Get multipart file upload link
Required capabilities:
filesAcl:WRITEOr
dataModelsAcl:READ(scope spaceId:cdf_cdm) anddataModelInstancesAcl:WRITE
Multipart file upload enables upload of files larger than 5000 MB, using a uniform API on all cloud environments for CDF.
Each file part must be larger than 5 MiB, and smaller than 4000 MiB. The file part for the last uploadURL can be smaller than 5 MiB. Maximum 250 upload URLs can be requested.
The supported maximum size for each file uploaded with the multi-part upload API is therefore 1000 GiB (1.073 TB / 0.976 TiB).
The client should calculate the ideal number of parts depending on predetermined or estimated file size, between 1 and the maximum.
Specify the number of parts in the parts URL query parameter. The parts parameter is required.
The request returns in addition to the file id, also a uploadId, and a list of uploadUrls for uploading the file contents.
To upload a file, send an HTTP PUT request to each of the uploadUrls, with the corresponding part of the file in the request body.
You may use a ‘Content-Length’ header in the PUT request for each part, but this is not required.
A failed part PUT upload can be retried.
If ‘contentEncoding’ is specified in the files/multiuploadlink request body, the full file contents must be compressed with that encoding before uploading the parts as byte ranges of the full content. When the file is later downloaded through a download link, the response will include a ‘Content-Encoding’ header set to this value, and the client is responsible for decompressing the content accordingly.
The client must ensure that the parts of the source file are stored in the correct order, using the order of the uploadUrls as specified in the response.
This to avoid ending up with a corrupt final file.
The parts can optionally be uploaded in parallel, preferably on a subset of parts at a time, for example maximum 3 concurrent PUT operations.
Once all file parts have been uploaded, the client should call the ‘files/completemultipartupload’ endpoint,
with the required file ID (as id or externalId) and uploadId fields in the request body. This will assemble the parts into one file.
The file’s uploaded flag will then eventually be set to true.
A standard sequence of calls to upload a large file with multipart upload would be for example as follows:
POST files/multiuploadlink?parts=8, to start a multipart upload session with 8 parts. Optionally specify thecontentEncodingrequest body property if the file content is compressed with a compression encoding. Use standard CDF project request headers such asAuthorizationandContent-Typefor thisPOSTrequest. Also optionally add aContent-Encodingrequest header to thisPOSTrequest, but note that it would be for the compressed JSON POST request body content, not for the full file content. Expect a 201 CREATED response code, and a response body with information to be used in thePUT <uploadURL>andPOST files/completemultipartuploadrequests.PUT <uploadUrl>, for each of theuploadUrlsin the response from thefiles/multiuploadlinkrequest. Expect a 200 OK or 201 CREATED response for each PUT request. NOTE: You can set theContent-Lengthrequest header for thisPUTrequest. Do not addAuthorization,Content-TypeorContent-Encodingheaders in thePUT <uploadURL>requests for the multipart upload based requests.POST files/completemultipartupload, with request body{ "id":123456789, "uploadId":"ABCD4321EFGH" }. This will assemble the file. Use standard CDF project POST request headers such asAuthorization,Content-Typefor this request. Expect a 200 OK response.
Consider verifying that the file is eventually marked as uploaded with a call to the getFileByInternalId endpoint.
NOTE: The uploadUrls expires after one week. A file that does not have the file content parts uploaded and completed within one week will be automatically deleted.
Request throttling
Please note that this endpoint is subject to the new throttling policy, which imposes limits on both request rate and concurrency. For more details, please refer to the Files resource documentation.
Authorizations
Access token issued by the CDF project's configured identity provider. Access token must be an OpenID Connect token, and the project must be configured to accept OpenID Connect tokens. Use a header key of 'Authorization' with a value of 'Bearer $accesstoken'. The token can be obtained through any flow supported by the identity provider.
Query Parameters
The 'parts' parameter specifies how many uploadURLs should be returned, for uploading the file contents in parts. See main endpoint description for more details.
1 <= x <= 250Body
Fields to be set for the file.
1 element- Option 1
- Option 2
Response
The response for a successful request to initiate upload of multiple parts for a file.