Skip to content

CompleteMultipartUpload endpoint

https://redmine.teklia.com/issues/9153

Requires #1927 (closed)

A new CompleteMultipartUpload as a POST /api/v1/multipart/complete/ allows to complete a multipart upload after every part has been uploaded successfully. It uses a MultipartUploadCompleteSerializer that inherits from MultipartUploadSerializer and adds:

  • checksum_algorithm: optional, nullable EnumField accepting the names of the multipart upload checksum algorithms
  • parts: A child serializer with many=True
    • part_number: required IntegerField, between 1 to 10000
    • checksum: optional, nullable CharField
    • md5_hash: optional, nullable CharField
    • etag: required CharField

The serializer's validate() method must verify that:

  • when checksum_algorithm is set, checksum is set on every part, or when it is not set, checksum is not set anywhere;
  • either all md5_hash fields are set, or none are set;
  • all part_number values are unique.

If fields are inconsistently defined, an HTTP 400 error occurs.

The serializer's create() method must:

  • call object.s3_object.MultipartUpload(upload_id).complete() with the Parts built from the part number, ETag and optional checksum of each part.
    • The Parts must be reordered by ascending part number.
    • The checksum must be base64-encoded.
  • call object.s3_object.load(ChecksumMode="ENABLED") to verify the object exists and get its metadata
  • handle botocore.exceptions.ClientError exceptions, returning HTTP 400 or 404 with the e.response["Error"]["Message"] when e.response["Error"]["Code"] == "400" or "404"
  • verify that object.size and object.s3_object.content_length match
  • if everything went well:
    • for a DataFile, update status to S3FileStatus.Checked
    • for a ModelVersion, update state to ModelVersionState.Available
  • if an error occurs or the sizes do not match:
    • for a DataFile, update status to S3FileStatus.Error
    • for a ModelVersion, update state to ModelVersionState.Error

The server-side checksum verification will be done at a later stage.

The API response should use DataFileSerializer or ModelVersionSerializer depending on the object type.