Skip to content

Replace ModelVersion.hash with ETag

https://redmine.teklia.com/issues/11341

  • The ModelVersion.hash field is removed.

  • The ModelVersion.check_hash method is removed. There are no server-side checks on the hash or the archive_hash, those will only be made after downloading a model version.

  • A new ModelVersion.s3_etag field is added instead, as a nullable CharField of up to 50 characters. It cannot be accessed or edited through the API. It is editable in the Django admin. This is the raw ETag header returned by S3 with no processing at all.

  • The s3_etag field should be set automatically from s3_object.e_tag:

    • When the state is set to Available through UpdateModelVersion;
    • When the state is set to Available through PartialUpdateModelVersion;
    • When CompleteMultipartUpload succeeds on a ModelVersion and will update its state to Available.
  • A new arkindex check_modelversions management command should, for each ModelVersion that is Available:

    1. Call s3_object.load();
    2. If it fails because the object doesn't exist, show a warning and skip;
    3. If it fails for any other reason, show an error and skip;
    4. If the s3_etag was None, set it to the s3_object.e_tag;
    5. If the s3_etag was set and it is different from the s3_object.e_tag, show a warning.
  • The arkindex cleanup command should perform the check through ListObjects instead of calling HeadObject once per ModelVersion.

Edited by Erwan Rouchet