Remove DataFile hash
This is some preparatory work for frontend#233 (closed). To help the frontend upload files directly to S3 using CreateDataFile
instead of UploadDataFile
and bypass time limits, it has been decided to remove the MD5 hash check on DataFiles, making MD5 hashes useless to them. Therefore, MD5 hashes are completely removed from the DataFile.
This implies a little bit of refactoring to arkindex.project.aws
to remove a now useless mixin, as well as updating the file checking algorithm to not update the attributes on a file, but just to check that what the client said the content type and length would be is what they actually are ("trust, but verify"). This also implies a lot of updates to unit tests, which took most of the time spent on this MR.
An issue has been noticed with the way the IIIF uploads are handled: When a JSON file gets uploaded, its @context
is checked and it is only treated as an IIIF manifest if it has the expected context. This is only detected when using UploadDataFile
and not CreateDataFile
, therefore exclusively using the S3 upload would break IIIF imports. This will be fixed in a later MR.