update_system_workers command

Closes #1832 (closed)

• This introduces some new helpers in arkindex.project.tests: the ArkindexTestCase and ArkindexAPITestCase include tools like assertExactQueries and clear_caches, but without the database fixtures, because the update_system_workers need to clear the get_by_feature cache and also are simpler with a blank database than with the fixtures.

• A new WorkerVersion.required_user_configuration_fields lists the user configuration fields that are required and do not have a default value. If this property is not empty, you need a worker configuration with those fields set to run the version. I used this property to detect feature versions with unsupported required fields.

• The Docker images are defined in the command directly, in the FEATURE_IMAGES constant. I did not want to put them in the Django settings because they would be drowning under the myriad of other variables that we normally don't touch even when releasing, and because it discourages using these parameters outside of the command. You should be using get_by_feature and should not rely on the hardcoded images we put in this command!

• Running this will break the S3 ingest worker until arkindex/workers/import/s3#5 (closed) is done, since it requires feature workers to not have any custom Docker comamnd defined.

• Declaring an image for a new feature (such as the file import worker) will involve setting it in FEATURE_IMAGES, then updating the tests.

• Updating the images for existing features (as would be done when releasing) will not break the tests.

• One remaining question (this is the test currently failing in CI): when the command needs to create a new system worker, but there is a worker with the same slug and a different type, the unique constraint on worker slugs will fail.

  • Do we use that existing worker, which may contain unrelated versions and may need to be unarchived?
  • Do we update the worker's slug to something else, and if so, what?
  • Do we create a system worker with a different slug, and if so, what?
  • Is this the only case where the command fails and instructs sysadmins to deal with it themselves?