AssertionError when calling RetryProcess on a process with an unavailable WorkerVersion
Sentry Issue: ARKINDEX-BACKEND-2WX
AssertionError: Worker Version 97793edd-50a0-4b4b-9a05-5fd705b1616f is not available and cannot be used to build a task.
(10 additional frame(s) were not displayed)
...
File "arkindex/process/models.py", line 497, in run
File "arkindex/process/builder.py", line 385, in build
File "arkindex/process/builder.py", line 49, in _wraps
File "arkindex/process/builder.py", line 327, in build_workers
File "arkindex/process/models.py", line 1211, in build_task
RetryProcess does not do the detailed checks that StartProcess does. This includes checking for unavailable worker versions, unavailable model versions, workers that require a model version but did not have a model version set, or required user_configuration fields (the new configuration validation is not made anywhere yet). This can result in errors because other methods like WorkerRun.build_task will make assertions on this.
The StartProcess validation likely needs to be moved to ProcessBuilder instead, which will allow it to be used for both start and retry. RetryProcess will also need to make the same complex prefetch for WorkerRuns as StartProcess does.