Skip to content

Endpoint to retrieve the full configuration for workers

https://redmine.teklia.com/issues/11531

A new RetrieveWorkerRunConfiguration endpoint will be intended for both workers and worker developers. It returns the full computed configuration, as a combination of the WorkerConfigurationFields, ModelVersion and WorkerConfiguration, similarly to the current configure method in base worker. This could simplify this method, and make it easier for developers to view the configuration that was actually applied for each WorkerRun and debug locally.

This endpoint should be at /workers/runs/{id}/configuration/. This URL does not match RetrieveWorkerRun, which is at /process/workers/{id}/, but the URL for that endpoint should probably be changed someday to avoid confusion with workers.

This endpoint should use the new error format powered by drf-standardized-errors.

This endpoint requires either admin access to the corpus of the process, or authentication as a Ponos agent. Ponos agents should only be able to access worker runs linked to a task assigned to that agent. Slurm agents will later need to be able to access this endpoint, because workers work offline within Slurm and will not be able to retrieve their configurations on their own.

Only worker versions that use the new configuration format are supported. If the version for this WorkerRun does not have modern_configuration enabled, return an HTTP 400 error.

This endpoint starts by building the configuration from all the WorkerConfigurationFields for this WorkerVersion:

  • group fields should be skipped, since they are not present in an actual configuration.
  • When a field does not have a parent, then its key should be the key in the final configuration.
  • When a field has a parent, then {parent.key}.{key} is the key in the final configuration.
  • The value is the default of the field.
  • Whether the field is a secret will also be necessary for the API response, because workers need to be able to tell when to call RetrieveSecret to get the actual value of a secret.
  • This is the modern equivalent of WorkerVersion.configuration, which we will be removing later.

Then, if the WorkerRun has a model version, the final configuration values get updated with the ModelVersion.configuration:

  • Whether or not a field is a secret must not be overwritten, if the field already existed.
  • If the ModelVersion's configuration defines a key that did not exist, it is created, and declared as not being a secret.

Finally, if there is a WorkerConfiguration on the WorkerRun, the configuration values get updated with the WorkerConfiguration.configuration. Each value in the configuration must be ignored when:

  • The key does not exist among the WorkerConfigurationFields, even after it got defined by a ModelVersion, and after taking into account the possible parent prefix;
  • The WorkerConfigurationField was declared as not editable.

The API response will then be:

{
  "configuration": [
    {
      "key": "...",
      "value": "...",
      "secret": false
    }
  ]
}

In the API documentation, the key must be a string, the value can be anything including null, and the secret must be a boolean and defaults to false. The configuration can also be an empty list.