Skip to content

Rework due to API changes

This depends on some work in progress in Base-Worker.

Arkindex 1.6.0 introduces breaking changes in its Dataset API. This worker will no longer depend on arkindex_worker.worker.DatasetWorker but instead inherit from arkindex_worker.worker.BaseWorker and arkindex_worker.worker.dataset.DatasetMixin.

Unlike the new DatasetWorkers, this worker will work at Dataset level, grouping all sets in the process to handle the dataset directly.

Rewrite run.

    def run(self):
        # Configure

        # List process sets

        # Group by dataset
        # Sets should already be sorted by dataset name so a simple groupby should be enough
        
        # Make sure the dataset state is either `Open` or `Error`
        # Update state to `Building`
        # Call process_dataset but using the iterator on the sets
        # Update state to `Complete` or `Error`

The idea is to still have error catching, just like we do in https://gitlab.teklia.com/workers/base-worker/-/blob/112551b9a853a5ae31117f46c1271d4e7617a31a/arkindex_worker/worker/init.py#L321

Edited by Yoann Schneider