Run local corpus exports on the high queue
https://redmine.teklia.com/issues/7571
Corpus exports need to be on different queues depending on whether they run on the default source, or any other source, to reduce issues when exports are running in parallel and running into database connection issues, and give more flexibility to sysadmins to manage the task distribution.
RQ's @job decorator does not allow to run a task on a specific queue, so we have to split the export task into two.
Please remove the @job from the export_corpus function and instead add two more functions to run exports on different queues:
@job('high', ...)
def local_export(corpus_export: CorpusExport) -> None:
assert corpus_export.source == "default"
export_corpus(*args, **kwargs)
@job('export', ...)
def remote_export(corpus_export: CorpusExport) -> None:
assert corpus_export.source != "default"
export_corpus(*args, **kwargs)
arkindex.project.triggers.export_corpus must be updated to pick the correct task to run.
Please add tests to check that local_export only allows the default source and remote_export does the opposite.