Skip to content

Support export sources

https://redmine.teklia.com/issues/5736

To support very large exports in EE, we need to allow exports to run on other databases.

A new EXPORT_SOURCES setting must be created, holding a set of database names. For now, it will only include default. replica must not be included, even if it is configured.

The CorpusExport model now includes a new source CharField, defaulting to default, and only allowing values found within settings.EXPORT_SOURCES.

The StartExport endpoint should allow setting this new source field.

The ListExports endpoint should include the source in its response.

When the source is not default, the export_corpus trigger should include " from source {source}" in the task's description, so that users can tell which export is running when looking at their current jobs.

The export_corpus RQ job should use the database specified as the CorpusExport's source for all of its export queries. It must not use this database to perform any unrelated queries, such as fetching the CorpusExport itself or updating its state when it finishes, as the selected database may be read-only or out of sync.