Skip to content

Use django_rq for async tasks

Erwan Rouchet requested to merge rq into master
  • Deprecates !1022 (closed) (dependency removed)
  • Requires a new base build on the next release
  • Requires an update to the settings docs; the redis.capacity setting is removed, and redis.port, redis.db, redis.password and redis.timeout are added
  • New worker added to the dev architecture in architecture!125
  • Requires updating some magical deployment YAML

This removes Channels entirely (we can add it later differently if we just want WebSocket support and no worker) and uses RQ instead via django-rq. Consumer classes have been replaced by functions with @job decorators and their syntax is similar to Celery's (reindex_start.delay()). The consumer's contents were not modified (the diff says otherwise because of indentation changes).

Although this chokes a bit on very large corpora like Madcat, GoLFor or Balsac and requires retries, this change alone makes it possible to delete any corpus.

Edited by Erwan Rouchet

Merge request reports

Loading