Skip to content

Stop tasks when they reach their TTL

https://redmine.teklia.com/issues/8785

Requires #1871 (closed)

A new Ponos task state, State.Cancelled, represents a task that has been stopped because it went over its TTL. It should be included in the FINAL_STATES, and comes in second place after Running in STATES_ORDERING.

The run_docker_task RQ task should react to a task that exceeds its TTL similarly to how it handles State.Stopping: it should call container.stop(), then set the task's state to State.Cancelled (rather than Stopped). A task exceeds its TTL when (timezone.now() - started).total_seconds() > ttl.

It should also append an error line to the logs, like what is done in run_task_rq, to report that the task has been cancelled due to reaching its TTL and make the cause of the failure clearer.