Celery-Eternal Documentation

Celery-Eternal provides a Celery task subclass for jobs that should run forever, based on celery-singleton and celery.contrib.abortable.

Like celery-singleton, your app must use a redis result backend.

API

class celery_eternal.EternalTask

Bases: celery.contrib.abortable.AbortableTask, celery_singleton.singleton.Singleton

Base class for a task that should run forever, and should be restarted if it ever exits. The task should periodically check is_aborted() and exit gracefully if it is set. During a warm shutdown, we will attempt to abort the task.

To create an abortable task, call the task() decorator with the keyword argument base=EternalTask.

Generally, you should also pass bind=True so that your task function has access to the Task instance and can call self.is_aborted().

Here is an example eternal task that calls a fictional function do_some_work() in a loop:

@app.task(base=EternalTask, bind=True, ignore_result=True)
def long_running_task(self):
    while not self.is_aborted():
        do_some_work()

class celery_eternal.EternalProcessTask

Bases: celery_eternal.EternalTask

Base class for an eternal task that runs in a subprocess.

This is useful for tasks that do not check the method is_aborted() but can be stopped by a KeyboardInterrupt triggered by receving the signal SIGINT.

The task itself launches and supervises a subprocess that runs the function. The subprocess has Python’s default SIGINT handler installed.