Skip to main content

Celery Domain Entities and Canvas Primitives

The data model for Celery's domain entities and Canvas primitives centers around the Defining and Registering Tasks and its execution context, as well as the Introduction to Task Signatures which allows for complex workflow compositions.

Key Entities:​

  • Task: The base class for all executable units. It defines execution parameters like retries, rate limits, and time limits.
  • Context: Represents the runtime state of a task execution (request). it tracks the task ID, arguments, and relationships to other tasks in a workflow (parent, root, chain, chord).
  • AsyncResult: A handle to a task's result and state. It maintains a recursive relationship with its parent (for chains) and children (for subtasks).
  • Signature: A "lazy" task invocation that wraps a task name with arguments and options. It is the building block for Canvas primitives.
  • Canvas Primitives:
    • Group: Executes a list of signatures in parallel.
    • Chain: Executes a list of signatures sequentially, passing the result of one to the next.
    • Chord: A group with a callback (body) that executes after all tasks in the group (header) complete.
  • Schedules: Define when periodic tasks should run, with implementations for fixed intervals (Scheduling & Periodic Tasks), cron-like patterns (Advanced Scheduling with Crontab), and solar events.

Relationships:​

  • A Task is invoked via a Signature.
  • Executing a Signature or Task returns an AsyncResult.
  • Canvas Primitives (Group, Chain, Chord) are specialized Signatures that compose other signatures.
  • GroupResult aggregates multiple AsyncResult objects from a group execution.
  • Context links a running task to its position in a workflow via parent_id, root_id, and chord.

Key Architectural Findings:

  • Signature is the base class for all Canvas primitives (Group, Chain, Chord) and inherits from Python's dict.
  • AsyncResult maintains a tree structure via 'parent' and 'children' fields, enabling result tracking across complex workflows.
  • Task execution state is encapsulated in a Context object, which is pushed onto a thread-local request_stack.
  • Chords are composed of a 'header' (a Group) and a 'body' (a Signature).
  • Schedules (schedule, crontab, solar) all inherit from BaseSchedule and provide logic for 'is_due' and 'remaining_estimate'.
Loading diagram...