model

EleanorAI Framework Task Execution Data Models

This module defines the data model classes used by the EleanorAI Task Execution Framework. These classes represent the various states, statuses, and policies associated with task execution stages. They are essential for managing the lifecycle of tasks, including checkpointing, rollbacks, and retries.

These models are used by the TaskEngine and BaseStage classes in the task execution framework to manage and persist the state of each stage in a task. They provide a structured way to track the progress, retries, and history of each stage, ensuring that tasks can be resumed or rolled back as needed.

Attributes:

• ACTIVE_STATES (set) –

  Set of active stage statuses.

• INACTIVE_STATES (set) –

  Set of inactive stage statuses.

• STAGE_NAME_MAX_LENGTH (int) –

  Maximum length of a stage name.

Attributes

ACTIVE_STATES module-attribute

ACTIVE_STATES = {CREATED, RUNNING, RETRY}

CANCELABLE_STATES module-attribute

CANCELABLE_STATES = {UNBOUND, CREATED, RUNNING, RETRY}

INACTIVE_STATES module-attribute

INACTIVE_STATES = {COMPLETE, CANCELED, ERROR}

STAGE_NAME_MAX_LENGTH module-attribute

STAGE_NAME_MAX_LENGTH = 150

STAGE_RESULT module-attribute

STAGE_RESULT = TypeVar('STAGE_RESULT', bound=BaseModel)

Classes

BaseStageState

Bases: BaseModel, Generic[STAGE_RESULT]

Base class for stage state data. This class should be subclassed to provide specific fields for managing state data.

Note that all stages in a task share the same state class. This is a deliberate design decision to ensure the currently executing stag as access to all relevant state details of every stage that came before it.

Attributes

attempts class-attribute instance-attribute

attempts: int = Field(
    default=0,
    ge=0,
    title="Run attempts",
    description="Number of times the stage has been run, value of 0 indicates it has not beenrun yet. Primary used to track retries.",
)

history class-attribute instance-attribute

history: List[StageStateHistoryEntry] = Field(
    default=[],
    title="History",
    description="Stage state history",
)

model_config class-attribute instance-attribute

model_config = ConfigDict(
    extra="forbid",
    strict=False,
    arbitrary_types_allowed=False,
)

result class-attribute instance-attribute

result: STAGE_RESULT | None = Field(
    default=None,
    title="Result",
    description="Contains the stage execution result. The intended use case is to store the output of the final stage such that external systems can inspect the final stage to determine the overall result of the task. A value of None indicates that the stage has not been completed successfully",
)

stage_name class-attribute instance-attribute

stage_name: str = Field(
    "",
    max_length=STAGE_NAME_MAX_LENGTH,
    title="Stage Name",
    description="Unique stage name / identifier. This field will be used to generate the statemetadata file/object and persisted during execution",
)

status class-attribute instance-attribute

status: StageStatus = Field(
    default=UNBOUND,
    title="Stage Status",
    description="Stage execution status, used by the task execution engine to determine whetherthis state needs to be executed or skipped",
)

task_id class-attribute instance-attribute

task_id: str = Field(
    default="", title="Task ID", description="Task ID"
)

RecordStatus

Bases: Enum

Record status enumeration

Attributes

ERROR class-attribute instance-attribute

ERROR = 'error'

SUCCESS class-attribute instance-attribute

SUCCESS = 'success'

UNPROCESSED class-attribute instance-attribute

UNPROCESSED = 'unprocessed'

RetryPolicy

Bases: Enum

Task retry policy enumeration.

Attributes:

• NONE (str) –

  No retry policy, a single stage failure will cause the entire task to fail.

• STAGE (str) –

  Retry counters are reset for each stage.

• GLOBAL (str) –

  Retry counters are shared across all stages, each stage execution consumes an attempt - even if it is successful.

Attributes

GLOBAL class-attribute instance-attribute

GLOBAL = 'GLOBAL'

NONE class-attribute instance-attribute

NONE = 'NONE'

STAGE class-attribute instance-attribute

STAGE = 'STAGE'

SimpleRecordsResult

Bases: BaseModel

Simple task result model indented to provide useful summary statistics in the RDBMS.

Attributes

fail_count class-attribute instance-attribute

fail_count: int = Field(
    default=0,
    ge=0,
    title="Fail Count",
    description="Number of failed records",
)

message class-attribute instance-attribute

message: str | None = Field(
    default=None,
    title="Message",
    description="Result message",
)

success_count class-attribute instance-attribute

success_count: int = Field(
    default=0,
    ge=0,
    title="Success Count",
    description="Number of successful records",
)

total_count class-attribute instance-attribute

total_count: int = Field(
    default=0,
    ge=0,
    title="Total Count",
    description="Total number of records",
)

unprocessed_count class-attribute instance-attribute

unprocessed_count: int = Field(
    default=0,
    ge=0,
    title="Unprocessed Count",
    description="Number of unprocessed records",
)

StageStateHistoryEntry

Bases: BaseModel

Stage state history

Attributes

attempt class-attribute instance-attribute

attempt: int = Field(
    ...,
    ge=0,
    title="Attempt",
    description="Run attempt number corresponding to this history entry",
)

begin_status class-attribute instance-attribute

begin_status: StageStatus = Field(
    ...,
    title="Begin Status",
    description="Status of the stage prior to the run",
)

begin_ts class-attribute instance-attribute

begin_ts: str = Field(
    ...,
    title="Start Timestamp",
    description="Run start timestamp",
)

end_status class-attribute instance-attribute

end_status: StageStatus = Field(
    ...,
    title="End Status",
    description="Status of the stage after the run",
)

end_ts class-attribute instance-attribute

end_ts: str = Field(
    ...,
    title="End Timestamp",
    description="Run end timestamp",
)

message class-attribute instance-attribute

message: str = Field(
    ..., title="Message", description="Status message"
)

run_time_seconds class-attribute instance-attribute

run_time_seconds: float = Field(
    ...,
    title="Run Time",
    description="Stage run time in seconds",
)

StageStatus

Bases: Enum

Task status enumeration.

Attributes:

• UNBOUND (str) –

  Task has not been bound to a task engine.

• CREATED (str) –

  Task has been created but not yet started.

• RUNNING (str) –

  Task is currently running.

• RETRY (str) –

  Task encountered an error and is eligible for to be retried.

• ERROR (str) –

  Task has encountered an error and cannot be retried.

• COMPLETE (str) –

  Task has completed successfully.

• CANCELED (str) –

  Task has been canceled.

Attributes

CANCELED class-attribute instance-attribute

CANCELED = 'CANCELED'

COMPLETE class-attribute instance-attribute

COMPLETE = 'COMPLETE'

CREATED class-attribute instance-attribute

CREATED = 'CREATED'

ERROR class-attribute instance-attribute

ERROR = 'ERROR'

RETRY class-attribute instance-attribute

RETRY = 'RETRY'

RUNNING class-attribute instance-attribute

RUNNING = 'RUNNING'

UNBOUND class-attribute instance-attribute

UNBOUND = 'UNBOUND'