A successful run is the easy case
A linear LLM pipeline is straightforward while every stage succeeds. One node produces an analysis, the next enriches it, and the last writes an external result. The difficult cases arrive when a container stops between stages, a retry replays a side effect, state becomes too large to checkpoint, or an automated review cannot decide whether the input is sufficient.
This workflow processes compliance evidence, but the problem is not specific to compliance. Document processing, approval flows, research pipelines, and enrichment jobs all need a clear answer to the same question: after an interruption, what can safely run again and what must not?
I replaced an eight-stage linear script with a LangGraph workflow so the execution model could carry that answer in its state, checkpoints, and routing rules. The important change was not adding more model calls. It was making persistence, safe re-execution, and escalation explicit parts of the system design.
Why the linear pipeline stopped being enough
The original shape was sequential: collect evidence, analyze a gap, draft remediation, and publish the result. A process interruption after an earlier stage completed left two bad choices: restart the whole run and repeat work, or reconstruct enough intermediate state to guess where to resume. If the restart reached a completed publish step, it could also replay an external side effect.
The graph separates the work into nodes with explicit state transitions. Reasoning nodes produce structured findings; deterministic nodes validate status, select the next route, or block unsafe progress. A LangGraph checkpointer persists state across node transitions. The documented production configuration uses DynamoDB, while smoke tests use LangGraph's in-process MemorySaver.
That makes a checkpoint an operational boundary, not just a debugging snapshot. A replacement task can resume from durable state after an interrupted run instead of reconstructing the workflow from its inputs. The tradeoff is that persisted state, node completion, and external effects all need deliberate semantics; a checkpoint alone cannot decide what is safe to repeat.
Turning the pipeline into a state machine
LangGraph supplies the directed graph and checkpointing behavior. CrewAI handles reasoning inside selected nodes; it does not choose the control flow. The graph contains both reasoning work and ordinary application logic: evidence merging, validation, routing, drafting, a pause before operator input, and a publisher behind a validation gate.
State transitions are typed with Pydantic models and validated on reads and writes. That gives malformed state a defined failure point at the edge rather than letting it travel as an untyped dictionary several nodes downstream. The project record describes the checkpointer as persisting state after each node transition, so a node boundary is also a recovery boundary.
For smoke tests, the same graph can use MemorySaver as an in-process checkpointer instead of DynamoDB. The project record documents that verification path, but not formal test counts or a broader test suite, so this is evidence of checkpoint behavior being exercised rather than a claim about overall coverage.
The routing policy below is simplified pseudocode based on the documented graph. It shows the important separation: the reviewer produces structured output, while regular Python decides whether to enrich, continue, or escalate.
# Simplified pseudocode; not literal production source.
def route_after_review(state):
if state.revision_count >= state.max_revisions:
return "human_interrupt"
if not state.review.schema_ok:
return "evidence_enricher"
if state.review.confidence < 0.75:
return "evidence_enricher"
return "remediation_drafter"The route is deterministic. The model may contribute a finding, but it does not decide whether the workflow loops, publishes, or stops.
The DynamoDB item-size constraint
DynamoDB has a 400 KB item limit. That is a poor fit for graph state that can accumulate evidence excerpts, screenshots, long narratives, and intermediate artifacts. Treating the checkpoint record as the place to store every payload makes a normal persistence operation fail when one field grows too large.
The workflow uses the Claim Check pattern for state serialization. Before a state field is written, the serializer measures its JSON payload. Values below the documented 50 KB threshold stay inline. Larger values are written to S3 and replaced in the checkpoint record with a typed S3Ref containing a kind marker, bucket, key, SHA-256 digest, and payload size.
On read, the serializer recognizes an S3Ref and restores the JSON value. Nodes use the state field without needing a separate code path for inline and offloaded data. The threshold is deliberately much smaller than DynamoDB's hard limit: it leaves headroom for several inline fields in the same item rather than allowing a collection of individually small fields to exceed the record limit together.
The source record shows SHA-256 being calculated and stored in the pointer and used in the S3 key. It does not document a digest check during hydration, so this design carries an integrity identifier rather than a claim of verified-on-read integrity.
The tradeoff is intentional. Small orchestration state remains inline for the normal checkpoint path, while large artifacts no longer make a checkpoint write fail. Large-state reads add an S3 retrieval, and the external objects introduce lifecycle and cleanup concerns; those are the costs of separating durable coordination state from artifact storage.
# Simplified representation of the documented serializer.
payload = json.dumps(value).encode("utf-8")
if len(payload) < 50_000:
return value
digest = sha256(payload).hexdigest()
key = f"state/{run_id}/{field_name}/{digest}.json"
s3.put_object(Bucket=state_bucket, Key=key, Body=payload)
return S3Ref(bucket=state_bucket, key=key,
sha256=digest, size_bytes=len(payload))Large payloads move to S3; graph state keeps a compact, typed reference. This is simplified from the project record rather than copied production code.
Retries need explicit side-effect semantics
A checkpoint makes resumption possible, but it does not make replay safe. A resumed workflow needs to distinguish work that can be repeated from a side effect that must not be issued again after a process interruption.
Every node in this workflow is wrapped by an @idempotent decorator. It records completion only after a normal return. If a node raises, no completion record is written, so the node remains eligible to run on resume. The completion key is based on the workflow thread and node name, with an optional revision component.
That revision component matters because replay semantics differ by node. gap_analyzer, evidence_enricher, evidence_merger, custom_test_runner, and reviewer use per_revision=True: another pass after new evidence is legitimate work. remediation_drafter, ui_mapper, jira_publisher, and human_interrupt use once-per-thread behavior because replaying a completed single-shot or terminal node can repeat an external outcome or violate the terminal state.
This is application-level side-effect protection, not a claim of distributed exactly-once execution or transaction semantics. It provides bounded re-execution: unfinished work can retry after resume, while successfully completed work is skipped according to the node's declared replay policy.
Bounded review is an operational control
The reviewer is deterministic. It grades a finding against a fixed rubric that includes schema completeness, evidence citation, and unsupported claims. Its output includes enrichment confidence and a list of missing evidence. A confidence score below the documented 0.75 threshold, or a schema failure, routes the graph to evidence_enricher. The model can participate in analysis and enrichment, but policy decisions about routing and stopping remain in code.
The enricher fetches only the signals identified as missing, appends them to the evidence state, increments revision_count, and returns to gap_analyzer. That is more useful than rerunning the whole pipeline with the same inputs, but it creates a loop and therefore needs a hard bound.
max_revisions is part of the state and ends the retry path. The project record does not specify a fixed numeric value, so the meaningful contract is the bound itself: the workflow cannot consume unbounded model calls or keep retrying an unresolved finding indefinitely.
Human escalation is a terminal state
When revision_count reaches max_revisions before the finding converges, deterministic routing sends the run to human_interrupt. The node records an ESCALATION REQUIRED log entry with structured run and control metadata, sets human_approval_status to pending, and ends the graph. This turns a non-converging automation path into an observable pending state rather than an unbounded retry.
The terminal behavior is deliberate. The documented node makes no LLM call, does not invoke the Jira publisher, and does not touch external services. It turns an unresolved automated result into visible pending work instead of treating an ambiguous outcome as something safe to publish.
There is a second human boundary before ui_mapper. The graph pauses at interrupt_before, an operator supplies evidence and sets a completion status, and ui_mapper accepts only complete, skipped, or not_applicable. Pending and unknown values raise instead. Because completion is recorded only after a normal return, that failed validation does not mark the node complete; the run can pause again rather than carrying unverified state to the publisher.
Why this workflow runs on ECS Fargate
The documented deployment uses Terraform to provision an ECS Fargate task definition, VPC, and IAM roles, with DynamoDB as the production checkpointer. The project rationale is specific to workflow semantics: a stateful graph can outlive Lambda's 15-minute execution ceiling, especially when it pauses at interrupt_before and resumes after operator input.
Lambda remains a good fit for short, stateless event handlers. For this graph, Fargate and a durable checkpointer avoid having to split one logical execution across multiple Lambda invocations and reconstruct its coordination state at each boundary. This is a workload-fit decision, not a claim that Fargate is cheaper, faster, or universally preferable.
Engineering lessons
Design workflow persistence before retries become complicated. Orchestration state and large artifacts have different access patterns and do not need the same storage strategy.
Declare replay semantics per node. Checkpointing without side-effect rules only makes unsafe replay easier. Automated review loops also need a deterministic route, a hard bound, and a terminal human state.
Choose the runtime around execution semantics. Model calls are only one component of a stateful workflow; the engineering work is in what the system persists, validates, retries, and refuses to do automatically.