Sagas: Distributed Transactions Without Two-Phase Commit

A checkout flow spans three services: payments charges the card, inventory reserves the items, shipping books the courier. Payments succeeds. Inventory succeeds. Shipping fails — the courier API is down, hard, for an hour.

Now enumerate your options. You can’t roll back — payments and inventory committed, in their own databases, minutes of real time ago. You can’t block and hold locks across three services for an hour. You can’t pretend it worked. What’s left is the option nobody teaches in database class: undo by doing new things — refund the charge, release the reservation — and accept that for a window of time, the system was in a state where money was taken for an order that will never ship.

That maneuver has a name, a theory, and a surprisingly old birthday. The saga was introduced by Hector Garcia-Molina and Kenneth Salem in their 1987 SIGMOD paper “Sagas” — written about long-running transactions inside a single database, decades before microservices existed. Their machinery turned out to be exactly what distributed architectures needed, which is why Chris Richardson’s microservices.io canonized it as the pattern for data consistency across services.

Why the “Real” Solution Is Off the Table

The textbook answer to multi-database atomicity is two-phase commit (2PC): a coordinator asks every participant to prepare (lock resources, promise to commit), then orders everyone to commit. Atomic, and correct — and almost universally rejected in service architectures, for reasons worth stating precisely rather than as folklore:

Blocking: between prepare and commit, every participant holds locks while waiting for the coordinator. A slow or dead coordinator leaves resources locked across your whole fleet — the cascading-failure anatomy with a protocol’s blessing.
Availability math: the transaction succeeds only if every participant and the coordinator are up simultaneously. Your checkout’s availability becomes the product of five services’ availability — the exact coupling that service boundaries were meant to dissolve.
Ecosystem reality: the infrastructure you actually use — Kafka consumers-to-DB, HTTP APIs, most managed cloud services — simply doesn’t speak XA/2PC. The pattern isn’t just costly; it’s unavailable.

Garcia-Molina and Salem’s 1987 motivation was structurally identical, just single-node: long-lived transactions “hold on to database resources for relatively long periods of time, significantly delaying the termination of shorter and more common transactions.” Their solution generalizes verbatim.

The Saga Contract

Their definition: a long transaction is a saga if it can be written as a sequence of sub-transactions T₁, T₂, … Tₙ, each locally atomic, where the system guarantees one of exactly two outcomes — all sub-transactions complete, or compensating transactions C₁…Cₖ run to amend a partial execution. Each Tᵢ ships with its compensation Cᵢ: refund for charge, release for reserve, cancel for book.

Notice what’s traded away. ACID’s atomicity is preserved in a weakened form (all-or-compensated). Isolation is gone — and this is the part that bites in production. Between T₁ committing and the saga finishing (or compensating), the intermediate state is visible to everyone. Another process can see the charged-but-not-shipped order. The literature calls the resulting bugs isolation anomalies, and the standard countermeasures are worth knowing by name: semantic locks (a status: PENDING field that other transactions respect), commutative updates (design steps so order doesn’t matter), and re-reading state before acting on possibly-stale intermediate data.

Compensation Is the Hard Part

The pattern looks mechanical until you sit down to write the compensations, whereupon three uncomfortable truths surface:

1. Compensation is business logic, not rollback. A database rollback erases history; a compensation adds to it. The refund is a new financial event — it appears on statements, it can itself fail, it may cost you card-processing fees. Designing Cᵢ means asking the business: what is our apology for having done Tᵢ?

2. Some actions have no undo. The email is sent. The push notification fired. The courier already picked up the package. For these, the vocabulary extends: sagas order steps so that compensatable transactions come first, followed by the pivot — the point of no return — after which remaining steps must be retriable (guaranteed to eventually succeed, retried forever). Checkout done right: reserve (compensatable) → charge (pivot) → book courier (retriable — keep trying, or escalate to a human). Sequencing steps by their undo-ability is the single most consequential design act in a saga.

3. Compensations themselves face the ambiguous-failure problem. A refund call that times out — did it refund? Every step and every compensation must be idempotent, keyed, and safe to retry. Sagas don’t replace this series’ fundamentals; they’re built out of them.

Choreography vs. Orchestration, Revisited

Richardson’s catalog documents two coordination styles, and the trade-off echoes the event-driven article at higher stakes:

Choreographed sagas: each service reacts to the previous service’s events — OrderCreated triggers payment, PaymentCompleted triggers inventory. No coordinator, maximal decoupling — and the failure paths are distributed too. Compensation logic for step 3 failing lives scattered across three services’ event handlers, and Fowler’s invisible-flow warning applies at maximum intensity: nobody can read the saga anywhere.

Orchestrated sagas: an explicit orchestrator — a state machine with persistence — invokes each step, records progress, and owns the failure playbook:

// The orchestrator is a persisted state machine. After a crash it
// reloads state and resumes — which is why every step must be
// idempotent: "resume" means "possibly re-issue the last command".
func (o *CheckoutSaga) Advance(ctx context.Context, s *SagaState) error {
    switch s.Step {
    case StepReserve:
        if err := o.inventory.Reserve(ctx, s.Key("reserve"), s.Items); err != nil {
            return o.fail(ctx, s) // nothing to compensate yet
        }
        s.Step = StepCharge
    case StepCharge: // the pivot
        if err := o.payments.Charge(ctx, s.Key("charge"), s.Amount); err != nil {
            o.inventory.Release(ctx, s.Key("release"), s.Items) // compensate T1
            return o.fail(ctx, s)
        }
        s.Step = StepShip
    case StepShip: // retriable — after the pivot we go forward, not back
        if err := o.shipping.Book(ctx, s.Key("ship"), s.Order); err != nil {
            return err // retried with backoff; escalates after N attempts
        }
        s.Step = StepDone
    }
    return o.store.Save(ctx, s) // persist progress — with an outbox, naturally
}

The honest recommendation mirrors the earlier one, sharpened: choreograph the happy-path notifications; orchestrate anything with compensation. A saga’s entire reason to exist is its failure handling — putting the failure handling where no one can see it defeats the point. This is also where workflow engines (Temporal, AWS Step Functions — whose reference implementation is literally this checkout example) earn consideration: they are orchestrators with the persistence, retries, and visibility pre-built.

The Objection: “This Is a Lot of Machinery to Avoid a Transaction”

Correct — and that instinct deserves to win most of the time. The saga pattern’s most important property is how rarely you should need it:

If the steps live in one database, use a transaction. A modular monolith runs this entire checkout in one BEGIN...COMMIT, no compensations, no anomalies, no orchestrator. This is among the strongest concrete arguments in the monolith-vs-microservices decision — covered properly later in this series — and it’s how my own systems handle multi-step operations: one process, one database, transactions doing what transactions do.
If only the last step is external, you don’t need a saga — you need the outbox plus a retriable delivery. Save order + charge locally, emit “book courier” as a retried job. A saga with one compensatable step is just a transaction with extra vocabulary.
If a human can be the compensation, let them. Low-volume flows fail rarely; a good alert plus a refund button may be cheaper than an orchestrator — if the volume genuinely stays low and someone owns the queue.

The pattern earns its complexity at the intersection: genuinely separate services (organizational or scale reasons — real ones), multi-step writes across them, and failure volume too high for humans. That intersection exists — it’s just further away than the conference talks imply.

Designing One, Step by Step

Write the steps in prose, then sort by undo-ability: compensatable → pivot → retriable. If you can’t identify the pivot, you don’t understand the flow yet.
Design each compensation with the business, as a business event with its own cost. Get the “apology” approved by whoever owns the P&L of that apology.
Make every step and compensation idempotent with saga-scoped keys (sagaID:step). Non-negotiable — the orchestrator’s crash-resume model depends on it.
Persist orchestrator state via the outbox pattern — the saga’s own progress must not be lost to the dual-write it’s supposed to prevent.
Name the isolation anomalies. Ask: who can see the intermediate state, and what’s the worst they can do with it? Add semantic locks (PENDING statuses) where the answer scares you.
Alert on stuck sagas. A saga neither done nor compensated after its SLA is a product incident — money taken, nothing shipped. The dashboard for in-flight saga age is not optional.
Reconsider the monolith. Genuinely. If this checklist feels heavier than merging two services, that’s data, not defeat.

Garcia-Molina and Salem solved a 1987 mainframe problem and accidentally wrote the operating manual for 2020s service architectures — the mark of work done at the level of principles. The next article stays at the boundary between services, but at design time instead of runtime: contract-first APIs, and making OpenAPI the source of truth your CI actually enforces.