All chapters
  1. Start hereA software factory, from first principles
  2. Primitive · 1A factory, before AI: input, transformation, check, output
  3. Primitive · 2From repetition to a pattern: what stays, what varies, what to leave alone
  4. Primitive · 3The specification as an executable contract
  5. Primitive · 4Build the smallest factory: four stations, two specs, one manifest
  6. Primitive · 5Assembly stations: typed contracts, owners and failure handling
  7. Primitive · 6AI enters the factory: what a model adds, and what stays deterministic
  8. WatchAlgo · 7The WatchAlgo production story: what the repository shows
  9. WatchAlgo · 8A problem as a production unit: the files one unit must contain
  10. WatchAlgo · 9Station by station: description, solution, implementations, visualization
  11. WatchAlgo · 10The visualizer factory: from algorithm state to renderable steps
  12. WatchAlgo · 11Three flavors and three languages: two different contracts
  13. WatchAlgo · 12Looping over a large backlog: scheduling, limits and a failed-case queue
  14. WatchAlgo · 13Long-running agent execution: context, checkpoints and token limits
  15. WatchAlgo · 14Evidence of correctness: tests the producer cannot talk its way past
  16. WatchAlgo · 15The factory learning loop: defects change the pattern, not only the unit
  17. Enterprise · 16Classify the enterprise demand before choosing a pattern
  18. Enterprise · 17Reference enterprise patterns and their variation points
  19. Enterprise · 18Platform foundations: golden paths, templates, registries and catalogs
  20. Enterprise · 19Pattern versus configuration versus business logic
  21. Enterprise · 20Insurance factory, part 1: building the intake contract
  22. Enterprise · 21Insurance factory, part 2: building the underwriting workflow
  23. Enterprise · 22Insurance factory, part 3: trust and observability built in
  24. Enterprise · 23Data and integration engineering: contracts that survive change
  25. Enterprise · 24The CI/CD and deployment factory: from pull request to production
  26. Enterprise · 25Enterprise governance: owners, approved patterns and provenance
  27. Enterprise · 26Scaling safely: many teams, many agents, one set of boundaries
  28. Operations · 27Factory quality engineering: measuring accepted output, not activity
  29. Operations · 28Failure and recovery scenarios: what breaks a factory, and what restores it
  30. Operations · 29The landscape: which tool for which job, read from its own documentation
  31. Operations · 30Design an enterprise software factory: the blueprint and the rollout
  32. ReferenceSources and verification
  33. ReferenceGlossary
  34. AboutAbout this site
Enterprise · 21

Insurance factory, part 2: building the underwriting workflow

Broker, intake, assessment, retrieval, recommendation and the underwriter's decision. Where sub-agents, queues, human gates and plain functions belong, and the difference between building this workflow and operating it.

You will separate
Building the underwriting workflow (factory run FR-0003) from operating it (runtime run run-7f3a on SUB-1004).
You will place
Each step of the flow with the least machinery that does it: plain code, an agent, a queue or a person.
You will weigh
Six ways to arrange the work, from one agent per request to a fixed graph with agents only where the work is semantic.

The design proposal for the insurer's underwriting review fits on one slide. "One agent handles each submission end to end. It reads the broker's email, checks the roof data, looks up the guidelines, writes the recommendation and updates the case. Its instructions say: never record a final decision; leave that to the underwriter."

The reviewer from the security team asks one question. "What stops it?" The answer is the sentence in the instructions. Nothing else. The agent holds one credential that can do everything the case system allows, and the only thing between that credential and a recorded decision is a paragraph the agent reads, alongside the broker's email and every retrieved passage, any of which might contain a paragraph of its own. A second engineer asks a different question: "When the factory builds this, does the factory's agent make the decisions in testing?" Two different things in the room are both called "the agent", and nobody has said which one they mean.

The learner's question: for the flow from a broker's email to an underwriter's decision, which steps are plain code, which deserve an agent, where do queues and human gates belong, and how does a factory build that flow without ever running it on a real case?

Two things called "the agent"#

Plain English. The factory is a machine that builds machines. The insurance service it builds contains its own agent, which works on real submissions after release. The factory's run and the service's run are different events, with different ids, owners and permissions.

Building vs operating is this site's name for that split. Building an insurance agent is a factory run: FR-0003 takes the pattern PAT-INTAKE-API 2.1.0 and the spec SPEC-INTAKE-001, and produces code, contracts, tests and a rollout manifest. Operating an underwriting agent is a runtime run: run-7f3a takes the broker's submission SUB-1004 and produces a recommendation that an underwriter reads before deciding. A tiny example of the difference: delete FR-0003's output and nothing happens to any insurance case; delete run-7f3a's record and the insurer loses part of the history of a real decision.

The operational consequence is strict. The factory never makes an underwriting decision and never touches production data. Its station workers see the spec, the pattern and synthetic fixtures. The service's agent sees submissions, and only at run time, in production, under the service's controls. The trade-off is a little duplication: the factory has its own run record, trace and permissions, and the service has its own. They must not be merged to save effort.

Building: factory run FR-0003Operating: runtime run run-7f3a
Inputpattern PAT-INTAKE-API 2.1.0 + spec SPEC-INTAKE-001submission SUB-1004 from a broker
Who actsfour deterministic stations, S1 to S4 (and in a larger factory, station workers)the service's code, the underwriting agent (a scripted stand-in here) and an underwriter
Outputopenapi.json, event_schemas.json, data_contract.json, service_config.json, rollout_manifest.json and a manifest with digestsa recommendation, a human decision, events, an audit trail
Data it may seethe spec and synthetic fixtures (cases.json)real submissions, in production only
What "done" meansevery station passed and every gate is greenan underwriter recorded a decision
IdentifierFR-0003, specification_hashrun_id, trace_id, submission_id

Diagram: Building vs operating · two runs that touch at release

Two lanes. The top lane is building: factory run FR-0003 reads the pattern and the spec, runs four deterministic stations, and publishes artifacts with a manifest. It sees synthetic fixtures only. Release carries the built service into the bottom lane. The bottom lane is operating: at run time, run-7f3a takes submission SUB-1004 through the service, the underwriting agent recommends, and an underwriter decides. Nothing flows from the bottom lane back into the factory's stations; what operation teaches comes back later as a defect or a spec change, through people.

  1. Building · factory run FR-0003
    1. pattern + spec: The reusable skeleton and the product's own fields, roles, routes and workflow.
    2. four stations: Deterministic code checks the spec, writes the contracts, assembles the service configuration and records digests.
    3. artifacts + manifest: Five files and a manifest with their sha256 digests; no submission, no recommendation, no decision.
  2. Operating · runtime run run-7f3a
    1. broker submits: A real case enters the released service; the factory is not involved.
    2. agent recommends: The underwriting agent (a scripted stand-in here) writes a recommendation; it cannot decide.
    3. underwriter decides: A person with the underwriter role records the decision; R-HUMAN-01.

Read the diagram as two lanes that touch at exactly one point. In the top lane the factory reads the pattern and the spec, runs four stations, and publishes artifacts with a manifest. Release carries the built service into the bottom lane. In the bottom lane, at run time, a broker's submission enters the service, the underwriting agent recommends, and an underwriter decides. No arrow goes from the bottom lane back to the factory's stations: what the factory learns from operation arrives later, as a defect report or a spec change, through people.

Before AI. A payroll team has always kept two things apart: the CI pipeline that builds the payroll system, and the monthly payroll run that pays people. Nobody lets the build server issue paychecks, and nobody debugs a payroll run by rerunning the build. The factory and the underwriting service are the same pair, with one new risk: both now contain something called an agent.

The flow, station by station#

The insurer's flow has six steps after the broker presses send. Teaching reconstruction: the full flow below is this site's reference design; Example C implements the steps from the typed submission onward, and the email reading and the guidance search are described, not run.

The rule for placing each step is the site's recurring one: use the least machinery that does the work reliably. A step whose logic can be written in advance is code. A step that interprets variable text is where a model can earn its place. A step that carries a consequence for a customer belongs to a person with the authority for it. A wait between two steps is a queue.

StepWho actsWhy that and nothing moreIn Example C
Broker sends an emailan outside partythe input arrives as free textnot modelled; a JSON body arrives
Intake: email to draft submissionthe intake agent drafts; the broker or a validation step confirmsreading free text is semanticnot run; part 1 builds the endpoint
Intake: validate and storecodethe schema is known in advancePOST /submissions, role broker
Assessment: roof evidencecode calling an outside provider, with bounded retriesthe call is known; only its timing variesStubRoofProvider, at most 3 attempts
Retrieval: guidance and similar casesa read tool, possibly a retrieval specialistfinding relevant passages is semantic; reading them must not grant powernot run; the rules come from UW-GUIDE 2026.2
Recommendationthe underwriting agentweighing evidence against rules and writing reasons is semanticthe scripted stand-in underwriting.py : recommend
Underwriter decisiona person with the underwriter roleR-HUMAN-01: an authorized underwriter decidesPOST /submissions/{submission_id}/decision, underwriter only
Auditcodea record must be exact and completeone audit record per request

The underwriter is the person who decides whether the insurer offers cover and on what terms; here, a principal with the underwriter role. The recommendation is the agent's written proposal to that person, in the series shape: escalate, the reasons, the rule ids it cites, what is missing, a stated confidence and requires_human: true. The model recommends; a person decides, and Example C keeps the two in separate routes, separate roles and separate records.

Diagram: Who acts at each step · code, agents, people, queues

Rows are who acts: outside parties, deterministic code, agents, and people. Columns are the flow from the broker's email to the audit. SUB-1004 starts as an email, enters the agent row only for the draft, spends most of its life in the code row (validation, the provider call with bounded retries, the audit), rises into the agent row a second time for the recommendation, and ends in the people row, where the underwriter decides. Two boxes out of eight are agents. Every arrow into the people row passes through a queue. Teaching reconstruction: Example C implements the code steps, the stand-in recommendation and the decision route; the email reading and the guidance search are described, not run.

  1. Outside parties
    1. broker email: The case arrives as text the service cannot trust.
  2. Deterministic code
    1. validate + store: Schema validation, then received to validated.
    2. provider call, bounded retries: Fetches roof evidence; a stub with scripted timeouts in Example C.
    3. awaiting_underwriter: The case waits for a person; underwriters work on their own schedule.
    4. audit record: One record per request, including refusals.
  3. Agents
    1. intake agent drafts submission: Reads the email and writes a draft that the broker or a validation step confirms.
    2. underwriting agent recommends: Writes the recommendation in the series shape, with reasons and cited rules.
  4. People
    1. underwriter decides: The human gate: a named role, a role check, a record of override.

The swimlane has four rows: outside parties, deterministic code, agents, and people. Follow SUB-1004 from left to right. It starts as the broker's email in the top row, drops into the agent row only for the draft, and spends most of its life in the code row: validation, storage, the provider call with retries, the audit. It rises into the agent row a second time for the recommendation, and the last step sits in the people row, where the underwriter decides. Two of the eight boxes are agents. Every arrow into the people row passes through a queue.

The smallest primitive: transitions as data#

A workflow transition is a permitted move of a case from one named state to the next. A tiny example: ["awaiting_underwriter", "decided"] lets a case move from waiting to decided, and nothing in the list lets a case move from validated straight to decided. In Example C the spec, not the service code, holds the list:

{
  "states": ["received", "validated", "assessed", "recommended", "awaiting_underwriter", "decided"],
  "transitions": [
    ["received", "validated"],
    ["validated", "assessed"],
    ["assessed", "recommended"],
    ["recommended", "awaiting_underwriter"],
    ["awaiting_underwriter", "decided"]
  ],
  "human_decision_state": "awaiting_underwriter"
}

The pattern's skeleton enforces it with four lines. From examples/c-insurance-service/service.py, method Service._transition, verbatim:

def _transition(self, rec: dict, to: str) -> None:
    if [rec["state"], to] not in self.cfg["workflow"]["transitions"]:
        raise TransitionError(f"{rec['state']} -> {to} is not allowed")
    rec["state"] = to
    rec["history"].append(to)

The consequence is that the order of the flow is a fact of the product, written in its spec and checked by code, not a habit of whoever calls the service. An agent that skips the assessment, a script that replays an old request, a person who clicks too early: each one meets the same list. The trade-off is that changing the flow means changing the spec and running the factory again, which is the point. A new step for a senior underwriter's review would be a spec change with a version, a review and new tests.

The executable demonstration#

The script below was written for this page and run on a scratch copy of examples/c-insurance-service/, from inside that folder. It prints the factory's record, then operates the built service on case UW-004 and tries three things the flow forbids.

"""Chapter 21 scratch demo: the factory record (building) next to the runtime record (operating),
and the workflow transitions the spec allows, refused when a request skips one."""
import factory
from clock import FixedClock
from run import req
from service import Service, TransitionError

pattern, spec, policy, cases = factory.load_inputs()
config, manifest = factory.assemble(pattern, spec)          # factory run FR-0003: BUILDS the service
print("BUILDING:", manifest["factory_run_id"], manifest["pattern_id"], manifest["pattern_version"],
      manifest["spec_id"], manifest["specification_hash"][:19] + "...")
print("  stations:", ", ".join(s["station_id"] for s in manifest["stations"]))
print("  output:", len(manifest["artifacts"]), "artifacts + a manifest; no submission, no recommendation, no decision")

svc, provider = factory.build_service(config, cases, policy, FixedClock())
b = cases["cases"]["UW-004"]["body"]
r = svc.handle(req("POST", "/submissions", "broker", b))
sid, run_id = r["body"]["submission_id"], r["body"]["run_id"]
print("OPERATING:", run_id, "for", sid, "state", r["body"]["state"])

# A decision before any recommendation: the spec has no transition validated -> decided.
r = svc.handle(req("POST", f"/submissions/{sid}/decision", "underwriter",
                   {"decision": "approve", "reason": "skip the review"}, key="d-early"))
print("  underwriter decides before assessment:", r["status"], r["body"])
r = svc.handle(req("POST", f"/submissions/{sid}/assessment", "underwriting-agent", key="a-1"))
print("  agent assesses:", r["status"], r["body"]["state"], r["body"]["recommendation"]["recommendation"])
r = svc.handle(req("POST", f"/submissions/{sid}/decision", "underwriting-agent",
                   {"decision": "refer", "reason": "agent tries"}, key="d-agent"))
print("  agent records a decision:", r["status"], r["body"])
r = svc.handle(req("POST", f"/submissions/{sid}/decision", "underwriter",
                   {"decision": "refer", "reason": "Roof ages differ by 13 years; inspection ordered."}, key="d-1"))
print("  underwriter decides:", r["status"], r["body"]["state"], r["body"]["human_decision"]["decision"],
      "override", r["body"]["human_decision"]["override"])
print("  history:", " > ".join(svc.store[sid]["history"]))
try:
    svc._transition(svc.store[sid], "assessed")
except TransitionError as e:
    print("  one more transition after decided:", e)
print("  provider calls:", provider.calls, "writes:", svc.writes)
BUILDING: FR-0003 PAT-INTAKE-API 2.1.0 SPEC-INTAKE-001 sha256:ecb151ab7f7f...
  stations: S1-validate-spec, S2-generate-contracts, S3-assemble-service, S4-write-manifest
  output: 5 artifacts + a manifest; no submission, no recommendation, no decision
OPERATING: run-7f3a for SUB-1004 state validated
  underwriter decides before assessment: 409 {'error': 'invalid_transition', 'state': 'validated'}
  agent assesses: 200 awaiting_underwriter escalate
  agent records a decision: 403 {'error': 'forbidden', 'required_roles': ['underwriter']}
  underwriter decides: 200 decided refer override False
  history: received > validated > assessed > recommended > awaiting_underwriter > decided
  one more transition after decided: decided -> assessed is not allowed
  provider calls: {'SUB-1004': 1} writes: {'recommendation': 1, 'decision': 1}

Read the output in three pairs. The first two lines show building: four stations passed, and the factory's output is files, not cases. The next pair shows two different refusals with two different causes. The early decision returns 409 because the case is in validated and the spec has no path from there to decided, even though the caller is an underwriter. The agent's decision returns 403 because its token carries the role underwriting-agent, and the decision route admits underwriter only, whatever state the case is in. The last pair shows the one path that works: the agent recommends escalate (the application says 9 years, the provider 22, and R-ROOF-03 allows a gap of 5), the underwriter records refer, and override False because refer is the decision that follows an escalate (service.py : FOLLOWS). One recommendation write, one decision write, one provider call.

The agent here is a scripted stand-in, so these results show how the factory mechanics work, not how good any model is. The last line of interest, the attempt to move a decided case back to assessed, calls the private _transition directly to show the list at work; no route in the service would even try it.

Authorization lives outside the agent's instructions#

The 403 above did not come from the agent's good behavior. The stand-in never asked; the script sent the request with the agent's token, as a confused or manipulated model might. The refusal came from step 3 of the pattern's skeleton, which runs before any business handler. From service.py, Service._pipeline, verbatim (an excerpt from inside the method, so it is shown as text):

# 3 role check (deny by default)
if not auth.has_role(principal, route["roles"]):
    return _r(403, {"error": "forbidden", "required_roles": route["roles"]})

and from auth.py, verbatim:

def has_role(principal: dict, allowed: list[str]) -> bool:
    """Role check (pattern skeleton step: role_check). Deny by default."""
    return any(r in allowed for r in principal.get("roles", []))

This is the sentence from the spec this page owns: ownership and authorization must be enforced independently of agent instructions. A shared tool, a shared route or a shared agent may serve many teams; who may do what is decided by the service that holds the data, from the caller's identity, every time. The instruction "never record a decision" is still worth writing, because it makes the agent less likely to try. It is not a control, because the thing it controls is the same thing that reads untrusted text.

The factory installs this in two layers. At build time, station 1 refuses a spec whose decision route admits anyone other than underwriter; Classify the enterprise demand shows that refusal running. At run time, the skeleton's role check refuses the request. Either layer alone would leave a gap: a correct spec does not stop a forged request, and a correct role check does not stop a team from shipping a spec that widens the role. How identity, roles and delegation work in depth belongs to Auth Engineering.

Diagram: Run run-7f3a on SUB-1004 · the role check refuses the agent's decision

Time runs down the page. The broker's submission passes the skeleton and starts run-7f3a. The agent's assessment request passes the role check for its own route; the skeleton calls the stub provider once, the stand-in writes the recommendation, and the case waits in awaiting_underwriter. The agent's request on the decision route stops at the role check with 403 before any handler runs, whatever the agent was told. The underwriter's request passes, the decision is recorded, and every request, including the refusal, leaves an audit record.

  1. Broker to Service skeleton: POST /submissions (SUB-1004). Body validated against schema v1; the run starts.
  2. Underwriting agent to Service skeleton: POST assessment + Idempotency-Key. The role check passes: this route admits underwriting-agent.
  3. Service skeleton to Roof provider (stub): get_roof_assessment(SUB-1004). One attempt of at most three.
  4. Service skeleton to Underwriting agent: 200 recommendation: escalate. Recorded once with the idempotency key; state awaiting_underwriter.
  5. Underwriting agent to Service skeleton: POST decision. Stopped at the role check: 403, required_roles underwriter. No handler runs.
  6. Underwriter to Service skeleton: POST decision: refer. Role check passes; state awaiting_underwriter allows the transition to decided.
  7. Service skeleton: audit every request. Each request, including the refusal, leaves one audit record.

The sequence follows run-7f3a down the page. The broker's request passes the skeleton and starts the run. The agent's assessment request passes the role check for its own route; the skeleton calls the stub provider once, the stand-in writes the recommendation, and the case waits for a person. The agent's attempt on the decision route stops at the role check with 403 before any handler runs. The underwriter's request passes, the decision is recorded, and the audit holds every step, including the refusal.

Six ways to arrange the work#

Once the steps are placed, the remaining question is how agents and code are arranged around them. The spec of this site lists six arrangements, and each one is right somewhere. Pattern: these are common architectures, not mandated ones.

ArrangementWhat it isFits whenFails when
One agent per full requestone agent, one long context, all tools, start to finisha low-risk internal task with few tools and a cheap check at the endpermissions must differ per step; the context grows past what the model reads well; the agent grades its own work
Sequential stationsa fixed chain; each station hands a typed artifact to the nextthe order is known and each step has a checkable outputsteps that could run side by side wait in line
Parallel specialistsseveral agents work at once on independent parts, then a step mergesparts are truly independent (two evidence sources)parts share files or facts; the merge becomes the hard work
Manager-workera coordinating agent splits the work and assigns it to workers at run timethe split differs a lot from case to casethe split is the same every time, so the manager adds cost and a place to go wrong
Planner-executorone agent writes a plan, another (or code) carries it out, with a check betweenplans can be reviewed or validated before anything runsthe plan cannot be checked, so the check is a formality
Deterministic graph, agents only at semantic stationsthe flow is fixed code; an agent staffs a station only where the work is interpretationthe business has already decided the flow, as in insurancethe work really is open-ended research, where a fixed graph cannot anticipate the steps

This site's recommendation for the insurer is the last row, and Example C is built that way. The six states and five transitions are fixed by the spec; the business decided them, and an agent choosing the next state would be choosing something already chosen. The agents sit at the two semantic stations: reading the email and writing the recommendation. Everything that judges, records or refuses is code. The arrangement buys a short list of things that can go wrong at each station, and a test for each. It costs flexibility: a case that needs a step the graph lacks goes to a person, not to an improvising agent.

The underwriting agent is the agent that gathers evidence with read tools and writes the recommendation, holding only the role that allows exactly that: in Example C, underwriting-agent, which may call the assessment route and nothing else. The intake agent is the agent at the front door that turns a broker's email into a draft submission for the broker or a validation step to confirm; Example C does not include it, because its service accepts JSON. They are two agents with two roles, not one agent with two jobs, because their inputs, tools and risks differ: the intake agent reads outside text and writes a draft; the underwriting agent reads internal evidence and writes a recommendation.

Where sub-agents belong#

A sub-agent earns its place when one semantic step needs a different context, tool set or permission from the agent that calls it. In the insurer's flow the candidate is retrieval: a specialist that searches the guidance with search_guidance(query) and finds past cases with find_similar_decisions(submission_id), then returns passages with their sources. Separating it keeps the underwriting agent's context small and keeps the search tools out of the hands of anything that writes. The cost is a handoff: if the specialist returns a summary instead of the passages and their ids, the recommendation can no longer cite what it relied on. Teaching reconstruction: Example C has no retrieval station; the uwmap lab shows it as "none of its own".

Where queues belong#

A queue goes wherever the next step works at a different speed or on a different schedule from the step before. The insurer's flow has two such places. Between intake and assessment, the roof-data provider is slow and sometimes unavailable, so submissions wait for it without holding the broker's request open. And before every human gate: underwriters work office hours, so the case waits in a work queue. In Example C the state awaiting_underwriter plays that role, and the in-memory EventBus stands in for a broker such as Kafka; no real queue runs. Looping over a large backlog owns how queues and limits behave.

Where human gates belong#

A human gate is a point in the flow where work stops until a named person with the authority for it decides. The rule for placing one is consequence, not difficulty: the decision to offer or refuse cover affects a business, so R-HUMAN-01 puts a person there for every case, including the easy ones the stand-in marks approve. A gate is not a review screen that someone clicks through; it has an owner, a role check, a record of who decided and whether they followed or overrode the recommendation, and a queue with a service level. A gate that underwriters clear by habit has become a step that looks like control and is not one; the folded block on rubber-stamping below says how to notice.

In a coding agent. When the factory itself uses a coding agent to build this flow, the agent is a station worker of FR-0003, never a participant in run-7f3a. It receives one work item (for example, "add the retrieval station's read tool"), a scoped checkout and synthetic fixtures; it never receives a production submission or a credential for the case system. The factory's tests decide whether its change advances, and a person with merge rights reviews it. The controls around that run are Harness Engineering.
What this is not. This page places agents, queues and gates inside one product the factory builds. The control plane that runs many agents at run time (routing, supervision, budgets) is Agent Orchestration Engineering; how agents exchange messages and tasks with each other is A2A Engineering. The trust controls the factory installs around this flow are the next chapter, Insurance factory: trust and observability.

Try it#

The lab maps the six stations of the insurer's line to Example C. In the build view it shows what factory run FR-0003 builds and tests for each station; switch to the run view to follow SUB-1004 through run-7f3a. For any station with a route, the "Try as" select shows which roles get through. The second embed opens directly on the run view.

StationRoutes (roles)Example C testsRequests for SUB-1004 (step, actor, status)Events of run-7f3a
IntakePOST /submissions (broker); GET /submissions/{submission_id} (auditor, broker, underwriter)11 of 11 pass1 broker 201; 3 no token 401; 5 other broker 404evt-0001 submission.received.v1, evt-0002 submission.validated.v1
AssessmentPOST /submissions/{submission_id}/assessment (underwriting-agent), Idempotency-Key5 of 5 pass6 underwriting-agent 200, 1 provider attemptevt-0003 submission.assessed.v1
Retrievalnone: Example C reads the policy fixture UW-GUIDE 2026.2 and the provider's evidence; no search indexnone of its ownnonenone
Recommendationinside the assessment route; record_recommendation with an idempotency key8 of 8 pass6: escalate, citations R-ROOF-03, R-HUMAN-01, stated_confidence 0.6 (canned)evt-0004 recommendation.recorded.v1
Underwriter decisionPOST /submissions/{submission_id}/decision (underwriter only), Idempotency-Key8 of 8 pass4 broker 403; 7 underwriting-agent 403; 8 underwriter 200, decision refer, override falseevt-0005 decision.recorded.v1
AuditGET /audit (auditor)5 of 5 pass18 auditor 200, saw 17 records (18 after its own read)none

States of run-7f3a: received, validated, assessed, recommended, awaiting_underwriter, decided. Test outcomes come from running Example C's suite (50 tests, all pass). The model recommends; a person decides.

WatchAlgo in practice#

WatchAlgo's line had no human gate and no run-time product agent, so the building-versus-operating split never arose there: the factory's output was static content, not a service that acts. What it does show is the arrangement this page recommends, chosen in practice. Observed in the repository (docs/EVIDENCE.md, section 5): in September 2026 the coordinator was pipeline.py run under supervise.sh, the queue was queue.json under a file lock, and the stations ran in a fixed order: a visualization task waited for its solutions. Acceptance was deterministic code (verify_solution, verify_viz), not an agent. That is a deterministic graph with agents only at the semantic stations, solution writing and tracer writing.

Observed (docs/EVIDENCE.md, matrix row 9): a separate engine in the CosmicKeys backend defines roles planner, worker, gate, reviewer and consensus per stage (engine.py, commit 93e2a0b1), a manager-and-planner design, and the repository shows no evidence that it produced published content. Observed (section 5, "Human role"): people started runs, set concurrency, requeued tasks and stood workers down; no human accepted or rejected units. For content with an executable oracle that was a defensible choice. For an insurance decision it is not available: R-HUMAN-01 puts a person at the end of every case.

Enterprise translation#

In an enterprise, the six-step flow becomes several systems with several owners. The platform team owns PAT-INTAKE-API and its skeleton: the role check, the transition check and the audit are theirs, identical in every product built from the pattern. The underwriting domain team owns the spec: the states, the transitions, the rules R-ROOF-01 to R-ROOF-03 and the roles. The identity team owns which people and workloads hold which roles; granting underwriter to a workload is a change in their system, reviewed by them, not a line in a spec or a prompt.

The queues become real infrastructure with service levels: a durable work queue for underwriters with an age alarm, and a retry queue for provider calls with a dead-letter path to a person. The flow itself may run in a workflow engine that persists each case's state across restarts; the choice of engine is the tooling landscape's question. What must not change in translation is the placement rule: the flow is fixed by the business, agents sit at semantic stations, and every consequence passes a person.

Failure test#

The failure: the underwriting agent records a decision. Three paths lead there: a spec that opens the decision route to the agent's role, a credential for the agent that carries the underwriter role, and an agent that is told, by a retrieved passage, to call the decision route.

How the factory catches it: the first path is refused at build time by station 1 (the demand chapter's scratch run shows the exact message). The second is outside the factory's code and is caught only if the factory's contract gate and the identity team's review agree on which principals hold underwriter; the contract check proves that the decision route lists underwriter alone, and the identity system decides who holds it. The third is refused at run time by the role check, as the demonstration's 403 shows, whatever the agent was told; the next chapter turns that into a test the factory installs in every product.

What to run or toggle: in the uwmap lab, select the underwriter decision station and set "Try as" to underwriting-agent. In a scratch copy of Example C, run the demonstration script above, then change the agent's decision request to use the role underwriter and watch it pass: the service trusts the token, so the token's roles are where the control lives. Run /opt/homebrew/bin/python3 -m unittest tests.test_workflow to see the 409 and 403 cases as tests.

Where to go deeper#

Checklist for Monday#

Exercises#

  1. A product owner asks the factory team to "run FR-0004 against last week's real submissions to check the new recommender". What is wrong with the request, and what should happen instead?
Answer 1The factory never touches production data

It asks the building system to operate on production data. The factory checks the recommender against synthetic golden cases; checking it against real submissions is an evaluation owned by the operating side, with its own approval, data access and record, and its results feed back as a spec change or a defect.

  1. Which of the six arrangements would you choose for a research assistant that helps an underwriter explore an unusual risk with no fixed steps, and what stays fixed even there?
Answer 2A looser arrangement, same authorization

One agent or planner-executor fits open-ended exploration better than a fixed graph. What stays fixed is that the assistant holds read tools only, its output is a note to the underwriter, and the decision route still admits underwriter alone.

  1. In the demonstration, why does the underwriter's early decision get 409 and not 403?
Answer 3Right person, wrong moment

The role check passes: the caller holds underwriter. The handler then checks the state, finds validated instead of awaiting_underwriter, and refuses the transition. 403 means "not you"; 409 means "not now".

Advanced perspective#

What a manager agent costs when the split never changesCoordination you pay for on every case

A manager-worker arrangement puts a model call at the start of every case to decide how to split it. When the split is the same every time, as in a flow fixed by the business, that call adds latency, model cost and a new failure mode (a manager that skips the assessment on an unusual email) in exchange for no flexibility anyone uses. The fixed graph does the same split in code at no cost and cannot skip a step.

The honest test is to log the manager's plans for a few hundred synthetic cases. If more than a handful of distinct plans appear, the manager is doing real work and the question is whether those variations should be in the spec. If one plan dominates, the plan belongs in the spec as transitions and the manager can go. Agent Orchestration Engineering covers the run-time control plane for the cases where dynamic routing is justified.

When a human gate becomes a rubber stampMeasure the gate, not only the agent

A gate is only as good as the attention behind it. Underwriters under time pressure who agree with a confident recommendation by habit turn R-HUMAN-01 into a formality. Example C records the signal needed to notice: each decision carries override true or false against the recommendation (service.py : FOLLOWS). An override rate near zero across many cases, with short decision times, is a warning, not a success.

Operational responses that keep the gate real: show the evidence and the cited rules before the recommendation, not after; route a sample of cases to a second underwriter and compare; seed the queue with synthetic cases whose correct answer differs from the recommendation and record whether they are caught. The last one is a golden case used on the people, and it needs the same governance as any test that touches staff: announced, owned and reviewed.

Retrying a step that writesEvery station that writes needs a key

The assessment route calls a provider and then writes a recommendation. If the caller times out and retries, a naive service would call the provider again and write a second recommendation. Example C requires an Idempotency-Key on that route: the same key and body replay the stored response without a second write or provider call, a different body with the same key is 409, and a 503 is not stored, so the caller may retry with the same key. In the README's run, SUB-1102 is assessed twice with one key: the provider count stays at 3 (the two timeouts and the success of the first request), and the total of recommendation writes stays at 4 for 4 assessed submissions.

The orchestration lesson is that the arrangement decides who retries. In a fixed graph the workflow retries a station with the same key. In a manager-worker arrangement the manager may decide to "try again" with a fresh request, and a fresh key defeats idempotency. Whatever arranges the work must reuse the key it was given. Long-running agent execution owns idempotency in depth; Agent Reliability Engineering owns retries as a discipline.

What changes when the stand-in becomes a modelThe arrangement holds; the evidence changes

Replacing underwriting.py : recommend with a model call changes one station's behavior from exact to variable. It does not change the transitions, the role check, the idempotency key or the human gate, which is why they were built as code. What changes is the evidence the factory needs before release: the stand-in passes the four recommendation cases by construction; a model must be measured on golden cases, including the injected passage of UW-005, with an agreed threshold, as an evaluation gate. The next chapter installs that gate.

One more change is easy to miss: stated_confidence stops being canned and starts being a number the model writes about itself. It is a self-report. The underwriter's screen should present it as such, beside the cited evidence, never as a measured probability.

In one breath#

Building an insurance agent and operating an underwriting agent are two runs with two ids: factory run FR-0003 turns a pattern and a spec into a tested service and never sees a real case, while runtime run run-7f3a processes SUB-1004 and ends with an underwriter's decision. Each step of the flow takes the least machinery that does it: code for validation, provider calls and audit; an agent only for reading the email and writing the recommendation; a queue where speeds differ; a person where there is a consequence. The workflow transitions live in the spec and the skeleton refuses any other move, and authorization is enforced by the service from the caller's role, independently of anything the agent was told.

The flow is now fixed and its decision is protected by a role check, but nobody can yet prove afterwards what happened in run-7f3a, who saw the broker's email address, or whether a poisoned passage would change the recommendation. Those are questions of trust and evidence, and the factory answers them by installing stations and gates in every product it builds. Insurance factory: trust and observability follows the same run through audit, trace, masking, replay and golden cases.