The design proposal for the insurer's underwriting review fits on one slide. "One agent handles each submission end to end. It reads the broker's email, checks the roof data, looks up the guidelines, writes the recommendation and updates the case. Its instructions say: never record a final decision; leave that to the underwriter."
The reviewer from the security team asks one question. "What stops it?" The answer is the sentence in the instructions. Nothing else. The agent holds one credential that can do everything the case system allows, and the only thing between that credential and a recorded decision is a paragraph the agent reads, alongside the broker's email and every retrieved passage, any of which might contain a paragraph of its own. A second engineer asks a different question: "When the factory builds this, does the factory's agent make the decisions in testing?" Two different things in the room are both called "the agent", and nobody has said which one they mean.
The learner's question: for the flow from a broker's email to an underwriter's decision, which steps are plain code, which deserve an agent, where do queues and human gates belong, and how does a factory build that flow without ever running it on a real case?
Two things called "the agent"#
Plain English. The factory is a machine that builds machines. The insurance service it builds contains its own agent, which works on real submissions after release. The factory's run and the service's run are different events, with different ids, owners and permissions.
Building vs operating is this site's name for that split. Building an insurance agent is a factory run: FR-0003 takes the pattern PAT-INTAKE-API 2.1.0 and the spec SPEC-INTAKE-001, and produces code, contracts, tests and a rollout manifest. Operating an underwriting agent is a runtime run: run-7f3a takes the broker's submission SUB-1004 and produces a recommendation that an underwriter reads before deciding. A tiny example of the difference: delete FR-0003's output and nothing happens to any insurance case; delete run-7f3a's record and the insurer loses part of the history of a real decision.
The operational consequence is strict. The factory never makes an underwriting decision and never touches production data. Its station workers see the spec, the pattern and synthetic fixtures. The service's agent sees submissions, and only at run time, in production, under the service's controls. The trade-off is a little duplication: the factory has its own run record, trace and permissions, and the service has its own. They must not be merged to save effort.
Building: factory run FR-0003 | Operating: runtime run run-7f3a | |
|---|---|---|
| Input | pattern PAT-INTAKE-API 2.1.0 + spec SPEC-INTAKE-001 | submission SUB-1004 from a broker |
| Who acts | four deterministic stations, S1 to S4 (and in a larger factory, station workers) | the service's code, the underwriting agent (a scripted stand-in here) and an underwriter |
| Output | openapi.json, event_schemas.json, data_contract.json, service_config.json, rollout_manifest.json and a manifest with digests | a recommendation, a human decision, events, an audit trail |
| Data it may see | the spec and synthetic fixtures (cases.json) | real submissions, in production only |
| What "done" means | every station passed and every gate is green | an underwriter recorded a decision |
| Identifier | FR-0003, specification_hash | run_id, trace_id, submission_id |
Diagram: Building vs operating · two runs that touch at release
Two lanes. The top lane is building: factory run FR-0003 reads the pattern and the spec, runs four deterministic stations, and publishes artifacts with a manifest. It sees synthetic fixtures only. Release carries the built service into the bottom lane. The bottom lane is operating: at run time, run-7f3a takes submission SUB-1004 through the service, the underwriting agent recommends, and an underwriter decides. Nothing flows from the bottom lane back into the factory's stations; what operation teaches comes back later as a defect or a spec change, through people.
- Building · factory run FR-0003
- pattern + spec: The reusable skeleton and the product's own fields, roles, routes and workflow.
- four stations: Deterministic code checks the spec, writes the contracts, assembles the service configuration and records digests.
- artifacts + manifest: Five files and a manifest with their sha256 digests; no submission, no recommendation, no decision.
- Operating · runtime run run-7f3a
- broker submits: A real case enters the released service; the factory is not involved.
- agent recommends: The underwriting agent (a scripted stand-in here) writes a recommendation; it cannot decide.
- underwriter decides: A person with the underwriter role records the decision; R-HUMAN-01.
Read the diagram as two lanes that touch at exactly one point. In the top lane the factory reads the pattern and the spec, runs four stations, and publishes artifacts with a manifest. Release carries the built service into the bottom lane. In the bottom lane, at run time, a broker's submission enters the service, the underwriting agent recommends, and an underwriter decides. No arrow goes from the bottom lane back to the factory's stations: what the factory learns from operation arrives later, as a defect report or a spec change, through people.
Before AI. A payroll team has always kept two things apart: the CI pipeline that builds the payroll system, and the monthly payroll run that pays people. Nobody lets the build server issue paychecks, and nobody debugs a payroll run by rerunning the build. The factory and the underwriting service are the same pair, with one new risk: both now contain something called an agent.
The flow, station by station#
The insurer's flow has six steps after the broker presses send. Teaching reconstruction: the full flow below is this site's reference design; Example C implements the steps from the typed submission onward, and the email reading and the guidance search are described, not run.
The rule for placing each step is the site's recurring one: use the least machinery that does the work reliably. A step whose logic can be written in advance is code. A step that interprets variable text is where a model can earn its place. A step that carries a consequence for a customer belongs to a person with the authority for it. A wait between two steps is a queue.
| Step | Who acts | Why that and nothing more | In Example C |
|---|---|---|---|
| Broker sends an email | an outside party | the input arrives as free text | not modelled; a JSON body arrives |
| Intake: email to draft submission | the intake agent drafts; the broker or a validation step confirms | reading free text is semantic | not run; part 1 builds the endpoint |
| Intake: validate and store | code | the schema is known in advance | POST /submissions, role broker |
| Assessment: roof evidence | code calling an outside provider, with bounded retries | the call is known; only its timing varies | StubRoofProvider, at most 3 attempts |
| Retrieval: guidance and similar cases | a read tool, possibly a retrieval specialist | finding relevant passages is semantic; reading them must not grant power | not run; the rules come from UW-GUIDE 2026.2 |
| Recommendation | the underwriting agent | weighing evidence against rules and writing reasons is semantic | the scripted stand-in underwriting.py : recommend |
| Underwriter decision | a person with the underwriter role | R-HUMAN-01: an authorized underwriter decides | POST /submissions/{submission_id}/decision, underwriter only |
| Audit | code | a record must be exact and complete | one audit record per request |
The underwriter is the person who decides whether the insurer offers cover and on what terms; here, a principal with the underwriter role. The recommendation is the agent's written proposal to that person, in the series shape: escalate, the reasons, the rule ids it cites, what is missing, a stated confidence and requires_human: true. The model recommends; a person decides, and Example C keeps the two in separate routes, separate roles and separate records.
Diagram: Who acts at each step · code, agents, people, queues
Rows are who acts: outside parties, deterministic code, agents, and people. Columns are the flow from the broker's email to the audit. SUB-1004 starts as an email, enters the agent row only for the draft, spends most of its life in the code row (validation, the provider call with bounded retries, the audit), rises into the agent row a second time for the recommendation, and ends in the people row, where the underwriter decides. Two boxes out of eight are agents. Every arrow into the people row passes through a queue. Teaching reconstruction: Example C implements the code steps, the stand-in recommendation and the decision route; the email reading and the guidance search are described, not run.
- Outside parties
- broker email: The case arrives as text the service cannot trust.
- Deterministic code
- validate + store: Schema validation, then received to validated.
- provider call, bounded retries: Fetches roof evidence; a stub with scripted timeouts in Example C.
- awaiting_underwriter: The case waits for a person; underwriters work on their own schedule.
- audit record: One record per request, including refusals.
- Agents
- intake agent drafts submission: Reads the email and writes a draft that the broker or a validation step confirms.
- underwriting agent recommends: Writes the recommendation in the series shape, with reasons and cited rules.
- People
- underwriter decides: The human gate: a named role, a role check, a record of override.
The swimlane has four rows: outside parties, deterministic code, agents, and people. Follow SUB-1004 from left to right. It starts as the broker's email in the top row, drops into the agent row only for the draft, and spends most of its life in the code row: validation, storage, the provider call with retries, the audit. It rises into the agent row a second time for the recommendation, and the last step sits in the people row, where the underwriter decides. Two of the eight boxes are agents. Every arrow into the people row passes through a queue.
The smallest primitive: transitions as data#
A workflow transition is a permitted move of a case from one named state to the next. A tiny example: ["awaiting_underwriter", "decided"] lets a case move from waiting to decided, and nothing in the list lets a case move from validated straight to decided. In Example C the spec, not the service code, holds the list:
{
"states": ["received", "validated", "assessed", "recommended", "awaiting_underwriter", "decided"],
"transitions": [
["received", "validated"],
["validated", "assessed"],
["assessed", "recommended"],
["recommended", "awaiting_underwriter"],
["awaiting_underwriter", "decided"]
],
"human_decision_state": "awaiting_underwriter"
}
The pattern's skeleton enforces it with four lines. From examples/c-insurance-service/service.py, method Service._transition, verbatim:
def _transition(self, rec: dict, to: str) -> None:
if [rec["state"], to] not in self.cfg["workflow"]["transitions"]:
raise TransitionError(f"{rec['state']} -> {to} is not allowed")
rec["state"] = to
rec["history"].append(to)
The consequence is that the order of the flow is a fact of the product, written in its spec and checked by code, not a habit of whoever calls the service. An agent that skips the assessment, a script that replays an old request, a person who clicks too early: each one meets the same list. The trade-off is that changing the flow means changing the spec and running the factory again, which is the point. A new step for a senior underwriter's review would be a spec change with a version, a review and new tests.
The executable demonstration#
The script below was written for this page and run on a scratch copy of examples/c-insurance-service/, from inside that folder. It prints the factory's record, then operates the built service on case UW-004 and tries three things the flow forbids.
"""Chapter 21 scratch demo: the factory record (building) next to the runtime record (operating),
and the workflow transitions the spec allows, refused when a request skips one."""
import factory
from clock import FixedClock
from run import req
from service import Service, TransitionError
pattern, spec, policy, cases = factory.load_inputs()
config, manifest = factory.assemble(pattern, spec) # factory run FR-0003: BUILDS the service
print("BUILDING:", manifest["factory_run_id"], manifest["pattern_id"], manifest["pattern_version"],
manifest["spec_id"], manifest["specification_hash"][:19] + "...")
print(" stations:", ", ".join(s["station_id"] for s in manifest["stations"]))
print(" output:", len(manifest["artifacts"]), "artifacts + a manifest; no submission, no recommendation, no decision")
svc, provider = factory.build_service(config, cases, policy, FixedClock())
b = cases["cases"]["UW-004"]["body"]
r = svc.handle(req("POST", "/submissions", "broker", b))
sid, run_id = r["body"]["submission_id"], r["body"]["run_id"]
print("OPERATING:", run_id, "for", sid, "state", r["body"]["state"])
# A decision before any recommendation: the spec has no transition validated -> decided.
r = svc.handle(req("POST", f"/submissions/{sid}/decision", "underwriter",
{"decision": "approve", "reason": "skip the review"}, key="d-early"))
print(" underwriter decides before assessment:", r["status"], r["body"])
r = svc.handle(req("POST", f"/submissions/{sid}/assessment", "underwriting-agent", key="a-1"))
print(" agent assesses:", r["status"], r["body"]["state"], r["body"]["recommendation"]["recommendation"])
r = svc.handle(req("POST", f"/submissions/{sid}/decision", "underwriting-agent",
{"decision": "refer", "reason": "agent tries"}, key="d-agent"))
print(" agent records a decision:", r["status"], r["body"])
r = svc.handle(req("POST", f"/submissions/{sid}/decision", "underwriter",
{"decision": "refer", "reason": "Roof ages differ by 13 years; inspection ordered."}, key="d-1"))
print(" underwriter decides:", r["status"], r["body"]["state"], r["body"]["human_decision"]["decision"],
"override", r["body"]["human_decision"]["override"])
print(" history:", " > ".join(svc.store[sid]["history"]))
try:
svc._transition(svc.store[sid], "assessed")
except TransitionError as e:
print(" one more transition after decided:", e)
print(" provider calls:", provider.calls, "writes:", svc.writes)
BUILDING: FR-0003 PAT-INTAKE-API 2.1.0 SPEC-INTAKE-001 sha256:ecb151ab7f7f...
stations: S1-validate-spec, S2-generate-contracts, S3-assemble-service, S4-write-manifest
output: 5 artifacts + a manifest; no submission, no recommendation, no decision
OPERATING: run-7f3a for SUB-1004 state validated
underwriter decides before assessment: 409 {'error': 'invalid_transition', 'state': 'validated'}
agent assesses: 200 awaiting_underwriter escalate
agent records a decision: 403 {'error': 'forbidden', 'required_roles': ['underwriter']}
underwriter decides: 200 decided refer override False
history: received > validated > assessed > recommended > awaiting_underwriter > decided
one more transition after decided: decided -> assessed is not allowed
provider calls: {'SUB-1004': 1} writes: {'recommendation': 1, 'decision': 1}
Read the output in three pairs. The first two lines show building: four stations passed, and the factory's output is files, not cases. The next pair shows two different refusals with two different causes. The early decision returns 409 because the case is in validated and the spec has no path from there to decided, even though the caller is an underwriter. The agent's decision returns 403 because its token carries the role underwriting-agent, and the decision route admits underwriter only, whatever state the case is in. The last pair shows the one path that works: the agent recommends escalate (the application says 9 years, the provider 22, and R-ROOF-03 allows a gap of 5), the underwriter records refer, and override False because refer is the decision that follows an escalate (service.py : FOLLOWS). One recommendation write, one decision write, one provider call.
The agent here is a scripted stand-in, so these results show how the factory mechanics work, not how good any model is. The last line of interest, the attempt to move a decided case back to assessed, calls the private _transition directly to show the list at work; no route in the service would even try it.
Authorization lives outside the agent's instructions#
The 403 above did not come from the agent's good behavior. The stand-in never asked; the script sent the request with the agent's token, as a confused or manipulated model might. The refusal came from step 3 of the pattern's skeleton, which runs before any business handler. From service.py, Service._pipeline, verbatim (an excerpt from inside the method, so it is shown as text):
# 3 role check (deny by default)
if not auth.has_role(principal, route["roles"]):
return _r(403, {"error": "forbidden", "required_roles": route["roles"]})
and from auth.py, verbatim:
def has_role(principal: dict, allowed: list[str]) -> bool:
"""Role check (pattern skeleton step: role_check). Deny by default."""
return any(r in allowed for r in principal.get("roles", []))
This is the sentence from the spec this page owns: ownership and authorization must be enforced independently of agent instructions. A shared tool, a shared route or a shared agent may serve many teams; who may do what is decided by the service that holds the data, from the caller's identity, every time. The instruction "never record a decision" is still worth writing, because it makes the agent less likely to try. It is not a control, because the thing it controls is the same thing that reads untrusted text.
The factory installs this in two layers. At build time, station 1 refuses a spec whose decision route admits anyone other than underwriter; Classify the enterprise demand shows that refusal running. At run time, the skeleton's role check refuses the request. Either layer alone would leave a gap: a correct spec does not stop a forged request, and a correct role check does not stop a team from shipping a spec that widens the role. How identity, roles and delegation work in depth belongs to Auth Engineering.
Diagram: Run run-7f3a on SUB-1004 · the role check refuses the agent's decision
Time runs down the page. The broker's submission passes the skeleton and starts run-7f3a. The agent's assessment request passes the role check for its own route; the skeleton calls the stub provider once, the stand-in writes the recommendation, and the case waits in awaiting_underwriter. The agent's request on the decision route stops at the role check with 403 before any handler runs, whatever the agent was told. The underwriter's request passes, the decision is recorded, and every request, including the refusal, leaves an audit record.
- Broker to Service skeleton: POST /submissions (SUB-1004). Body validated against schema v1; the run starts.
- Underwriting agent to Service skeleton: POST assessment + Idempotency-Key. The role check passes: this route admits underwriting-agent.
- Service skeleton to Roof provider (stub): get_roof_assessment(SUB-1004). One attempt of at most three.
- Service skeleton to Underwriting agent: 200 recommendation: escalate. Recorded once with the idempotency key; state awaiting_underwriter.
- Underwriting agent to Service skeleton: POST decision. Stopped at the role check: 403, required_roles underwriter. No handler runs.
- Underwriter to Service skeleton: POST decision: refer. Role check passes; state awaiting_underwriter allows the transition to decided.
- Service skeleton: audit every request. Each request, including the refusal, leaves one audit record.
The sequence follows run-7f3a down the page. The broker's request passes the skeleton and starts the run. The agent's assessment request passes the role check for its own route; the skeleton calls the stub provider once, the stand-in writes the recommendation, and the case waits for a person. The agent's attempt on the decision route stops at the role check with 403 before any handler runs. The underwriter's request passes, the decision is recorded, and the audit holds every step, including the refusal.
Six ways to arrange the work#
Once the steps are placed, the remaining question is how agents and code are arranged around them. The spec of this site lists six arrangements, and each one is right somewhere. Pattern: these are common architectures, not mandated ones.
| Arrangement | What it is | Fits when | Fails when |
|---|---|---|---|
| One agent per full request | one agent, one long context, all tools, start to finish | a low-risk internal task with few tools and a cheap check at the end | permissions must differ per step; the context grows past what the model reads well; the agent grades its own work |
| Sequential stations | a fixed chain; each station hands a typed artifact to the next | the order is known and each step has a checkable output | steps that could run side by side wait in line |
| Parallel specialists | several agents work at once on independent parts, then a step merges | parts are truly independent (two evidence sources) | parts share files or facts; the merge becomes the hard work |
| Manager-worker | a coordinating agent splits the work and assigns it to workers at run time | the split differs a lot from case to case | the split is the same every time, so the manager adds cost and a place to go wrong |
| Planner-executor | one agent writes a plan, another (or code) carries it out, with a check between | plans can be reviewed or validated before anything runs | the plan cannot be checked, so the check is a formality |
| Deterministic graph, agents only at semantic stations | the flow is fixed code; an agent staffs a station only where the work is interpretation | the business has already decided the flow, as in insurance | the work really is open-ended research, where a fixed graph cannot anticipate the steps |
This site's recommendation for the insurer is the last row, and Example C is built that way. The six states and five transitions are fixed by the spec; the business decided them, and an agent choosing the next state would be choosing something already chosen. The agents sit at the two semantic stations: reading the email and writing the recommendation. Everything that judges, records or refuses is code. The arrangement buys a short list of things that can go wrong at each station, and a test for each. It costs flexibility: a case that needs a step the graph lacks goes to a person, not to an improvising agent.
The underwriting agent is the agent that gathers evidence with read tools and writes the recommendation, holding only the role that allows exactly that: in Example C, underwriting-agent, which may call the assessment route and nothing else. The intake agent is the agent at the front door that turns a broker's email into a draft submission for the broker or a validation step to confirm; Example C does not include it, because its service accepts JSON. They are two agents with two roles, not one agent with two jobs, because their inputs, tools and risks differ: the intake agent reads outside text and writes a draft; the underwriting agent reads internal evidence and writes a recommendation.
Where sub-agents belong#
A sub-agent earns its place when one semantic step needs a different context, tool set or permission from the agent that calls it. In the insurer's flow the candidate is retrieval: a specialist that searches the guidance with search_guidance(query) and finds past cases with find_similar_decisions(submission_id), then returns passages with their sources. Separating it keeps the underwriting agent's context small and keeps the search tools out of the hands of anything that writes. The cost is a handoff: if the specialist returns a summary instead of the passages and their ids, the recommendation can no longer cite what it relied on. Teaching reconstruction: Example C has no retrieval station; the uwmap lab shows it as "none of its own".
Where queues belong#
A queue goes wherever the next step works at a different speed or on a different schedule from the step before. The insurer's flow has two such places. Between intake and assessment, the roof-data provider is slow and sometimes unavailable, so submissions wait for it without holding the broker's request open. And before every human gate: underwriters work office hours, so the case waits in a work queue. In Example C the state awaiting_underwriter plays that role, and the in-memory EventBus stands in for a broker such as Kafka; no real queue runs. Looping over a large backlog owns how queues and limits behave.
Where human gates belong#
A human gate is a point in the flow where work stops until a named person with the authority for it decides. The rule for placing one is consequence, not difficulty: the decision to offer or refuse cover affects a business, so R-HUMAN-01 puts a person there for every case, including the easy ones the stand-in marks approve. A gate is not a review screen that someone clicks through; it has an owner, a role check, a record of who decided and whether they followed or overrode the recommendation, and a queue with a service level. A gate that underwriters clear by habit has become a step that looks like control and is not one; the folded block on rubber-stamping below says how to notice.
In a coding agent. When the factory itself uses a coding agent to build this flow, the agent is a station worker ofFR-0003, never a participant inrun-7f3a. It receives one work item (for example, "add the retrieval station's read tool"), a scoped checkout and synthetic fixtures; it never receives a production submission or a credential for the case system. The factory's tests decide whether its change advances, and a person with merge rights reviews it. The controls around that run are Harness Engineering.
What this is not. This page places agents, queues and gates inside one product the factory builds. The control plane that runs many agents at run time (routing, supervision, budgets) is Agent Orchestration Engineering; how agents exchange messages and tasks with each other is A2A Engineering. The trust controls the factory installs around this flow are the next chapter, Insurance factory: trust and observability.
Try it#
The lab maps the six stations of the insurer's line to Example C. In the build view it shows what factory run FR-0003 builds and tests for each station; switch to the run view to follow SUB-1004 through run-7f3a. For any station with a route, the "Try as" select shows which roles get through. The second embed opens directly on the run view.
| Station | Routes (roles) | Example C tests | Requests for SUB-1004 (step, actor, status) | Events of run-7f3a |
|---|---|---|---|---|
| Intake | POST /submissions (broker); GET /submissions/{submission_id} (auditor, broker, underwriter) | 11 of 11 pass | 1 broker 201; 3 no token 401; 5 other broker 404 | evt-0001 submission.received.v1, evt-0002 submission.validated.v1 |
| Assessment | POST /submissions/{submission_id}/assessment (underwriting-agent), Idempotency-Key | 5 of 5 pass | 6 underwriting-agent 200, 1 provider attempt | evt-0003 submission.assessed.v1 |
| Retrieval | none: Example C reads the policy fixture UW-GUIDE 2026.2 and the provider's evidence; no search index | none of its own | none | none |
| Recommendation | inside the assessment route; record_recommendation with an idempotency key | 8 of 8 pass | 6: escalate, citations R-ROOF-03, R-HUMAN-01, stated_confidence 0.6 (canned) | evt-0004 recommendation.recorded.v1 |
| Underwriter decision | POST /submissions/{submission_id}/decision (underwriter only), Idempotency-Key | 8 of 8 pass | 4 broker 403; 7 underwriting-agent 403; 8 underwriter 200, decision refer, override false | evt-0005 decision.recorded.v1 |
| Audit | GET /audit (auditor) | 5 of 5 pass | 18 auditor 200, saw 17 records (18 after its own read) | none |
States of run-7f3a: received, validated, assessed, recommended, awaiting_underwriter, decided. Test outcomes come from running Example C's suite (50 tests, all pass). The model recommends; a person decides.
WatchAlgo in practice#
WatchAlgo's line had no human gate and no run-time product agent, so the building-versus-operating split never arose there: the factory's output was static content, not a service that acts. What it does show is the arrangement this page recommends, chosen in practice. Observed in the repository (docs/EVIDENCE.md, section 5): in September 2026 the coordinator was pipeline.py run under supervise.sh, the queue was queue.json under a file lock, and the stations ran in a fixed order: a visualization task waited for its solutions. Acceptance was deterministic code (verify_solution, verify_viz), not an agent. That is a deterministic graph with agents only at the semantic stations, solution writing and tracer writing.
Observed (docs/EVIDENCE.md, matrix row 9): a separate engine in the CosmicKeys backend defines roles planner, worker, gate, reviewer and consensus per stage (engine.py, commit 93e2a0b1), a manager-and-planner design, and the repository shows no evidence that it produced published content. Observed (section 5, "Human role"): people started runs, set concurrency, requeued tasks and stood workers down; no human accepted or rejected units. For content with an executable oracle that was a defensible choice. For an insurance decision it is not available: R-HUMAN-01 puts a person at the end of every case.
Enterprise translation#
In an enterprise, the six-step flow becomes several systems with several owners. The platform team owns PAT-INTAKE-API and its skeleton: the role check, the transition check and the audit are theirs, identical in every product built from the pattern. The underwriting domain team owns the spec: the states, the transitions, the rules R-ROOF-01 to R-ROOF-03 and the roles. The identity team owns which people and workloads hold which roles; granting underwriter to a workload is a change in their system, reviewed by them, not a line in a spec or a prompt.
The queues become real infrastructure with service levels: a durable work queue for underwriters with an age alarm, and a retry queue for provider calls with a dead-letter path to a person. The flow itself may run in a workflow engine that persists each case's state across restarts; the choice of engine is the tooling landscape's question. What must not change in translation is the placement rule: the flow is fixed by the business, agents sit at semantic stations, and every consequence passes a person.
Failure test#
The failure: the underwriting agent records a decision. Three paths lead there: a spec that opens the decision route to the agent's role, a credential for the agent that carries the underwriter role, and an agent that is told, by a retrieved passage, to call the decision route.
How the factory catches it: the first path is refused at build time by station 1 (the demand chapter's scratch run shows the exact message). The second is outside the factory's code and is caught only if the factory's contract gate and the identity team's review agree on which principals hold underwriter; the contract check proves that the decision route lists underwriter alone, and the identity system decides who holds it. The third is refused at run time by the role check, as the demonstration's 403 shows, whatever the agent was told; the next chapter turns that into a test the factory installs in every product.
What to run or toggle: in the uwmap lab, select the underwriter decision station and set "Try as" to underwriting-agent. In a scratch copy of Example C, run the demonstration script above, then change the agent's decision request to use the role underwriter and watch it pass: the service trusts the token, so the token's roles are where the control lives. Run /opt/homebrew/bin/python3 -m unittest tests.test_workflow to see the 409 and 403 cases as tests.
Where to go deeper#
- Insurance factory: trust and observability (next): the audit, trace, masking, guardrails and golden cases the factory installs around this flow.
- Insurance factory: contract intake: the intake endpoint, its schema and its error responses.
- AI enters the factory: the one agent, specialized agents or functions decision for a single unit of work.
- Enterprise reference patterns: why the intake service is an integration service with a document-intake front door.
- Agent Orchestration Engineering: the control plane for running agents at run time.
- A2A Engineering: messages and tasks between agents.
- Auth Engineering: identity, roles and delegation for people and workloads.
Checklist for Monday#
- Give the factory run and the runtime run different ids, records and credentials; write down that the factory sees synthetic data only.
- Place every step with the least machinery: code where the logic is known, an agent where the work is interpretation, a person where there is a consequence, a queue where the speed changes.
- Write the workflow states and transitions in the spec, and have the skeleton refuse any transition not in the list.
- Enforce roles in the service, from the caller's identity, before any handler runs; treat instructions to the agent as guidance, never as the control.
- Give each agent its own role with only the routes it needs; two jobs with different inputs mean two agents.
- For every human gate, name the role, the queue, the service level and the record of override.
Exercises#
- A product owner asks the factory team to "run FR-0004 against last week's real submissions to check the new recommender". What is wrong with the request, and what should happen instead?
Answer 1The factory never touches production data
It asks the building system to operate on production data. The factory checks the recommender against synthetic golden cases; checking it against real submissions is an evaluation owned by the operating side, with its own approval, data access and record, and its results feed back as a spec change or a defect.
- Which of the six arrangements would you choose for a research assistant that helps an underwriter explore an unusual risk with no fixed steps, and what stays fixed even there?
Answer 2A looser arrangement, same authorization
One agent or planner-executor fits open-ended exploration better than a fixed graph. What stays fixed is that the assistant holds read tools only, its output is a note to the underwriter, and the decision route still admits underwriter alone.
- In the demonstration, why does the underwriter's early decision get 409 and not 403?
Answer 3Right person, wrong moment
The role check passes: the caller holds underwriter. The handler then checks the state, finds validated instead of awaiting_underwriter, and refuses the transition. 403 means "not you"; 409 means "not now".
Advanced perspective#
What a manager agent costs when the split never changesCoordination you pay for on every case
A manager-worker arrangement puts a model call at the start of every case to decide how to split it. When the split is the same every time, as in a flow fixed by the business, that call adds latency, model cost and a new failure mode (a manager that skips the assessment on an unusual email) in exchange for no flexibility anyone uses. The fixed graph does the same split in code at no cost and cannot skip a step.
The honest test is to log the manager's plans for a few hundred synthetic cases. If more than a handful of distinct plans appear, the manager is doing real work and the question is whether those variations should be in the spec. If one plan dominates, the plan belongs in the spec as transitions and the manager can go. Agent Orchestration Engineering covers the run-time control plane for the cases where dynamic routing is justified.
When a human gate becomes a rubber stampMeasure the gate, not only the agent
A gate is only as good as the attention behind it. Underwriters under time pressure who agree with a confident recommendation by habit turn R-HUMAN-01 into a formality. Example C records the signal needed to notice: each decision carries override true or false against the recommendation (service.py : FOLLOWS). An override rate near zero across many cases, with short decision times, is a warning, not a success.
Operational responses that keep the gate real: show the evidence and the cited rules before the recommendation, not after; route a sample of cases to a second underwriter and compare; seed the queue with synthetic cases whose correct answer differs from the recommendation and record whether they are caught. The last one is a golden case used on the people, and it needs the same governance as any test that touches staff: announced, owned and reviewed.
Retrying a step that writesEvery station that writes needs a key
The assessment route calls a provider and then writes a recommendation. If the caller times out and retries, a naive service would call the provider again and write a second recommendation. Example C requires an Idempotency-Key on that route: the same key and body replay the stored response without a second write or provider call, a different body with the same key is 409, and a 503 is not stored, so the caller may retry with the same key. In the README's run, SUB-1102 is assessed twice with one key: the provider count stays at 3 (the two timeouts and the success of the first request), and the total of recommendation writes stays at 4 for 4 assessed submissions.
The orchestration lesson is that the arrangement decides who retries. In a fixed graph the workflow retries a station with the same key. In a manager-worker arrangement the manager may decide to "try again" with a fresh request, and a fresh key defeats idempotency. Whatever arranges the work must reuse the key it was given. Long-running agent execution owns idempotency in depth; Agent Reliability Engineering owns retries as a discipline.
What changes when the stand-in becomes a modelThe arrangement holds; the evidence changes
Replacing underwriting.py : recommend with a model call changes one station's behavior from exact to variable. It does not change the transitions, the role check, the idempotency key or the human gate, which is why they were built as code. What changes is the evidence the factory needs before release: the stand-in passes the four recommendation cases by construction; a model must be measured on golden cases, including the injected passage of UW-005, with an agreed threshold, as an evaluation gate. The next chapter installs that gate.
One more change is easy to miss: stated_confidence stops being canned and starts being a number the model writes about itself. It is a self-report. The underwriter's screen should present it as such, beside the cited evidence, never as a measured probability.
In one breath#
Building an insurance agent and operating an underwriting agent are two runs with two ids: factory run FR-0003 turns a pattern and a spec into a tested service and never sees a real case, while runtime run run-7f3a processes SUB-1004 and ends with an underwriter's decision. Each step of the flow takes the least machinery that does it: code for validation, provider calls and audit; an agent only for reading the email and writing the recommendation; a queue where speeds differ; a person where there is a consequence. The workflow transitions live in the spec and the skeleton refuses any other move, and authorization is enforced by the service from the caller's role, independently of anything the agent was told.
The flow is now fixed and its decision is protected by a role check, but nobody can yet prove afterwards what happened in run-7f3a, who saw the broker's email address, or whether a poisoned passage would change the recommendation. Those are questions of trust and evidence, and the factory answers them by installing stations and gates in every product it builds. Insurance factory: trust and observability follows the same run through audit, trace, masking, replay and golden cases.