All chapters
  1. Start hereA software factory, from first principles
  2. Primitive · 1A factory, before AI: input, transformation, check, output
  3. Primitive · 2From repetition to a pattern: what stays, what varies, what to leave alone
  4. Primitive · 3The specification as an executable contract
  5. Primitive · 4Build the smallest factory: four stations, two specs, one manifest
  6. Primitive · 5Assembly stations: typed contracts, owners and failure handling
  7. Primitive · 6AI enters the factory: what a model adds, and what stays deterministic
  8. WatchAlgo · 7The WatchAlgo production story: what the repository shows
  9. WatchAlgo · 8A problem as a production unit: the files one unit must contain
  10. WatchAlgo · 9Station by station: description, solution, implementations, visualization
  11. WatchAlgo · 10The visualizer factory: from algorithm state to renderable steps
  12. WatchAlgo · 11Three flavors and three languages: two different contracts
  13. WatchAlgo · 12Looping over a large backlog: scheduling, limits and a failed-case queue
  14. WatchAlgo · 13Long-running agent execution: context, checkpoints and token limits
  15. WatchAlgo · 14Evidence of correctness: tests the producer cannot talk its way past
  16. WatchAlgo · 15The factory learning loop: defects change the pattern, not only the unit
  17. Enterprise · 16Classify the enterprise demand before choosing a pattern
  18. Enterprise · 17Reference enterprise patterns and their variation points
  19. Enterprise · 18Platform foundations: golden paths, templates, registries and catalogs
  20. Enterprise · 19Pattern versus configuration versus business logic
  21. Enterprise · 20Insurance factory, part 1: building the intake contract
  22. Enterprise · 21Insurance factory, part 2: building the underwriting workflow
  23. Enterprise · 22Insurance factory, part 3: trust and observability built in
  24. Enterprise · 23Data and integration engineering: contracts that survive change
  25. Enterprise · 24The CI/CD and deployment factory: from pull request to production
  26. Enterprise · 25Enterprise governance: owners, approved patterns and provenance
  27. Enterprise · 26Scaling safely: many teams, many agents, one set of boundaries
  28. Operations · 27Factory quality engineering: measuring accepted output, not activity
  29. Operations · 28Failure and recovery scenarios: what breaks a factory, and what restores it
  30. Operations · 29The landscape: which tool for which job, read from its own documentation
  31. Operations · 30Design an enterprise software factory: the blueprint and the rollout
  32. ReferenceSources and verification
  33. ReferenceGlossary
  34. AboutAbout this site
Primitive · 4

Build the smallest factory: four stations, two specs, one manifest

A runnable assembly line with no agent: validate the spec, render the artifact, compile and test, write a manifest with hashes. Run it twice and prove the output is identical.

You will run
Example A end to end on your own machine: four stations, three specs, one command, with the exact commands to repeat it.
You will read
A manifest that binds one artifact to the spec, the pattern and the test evidence that produced it.
You will prove
A second run is byte-identical, a changed spec is caught at station 3, and an edited template cannot hide.

Two weeks after the tiny factory ran, a colleague opens running_sum.py in the shared folder and asks three questions. Which spec was this built from? Did the tests that passed run against this file, or an earlier one? If I run the factory again today, do I get the same file?

Without a record, every answer is a memory. The file says nothing about where it came from. The test run printed "OK" on someone's screen and disappeared. And if a template changed in the meantime, a rerun might produce a slightly different file, and nobody could say whether the difference matters.

The previous three chapters gave the factory a primitive, a pattern and a validated specification. This chapter closes the loop: it runs all four stations of Example A, keeps a record that answers the colleague's three questions with hashes instead of memories, and shows that the answers hold up when the factory runs again.

The learner's question: what does the smallest complete factory look like when it runs, and what must it leave behind so that anyone can check what it built, from what, and get the same result again?

The line in one picture#

Plain English. The smallest factory reads a request, checks it, builds the product from fixed templates, tests the product, and writes a receipt that names every input and every output by its fingerprint.

Beginner definition: a tiny factory is a script that runs the same fixed stations over every specification it receives and always writes a receipt, whether the product was accepted or refused.

Example A (examples/a-assembly-line/) has four stations, and every one of them is plain Python. Station 1 validates the spec, station 2 renders a module and its test file from templates, station 3 compiles both files and runs the tests in a separate process, and station 4 writes the receipt. One run, factory run FR-0001, takes three specs through the line: running-sum (SPEC-ALGO-101), clamp-list (SPEC-ALGO-102) and the incomplete median spec (SPEC-ALGO-103) from the previous chapter.

Before AI. A Maven build already has this shape: it reads a pom.xml, compiles, runs the tests and packages a jar. A Make target with a test prerequisite refuses to produce dist/ when a test fails. Example A is the same idea, cut down until every line of it fits on one page.

Diagram: Factory run FR-0001 · three specs, four stations, a receipt for every one

Read left to right. Three spec files enter station 1. Running-sum and clamp-list pass and go on to rendering from templates and to compile-and-test in a separate process. The median spec fails station 1 with six errors; stations 2 and 3 are skipped for it. Every path ends at station 4, which writes a manifest for accepted and rejected specs alike. On disk: two accepted units with their files and manifests, one rejection with a manifest and no code, and run.json listing all three outcomes.

  1. Specs
    1. running-sum.json: Valid spec using the prefix_sum body.
    2. clamp-list.json: Valid spec using the clamp_each body; one example expects ValueError.
    3. median-incomplete.json: Deliberately incomplete spec from chapter 3.
  2. Stations
    1. S1 validate: Checks the spec against schema.json and the pattern; returns every error.
    2. S2 render: Fills the module, test and body templates; no judgment, same bytes every time.
    3. S3 compile + test: py_compile both files, then unittest in a subprocess; returns evidence.
    4. S4 manifest: Writes the receipt with spec hash, pattern digest, artifact hashes and evidence.
  3. On disk
    1. 2 accepted units: Each folder holds the rendered files and the manifest that names them by hash.
    2. 1 recorded rejection: No code was built; the manifest lists the six errors and the skipped stations.
    3. run.json: Lists each spec's status and the SHA-256 of its manifest.

Read the diagram left to right. Each spec enters station 1. A valid spec moves on to rendering and testing; the median spec stops at station 1, and stations 2 and 3 are marked skipped. All three paths meet again at station 4, because the receipt is written for every spec. The rightmost lane shows what lands on disk: two accepted units with their files, one rejection that is a record rather than a silence, and a run.json that lists all three.

Run it yourself#

This is the tiny factory the site promised a beginner could run. You need Python 3 and nothing else: no network, no key, no package to install. The outputs on this page come from Python 3.14 at /opt/homebrew/bin/python3; use your own python3 if it lives elsewhere. Work in a copy so the repository stays clean:

cp -R examples/a-assembly-line /tmp/tiny-factory
cd /tmp/tiny-factory
/opt/homebrew/bin/python3 run.py --verbose
/opt/homebrew/bin/python3 -m unittest discover -s tests -t .

The first command copies the example from the repository root. The third runs the factory and prints every station. The last runs the example's own 14 tests, which check the line itself (Ran 14 tests and OK on 2026-11-20). Here is the first part of the factory's output, from a run in a scratch copy on 2026-11-20:

== FR-0001: three specs through four stations (pattern PAT-ALGO-UNIT 1.3.0)
FR-0001 running-sum.json
  S1-validate-spec: pass (0 errors)
  S2-render-artifact: pass (running_sum.py, test_running_sum.py)
  S3-compile-test: pass (tests_run=4, failed=[])
  S4-write-manifest: pass (manifest.json, status=accepted)
FR-0001 clamp-list.json
  S1-validate-spec: pass (0 errors)
  S2-render-artifact: pass (clamp_list.py, test_clamp_list.py)
  S3-compile-test: pass (tests_run=5, failed=[])
  S4-write-manifest: pass (manifest.json, status=accepted)
FR-0001 median-incomplete.json
  S1-validate-spec: fail (6 errors)
  S4-write-manifest: pass (manifest.json, status=rejected_at_S1)
spec_id        problem         S1    S2       S3       status          spec_hash
SPEC-ALGO-101  running-sum     pass  pass     pass     accepted        1deb0c4b830f
SPEC-ALGO-102  clamp-list      pass  pass     pass     accepted        7af02f7aef73
SPEC-ALGO-103  median-of-list  fail  skipped  skipped  rejected_at_S1  32c1e1899da6

SPEC-ALGO-103 rejected at S1-validate-spec with 6 errors:
  - $.pattern: required field missing
  - $.acceptance_criteria: required field missing
  - $.body_template: 'median_sorted' is not in the pattern's template catalog (clamp_each, prefix_sum)
  - $.requirements: must be a non-empty list of non-empty strings
  - $.examples[1]: needs exactly one of 'output' or 'raises'
  - $.examples[2]: has 2 arguments, params declares 1

Two specs are accepted and one is refused with six exact errors. The previous chapter walked through those errors and the validator that produces them, so this chapter takes station 1 as given and follows the accepted specs through the rest of the line. The run leaves this on disk: out/FR-0001/running-sum/ and out/FR-0001/clamp-list/ each hold a module, its test file and a manifest.json; out/FR-0001/median-of-list/ holds only a manifest.json; and out/FR-0001/run.json lists the three outcomes.

Station 2: rendering from templates#

A template is a file with named holes that a program fills with values to produce a concrete file. Example A has a module template, a test template, and one body template per kind of logic. A form letter is the non-AI analogy: the text is fixed, the name and address are filled in. The minimal example is the body template templates/bodies/prefix_sum.py.tmpl, quoted as it is:

    total = 0
    result = []
    for x in ${first_param}:
        total += x
        result.append(total)
    return result

The only hole is ${first_param}, which becomes xs for the running sum. The operational consequence is that rendering involves no judgment: the same spec and the same templates always give the same bytes. The trade-off is the one the pattern chapter named: a template can only produce what someone already wrote into it.

A scaffold is the set of starting files a generator lays down for a new unit or project: here, the two rendered files. Station 2 is the function render_artifact in factory.py, quoted verbatim:

def render_artifact(spec: dict, pattern: dict, s_hash: str) -> dict[str, str]:
    """Render the module and its test file from the pattern's templates. Returns {relative path: text}."""
    module_name = spec["function_name"]
    body = Template((HERE / pattern["bodies"][spec["body_template"]]).read_text()).substitute(first_param=spec["params"][0])
    common = {
        "pattern_id": pattern["pattern_id"],
        "pattern_version": pattern["version"],
        "spec_id": spec["spec_id"],
        "spec_hash": s_hash,
        "function_name": spec["function_name"],
        "module_name": module_name,
        "problem_name": spec["problem_name"],
    }
    module = Template((HERE / pattern["templates"]["module"]).read_text()).substitute(
        common,
        params=", ".join(spec["params"]),
        requirements="\n".join(f"- {r}" for r in spec["requirements"]),
        body=body.rstrip("\n"),
    )
    test = Template((HERE / pattern["templates"]["test"]).read_text()).substitute(
        common, class_name=_class_name(spec["problem_name"]), test_methods=_test_methods(spec)
    )
    return {f"{module_name}.py": module, f"test_{module_name}.py": test}

Notice two things. The rendered module carries its own origin: the pattern id and version, the spec id and the spec hash go into its docstring, followed by "Do not edit by hand. Change the spec and run the line again." And the test file is rendered from the spec's examples by _test_methods, a different function from the one that renders the body. The product and its check come from the same spec through separate paths, which is the rule from chapter 1: the producer does not write the check.

Common confusion. A scaffold is a starting point; a factory station is a contract. Spring Initializr gives you a project and walks away. Station 2 renders the files and then hands them to station 3, which decides whether they are accepted, and station 4 records that decision.

Station 3: compile and test in a separate process#

Station 3 is compile_and_test in factory.py, quoted verbatim:

def compile_and_test(artifact_dir: Path, files: list[str]) -> dict:
    """py_compile every file (bytecode goes to a throwaway folder), then run the tests in a subprocess.

    The evidence keeps what is stable (counts, failing test names), never timings or absolute paths.
    """
    with tempfile.TemporaryDirectory() as tmp:
        for name in sorted(files):
            try:
                py_compile.compile(str(artifact_dir / name), cfile=os.path.join(tmp, name + "c"), doraise=True)
            except py_compile.PyCompileError as exc:
                return {"compiled": False, "compile_error": f"{name}: {exc.exc_type_name}", "result": "fail",
                        "tests_run": 0, "failed_tests": [], "command": "python -B -m unittest discover -s . -p test_*.py"}
    env = dict(os.environ, PYTHONDONTWRITEBYTECODE="1", PYTHONHASHSEED="0")
    proc = subprocess.run(TEST_COMMAND, cwd=artifact_dir, capture_output=True, text=True, env=env, timeout=60)
    ran = re.search(r"^Ran (\d+) tests?", proc.stderr, re.M)
    failed = sorted(set(re.findall(r"^(?:FAIL|ERROR): (\w+)", proc.stderr, re.M)))
    return {
        "command": "python -B -m unittest discover -s . -p test_*.py",
        "compiled": True,
        "exit_code": proc.returncode,
        "failed_tests": failed,
        "result": "pass" if proc.returncode == 0 else "fail",
        "tests_run": int(ran.group(1)) if ran else 0,
    }

The tests run in a subprocess, not inside the factory's own process, so a crashing test cannot take the factory down and the factory's state cannot leak into the test. The function returns evidence, not a yes or no: how many tests ran, which ones failed, the exit code and the command. That evidence is what station 4 writes down. Its last design choice matters most for the next section: it keeps counts and test names and drops timings and absolute paths, because those change from run to run for no business reason.

Station 4: the manifest#

A manifest is a machine-readable receipt for one artifact: which inputs produced it, which files it consists of, which checks ran and what they found. The analogy is the delivery note that travels with a pallet, listing the order number, the batch and the inspection stamp. The minimal example is the file below. The consequence is that the colleague's three questions now have answers that do not depend on anyone's memory. The trade-off is discipline: a manifest is only as trustworthy as its least stable field, so everything in it must be either a stable value or a fingerprint.

That fingerprint is a content hash: a short, fixed-length value computed from the exact bytes of a file or document, such that any change to the bytes gives a different value. Example A uses SHA-256. A change of one character in running_sum.py changes its hash completely, so the hash in the manifest identifies the file without storing it. Two content hashes in Example A are worth knowing by name. The spec hash (factory.py : spec_hash) is computed over the spec's canonical JSON, with sorted keys and fixed separators, so reformatting the file does not change it and changing one example does. The pattern digest (factory.py : pattern_digest) is computed over pattern.json and every template it names, so editing any template changes it.

Station 4 is the last block of run_line in factory.py, quoted verbatim:

    # S4
    started = clock.now()
    manifest = {
        "factory_run_id": run_id,
        "spec_id": spec.get("spec_id") if isinstance(spec, dict) else None,
        "problem_name": name,
        "spec_file": spec_path.name,
        "spec_hash": s_hash,
        "pattern": {"id": pattern["pattern_id"], "version": pattern["version"], "digest": pattern_digest(pattern)},
        "artifacts": artifacts,
        "test_evidence": evidence,
        "status": status,
        "stations": stations + [{"station_id": STATIONS[3], "status": "pass", "started_at": started}],
        "simulated": "No model is used. Station 2 is template rendering; the clock is injected.",
    }
    text = canonical_json(manifest)
    manifest_path = work / "manifest.json"
    manifest_path.write_text(text)

The snippet keeps its indentation because it sits inside run_line; it is shown as plain text for that reason. Here is the manifest it wrote for the running sum, out/FR-0001/running-sum/manifest.json, as it came out of the scratch run:

{
  "artifacts": [
    {
      "bytes": 504,
      "path": "running_sum.py",
      "sha256": "ac6459d79b3d6e1bca077bbbaf66b8b5c60888c630dfe380caa94b6bd2be7904"
    },
    {
      "bytes": 756,
      "path": "test_running_sum.py",
      "sha256": "ad5a9ca2a67e31c5166e71708e13e4e4f6e3c444895492a9ed177ee327374033"
    }
  ],
  "factory_run_id": "FR-0001",
  "pattern": {
    "digest": "b33a45fb114c16a0dbf45f06eccc1bfbd8b79d66f7a7a38d7c0b56f046fabd18",
    "id": "PAT-ALGO-UNIT",
    "version": "1.3.0"
  },
  "problem_name": "running-sum",
  "simulated": "No model is used. Station 2 is template rendering; the clock is injected.",
  "spec_file": "running-sum.json",
  "spec_hash": "1deb0c4b830fc8ac01b0ed6f22844ee5e7dcc4f7fd0c138ec007dd424b41db3d",
  "spec_id": "SPEC-ALGO-101",
  "stations": [
    {
      "errors": [],
      "finished_at": "2026-11-20T09:00:01Z",
      "started_at": "2026-11-20T09:00:00Z",
      "station_id": "S1-validate-spec",
      "status": "pass"
    },
    {
      "finished_at": "2026-11-20T09:00:03Z",
      "outputs": [
        "running_sum.py",
        "test_running_sum.py"
      ],
      "started_at": "2026-11-20T09:00:02Z",
      "station_id": "S2-render-artifact",
      "status": "pass"
    },
    {
      "finished_at": "2026-11-20T09:00:05Z",
      "started_at": "2026-11-20T09:00:04Z",
      "station_id": "S3-compile-test",
      "status": "pass"
    },
    {
      "started_at": "2026-11-20T09:00:06Z",
      "station_id": "S4-write-manifest",
      "status": "pass"
    }
  ],
  "status": "accepted",
  "test_evidence": {
    "command": "python -B -m unittest discover -s . -p test_*.py",
    "compiled": true,
    "exit_code": 0,
    "failed_tests": [],
    "result": "pass",
    "tests_run": 4
  }
}

Diagram: What the manifest binds · three questions, answered by hashes

Left: the inputs and outputs of one line, each reduced to a fingerprint. The spec becomes spec_hash over its canonical JSON. The pattern file and every template become the pattern digest. Each rendered file becomes its own SHA-256. Station 3's evidence is kept as counts and test names. Middle: the manifest holds all of them in one canonical file. Right: the three questions a colleague asks, each answered by one or two fields. The version string states intent; the digest states the exact bytes.

  1. Inputs and outputs
    1. spec file: Hashed with sorted keys and fixed separators, so reformatting does not change the hash.
    2. pattern + templates: pattern.json and every template it names, hashed together.
    3. rendered files: Each file hashed on its own and listed with its size.
    4. test evidence: What station 3 found, without timings or paths.
  2. manifest.json
    1. spec_hash: SHA-256 of the canonical spec.
    2. pattern.version + digest: Intent and identity of the pattern, side by side.
    3. artifacts[].sha256: One hash per file the unit consists of.
    4. status + test_evidence: The verdict and the evidence it rests on.
  3. The colleague's questions
    1. Which spec?: Hash the spec in the repository and compare.
    2. Did the tests run on this file?: Hash the file in the folder; if it matches, it is the file station 3 tested.
    3. Same result again?: Run again in another folder and compare hashes.

The diagram maps the colleague's three questions onto manifest fields. "Which spec?" is answered by spec_id and spec_hash. "Did the tests run against this file?" is answered by the artifact's sha256, recorded in the same manifest as the test evidence: hash the file in the folder, and if it matches, it is the file station 3 tested. "Which pattern built it?" is answered by pattern.version for intent and pattern.digest for the exact bytes of every template. The timestamps come from an injected clock that starts at 2026-11-20T09:00:00Z and moves one second per reading, so they are stable too; real wall-clock times belong in a run record, not in the artifact's receipt.

Two details make the manifest honest about failure. Station 4 runs for every spec, so the median spec has a manifest with status rejected_at_S1, its six errors, and stations 2 and 3 marked skipped with the reason. And the status field has three values, accepted, rejected_at_S1 and rejected_at_S3, so a refusal always says where it happened.

The second run: byte-identical#

Byte-identical means two files have exactly the same bytes, which you can check by comparing their content hashes. It is a stronger claim than "equivalent" or "the same output": a reordered key, a trailing space or a timestamp breaks it. The analogy is two prints from the same photographic negative compared under a loupe rather than at arm's length. run.py makes the claim and checks it: after the first run it runs the same specs again into a fresh temporary folder and compares the manifests byte by byte.

== Reproducibility: run the same specs again into a fresh folder, compare bytes
running-sum     manifest sha256 dfac170b404d6b4b vs dfac170b404d6b4b  identical=True
clamp-list      manifest sha256 4efcde45091a16a6 vs 4efcde45091a16a6  identical=True
median-of-list  manifest sha256 5a11a95861994452 vs 5a11a95861994452  identical=True
3/3 manifests byte-identical; run.json identical=True

A check written by the factory about itself deserves an outside check. This one was run for this page: two separate copies of the example in two different folders, each run once, then every output file hashed with the operating system's shasum. The commands use /tmp; the run for this page used two scratch folders, which gives the same relative paths.

cp -R examples/a-assembly-line /tmp/copy1
cp -R examples/a-assembly-line /tmp/copy2
(cd /tmp/copy1 && /opt/homebrew/bin/python3 run.py > /dev/null && shasum -a 256 out/FR-0001/run.json out/FR-0001/*/* > ../h1.txt)
(cd /tmp/copy2 && /opt/homebrew/bin/python3 run.py > /dev/null && shasum -a 256 out/FR-0001/run.json out/FR-0001/*/* > ../h2.txt)
diff /tmp/h1.txt /tmp/h2.txt && echo SAME
99b69507c53bab6fca7af05083453cc96c6f37ee3c28249c0ad22097343afa62  out/FR-0001/run.json
45761dbdc87f5368fd77e00338b2ba36fdfc02805e080ed3670f41c2682b493d  out/FR-0001/clamp-list/clamp_list.py
4efcde45091a16a65cc8ab1d52d387df103fdf3e328f29136871d35e118cf147  out/FR-0001/clamp-list/manifest.json
3bab294b53747be0e11f11b68d151198a37fc4e2d0373c1b71e0c4a122523627  out/FR-0001/clamp-list/test_clamp_list.py
5a11a9586199445268270750022eae008bae62d084026a479e8e496fad9a7285  out/FR-0001/median-of-list/manifest.json
dfac170b404d6b4b2ece97fc7b12fc38f9170397c21b4527a6f694374dfb3b7f  out/FR-0001/running-sum/manifest.json
ac6459d79b3d6e1bca077bbbaf66b8b5c60888c630dfe380caa94b6bd2be7904  out/FR-0001/running-sum/running_sum.py
ad5a9ca2a67e31c5166e71708e13e4e4f6e3c444895492a9ed177ee327374033  out/FR-0001/running-sum/test_running_sum.py
SAME

The listing is the hashes of the first copy; diff found no difference with the second, so it printed SAME. The hash of running_sum.py in the listing equals the sha256 in the manifest above, and the manifest hashes equal the ones run.py printed. The folders were different, which matters: if an absolute path had leaked into a manifest, the two copies would differ. What makes the line deterministic (an injected clock, sorted keys, a fixed hash seed for the test process, no timings) is described in chapter 1's folded block on determinism; here you have seen that it works.

A changed spec is a new input#

The last part of run.py changes one expected value in the running-sum spec ([5, -2, 7] now expects [5, 3, 9]) and runs it as factory run FR-0008:

== FR-0008: the running-sum spec with one example changed ([5, -2, 7] now expects [5, 3, 9])
FR-0008 running-sum-changed.json
  S1-validate-spec: pass (0 errors)
  S2-render-artifact: pass (running_sum.py, test_running_sum.py)
  S3-compile-test: fail (tests_run=4, failed=['test_example_3'])
  S4-write-manifest: pass (manifest.json, status=rejected_at_S3)
spec_hash before 1deb0c4b830f  after df066a312212  differ=True
S3-compile-test: fail  tests_run=4  failed_tests=['test_example_3']  status=rejected_at_S3
The line does not decide whether the code or the example is wrong. It refuses to accept the unit and names the failing test.

Read the hashes first. The spec changed by one number, so its hash changed completely, and the manifest records a different input. The body template did not change, so station 2 rendered the same function body; the rendered module still differs, because its docstring carries the new spec hash. Station 3 then found the disagreement between the function and the changed example and refused the unit. Chapter 3 showed the other way to reach station 3 with a bad spec: rename the clamp's parameters, and station 1 passes while all five tests fail.

Your own spec through the line#

The definition of done for this part of the site is that you can run the tiny factory on a request of your own. Here is the shortest way. Write a spec that reuses a body from the catalog, for example a running total of game scores, as specs/mine/cumulative-scores.json in your copy. The id SPEC-ALGO-104 is this page's own; Example A ships only 101 to 103.

{
  "schema_version": "1.0",
  "spec_id": "SPEC-ALGO-104",
  "problem_name": "cumulative-scores",
  "function_name": "cumulative_scores",
  "params": ["scores"],
  "pattern": {"id": "PAT-ALGO-UNIT", "version": "1.3.0"},
  "body_template": "prefix_sum",
  "requirements": [
    "Return the score total after each round, as a new list.",
    "Do not change the caller's list."
  ],
  "examples": [
    {"input": [[10, 0, 5]], "output": [10, 10, 15]},
    {"input": [[-1]], "output": [-1]}
  ],
  "acceptance_criteria": [
    {"id": "AC-1", "text": "Every example in the spec passes.", "check": "all_examples_pass"},
    {"id": "AC-2", "text": "The input list is unchanged after the call.", "check": "input_not_mutated"}
  ]
}

Then save this driver as my_line.py next to factory.py. It was written for this page; it calls the example's own run_factory with a run id this page picked, FR-0009:

"""Run your own spec files through Example A's four stations. Usage: python3 my_line.py specs/mine/*.json"""
import sys
from pathlib import Path

import factory

paths = [Path(p) for p in sys.argv[1:]]
results = factory.run_factory(paths, Path("out"), run_id="FR-0009", log=print)
for r in results:
    print(r.spec_id, r.status, r.spec_hash[:12])
    for e in r.errors:
        print("  -", e)

Run /opt/homebrew/bin/python3 my_line.py specs/mine/cumulative-scores.json. Output from the scratch copy:

FR-0009 cumulative-scores.json
  S1-validate-spec: pass (0 errors)
  S2-render-artifact: pass (cumulative_scores.py, test_cumulative_scores.py)
  S3-compile-test: pass (tests_run=3, failed=[])
  S4-write-manifest: pass (manifest.json, status=accepted)
SPEC-ALGO-104 accepted 773ddc68b0ea

Three tests ran: two examples and the mutation rule. Running the same command a second time gave the same manifest hash both times (9b6223af2a67…). Now break it on purpose: change an expected value, delete the acceptance criteria, or ask for a body that is not in the catalog, and watch which station refuses it. You have a reproducible tiny factory.

In a coding agent. Put a coding agent such as Claude Code, running headless, in the place of station 2 and every other station stays as it is: the spec still passes station 1 first, the tests still come from the spec, and station 4 still records the hashes of whatever the agent wrote. What changes is the claim of this section: an agent's output is not byte-identical between runs, so the manifest records what was produced and tested, not what would be produced again. AI enters the factory takes this up; the runtime controls around the agent belong to Harness Engineering.
What this is not. A manifest here is a receipt written by the factory for one artifact. Signed build provenance that a third party can verify (signatures, attestations, supply-chain standards) is a later and larger step, covered on Enterprise governance. A manifest is also not a run record: the durable log of every attempt, retry and resume point lives in Example D and on Long-running agent execution. And this page is not about the runtime controls around one coding agent's run; those belong to Harness Engineering.

Try it#

Two labs. The first, primitive, runs Example A's validator and renderer in the browser on the rejected median spec (preset broken). Edit it until station 1 passes: add the pattern, add acceptance criteria, fix the two examples. You will find that station 1 still refuses median_sorted until you pick a body that exists in the catalog, which is the point of the pattern chapter.

The table holds the same results for a reader without JavaScript. They are real outputs of examples/a-assembly-line/run.py, run on 2026-11-20 in a scratch copy; the last row is this page's own spec.

SpecStation 1Station 2Station 3StatusSpec hash (first 12)
SPEC-ALGO-101 running-sumpass, 0 errors2 files rendered4 run, 0 failedaccepted1deb0c4b830f
SPEC-ALGO-102 clamp-listpass, 0 errors2 files rendered5 run, 0 failedaccepted7af02f7aef73
SPEC-ALGO-103 median (preset broken)fail, 6 errorsskippedskippedrejected_at_S132c1e1899da6
SPEC-ALGO-101 changed (FR-0008)pass, 0 errors2 files rendered4 run, test_example_3 failedrejected_at_S3df066a312212
SPEC-ALGO-104 cumulative-scores (FR-0009)pass, 0 errors2 files rendered3 run, 0 failedaccepted773ddc68b0ea

The second lab, conveyor, steps the same line one event at a time (preset a, the running-sum spec). Pick a station to see its contract: inputs, outputs, validator, who does the work, dependencies and what happens on failure. The next chapter is about exactly those contracts.

The events, for a reader without JavaScript (labcore a_conveyor.py, built from a real run of running-sum.json):

StepStationStateNote
1 to 2S1-validate-specrunning, passed0 errors
3 to 4S2-render-artifactrunning, passedrunning_sum.py, test_running_sum.py
5 to 6S3-compile-testrunning, passed4 tests run, 0 failed
7 to 8S4-write-manifestrunning, passedstatus accepted

Example A has no model anywhere, not even a stand-in. Where later examples put one at a station, the agent is a scripted stand-in, so those results show how the factory mechanics work, not how good any model is.

WatchAlgo in practice#

WatchAlgo, the author's algorithm-learning product and the case study of Part II, shows what happens to the colleague's three questions at the scale of thousands of units when there is no per-unit manifest.

Observed in the repository (zenalgo-frontend, HEAD 1b183564): each unit is a folder wa-NNNNNN with up to three JSON files, and nothing in the folder records which validator version accepted each file, or when. The September pipeline checkpointed by committing verified files to git every 30 minutes (scripts/viz/supervise.sh, commit 017ed6b9), so git history is the closest thing to a receipt: it says when a file arrived, not what checked it. The consequence is visible in the evidence: 70 of 281 sampled solutions written before September fail the solution validator as it stands today, because its rules were tightened after the files were written, and nothing in those files says which rules they passed (docs/EVIDENCE.md, matrix row 7 and section 10.3).

Observed in the repository (docs/EVIDENCE.md, matrix row 15): publication has no recorded source commit either. The live site's content was matched to the working tree of 2026-09-03 only by probing it; that match is an inference, not a record.

Teaching reconstruction: a WatchAlgo-sized factory would write one manifest per unit listing its facets, the schema version, the hash of each file and the version of the validator that accepted it, which is Example A's manifest with more fields. A problem as a production unit develops that contract. Reported by the author: 3,246 problems with descriptions, solutions and visualizations; the WatchAlgo story reconciles that number with the repository.

Enterprise translation#

Example C, the enterprise factory of Part III, has the same four-station shape for a much larger product. Factory run FR-0003 takes pattern PAT-INTAKE-API 2.1.0 and the product spec SPEC-INTAKE-001 through S1-validate-spec, S2-generate-contracts, S3-assemble-service and S4-write-manifest, and writes a data contract, event schemas, an OpenAPI-shaped API contract, a rollout manifest and a service configuration. Its README records the same properties this page proved: the assembly is byte-identical across runs, and changing one value in the spec (canary_percentage from 5 to 10) changes the specification hash and the rollout manifest's digest while the API contract's digest stays the same. The first lines of its output, from a scratch copy:

factory run FR-0003: PAT-INTAKE-API 2.1.0 + SPEC-INTAKE-001 (sha256:ecb151ab7f7f...)
  S1-validate-spec       passed
  S2-generate-contracts  passed
  S3-assemble-service    passed
  S4-write-manifest      passed
  artifacts: data_contract.json, event_schemas.json, openapi.json, rollout_manifest.json, service_config.json

What changes in an enterprise is who reads the manifest. A release approver reads it to see which gates passed. An auditor reads it a year later to find which spec version a service in production was built from. A security team reads it to find every service built from a pattern version with a known flaw. Each of those readers needs the manifest to be stable and hashed for the same reason the colleague did. In an enterprise the manifest is usually the cheap part; the expensive parts of the critical path are the gates whose results it records (integration tests against real systems, security approval) and the people who must sign. The CI/CD and deployment factory and Enterprise governance take those up.

Failure test#

The failure: someone fixes a comment in a body template and does not release a new pattern version. The pattern file still says 1.3.0. Every manifest built afterwards claims the same pattern as before, but the templates are not the same.

How the factory catches it: the pattern digest covers every template, not the version string. For this page a comment was added to templates/bodies/prefix_sum.py.tmpl in a scratch copy (the line result = [] became result = [] # running totals) and the line was run again:

$ /opt/homebrew/bin/python3 run.py
spec_id        problem         S1    S2       S3       status          spec_hash
SPEC-ALGO-101  running-sum     pass  pass     pass     accepted        1deb0c4b830f
SPEC-ALGO-102  clamp-list      pass  pass     pass     accepted        7af02f7aef73
SPEC-ALGO-103  median-of-list  fail  skipped  skipped  rejected_at_S1  32c1e1899da6
$ grep '"digest"' out/FR-0001/running-sum/manifest.json
    "digest": "86f7b6238a06e84732c229d2da3c9c47fbd6b1a6aff05b414fcdb7fa4ca758a0",

(The first command's output is cut to its summary table.)

The spec hashes did not change and every test still passed, so a reader who looked only at statuses would see nothing. The pattern digest changed from b33a45fb… to 86f7b623…, while the version stayed 1.3.0, and so all three manifests changed hash: running-sum (bbfd7c60…), clamp-list (fb30ad4d…) and even the rejected median (2565b486…), which never used that template. A comparison of manifests between two runs shows the edit at once. Notice what the factory did not do: it did not refuse to build. Example A records the digest; a stricter line would also refuse any digest that is not registered for the version it claims.

What to run or toggle: in your copy, edit any template, run /opt/homebrew/bin/python3 run.py, and compare out/FR-0001/run.json with the hashes above. Then undo the edit and confirm the hashes return.

Where to go deeper#

Checklist for Monday#

Exercises#

  1. You change the wording of one requirement in running-sum.json and rerun. Which hashes change, and does station 3 pass?
Answer 1Spec hash, module hash, manifest hash

The spec hash changes, so the module changes too (its docstring lists the requirements and the spec hash), and so does the manifest. Station 3 still passes, because the examples and the body did not change.

  1. Two teammates run Example A on different laptops and get different manifest hashes for running-sum. Name two likely causes.
Answer 2Different inputs or a different interpreter

A local edit to a spec or template (compare spec_hash and pattern.digest first), or a different Python version: the README claims byte-identical output for the same interpreter only.

  1. Why does the median spec get a manifest when nothing was built?
Answer 3A refusal is an outcome

So that "not built" is a record with a reason and a station, not a missing folder that nobody notices. Run reports can then count refusals alongside acceptances.

Advanced perspective#

What byte-identical proves, and what it does notSame inputs, same process, same bytes; nothing about correctness

A byte-identical rerun proves that the factory's output depends only on the inputs it records: the spec, the pattern and its templates, and the interpreter. That is valuable in three ways. A reviewer can trust a run they did not watch by repeating it. A difference between two runs always means a recorded input changed, so investigations start from the manifest instead of from guesswork. And caching becomes safe: if the inputs hash the same, the recorded artifact can be reused, which Example D does with its input_hash (Long-running agent execution).

It proves nothing about correctness. A wrong expected value in a spec produces a wrong test, and a wrong function can pass it identically on every run. It also does not cover the environment beyond what is recorded: Example A's README limits the claim to the same Python version, and a real factory with dependencies would need their versions, ideally their hashes, in the manifest. Reproducible builds as a discipline go further (fixed toolchains, recorded build environments), and that is where the governance chapter picks up.

Version numbers and content digests do different jobsIntent for people, identity for machines

A version number such as 1.3.0 states intent: a person decided that this set of templates is a release, and semantic versioning tells consumers what kind of change to expect. A digest states identity: these exact bytes, nothing else. The failure test shows why a factory needs both. The version alone let an edited template pass as 1.3.0. The digest alone would tell you something changed, but not whether anyone meant it to.

A practical rule follows: a registry maps each released version to exactly one digest, and the factory refuses to build when the digest it computes does not match the one registered for the version the spec asks for. Example A stops one step short (it records the digest but does not compare it with a registry), which is honest for a teaching line and not enough for a production one. The factory learning loop handles releasing a changed pattern and measuring its blast radius; the failure test above already shows the blast radius of even a comment, three manifests out of three.

From a manifest to provenance someone else can verifyWho wrote the receipt, and can it be forged?

Example A's manifest is written by the same process that built the artifact, stored next to it, and unsigned. Anyone with write access to the folder could change the artifact and the manifest together. Inside one team that is acceptable; across organizational boundaries it is not, because the consumer has to trust the producer's word, which is the very thing this site keeps refusing to do.

The upgrade path has three steps. Store manifests where the builder cannot later rewrite them (an append-only store, or a commit whose hash is recorded elsewhere). Have the build platform, not the build script, produce the receipt, so a compromised script cannot lie about its inputs. And sign the receipt so a consumer can verify who produced it. Those steps are what supply-chain provenance standards formalize; Enterprise governance describes them from their own documentation. The fields stay recognizable: inputs by hash, outputs by hash, the process that connected them.

Why the tiny factory is one script and not a workflow engineThe least machinery that produces the artifact reliably

Example A runs its stations as four function calls in one process, in a fixed order, with no queue, no scheduler and no database. For three small specs that is the right amount of machinery: it is reproducible, it records refusals, and a reader can understand all of it. A workflow engine would add durable state, retries and parallelism that this line does not need.

The case for more machinery appears when one of four things becomes true: stations take long enough that a crash must not lose finished work, some stations can run in parallel, failures are transient and worth retrying, or many factory runs share workers. Example D adds exactly those controls, with a SQLite record of every attempt, and Assembly stations explains the contracts that make them safe. Until one of the four is true, a script with a manifest is the stronger design, because there is less of it to be wrong.

In one breath#

The smallest complete factory runs fixed stations over every spec it receives (validate, render from templates, compile and test in a separate process, write a manifest) and records refusals as carefully as acceptances. The manifest answers which spec, which pattern and which tests by content hash, and a second run in a different folder produces byte-identical files, which you can verify with shasum. A changed spec becomes a new input that station 3 refuses, and an edited template shows up in the pattern digest even when the version number lies.

The four stations so far are function calls in a fixed order, each with an implicit contract. The next chapter, Assembly stations, writes those contracts down: what each station takes and gives, who owns it, what it depends on, and what the line does when a station fails, including the difference between trying again and sending work back.