All chapters
  1. Start hereA software factory, from first principles
  2. Primitive · 1A factory, before AI: input, transformation, check, output
  3. Primitive · 2From repetition to a pattern: what stays, what varies, what to leave alone
  4. Primitive · 3The specification as an executable contract
  5. Primitive · 4Build the smallest factory: four stations, two specs, one manifest
  6. Primitive · 5Assembly stations: typed contracts, owners and failure handling
  7. Primitive · 6AI enters the factory: what a model adds, and what stays deterministic
  8. WatchAlgo · 7The WatchAlgo production story: what the repository shows
  9. WatchAlgo · 8A problem as a production unit: the files one unit must contain
  10. WatchAlgo · 9Station by station: description, solution, implementations, visualization
  11. WatchAlgo · 10The visualizer factory: from algorithm state to renderable steps
  12. WatchAlgo · 11Three flavors and three languages: two different contracts
  13. WatchAlgo · 12Looping over a large backlog: scheduling, limits and a failed-case queue
  14. WatchAlgo · 13Long-running agent execution: context, checkpoints and token limits
  15. WatchAlgo · 14Evidence of correctness: tests the producer cannot talk its way past
  16. WatchAlgo · 15The factory learning loop: defects change the pattern, not only the unit
  17. Enterprise · 16Classify the enterprise demand before choosing a pattern
  18. Enterprise · 17Reference enterprise patterns and their variation points
  19. Enterprise · 18Platform foundations: golden paths, templates, registries and catalogs
  20. Enterprise · 19Pattern versus configuration versus business logic
  21. Enterprise · 20Insurance factory, part 1: building the intake contract
  22. Enterprise · 21Insurance factory, part 2: building the underwriting workflow
  23. Enterprise · 22Insurance factory, part 3: trust and observability built in
  24. Enterprise · 23Data and integration engineering: contracts that survive change
  25. Enterprise · 24The CI/CD and deployment factory: from pull request to production
  26. Enterprise · 25Enterprise governance: owners, approved patterns and provenance
  27. Enterprise · 26Scaling safely: many teams, many agents, one set of boundaries
  28. Operations · 27Factory quality engineering: measuring accepted output, not activity
  29. Operations · 28Failure and recovery scenarios: what breaks a factory, and what restores it
  30. Operations · 29The landscape: which tool for which job, read from its own documentation
  31. Operations · 30Design an enterprise software factory: the blueprint and the rollout
  32. ReferenceSources and verification
  33. ReferenceGlossary
  34. AboutAbout this site
WatchAlgo · 8

A problem as a production unit: the files one unit must contain

The real contract of one WatchAlgo unit: problem, solution and visualization files, the fields inside them, how units vary, and what is the source of truth.

You will open
One WatchAlgo unit folder and its three files: problem, solution, visualization, with their real fields.
You will compare
How units vary: sizes, test cases, states, declared variants, and the ones that drifted from the shape.
You will decide
What the source of truth is when the type files, the validator, the renderer and the data disagree.

Unit wa-002025 is a 168 KB solution file, about twice the median. Open it and the reason shows at once: its explanation, both code approaches, its six test cases and its run instructions appear twice, once nested under solution where every reader expects them, and once again at the top level of the file. The two copies are not identical. The top-level brute-force and optimal sections differ from the nested ones.

The page for that problem renders without complaint. The solution validator passes the file. Nothing ever noticed, because every consumer reads only the nested copy and none of them was told what else may not be in the file.

That is the problem of this page. A factory that produces thousands of units must say exactly what one unit contains, so that every station produces it the same way and every check can reject what does not belong. WatchAlgo's units have such a contract, partly written in code and partly only implied. Reading it field by field shows both halves.

The learner's question: what exactly must one production unit contain, who decides that, and how would a factory notice a unit that only looks right?

One folder, three files#

Plain English. A production unit is one finished thing the factory makes, kept together in one place, with a fixed list of parts.

On this site a production unit is the smallest deliverable a factory is accountable for: one item it can register, build, check and publish on its own. In WatchAlgo the unit is one algorithm problem. In the insurance factory of Part III it is one built service. The unit contract is the written statement of what a unit must contain: which files, which fields inside them, which variants are allowed, and which check enforces each rule.

WatchAlgo keeps each unit in a unit folder: a directory named wa- plus six digits under public/content/, holding up to three JSON files, called facets on this site.

FacetFileWhat it holdsUnits holding it
Problemproblem.jsonthe problem told in three narrative flavors3,246
Solutionsolution.jsonexplanation, two approaches in three languages, test cases, run instructions, per flavor2,773
Visualizationvisualization.jsonstructured test cases with step-by-step states, annotated code, algorithm metadata, per-flavor explanation2,380

Observed in the repository (public/content/, commit 1b183564): 2,380 folders hold all three facets, 393 hold problem and solution, 473 hold only the problem. A page whose facet is missing shows a "Coming Soon" placeholder (src/app/problems/[code]/[slug]/ProblemPageClient.tsx line 692).

Before AI. A Java library release is the same idea: one set of coordinates, a fixed set of files (the JAR, its POM, a sources JAR, checksums), and a publishing rule that refuses a release missing a required file. The contract lives in that rule, not in the head of whoever ran the build.

The smallest primitive: print the shape of one unit#

Before reading fields, look at the outline. The script below walks the three files of one unit and prints one line per key path. It collapses the two independent axes, so <flavor> stands for technical, fun and spiritual (the technical branch is shown) and <language> for javascript, python and java. It was written for this page and run on 2026-11-20 on a copy of wa-001689 in a scratch folder. It proves what keys this one unit has; it does not prove that other units have the same keys.

import json
import os
import sys

FLAVORS = {"technical", "fun", "spiritual"}
LANGS = {"javascript", "python", "java"}


def shape(node, path, depth, out):
    """Print one line per key path; collapse the flavor and language axes."""
    if depth == 0:
        out.append(f"{path}  {{{len(node)} keys}}" if isinstance(node, dict) else path)
        return
    if isinstance(node, dict):
        keys = list(node)
        if set(keys) == FLAVORS:
            shape(node["technical"], path + ".<flavor>", depth, out)
            return
        if set(keys) == LANGS:
            out.append(f"{path}.<language>  ({type(node['javascript']).__name__})")
            return
        for k in keys:
            shape(node[k], f"{path}.{k}", depth - 1, out)
    elif isinstance(node, list):
        out.append(f"{path}[{len(node)}]")
    else:
        out.append(path)


root, unit = sys.argv[1], sys.argv[2]
for facet, depth in (("problem", 3), ("solution", 4), ("visualization", 2)):
    with open(os.path.join(root, unit, facet + ".json"), encoding="utf-8") as f:
        data = json.load(f)
    lines = []
    shape(data, facet, depth, lines)
    print(f"{facet}.json: {len(lines)} paths")
    for line in lines:
        print("  " + line)
problem.json: 15 paths
  problem.problem.<flavor>.title
  problem.problem.<flavor>.title_emoji
  problem.problem.<flavor>.theme_color
  problem.problem.<flavor>.problem_statement
  problem.problem.<flavor>.enhanced_definition.what_this_means
  problem.problem.<flavor>.enhanced_definition.key_idea
  problem.problem.<flavor>.enhanced_definition.visual
  problem.problem.<flavor>.constraints[3]
  problem.problem.<flavor>.examples[3]
  problem.problem.<flavor>.key_requirements[4]
  problem.problem.<flavor>.walkthrough.title
  problem.problem.<flavor>.walkthrough.emoji
  problem.problem.<flavor>.walkthrough.steps
  problem.problem.<flavor>.follow_up_questions[3]
  problem.problem.<flavor>.additional_examples[2]
solution.json: 44 paths
  solution.solution.<flavor>.thinking.restatement
  solution.solution.<flavor>.thinking.firstPrincipalThinking
  solution.solution.<flavor>.thinking.inputsConstraints[4]
  solution.solution.<flavor>.thinking.naiveBaseline
  solution.solution.<flavor>.thinking.signalsPatterns[3]
  solution.solution.<flavor>.thinking.dsChoice
  solution.solution.<flavor>.thinking.planSteps[6]
  solution.solution.<flavor>.thinking.edgeCases[6]
  solution.solution.<flavor>.thinking.intuition[2]
  solution.solution.<flavor>.thinking.complexity.time
  solution.solution.<flavor>.thinking.complexity.space
  solution.solution.<flavor>.thinking.pitfalls[3]
  solution.solution.<flavor>.bruteForce.applicable
  solution.solution.<flavor>.bruteForce.title
  solution.solution.<flavor>.bruteForce.description
  solution.solution.<flavor>.bruteForce.complexity.time
  solution.solution.<flavor>.bruteForce.complexity.space
  solution.solution.<flavor>.bruteForce.complexity.why
  solution.solution.<flavor>.bruteForce.code.<language>  (str)
  solution.solution.<flavor>.bruteForce.stepCount
  solution.solution.<flavor>.optimalSolution.title
  solution.solution.<flavor>.optimalSolution.description
  solution.solution.<flavor>.optimalSolution.complexity.time
  solution.solution.<flavor>.optimalSolution.complexity.space
  solution.solution.<flavor>.optimalSolution.code.<language>  (str)
  solution.solution.<flavor>.optimalSolution.stepCount
  solution.solution.<flavor>.testCases[6]
  solution.solution.<flavor>.runInIDE.fileName.<language>  (str)
  solution.solution.<flavor>.runInIDE.runCommand.<language>  (str)
  solution.solution.<flavor>.runInIDE.setupNotes
  solution.solution.<flavor>.runInIDE.runnableCode.<language>  (str)
  solution.visualization_spec.algorithm_type
  solution.visualization_spec.visual_template
  solution.visualization_spec.data_structure
  solution.visualization_spec.canvas.primary_element
  solution.visualization_spec.canvas.element_count_range[2]
  solution.visualization_spec.canvas.needs_secondary_view
  solution.visualization_spec.canvas.secondary_element
  solution.visualization_spec.state_variables[3]
  solution.visualization_spec.animation_phases[4]
  solution.visualization_spec.key_moments[3]
  solution.visualization_spec.complexity_visual.time_metaphor
  solution.visualization_spec.complexity_visual.space_metaphor
  solution.visualization_spec.test_case_for_demo
visualization.json: 8 paths
  visualization.testCases[3]
  visualization.annotatedCode.<language>  (dict)
  visualization.algorithmMeta.type
  visualization.algorithmMeta.primaryStructure
  visualization.algorithmMeta.name
  visualization.algorithmMeta.secondaryStructure
  visualization.thinkingContent.<flavor>.title
  visualization.thinkingContent.<flavor>.sections  {3 keys}

Three things stand out before any field is read. The flavor axis wraps whole branches of the problem and the solution, so a solution holds every section three times. The language axis sits deep inside, at the code. And one line is missing that the solution prompt asks for: optimalSolution.complexity.why. This unit has why under brute force but not under the optimal approach. Keep that in mind; it returns in the validator run below.

Diagram: One unit folder as a bill of materials: facets, key fields, readers

Left to right: the unit folder, its three facets with the fields that matter, and the stations and programs that read them. The problem facet feeds the solution station. The solution's optimal JavaScript code and its step numbers feed the visualization station, which turns step numbers into shared line ids. The visualization's structured test cases feed the executable cross-check, and the renderer reads all three facets. A field with no reader, like the duplicate top-level sections of wa-002025, has no arrow, which is exactly why nobody noticed it.

  1. Unit folder
    1. public/content/ wa-NNNNNN/: The unit's home: a directory named wa- plus six digits holding up to three JSON files.
  2. Facets and key fields
    1. problem.json: The problem in three narrative flavors, each with its own fields (the spiritual one has a koan).
    2. solution.json: Per flavor: 11 thinking sections, brute force and optimal code in three languages, 6 test cases as display strings, run instructions; top-level visualization_spec.
    3. visualization.json: Structured test cases with recorded states; code lines with shared ids per language; algorithm metadata; per-flavor explanation.
  3. Readers
    1. solution worker: The solution station writes solution.json from the problem, the instructions and a reference solution.
    2. tracer + vizlib: vizlib.load_solution_code reads solution.technical.optimalSolution.code; step comments become line ids L1, L2.
    3. executable cross-check: Runs the real solution in Node on visualization test inputs and compares with the tracer's output.
    4. renderer: The problem page reads solution.solution[flavor] and the visualization; a missing facet shows Coming Soon.

Read the diagram as a bill of materials. Each facet has its own producer and its own consumers, and the arrows show which station reads which field. The solution's optimal JavaScript code feeds the visualization station; the visualization's structured test cases feed the executable cross-check; the renderer reads all three. A field that nobody reads, such as the duplicate sections of wa-002025, has no arrow, which is why nobody noticed it.

problem.json: one problem, three voices#

The problem facet holds the same problem told three ways. Observed in the repository (wa-000001/problem.json, commit cb6e7ac3): technical "Remove Duplicates from Sorted Array", fun "Remove Extra Fruits", spiritual "One of Each Stone". Each flavor uses different names for the same things, and the flavors carry different fields. The technical flavor has constraints, enhanced_definition and follow_up_questions; the fun flavor has intro, full_problem and approach_sections; the spiritual flavor has intro, approach_sections and a koan.

So the problem contract is not one schema with three copies. It is three related schemas that must describe one problem. Observed (scripts/validate_problems.py, commit cb6e7ac3): a "10-point content QA" checks that every flavor is present, that the technical definition has its keys, that the walkthrough has at least four steps, that there are three follow-up questions and two additional examples per flavor, that the spiritual flavor has a koan, and that no Big-O notation appears. On a scratch copy of every tenth unit, 328 of 328 problems pass. What it does not check is whether an example's stated output is right; 15 wrong examples were caught later, by the solution station.

solution.json: three flavors times three languages times two approaches#

The solution facet is the largest contract. Observed (prompt in scripts/viz/pipeline.py prompt_for, commit 971863f2; validator validate_payload in scripts/generate_batch37_next10.py, commit cb6e7ac3): per flavor, thinking with eleven named sections, a bruteForce and an optimalSolution each with complexity (including a why), code in the three languages and a stepCount, exactly six testCases split two basic, two edge and two complex, and runInIDE with a runnable file per language; at the top level, a visualization_spec describing how the problem should be drawn; no Big-O notation anywhere; at least 30,000 bytes.

Here is one real test case from the technical flavor of wa-001689 (read from a scratch copy, unmodified):

{"name": "Basic: two separate groups", "input": "n = 5, edges = [[0,1],[1,2],[3,4]]", "expectedOutput": "2",
 "executionTrace": [{"step": 1, "action": "Build adjacency list. Start DFS from node 0.", "state": "Visiting 0 -> 1 -> 2. componentCount=1"},
                    {"step": 2, "action": "Nodes 1 and 2 already visited. Node 3 unvisited.", "state": "Start new DFS. componentCount=2"},
                    {"step": 3, "action": "Visit 3 -> 4. Node 4 visited. All nodes done.", "state": "Result: 2"}],
 "category": "basic"}

Two facts about it matter for every later station. The input and expected output are display strings, readable by a person and by a model, but no code can run them without a parser. And this test case carries an executionTrace field that the prompt does not ask for, narrating a depth-first search, while the unit's optimal code and its visualization use union-find. Both reach the answer 2. Nothing checks that the narration matches the code.

Each flavor's code also carries numbered step comments (1️⃣, 2️⃣, …) in all three languages, and stepCount declares how many there are. The validator counts them. That field is not decoration: the visualization station turns those step numbers into shared line ids.

visualization.json: states, line ids, metadata#

The visualization facet holds a different kind of data: a recorded run of the algorithm. Observed (scripts/viz/vizlib.py validate, commit d08ba4f0): four top-level keys. testCases holds three to five cases, each with structured input and output and a list of states. annotatedCode holds the code per language as lines with ids. algorithmMeta names the algorithm type and the structures drawn. thinkingContent holds a short explanation per flavor.

The structured test case of the same unit, wa-001689, is executable as it stands:

{"input": {"n": 5, "edges": [[0, 1], [1, 2], [3, 4]]}, "output": 2}

One state of that case, trimmed for length (the variables, dataStructureState and annotation fields are shortened or omitted; every other field is as stored):

{
  "step": 2,
  "codeLineId": "L2",
  "phase": "find",
  "description": "find(0) = 0, find(1) = 1",
  "variables": {"rootX": {"value": 0, "changed": true, "displayValue": "0"},
                "rootY": {"value": 1, "changed": true, "displayValue": "1"}},
  "pointers": {"u": {"position": 0, "color": "#f97316", "label": "u"},
               "v": {"position": 1, "color": "#22d3ee", "label": "v"}},
  "calculation": [{"label": "Compare", "expression": "0 === 1", "value": "false", "highlight": "amber"}]
}

codeLineId is the field that ties the facets together. Line L1 of this unit is const parent = Array.from({length: n}, (_, i) => i); in JavaScript, parent = list(range(n)) in Python and parent = new int[n]; in Java. One state highlights the same step in all three languages because the line ids are shared, and vizlib.validate refuses a state whose codeLineId is missing from any language. The visualizer factory is the home of the state contract; here it matters only as a field the unit must carry.

Two test-case representations in one unit#

The unit holds its test cases twice, in two different forms, and the difference explains where WatchAlgo's strongest check lives.

WhereExample from wa-000001Who reads it
solution.technical.testCases[0]input "nums = [2, 2, 5]", expected "k = 2, front = [2, 5]" (display strings)the Solution tab; the tracer worker parses them
visualization.testCases[0]input {"nums": [2, 2, 5]}, output 2 (structured values)the animation; the executable cross-check

A check can only run what it can parse. The solution's strings cannot be executed directly, so the solution validator checks wording and counts. The visualization's structured values can be executed, so the September visualization station runs the real JavaScript solution on them in Node and compares. That is how 86 solution defects were found downstream of the station that wrote them (the WatchAlgo story has the defect log).

How units vary#

A contract that fits one unit is easy. The hard part is variation that is legitimate and variation that is drift, side by side in the same folder.

Observed (inventory adapter, examples/watchalgo-adapter/README.md): problem files range from 6.7 to 15.5 KB, solutions from 27 to 179 KB, visualizations from 22 to 786 KB. Visualizations have three to five test cases (median three) and 1 to 101 states per test case (median 11, 7,142 test cases in all). The algorithm types spread wide, from hash-map (199 units) and linear-scan (136) to union-find (47); 1,172 visualizations draw a secondary structure and the rest do not.

Some variation is declared. Observed (copy of wa-001378/solution.json): a tree traversal has no meaningful brute force, so its bruteForce section carries "applicable": false and an omitReason instead of code. That is a reasonable variant. The solution validator does not know it exists: run on a copy, it stops at this unit with a missing-key error, because it expects complexity in every brute-force section.

Some variation is only in names. Observed (fixture examples/watchalgo-adapter/fixtures/watchalgo-excerpts.json): the problem example of wa-001830 (Flood Fill) calls its inputs image, sr, sc and color; the visualization calls the same inputs canvas, startRow, startCol and paintColor, the names the solution's JavaScript uses. Different names per flavor are required by the prompt; different names between facets of one flavor are not mentioned anywhere.

What is the source of truth?#

A source of truth is the one place whose content wins when two copies disagree. WatchAlgo has several candidates, and they do not all agree.

The type files src/components/solution/types.ts and src/components/visualization/types.ts describe the shapes the renderer expects. The validators describe the shapes the factory accepts. The prompts and briefs describe what workers are asked to write. The files on disk are what is actually there.

Observed in the repository: the factory treats disk as the truth. The September driver can rebuild its whole queue by scanning public/content/ for missing solution.json and visualization.json files (pipeline.py build), the renderer fetches the static files, and the visualization station reads the solution's code from the file, not from a database or a cache. That is a sound choice for a single-machine content factory: there is one copy, and every process looks at it.

Diagram: Four candidates for the source of truth, and what each one decides

Four lanes. Intentions: the prompts, briefs and type files describe what a unit should look like. Gatekeepers: the validators decide what may be written, and they changed over time. Data: the files on disk are what is actually there, and the factory treats them as the truth: the queue is rebuilt from them and the renderer serves them. Missing: a single versioned contract with a record of which validator accepted each unit. Without it, valid means valid by whichever rules ran when the file was written.

  1. Intentions
    1. prompts + briefs: prompt_for in pipeline.py, .claude/SOLUTION_ASSEMBLY_LINE.md, scripts/viz/WORKER_BRIEF.md.
    2. type files: src/components/solution/types.ts and src/components/visualization/types.ts.
  2. Gatekeepers
    1. validators: validate_problems.py, validate_payload, vizlib.validate. Rules tightened over time, without versions.
  3. Data
    1. files on disk: public/content as it is. The queue is rebuilt by scanning it; the renderer serves it.
  4. Missing
    1. versioned contract + accepted-by record: One contract file with a version, and per unit the validator version that accepted it.

The diagram separates the four sources by role. Disk holds the data; the validators decide what may enter it; the types and prompts describe intentions. When they disagree, the evidence notes' rule applies: code and data win over notes. The gap the diagram shows is that no single artifact holds the contract itself, with a version, so "valid" means "valid by whichever validator ran when the file was written". On a scratch copy of every tenth unit, 70 of 281 solutions fail today's validate_payload, all numbered wa-001761 or lower: the files did not change, the rules did.

Observed schema drift and defects#

Schema drift is the slow divergence of real files from their intended shape: fields that move, appear, disappear or change type while consumers keep working. The inventory names the cases, and this page checked each one again on a scratch copy.

UnitWhat is differentWhat still worksSource
wa-002025sections duplicated at the top level, not identical to the nested ones; the technical flavor also holds a visualization_specrenderer, tracer and validator read only the nested copy; validate_payload passesinventory; copy inspected 2026-11-20; last changed 1169fae3
wa-001609optimalSolution.code and bruteForce.code are one string, not a map of three languagesthe problem page; the unit has no visualizationinventory; copy inspected 2026-11-20
wa-000001, wa-002157, wa-002158, wa-002159no top-level visualization_specrendering, since nothing on the page reads itinventory
wa-002157five test cases instead of six (one edge case short)renderingcopy inspected 2026-11-20

One detail on wa-002157 deserves its own sentence, because it shows how a checkpoint can mislead. Its solution file was first committed in a September checkpoint (5e6c6040, 2026-09-02 23:25), and it fails today's validator. Observed (local run.log): visualization tasks for wa-002153 to wa-002158 were already queued at 20:20 that evening, so the file was on disk before the supervisor's first checkpoint. Which process wrote it is not recorded. The supervisor stages every solution file present on disk. A checkpoint commit records when a file was committed, not which station accepted it.

Common confusion. "Valid JSON" and "valid unit" are different claims. All 8,399 facet files parse. wa-002025 parses, renders and passes the solution validator, and it still carries two different versions of its own answer.
In a coding agent. When a coding agent writes a unit, the unit contract is what lets the factory reject its work without reading it. The agent receives the contract with the work item; the factory's own shape check and oracle run after it exits; a file that carries an extra top-level section is refused even if every field the page reads is correct. Harness Engineering owns the controls around the agent's run; the contract belongs to the factory.
What this is not. This page is not a guide to schema languages, typed validation or repair loops: that is Schema Engineering. It describes one real unit contract, partly enforced and partly implied, and what a factory needs from such a contract.

Run the unit's own checks, on a copy#

The demonstration below runs WatchAlgo's own solution validator, validate_payload, on copies of seven units: the four fixture units and the three drift cases above. It shows what the factory's acceptance at the solution station actually enforces. A successful run proves only the result for these seven files.

The copies were made with cp -R from ~/Projects/zenalgo-frontend/public/content/<unit> into a scratch folder, next to a copy of scripts/generate_batch37_next10.py. The runner is the one docs/EVIDENCE.md section 10.3 gives:

# run_validate_payload.py: place next to a COPY of scripts/generate_batch37_next10.py
import glob, os, sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from generate_batch37_next10 import validate_payload  # the copy, not the original
ok, bad = 0, []
for p in sorted(glob.glob(os.path.join(sys.argv[1], "wa-*", "solution.json"))):
    try:
        validate_payload(p); ok += 1
    except Exception as e:  # AssertionError carries (path, flavor, section, ...)
        bad.append((p.split("/")[-2], repr(e)[:120]))
print(f"validate_payload: {ok} pass, {len(bad)} fail of {ok + len(bad)} solution.json files")
for code, why in bad:
    print(f"  {code}: {why}")
validate_payload: 2 pass, 5 fail of 7 solution.json files
  wa-000001: AssertionError('content/wa-000001/solution.json')
  wa-001378: KeyError('complexity')
  wa-001609: AttributeError("'str' object has no attribute 'values'")
  wa-001689: AssertionError(('content/wa-001689/solution.json', 'technical', 'optimal'))
  wa-002157: AssertionError(('content/wa-002157/solution.json', 'technical', 'tests'))

Read the result against what you now know. wa-000001 fails the file-level assertions (it has no visualization_spec). wa-001378 fails on its declared no-brute-force variant, which the validator does not model. wa-001609 crashes on code stored as a string. wa-001689 lacks optimalSolution.complexity.why, the gap the shape primitive showed. wa-002157 has five test cases. And the two passes are wa-001830 and wa-002025: the unit with two versions of its answer passes, because the validator reads only data["solution"]. All five failing units serve their solution on the public site (read-only HEAD requests on 2026-11-20 returned 200 for each solution.json); they reached it under earlier rules or by a path this validator never saw.

Try it#

The explorer opens on the file contract view. Pick one of the four real units and see its three files side by side: which fields each holds, how the flavors and languages nest, and which commit last changed each file.

The same facts for a reader without JavaScript (from the fixture, and from the full files of the scratch copies):

Unitproblem.jsonsolution.jsonvisualization.jsonVariant or drift
wa-0000013 flavors, last changed cb6e7ac36 test cases, 3 languages, no visualization_spec3 test cases, 8 states in case 0, two-pointer on an arrayno top-level visualization_spec
wa-0013783 flavors, cb6e7ac3brute force not applicable, with omitReason13 states in case 0, recursive-dfs on a treedeclared variant the validator rejects
wa-0016893 flavors, cb6e7ac3test case narrates a depth-first search; code is union-find11 states in case 0, union-find on a graphno optimalSolution.complexity.why
wa-0018303 flavors; inputs image, sr, sc, colorlast changed f88d3c22; inputs canvas, startRow, startCol, paintColor21 states in case 0, bfs on a matrixnames differ between facets

WatchAlgo in practice#

Observed in the repository: the unit contract exists in four places at once: the prompt (prompt_for), the briefs (.claude/SOLUTION_ASSEMBLY_LINE.md, scripts/viz/WORKER_BRIEF.md), the type files the renderer compiles against, and the validators. The strictest of them, vizlib.validate, guards the visualization facet: it checks every state's keys, the shared line ids, allowed statuses and pointer positions, and nothing is written unless it passes. The loosest guard is on the problem facet's examples, which no code executes.

Reported by the author: each problem has descriptions, solutions and visualizations. The files support that sentence for 2,380 units, and support "problem and solution" for 393 more.

Teaching reconstruction: a versioned unit manifest would close most of the gaps on this page. One small file per unit would list its facets, the schema version of each, and the validator version that accepted it. Drift would then be a query ("which units were accepted by a validator older than 1.4?") instead of an archaeology project.

Enterprise translation#

The enterprise factory of Part III produces a different unit: one built service. In Example C, factory run FR-0003 takes the pattern PAT-INTAKE-API 2.1.0 and the product specification SPEC-INTAKE-001 and produces five artifacts: data_contract.json, event_schemas.json, openapi.json, rollout_manifest.json and service_config.json. That list is the unit contract of an intake service in this site's teaching example. A unit missing rollout_manifest.json is not a smaller unit; it is an incomplete one, and the factory must say so.

The WatchAlgo lessons carry over almost word for word. Write the contract once, with a version. Declare legitimate variants (a service with no events still says "no events", with a reason, the way wa-001378 says "no brute force"). Reject unknown top-level content instead of ignoring it, because an ignored section is a second version of the truth waiting to be read by the next consumer. And record which validator version accepted each unit, because enterprise rules tighten too: a security baseline added in March makes every service built in February a question.

Failure test#

The failure: a unit carries content the contract does not allow, and every check passes because each one looks only where it expects data. wa-002025 is the real case.

How a factory catches it: a strict shape check that lists what may exist, not only what must. The script below is a teaching reconstruction, not WatchAlgo code. It allows two top-level keys, requires code as a map of the three languages, accepts the declared no-brute-force variant, and checks the 2/2/2 test-case split. It was run on 2026-11-20 on the same seven copies.

"""drift_check.py: a strict shape check for solution.json (teaching reconstruction, not WatchAlgo code)."""
import json
import os
import sys

TOP = {"solution", "visualization_spec"}
FLAVORS = ("technical", "fun", "spiritual")
LANGS = {"javascript", "python", "java"}


def problems(sol):
    found = []
    extra = sorted(set(sol) - TOP)
    if extra:
        found.append("unexpected top-level keys: " + ", ".join(extra))
    if "visualization_spec" not in sol:
        found.append("no top-level visualization_spec")
    for flavor in FLAVORS:
        f = sol.get("solution", {}).get(flavor, {})
        for section in ("bruteForce", "optimalSolution"):
            part = f.get(section, {})
            if section == "bruteForce" and part.get("applicable") is False and part.get("omitReason"):
                continue  # a declared variant: no brute force, with a written reason
            code = part.get("code")
            if not isinstance(code, dict) or set(code) != LANGS:
                found.append(f"{flavor}.{section}.code is {type(code).__name__}, not a map of 3 languages")
        cats = sorted(t.get("category", "?") for t in f.get("testCases", []))
        if cats != ["basic", "basic", "complex", "complex", "edge", "edge"]:
            found.append(f"{flavor}.testCases split is {cats}")
    return found


root = sys.argv[1]
for unit in sorted(os.listdir(root)):
    path = os.path.join(root, unit, "solution.json")
    if not os.path.exists(path):
        continue
    with open(path, encoding="utf-8") as fh:
        found = problems(json.load(fh))
    print(f"{unit}: {'ok' if not found else str(len(found)) + ' finding(s)'}")
    for p in found[:3]:
        print("   - " + p)
wa-000001: 1 finding(s)
   - no top-level visualization_spec
wa-001378: ok
wa-001609: 6 finding(s)
   - technical.bruteForce.code is str, not a map of 3 languages
   - technical.optimalSolution.code is str, not a map of 3 languages
   - fun.bruteForce.code is str, not a map of 3 languages
wa-001689: ok
wa-001830: ok
wa-002025: 1 finding(s)
   - unexpected top-level keys: bruteForce, optimalSolution, runInIDE, testCases, thinking
wa-002157: 4 finding(s)
   - no top-level visualization_spec
   - technical.testCases split is ['basic', 'basic', 'complex', 'complex', 'edge']
   - fun.testCases split is ['basic', 'basic', 'complex', 'complex', 'edge']

The two checks disagree in both directions, and both disagreements teach. wa-002025 passes the observed validator and fails the strict one. wa-001378 fails the observed validator and passes the strict one, because the strict check models the declared variant. wa-001689 passes here only because this check does not look at complexity.why. What to run or toggle: save the script, copy a few unit folders to a scratch directory, and run /opt/homebrew/bin/python3 drift_check.py <copy>; then delete the omitReason line and watch wa-001378 fail.

Where to go deeper#

Checklist for Monday#

Exercises#

  1. wa-002025 passes validate_payload. Which single line of the validator explains why?
Answer 1It only reads the nested object

flavor_data = data["solution"][flavor]: every check runs on the nested solution object, so top-level content is never examined.

  1. Why can the visualization station run an executable cross-check when the solution station cannot?
Answer 2Structured versus display values

The visualization's test cases hold structured input and output values that can be passed to the JavaScript function; the solution's test cases hold display strings such as "nums = [2, 2, 5]".

  1. wa-001378 has no brute-force code. Is that drift or a variant, and how should the contract say so?
Answer 3A declared variant

A variant: it carries "applicable": false and an omitReason. The contract should list that shape as allowed, so validators accept it and still reject a section that is merely missing.

Advanced perspective#

Files per unit or a database?Why one folder per unit worked here, and when it stops working

WatchAlgo stores 8,399 JSON files, about 588 MB in total (problems 31.6 MB, solutions 226.9 MB, visualizations 329.5 MB), as static files served with long CDN cache headers (next.config.ts, commit 9f43618c). For a content site that changes in batches and is read far more than written, this is a strong design: no database at read time, git as the history, any file inspectable with a text editor, and a work list rebuildable with a directory scan.

It stops working when several writers need to change the same unit concurrently, when a unit's facets must change together atomically, or when queries cross units ("all visualizations accepted by validator 1.2 that draw a tree"). A database gives transactions and queries, at the price of a second source of truth unless the files become an export of it. The middle road many factories take: files stay the product, and a small per-unit manifest or index table holds the metadata that queries need, rebuilt from the files.

Validator drift is a versioning problem70 of 281 sampled solutions fail today's rules

The 70 failures in the every-tenth sample are not defects introduced after acceptance. They are files accepted under an older, looser validator and judged by a newer one. Thirty-two lack optimalSolution.complexity.why, sixteen have a step-count mismatch, eight lack a complexity block, three lack bruteForce.complexity.why, and eleven fail the no-Big-O, visualization_spec or size assertion. None of the sampled solutions numbered above wa-002134 fails.

Without a recorded validator version per unit, a team cannot tell "this unit is broken" from "this unit predates the rule". With one, the decision becomes explicit: re-run the new rule over old units as a planned batch, fix or grandfather each, and record the decision. The factory learning loop treats rule changes as pattern versions with a blast radius; the unit manifest is what makes the blast radius computable.

Prose fields are part of the contract tooNarration that disagrees with the code

wa-001689's solution test case narrates a depth-first search in executionTrace, while its optimal code and visualization use union-find. A learner who reads the narration next to the animation sees two different algorithms reach the same answer. No schema check can catch that, because both texts are well-formed.

Three designs reduce the risk, in rising cost. Remove free-text fields that duplicate structured ones, so there is less to disagree. Generate narration from the same artifact that generates the trace (WatchAlgo's tracer design already does this for states, which are written by code from an executed run). Or add an inferential check, a model or a person comparing narration with code, with its own recorded error rate. Evidence of correctness discusses when a semantic check is worth that cost, and Eval Engineering owns the graders that would perform it.

In one breath#

A WatchAlgo production unit is one folder with three JSON files: a problem in three narrative flavors, a solution with three flavors times three languages times two approaches, and a visualization of recorded states tied to shared line ids. Its contract lives in prompts, briefs, type files and validators at once, with no single versioned copy, so legitimate variants like wa-001378 get rejected while drift like wa-002025 passes. The fix is a contract that lists what may exist, declares its variants, and records which validator version accepted each unit.

The unit now has a contract and a known set of exceptions. The next question is who produces each facet and which check accepts it. Station by station maps those producers to the modules the repository actually holds, marks the ones it does not, and only then gives the stations names.