Unit wa-002025 is a 168 KB solution file, about twice the median. Open it and the reason shows at once: its explanation, both code approaches, its six test cases and its run instructions appear twice, once nested under solution where every reader expects them, and once again at the top level of the file. The two copies are not identical. The top-level brute-force and optimal sections differ from the nested ones.
The page for that problem renders without complaint. The solution validator passes the file. Nothing ever noticed, because every consumer reads only the nested copy and none of them was told what else may not be in the file.
That is the problem of this page. A factory that produces thousands of units must say exactly what one unit contains, so that every station produces it the same way and every check can reject what does not belong. WatchAlgo's units have such a contract, partly written in code and partly only implied. Reading it field by field shows both halves.
The learner's question: what exactly must one production unit contain, who decides that, and how would a factory notice a unit that only looks right?
One folder, three files#
Plain English. A production unit is one finished thing the factory makes, kept together in one place, with a fixed list of parts.
On this site a production unit is the smallest deliverable a factory is accountable for: one item it can register, build, check and publish on its own. In WatchAlgo the unit is one algorithm problem. In the insurance factory of Part III it is one built service. The unit contract is the written statement of what a unit must contain: which files, which fields inside them, which variants are allowed, and which check enforces each rule.
WatchAlgo keeps each unit in a unit folder: a directory named wa- plus six digits under public/content/, holding up to three JSON files, called facets on this site.
| Facet | File | What it holds | Units holding it |
|---|---|---|---|
| Problem | problem.json | the problem told in three narrative flavors | 3,246 |
| Solution | solution.json | explanation, two approaches in three languages, test cases, run instructions, per flavor | 2,773 |
| Visualization | visualization.json | structured test cases with step-by-step states, annotated code, algorithm metadata, per-flavor explanation | 2,380 |
Observed in the repository (public/content/, commit 1b183564): 2,380 folders hold all three facets, 393 hold problem and solution, 473 hold only the problem. A page whose facet is missing shows a "Coming Soon" placeholder (src/app/problems/[code]/[slug]/ProblemPageClient.tsx line 692).
Before AI. A Java library release is the same idea: one set of coordinates, a fixed set of files (the JAR, its POM, a sources JAR, checksums), and a publishing rule that refuses a release missing a required file. The contract lives in that rule, not in the head of whoever ran the build.
The smallest primitive: print the shape of one unit#
Before reading fields, look at the outline. The script below walks the three files of one unit and prints one line per key path. It collapses the two independent axes, so <flavor> stands for technical, fun and spiritual (the technical branch is shown) and <language> for javascript, python and java. It was written for this page and run on 2026-11-20 on a copy of wa-001689 in a scratch folder. It proves what keys this one unit has; it does not prove that other units have the same keys.
import json
import os
import sys
FLAVORS = {"technical", "fun", "spiritual"}
LANGS = {"javascript", "python", "java"}
def shape(node, path, depth, out):
"""Print one line per key path; collapse the flavor and language axes."""
if depth == 0:
out.append(f"{path} {{{len(node)} keys}}" if isinstance(node, dict) else path)
return
if isinstance(node, dict):
keys = list(node)
if set(keys) == FLAVORS:
shape(node["technical"], path + ".<flavor>", depth, out)
return
if set(keys) == LANGS:
out.append(f"{path}.<language> ({type(node['javascript']).__name__})")
return
for k in keys:
shape(node[k], f"{path}.{k}", depth - 1, out)
elif isinstance(node, list):
out.append(f"{path}[{len(node)}]")
else:
out.append(path)
root, unit = sys.argv[1], sys.argv[2]
for facet, depth in (("problem", 3), ("solution", 4), ("visualization", 2)):
with open(os.path.join(root, unit, facet + ".json"), encoding="utf-8") as f:
data = json.load(f)
lines = []
shape(data, facet, depth, lines)
print(f"{facet}.json: {len(lines)} paths")
for line in lines:
print(" " + line)
problem.json: 15 paths
problem.problem.<flavor>.title
problem.problem.<flavor>.title_emoji
problem.problem.<flavor>.theme_color
problem.problem.<flavor>.problem_statement
problem.problem.<flavor>.enhanced_definition.what_this_means
problem.problem.<flavor>.enhanced_definition.key_idea
problem.problem.<flavor>.enhanced_definition.visual
problem.problem.<flavor>.constraints[3]
problem.problem.<flavor>.examples[3]
problem.problem.<flavor>.key_requirements[4]
problem.problem.<flavor>.walkthrough.title
problem.problem.<flavor>.walkthrough.emoji
problem.problem.<flavor>.walkthrough.steps
problem.problem.<flavor>.follow_up_questions[3]
problem.problem.<flavor>.additional_examples[2]
solution.json: 44 paths
solution.solution.<flavor>.thinking.restatement
solution.solution.<flavor>.thinking.firstPrincipalThinking
solution.solution.<flavor>.thinking.inputsConstraints[4]
solution.solution.<flavor>.thinking.naiveBaseline
solution.solution.<flavor>.thinking.signalsPatterns[3]
solution.solution.<flavor>.thinking.dsChoice
solution.solution.<flavor>.thinking.planSteps[6]
solution.solution.<flavor>.thinking.edgeCases[6]
solution.solution.<flavor>.thinking.intuition[2]
solution.solution.<flavor>.thinking.complexity.time
solution.solution.<flavor>.thinking.complexity.space
solution.solution.<flavor>.thinking.pitfalls[3]
solution.solution.<flavor>.bruteForce.applicable
solution.solution.<flavor>.bruteForce.title
solution.solution.<flavor>.bruteForce.description
solution.solution.<flavor>.bruteForce.complexity.time
solution.solution.<flavor>.bruteForce.complexity.space
solution.solution.<flavor>.bruteForce.complexity.why
solution.solution.<flavor>.bruteForce.code.<language> (str)
solution.solution.<flavor>.bruteForce.stepCount
solution.solution.<flavor>.optimalSolution.title
solution.solution.<flavor>.optimalSolution.description
solution.solution.<flavor>.optimalSolution.complexity.time
solution.solution.<flavor>.optimalSolution.complexity.space
solution.solution.<flavor>.optimalSolution.code.<language> (str)
solution.solution.<flavor>.optimalSolution.stepCount
solution.solution.<flavor>.testCases[6]
solution.solution.<flavor>.runInIDE.fileName.<language> (str)
solution.solution.<flavor>.runInIDE.runCommand.<language> (str)
solution.solution.<flavor>.runInIDE.setupNotes
solution.solution.<flavor>.runInIDE.runnableCode.<language> (str)
solution.visualization_spec.algorithm_type
solution.visualization_spec.visual_template
solution.visualization_spec.data_structure
solution.visualization_spec.canvas.primary_element
solution.visualization_spec.canvas.element_count_range[2]
solution.visualization_spec.canvas.needs_secondary_view
solution.visualization_spec.canvas.secondary_element
solution.visualization_spec.state_variables[3]
solution.visualization_spec.animation_phases[4]
solution.visualization_spec.key_moments[3]
solution.visualization_spec.complexity_visual.time_metaphor
solution.visualization_spec.complexity_visual.space_metaphor
solution.visualization_spec.test_case_for_demo
visualization.json: 8 paths
visualization.testCases[3]
visualization.annotatedCode.<language> (dict)
visualization.algorithmMeta.type
visualization.algorithmMeta.primaryStructure
visualization.algorithmMeta.name
visualization.algorithmMeta.secondaryStructure
visualization.thinkingContent.<flavor>.title
visualization.thinkingContent.<flavor>.sections {3 keys}
Three things stand out before any field is read. The flavor axis wraps whole branches of the problem and the solution, so a solution holds every section three times. The language axis sits deep inside, at the code. And one line is missing that the solution prompt asks for: optimalSolution.complexity.why. This unit has why under brute force but not under the optimal approach. Keep that in mind; it returns in the validator run below.
Diagram: One unit folder as a bill of materials: facets, key fields, readers
Left to right: the unit folder, its three facets with the fields that matter, and the stations and programs that read them. The problem facet feeds the solution station. The solution's optimal JavaScript code and its step numbers feed the visualization station, which turns step numbers into shared line ids. The visualization's structured test cases feed the executable cross-check, and the renderer reads all three facets. A field with no reader, like the duplicate top-level sections of wa-002025, has no arrow, which is exactly why nobody noticed it.
- Unit folder
- public/content/ wa-NNNNNN/: The unit's home: a directory named wa- plus six digits holding up to three JSON files.
- Facets and key fields
- problem.json: The problem in three narrative flavors, each with its own fields (the spiritual one has a koan).
- solution.json: Per flavor: 11 thinking sections, brute force and optimal code in three languages, 6 test cases as display strings, run instructions; top-level visualization_spec.
- visualization.json: Structured test cases with recorded states; code lines with shared ids per language; algorithm metadata; per-flavor explanation.
- Readers
- solution worker: The solution station writes solution.json from the problem, the instructions and a reference solution.
- tracer + vizlib: vizlib.load_solution_code reads solution.technical.optimalSolution.code; step comments become line ids L1, L2.
- executable cross-check: Runs the real solution in Node on visualization test inputs and compares with the tracer's output.
- renderer: The problem page reads solution.solution[flavor] and the visualization; a missing facet shows Coming Soon.
Read the diagram as a bill of materials. Each facet has its own producer and its own consumers, and the arrows show which station reads which field. The solution's optimal JavaScript code feeds the visualization station; the visualization's structured test cases feed the executable cross-check; the renderer reads all three. A field that nobody reads, such as the duplicate sections of wa-002025, has no arrow, which is why nobody noticed it.
problem.json: one problem, three voices#
The problem facet holds the same problem told three ways. Observed in the repository (wa-000001/problem.json, commit cb6e7ac3): technical "Remove Duplicates from Sorted Array", fun "Remove Extra Fruits", spiritual "One of Each Stone". Each flavor uses different names for the same things, and the flavors carry different fields. The technical flavor has constraints, enhanced_definition and follow_up_questions; the fun flavor has intro, full_problem and approach_sections; the spiritual flavor has intro, approach_sections and a koan.
So the problem contract is not one schema with three copies. It is three related schemas that must describe one problem. Observed (scripts/validate_problems.py, commit cb6e7ac3): a "10-point content QA" checks that every flavor is present, that the technical definition has its keys, that the walkthrough has at least four steps, that there are three follow-up questions and two additional examples per flavor, that the spiritual flavor has a koan, and that no Big-O notation appears. On a scratch copy of every tenth unit, 328 of 328 problems pass. What it does not check is whether an example's stated output is right; 15 wrong examples were caught later, by the solution station.
solution.json: three flavors times three languages times two approaches#
The solution facet is the largest contract. Observed (prompt in scripts/viz/pipeline.py prompt_for, commit 971863f2; validator validate_payload in scripts/generate_batch37_next10.py, commit cb6e7ac3): per flavor, thinking with eleven named sections, a bruteForce and an optimalSolution each with complexity (including a why), code in the three languages and a stepCount, exactly six testCases split two basic, two edge and two complex, and runInIDE with a runnable file per language; at the top level, a visualization_spec describing how the problem should be drawn; no Big-O notation anywhere; at least 30,000 bytes.
Here is one real test case from the technical flavor of wa-001689 (read from a scratch copy, unmodified):
{"name": "Basic: two separate groups", "input": "n = 5, edges = [[0,1],[1,2],[3,4]]", "expectedOutput": "2",
"executionTrace": [{"step": 1, "action": "Build adjacency list. Start DFS from node 0.", "state": "Visiting 0 -> 1 -> 2. componentCount=1"},
{"step": 2, "action": "Nodes 1 and 2 already visited. Node 3 unvisited.", "state": "Start new DFS. componentCount=2"},
{"step": 3, "action": "Visit 3 -> 4. Node 4 visited. All nodes done.", "state": "Result: 2"}],
"category": "basic"}
Two facts about it matter for every later station. The input and expected output are display strings, readable by a person and by a model, but no code can run them without a parser. And this test case carries an executionTrace field that the prompt does not ask for, narrating a depth-first search, while the unit's optimal code and its visualization use union-find. Both reach the answer 2. Nothing checks that the narration matches the code.
Each flavor's code also carries numbered step comments (1️⃣, 2️⃣, …) in all three languages, and stepCount declares how many there are. The validator counts them. That field is not decoration: the visualization station turns those step numbers into shared line ids.
visualization.json: states, line ids, metadata#
The visualization facet holds a different kind of data: a recorded run of the algorithm. Observed (scripts/viz/vizlib.py validate, commit d08ba4f0): four top-level keys. testCases holds three to five cases, each with structured input and output and a list of states. annotatedCode holds the code per language as lines with ids. algorithmMeta names the algorithm type and the structures drawn. thinkingContent holds a short explanation per flavor.
The structured test case of the same unit, wa-001689, is executable as it stands:
{"input": {"n": 5, "edges": [[0, 1], [1, 2], [3, 4]]}, "output": 2}
One state of that case, trimmed for length (the variables, dataStructureState and annotation fields are shortened or omitted; every other field is as stored):
{
"step": 2,
"codeLineId": "L2",
"phase": "find",
"description": "find(0) = 0, find(1) = 1",
"variables": {"rootX": {"value": 0, "changed": true, "displayValue": "0"},
"rootY": {"value": 1, "changed": true, "displayValue": "1"}},
"pointers": {"u": {"position": 0, "color": "#f97316", "label": "u"},
"v": {"position": 1, "color": "#22d3ee", "label": "v"}},
"calculation": [{"label": "Compare", "expression": "0 === 1", "value": "false", "highlight": "amber"}]
}
codeLineId is the field that ties the facets together. Line L1 of this unit is const parent = Array.from({length: n}, (_, i) => i); in JavaScript, parent = list(range(n)) in Python and parent = new int[n]; in Java. One state highlights the same step in all three languages because the line ids are shared, and vizlib.validate refuses a state whose codeLineId is missing from any language. The visualizer factory is the home of the state contract; here it matters only as a field the unit must carry.
Two test-case representations in one unit#
The unit holds its test cases twice, in two different forms, and the difference explains where WatchAlgo's strongest check lives.
| Where | Example from wa-000001 | Who reads it |
|---|---|---|
solution.technical.testCases[0] | input "nums = [2, 2, 5]", expected "k = 2, front = [2, 5]" (display strings) | the Solution tab; the tracer worker parses them |
visualization.testCases[0] | input {"nums": [2, 2, 5]}, output 2 (structured values) | the animation; the executable cross-check |
A check can only run what it can parse. The solution's strings cannot be executed directly, so the solution validator checks wording and counts. The visualization's structured values can be executed, so the September visualization station runs the real JavaScript solution on them in Node and compares. That is how 86 solution defects were found downstream of the station that wrote them (the WatchAlgo story has the defect log).
How units vary#
A contract that fits one unit is easy. The hard part is variation that is legitimate and variation that is drift, side by side in the same folder.
Observed (inventory adapter, examples/watchalgo-adapter/README.md): problem files range from 6.7 to 15.5 KB, solutions from 27 to 179 KB, visualizations from 22 to 786 KB. Visualizations have three to five test cases (median three) and 1 to 101 states per test case (median 11, 7,142 test cases in all). The algorithm types spread wide, from hash-map (199 units) and linear-scan (136) to union-find (47); 1,172 visualizations draw a secondary structure and the rest do not.
Some variation is declared. Observed (copy of wa-001378/solution.json): a tree traversal has no meaningful brute force, so its bruteForce section carries "applicable": false and an omitReason instead of code. That is a reasonable variant. The solution validator does not know it exists: run on a copy, it stops at this unit with a missing-key error, because it expects complexity in every brute-force section.
Some variation is only in names. Observed (fixture examples/watchalgo-adapter/fixtures/watchalgo-excerpts.json): the problem example of wa-001830 (Flood Fill) calls its inputs image, sr, sc and color; the visualization calls the same inputs canvas, startRow, startCol and paintColor, the names the solution's JavaScript uses. Different names per flavor are required by the prompt; different names between facets of one flavor are not mentioned anywhere.
What is the source of truth?#
A source of truth is the one place whose content wins when two copies disagree. WatchAlgo has several candidates, and they do not all agree.
The type files src/components/solution/types.ts and src/components/visualization/types.ts describe the shapes the renderer expects. The validators describe the shapes the factory accepts. The prompts and briefs describe what workers are asked to write. The files on disk are what is actually there.
Observed in the repository: the factory treats disk as the truth. The September driver can rebuild its whole queue by scanning public/content/ for missing solution.json and visualization.json files (pipeline.py build), the renderer fetches the static files, and the visualization station reads the solution's code from the file, not from a database or a cache. That is a sound choice for a single-machine content factory: there is one copy, and every process looks at it.
Diagram: Four candidates for the source of truth, and what each one decides
Four lanes. Intentions: the prompts, briefs and type files describe what a unit should look like. Gatekeepers: the validators decide what may be written, and they changed over time. Data: the files on disk are what is actually there, and the factory treats them as the truth: the queue is rebuilt from them and the renderer serves them. Missing: a single versioned contract with a record of which validator accepted each unit. Without it, valid means valid by whichever rules ran when the file was written.
- Intentions
- prompts + briefs: prompt_for in pipeline.py, .claude/SOLUTION_ASSEMBLY_LINE.md, scripts/viz/WORKER_BRIEF.md.
- type files: src/components/solution/types.ts and src/components/visualization/types.ts.
- Gatekeepers
- validators: validate_problems.py, validate_payload, vizlib.validate. Rules tightened over time, without versions.
- Data
- files on disk: public/content as it is. The queue is rebuilt by scanning it; the renderer serves it.
- Missing
- versioned contract + accepted-by record: One contract file with a version, and per unit the validator version that accepted it.
The diagram separates the four sources by role. Disk holds the data; the validators decide what may enter it; the types and prompts describe intentions. When they disagree, the evidence notes' rule applies: code and data win over notes. The gap the diagram shows is that no single artifact holds the contract itself, with a version, so "valid" means "valid by whichever validator ran when the file was written". On a scratch copy of every tenth unit, 70 of 281 solutions fail today's validate_payload, all numbered wa-001761 or lower: the files did not change, the rules did.
Observed schema drift and defects#
Schema drift is the slow divergence of real files from their intended shape: fields that move, appear, disappear or change type while consumers keep working. The inventory names the cases, and this page checked each one again on a scratch copy.
| Unit | What is different | What still works | Source |
|---|---|---|---|
wa-002025 | sections duplicated at the top level, not identical to the nested ones; the technical flavor also holds a visualization_spec | renderer, tracer and validator read only the nested copy; validate_payload passes | inventory; copy inspected 2026-11-20; last changed 1169fae3 |
wa-001609 | optimalSolution.code and bruteForce.code are one string, not a map of three languages | the problem page; the unit has no visualization | inventory; copy inspected 2026-11-20 |
wa-000001, wa-002157, wa-002158, wa-002159 | no top-level visualization_spec | rendering, since nothing on the page reads it | inventory |
wa-002157 | five test cases instead of six (one edge case short) | rendering | copy inspected 2026-11-20 |
One detail on wa-002157 deserves its own sentence, because it shows how a checkpoint can mislead. Its solution file was first committed in a September checkpoint (5e6c6040, 2026-09-02 23:25), and it fails today's validator. Observed (local run.log): visualization tasks for wa-002153 to wa-002158 were already queued at 20:20 that evening, so the file was on disk before the supervisor's first checkpoint. Which process wrote it is not recorded. The supervisor stages every solution file present on disk. A checkpoint commit records when a file was committed, not which station accepted it.
Common confusion. "Valid JSON" and "valid unit" are different claims. All 8,399 facet files parse. wa-002025 parses, renders and passes the solution validator, and it still carries two different versions of its own answer.
In a coding agent. When a coding agent writes a unit, the unit contract is what lets the factory reject its work without reading it. The agent receives the contract with the work item; the factory's own shape check and oracle run after it exits; a file that carries an extra top-level section is refused even if every field the page reads is correct. Harness Engineering owns the controls around the agent's run; the contract belongs to the factory.
What this is not. This page is not a guide to schema languages, typed validation or repair loops: that is Schema Engineering. It describes one real unit contract, partly enforced and partly implied, and what a factory needs from such a contract.
Run the unit's own checks, on a copy#
The demonstration below runs WatchAlgo's own solution validator, validate_payload, on copies of seven units: the four fixture units and the three drift cases above. It shows what the factory's acceptance at the solution station actually enforces. A successful run proves only the result for these seven files.
The copies were made with cp -R from ~/Projects/zenalgo-frontend/public/content/<unit> into a scratch folder, next to a copy of scripts/generate_batch37_next10.py. The runner is the one docs/EVIDENCE.md section 10.3 gives:
# run_validate_payload.py: place next to a COPY of scripts/generate_batch37_next10.py
import glob, os, sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from generate_batch37_next10 import validate_payload # the copy, not the original
ok, bad = 0, []
for p in sorted(glob.glob(os.path.join(sys.argv[1], "wa-*", "solution.json"))):
try:
validate_payload(p); ok += 1
except Exception as e: # AssertionError carries (path, flavor, section, ...)
bad.append((p.split("/")[-2], repr(e)[:120]))
print(f"validate_payload: {ok} pass, {len(bad)} fail of {ok + len(bad)} solution.json files")
for code, why in bad:
print(f" {code}: {why}")
validate_payload: 2 pass, 5 fail of 7 solution.json files
wa-000001: AssertionError('content/wa-000001/solution.json')
wa-001378: KeyError('complexity')
wa-001609: AttributeError("'str' object has no attribute 'values'")
wa-001689: AssertionError(('content/wa-001689/solution.json', 'technical', 'optimal'))
wa-002157: AssertionError(('content/wa-002157/solution.json', 'technical', 'tests'))
Read the result against what you now know. wa-000001 fails the file-level assertions (it has no visualization_spec). wa-001378 fails on its declared no-brute-force variant, which the validator does not model. wa-001609 crashes on code stored as a string. wa-001689 lacks optimalSolution.complexity.why, the gap the shape primitive showed. wa-002157 has five test cases. And the two passes are wa-001830 and wa-002025: the unit with two versions of its answer passes, because the validator reads only data["solution"]. All five failing units serve their solution on the public site (read-only HEAD requests on 2026-11-20 returned 200 for each solution.json); they reached it under earlier rules or by a path this validator never saw.
Try it#
The explorer opens on the file contract view. Pick one of the four real units and see its three files side by side: which fields each holds, how the flavors and languages nest, and which commit last changed each file.
The same facts for a reader without JavaScript (from the fixture, and from the full files of the scratch copies):
| Unit | problem.json | solution.json | visualization.json | Variant or drift |
|---|---|---|---|---|
wa-000001 | 3 flavors, last changed cb6e7ac3 | 6 test cases, 3 languages, no visualization_spec | 3 test cases, 8 states in case 0, two-pointer on an array | no top-level visualization_spec |
wa-001378 | 3 flavors, cb6e7ac3 | brute force not applicable, with omitReason | 13 states in case 0, recursive-dfs on a tree | declared variant the validator rejects |
wa-001689 | 3 flavors, cb6e7ac3 | test case narrates a depth-first search; code is union-find | 11 states in case 0, union-find on a graph | no optimalSolution.complexity.why |
wa-001830 | 3 flavors; inputs image, sr, sc, color | last changed f88d3c22; inputs canvas, startRow, startCol, paintColor | 21 states in case 0, bfs on a matrix | names differ between facets |
WatchAlgo in practice#
Observed in the repository: the unit contract exists in four places at once: the prompt (prompt_for), the briefs (.claude/SOLUTION_ASSEMBLY_LINE.md, scripts/viz/WORKER_BRIEF.md), the type files the renderer compiles against, and the validators. The strictest of them, vizlib.validate, guards the visualization facet: it checks every state's keys, the shared line ids, allowed statuses and pointer positions, and nothing is written unless it passes. The loosest guard is on the problem facet's examples, which no code executes.
Reported by the author: each problem has descriptions, solutions and visualizations. The files support that sentence for 2,380 units, and support "problem and solution" for 393 more.
Teaching reconstruction: a versioned unit manifest would close most of the gaps on this page. One small file per unit would list its facets, the schema version of each, and the validator version that accepted it. Drift would then be a query ("which units were accepted by a validator older than 1.4?") instead of an archaeology project.
Enterprise translation#
The enterprise factory of Part III produces a different unit: one built service. In Example C, factory run FR-0003 takes the pattern PAT-INTAKE-API 2.1.0 and the product specification SPEC-INTAKE-001 and produces five artifacts: data_contract.json, event_schemas.json, openapi.json, rollout_manifest.json and service_config.json. That list is the unit contract of an intake service in this site's teaching example. A unit missing rollout_manifest.json is not a smaller unit; it is an incomplete one, and the factory must say so.
The WatchAlgo lessons carry over almost word for word. Write the contract once, with a version. Declare legitimate variants (a service with no events still says "no events", with a reason, the way wa-001378 says "no brute force"). Reject unknown top-level content instead of ignoring it, because an ignored section is a second version of the truth waiting to be read by the next consumer. And record which validator version accepted each unit, because enterprise rules tighten too: a security baseline added in March makes every service built in February a question.
Failure test#
The failure: a unit carries content the contract does not allow, and every check passes because each one looks only where it expects data. wa-002025 is the real case.
How a factory catches it: a strict shape check that lists what may exist, not only what must. The script below is a teaching reconstruction, not WatchAlgo code. It allows two top-level keys, requires code as a map of the three languages, accepts the declared no-brute-force variant, and checks the 2/2/2 test-case split. It was run on 2026-11-20 on the same seven copies.
"""drift_check.py: a strict shape check for solution.json (teaching reconstruction, not WatchAlgo code)."""
import json
import os
import sys
TOP = {"solution", "visualization_spec"}
FLAVORS = ("technical", "fun", "spiritual")
LANGS = {"javascript", "python", "java"}
def problems(sol):
found = []
extra = sorted(set(sol) - TOP)
if extra:
found.append("unexpected top-level keys: " + ", ".join(extra))
if "visualization_spec" not in sol:
found.append("no top-level visualization_spec")
for flavor in FLAVORS:
f = sol.get("solution", {}).get(flavor, {})
for section in ("bruteForce", "optimalSolution"):
part = f.get(section, {})
if section == "bruteForce" and part.get("applicable") is False and part.get("omitReason"):
continue # a declared variant: no brute force, with a written reason
code = part.get("code")
if not isinstance(code, dict) or set(code) != LANGS:
found.append(f"{flavor}.{section}.code is {type(code).__name__}, not a map of 3 languages")
cats = sorted(t.get("category", "?") for t in f.get("testCases", []))
if cats != ["basic", "basic", "complex", "complex", "edge", "edge"]:
found.append(f"{flavor}.testCases split is {cats}")
return found
root = sys.argv[1]
for unit in sorted(os.listdir(root)):
path = os.path.join(root, unit, "solution.json")
if not os.path.exists(path):
continue
with open(path, encoding="utf-8") as fh:
found = problems(json.load(fh))
print(f"{unit}: {'ok' if not found else str(len(found)) + ' finding(s)'}")
for p in found[:3]:
print(" - " + p)
wa-000001: 1 finding(s)
- no top-level visualization_spec
wa-001378: ok
wa-001609: 6 finding(s)
- technical.bruteForce.code is str, not a map of 3 languages
- technical.optimalSolution.code is str, not a map of 3 languages
- fun.bruteForce.code is str, not a map of 3 languages
wa-001689: ok
wa-001830: ok
wa-002025: 1 finding(s)
- unexpected top-level keys: bruteForce, optimalSolution, runInIDE, testCases, thinking
wa-002157: 4 finding(s)
- no top-level visualization_spec
- technical.testCases split is ['basic', 'basic', 'complex', 'complex', 'edge']
- fun.testCases split is ['basic', 'basic', 'complex', 'complex', 'edge']
The two checks disagree in both directions, and both disagreements teach. wa-002025 passes the observed validator and fails the strict one. wa-001378 fails the observed validator and passes the strict one, because the strict check models the declared variant. wa-001689 passes here only because this check does not look at complexity.why. What to run or toggle: save the script, copy a few unit folders to a scratch directory, and run /opt/homebrew/bin/python3 drift_check.py <copy>; then delete the omitReason line and watch wa-001378 fail.
Where to go deeper#
- Station by station (next): which station produces each facet, and which check accepts it.
- The visualizer factory: the state contract inside
visualization.json, with invariants you can corrupt. - The multi-flavor contract: flavors against languages, and checking that three voices mean one thing.
- The specification as an executable contract: how a spec is validated before any station runs.
- Data and integration engineering: schema evolution and backward compatibility in the enterprise factory.
- Schema Engineering: typed contracts, validation and repair as their own discipline.
Checklist for Monday#
- Write your unit contract in one file with a version: required parts, optional parts, declared variants.
- List what may exist, not only what must. Reject unknown top-level content.
- For every field, name its producer and its consumers. A field nobody reads is where drift hides.
- Store test cases in a form code can execute, not only as display text.
- Record which validator version accepted each unit, and re-run new rules over old units on purpose.
- Keep disk (or one store) as the single source of truth, and rebuild work lists from it.
Exercises#
wa-002025passesvalidate_payload. Which single line of the validator explains why?
Answer 1It only reads the nested object
flavor_data = data["solution"][flavor]: every check runs on the nested solution object, so top-level content is never examined.
- Why can the visualization station run an executable cross-check when the solution station cannot?
Answer 2Structured versus display values
The visualization's test cases hold structured input and output values that can be passed to the JavaScript function; the solution's test cases hold display strings such as "nums = [2, 2, 5]".
wa-001378has no brute-force code. Is that drift or a variant, and how should the contract say so?
Answer 3A declared variant
A variant: it carries "applicable": false and an omitReason. The contract should list that shape as allowed, so validators accept it and still reject a section that is merely missing.
Advanced perspective#
Files per unit or a database?Why one folder per unit worked here, and when it stops working
WatchAlgo stores 8,399 JSON files, about 588 MB in total (problems 31.6 MB, solutions 226.9 MB, visualizations 329.5 MB), as static files served with long CDN cache headers (next.config.ts, commit 9f43618c). For a content site that changes in batches and is read far more than written, this is a strong design: no database at read time, git as the history, any file inspectable with a text editor, and a work list rebuildable with a directory scan.
It stops working when several writers need to change the same unit concurrently, when a unit's facets must change together atomically, or when queries cross units ("all visualizations accepted by validator 1.2 that draw a tree"). A database gives transactions and queries, at the price of a second source of truth unless the files become an export of it. The middle road many factories take: files stay the product, and a small per-unit manifest or index table holds the metadata that queries need, rebuilt from the files.
Validator drift is a versioning problem70 of 281 sampled solutions fail today's rules
The 70 failures in the every-tenth sample are not defects introduced after acceptance. They are files accepted under an older, looser validator and judged by a newer one. Thirty-two lack optimalSolution.complexity.why, sixteen have a step-count mismatch, eight lack a complexity block, three lack bruteForce.complexity.why, and eleven fail the no-Big-O, visualization_spec or size assertion. None of the sampled solutions numbered above wa-002134 fails.
Without a recorded validator version per unit, a team cannot tell "this unit is broken" from "this unit predates the rule". With one, the decision becomes explicit: re-run the new rule over old units as a planned batch, fix or grandfather each, and record the decision. The factory learning loop treats rule changes as pattern versions with a blast radius; the unit manifest is what makes the blast radius computable.
Prose fields are part of the contract tooNarration that disagrees with the code
wa-001689's solution test case narrates a depth-first search in executionTrace, while its optimal code and visualization use union-find. A learner who reads the narration next to the animation sees two different algorithms reach the same answer. No schema check can catch that, because both texts are well-formed.
Three designs reduce the risk, in rising cost. Remove free-text fields that duplicate structured ones, so there is less to disagree. Generate narration from the same artifact that generates the trace (WatchAlgo's tracer design already does this for states, which are written by code from an executed run). Or add an inferential check, a model or a person comparing narration with code, with its own recorded error rate. Evidence of correctness discusses when a semantic check is worth that cost, and Eval Engineering owns the graders that would perform it.
In one breath#
A WatchAlgo production unit is one folder with three JSON files: a problem in three narrative flavors, a solution with three flavors times three languages times two approaches, and a visualization of recorded states tied to shared line ids. Its contract lives in prompts, briefs, type files and validators at once, with no single versioned copy, so legitimate variants like wa-001378 get rejected while drift like wa-002025 passes. The fix is a contract that lists what may exist, declares its variants, and records which validator version accepted each unit.
The unit now has a contract and a known set of exceptions. The next question is who produces each facet and which check accepts it. Station by station maps those producers to the modules the repository actually holds, marks the ones it does not, and only then gives the stations names.