A reviewer wants a quick consistency check for WatchAlgo's solutions. Every unit tells its problem in three flavors, so the reviewer writes a short script: for each unit, the six test cases should state the same expected outputs in the technical, fun and spiritual flavors. The script runs read-only over all 2,773 solution files and reports that 1,101 units disagree.
The first hit is the first unit. Observed in the repository (wa-000001/solution.json, commit cb6e7ac3, read-only, 2026-11-20): the technical flavor expects k = 2, front = [2, 5], the spiritual flavor the same, and the fun flavor k = 2, front = [apple, banana]. Its input is fruitShelf = [apple, apple, banana] instead of nums = [2, 2, 5]. Nothing is wrong. The fun flavor tells the same problem with fruit, as it was asked to. The check was wrong, because it treated a change of voice as a change of facts.
Then a colleague runs the Python version of a tree traversal on a chain of 1,500 nodes and gets RecursionError, while the JavaScript and Java versions of the same unit return the list (the run is in the trap table below). That one is not about voice at all.
The learner's question: when one unit is delivered in three voices and in three programming languages, what exactly must agree across them, and which check proves it?
Two axes, two contracts#
Plain English. A flavor is the same problem told in a different voice: other names, other images, other wording. A language implementation is the same solution written in another programming language. Flavors must agree on facts a reader learns. Languages must agree on what the code returns for every allowed input. Those are different promises, so they need different checks.
A flavor is one narrative variant of a unit: the same problem, the same algorithm and the same constraints, told in a different voice with different names. A tiny example: WatchAlgo's wa-001689 is "Number of Connected Components in an Undirected Graph" in the technical flavor, "Count the Neighborhoods" in the fun flavor and "Islands of Stillness" in the spiritual flavor, with nodes becoming houses or stones.
A language implementation is the solution of a unit written in one programming language, expected to behave like the others. In wa-001689 the technical flavor's code is countComponents in JavaScript, count_components in Python and CountComponents.countComponents in Java.
Before AI. Both axes predate models. Flavors resemble localization: a product's help text in several languages, where a translator may change idioms but a number in the text must not change. Language implementations resemble client libraries generated for one API in Java, Python and TypeScript, where every client must send the same request and parse the same response, and a shared contract-test suite proves it. Teams that run both know the checks differ: translation review for the first, executable tests for the second.
| Flavors (narrative voice) | Languages (code) | |
|---|---|---|
| What may vary | names, images, wording, examples' surface form | syntax, types, standard library, naming convention |
| What must stay | the problem, the constraints, the algorithm, every fact a reader learns | the output for every allowed input, including edge cases |
| The check | consistency of facts across flavors, through a declared mapping | semantic equivalence on shared fixtures |
| Kind of evidence | comparison of stated facts, partly human or model judgment | executed results, deterministic |
| Typical failure | a renamed variable that now means something else | overflow, mutation, sort order, recursion depth, floating point |
Semantic equivalence means that two implementations give the same result for every input the specification allows, whatever their code looks like. Its practical form is a shared fixture: one file of inputs and expected outputs that every implementation reads, so all languages are judged on exactly the same cases. A cross-language check runs every implementation on the shared fixtures and compares the results with each other and with the expected values.
Common confusion. "Three flavors" in WatchAlgo does not mean three languages. Observed (docs/EVIDENCE.md, matrix row 3): the flavors aretechnical,funandspiritual; the languagesjavascript,pythonandjavaare a separate axis. A solution file holds three flavors, each with code in three languages and two approaches. The WatchAlgo story reconciles the counts.
What the repository holds on each axis#
Observed in the repository (docs/EVIDENCE.md, rows 3 and 4): every present facet carries all three flavors: the problem (problem.problem.<flavor>), the solution (solution.solution.<flavor>), and the visualization's short explanation (visualization.thinkingContent.<flavor>). The animation itself, testCases[].states, is one per test case and shared by all flavors. The site's UI labels the flavors "Technical", "Fun" and "Spiritual", with technical as the default.
The flavor contract is enforced in parts. Observed (scripts/validate_problems.py, cb6e7ac3): every flavor present, a "stands for" mapping line in the fun flavor, a koan in the spiritual flavor. Observed (prompt in scripts/viz/pipeline.py prompt_for, 971863f2): "different variable names per flavor". No validator compares the facts of one flavor with another.
Reading units side by side shows how much a flavor may change. These were read on copies on 2026-11-20:
| Unit | What the flavors share | What differs between flavors |
|---|---|---|
wa-000001 | inputs and outputs, re-encoded | fun encodes numbers as fruit ([apple, apple, banana]), declared by its "stands for" line |
wa-001689 | the same six test inputs and outputs, only renamed (n becomes numHouses, numStones) | names; the fun flavor names its rank array groupSize |
wa-000003 | the problem and its rule | entirely different examples: s = "skyline over quiet lake" expects 4; nameTag = "Alice Bob Charlie" expects 7; mantra = "peace flows through quiet waters" expects 6 |
wa-001830 | the same grids, start cells and colors | names differ even between languages of one flavor: the fun flavor's JavaScript says freshPaint, its Python paint_paint |
The wa-001689 detail is the subtle one. The fun flavor's array called groupSize starts at 1 and is incremented only when two groups of equal value merge. It behaves like the technical flavor's rank plus one, not like a size. The answers are identical, as the run below shows; the explanation a learner reads in the variable's name is not.
The language contract is enforced in a different place. Observed (EVIDENCE.md row 4; vizlib.validate, d08ba4f0): 2,772 of 2,773 solutions store code as a map of three languages (wa-001609 stores one string); every visualization has annotatedCode in all three; line ids are shared across languages. But only the JavaScript version is executed by WatchAlgo's checks: the Node cross-check runs the JavaScript solution against the tracer's output. Python and Java are never run by the factory.
The smallest primitive: one fixture file, three programs#
Example B states its shared fixtures once, in examples/b-algorithm-factory/fixtures.json. An excerpt (four of its ten cases; the file holds a version and a note as well):
{"fixtures_version": "1", "cases": [
{"name": "empty", "values": [], "target": 5, "expected": 0},
{"name": "all-smaller", "values": [1, 2, 3], "target": 9, "expected": 3},
{"name": "duplicates", "values": [1, 2, 2, 2, 3], "target": 2, "expected": 1},
{"name": "negatives", "values": [-5, -3, -3, 0, 2], "target": -3, "expected": 1}
]}
Three implementations read that same file. The functions, quoted from the example (the Python is the stand-in's accepted attempt in generator.py; the JavaScript and Java files also contain a reader for the fixtures and a print loop, not shown):
Python
def lower_bound(values, target):
lo, hi = 0, len(values)
while lo < hi:
mid = (lo + hi) // 2
if values[mid] < target:
lo = mid + 1
else:
hi = mid
return loJavaScript
function lowerBound(values, target) {
let lo = 0;
let hi = values.length;
while (lo < hi) {
const mid = (lo + hi) >>> 1;
if (values[mid] < target) lo = mid + 1;
else hi = mid;
}
return lo;
}Java
static int lowerBound(long[] values, long target) {
int lo = 0;
int hi = values.length;
while (lo < hi) {
int mid = (lo + hi) >>> 1;
if (values[mid] < target) lo = mid + 1;
else hi = mid;
}
return lo;
}The build does not compile JavaScript or Java blocks; these two are quoted from lower_bound.js and LowerBound.java, which ran in the demonstration below. The three look alike, and the small differences are deliberate. Python's integers do not overflow, so (lo + hi) // 2 is safe. JavaScript and Java use >>> 1, an unsigned shift, so the midpoint stays correct even when lo + hi exceeds the signed 32-bit range. Java reads values as long, so a fixture with large numbers cannot overflow the comparison. Each line is a language trap avoided; the fixtures are what prove the three still agree.
Diagram: Two independent axes: three flavors across, three languages down, one shared trace
Read the lanes as the three flavors. Each flavor holds the same problem in its own voice and code in three languages, so every box is one program. The shared fixtures feed every program and decide the language axis by execution. The fact check compares what each flavor states, through its declared mapping, and decides the flavor axis. The visualization's states are one per test case and shared by all nine programs; only a short explanation text differs per flavor. Names and counts are observed in wa-001689.
- Shared inputs
- shared fixtures: One set of inputs and expected outputs that every language implementation reads.
- facts + mapping: The unit's facts (constraints, examples, outputs) and each flavor's declared encoding of them.
- Technical flavor
- countComponents: The technical flavor's optimal solution in three languages.
- Fun flavor
- countNeighborhoods: The same algorithm told with houses and streets; names change, behavior must not.
- Spiritual flavor
- countIslands: The same algorithm told with stones and sand paths.
- Checks
- cross-language check: Every program runs on the shared fixtures; results are compared with each other and with the expected values.
- fact check: Stated facts are mapped back and compared; mismatches go to a reviewer.
- one trace per test case: The visualization's states do not vary by flavor or language; only thinkingContent text varies by flavor.
The diagram lays the two axes out as a grid. Flavors run across, languages down, and every cell is one program. The shared fixtures feed every cell and decide the language axis. The fact check reads across a row of facts, through each flavor's declared mapping, and decides the flavor axis. The visualization sits outside the grid: one trace per test case, shared by all nine cells.
The demonstration: Example B in three runtimes#
Copied to a scratch folder with cp -R examples/b-algorithm-factory, then run on 2026-11-20 with /opt/homebrew/bin/python3 run.py --verbose. The cross-language part of the output, as printed:
cross-language over fixtures.json (10 cases)
case python node java
empty 0 0 0
all-smaller 3 3 3
all-larger 0 0 0
duplicates 1 1 1
duplicates-at-end 2 2 2
single-below 0 0 0
single-equal 0 0 0
single-above 1 1 1
between 3 3 3
negatives 1 1 1
equivalent: yes (2 runtimes compared with python)
Exactly what ran: crosslang.py : compare called the accepted Python function in-process on each case; started node lower_bound.js fixtures.json (Node v26.3.1); compiled LowerBound.java with javac and ran java -cp <build dir> LowerBound fixtures.json (OpenJDK 23.0.1). Each program printed one line per case, and the table compares the Node and Java lines with Python's. The example's own test file proves the same thing and refuses to pass when a runtime is missing:
$ /opt/homebrew/bin/python3 -m unittest tests.test_crosslang -v
test_java_matches_python (tests.test_crosslang.CrossLanguage.test_java_matches_python) ... ok
test_node_matches_python (tests.test_crosslang.CrossLanguage.test_node_matches_python) ... ok
test_python_matches_fixture_expectations (tests.test_crosslang.CrossLanguage.test_python_matches_fixture_expectations) ... ok
----------------------------------------------------------------------
Ran 3 tests in 0.407s
OK
If node or javac is absent, those tests are skipped with a message, never counted as passes. The agent here is a scripted stand-in, so these results show how the factory mechanics work, not how good any model is.
A real unit as nine programs#
Example B is a toy written for the purpose. The stronger test is a real WatchAlgo unit, where three flavors and three languages were produced by workers. Observed, run for this page on copies (2026-11-20): wa-001689 was copied from ~/Projects/zenalgo-frontend/public/content/ to a scratch folder. A harness written for this page took each flavor's optimal solution in each language exactly as stored, added only a call per fixture and a print, and ran it: JavaScript with Node v26.3.1, Python with 3.14.7, Java with OpenJDK 23.0.1 (javac then java). The fixtures are the unit's three structured visualization test cases plus the six solution test cases, parsed from the technical flavor's display strings.
wa-001689: 9 fixtures x 3 flavors x 3 languages
fixture expected tech/js tech/py tech/jv fun/js fun/py fun/jv spir/js spir/py spir/jv
viz-tc0 2 2 2 2 2 2 2 2 2 2 ok
viz-tc1 4 4 4 4 4 4 4 4 4 4 ok
viz-tc2 2 2 2 2 2 2 2 2 2 2 ok
sol-tc0 2 2 2 2 2 2 2 2 2 2 ok
sol-tc1 1 1 1 1 1 1 1 1 1 1 ok
sol-tc2 4 4 4 4 4 4 4 4 4 4 ok
sol-tc3 1 1 1 1 1 1 1 1 1 1 ok
sol-tc4 3 3 3 3 3 3 3 3 3 3 ok
sol-tc5 2 2 2 2 2 2 2 2 2 2 ok
fixtures where all 9 programs equal the expected value: 9 of 9
runtimes: v26.3.1; python 3.14.7; openjdk version "23.0.1" 2024-10-15
All 81 results agree. The fun flavor's misleading groupSize name changed no answer, which is the point of separating the axes: the language check passes and says nothing about the name.
For wa-000003, where every flavor has its own examples, the right check is different: every program must reproduce, for every input, the output written next to that input. The same kind of harness ran each of the nine programs on all eighteen inputs of the three flavors:
technical js: 18 of 18 inputs match the written expectedOutput
technical py: 18 of 18 inputs match the written expectedOutput
technical jv: 18 of 18 inputs match the written expectedOutput
fun js: 18 of 18 inputs match the written expectedOutput
fun py: 18 of 18 inputs match the written expectedOutput
fun jv: 18 of 18 inputs match the written expectedOutput
spiritual js: 18 of 18 inputs match the written expectedOutput
spiritual py: 18 of 18 inputs match the written expectedOutput
spiritual jv: 18 of 18 inputs match the written expectedOutput
all programs x all inputs: 162 of 162 agree
Running every flavor's code on every flavor's examples turns per-flavor examples into a shared fixture set after the fact. What the two runs prove: these two units' stored programs agree on these 27 inputs in three runtimes. They do not prove anything about the other units, and they say nothing about whether the prose of each flavor is accurate.
One more thing the harness had to handle is itself a language difference. In wa-001689 the Java method is static; in wa-000003 it is an instance method of a class named Solution, so the first attempt to call it failed to compile ("non-static method lengthOfLastWord(String) cannot be referenced from a static context"). A cross-language check needs a per-language adapter, and the adapter is code the factory must own and test.
Language-specific traps#
Semantic equivalence fails in predictable places. Each row below was run on 2026-11-20 on this machine, from small files written for this page (the recursion row uses the stored code of wa-001378 on a copy).
| Trap | Input | JavaScript | Python | Java |
|---|---|---|---|---|
| Integer overflow in a midpoint | lo = 1,500,000,000, hi = 2,000,000,000 | (lo + hi) >> 1 gives -397483648; >>> 1 gives 1750000000 | (lo + hi) // 2 gives 1750000000 | (lo + hi) / 2 gives -397483648; lo + (hi - lo) / 2 and >>> 1 give 1750000000 |
| Default sort order | [10, 9, 1] | .sort() gives [1,10,9]; with (a, b) => a - b gives [1,9,10] | sorted gives [1, 9, 10] | Arrays.sort gives [1, 9, 10] |
| Floating point | 0.1 + 0.2 | 0.30000000000000004 | 0.30000000000000004 | 0.30000000000000004 |
| Recursion depth | wa-001378 inorder traversal on a right-leaning chain of N nodes | N = 10,000 ok; 20,000 RangeError | N = 500 ok; 1,500 RecursionError (limit printed: 1000) | N = 10,000 ok; 20,000 StackOverflowError |
Three lessons sit in the table. First, identical source text can mean different things: >> 1 and / 2 look harmless and break at the same large sum in JavaScript and Java. Second, a language's default is part of its semantics: Product (MDN, read 2026-11-20): JavaScript's default sort converts elements to strings and compares "their sequences of UTF-16 code unit values", which is why [10, 9, 1] sorts as [1, 10, 9]. Third, floating point agrees across these three runtimes and is still not equal to 0.3, so fixtures with decimals need a tolerance, stated in the specification.
The recursion row needs its context. Observed (copy of wa-001378/problem.json): the unit's constraint is "The tree has between 0 and 100 nodes". Every run within the constraint passed in all three languages, including the three structured test cases. The failures appear far outside it, at depths the specification does not allow. That is the right way to read a language trap: a fixture set must include the constraint boundary (here a 100-node chain), and an input beyond it is a different specification. A factory whose units claim larger limits must test at those limits in every language, because Python fails first.
Two more traps belong on every checklist. Sort stability: equal elements may keep or lose their order. Product (Python documentation, read 2026-11-20): "Sorts are guaranteed to be stable." Product (Java SE 23 API, List.sort): "The sort is stable: this method must not reorder equal elements." Product (MDN): JavaScript's Array.prototype.sort is stable "since version 10 (or ECMAScript 2019)". A port to an older runtime, or to an unstable sort such as one written by hand, can change outputs on ties. Mutation: a function may change its input instead of returning a new value. Observed (docs/EVIDENCE.md §10.2): WatchAlgo's wa-000161 ("Merge Sorted Array") mutates its input and returns nothing, and in the evidence notes' sample, seven units whose tracers opt out of the Node cross-check give reasons that include in-place mutation. A cross-language check that compares only return values silently passes such code; the harness must compare the mutated argument too.
Diagram: The language check is a machine; the flavor check is a machine plus a reviewer
Top lane, languages: shared fixtures and a per-language adapter feed every implementation, the results are compared, and the verdict is deterministic. Bottom lane, flavors: facts are extracted from each flavor, mapped back through the flavor's declared encoding, executed against the reference solution where possible, and only true mismatches go to a person, who decides whether the difference is voice or error. The model recommends; a person decides.
- Language axis
- fixtures + adapters: The fixture file plus code that builds trees, lists and calls static or instance methods in each language.
- run every language: Each implementation prints its results for every fixture; mutated arguments are compared too.
- equal?: All results equal the expected values, or the work item goes back as rework.
- Flavor axis
- extract stated facts: Pull examples, outputs and constraints out of each flavor's text; a model can help triage.
- map and execute: Map names back to facts and run the reference solution on mapped inputs.
- reviewer decides: A person judges what no fixture settles, such as a misleading variable name.
The diagram puts the two checks side by side. The language check is a machine: shared fixtures in, nine executions, a comparison, a verdict. The flavor check has a machine part and a judgment part: extract stated facts from each flavor, map them through the flavor's declared encoding, compare, and send what does not match to a reviewer, who decides whether it is a voice change or an error.
In a coding agent. A coding agent asked to "port this solution to Java" will report success when its port compiles and its own example runs. In a factory, the agent receives the work item and the shared fixture file, writes the port in a scoped checkout, and the factory's cross-language suite, not the agent's example, decides whether the port advances; a person with merge rights reviews what passes. The agent never edits the fixtures it is judged by. Harness Engineering owns the controls that make that last rule hold.
What this is not. This page is not about translating content into human languages, and not a general guide to differential or property-based testing; Evidence of correctness is the home of test techniques, and Eval Engineering owns graders for judgments no fixture can settle, such as whether a flavor's prose is accurate.
Try it#
The explorer opens on the code tabs. Pick a unit, switch between JavaScript, Python and Java, and step the visualization: the same states drive all three tabs. On wa-001689 and wa-001378, compare the highlighted line across the tabs (see the visualizer chapter for why it moves).
The four explorer units, for a reader without JavaScript (from examples/watchalgo-adapter/fixtures/watchalgo-excerpts.json, extracted read-only at commit 1b183564; this table was written for this page):
| Unit | Technical · fun · spiritual title | First line in JavaScript · Python · Java | Code lines (JS · Py · Java) |
|---|---|---|---|
wa-000001 | Remove Duplicates from Sorted Array · Remove Extra Fruits · One of Each Stone | function removeDuplicates(nums) { · def remove_duplicates(nums): · public class Solution { | 23 · 21 · 25 |
wa-001378 | Binary Tree Inorder Traversal · Visit Family Left-First · Walking the Left Path First | function inorderTraversal(root) { · from typing import Optional, List · import java.util.ArrayList; | 23 · 30 · 34 |
wa-001689 | Number of Connected Components in an Undirected Graph · Count the Neighborhoods · Islands of Stillness | function countComponents(n, edges) { · def count_components(n, edges): · public class CountComponents { | 29 · 33 · 32 |
wa-001830 | Flood Fill · Painting Desk Groups · Color Flowing Through Stones | function floodFill(canvas, startRow, startCol, paintColor) { · from collections import deque · import java.util.*; | 38 · 33 · 44 |
Or run the cross-language check yourself: copy examples/b-algorithm-factory to a scratch folder and run /opt/homebrew/bin/python3 run.py; then change >>> 1 to >> 1 in lower_bound.js and add a fixture whose answer needs a large index, and watch which runtimes still agree.
WatchAlgo in practice#
Reported by the author: WatchAlgo has descriptions, solutions and visualizations in three flavors. Observed in the repository: the three flavors are narrative styles present in every facet file; the code languages are a second axis present in every solution and visualization except the one-string drift case wa-001609; the animation is one per test case for all flavors. The report holds, read as narrative flavors.
Observed: flavor consistency is checked for presence and form (validate_problems.py: every flavor, a "stands for" line, a koan), not for facts. Language consistency is enforced for line ids (vizlib.validate) and checked by execution only in JavaScript (the Node cross-check, which found 86 solution defects in one category, .ai-context/VIZ_SOLUTION_BUGS_FOUND.md, 4426f119). Python and Java code is shipped to readers and run by no factory check.
Observed, run for this page on copies: two units' nine programs agree on all their fixtures (81 of 81 and 162 of 162 results). A naive cross-flavor comparison of expected outputs over all 2,773 solutions flags 1,101 units; among the first hits are voice re-encodings (fruit for numbers) and per-flavor examples (wa-000003), both legitimate. The number measures the naivety of the check, not defects.
Teaching reconstruction: a WatchAlgo-shaped factory would treat each flavor as a variant output of one station, generated from one source of facts, and would run all three languages on the structured test cases, extending the existing Node cross-check to Python and Java.
Enterprise translation#
Enterprises meet both axes, usually without naming them. The flavor axis appears whenever one fact is told to different audiences: a broker-facing and an underwriter-facing explanation of the same validation error, a customer notice and an internal ticket for the same incident, marketing copy and API documentation for the same limit. The fact (a missing roof age, a 63-field submission, a rule id) must survive every voice. The language axis appears whenever one behavior is implemented twice: a validation rule in a Python service and in a Java legacy adapter, a pricing function in the backend and in a mobile client, an SDK in three languages for one API.
In the insurance factory of Part III, Example C's factory run FR-0003 builds the intake service from PAT-INTAKE-API 2.1.0 and the product specification, with fixtures that expect 401 without a token, 403 when a broker tries to record a decision, and 422 with field errors. This site's recommendation: if the insurer's factory also generated a client in a second language, those fixtures, not new ones written for the client, would become its cross-language suite, and the broker-facing and underwriter-facing error texts would be checked for the same field names and rule ids through a declared mapping. The enterprise difference is scale and ownership: each language belongs to a team, so the shared fixture file needs an owner and a version, and a change to it is a change to every implementation's contract. The insurance intake chapter builds the service.
Failure test#
The failure: a JavaScript port sorts numbers with .sort() and no compare function. Every fixture with single-digit values passes, because string order and numeric order agree on them. Python and Java pass everything.
How the factory catches it: a shared fixture that includes values of different digit counts. The cross-language check then disagrees on that fixture, and the factory blocks the port. The underlying behavior, run on 2026-11-20 from a file written for this page:
$ node more.js
js 0.1 + 0.2 = 0.30000000000000004
js [10, 9, 1].sort() = [1,10,9]
js [10, 9, 1].sort((a, b) => a - b) = [1,9,10]
$ /opt/homebrew/bin/python3 more.py
py 0.1 + 0.2 = 0.30000000000000004
py sorted([10, 9, 1]) = [1, 9, 10]
py sys.getrecursionlimit() = 1000
What to run or toggle: in a scratch copy of Example B, look at fixtures.json and ask which traps its ten cases would catch (negatives yes; multi-digit values, large indices and decimals no), then add a case and rerun run.py. The general rule: every row of the trap table should have at least one fixture that exposes it, within the constraints the specification allows.
Decision: one source or independent implementations?#
| Option | What it looks like | Choose it when | Counterexample | Failure mode |
|---|---|---|---|---|
| One reference, translations checked against it | one language is canonical; the others are ports judged by shared fixtures | one team owns the algorithm; the reference is executable | WatchAlgo executes only JavaScript today, so its ports are unchecked | ports drift silently when only the reference is tested |
| Independent implementations, differential check | each language written separately; disagreement is a signal | a wrong answer is costly; independent writers are available | Example B's three files agree on ten fixtures | shared misreadings of the specification agree with each other |
| Generate all languages from one source | a transpiler or a code generator emits each language | the logic fits a common subset; the generator is trusted | client SDKs from one API description | the generator's own bugs reach every language at once |
The criterion is where independent evidence comes from. This site's recommendation: whichever option you choose, keep one shared fixture file per unit, owned and versioned, run every language on it in the factory, and never let the code under test edit it. For flavors, generate every voice from one structured source of facts, and check the facts, not the strings.
Where to go deeper#
- Looping over a large backlog (next): scheduling thousands of such units.
- The visualizer factory: the trace shared by all flavors, and the line ids that link it to each language.
- A problem as a production unit: where flavors and languages live in the unit's files.
- Evidence of correctness: differential, randomized and metamorphic testing in general.
- Eval Engineering: graders for prose consistency, where no fixture can decide.
- Schema Engineering: typed contracts that keep fixture files honest.
Checklist for Monday#
- Name your axes: which variants are voice, which are behavior. Give each its own check.
- Keep one shared fixture file per unit, versioned and owned; every language reads it in the factory.
- Cover each language trap with a fixture inside the constraints: large sums, multi-digit sorts, decimals with a tolerance, depth at the size limit, ties, mutated inputs.
- Compare mutated arguments as well as return values.
- For flavors, declare the mapping (what stands for what) and compare facts through it; send mismatches to a reviewer.
- Pin runtime versions in the factory's record: a cross-language result is only as reproducible as the runtimes that produced it.
Exercises#
- Why did the reviewer's naive flavor check flag
wa-000001, and what should it have compared?
Answer 1A declared re-encoding
The fun flavor encodes numbers as fruit, declared by its "stands for" line. The check should map fruit back to numbers through that declaration, then compare the values.
- Example B's Python uses
(lo + hi) // 2but JavaScript and Java use>>> 1. Would(lo + hi) // 2be wrong in Python?
Answer 2No
Python integers have arbitrary precision, so the sum cannot overflow (the midpoint run printed 1750000000). The trap is language-specific.
- The
wa-001378traversal fails in Python at 1,500 nodes. Is that a defect of the unit?
Answer 3Not under its constraints
The unit allows at most 100 nodes, and all three languages pass there. It becomes a defect only if the specification raises the limit; then Python needs an iterative version or a stated limit.
Advanced perspective#
Designing a flavor fact checkFacts, mappings and the share left to judgment
A flavor check that compares strings fails both ways: it flags voice (1,101 units in the naive run) and misses meaning (the groupSize name passes any string check). A workable design has three layers. First, a structured source of facts per unit: constraints, example inputs and outputs, the algorithm's name and complexity class. Second, a declared mapping per flavor: which names and images stand for which facts, the "stands for" line made machine-readable. Third, an extraction step that reads each flavor's facts, maps them back, and compares with the source.
The third layer is where a model can help, and where the series' rule applies: the model recommends, a person decides. A model can extract "the example says 7" from "Alice Bob Charlie expects 7" reliably enough to triage, and the factory can then execute the reference solution on the mapped input to confirm. What remains for a reviewer is the judgment no fixture settles: does "groupSize" mislead a learner? Scoring such judgments belongs to Eval Engineering; the factory's job is to route only the true mismatches to it.
The multiplication costNine programs per unit, and where the checks should scale
Each WatchAlgo solution holds two approaches in three languages in three flavors: up to eighteen programs per unit before the runnable harness copies (fewer where a brute force is declared not applicable). Across 2,773 solutions that is up to about fifty thousand stored programs, of which the factory executes only the JavaScript optimal solution, and only through the visualization cross-check. Generation cost scales with that multiplication; verification, today, does not.
Two ways to bring them back into line. Reduce what must be verified: if every flavor's code were generated from one source by renaming identifiers mechanically, a check that the renaming is consistent would replace eight of nine executions, and the remaining execution per language would carry the evidence. Or scale verification with generation: execution on shared fixtures needs no model and no reviewer, so running all eighteen programs per unit is a modest addition to a line that already runs JavaScript. The expensive part is the per-language adapter (static versus instance methods, tree and list construction, mutated arguments), which is written once per input shape, not once per unit.
Runtime versions are part of the fixtureWhy a cross-language pass must record what it ran on
The trap table's results belong to this machine on 2026-11-20: Node v26.3.1, Python 3.14.7, OpenJDK 23.0.1. Some behaviors are fixed by the language (Java's int arithmetic, Python's unbounded integers). Others are runtime defaults that differ between versions and configurations: the depth at which JavaScript or Java runs out of stack depends on stack size settings, and JavaScript sort stability depends on the runtime implementing ECMAScript 2019 or later.
This site's recommendation: treat the runtime as an input of the cross-language station. Record each runtime's version in the station's evidence next to the fixture version, include both in the station's input hash, so that a runtime upgrade re-runs the check instead of reusing a stale pass (the smallest factory is the home of input hashes), and pin the versions in the container or toolchain file the factory uses. A pass that cannot say which runtimes produced it is a self-report of the factory itself.
In one breath#
A WatchAlgo unit varies along two independent axes: three narrative flavors that must keep every fact while changing voice, and three code languages that must return the same results for every allowed input. The first is checked by comparing facts through a declared mapping, with a reviewer for what no machine can settle; the second by running every language on one shared fixture file, as Example B does in Python, Node and Java, and as one real unit's nine programs did here on copies, with 81 of 81 results agreeing. Language traps (overflow, default sort order, floating point, recursion depth, stability, mutation) are where equivalence breaks, so the fixtures must expose each of them within the specification's constraints.
One unit is now fully specified: its files, its stations, its trace and its two axes of variation. WatchAlgo has 3,246 of them. Looping over a large backlog turns from the single unit to the line: how to schedule thousands of independent units, limit work in progress, retry what fails, and keep a queue of failed cases from turning into a storm.