bigmerge
daad2fa2a0
git diff errored to stderr on a malformed revision while wc -l counted empty
stdout, producing three confident IDENTICAL results that meant nothing. Had the
promoted trees actually differed, I would have reported the promotion clean.
The correct check is not 'how many files differ' but 'is the tree object the
same object' -- all three share hash 2acd9374.
All five defects are now visibly one shape: reading a PROXY instead of the
thing. One file instead of the operation, a variable name instead of the shape,
a scope instead of the whole, a pipe's exit instead of the program's, a line
count instead of object identity.
2.6 KiB
Defects in my own measurement
Recorded because the pattern is the point: five of these, and every one is the same shape — reading a proxy instead of the thing. A file instead of the operation, a variable name instead of the shape, a scope instead of the whole, a pipe's exit instead of the program's, a line count instead of the object identity. Each was caught by running something, never by reading.
1. Scoped the search to one file
Reported test as an inert keyword by checking only parser.el. codegen
consumes it at 4135 for --test mode, and the tree has 408 uses. Removing it
would have broken every test in the suite — including the ones used to verify
the removal.
2. Searched by variable name, not by operation
Grepped for native_list_append(tokens to find direct token appends.
interp_tokens_append_all calls its parameters dst/result, carries its own
copy of the stride, and corrupted generation 2 — while generation 1 built fine,
because the compiler's own source uses string interpolation.
3. Scoped to compiler sources; the stride had escaped into tests
tok_count in test_compiler.el computed len/2 independently. 21 tests failed
after the token layout changed.
4. Read the wrong exit code
timeout 10 /tmp/leakrun 2>&1 | head -2; echo "exit=$?" # reports head's exit
Reported exit=0 for a program that was returning 139 (SIGSEGV). I nearly
recorded a segfault as a clean run.
5. Read a count that was not counting
Comparing the three promoted branches:
for pair in "dev stage" ...; do set -- $pair
n=$(git diff --stat origin/$1 origin/$2 | wc -l) # git errored to STDERR
... # wc counted empty STDOUT
git diff failed on a malformed revision, wrote its error to stderr, and wc -l
counted zero lines of stdout. Three confident IDENTICAL results, all
meaningless. Had the trees actually differed, I would have reported the
promotion clean.
Redone correctly, the three trees share one hash — 2acd9374 — which is the
check that should have been run first: not "how many files differ" but "is the
tree object the same object".
And one that was not a measurement defect but a method defect
One cycle was run without committing predictions first — see
cycles/18-async-half-expressible.md. The test joined the thread immediately
after creating it and printed the word DEFERRED itself. A test authored by the
party holding the conclusion, with nothing committed beforehand, cannot fail.
It had to be discarded and re-run.