bigmerge
daad2fa2a0
El SDK CI - dev / build-and-test (pull_request) Failing after 10m9s
git diff errored to stderr on a malformed revision while wc -l counted empty
stdout, producing three confident IDENTICAL results that meant nothing. Had the
promoted trees actually differed, I would have reported the promotion clean.
The correct check is not 'how many files differ' but 'is the tree object the
same object' -- all three share hash 2acd9374.
All five defects are now visibly one shape: reading a PROXY instead of the
thing. One file instead of the operation, a variable name instead of the shape,
a scope instead of the whole, a pipe's exit instead of the program's, a line
count instead of object identity.
63 lines
2.6 KiB
Markdown
63 lines
2.6 KiB
Markdown
# Defects in my own measurement
|
|
|
|
Recorded because the pattern is the point: **five of these, and every one is the same shape —
|
|
reading a proxy instead of the thing.** A file instead of the operation, a
|
|
variable name instead of the shape, a scope instead of the whole, a pipe's exit
|
|
instead of the program's, a line count instead of the object identity. Each was caught
|
|
by running something, never by reading.
|
|
|
|
### 1. Scoped the search to one file
|
|
|
|
Reported `test` as an inert keyword by checking only `parser.el`. **codegen**
|
|
consumes it at 4135 for `--test` mode, and the tree has 408 uses. Removing it
|
|
would have broken every test in the suite — including the ones used to verify
|
|
the removal.
|
|
|
|
### 2. Searched by variable name, not by operation
|
|
|
|
Grepped for `native_list_append(tokens` to find direct token appends.
|
|
`interp_tokens_append_all` calls its parameters `dst`/`result`, carries its own
|
|
copy of the stride, and corrupted generation 2 — while generation 1 built fine,
|
|
because the compiler's own source uses string interpolation.
|
|
|
|
### 3. Scoped to compiler sources; the stride had escaped into tests
|
|
|
|
`tok_count` in `test_compiler.el` computed `len/2` independently. 21 tests failed
|
|
after the token layout changed.
|
|
|
|
### 4. Read the wrong exit code
|
|
|
|
```bash
|
|
timeout 10 /tmp/leakrun 2>&1 | head -2; echo "exit=$?" # reports head's exit
|
|
```
|
|
|
|
Reported `exit=0` for a program that was returning **139 (SIGSEGV)**. I nearly
|
|
recorded a segfault as a clean run.
|
|
|
|
### 5. Read a count that was not counting
|
|
|
|
Comparing the three promoted branches:
|
|
|
|
```bash
|
|
for pair in "dev stage" ...; do set -- $pair
|
|
n=$(git diff --stat origin/$1 origin/$2 | wc -l) # git errored to STDERR
|
|
... # wc counted empty STDOUT
|
|
```
|
|
|
|
`git diff` failed on a malformed revision, wrote its error to stderr, and `wc -l`
|
|
counted zero lines of stdout. Three confident `IDENTICAL` results, all
|
|
meaningless. **Had the trees actually differed, I would have reported the
|
|
promotion clean.**
|
|
|
|
Redone correctly, the three trees share one hash — `2acd9374` — which is the
|
|
check that should have been run first: not "how many files differ" but "is the
|
|
tree object the same object".
|
|
|
|
### And one that was not a measurement defect but a method defect
|
|
|
|
One cycle was run **without committing predictions first** — see
|
|
`cycles/18-async-half-expressible.md`. The test joined the thread immediately
|
|
after creating it and printed the word `DEFERRED` itself. A test authored by the
|
|
party holding the conclusion, with nothing committed beforehand, cannot fail.
|
|
It had to be discarded and re-run.
|