Files
el/docs/v1/experiments
bigmerge daad2fa2a0
El SDK CI - dev / build-and-test (pull_request) Failing after 10m9s
log the fifth measurement defect: a count that was not counting
git diff errored to stderr on a malformed revision while wc -l counted empty
stdout, producing three confident IDENTICAL results that meant nothing. Had the
promoted trees actually differed, I would have reported the promotion clean.

The correct check is not 'how many files differ' but 'is the tree object the
same object' -- all three share hash 2acd9374.

All five defects are now visibly one shape: reading a PROXY instead of the
thing. One file instead of the operation, a variable name instead of the shape,
a scope instead of the whole, a pipe's exit instead of the program's, a line
count instead of object identity.
2026-08-17 11:10:22 -05:00
..

v1 — Experiments

Every change to El on iteration-1 was produced by one loop, run repeatedly:

Ishikawa  →  scientific method  →  Six Sigma  →  repeat
  • Ishikawa — name the root cause, not the symptom. Why is this table here? never why is this table ugly?
  • Scientific method — state a hypothesis, commit predictions before running, then run it in an isolated worktree and grade every prediction including the ones that failed.
  • Six Sigma — eliminate the defect class, then add a control so it cannot silently return.

The organising finding

Predictions that came back FALSE were worth more than the ones that held.

Nineteen cycles, sixty-one predictions. The eleven that failed produced every significant result:

Failed prediction What it found
"the arity table has drifted from the header" Zero drift — but 110 functions had no entry at all. The table was not wrong, it was 40% incomplete.
"codegen drops below baseline" (×4) The traversal is irreducible. Walking an AST to find calls does not move no matter who decides. Only the rule and the judgment leave.
"guards cannot refuse through the seam" One line, and refusal works. Six compile-time kinds were unnecessary.
"C forbids the struct redefinition" C allows shadowing — and a different defect surfaced: an exit injection emitted with an empty target.
"routing el_bin_lookup through the gate fixes the SIGSEGV" It did not. The fallback was the hazard: strlen() on an integer. I would have shipped the wrong fix and called it verified.

A prediction that only ever confirms is a demonstration, not a test. One cycle was run without committing predictions first — async-half-expressible — and it produced a rigged result: pthread_join immediately after pthread_create, with the word DEFERRED printed by the test itself. It had to be discarded and re-run.

Layout

cycles/     one file per loop, numbered in order, named for the DEFECT
findings/   what the cycles produced, cross-cut by kind

Scoreboard

cycles run                     19
predictions committed          61
predictions FALSE              11   ← the useful ones
silent miscompilations found    4
security-relevant defects       2
architecture questions closed   5
defects in my own measurement   4

Every cycle verified the same three things before landing: the compiler self-hosts byte-identically (gen2 == gen3), the native suite passes, and the integration harnesses pass. A cycle that could not show all three did not land.