Files
el/docs/v1/experiments/README.md
T
bigmerge c6ba0677f0 log v1 experiments: nineteen cycles, organised by the method that produced them
cycles/    one file per Ishikawa -> scientific method -> Six Sigma loop, named
           for the DEFECT not the fix, carrying the commit record as written at
           the time
findings/  what the cycles produced, cross-cut: live bugs, architecture answers,
           and defects in my own measurement

The organising finding is that predictions which came back FALSE produced every
significant result. Eleven of sixty-one failed, and those eleven found: that the
arity table was not drifted but 40% incomplete; that the AST traversal is
irreducible and only rules and judgments move; that guards could refuse through
the seam after all; and that routing el_bin_lookup through the gate did NOT fix
the SIGSEGV, because the fallback strlen was the hazard -- a wrong fix I would
otherwise have shipped as verified.

One cycle was run without committing predictions first and had to be discarded
as rigged. It is kept, in full, as 18-async-half-expressible.md.
2026-08-17 10:52:17 -05:00

2.6 KiB
Raw Blame History

v1 — Experiments

Every change to El on iteration-1 was produced by one loop, run repeatedly:

Ishikawa  →  scientific method  →  Six Sigma  →  repeat
  • Ishikawa — name the root cause, not the symptom. Why is this table here? never why is this table ugly?
  • Scientific method — state a hypothesis, commit predictions before running, then run it in an isolated worktree and grade every prediction including the ones that failed.
  • Six Sigma — eliminate the defect class, then add a control so it cannot silently return.

The organising finding

Predictions that came back FALSE were worth more than the ones that held.

Nineteen cycles, sixty-one predictions. The eleven that failed produced every significant result:

Failed prediction What it found
"the arity table has drifted from the header" Zero drift — but 110 functions had no entry at all. The table was not wrong, it was 40% incomplete.
"codegen drops below baseline" (×4) The traversal is irreducible. Walking an AST to find calls does not move no matter who decides. Only the rule and the judgment leave.
"guards cannot refuse through the seam" One line, and refusal works. Six compile-time kinds were unnecessary.
"C forbids the struct redefinition" C allows shadowing — and a different defect surfaced: an exit injection emitted with an empty target.
"routing el_bin_lookup through the gate fixes the SIGSEGV" It did not. The fallback was the hazard: strlen() on an integer. I would have shipped the wrong fix and called it verified.

A prediction that only ever confirms is a demonstration, not a test. One cycle was run without committing predictions first — async-half-expressible — and it produced a rigged result: pthread_join immediately after pthread_create, with the word DEFERRED printed by the test itself. It had to be discarded and re-run.

Layout

cycles/     one file per loop, numbered in order, named for the DEFECT
findings/   what the cycles produced, cross-cut by kind

Scoreboard

cycles run                     19
predictions committed          61
predictions FALSE              11   ← the useful ones
silent miscompilations found    4
security-relevant defects       2
architecture questions closed   5
defects in my own measurement   4

Every cycle verified the same three things before landing: the compiler self-hosts byte-identically (gen2 == gen3), the native suite passes, and the integration harnesses pass. A cycle that could not show all three did not land.