el-retired/docs/v2/experiments/evidence/cycles/02-seven-tables-and-a-shipped-defect/CLAIMS.md

CLAIMS — v2 cycle 02, "seven tables, and a defect I shipped that morning"

Re-run 2026-08-17. Source doc: docs/v2/experiments/cycles/02-seven-tables-and-a-shipped-defect.md (not modified; it is the historical record).

Pinned commits

Role	SHA	Subject
BUGGY — before the repair	1a5246667d80e0bf5830a06a04e1140c2867e4ce	log the experiments that were run and never written down
FIXED — the code commit	abb0ab41980f577ee8bfda04d4339e6355adfa48	finish the seven tables, and repair a worse defect they exposed
Doc commits	69f2ab2, ebb4f1f	v2 cycle 02: …

Two detached worktrees (/tmp/rerun-v2-02a at the buggy parent, /tmp/rerun-v2-02b at the fix), a compiler built in each, and the probe programs compiled and linked against each worktree's own runtime — so the runtime under test is the historical one, not today's. Every .prov carries 1a5246667… or abb0ab419…; none is flagged DIRTY-TREE by verify-manifest.sh.

Verdicts

#	Claim as stated in the doc	Artifact	Commit	Verdict
A1	P1 — six of seven are one shape; 5 take a call_expr, 1 takes a name	0021, 0022	both	REPRODUCED — six collapse to 3–6 line wrappers, duration_unit_nanos does not
A2	P2 — 49 names moved to signatures.rel	0020	both	REPRODUCED exactly — 51 → 100 entries, delta 49
A3	P3 — duration_unit_nanos breaks the shape (name → MULTIPLIER, not name → type)	0022	abb0ab4	REPRODUCED — it alone keeps 17 lines and all 14 string literals
A4	P4 — codegen drops ~120 lines: FALSE, 80	0019	both	DIVERGED (−79, off by one) — see below
A5	each table became a 3-line wrapper	0022	abb0ab4	REPRODUCED — five are exactly 3 lines; is_void_builtin is 6
A6	P5 — fixpoint holds	0031	abb0ab4	REPRODUCED — gen2 == gen3, identical sha256
A7	P5 — all suites hold; 105/105 native, 253 assertions	0032	abb0ab4	REPRODUCED exactly
A8	P5 — 10/10 harnesses, 53 → 58 assertions	0033	abb0ab4	REPRODUCED exactly — 10 harnesses, 0 failed, 58 assertions
A9	P8 — an unrecognised unit silently becomes nanoseconds: FALSE	0023, 0024	1a52466	REPRODUCED — it is compiled as a field access on the integer
A10	5.fortnights → el_get_field(5, …) → exit 139, SIGSEGV	0023	1a52466	REPRODUCED — exit code 139, captured directly
A11	as_map was a bare cast with no validation	0028	1a52466	REPRODUCED verbatim: static ElMap* as_map(el_val_t v) { return (ElMap*)(uintptr_t)v; }
A12	P9 — routing through el_tagged stops the SIGSEGV	0026, 0028	abb0ab4	REPRODUCED — same program now prints survived, exit 0
A13	P10 — legitimate access unaffected: {"a":1}["a"] → 1, [10,20,30][1] → 20	0025, 0027	both	REPRODUCED — and the two outputs are byte-identical (same sha256 783e5bca6f66) before and after the fix
A14	P11 — el_list_get / el_list_len had the identical defect, now gated	0028	both	REPRODUCED
A15	the guard was looks_like_heap_obj, testing p & 0x7	0004	1a52466	REPRODUCED verbatim
A16	the shipped fallback was if (!looks_like_heap_obj(...)) return 0; return strlen(s);	0003	1a52466	REPRODUCED verbatim
A17	el_input_len silently returned 0 for roughly 60% of string literals	0006, 0007, 0008, 0009	1a52466	DIVERGED — the real rate is worse. See below
A18	which literals failed depended on link layout	0008, 0009	1a52466	REPRODUCED — and this is the reason A17 diverges
A19	"abc" happened to land on an 8-boundary in that build	0008	—	REPRODUCED — abc is still aligned8=1 today, while 4 of the doc's other 5 are not
A20	sha256_hex would hash the empty string instead of its input	0006, 0007	1a52466	REPRODUCED — 20 of 20 literals return the empty-string hash
A21	the repair checks "not a small integer, not a low address", and nothing more	0013	abb0ab4	REPRODUCED
A22	tagged_gate.sh grew from 5 assertions to 10	0016, 0017, 0018	both	REPRODUCED — 10 assertions, 10 passed
A23	including an explicit assertion that no literal silently hashed as empty	0018	abb0ab4	REPRODUCED — the assertion is present and green

Counts: 21 REPRODUCED, 2 DIVERGED, 0 NOT-REPRODUCIBLE, 0 NOT-CAPTURED.

---

A17 — THE DIVERGENCE THAT MATTERS: the shipped defect was worse than recorded

The doc says el_input_len returned 0 for "roughly 60%" of string literals. Measured today at the buggy commit, with a probe of 20 string literals of lengths 1 through 20, compiled and linked against 1a52466's own runtime and checked against Python's hashlib.sha256 as an independent oracle (0007):

literals tested              = 20
correct vs hashlib.sha256    = 0
incorrect                    = 20
hashed as the EMPTY STRING   = 20
failure rate                 = 100%

Every one of the twenty. Not 60% — 100% in this build. The same program at the fixed commit (0015):

literals tested              = 20
correct vs hashlib.sha256    = 20
incorrect                    = 0
hashed as the EMPTY STRING   = 0
failure rate                 = 0%

The mechanism is exactly as the doc describes — looks_like_heap_obj rejects any pointer with p & 0x7 set, and the linker packs string literals contiguously with no padding, so consecutive short literals are almost never 8-aligned. Measured directly (0008, 0009):

the doc's own five literals              4 of 5 misaligned  (80%)
twenty literals, lengths 1..20          18 of 20 misaligned  (90%)

Note abc is still aligned8=1 today — the doc's explanation for why the test passed reproduces precisely — while hello, which the doc measured as aligned, is not aligned in today's build. The specific set of victims is link-layout dependent, exactly as the doc says. The rate is not a stable quantity at all, and the doc's 60% was a five-literal sample.

This divergence flatters nothing and is reported for that reason: the doc understates the severity of the worst defect in the v2 record. And it understates it by the same error the cycle exists to teach — a sample of five is not a measurement of a rate. That is defect 7's shape ("a single literal is not a test of a property"), committed inside the paragraph that names it.

The correct statement, with artifacts behind it: el_input_len returned 0 for between 80% and 100% of string literals in the builds measured, the exact set depending on link layout, and in the end-to-end El program measured today it was all of them.

A4 — the second divergence, cosmetic

doc:      codegen drops 80 lines
measured: TOTAL 4524 -> 4445 = -79        (0019)
          CODE  3277 -> 3209 = -68

Off by one, the same +1 pattern that appears in cycle 05's totals (documented in that cycle's CLAIMS.md). The cycle's conclusion — that the predicted ~120 did not materialise because each table left a wrapper — is unaffected: 79 is as far from 120 as 80 is. Recorded because a document about miscounting should not carry an uncorrected miscount.

Cycle 05 later established that the metric worth quoting is code, not total. On that metric the drop is 68, not 80.

Instruments

instruments/verify-hashes.py and instruments/run-oracle.sh are in this directory rather than inline, so the oracle's criterion is auditable and so the capture command stays one line — capture.sh records commands with printf '%s\n' "$*", and a multi-line command splits one manifest record across several lines and desynchronises its artifact counter. That happened on the first pass of this cycle. The whole directory was deleted and every capture re-run from scratch rather than repairing MANIFEST.tsv by hand, because editing a manifest to match its artifacts inverts what the manifest is for. The final manifest is 33 records, 9 fields each, and verify-manifest.sh reports 0 altered and 0 missing.

0029 is a real failed run, retained: scripts/verify.sh did not exist at abb0ab4 (exit 127). It first appears in 6c80dee, cycle 04. P5 was therefore verified by running fixpoint, the native suite and every harness directly (0031, 0032, 0033) rather than through a gauntlet that had not been built yet.