el-retired/docs/v1/experiments/evidence/cycles/16-tokens-carry-no-position/CLAIMS.md

CLAIMS — cycle 16, tokens carry no position

Source doc: docs/v1/experiments/cycles/16-tokens-carry-no-position.md

Pinned commits

role	sha	subject
parent / before-state	1086ac965	record the module answer: the partition is a path, not a neighbourhood
experiment (system under test)	6c975b1d5019446c29d84d99b9b1c0ec512be8a6	thread provenance through resolve_imports
land (merge)	cb7289f06510c002b6ab7c5ca8c2bd182d8216bc	thread provenance end to end: a diagnostic can finally name a place

Verdicts

#	claim	artifact	commit	verdict
A1	tokens had no position at all — a token was a flat (kind, value) pair	0001-pre-token-is-flat-pair.out	1086ac9	REPRODUCED (exact)
A2	no diagnostic in El could name a place — every error named a symbol, never a line	0001 (tok_line absent from all five compiler sources)	1086ac9	REPRODUCED
A3	P1: 15 stride sites, encapsulated in tok_kind/tok_value	0014, 0015-P1-stride-sites-all-files-clean-tree.out	1086ac9→6c975b1	REPRODUCED (see note)
A4	P2: adding a line field is mechanical; P3: the lexer must count newlines — stride 2 → 3	0014, 0015	6c975b1	REPRODUCED
A5	P4: resolve_imports can record per-file line ranges (<file> spans <start> <end>)	0009-P5-message-names-both-files.out	6c975b1	REPRODUCED (via effect)
A6	P5: the message can then name both files	0009	6c975b1	REPRODUCED (exact shape)
A7	P6: token memory grows — 25.0 → 33.9 MB (+36%)	0005, 0006, 0007-P6-rss-five-runs-each.out, 0008-P6-growth-arithmetic.out	1086ac9/6c975b1	DIVERGED (endpoints reproduce; the percentage does not)
A8	105/105 native	0011-post-native-suite.out	6c975b1	REPRODUCED (exact)
A9	37/37 integration	0012-post-integration-harnesses.out	6c975b1	REPRODUCED (exact)
A10	fixpoint ok	0010-post-fixpoint.out	6c975b1	REPRODUCED
A11	compiler self-checks clean	0013-post-compiler-self-check.out	6c975b1	REPRODUCED
A12	defect 1: interp_tokens_append_all walks the token list with its own copy of the stride; gen1 built, gen2 emitted corrupt C	—	—	NOT-CAPTURED
A13	defect 2: tok_count in test_compiler.el carried the stride too, having escaped a compiler-scoped search	0015	1086ac9	REPRODUCED (corroborated — see note)
A14	defect 3: nested resolve_imports accumulated spans into shared state	—	—	NOT-CAPTURED
A15	defect 4: the first working version reported b.el:3 — the COMBINED line against a file with no line 3	0009 (shows the corrected behaviour only)	6c975b1	NOT-CAPTURED
A16	both compilers build	0003, 0004	1086ac9, 6c975b1	REPRODUCED

A1 / A4 — the token shape

tok_append at 1086ac9 (0001):

fn tok_append(tokens: [Any], kind: String, value: String) -> [Any] {
    let tokens = native_list_append(tokens, kind)
    native_list_append(tokens, value)
}

Two slots. tok_line does not appear anywhere in lexer.el, parser.el, codegen.el, codegen-js.el or compiler.el — count 0 in all five. At 6c975b1 a third slot is appended from state_get("__lex_line") and tok_line(tokens, pos) is defined at parser.el:41.

A3 / A13 — the stride count, and the escapee

0015, run against 1086ac9, finds 16 sites that hardcode the stride:

codegen.el             3 sites   (3633, 3807, 4018)
parser.el             12 sites   (20, 21, 31, 38, 42, 49, 1233, 2161, 2204, 2256, 2422, 2564)
tests/native/test_compiler.el   1 site (21)

Compiler sources alone: 3 + 12 = 15 — exactly P1's figure. The sixteenth is tok_count in test_compiler.el, which is precisely what defect 2 in the record says: "tok_count in test_compiler.el carried the stride too. I had scoped the search to compiler sources and it had escaped into the tests."

So P1's 15 is the compiler-source count, it is correct as such, and the independent count corroborates the recorded defect rather than contradicting the prediction. After the change parser.el exposes three accessors (tok_kind:25, tok_line:41, tok_value:48) with 187 call sites (0014).

A6 — the diagnostic, exactly as recorded

Fixtures fixtures/a.el, fixtures/b.el, fixtures/m.el. At 6c975b1 (0009, checker exit 1):

duplicate definition: 'helper' is defined 2 times — El has no namespacing, so imported modules share one global scope
    …/mod16/a.el:1
    …/mod16/b.el:1

Same shape as the record's

    /tmp/modtest/a.el:1
    /tmp/modtest/b.el:1

(the directory differs because the fixture was re-created in this re-run's scratch space; /tmp/modtest from the original session no longer exists). Both files are named, and each line number is the line within that file — which is defect 4's repair: the combined-source line is no longer reported against a file that has no such line.

A7 — the divergence, in full

Peak RSS of the same compile (elc-cli.el), five runs each, /usr/bin/time -l (0007):

pre  run1 25018368   23.86 MiB   25.02 MB        post run1 35536896   33.89 MiB   35.54 MB
pre  run2 25001984   23.84 MiB   25.00 MB        post run2 33980416   32.41 MiB   33.98 MB
pre  run3 25067520   23.91 MiB   25.07 MB        post run3 35061760   33.44 MiB   35.06 MB
pre  run4 24887296   23.73 MiB   24.89 MB        post run4 34947072   33.33 MiB   34.95 MB
pre  run5 25067520   23.91 MiB   25.07 MB        post run5 34848768   33.23 MiB   34.85 MB

Both endpoint numbers in the record reproduce — but not in the same unit.

• 25.0 is the pre figure read in decimal MB (25.00–25.07 MB; the same bytes are 23.84 MiB).
• 33.9 is the post figure read in MiB (33.89 MiB on run 1; the same bytes are 35.54 MB).

Computed in consistent units (0008):

pre mean   25008537 bytes = 23.84 MiB = 25.00 MB
post mean  34874982 bytes = 33.25 MiB = 34.87 MB
growth, consistent units = 39.45 %
claimed 25.0 -> 33.9 implies 35.60 %

So the true growth is ≈ +39.5 %, not +36 %. The +36 % figure is the arithmetic of one decimal-MB reading against one MiB reading. P6's prediction ("token memory grows") is REPRODUCED and the direction is unaffected; the magnitude as printed is DIVERGED, and understates the cost.

A note on what is being measured: /usr/bin/time -l reports process peak resident set, not the token list in isolation. The record does not state the method used for its own figures. Since both of its endpoints fall inside the peak-RSS distribution measured here, peak RSS is the most likely original instrument; but the +36 % is wrong under any single consistent unit, which is what the divergence rests on. A stride change from 2 to 3 slots implies +50 % on the token list alone, and +39.5 % on whole-process peak is consistent with the token list being the dominant but not sole allocation.

Suite results at 6c975b1

105 tests, 105 passed, 0 failed, 253 assertions       (0011)
FIXPOINT-OK gen2 == gen3                              (0010)
definitions: clean  exit=0  (compiler's own source)   (0013)

arity_query.sh          exit=0 assertions=5
capability_query.sh     exit=0 assertions=5
definitions_query.sh    exit=0 assertions=6
prohibition_query.sh    exit=0 assertions=4
seam_binding.sh         exit=0 assertions=7
temporal_query.sh       exit=0 assertions=6
temporal_signatures.sh  exit=0 assertions=4
TOTAL: 37                                             (0012)

105/105 and 37/37 — both exact.

Provenance notes

0002 carries a DIRTY-TREE flag (git show measurement run from the shared main checkout while other agents were committing to it). It was re-run from a clean worktree as 0014, and extended as 0015. Cite the -clean-tree artifacts. Everything else is from /tmp/rerun-v1b-16-pre (1086ac9) and /tmp/rerun-v1b-16-post (6c975b1).

The four defects the record lists (A12–A15) were found and fixed within the development of 6c975b1; no intermediate states were committed, so no artifact of them exists and none was manufactured. A13 is graded REPRODUCED only because the escapee it names is independently visible in the stride census at the parent commit.