This repository has been archived on 2026-08-20. You can view files and clone it. You cannot open issues or pull requests or push a commit.
Files
el-retired/docs/v1/experiments/evidence/cycles/07-invocation-is-not-composable/CLAIMS.md
T
will d3495476f4
El SDK Release / build-and-release (push) Failing after 13m0s
kill: purge old-paradigm dist/platform binaries from tree
The generated C, amalgams, vendored runtime pins, and compiled binaries
from the Claude Code era are removed from the worktree. The El sources
survive; this tree is now source-only for the first-principles rebuild.

Per Principal direction 2026-08-19.
2026-08-19 19:46:15 -05:00

258 lines
20 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# CLAIMS — cycle 07, invocation is not composable
Source doc: `docs/v1/experiments/cycles/07-invocation-is-not-composable.md`
## Pinned commits
| | |
|---|---|
| Cycle commit (the EXPERIMENT) | `bc2f26ddfc35119e242c0a38d09e2b0023136462` |
| Committed | 2026-08-17 09:06:05 -0500 |
| Subject | EXPERIMENT: invocation control resolves at runtime |
| Parent (the "before" state the record describes) | `b40754f07b5eff3700a69681397e46949a5c4f41` — "land unconditional wrapper: exit crossings resolve at runtime", 2026-08-17 09:01:55 -0500 |
| Land commit (second record block, no numeric claims) | `c04d68f9c…` — "land runtime invocation control: only prohibition remains compiled", 2026-08-17 09:06:16 -0500 |
| Re-run method | detached worktree `/tmp/rerun-v1-07` at `bc2f26d`, tree clean |
| Artifact provenance | every artifact pinned to `bc2f26ddfc35`, tree `clean` (see `MANIFEST.tsv`) |
| Machine | Apple M4 Pro, Darwin 25.5.0 arm64, Apple clang 21.0.0, load avg ≈56 during timing |
**`c04d68f` is tree-identical to `bc2f26d`.** `git diff --stat bc2f26d c04d68f` is empty and
`codegen.el` hashes identically at both (`0015`). The land commit changes no file; it only
re-records the experiment as landed. Every measurement below therefore applies to both.
## Verdicts
| # | Claim (verbatim from the record) | Artifact | Commit | Verdict |
|---|---|---|---|---|
| 1 | "P1 wrap becomes runtime-bindable **TRUE** body x3 -> 21, never invoked -> 111" | `0016-P1-wrap-probe-reconstructed.out` (reconstructed probe), `0040-REDO-seam-caller-and-wrap-assertion-search.out` | `bc2f26d` | **REPRODUCED (behaviour) / DIVERGED (evidence)** — a reconstructed probe does observe `21` and `111`. But **no test or harness in the tree at `bc2f26d` asserts either number, and none ever exercises the wrap seam at all.** See divergence detail A. |
| 2 | "P2 codegen shrinks **TRUE** 5042 -> 4977" | `0009-codegen-linecount-parent-and-commit.out`, `0011-codegen-linecount-every-commit-in-range.out` | `bc2f26d^`, `bc2f26d` | **DIVERGED** — observed `5043 -> 4976` (67). Claimed `5042 -> 4977` (65). Neither endpoint matches; no commit in `5718943^..c04d68f` has codegen.el at 5042 or 4977 lines. Direction of the claim (it shrinks) holds. See detail B. |
| 3 | "P3 cost 5-10% from an indirect call on every fn **TRUE** 0.36s -> 0.39s, ~8%" | `0036-P3-timing-parent-vs-commit-gen3.out`, `0037-P3-timing-two-workloads-n21.out` | `bc2f26d^` vs `bc2f26d` | **DIVERGED** on the ratio; **NOT-REPRODUCIBLE** on the absolutes. Re-measured overhead is **+0.6 % to +2.3 %**, not ~8 %. No workload in the tree runs anywhere near 0.36 s. See detail C. |
| 4 | "P4 zero-param fns break on the empty struct **TRUE** — empty struct is a GNU extension, empty init is C23. Fixed with a char field." | `0017-P4-zero-param-env-struct-emitted-c.out`, `0005/0041-native-compiler-tests*.out` | `bc2f26d` | **REPRODUCED** — emitted C contains `struct __env_noargs { char __e0; };` and `struct __env_noargs __env = { 0 };`; no empty struct, no empty initialiser; the emitted C passes `-std=c11 -pedantic-errors -fsyntax-only`. The `char __e0` return is at `codegen.el:3226` at `bc2f26d` and absent at the parent. Test `zero-param-fn-emits-valid-c` passes. |
| 5 | "P5 fixpoint holds **TRUE**" | `0038-selfhost-fixpoint-gen2-gen3.out` (and `0008-REDO-…`) | `bc2f26d` | **REPRODUCED** — gen2 == gen3, identical sha256 `02c9b3e2…`, 14360 lines each. gen1 (from the committed `dist/platform/elc`) differs, which is expected because this commit changes codegen's output. See measurement defect 1. |
| 6 | "PROCESS FAILURE … The build then failed with 'undeclared identifier __thunk_noargs'" | `0039-thunk-noargs-and-seam-caller-search.out` | range `5718943^..c04d68f` | **NOT-CAPTURED** — the state described was never committed. The literal `__thunk_noargs` appears in no committed file at any commit in the range; it is a name codegen *generates* (`codegen.el:3356`). In the C emitted at `bc2f26d` the thunk is declared at line 12 and used at line 19, and the file compiles and runs clean. The failure is a real-but-transient working-tree state with no artifact. |
| 7 | "Removed: declare_wrap, decorator_wrap, cg_wrap_target, cg_wrap_construct, params_to_call_args, and the wraps_body scanner branch" | `0012-removed-symbols-parent-vs-commit.out`, `0013-params_to_call_args-orphan-comment.out`, `0045-wraps_body-scanner-branch-at-parent.out` | `bc2f26d^`, `bc2f26d` | **DIVERGED (one of six)** — five of six are gone from the whole tree. `params_to_call_args` loses its definition (parent codegen.el:3299) and its only call site (parent codegen.el:3415), but its **two-line doc comment survives at `bc2f26d` codegen.el:3269**, describing a function that no longer exists. See detail D. |
| 8 | "prohibits_outside is now the ONLY construct kind left at compile time" | `0014-construct-kinds-parent-vs-commit.out` | `bc2f26d` | **REPRODUCED, with a scope note**`prohibits_outside` is the only kind name with executable dispatch in codegen.el (`str_eq(dkind, "prohibits_outside")` at line 4326). `injects_at_entry`, `guards_at_entry`, `injects_at_exit` each survive at exactly one line, all of them `//` doc comments. `wraps_body` is gone tree-wide. Note the claim was already true of the other three at the **parent** — this commit's contribution is removing `wraps_body` only. |
| 9 | (not stated in the doc) native compiler test result at `bc2f26d` | `0005-native-compiler-tests.out`, `0041-native-compiler-tests-gen2-binary.out` | `bc2f26d` | **REPRODUCED as fact, but the doc records nothing to check it against**`100 tests, 100 passed, 0 failed, 241 assertions`. Parent: `99 tests, 99 passed, 0 failed, 239 assertions` (`0043`). The doc gives no test count for this commit; its absence is itself a gap, since the commit deletes a test (`declared-wrap-emits-closure-and-convention`) and adds two. |
| 10 | (not stated in the doc) integration harness result at `bc2f26d` | `0007`, `0042`, `0043` | `bc2f26d` | **REPRODUCED as fact**`seam_binding.sh`: `7 assertions, 7 passed, 0 failed`. At the parent the same harness reports `7 assertions, 6 passed, 0 failed` — an arithmetic bug (`$((6-FAILS))`) that this commit fixes. **The commit adds no assertion**; its only substantive harness change is appending a dead `thrice` target. |
| 11 | "a construct bound after the build decides how and whether to invoke it" (ISHIKAWA) | `0016`, `0007`/`0042` | `bc2f26d` | **REPRODUCED** — via the reconstructed probe for the wrap phase, and directly by the shipped harness for the entry/exit/refuse phases. |
| 12 | (not stated in the doc) `bc2f26d` silently fixes a harness pass-count bug | `0046-seam_binding-tally-history.out` | `bc2f26d` | **OBSERVED BUT UNCLAIMED** — see detail E. |
**Counts: 6 REPRODUCED · 4 DIVERGED · 1 NOT-CAPTURED · 1 OBSERVED-BUT-UNCLAIMED · (0 pure NOT-REPRODUCIBLE; claim 3 carries a NOT-REPRODUCIBLE component on its absolute figures).**
---
## Divergence detail A — P1's 21 and 111 have no evidence in the tree
This is the substantive finding for cycle 07.
The numbers are **behaviourally correct** — I observed `21` and `111`. What is not correct is
the implication that the commit *demonstrated* them.
What `bc2f26d` actually did to `lang/tests/integration/seam_binding.sh` (the whole diff, from
`git diff bc2f26d^ bc2f26d -- lang/tests/integration/`):
1. appended a `thrice` wrap target to `$WORK/targets.c`, and
2. changed the summary line from `$((6-FAILS))` to `$((7-FAILS))`.
Both are inert with respect to P1:
- **`thrice` is never compiled.** `targets.c` is written at harness line 29 and passed to `cc`
at line 57. The `cat >> "$WORK/targets.c"` that adds `thrice` is at **line 86** — twenty-nine
lines *after* the binary was already linked. Nothing recompiles it (`0040` §5d).
- **No `wrap`-phase binding is ever written.** `el_seam_load` at `bc2f26d` recognises three
phases (`entry`, `exit`, `wrap`); the harness writes `entry` and `exit` binding files only.
`0040` §5e: no `printf '… wrap …'` anywhere in the harness. §6: no `wrap` binding in any
`.sh`/`.txt` in the tree.
- **The strings `21` and `111` do not occur in the harness at all** (`0040` §5b). Its seven
expectations are `7`, `SEEN work/audited\n7`, `42`, `7`, `7`, `SEEN work/a\nSEEN work/b\n7`,
`14` (`0040` §5c).
- The commit's own native test comments say the N-times behaviour "lives in
`tests/integration/seam_binding.sh`" — it does not.
- `tests/integration/seam_caller.sh` **does not exist at any commit in `5718943^..c04d68f`**
(`0040` §4). It first appears at `d9d1571`, a v2-era commit, and is unrelated to this cycle.
(The parent agent's hint to look for it is answered: not here.)
So the 7→7 pass count reported by the harness at `bc2f26d` is the *same seven assertions* the
parent already ran; the commit fixed the arithmetic that had been under-reporting them as 6.
**The reconstruction (an INPUT, not evidence).** `evidence/.../probe07.sh`, output captured as
`0016`. It is modelled line-for-line on `seam_binding.sh` at `bc2f26d`: the same `prog.el`
(`fn work() -> Int { return 7 }`), the same `thrice` target *verbatim including its comment*,
the same `EL_CONSTRUCTS` file format, and the phase name `wrap` read from `el_seam_load` in
`el_runtime.c` at `bc2f26d`. The one thing not recoverable from the record is the
"never invoked" target — the record names no symbol — so `never()` (returns 111 without
calling the body) is invented to match the stated semantics. Observed:
```
== baseline, no binding (work() returns 7) == 7
== wrap binding -> thrice (expect 21) == 21
== wrap binding -> never (expect 111) == 111
```
`7 × 3 = 21` corroborates that the record's "body x3" was measured against this same
`work() -> 7` fixture. `111` is only as faithful as the invented target.
## Divergence detail B — P2's line counts
| | claimed | observed (`wc -l`, blob straight out of git) |
|---|---|---|
| parent `bc2f26d^` | 5042 | **5043** |
| commit `bc2f26d` | 4977 | **4976** |
| delta | 65 | **67** |
Both blobs end in a newline (`0009` checks the last byte), so `wc -l` is exact; there is no
trailing-newline ambiguity to absorb the difference. `git diff --numstat` for codegen.el is
`12 79` (+12 / 79 = 67), independently confirming the observed delta.
**No commit in `5718943^..c04d68f` ever has codegen.el at 5042 or 4977 lines** (`0011`,
explicit search: `NONE`). The full sequence over the range is:
```
5718943f2 4800 60737b030 4850 4f7568b07 4953 2bed8483f 5084
7d01608a9 5084 1b324a071 5152 35b07bade 5156 82e998273 5152
886626a64 5156 28d19da7f 5095 8bbb750c2 5095 24f7fb514 5095
285166c25 5043 b40754f07 5043 bc2f26ddf 4976 c04d68f9c 4976
```
Cross-checking the parent agent's note: **cycle 03** claims "5094 -> 5044" — observed
`24f7fb5 = 5095``285166c = 5043`. **Cycle 06** claims "5157 -> 5096" — observed
`35b07ba = 5156`, `28d19da = 5095`. So every one of these three records is off by one on
every endpoint, but **not in a consistent direction**: cycle 06 is +1 on both ends, while
cycles 03 and 07 are 1 on the parent and +1 on the commit. A single systematic counting
convention (e.g. counting a missing final newline) would produce a uniform offset; this does
not. The line numbers in these records were not produced by a reproducible instrument.
The qualitative claim — "codegen shrinks" — is unambiguously true, by 67 lines.
## Divergence detail C — P3's ~8 % cost
The record's absolute figures (`0.36s -> 0.39s`) are **NOT-REPRODUCIBLE**: the workload is not
named, and nothing in the tree at `bc2f26d` takes anything like 0.36 s on this machine
(`0037`, tail): `elc elc-cli.el` 0.067 s, `elc --test tests/native/test_compiler.el` 0.068 s,
`elc --test tests/native/test_lexer_scaling.el` 0.030 s. The compiled native test binary runs
in 5.9 ms. Whatever was timed, it is not identifiable from the record.
The **ratio** is the load-bearing part, so I re-measured it. Construction (`time07b.sh`, both
scripts stored beside this file):
- The cost under test is paid by code **emitted** by the new codegen, so both binaries must be
gen2 — self-emitted by their own commit's compiler. `/tmp/elc07pb` = elc self-emitted at
`bc2f26d^` (wrapper calls the body **directly**; 0 occurrences of `el_seam_wrap` in its C);
`/tmp/elc07b` = elc self-emitted at `bc2f26d` (254 occurrences of `el_seam_wrap`).
- Both are handed the **same** input, in the same CWD (`bc2f26d`'s `lang/`), so the only
difference is the calling convention compiled into the binary.
- Runs are interleaved, warmed, n = 21 per arm, two workloads.
Observed (`0037`):
| workload | parent median | commit median | ratio (median) | ratio (min) | ratio (mean) |
|---|---|---|---|---|---|
| `elc elc-cli.el` | 0.06482 s | 0.06576 s | **+1.45 %** | +2.31 % | +1.92 % |
| `elc --test tests/native/test_compiler.el` | 0.06758 s | 0.06821 s | **+0.93 %** | +0.62 % | +1.36 % |
An earlier, independent n = 11 run (`0036`) on the first workload gave **+0.64 % median /
+0.16 % min**. The overhead is real and positive in every arm of every run, but it sits at
**roughly 1 %, not ~8 %** — outside the predicted 510 % band, i.e. the prediction as scored
would be FALSE on this machine, not TRUE.
Caveats stated plainly, not to rescue the claim: this is an Apple M4 Pro under load avg ≈56;
`el_seam_wrap`'s fast path is `if (_el_seam_n == 0) return body(env);` and a wide
out-of-order core hides that better than most; and the workload the record actually used is
unknown, so I cannot rule out that some other program shows 8 %. No number here is rounded
toward the claimed value.
## Divergence detail D — `params_to_call_args` is not fully removed
Of the six things the record says were removed, five vanish from the entire tree between
`bc2f26d^` and `bc2f26d`: `declare_wrap`, `decorator_wrap`, `cg_wrap_target`,
`cg_wrap_construct`, and the `wraps_body` scanner branch (`0012`; the branch itself is shown at
`0045`, parent codegen.el:4385, together with the `declare_wrap(fname, pending_wrap)` call it
guarded).
`params_to_call_args` does not. At the parent it has three occurrences — a doc comment (3297),
the definition (3299), and one call site (3415). At `bc2f26d` the definition and the call site
are gone but the comment remains:
```
bc2f26d:lang/el-compiler/src/codegen.el:3269:// params_to_call_args — "a, b, c" from the param list, for the wrapper's call
bc2f26d:lang/el-compiler/src/codegen.el:3270:// into the body helper.
```
`git grep "^fn params_to_call_args"` at `bc2f26d` → no definition. It is an orphaned two-line
comment sitting immediately above `fn cg_fn`, documenting a function that no longer exists, and
it survives unchanged into `c04d68f`. Harmless, but the record's "Removed:" list overstates by
one item, and the residue is exactly the kind of thing the commit message's own
"the guard that would have caught it existed and I removed it" paragraph is about.
## Detail E — an undocumented fix: the harness pass tally
`bc2f26d` also fixes a bug its own record never mentions. Independently flagged by the cycle 08
agent and confirmed here (`0046`):
```
8bbb750c2 check-calls=6 echo " 6 assertions, $((6-FAILS)) passed, $FAILS failed"
24f7fb514 check-calls=6 echo " 6 assertions, $((6-FAILS)) passed, $FAILS failed"
285166c25 check-calls=7 echo " 7 assertions, $((6-FAILS)) passed, $FAILS failed" <- seventh check added, tally not bumped
b40754f07 check-calls=7 echo " 7 assertions, $((6-FAILS)) passed, $FAILS failed"
bc2f26ddf check-calls=7 echo " 7 assertions, $((7-FAILS)) passed, $FAILS failed" <- fixed here
c04d68f9c check-calls=7 echo " 7 assertions, $((7-FAILS)) passed, $FAILS failed"
```
`285166c` (cycle 03's experiment) added the seventh `check` and updated the header text to
"7 assertions" but left the arithmetic at `$((6-FAILS))`, so a fully green run printed
`7 assertions, 6 passed, 0 failed` — confirmed by running the harness at the parent (`0043`),
which prints exactly that with zero failures. `bc2f26d` changes it to `$((7-FAILS))` and the
run at this commit prints `7 assertions, 7 passed, 0 failed` (`0007`, `0042`).
Worth recording for two reasons. First, a commit doing something its record does not mention is
a gap in the record regardless of whether the change is a fix. Second, it is the *only*
substantive thing `bc2f26d` does to the harness that has any effect — the `thrice` target it
also adds is dead code (detail A) — which sharpens the point that the record's headline P1
result was never wired into the test suite. The harness self-report was under-counting a green
run for two cycles before anyone noticed, which is the same class of silent-instrument defect
the commit message's PROCESS FAILURE paragraph is about.
---
## Measurement defects in this re-run, recorded rather than removed
1. **`0006-selfhost-fixpoint.out` is defective and retained.** It ran the recipe's original
fixpoint command, which compares **gen1 vs gen2** — gen1 being the output of the committed
`dist/platform/elc`, which is stale at this commit and predates the seam entirely (0
occurrences of `el_seam_wrap`). Since `bc2f26d` changes what codegen emits, gen1 ≠ gen2 is
*expected*, and the artifact's `differ: char 12626, line 259` is not a finding. The correct
check is gen2 == gen3, captured as `0038-selfhost-fixpoint-gen2-gen3.out` (and, under a
confusing name that used 1-based generation numbering, `0008-REDO-selfhost-fixpoint-gen3-gen4.out`
— same two files compared, `/tmp/g07b.c` vs `/tmp/g07c.c`). This was a recipe defect, not a
tree defect; it is left in the manifest because deleting a failed measurement is the failure
mode this record exists to control for. Do not cite `0006`.
2. **`0039-thunk-noargs-and-seam-caller-search.out` sections 4 and 5 are defective and
retained.** The script `cd`'d into `<worktree>/lang` for section 3 and never returned, so
sections 4 and 5 ran with CWD = `lang/` and their pathspecs (`lang/tests/integration/`)
resolved to `lang/lang/…` and matched nothing. The "NONE" printed by section 4 was an
artifact of the CWD, not a result — this is precisely the cycle-01 defect repeating.
Sections 13 of `0039` ran from the repo root and are sound. The corrected re-run is
`0040-REDO-seam-caller-and-wrap-assertion-search.out`, which prints its own CWD as a guard.
Do not cite `0039` §45.
3. **Artifact index numbers are not contiguous** (0017 jumps to 0036). `capture.sh` derives the
next index from `grep -c . MANIFEST.tsv`, and several of my commands were multi-line shell
scripts passed to `bash -c`, so the command field written to the manifest contained embedded
newlines and inflated the line count. No data is affected — `verify-manifest.sh` reports
0 altered, 0 missing — but the numbering gap is an instrument artifact, not a lost artifact.
Subsequent captures put multi-line scripts in files (`probe07.sh`, `thunk07b.sh`, …, stored
alongside this file) and pass a single-line command.
4. **The parent-commit context measurement (`0043`) is stamped with the wrong SHA.** The parent
tree was materialised with `git archive bc2f26d^ | tar -x -C /tmp/el07parent` rather than a
worktree (worktree creation is owned by the parent agent for this run), and `/tmp/el07parent`
is not a git repo, so `capture.sh` was invoked from `/tmp/rerun-v1-07` and stamped `bc2f26d`.
The **subject** of that measurement is `bc2f26d^`. The script defends against a bad extraction
by hashing `codegen.el` and `test_compiler.el` against the `bc2f26d^` blobs before measuring
(both MATCH, printed in the artifact). Read `0043`'s provenance as "taken during the bc2f26d
run", not "measured at bc2f26d". The same applies to the parent arm of the timing artifacts
`0036`/`0037`.
## Inputs stored alongside the evidence
`probe07.sh`, `time07.sh`, `time07b.sh`, `thunk07.sh`, `thunk07b.sh`, `parent07.sh` are the
scripts whose captured *output* is the evidence. They are inputs, not artifacts, and are not in
`MANIFEST.tsv`. `probe07.sh` in particular is a **reconstruction** — see divergence detail A.