Archived
will
d3495476f4
El SDK Release / build-and-release (push) Failing after 13m0s
The generated C, amalgams, vendored runtime pins, and compiled binaries from the Claude Code era are removed from the worktree. The El sources survive; this tree is now source-only for the first-principles rebuild. Per Principal direction 2026-08-19.
177 lines
9.4 KiB
Markdown
177 lines
9.4 KiB
Markdown
# CLAIMS — v2 cycle 07, "the caller was already there"
|
||
|
||
Re-run 2026-08-17. Source doc: `docs/v2/experiments/cycles/07-the-caller-was-already-there.md`
|
||
(not modified; it is the historical record).
|
||
|
||
## Pinned commits
|
||
|
||
| Role | SHA | Subject |
|
||
|---|---|---|
|
||
| Code commit (system under test) | `d9d1571f2a17024323a3f7be6966bc476c2f2a78` | a construct can see its caller, with no new emission point |
|
||
| Its parent — the BASE for timing | `d403a5cf2e728ce00ecdc2d84290e471e74159bb` | v2 cycle 06: form is half bindable |
|
||
| Merge that landed it | `ffab17b` | merge experiment/seam-caller |
|
||
| Cycle 07 doc commit | `47b56a3` | v2 cycle 07: the caller was already there |
|
||
| Standing critique of this cycle | `fbb3a52` | cycle 07's central claim is backed by a grep, not a measurement |
|
||
|
||
All measurements were taken in detached worktrees at `d9d1571` and `d403a5c`,
|
||
so each `.prov` carries the system under test's SHA, not the re-run host's.
|
||
|
||
`d9d1571` adds 80 lines across three files and touches **no compiler source** —
|
||
`el_runtime.c` (+33), `el_runtime.h` (+1), `seam_caller.sh` (+46). The compiler
|
||
sources are byte-identical between BASE and STACK worktrees (`0006`,
|
||
`diff -rq … IDENTICAL`), so the timing below isolates the runtime change and
|
||
nothing else.
|
||
|
||
## Verdicts
|
||
|
||
| # | Claim as stated in the doc | Artifact | Commit | Verdict |
|
||
|---|---|---|---|---|
|
||
| A1 | P1 — exactly three emission points per fn | `0026`, `0027`, `0028` | `d9d1571` | **REPRODUCED** — and by a *better* instrument than the doc's, see below |
|
||
| A2 | P2 — no other emission point exists | `0027`, `0028`, `0029`, `0030` | `d9d1571` | **REPRODUCED** — loops, call sites and field accesses emit none |
|
||
| A3 | entry is `el_seam_run(fn, 0, 0)`, refusable | `0026`, `0028` | `d9d1571` | **REPRODUCED** verbatim |
|
||
| A4 | wrap is `el_seam_wrap(fn, thunk, env)` | `0026`, `0028` | `d9d1571` | **REPRODUCED** verbatim |
|
||
| A5 | exit is `el_seam_run(fn, 1, __r)` | `0026`, `0028` | `d9d1571` | **REPRODUCED** verbatim |
|
||
| A6 | all three keyed on `fn_name` | `0028` | `d9d1571` | **REPRODUCED** — every point carries `EL_STR("<fn>")` |
|
||
| A7 | P3 — a construct cannot see WHO called it (before the change) | `0032` | `d9d1571` | **REPRODUCED** — unbound run reports nothing |
|
||
| A8 | P4 — the call graph is static, the gap is runtime | — | — | **NOT-CAPTURED** — asserted in prose, no measurement stated |
|
||
| A9 | P5 — a seam-maintained stack gives the caller with NO new emission point | `0027`, `0033` | `d9d1571` | **REPRODUCED** |
|
||
| A10 | the result: `AUDIT save called by handle_request` | `0033` | `d9d1571` | **REPRODUCED** verbatim |
|
||
| A11 | and `AUDIT save called by migrate` | `0033` | `d9d1571` | **REPRODUCED** verbatim |
|
||
| A12 | a construct bound **after the build** sees its caller | `0032`, `0033` | `d9d1571` | **REPRODUCED** — same binary, binding supplied at run time via `EL_CONSTRUCTS` |
|
||
| A13 | P6 — maintaining the stack costs something measurable — FALSE | `0007`–`0021`, `0024` | `d9d1571` vs `d403a5c` | **REPRODUCED** — 7 interleaved rounds, means identical to 0.00 ms |
|
||
| A14 | P7 — moving it behind the early-out removes the cost — FALSE, there was no cost | `0024` | — | **NOT-REPRODUCIBLE** — the "before the early-out" variant was never committed |
|
||
| A15 | interleaved rounds base 0.25/0.25/0.24 s, stack 0.24/0.24/0.24 s | `0024`, `0025` | `d9d1571` vs `d403a5c` | **DIVERGED on absolute magnitude, REPRODUCED on the conclusion** — see below |
|
||
| A16 | "identical within noise. There was no regression." | `0024` | `d9d1571` vs `d403a5c` | **REPRODUCED**, and more strongly than the doc claims |
|
||
| A17 | the baseline had drifted from 0.37 s — the machine, not the code | `0025` | — | **REPRODUCED in mechanism** — cold/warm alone is 4.30× |
|
||
| A18 | `seam_caller.sh` — 5 assertions | `0031` | `d9d1571` | **REPRODUCED** — 5 assertions, 5 passed, exit 0 |
|
||
| A19 | defect 11 — `arity_query.sh` asserted exactly 503 signatures | `0039` | `d9d1571^` | **REPRODUCED** verbatim |
|
||
| A20 | the assertion now tests the property directly | `0040` | `47b56a3` | **REPRODUCED** — now `SIGN -gt 400` |
|
||
| A21 | "the gauntlet built in cycle 04 caught this" | — | — | **NOT-CAPTURED** — no artifact of the original failing run survives |
|
||
|
||
Counts: **17 REPRODUCED, 1 DIVERGED, 1 NOT-REPRODUCIBLE, 2 NOT-CAPTURED.**
|
||
|
||
---
|
||
|
||
## A1/A2 — measured from emitted C, because the doc's instrument does not measure the system
|
||
|
||
The cycle's headline — *three emission points, all keyed on `fn_name`* — is
|
||
asserted in `seam_caller.sh` by this line (`0034`):
|
||
|
||
```
|
||
chk "no fourth emission point was added" "3" \
|
||
"$(grep -c 'el_seam_run(EL_STR\|el_seam_wrap(EL_STR' el-compiler/src/codegen.el)"
|
||
```
|
||
|
||
That greps **compiler source text**, while the harness runs a **prebuilt
|
||
compiler** passed in as `$1`. The source it counts is never compiled by the
|
||
harness. Commit `fbb3a52` records this as a standing defect. I confirmed it
|
||
independently, in both directions, with my own artifacts:
|
||
|
||
| | grep of source | emitted-C emission points | compiler sha256 | harness |
|
||
|---|---|---|---|---|
|
||
| baseline (`0035`) | 3 | 9 | `e99058b8…` | 5/5 green |
|
||
| after appending one **comment line** (`0036`, `0037`) | **4** | **9** | **`e99058b8…` unchanged** | **FAILS, "expected 3 got 4"** |
|
||
|
||
A comment — zero semantic change, byte-identical binary, identical emitted C —
|
||
turns the cycle's central assertion red. The assertion measures a text file, not
|
||
the system.
|
||
|
||
**So I did not rely on it.** A1–A6 above are graded against the **emitted C
|
||
produced by the binary under test**, which is causally downstream of the
|
||
compiler and therefore an actual measurement. Compiling a probe containing a
|
||
loop, a call site, a map field access and four functions (`0030`) yields
|
||
(`0028`):
|
||
|
||
```
|
||
fields -> entry wrap exit
|
||
loops -> entry wrap exit
|
||
helper -> entry wrap exit
|
||
|
||
total emission points = 9
|
||
```
|
||
|
||
Three functions with bodies, three points each, nine total, every one keyed on
|
||
the function name. `main` gets none. **Loops, call sites and field accesses get
|
||
none** — which is precisely the bound the cycle claims, now established by
|
||
observing the compiler's output rather than by counting strings in its source.
|
||
|
||
This is a stronger result than the document's, obtained because the document's
|
||
own instrument was found untrustworthy. The conclusion the cycle drew was right;
|
||
the evidence it offered for it was not.
|
||
|
||
---
|
||
|
||
## A15 — the one divergence, reported loudly
|
||
|
||
The conclusion reproduces. The numbers do not.
|
||
|
||
```
|
||
doc: round 1 base 0.25 s stack 0.24 s
|
||
round 2 base 0.25 s stack 0.24 s
|
||
round 3 base 0.24 s stack 0.24 s
|
||
|
||
measured: 7 interleaved rounds, alternating, identical workload
|
||
round BASE STACK
|
||
1 51 53
|
||
2 53 52
|
||
3 52 53
|
||
4 53 53
|
||
5 52 52
|
||
6 52 52
|
||
7 53 51
|
||
|
||
BASE mean 52.29 ms min 51 max 53 spread 2 sd 0.70
|
||
STACK mean 52.29 ms min 51 max 53 spread 2 sd 0.70
|
||
STACK/BASE ratio of means = 1.0000 (+0.00%)
|
||
```
|
||
|
||
**Today's absolute figures are ~4.8× faster than the doc's** — 52 ms where the
|
||
doc reports 240–250 ms. The doc's conclusion is nonetheless confirmed, and
|
||
confirmed harder: where the doc showed a 1-round-in-3 wobble of 0.01 s, seven
|
||
interleaved rounds give means equal to two decimal places and identical spread.
|
||
|
||
The magnitude gap is explained, and the explanation is the point of the cycle
|
||
(`0025`):
|
||
|
||
```
|
||
FIRST (cold) run, BASE = 225 ms
|
||
FIRST (cold) run, STACK = 229 ms
|
||
steady-state warm mean over 14 runs = 52.29 ms
|
||
cold/warm ratio, BASE = 4.30x
|
||
```
|
||
|
||
The doc's 0.24–0.25 s figures sit in the cold-cache band; its steady-state
|
||
equivalent is 52 ms. **Cache state alone moves this workload by 4.30×, which is
|
||
larger than the 50% (0.48 → 0.72 s) "regression" that was originally reported
|
||
and then retracted.** That single number does more to explain the phantom
|
||
regression than the retraction in the doc does: the original pair of timings was
|
||
not merely noisy, it was almost certainly comparing a cold run against a warm
|
||
one.
|
||
|
||
Recorded as DIVERGED because the doc presents 0.25 s and 0.24 s as measurements
|
||
of this workload and they are not reproducible as such — they are reproducible
|
||
only as cold-start measurements. The *finding* they support — no regression —
|
||
is REPRODUCED (A13, A16).
|
||
|
||
## A14 — why NOT-REPRODUCIBLE
|
||
|
||
P7 concerns a variant in which the depth-stack maintenance sits **before** the
|
||
unbound early-out. That variant was never committed; only the post-early-out
|
||
form exists in history (`0003`). Reconstructing it would mean writing a mutant
|
||
today and grading a historical prediction against it, which is a new experiment,
|
||
not a reproduction. The runtime comment in `0003` states the 50% figure that
|
||
motivated the move — but a comment is a claim, not a measurement, and no
|
||
artifact of it survives.
|
||
|
||
## Notes on tree state
|
||
|
||
Every `.prov` records `d9d1571f2…` or `d403a5cf2…`. `0041` shows the worktree
|
||
ended with only untracked build scaffolding (`elc-stack`, `g2.c`, `work/`) and
|
||
no modified tracked file: the deliberate source mutation in `0036` was reverted
|
||
in `0038` and the revert is in the record. Both worktrees were removed after
|
||
capture.
|
||
|
||
`0023` is a failed instrument run (a malformed `awk`, exit 2) and is retained
|
||
rather than deleted. Removing a failed attempt from an evidence record is itself
|
||
a measurement defect; `0024` is the corrected run.
|