Archived
kill: purge old-paradigm dist/platform binaries from tree
El SDK Release / build-and-release (push) Failing after 13m0s
El SDK Release / build-and-release (push) Failing after 13m0s
The generated C, amalgams, vendored runtime pins, and compiled binaries from the Claude Code era are removed from the worktree. The El sources survive; this tree is now source-only for the first-principles rebuild. Per Principal direction 2026-08-19.
This commit is contained in:
will
@@ -0,0 +1,207 @@
|
||||
# CLAIMS — ITERATION-level evidence for v2
|
||||
|
||||
Re-run 2026-08-17. This directory holds the evidence that is about **iteration 2
|
||||
as a whole** rather than about any one cycle. Per-cycle evidence and verdicts
|
||||
are in `../cycles/<NN>-<slug>/CLAIMS.md`.
|
||||
|
||||
Before this run, `docs/` held 42 markdown files and **zero data files**. It now
|
||||
holds a captured artifact for every re-runnable claim, each one recording the
|
||||
command verbatim, stdout, stderr, the exit code captured directly and never
|
||||
through a pipe, the commit SHA, the tree state, UTC, and the sha256 of both
|
||||
streams.
|
||||
|
||||
## The measurement window, pinned
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Window start | `a116710bacba679615cb84b4ea9ce98c3acaa380` |
|
||||
| Window end | `a0cc95e3d…` (see `0026-zz-head-at-end.out`) |
|
||||
| Commits in window | **5** |
|
||||
| Commits touching `lang/` | **0** |
|
||||
| Commits touching `scripts/` or `tools/evidence/capture.sh` | **0** |
|
||||
|
||||
`0022-measurement-window-lang-untouched.out` is the proof. A concurrent agent
|
||||
was landing v1 evidence documents into this repository throughout, so HEAD moved
|
||||
five times and `git status` read dirty for most of the run — but **no compiler
|
||||
source, no runtime source, and no measuring instrument changed while any of this
|
||||
was measured.** The system under test and the recorder were both stable.
|
||||
|
||||
Historical claims were measured inside detached worktrees pinned at their own
|
||||
commits, so those artifacts carry the system-under-test's SHA and a clean tree.
|
||||
`verify-manifest.sh` flags every dirty-tree artifact as provenance-weak; the
|
||||
load-bearing measurements are deliberately not among them.
|
||||
|
||||
## What was captured at iteration level
|
||||
|
||||
| Claim | Artifact | Result |
|
||||
|---|---|---|
|
||||
| the full gauntlet | `0003-gauntlet-verify-sh` | **VERIFIED, exit 0** |
|
||||
| fixpoint gen2 == gen3 | `0004-fixpoint-gen2-eq-gen3` | **OK**, by sha256 |
|
||||
| native suite | `0005-native-suite` | **105 tests, 105 passed, 0 failed, 253 assertions**, exit 0 |
|
||||
| every harness individually, own exit code | `0006`–`0018` | **13 harnesses, all exit 0** |
|
||||
| assertion tally | `0020-harness-assertion-tally` | **76 assertions across 13 harnesses** |
|
||||
| `codegen.el` total / comment / code | `0019-codegen-total-vs-comment-vs-code` | **4434 total, 3197 code, 981 comment, 256 blank** |
|
||||
| unguarded-deref ratchet | `0021-unguarded-deref-ratchet` | **11 unguarded casts, at baseline 11**, exit 0 |
|
||||
| control over the whole record | `0023-CONTROL-verify-manifest-over-all-v2-cycles` | **250 artifacts verified, 0 altered, 0 missing** |
|
||||
|
||||
### The gauntlet
|
||||
|
||||
```
|
||||
fixpoint gen2 == gen3 ok
|
||||
native suite 105 tests, 105 passed, 0 failed, 253 assertions in 7.115ms
|
||||
integration harnesses 13 green, 76 assertions
|
||||
unguarded-deref ratchet at or below baseline
|
||||
|
||||
VERIFIED
|
||||
```
|
||||
|
||||
### The fixpoint proof
|
||||
|
||||
```
|
||||
gen1 sha256 = aa340a723e7b29e585e30c07d6dc9834e072ef15a07df5e1afb363c18e168c76
|
||||
gen2 sha256 = 61210655962abd07ce395d66a9232582fd10c2a878c80f9b85fefc9081c5313e
|
||||
gen3 sha256 = 61210655962abd07ce395d66a9232582fd10c2a878c80f9b85fefc9081c5313e
|
||||
FIXPOINT gen2 == gen3 OK
|
||||
note: gen1 != gen2, as expected whenever codegen output changes
|
||||
```
|
||||
|
||||
The `gen1 != gen2` line matters: comparing gen1 to gen2 would be the wrong test,
|
||||
and it is recorded here as an observed fact rather than as a design comment.
|
||||
|
||||
### `codegen.el` composition at HEAD
|
||||
|
||||
```
|
||||
total_lines = 4434
|
||||
code = 3197 72.1%
|
||||
comment = 981 22.1%
|
||||
blank = 256
|
||||
```
|
||||
|
||||
Cycle 05 established that the metric worth quoting is code, not total. On that
|
||||
metric the compiler stands at **3197 code lines**, against the **3589** baseline
|
||||
at `45325f7` — **−392, −10.9%** across v1 and v2 together.
|
||||
|
||||
## v2's standing, with references
|
||||
|
||||
### Verdicts across all eight cycles
|
||||
|
||||
| Cycle | R | D | NR | NC | total |
|
||||
|---|---|---|---|---|---|
|
||||
| 01 the-compiler-was-never-measured | 26 | 5 | 5 | 1 | 37 |
|
||||
| 02 seven-tables-and-a-shipped-defect | 21 | 2 | 0 | 0 | 23 |
|
||||
| 03 a-gate-with-no-detector | 23 | 2 | 0 | 0 | 25 |
|
||||
| 04 verification-was-a-convention | 7 | 2 | 0 | 3 | 12 |
|
||||
| 05 the-metric-counted-comments | 12 | 1 | 7 | 0 | 20 |
|
||||
| 06 form-is-half-bindable | 12 | 0 | 0 | 1 | 13 |
|
||||
| 07 the-caller-was-already-there | 17 | 1 | 1 | 2 | 21 |
|
||||
| 08 PREDICTIONS | 15 | 0 | 0 | 7 | 22 |
|
||||
| **total** | **133** | **13** | **13** | **14** | **173** |
|
||||
|
||||
R = REPRODUCED, D = DIVERGED, NR = NOT-REPRODUCIBLE, NC = NOT-CAPTURED.
|
||||
|
||||
**133 of 173 assertions reproduced from raw data. Nothing was reconstructed.**
|
||||
|
||||
### The load-bearing results, re-verified
|
||||
|
||||
The index classifies v2 as one SUPPORTS pair plus a BOUNDS, and commit
|
||||
`9540f23` weights the whole 27-cycle record at four load-bearing results. Those
|
||||
that live in v2 were given the most care in this re-run:
|
||||
|
||||
**Cycle 06 — a new suffix form by a two-line data edit, no rebuild.** Fully
|
||||
reproduced. The compiler binary's sha256 and mtime are byte-identical before and
|
||||
after the edit and its mtime predates the edit by 37.4 s; `git diff --stat`
|
||||
reads `1 file changed, 2 insertions(+)`; the same binary then emits
|
||||
`3LL * 1209600000000000LL`; the linked program prints `3628800000000000`; and
|
||||
reverting the two lines reverts the behaviour. **12 REPRODUCED, 0 DIVERGED.**
|
||||
|
||||
**Cycle 07 — exactly three emission points, and a construct that sees its
|
||||
caller.** Reproduced, but *not* by the instrument the cycle used. Commit
|
||||
`fbb3a52` records that the cycle's central assertion is a grep of compiler
|
||||
source the harness never compiles; I confirmed that independently — appending a
|
||||
**comment** to `codegen.el` turns the assertion red while the binary's sha256
|
||||
and the emitted C are unchanged. So the bound was re-established from the
|
||||
**emitted C of the binary under test**: three functions, `entry wrap exit` each,
|
||||
nine points, every one keyed on the function name, and **zero** for loops, call
|
||||
sites and field accesses. `AUDIT save called by handle_request` /
|
||||
`AUDIT save called by migrate` reproduce verbatim.
|
||||
|
||||
**Cycle 02 — 49 builtin names moved to data.** Reproduced exactly: 51 → 100
|
||||
entries in `signatures.rel`, delta 49; six of seven tables collapse to 3–6 line
|
||||
wrappers; `duration_unit_nanos` alone keeps its 14 literals, which is P3.
|
||||
|
||||
### The divergences that matter
|
||||
|
||||
Thirteen divergences across the record. Four are substantive:
|
||||
|
||||
1. **Cycle 02, A17 — the shipped defect was worse than recorded.** The doc says
|
||||
`el_input_len` returned 0 for "roughly 60%" of string literals. Measured at
|
||||
the buggy commit against 20 literals of lengths 1–20, checked by an
|
||||
independent `hashlib` oracle: **20 of 20, a 100% failure rate.** Standalone
|
||||
alignment probes give 80% on the doc's own five literals and 90% on twenty.
|
||||
The rate is link-layout dependent and is not a stable quantity; the doc's 60%
|
||||
was a five-literal sample. That is the same error — *a sample of five is not a
|
||||
measurement of a rate* — inside the paragraph that teaches it.
|
||||
|
||||
2. **Cycle 03, D-1 — the ratchet is not a ratchet.** It compares a *total*
|
||||
against `BASELINE=11` rather than a *set of sites*, so guarding one site while
|
||||
adding a new dereference passes: it printed a brand-new unguarded deref in its
|
||||
own output while reporting `ok 11 unguarded casts, at baseline 11`. Its regex
|
||||
also misses `EL_CSTR`, leaving **452 casts invisible to the detector**. And
|
||||
`let s: String = 42` — defect #1 in that cycle's own list — **still segfaults
|
||||
at HEAD, exit 139.**
|
||||
|
||||
3. **Cycle 07, A15 — the timing numbers do not reproduce; the conclusion does.**
|
||||
Seven interleaved rounds gave BASE mean 52.29 ms and STACK mean 52.29 ms,
|
||||
ratio 1.0000, spread 2 ms each. The doc's 0.24–0.25 s are cold-cache figures:
|
||||
measured cold/warm on this workload is **4.30×**, which is larger than the
|
||||
50% "regression" that was reported and retracted.
|
||||
|
||||
4. **Cycle 04, A5 — P3 as worded is false.** `verify.sh` contains no detector
|
||||
for an exit code read through a pipe; it *avoids* the defect by construction.
|
||||
Prevention is not detection, so the cycle's 5/5 is 4/5 as worded and 5/5 in
|
||||
substance.
|
||||
|
||||
The remaining nine are off-by-one line counts (`4525` for 4524, `4662` for 4661,
|
||||
`4446` for 4445, `80` for 79, `3267` for 3266) and one misattribution in cycle 01
|
||||
(`capabilities.md` names a different table than the doc claims). None changes a
|
||||
conclusion; all are recorded because a record about miscounting must not carry
|
||||
uncorrected miscounts.
|
||||
|
||||
### What could not be reproduced, and why
|
||||
|
||||
- **13 NOT-REPRODUCIBLE.** Almost all are the emit/table/other bucket split from
|
||||
cycles 01 and 05. **The classifier that produced 57.2%, 2578 lines, 21 fns,
|
||||
42.8%, 15.4% and 27.4% was never committed**, and its criterion for
|
||||
"emission" versus "table" versus "other" was never written down. No
|
||||
replacement classifier was written and passed off as a reproduction. What could
|
||||
be checked was the arithmetic, and it holds: emit 1909 + table 506 + other 794
|
||||
= **3209**, exactly the independently measured code total at `8cb7613`.
|
||||
|
||||
- **14 NOT-CAPTURED.** Seven are cycle 08's predictions P1–P7, which are
|
||||
ungraded by construction: the mutation study was never run, and running it
|
||||
today would be a new experiment, not a reproduction. The seal is intact —
|
||||
P1–P7 are byte-identical to their form at `d936bde`, which makes this genuine
|
||||
pre-registration. The rest are prose assertions with no measurement behind
|
||||
them, including cycle 06's P3, which the document itself labels
|
||||
"NOT TESTED — an argument, not a measurement".
|
||||
|
||||
## The instrument, and one defect in it
|
||||
|
||||
`capture.sh` writes its manifest line with `printf '%s\n' "$*"`. **A command
|
||||
containing a newline therefore splits one manifest record across several lines
|
||||
and desynchronises the artifact counter.** This was hit independently on cycle 01
|
||||
and on cycle 02. In both cases the fix was the same and it was not to edit the
|
||||
manifest: the evidence directory was deleted and every capture re-run from
|
||||
scratch with the logic moved into instrument files. Editing a manifest to match
|
||||
its artifacts inverts what a manifest is for.
|
||||
|
||||
`verify-manifest.sh` globs `"$ROOT"/*/MANIFEST.tsv`, so it must be given the
|
||||
**parent** of the cycle directories. Handed a cycle directory it prints
|
||||
`0 artifacts verified, 0 altered, 0 missing` and **exits 0** — a green verdict
|
||||
having checked nothing. That is the same shape as cycle 03's subject: a gate
|
||||
nothing is forced through. The correct invocation is recorded in `0023`.
|
||||
|
||||
Failed runs are retained rather than deleted — a SIGPIPE-killed build in cycle 03,
|
||||
a malformed `awk` in cycles 03 and 07, and `scripts/verify.sh` returning exit 127
|
||||
at `abb0ab4` because it did not exist yet. Removing a failed attempt from an
|
||||
evidence record is itself a measurement defect.
|
||||
Reference in New Issue
Block a user