docs: the builtin recipe never required a test
El SDK CI - dev / build-and-test (pull_request) Failing after 10m55s
El SDK CI - dev / build-and-test (pull_request) Failing after 10m55s
lang/AGENTS.md:71-77 gives four steps for adding a C builtin and ends at 'confirm the self-host fixpoint is byte-identical'. No step asks for a test. The only 'verify' in the file is that fixpoint, which proves the COMPILER REPRODUCES ITSELF and says nothing about whether the builtin works — so the recipe reads as complete while having checked nothing about the thing just added. Measured on 2026-08-16: engram_node_set_emb, engram_curiosity_json and dream_set_handler were all added in a single session with zero tests, by an agent following this recipe. Separately a UTF-8 fix was written and tested and THE TEST PASSED ON THE UNPATCHED BUILD — the real defect was elsewhere, and only building the pre-fix binary exposed it. Without a negative control that fix would have merged as verified. Adds step 5 with the two failure shapes actually encountered: a test that never exercises the change (a route default bypassed the code under test), and an induction that loses a race (curl --max-time left BOTH builds alive; only SO_LINGER 0, a real RST, reproduced it). Plus the port-binding check, because a stale instance answering has silently produced false results here more than once and pkill -f does not reliably match argv './engram'. Documentation only. Does not touch the (a) split-the-C / (b) close-the- compiler-gap question, which is a separate decision.
This commit is contained in:
@@ -73,6 +73,17 @@ When you add a C builtin (verbatim-emit recipe — the El name is emitted as the
|
||||
2. Add a `__`-prefixed thin wrapper in `el_seed.c` and declare it in `el_seed.h`.
|
||||
3. Add the name to `builtin_arity` in `el-compiler/src/codegen.el` — add **both** the plain and `__`-prefixed spellings.
|
||||
4. Rebuild the elc binary (see below) and confirm the self-host fixpoint is byte-identical.
|
||||
5. **Prove it with a NEGATIVE CONTROL.** Show the test FAILING on a build without your change, then passing with it. A test that has never been seen to fail has proven nothing.
|
||||
|
||||
> **Step 5 is not optional, and step 4 does not cover it.** The fixpoint proves the *compiler reproduces itself*. It says nothing whatsoever about whether your builtin works. A recipe ending at "byte-identical" reads as complete while having verified nothing about the thing just added — which is why this file, until 2026-08-16, produced builtins with no tests at all.
|
||||
>
|
||||
> Measured cost of the omission (2026-08-16): `engram_node_set_emb`, `engram_curiosity_json` and `dream_set_handler` were all added in one session with zero tests. Separately, a UTF-8 fix was written, tested, and **the test passed on the unpatched build too** — the defect was elsewhere entirely, and only building the pre-fix binary exposed it. Without a negative control that fix would have merged as verified.
|
||||
>
|
||||
> Two shapes that pass while proving nothing, both hit the same day:
|
||||
> - A test that never exercises your change (the route supplied a default that bypassed the code under test).
|
||||
> - An induction that loses a race. `curl --max-time` on a large response left *both* builds alive; only `SO_LINGER 0` — a genuine RST, so the peer is provably gone — reproduced the failure. Six of ten attempts is not a control.
|
||||
>
|
||||
> Before every probe, confirm **your** process bound the port (`lsof -nP -iTCP:<port>`, match the PID). A stale instance answering on the port has silently produced false results here more than once, and `pkill -f` does not reliably match an argv like `./engram`.
|
||||
|
||||
Worked example: the `engram_assert_json` (op_assert seam) and `engram_node_full_in`/`engram_connect_in` (purview write-side) primitives added 2026-08-15 follow exactly this recipe.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user