From fe820928b0e1b78fd80b16033223dff1df4117b9 Mon Sep 17 00:00:00 2001 From: Neuron Date: Sun, 16 Aug 2026 13:53:08 -0500 Subject: [PATCH] docs: the builtin recipe never required a test MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit lang/AGENTS.md:71-77 gives four steps for adding a C builtin and ends at 'confirm the self-host fixpoint is byte-identical'. No step asks for a test. The only 'verify' in the file is that fixpoint, which proves the COMPILER REPRODUCES ITSELF and says nothing about whether the builtin works — so the recipe reads as complete while having checked nothing about the thing just added. Measured on 2026-08-16: engram_node_set_emb, engram_curiosity_json and dream_set_handler were all added in a single session with zero tests, by an agent following this recipe. Separately a UTF-8 fix was written and tested and THE TEST PASSED ON THE UNPATCHED BUILD — the real defect was elsewhere, and only building the pre-fix binary exposed it. Without a negative control that fix would have merged as verified. Adds step 5 with the two failure shapes actually encountered: a test that never exercises the change (a route default bypassed the code under test), and an induction that loses a race (curl --max-time left BOTH builds alive; only SO_LINGER 0, a real RST, reproduced it). Plus the port-binding check, because a stale instance answering has silently produced false results here more than once and pkill -f does not reliably match argv './engram'. Documentation only. Does not touch the (a) split-the-C / (b) close-the- compiler-gap question, which is a separate decision. --- lang/AGENTS.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/lang/AGENTS.md b/lang/AGENTS.md index 7941192..24fd3fd 100644 --- a/lang/AGENTS.md +++ b/lang/AGENTS.md @@ -73,6 +73,17 @@ When you add a C builtin (verbatim-emit recipe — the El name is emitted as the 2. Add a `__`-prefixed thin wrapper in `el_seed.c` and declare it in `el_seed.h`. 3. Add the name to `builtin_arity` in `el-compiler/src/codegen.el` — add **both** the plain and `__`-prefixed spellings. 4. Rebuild the elc binary (see below) and confirm the self-host fixpoint is byte-identical. +5. **Prove it with a NEGATIVE CONTROL.** Show the test FAILING on a build without your change, then passing with it. A test that has never been seen to fail has proven nothing. + +> **Step 5 is not optional, and step 4 does not cover it.** The fixpoint proves the *compiler reproduces itself*. It says nothing whatsoever about whether your builtin works. A recipe ending at "byte-identical" reads as complete while having verified nothing about the thing just added — which is why this file, until 2026-08-16, produced builtins with no tests at all. +> +> Measured cost of the omission (2026-08-16): `engram_node_set_emb`, `engram_curiosity_json` and `dream_set_handler` were all added in one session with zero tests. Separately, a UTF-8 fix was written, tested, and **the test passed on the unpatched build too** — the defect was elsewhere entirely, and only building the pre-fix binary exposed it. Without a negative control that fix would have merged as verified. +> +> Two shapes that pass while proving nothing, both hit the same day: +> - A test that never exercises your change (the route supplied a default that bypassed the code under test). +> - An induction that loses a race. `curl --max-time` on a large response left *both* builds alive; only `SO_LINGER 0` — a genuine RST, so the peer is provably gone — reproduced the failure. Six of ten attempts is not a control. +> +> Before every probe, confirm **your** process bound the port (`lsof -nP -iTCP:`, match the PID). A stale instance answering on the port has silently produced false results here more than once, and `pkill -f` does not reliably match an argv like `./engram`. Worked example: the `engram_assert_json` (op_assert seam) and `engram_node_full_in`/`engram_connect_in` (purview write-side) primitives added 2026-08-15 follow exactly this recipe.