docs: the builtin recipe never required a test #154

2026-08-16T18:53:10Z

will.anderson commented

2026-08-16 18:53:10 +00:00

lang/AGENTS.md:71-77 gives four steps for adding a C builtin and ends at "confirm the self-host fixpoint is byte-identical." No step asks for a test.

The only verify in the entire file is that fixpoint — which proves the compiler reproduces itself and says nothing about whether your builtin works. So the recipe reads as complete while having checked nothing about the thing just added.

Measured cost

engram_node_set_emb, engram_curiosity_json, dream_set_handler — all added in one session on 2026-08-16, following this recipe, with zero tests.
A UTF-8 fix was written and tested and the test passed on the unpatched build too. The real defect was elsewhere entirely; only building the pre-fix binary exposed it. Without a negative control it would have merged as verified.

Adds step 5, with the two failure shapes actually hit

A test that never exercises the change — a route supplied a default that bypassed the code under test.
An induction that loses a race — curl --max-time on a large response left both builds alive. Only SO_LINGER 0 (a real RST, so the peer is provably gone) reproduced the failure.

Plus the port-binding check: confirm your process bound the port before every probe. A stale instance answering has silently produced false results here more than once, and pkill -f does not reliably match an argv like ./engram.

Documentation only. Does not touch the (a) split-the-C / (b) close-the-compiler-gap question — that is a separate decision.

`lang/AGENTS.md:71-77` gives four steps for adding a C builtin and ends at *"confirm the self-host fixpoint is byte-identical."* **No step asks for a test.** The only `verify` in the entire file is that fixpoint — which proves the **compiler reproduces itself** and says nothing about whether your builtin works. So the recipe reads as complete while having checked nothing about the thing just added. ### Measured cost - `engram_node_set_emb`, `engram_curiosity_json`, `dream_set_handler` — all added in one session on 2026-08-16, following this recipe, with **zero tests**. - A UTF-8 fix was written and tested and **the test passed on the unpatched build too**. The real defect was elsewhere entirely; only building the pre-fix binary exposed it. Without a negative control it would have merged as verified. ### Adds step 5, with the two failure shapes actually hit - **A test that never exercises the change** — a route supplied a default that bypassed the code under test. - **An induction that loses a race** — `curl --max-time` on a large response left *both* builds alive. Only `SO_LINGER 0` (a real RST, so the peer is provably gone) reproduced the failure. Plus the port-binding check: confirm **your** process bound the port before every probe. A stale instance answering has silently produced false results here more than once, and `pkill -f` does not reliably match an argv like `./engram`. **Documentation only.** Does not touch the (a) split-the-C / (b) close-the-compiler-gap question — that is a separate decision.

will.anderson added 1 commit 2026-08-16 18:53:11 +00:00

docs: the builtin recipe never required a test

El SDK CI - dev / build-and-test (pull_request) Failing after 10m55s

Details

fe820928b0

lang/AGENTS.md:71-77 gives four steps for adding a C builtin and ends at
'confirm the self-host fixpoint is byte-identical'. No step asks for a test.
The only 'verify' in the file is that fixpoint, which proves the COMPILER
REPRODUCES ITSELF and says nothing about whether the builtin works — so the
recipe reads as complete while having checked nothing about the thing just
added.

Measured on 2026-08-16: engram_node_set_emb, engram_curiosity_json and
dream_set_handler were all added in a single session with zero tests, by an
agent following this recipe. Separately a UTF-8 fix was written and tested
and THE TEST PASSED ON THE UNPATCHED BUILD — the real defect was elsewhere,
and only building the pre-fix binary exposed it. Without a negative control
that fix would have merged as verified.

Adds step 5 with the two failure shapes actually encountered: a test that
never exercises the change (a route default bypassed the code under test),
and an induction that loses a race (curl --max-time left BOTH builds alive;
only SO_LINGER 0, a real RST, reproduced it). Plus the port-binding check,
because a stale instance answering has silently produced false results here
more than once and pkill -f does not reliably match argv './engram'.

Documentation only. Does not touch the (a) split-the-C / (b) close-the-
compiler-gap question, which is a separate decision.

will.anderson merged commit 4a57b4faa8 into dev

2026-08-16 20:49:22 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: neuron-technologies/el#154