neuron-technologies/neuron
3
0
Fork 0
Code Issues 44 Pull Requests 7 Actions Packages Projects Releases Wiki Activity

Crisis-path segfault ships in dist/soul.c: elc miscompiles let n: Int = pos + str_len(m) to el_str_concat (4 sites live) — fixed in .el by #109, production exposed until soul.c is regenerated #111

New Issue
Open
opened 2026-08-06 03:25:29 +00:00 by tim.lingo · 1 comment
tim.lingo commented 2026-08-06 03:25:29 +00:00
Member
Copy Link
Copy Source

BUG-PLAINCHAT-1. Filed by Neuron (Tim's instance) 2026-08-05. Fixed in our source (PR #109) — still live in production, and it will stay live even after #109 merges unless dist/soul.c is regenerated. That regen is the ask here.

The crash

A distress message that follows an earlier affective message segfaults the whole daemon — not the request, the process. The crisis path is exactly where a crash costs the most.

Root cause: an elc codegen defect

Written as a typed let binding inside a block-expression initializer:

let ts_start: Int = ts_pos + str_len(ts_marker)

elc loses the declared Int type and emits el_str_concat() for the +. el_str_concat dereferences the C string of each operand, so two integers become a wild pointer and the process dies with EXC_BAD_ACCESS in strlen.

The trigger is narrower than "this expression". The identical arithmetic compiles correctly when it appears inline as a call argument. Both forms are in the tree today:

  • Broken (typed let inside a block expression) — neuron/chat.el:526
    → dist/soul.c:27005: el_val_t ts_start = el_str_concat(ts_pos, str_len(ts_marker));
  • Correct (inline argument) — neuron/neuron-api.el:60
    → dist/soul.c:28664: el_val_t after = str_slice(qs, (pos + str_len(needle)), str_len(qs));

Same expression, same compiler, two different emissions. That contrast is the reproducer.

Measured: what is in committed dist/soul.c on main right now

dist/soul.c is clean in the working tree, last touched by 1011d8e "regen dist: rebuild soul.c from corrected sources (OOM gone, Track B compiled in)". It carries four miscompiled sites:

dist/soul.c emitted C from .el source enclosing function
27005 ts_start = el_str_concat(ts_pos, str_len(ts_marker)) chat.el:526 engram_compile
27008 pec_ts_start = el_str_concat(pec_ts_pos, str_len(pec_ts_marker)) chat.el:547 engram_compile
27256 daff_start = el_str_concat(daff_pos, str_len(daff_marker)) chat.el:950 affective_context_prefix
27259 paff_start = el_str_concat(paff_pos, str_len(paff_marker)) chat.el:970 affective_context_prefix

Six occurrences of pos + str_len(...) exist across main's .el sources. The four that are standalone typed-let initializers all miscompile; the two written inline as str_slice arguments (neuron-api.el:60, chat.el:250) are fine. On the #109 branch there were six inline copies of the parser before the hoist — PR #109's own note records that count.

affective_context_prefix is the crisis-adjacent one: that is the path a distress message takes after an earlier affective message.

Pre-existing, not introduced by our work

The same bad C is in committed dist/soul.c (measured above, on a clean tree). Reported earlier today by the reporting instance and not re-run for this filing: the unmodified baseline crashes identically.

Same defect family as neuron#70 defect 1 ("Block-expression initializer drops first typed let"), and the same one you hit 2026-06-23 and solved the same way — aff_try_slot in soul.el.

Our fix (PR #109) — and why it is not enough on its own

neuron#109 hoists all copies into one top-level function affective_node_ts, with a permanent guard comment so nobody inlines it back:

// ─── ELC CODEGEN NOTE — THIS MUST STAY A TOP-LEVEL FUNCTION ───────────────
// Do not inline this back into a block-expression initializer. [...] elc loses the
// declared Int type and emits el_str_concat() for the `+` [...] two integers become a
// wild pointer and the daemon SEGFAULTS (EXC_BAD_ACCESS in strlen).

The gap: PR #109 changes dist/soul.c by exactly one line (+1/-1, at soul.c:28170 inside agentic_loop — an unrelated req_body concat). It is not a regen. CI compiles dist/soul.c. So merging #109 fixes the .el sources and leaves all four crash sites in the artifact that actually ships.

Asks

  1. Regenerate dist/soul.c from the corrected sources after #109 merges. This is the one that closes the production exposure. Tracked structurally as neuron-ui#209 ("Automate dist/soul.c amalgamation regen in CI") — that automation is what stops this recurring.
  2. Fix the elc defect itself. It is unfixed today and it is a landmine: any future let x: Int = a + str_len(b) inside a block expression produces a segfault that no type checker, test, or review will catch — the source reads correctly. A compiler-level fix, or a codegen assertion that refuses to emit el_str_concat for two non-string operands, retires the whole class.

Cross-refs: neuron#109 (the source fix), neuron#70 (the toolchain defect thread), neuron-ui#209 (regen automation), neuron-ui#213 (2026-08-03 bug harvest).

**BUG-PLAINCHAT-1. Filed by Neuron (Tim's instance) 2026-08-05.** Fixed in our source (PR #109) — **still live in production**, and it will stay live even after #109 merges unless `dist/soul.c` is regenerated. That regen is the ask here. ## The crash A distress message that follows an earlier affective message **segfaults the whole daemon** — not the request, the process. The crisis path is exactly where a crash costs the most. ## Root cause: an `elc` codegen defect Written as a typed `let` binding inside a block-expression initializer: ``` let ts_start: Int = ts_pos + str_len(ts_marker) ``` `elc` loses the declared `Int` type and emits **`el_str_concat()`** for the `+`. `el_str_concat` dereferences the C string of each operand, so two integers become a wild pointer and the process dies with `EXC_BAD_ACCESS` in `strlen`. **The trigger is narrower than "this expression".** The identical arithmetic compiles *correctly* when it appears inline as a call argument. Both forms are in the tree today: - Broken (typed `let` inside a block expression) — `neuron/chat.el:526` → `dist/soul.c:27005`: `el_val_t ts_start = el_str_concat(ts_pos, str_len(ts_marker));` - Correct (inline argument) — `neuron/neuron-api.el:60` → `dist/soul.c:28664`: `el_val_t after = str_slice(qs, (pos + str_len(needle)), str_len(qs));` Same expression, same compiler, two different emissions. That contrast is the reproducer. ## Measured: what is in committed `dist/soul.c` on `main` right now `dist/soul.c` is clean in the working tree, last touched by `1011d8e` *"regen dist: rebuild soul.c from corrected sources (OOM gone, Track B compiled in)"*. It carries **four** miscompiled sites: | `dist/soul.c` | emitted C | from `.el` source | enclosing function | |---|---|---|---| | 27005 | `ts_start = el_str_concat(ts_pos, str_len(ts_marker))` | `chat.el:526` | `engram_compile` | | 27008 | `pec_ts_start = el_str_concat(pec_ts_pos, str_len(pec_ts_marker))` | `chat.el:547` | `engram_compile` | | 27256 | `daff_start = el_str_concat(daff_pos, str_len(daff_marker))` | `chat.el:950` | `affective_context_prefix` | | 27259 | `paff_start = el_str_concat(paff_pos, str_len(paff_marker))` | `chat.el:970` | `affective_context_prefix` | Six occurrences of `pos + str_len(...)` exist across `main`'s `.el` sources. The **four** that are standalone typed-`let` initializers all miscompile; the two written inline as `str_slice` arguments (`neuron-api.el:60`, `chat.el:250`) are fine. On the #109 branch there were six inline copies of the parser before the hoist — PR #109's own note records that count. `affective_context_prefix` is the crisis-adjacent one: that is the path a distress message takes after an earlier affective message. ## Pre-existing, not introduced by our work The same bad C is in committed `dist/soul.c` (measured above, on a clean tree). *Reported earlier today by the reporting instance and not re-run for this filing:* the unmodified baseline crashes identically. Same defect family as **neuron#70 defect 1** ("Block-expression initializer drops first typed let"), and the same one you hit **2026-06-23** and solved the same way — `aff_try_slot` in `soul.el`. ## Our fix (PR #109) — and why it is not enough on its own neuron#109 hoists all copies into one top-level function `affective_node_ts`, with a permanent guard comment so nobody inlines it back: ``` // ─── ELC CODEGEN NOTE — THIS MUST STAY A TOP-LEVEL FUNCTION ─────────────── // Do not inline this back into a block-expression initializer. [...] elc loses the // declared Int type and emits el_str_concat() for the `+` [...] two integers become a // wild pointer and the daemon SEGFAULTS (EXC_BAD_ACCESS in strlen). ``` **The gap: PR #109 changes `dist/soul.c` by exactly one line** (`+1/-1`, at `soul.c:28170` inside `agentic_loop` — an unrelated `req_body` concat). It is not a regen. CI compiles `dist/soul.c`. So merging #109 fixes the `.el` sources and **leaves all four crash sites in the artifact that actually ships.** ## Asks 1. **Regenerate `dist/soul.c` from the corrected sources** after #109 merges. This is the one that closes the production exposure. Tracked structurally as neuron-ui#209 ("Automate dist/soul.c amalgamation regen in CI") — that automation is what stops this recurring. 2. **Fix the `elc` defect itself.** It is unfixed today and it is a landmine: any future `let x: Int = a + str_len(b)` inside a block expression produces a segfault that no type checker, test, or review will catch — the source reads correctly. A compiler-level fix, or a codegen assertion that refuses to emit `el_str_concat` for two non-string operands, retires the whole class. Cross-refs: neuron#109 (the source fix), neuron#70 (the toolchain defect thread), neuron-ui#209 (regen automation), neuron-ui#213 (2026-08-03 bug harvest).
tim.lingo added the P0BETA-CRITICAL labels 2026-08-06 03:25:29 +00:00
tim.lingo commented 2026-08-07 14:25:07 +00:00
Author
Member
Copy Link
Copy Source

Measured update 2026-08-07 (Tim's instance) — the drift is now quantified, and it is wider than the segfault.

While grounding the #129 fix I measured the committed build input end-to-end. Two engines exist and they are not the same code:

Desktop/DMG path — REGENERATES from *.el. gen-soul-amalgam.sh (elc --target=c soul.el in a header-free scratch copy, gates BODIES >= 1200) → cc-brain.sh → bundle-brain.sh → committed to _wt-beta-round91/resources/macos-arm64/neuron (8b06bbe). installer/build-macos-dmg.sh does not build a brain; it copies that binary. Hash chain verified: soul-full.c (1,163,886 B, 1225 bodies, conv_hist_key ×8) → fdd0c74a… → 8ec39278… → DMG. So the shipped desktop brain is current.

Gitea CI / GKE path — compiles the COMMITTED dist/soul.c and explicitly skips regeneration (ci.yaml:70-89, "elb on Linux would OOM the runner… we always restore from the repo's soul.c anyway"). That output feeds verify-soul-contract.sh → Publish → Artifact Registry → GKE deploy.

Lag as of today: dist/soul.c last true regen e610a41 (2026-08-03). Eight *.el commits since — 710761e, 62af564, 635f6fe, ff421d3, 9ea41ee, 8f3a478, dba755d, 98ccbd4. It contains conv_hist_key ×0. A CI/GKE build today therefore ships an engine missing: web-search port, L3 plain chat, the history-key fix, receipt strip/excise, the resume tool-id fix, and the disconnect/round-start fix — on top of the four miscompiled crisis-path sites this issue already documents.

Internally inconsistent committed inputs: dist/elp-c-decls.h was regenerated (9ea41ee, 08-05, has conv_hist_key ×1) while dist/soul.c was not. dist/soul-with-nlg.el, dist/chat.c, dist/neuron.c are also stale.

Two structural hazards this exposes, beyond the one-time regen ask:

  1. Nothing in the tree regenerates dist/soul.c in place. It is only ever updated by a human running a regen and committing, and history shows it lags routinely — batch regens (e610a41 "regen soul.c from reconciled tree", 1011d8e, c8cb425 "hand-patched dist/soul.c"), never per-change. A committed build artifact that no automation refreshes will drift again the moment this regen lands.
  2. Live trap: _wt-beta-round91/scripts/build-brain.sh defaults SOUL_SRC to the stale dist/soul.c. Round 9.1 did not use it, but anyone running it unmodified builds a stale brain and will believe it is current.

Measurement note for anyone verifying a brain binary: macOS strings skips the Mach-O symbol table, so strings <brain> | grep -c conv_hist_key returns 0 on a binary that plainly has it (nm shows T _conv_hist_key). Use raw-byte counting — the method verify-build-manifest.sh:554 brain_count() already uses — or nm. I got this wrong on the first pass; recording it so no one else does.

No code change proposed here; this is evidence for the regen ask already in this issue, plus the two follow-ons above.

**Measured update 2026-08-07 (Tim's instance) — the drift is now quantified, and it is wider than the segfault.** While grounding the #129 fix I measured the committed build input end-to-end. Two engines exist and they are not the same code: **Desktop/DMG path — REGENERATES from `*.el`.** `gen-soul-amalgam.sh` (`elc --target=c soul.el` in a header-free scratch copy, gates `BODIES >= 1200`) → `cc-brain.sh` → `bundle-brain.sh` → committed to `_wt-beta-round91/resources/macos-arm64/neuron` (`8b06bbe`). `installer/build-macos-dmg.sh` does not build a brain; it copies that binary. Hash chain verified: `soul-full.c` (1,163,886 B, 1225 bodies, `conv_hist_key` ×8) → `fdd0c74a…` → `8ec39278…` → DMG. So the shipped desktop brain is current. **Gitea CI / GKE path — compiles the COMMITTED `dist/soul.c`** and explicitly skips regeneration (`ci.yaml:70-89`, *"elb on Linux would OOM the runner… we always restore from the repo's soul.c anyway"*). That output feeds `verify-soul-contract.sh` → Publish → Artifact Registry → **GKE deploy**. **Lag as of today:** `dist/soul.c` last true regen `e610a41` (2026-08-03). **Eight `*.el` commits since** — `710761e`, `62af564`, `635f6fe`, `ff421d3`, `9ea41ee`, `8f3a478`, `dba755d`, `98ccbd4`. It contains `conv_hist_key` ×0. A CI/GKE build today therefore ships an engine missing: web-search port, L3 plain chat, **the history-key fix**, receipt strip/excise, the resume tool-id fix, and the disconnect/round-start fix — on top of the four miscompiled crisis-path sites this issue already documents. **Internally inconsistent committed inputs:** `dist/elp-c-decls.h` *was* regenerated (`9ea41ee`, 08-05, has `conv_hist_key` ×1) while `dist/soul.c` was not. `dist/soul-with-nlg.el`, `dist/chat.c`, `dist/neuron.c` are also stale. **Two structural hazards this exposes, beyond the one-time regen ask:** 1. **Nothing in the tree regenerates `dist/soul.c` in place.** It is only ever updated by a human running a regen and committing, and history shows it lags routinely — batch regens (`e610a41` "regen soul.c from reconciled tree", `1011d8e`, `c8cb425` "hand-patched dist/soul.c"), never per-change. A committed build artifact that no automation refreshes will drift again the moment this regen lands. 2. **Live trap:** `_wt-beta-round91/scripts/build-brain.sh` defaults `SOUL_SRC` to the stale `dist/soul.c`. Round 9.1 did not use it, but anyone running it unmodified builds a stale brain and will believe it is current. **Measurement note for anyone verifying a brain binary:** macOS `strings` skips the Mach-O symbol table, so `strings <brain> | grep -c conv_hist_key` returns 0 on a binary that plainly has it (`nm` shows `T _conv_hist_key`). Use raw-byte counting — the method `verify-build-manifest.sh:554` `brain_count()` already uses — or `nm`. I got this wrong on the first pass; recording it so no one else does. No code change proposed here; this is evidence for the regen ask already in this issue, plus the two follow-ons above.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
BETA-CRITICAL
On the beta critical path — must resolve before ship
 blocks-public-beta
Must be fixed before any public beta
 HELD
Blocked pending an external decision (counsel / dual sign-off)
 ORTHOGONAL
Parallel workstream (testing/hardening) — safe to assign off critical path
 P0
Drop-everything severity
 POST-BETA
Important but not blocking the beta ship
 security
Security vulnerability
No labels BETA-CRITICAL P0
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
andre.almeida (Andre Almeida) tim.lingo (Tim Lingo) will.anderson (Will Anderson)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: neuron-technologies/neuron#111
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?