Union of all ship-critical soul fixes for the Mac beta:
- Keeps main's honest receipts (#100/#101), immutability arc + #199 by-id
gate (#83), Track B threat routing (#76), bounded beginSession (#103),
CI hardening (#85/#86), elc typo hotfix (#77), neuron-dev-setup (#84).
- Brings WIP's genesis-boot SIGSEGV fix (#150/#95), safety-contact 988
truncation fix (#96), bounded-persona floor (#93), and 10 self-review
commits (importance flattening, curiosity DF gating, WM/heartbeat
observability, boot-counter telemetry).
- Folds the multi-word ranked-search fix: api_query_param now url_decode()s
the extracted value so q=foo%20bar / foo+bar tokenize as two words.
Conflicts (neuron-api.el payload-bound comments, mcp-wrapper tool_forget)
resolved toward the correct end state: main's verified honest-receipt
read-back is kept; WIP's improved forget description is kept. Generated
dist/*.c taken from main and will be regenerated from this reconciled
source in the following commit.
Port the payload-bounding fix (PR #102, commit 872120c) onto main. The
session-init endpoints projected unbounded engram nodes (~900KB), closing
the MCP client socket on every beginSession. Cap the lists (8 activated /
10 recent for begin_session, 10/20 for compile_ctx) and project each node
to a light identity + a bounded, UTF-8-safe content snippet via
api_compact_node / api_compact_activated / api_utf8_trunc. Response drops
~900KB -> ~12KB; full content stays available via recall/fetch/inspectGraph.
Regenerate dist/soul.c (the CI-built amalgamation) and dist/neuron-api.c
from source. The soul.c regen also compiles in already-merged source the
previously-committed soul.c was stale against (agent write/edit receipt
fixes, #100/#101); verified via scripts/verify-soul-contract.sh
(PRESENCE + IMMUTABILITY PASS) and a clean CI-style cc build.
Four sites passed the Float local 'sal' through el_from_float() a second
time. el_val_t is the bit-pattern of the double, so re-wrapping performs
an int64->double VALUE conversion of the bits before re-bitcasting —
garbage that fails engram_decode_score's range check and clamps to
defaults. Net effect: importance="critical" stored 0.5/0.5 — importance
levels were cosmetic on the MCP memory path. Verified fixed live:
critical now stores salience/importance 0.95/0.95. Same bug fixed today
in engram server.el route_create_node (foundation/el 7f03876). Literal
wraps (el_from_float(0.9)) are safe — elc passes numeric literals raw.
The session-init endpoints concatenated unbounded engram activate/scan
results as FULL node objects (content up to ~90KB per node), producing a
~900KB response. After the MCP wrapper re-escapes that into a stringified
text block the client dropped the socket ('connection closed unexpectedly')
on every beginSession call. Cap each list (8-10 activated, 10-20 recent)
and project every node to a light identity plus a bounded, UTF-8-safe
content snippet. Response drops from ~900KB to ~12KB; full content stays
available on demand via recall/fetch/inspectGraph.
- awareness.el: curiosity auto_term was the raw first word of a WM
label with no term-quality scoring — observed seeds included What,
Colon, Prose, Context. Replaced the 7-word genre blocklist whack-a-
mole with a delimited stopword membership test (function words +
document-structure words); topical terms pass untouched. Verified
live: seeds now ReasonEdit, Reasoning-model, Self-review.
- routes.el + awareness.el: idle counter only reset on rare inbox
synthesis-requests, so idle==pulse always (zero information).
handle_request now stamps soul.last_activity_ts on every inbound
HTTP request; heartbeat emits idle_ms = ms since last request
(-1 until first request of a boot).
- neuron-api.el: beginSession concatenated a depth-2 spread plus the
unbounded self-hub neighbor dump — multi-MB response, doubled by
wrapper re-escaping, socket died on every call. Now depth-1 and
the hub dump dropped (identity loading has its own tool). Verified:
beginSession returns instead of closing the socket.
Sentinel labels (knowledge:captured/evolved/canonical) made every capture
anonymous in WM telemetry — 35 identical wm_top entries — and starved the
curiosity auto-term seeder, which derives scan seeds from WM top-10 labels
and had returned empty on every scan since boot 6 because WM became
Knowledge-dominated while Knowledge was excluded from seeding.
- capture/evolve/promote now pass title (or empty → engram_node_full's
content[:60] derivation) instead of sentinels
- auto_term_try_slot admits Knowledge slots; sentinel-shaped labels
(colon, no space) are skipped so legacy nodes cannot seed 'knowledge'
- verified: probe capture labeled 'Label derivation probe 2026-07-23'
A fresh-install (SOUL_CGI_ID=ntn-genesis) boot crashed with
"Segmentation fault: 11" right after the http server came up — a real
customer's very first boot. Backtrace:
strcmp(0x1) <- str_eq (el_runtime.c:219) <- mem_save <- awareness_run
Root cause: the el runtime's engram_save returns an Int (1 = ok, 0 =
failure), but mem_save did `str_eq(engram_save(path), "")`, treating the
return as a String. str_eq runs EL_CSTR on it, which is a raw cast:
EL_CSTR(1) = (char*)0x1. On a SUCCESSFUL save (return 1) strcmp then
dereferences 0x1 and segfaults. Genesis is the first path that both seeds
the brain AND saves it successfully on the very first awareness pass, so it
crashes there; non-genesis boots (contract gate, refusal test) don't hit a
successful early mem_save, which is why they passed. handle_api_consolidate
had the identical latent bug.
Fix: read engram_save's Int result and compare `== 0` instead of str_eq'ing
it — in mem_save (memory.el) and handle_api_consolidate (neuron-api.el).
Regression: pre-existing, NOT introduced by the immutability/floor rebuild.
The pre-immutability build (1442ce2) genesis-crashes identically in the same
unchanged mem_save; #159 never actually fixed#150 for a release-runtime
build.
Regenerated dist/soul.c + per-module dist/{memory,neuron-api}.c (3GB RSS
watchdog, built against release el_runtime v1.0.0-20260501). Verified:
genesis boot survives (/health 200, no segfault), verify-soul-contract.sh
GATE PASS (PRESENCE + IMMUTABILITY), and the bounded-persona floor is still
compiled in (BOUNDED PERSONA / SOUL_PERSONA_NAME strings present).
The ship-soul builds from this branch, which has the bounded-persona floor
(#93) but never received the tombstone/supersede immutability fix (that
went to main; hotfix diverged before it). So the launch soul failed
verify-soul-contract IMMUTABILITY on the delete/update/forget routes —
they hard-removed engram nodes via engram_forget/mem_forget.
Apply the same fix, mirroring the knowledge routes' supersede pattern:
- node/update -> create new node + "supersedes" edge to the original, KEEP
the original (no engram_forget).
- node/delete, memory/delete, memory/forget, cultivate forget, and the
autonomous awareness forget -> TOMBSTONE via the canonical mem_tombstone
(memory.el): keep the node + its edges, record a Tombstone marker, hide
from default bounded list reads (?include_deleted recovers). Never
engram_forget. The MCP forget tool now routes to the tombstoning delete
instead of faking a delete.
Internal GC that genuinely removes transient nodes (awareness inbox-trigger
consume, consolidation dedup, session-summary replace, telemetry pruning)
still calls engram_forget directly and is unchanged.
Regenerated dist/soul.c (single-TU) + per-module dist/{memory,awareness,
neuron-api}.c from THIS branch's sources under a 3GB physical-RSS watchdog
(peak ~32MB), built against the release el_runtime (v1.0.0-20260501). The
bounded-persona floor is preserved — verified in the emitted C and the
linked binary (BOUNDED PERSONA / SOUL_PERSONA_NAME strings present).
verify-soul-contract.sh: GATE PASS — PRESENCE all 27 routes, IMMUTABILITY
5/5 KEPT (memory-update, memory-delete, node-update, node-delete,
memory-forget).
Phase 1 tombstoned memory/delete + node/delete, but the generic forget
path was still destructive. This routes every forget through the same
tombstone semantics so nothing in the daemon can hard-delete an engram
node anymore.
Canonical helper: mem_tombstone (memory.el, imported first so every module
can call it) — keep the node + its edges, record a Tombstone marker, never
engram_forget. neuron-api's tombstone_node now delegates to it (single
source of truth).
Per-path before -> after:
- memory.el `mem_forget` hard delete (engram_forget) -> tombstone. This
alone fixes both callers: the /api/neuron/memory/forget route
(handle_api_forget) and the cultivate op=="forget".
- awareness.el autonomous `forget` action engram_forget -> mem_tombstone.
The soul can no longer autonomously hard-delete a memory.
- mcp-wrapper tool_forget was a FAKE no-op that returned {"ok","deleted"}
without deleting OR tombstoning -> now routes to the soul's tombstoning
/api/neuron/memory/delete; tool description fixed to say it
supersedes/tombstones (recoverable), not "Remove a node".
Left intentionally as hard deletes (internal GC / lifecycle, not user
memory, all call engram_forget directly): session-summary replace
(chat.el), mem_consolidate dedup (memory.el), session-start telemetry
pruning (soul.el), session lifecycle (sessions.el).
Regenerated both ship paths under a 3GB physical-RSS watchdog (peak ~32MB):
dist/soul.c (single-TU amalgamation, macOS/Linux) and the per-module
dist/{memory,awareness,neuron-api}.c (Windows/Linux build). Verified: no
forget handler calls engram_forget; a forget leaves the node present +
tombstone marker. Gate (neuron-ui verify-soul-contract.sh, new memory-forget
row): PRESENCE + IMMUTABILITY PASS.
The soul was hard-deleting engram nodes: memory/delete and node/delete
called engram_forget (frees the node and drops its incident edges), and
node/update created a replacement then forgot the original with no link
back. That violates the day-one rule that engram nodes are immutable —
memory could be silently, irrecoverably destroyed via the API.
Convert the three destructive handlers to Will's immutability semantics,
mirroring the pattern memory/update and knowledge evolve/promote already
use:
- node/update -> create the new node, wire a "supersedes" edge new->old,
KEEP the original. No engram_forget. (Was: create + forget old, no edge.)
- memory/delete and node/delete -> TOMBSTONE: keep the node AND its edges,
create a Tombstone marker node (content = target id, label
"tombstone:<id>") wired with a "tombstones" edge. Never engram_forget.
Default bounded list reads (handle_api_list_typed / the memory list) hide
tombstoned nodes and the markers; ?include_deleted=1 returns them, and
internal cognition + /api/graph/nodes still traverse them. memory/update
was already correct and is unchanged.
Full-graph hiding on /api/graph/nodes is deliberately NOT done at the el
layer: json_array_get is O(index), so filtering that endpoint (called with
limit up to 999999) would be O(n^2). That hide needs a runtime scan filter
and is a separate follow-up; nodes there remain traversable, tagged
status:deleted via the marker edge.
Regenerated dist/soul.c from these sources (flat single-TU amalgamation,
compiled under a 3GB physical-RSS watchdog, peak ~32MB). Verified with
scripts/verify-soul-contract.sh in neuron-ui: PRESENCE passes (all 27
routes) and IMMUTABILITY passes (all four mutation routes KEPT; deletes
produce a real tombstone marker and hide from the default list).
Fix 1: mem_boot_count_inc prunes all existing soul:boot_count nodes before
inserting the new one — keeps exactly one boot counter node instead
of accumulating a new node per boot. Also fixes a latent ordering
bug where engram_search_json oldest-first results caused the counter
to read stale (low) values once >3 copies accumulated.
Fix 3: handle_api_node_delete comment clarified — the no-verify exception
is correct for deletes (not a write path); read-back-verify is for
writes only.
Fix 4: emit_session_start_event prunes old session-start InternalStateEvent
nodes after each boot, keeping the 10 most recent and forgetting
older ones. Prevents unbounded accumulation of ~120+ copies.
Every engram_node_full call that dropped its return value now binds it
and emits a println on empty string. engram_save calls in consolidate,
heartbeat, and dharma-room-turn are checked for failure. The two API
handlers (log_state_event, tune_config) that skipped api_persisted()
now match the read-back-after-write contract used everywhere else in
neuron-api.el.
Files changed:
- chat.el: conv_history_persist, handle_dharma_room_turn, auto_persist
- soul.el: emit_session_start_event, seed_persona_from_env HTTP check
- memory.el: mem_save, mem_boot_count_inc
- neuron-api.el: handle_api_log_state_event, handle_api_tune_config,
handle_api_consolidate (engram_save + session summary write)
- awareness.el: ise_post local-engram fallback path
TODO comments added for non-atomic patterns (issues #12, #13) and
the missing circuit breaker (#14) — these require new primitives.
fix(soul): add HTTP-engram guard to safe_to_seed — when ENGRAM_URL is set
the HTTP Engram owns persistence; genesis must never save to local snapshot
regardless of node counts (was: guard_disk forced to empty string, making
the ratio check vacuously true and allowing init_soul_edges+engram_save).
fix(soul): use multiplication form for ratio guard — node_count * 16000 <
disk_len avoids floor-division truncation that underestimated boundary files
(250KB / 16000 = 15.6, floors to 15; a 15-node graph wrongly passed old guard).
fix(chat): add safety_augment_system to handle_chat_as_soul,
handle_dharma_room_turn, and handle_dharma_room_turn_agentic — all three
called the LLM without Hard Bell evaluation, leaving users in dharma rooms
without crisis resource routing.
fix(neuron-api): add api_persisted read-back to handle_api_define_process —
was the only write handler that returned ok:true without verifying the node
was actually written to engram.
fix(routes): unique temp file path in connectd_post — replaces fixed
/tmp/neuron-connectors-req.json with a timestamped path to prevent
collision if concurrency is added or two soul instances share a machine.
test: add tests/test_bell_safety.el — covers safety_detect_bell_level
(none/soft/hard), safety_classify_hard_bell (abuse/self_harm routing),
safety_normalize (smart-quote), safety_augment_system, and
handle_safety_contact_post (validation + read-back).
test: add tests/test_soul_guard.el — pure-function logic tests for the
safe_to_seed predicate: 200KB boundary, 47MB/63-node clobber scenario,
HTTP-engram mode, multiplication vs division truncation at 250KB.
test: add tests/test_api_define_process.el — verifies the define_process
write is read-back verified after the fix.
The UI needs full memory CRUD; the soul had create (handle_api_remember)
and the older /memory/forget + /memory/evolve, but no endpoints matching
the UI's delete/update contract.
POST /api/neuron/memory/delete {"id"}
Hard delete. engram_forget is a true delete primitive (removes the node
and all incident edges from the engram store), so no soft-delete
fallback is needed. Unlike /memory/forget, this checks the node exists
first - engram_forget silently no-ops on unknown ids, and a bad id must
return an error, not fake success. Protected identity/values nodes are
blocked, same as the other accumulation-path handlers.
POST /api/neuron/memory/update {"id","content"}
Evolve-style update. The engram runtime has no in-place node mutation
primitive (only node-create, strengthen, forget, connect), so update
creates a new Memory node and wires a supersedes edge to the prior one,
same pattern as handle_api_evolve_knowledge. Unlike /memory/evolve, id
is required and must reference an existing node; create+link delegates
to handle_api_evolve_memory. Returns {id, supersedes, ok}.
Both files syntax-checked with elc --target=c (exit 0, no stderr).
Compile-verified only - local builds cannot run the soul; needs Will's
build for runtime verification.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Block evolve_knowledge, evolve_memory, forget, and link_entities (to_id
direction) from modifying the 15 hardcoded identity and values node IDs.
Returns HTTP 403 with a hint to use the cultivation path instead.
Add POST /api/neuron/cultivate — the bypass endpoint for intentional
cultivation sessions. Accepts { "operation": "...", ...args } and performs
the same operations without the protection check.
Add handle_api_forget and handle_api_evolve_memory as new protected-by-
default handlers, routed at /api/neuron/memory/forget and
/api/neuron/memory/evolve respectively.
Tested: 10 verification cases — 403 on all blocked targets, 200 on
non-protected nodes and FROM-direction links, cultivate bypass confirmed.
neuron-api.el is a new first-class El module that implements all Neuron
cognitive API handlers natively — no HTTP round-trips, no MCP wrapper,
direct engram builtin calls. All capabilities that previously lived in
the MCP wrapper adapter now live here in the soul.
Handlers: begin_session, compile_ctx, remember, recall, search_knowledge,
browse_knowledge, capture_knowledge, evolve_knowledge, promote_knowledge,
browse_processes, define_process, log_state_event, list_state_events,
inspect_config, tune_config, inspect_graph, link_entities, list_typed,
consolidate.
Routes wired in routes.el under /api/neuron/* (GET + POST).
Also compiles all loop-1/loop-2 .el source changes into dist/*.c and
rebuilds the binary. memory.elh and neuron-api.elh updated with new exports.
will.anderson
Tim Lingo