Saving the 988 crisis-line contact returned truncated, unparseable JSON —
cut mid-"set_at" at the file's byte length (e.g. 178 of a 218-byte
response). The contact written to disk was complete; only the HTTP response
was clipped, so a real customer's crisis-contact save came back corrupt.
Root cause is in the el runtime's response writer, not a handler buffer:
fs_read stores the file's byte count in a thread-local (_tl_fs_read_len)
for binary-safe file serving, and the response writer uses that length when
non-zero instead of strlen(body) (el_runtime.c:1409). Both safety-contact
handlers call fs_read (the POST read-back verify; the GET file read) and
then return a LONGER wrapped JSON string, so the response is capped to the
file size.
Soul-source fix (no runtime change needed):
- POST: verify persistence via fs_write's return (1 = all bytes written)
instead of an fs_read read-back — removes the fs_read, so nothing caps the
response.
- GET: fs_read is required, so reset the thread-local after it with a no-op
fs_read("") (fs_read zeroes the length before it opens a path) so the
wrapped response is sent in full.
Verified: POST (crisis-line + custom) and GET now return complete, valid
JSON (parses cleanly, full contact incl. set_at). Regenerated dist/soul.c +
dist/safety.c (3GB RSS watchdog, release el_runtime v1.0.0-20260501).
Full suite still green: verify-soul-contract GATE PASS (PRESENCE +
IMMUTABILITY), genesis boot survives (/health 200, no segfault), bounded-
persona floor still compiled in.
NOTE: the underlying runtime leak (any handler that fs_reads then returns a
longer string) is worth a proper fix in el_runtime.c (use the max of
strlen and _tl_fs_read_len) so this class can't recur.
A fresh-install (SOUL_CGI_ID=ntn-genesis) boot crashed with
"Segmentation fault: 11" right after the http server came up — a real
customer's very first boot. Backtrace:
strcmp(0x1) <- str_eq (el_runtime.c:219) <- mem_save <- awareness_run
Root cause: the el runtime's engram_save returns an Int (1 = ok, 0 =
failure), but mem_save did `str_eq(engram_save(path), "")`, treating the
return as a String. str_eq runs EL_CSTR on it, which is a raw cast:
EL_CSTR(1) = (char*)0x1. On a SUCCESSFUL save (return 1) strcmp then
dereferences 0x1 and segfaults. Genesis is the first path that both seeds
the brain AND saves it successfully on the very first awareness pass, so it
crashes there; non-genesis boots (contract gate, refusal test) don't hit a
successful early mem_save, which is why they passed. handle_api_consolidate
had the identical latent bug.
Fix: read engram_save's Int result and compare `== 0` instead of str_eq'ing
it — in mem_save (memory.el) and handle_api_consolidate (neuron-api.el).
Regression: pre-existing, NOT introduced by the immutability/floor rebuild.
The pre-immutability build (1442ce2) genesis-crashes identically in the same
unchanged mem_save; #159 never actually fixed#150 for a release-runtime
build.
Regenerated dist/soul.c + per-module dist/{memory,neuron-api}.c (3GB RSS
watchdog, built against release el_runtime v1.0.0-20260501). Verified:
genesis boot survives (/health 200, no segfault), verify-soul-contract.sh
GATE PASS (PRESENCE + IMMUTABILITY), and the bounded-persona floor is still
compiled in (BOUNDED PERSONA / SOUL_PERSONA_NAME strings present).
The ship-soul builds from this branch, which has the bounded-persona floor
(#93) but never received the tombstone/supersede immutability fix (that
went to main; hotfix diverged before it). So the launch soul failed
verify-soul-contract IMMUTABILITY on the delete/update/forget routes —
they hard-removed engram nodes via engram_forget/mem_forget.
Apply the same fix, mirroring the knowledge routes' supersede pattern:
- node/update -> create new node + "supersedes" edge to the original, KEEP
the original (no engram_forget).
- node/delete, memory/delete, memory/forget, cultivate forget, and the
autonomous awareness forget -> TOMBSTONE via the canonical mem_tombstone
(memory.el): keep the node + its edges, record a Tombstone marker, hide
from default bounded list reads (?include_deleted recovers). Never
engram_forget. The MCP forget tool now routes to the tombstoning delete
instead of faking a delete.
Internal GC that genuinely removes transient nodes (awareness inbox-trigger
consume, consolidation dedup, session-summary replace, telemetry pruning)
still calls engram_forget directly and is unchanged.
Regenerated dist/soul.c (single-TU) + per-module dist/{memory,awareness,
neuron-api}.c from THIS branch's sources under a 3GB physical-RSS watchdog
(peak ~32MB), built against the release el_runtime (v1.0.0-20260501). The
bounded-persona floor is preserved — verified in the emitted C and the
linked binary (BOUNDED PERSONA / SOUL_PERSONA_NAME strings present).
verify-soul-contract.sh: GATE PASS — PRESENCE all 27 routes, IMMUTABILITY
5/5 KEPT (memory-update, memory-delete, node-update, node-delete,
memory-forget).
Regenerates the combined dist/soul.c and per-module dist/*.c from the current
El sources, on top of the elc-source-typo fixes (PR #77) and the Track B
threat-to-others routing (PR #76), both already on this branch.
Validated end to end under a physical-RSS watchdog (macOS silently ignores
ulimit -v / RLIMIT_AS, so every elc/elb run was RSS-polled and kill -9'd at a
3GB ceiling, one module at a time):
- OOM is GONE. The stale dist/soul-with-nlg.el (which still carries the
malformed string literals) explodes to 3.3GB+ and is watchdog-killed at ~90%.
With the typos fixed, every one of the 48 modules compiles at <=18MB peak RSS,
and the full flat amalgamation compiles as a single translation unit at ~68MB.
The 700GB pathology was purely the unbounded-parser-on-malformed-literal loop;
no malformed construct means no loop.
- The regenerated soul.c contains Track B: safety_classify_hard_bell ->
threat_other -> safety_hard_directive routes credible threat-to-others to 911
and explicitly NOT to 988 / the safety contact. Verified in source, in the
emitted C, and in the linked binary's strings. Track A (abuse / self_harm)
is unchanged and still checked first.
- The regenerated soul links to a working native arm64 binary and boots: serves
on a throwaway port, /health returns 200, awareness loop runs.
Also fixes one source blocker discovered during regen (unrelated to the typos
or Track B): chat.el handle_chat_agentic left a void `if { println(...) }` in
value position, which the current elc lowers to `_if_result = (println(...))`
(assigning void) -> invalid C. Bound an explicit Bool so the branch is
non-void; behavior unchanged (still only logs on persist failure).
NOTE (runtime dependency, for controlled deploy): this branch's chat.el calls
engram_get_node_by_label, which the canonical el-compiler/runtime does not yet
declare/define (the release runtime v1.0.0-20260501 has it; the newest runtime
has arena + http_serve_async but not this). Building the soul requires a runtime
that has all three. Land engram_get_node_by_label into the runtime package
before this soul.c can be built in CI.
Do not merge — regen + Track B going live is a controlled-deploy call.
awareness_run's while-loop ran outside any request arena, so every
allocation in every 1s tick (search JSON, heartbeat payloads, curiosity
activations) was treated as permanent by the runtime — 7.5GB RSS in
under a minute. Bracket each iteration with el_arena_push/el_arena_pop
(same pattern the compiler emits for scoped blocks; state_set/state_get
persist separately via el_strdup_persist and are unaffected).
dist/soul.c carries the same change hand-patched at the compiled
awareness_run site — elc is currently unsafe to run locally (pathological
memory on sessions.el), so the generated C was patched to match the
source, verified line-for-line against the compiler's own conventions.
MUST be paired with el repo PR #64 (el_strdup_persist for stored engram
fields): per-tick arena reclamation widens the write-corruption window
without it. Verified together: 5h live soak on the recovered production
snapshot, flat RSS, write-field-integrity clean.
Note: dist/soul.c still needs a full elc regen to pick up PR #73's
source changes (consent tiers) — tracked separately; this patch does not
regress that (those changes were never in dist).
Fix 1: mem_boot_count_inc prunes all existing soul:boot_count nodes before
inserting the new one — keeps exactly one boot counter node instead
of accumulating a new node per boot. Also fixes a latent ordering
bug where engram_search_json oldest-first results caused the counter
to read stale (low) values once >3 copies accumulated.
Fix 3: handle_api_node_delete comment clarified — the no-verify exception
is correct for deletes (not a write path); read-back-verify is for
writes only.
Fix 4: emit_session_start_event prunes old session-start InternalStateEvent
nodes after each boot, keeping the 10 most recent and forgetting
older ones. Prevents unbounded accumulation of ~120+ copies.
- PR #60: inject operator home dir into system prompt (#30)
Adds OPERATOR IDENTITY section so the LLM correctly resolves
'my files/notes/desktop' to the actual running user's $HOME.
Prevents identity confusion between imprint author and operator.
- PR #61: plan-mode endpoint POST /api/chat {mode:'plan'} (#27)
Adds handle_chat_plan — returns {steps:[{id,title,detail}]} JSON.
Wired into all three /api/chat route handlers. Grounds the plan
via engram_compile (same as agentic path) for context awareness.
dist changes:
- soul.c: both PRs compiled in; build_system_prompt updated to
2-param signature (ctx, chat_mode); handle_chat_plan added
- chat.c/routes.c/chat.elh: individual module outputs updated
- elp-c-decls.h: remove stale 1-param build_system_prompt decl,
add handle_chat_plan declaration
- soul.elh.c: new soul header declarations file (from PR #60)
Compile verified: cc -O2 -DHAVE_CURL soul.c el_runtime.c -lcurl
Binary: 805K arm64, smoke test passes (port in use = expected).
REPRODUCED: in the non-agentic path (Tools off / 'Just chat'), asking for
tool-work makes the model role-play tool use — it emits a fake ```json {...}```
'tool call' and says 'let me search/query/pull your sessions' while NOTHING
runs. Reads as a broken/lying app. (The agentic path is fine: verified it
calls search_memory and reports honestly.)
Root cause: build_system_prompt (handle_chat, the tool-less path) never told
the model it has no tools this turn, so it fabricated.
Fix: add a NO-TOOLS directive to the non-agentic system prompt — never emit
tool calls / JSON tool blocks / 'let me pull...' narration; answer from context
only; if a tool is truly needed, say so in one sentence and tell the user to
turn Tools on. Applied to chat.el (source) AND dist/soul.c (the curated TU the
CI compiles), so the CI-built binary carries it.
Verified the FABRICATION repro on the live local soul; could not verify the
patched binary locally (no matching el-runtime version on this machine — a
hand-link against origin/main runtime 404s on all routes). Builds correctly via
CI, which links soul.c against the pinned runtime.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Incorporates PRs #22/#23/#24:
- agentic_tools_all dedup fix (no duplicate web_search tool)
- workspace scope functions (agent_workspace_root, path_within_root, resolve_in_root)
- updated dispatch_tool with workspace confinement
- canonical-self bridge (ensure_self_canonical_bridge)
Also incorporates CI link fix from PR #26 (soul.c is self-contained, no
other dist/*.c needed). Fixes the CI build step which was compiling the
old June-16 soul.c that predated all these changes.
Genesis boot previously seeded a fresh identity and saved it over snapshot.json
whenever the in-memory graph looked empty. Replace the fixed node-count threshold
with a ratio guard: refuse to seed when the on-disk snapshot is large
(>200KB) but the loaded graph is sparse (< disk/16000 nodes).
KNOWN LIMITATION: this gates only the seed/pre-serve-save path. The deeper cause
is a non-atomic engram_save (fopen wb truncates to 0 before writing 47MB), which
creates a window where a concurrent load reads an empty file -> genesis -> and if
guard_disk is read in that same window the guard passes. The real fix is an
atomic engram_save (temp + fsync + rename) in el_runtime.c, tracked separately.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- sessions.el: new sessions module with session management and approval gate
- routes.el: wire /api/sessions routes (list, get, create, approve, tool_result)
- chat.el: thread-aware activation — short messages anchor to last reply
before engram compilation so follow-ups stay on-topic
- chat.el: agentic path tracks per-session history (session_hist_{id})
instead of shared conv_history, seeding each turn with prior context
- chat.el: add call_neuron_mcp, dispatch_tool, is_builtin_tool, next_bridge_id
agentic_loop, bridge_save, agentic_resume, handle_tool_result
- dist/soul: rebuild with all of the above
On startup, prefer the local engram snapshot if it has >50 nodes.
HTTP Engram is only used on first boot (no snapshot yet). This means
sessions, conversation history, and in-process state survive daemon
restarts.
awareness.el: sync source with compiled binary (periodic mem_save
on heartbeat was already in the binary but not in source).
Rebuilds soul.c with the new startup logic and ships updated binary.
neuron-api.el is a new first-class El module that implements all Neuron
cognitive API handlers natively — no HTTP round-trips, no MCP wrapper,
direct engram builtin calls. All capabilities that previously lived in
the MCP wrapper adapter now live here in the soul.
Handlers: begin_session, compile_ctx, remember, recall, search_knowledge,
browse_knowledge, capture_knowledge, evolve_knowledge, promote_knowledge,
browse_processes, define_process, log_state_event, list_state_events,
inspect_config, tune_config, inspect_graph, link_entities, list_typed,
consolidate.
Routes wired in routes.el under /api/neuron/* (GET + POST).
Also compiles all loop-1/loop-2 .el source changes into dist/*.c and
rebuilds the binary. memory.elh and neuron-api.elh updated with new exports.
Adds handle_dharma_room_turn_agentic to chat.el — same full tool loop
as handle_chat_agentic but reads transcript directly (not message), and
returns {response, cgi_id, tools_used} to match the dharma room shape.
Registers dharma_room_turn_agentic as a new event type in routes.el so
the studio can dispatch @neuron turns through this dedicated path.
- soul.el: SOUL_CGI_ID, SOUL_ENGRAM_PATH, SOUL_IDENTITY env vars;
state_set("soul_snapshot_path") so callers can find it; only call
init_soul_edges() when cgi_id == "ntn-genesis"
- chat.el: handle_dharma_room_turn — soul builds its own context from its
own engram, assembles system prompt, calls LLM, persists episodic memory;
also fix is_new_tool scoping bug in handle_chat_agentic (use has_tool)
- routes.el: wire dharma_room_turn event type before chat_as_soul branch
- rebuild dist/neuron: handle_dharma_room_turn now compiled in
Replace scan-by-offset fallback with engram_get_node_json calls for the
known high-salience identity nodes (family, origin). Offset-based scanning
is order-dependent and unreliable; direct ID fetch is stable regardless of
snapshot position. Ensures biographical context (Fox, Bobby, etc.) is
always in the system prompt when vector search returns nothing.
Routes a new event_type "chat_as_soul" through dharma/recv. The Studio
preassembles the system_prompt + transcript and dispatches per-speaker;
the soul-binary just performs the LLM call as the requested speaker_slug.
No engram_compile here — each soul has its own engram (88xx) and the
Studio queries it before composing the prompt.
Also: track the previously-untracked split source modules (chat, routes,
memory, awareness, studio) and add build.sh so the binary can be rebuilt
without the studio’s concat trick. elb resolves the import graph and
emits one .c per .el; we link them together with cc. dist/soul-el now
points at dist/neuron via symlink (matching the launchctl plist).
will.anderson
Tim Lingo