Phase 1 tombstoned memory/delete + node/delete, but the generic forget
path was still destructive. This routes every forget through the same
tombstone semantics so nothing in the daemon can hard-delete an engram
node anymore.
Canonical helper: mem_tombstone (memory.el, imported first so every module
can call it) — keep the node + its edges, record a Tombstone marker, never
engram_forget. neuron-api's tombstone_node now delegates to it (single
source of truth).
Per-path before -> after:
- memory.el `mem_forget` hard delete (engram_forget) -> tombstone. This
alone fixes both callers: the /api/neuron/memory/forget route
(handle_api_forget) and the cultivate op=="forget".
- awareness.el autonomous `forget` action engram_forget -> mem_tombstone.
The soul can no longer autonomously hard-delete a memory.
- mcp-wrapper tool_forget was a FAKE no-op that returned {"ok","deleted"}
without deleting OR tombstoning -> now routes to the soul's tombstoning
/api/neuron/memory/delete; tool description fixed to say it
supersedes/tombstones (recoverable), not "Remove a node".
Left intentionally as hard deletes (internal GC / lifecycle, not user
memory, all call engram_forget directly): session-summary replace
(chat.el), mem_consolidate dedup (memory.el), session-start telemetry
pruning (soul.el), session lifecycle (sessions.el).
Regenerated both ship paths under a 3GB physical-RSS watchdog (peak ~32MB):
dist/soul.c (single-TU amalgamation, macOS/Linux) and the per-module
dist/{memory,awareness,neuron-api}.c (Windows/Linux build). Verified: no
forget handler calls engram_forget; a forget leaves the node present +
tombstone marker. Gate (neuron-ui verify-soul-contract.sh, new memory-forget
row): PRESENCE + IMMUTABILITY PASS.
The soul was hard-deleting engram nodes: memory/delete and node/delete
called engram_forget (frees the node and drops its incident edges), and
node/update created a replacement then forgot the original with no link
back. That violates the day-one rule that engram nodes are immutable —
memory could be silently, irrecoverably destroyed via the API.
Convert the three destructive handlers to Will's immutability semantics,
mirroring the pattern memory/update and knowledge evolve/promote already
use:
- node/update -> create the new node, wire a "supersedes" edge new->old,
KEEP the original. No engram_forget. (Was: create + forget old, no edge.)
- memory/delete and node/delete -> TOMBSTONE: keep the node AND its edges,
create a Tombstone marker node (content = target id, label
"tombstone:<id>") wired with a "tombstones" edge. Never engram_forget.
Default bounded list reads (handle_api_list_typed / the memory list) hide
tombstoned nodes and the markers; ?include_deleted=1 returns them, and
internal cognition + /api/graph/nodes still traverse them. memory/update
was already correct and is unchanged.
Full-graph hiding on /api/graph/nodes is deliberately NOT done at the el
layer: json_array_get is O(index), so filtering that endpoint (called with
limit up to 999999) would be O(n^2). That hide needs a runtime scan filter
and is a separate follow-up; nodes there remain traversable, tagged
status:deleted via the marker edge.
Regenerated dist/soul.c from these sources (flat single-TU amalgamation,
compiled under a 3GB physical-RSS watchdog, peak ~32MB). Verified with
scripts/verify-soul-contract.sh in neuron-ui: PRESENCE passes (all 27
routes) and IMMUTABILITY passes (all four mutation routes KEPT; deletes
produce a real tombstone marker and hide from the default list).
Regenerates the combined dist/soul.c and per-module dist/*.c from the current
El sources, on top of the elc-source-typo fixes (PR #77) and the Track B
threat-to-others routing (PR #76), both already on this branch.
Validated end to end under a physical-RSS watchdog (macOS silently ignores
ulimit -v / RLIMIT_AS, so every elc/elb run was RSS-polled and kill -9'd at a
3GB ceiling, one module at a time):
- OOM is GONE. The stale dist/soul-with-nlg.el (which still carries the
malformed string literals) explodes to 3.3GB+ and is watchdog-killed at ~90%.
With the typos fixed, every one of the 48 modules compiles at <=18MB peak RSS,
and the full flat amalgamation compiles as a single translation unit at ~68MB.
The 700GB pathology was purely the unbounded-parser-on-malformed-literal loop;
no malformed construct means no loop.
- The regenerated soul.c contains Track B: safety_classify_hard_bell ->
threat_other -> safety_hard_directive routes credible threat-to-others to 911
and explicitly NOT to 988 / the safety contact. Verified in source, in the
emitted C, and in the linked binary's strings. Track A (abuse / self_harm)
is unchanged and still checked first.
- The regenerated soul links to a working native arm64 binary and boots: serves
on a throwaway port, /health returns 200, awareness loop runs.
Also fixes one source blocker discovered during regen (unrelated to the typos
or Track B): chat.el handle_chat_agentic left a void `if { println(...) }` in
value position, which the current elc lowers to `_if_result = (println(...))`
(assigning void) -> invalid C. Bound an explicit Bool so the branch is
non-void; behavior unchanged (still only logs on persist failure).
NOTE (runtime dependency, for controlled deploy): this branch's chat.el calls
engram_get_node_by_label, which the canonical el-compiler/runtime does not yet
declare/define (the release runtime v1.0.0-20260501 has it; the newest runtime
has arena + http_serve_async but not this). Building the soul requires a runtime
that has all three. Land engram_get_node_by_label into the runtime package
before this soul.c can be built in CI.
Do not merge — regen + Track B going live is a controlled-deploy call.
awareness_run's while-loop ran outside any request arena, so every
allocation in every 1s tick (search JSON, heartbeat payloads, curiosity
activations) was treated as permanent by the runtime — 7.5GB RSS in
under a minute. Bracket each iteration with el_arena_push/el_arena_pop
(same pattern the compiler emits for scoped blocks; state_set/state_get
persist separately via el_strdup_persist and are unaffected).
dist/soul.c carries the same change hand-patched at the compiled
awareness_run site — elc is currently unsafe to run locally (pathological
memory on sessions.el), so the generated C was patched to match the
source, verified line-for-line against the compiler's own conventions.
MUST be paired with el repo PR #64 (el_strdup_persist for stored engram
fields): per-tick arena reclamation widens the write-corruption window
without it. Verified together: 5h live soak on the recovered production
snapshot, flat RSS, write-field-integrity clean.
Note: dist/soul.c still needs a full elc regen to pick up PR #73's
source changes (consent tiers) — tracked separately; this patch does not
regress that (those changes were never in dist).
Fix 1: mem_boot_count_inc prunes all existing soul:boot_count nodes before
inserting the new one — keeps exactly one boot counter node instead
of accumulating a new node per boot. Also fixes a latent ordering
bug where engram_search_json oldest-first results caused the counter
to read stale (low) values once >3 copies accumulated.
Fix 3: handle_api_node_delete comment clarified — the no-verify exception
is correct for deletes (not a write path); read-back-verify is for
writes only.
Fix 4: emit_session_start_event prunes old session-start InternalStateEvent
nodes after each boot, keeping the 10 most recent and forgetting
older ones. Prevents unbounded accumulation of ~120+ copies.
- PR #60: inject operator home dir into system prompt (#30)
Adds OPERATOR IDENTITY section so the LLM correctly resolves
'my files/notes/desktop' to the actual running user's $HOME.
Prevents identity confusion between imprint author and operator.
- PR #61: plan-mode endpoint POST /api/chat {mode:'plan'} (#27)
Adds handle_chat_plan — returns {steps:[{id,title,detail}]} JSON.
Wired into all three /api/chat route handlers. Grounds the plan
via engram_compile (same as agentic path) for context awareness.
dist changes:
- soul.c: both PRs compiled in; build_system_prompt updated to
2-param signature (ctx, chat_mode); handle_chat_plan added
- chat.c/routes.c/chat.elh: individual module outputs updated
- elp-c-decls.h: remove stale 1-param build_system_prompt decl,
add handle_chat_plan declaration
- soul.elh.c: new soul header declarations file (from PR #60)
Compile verified: cc -O2 -DHAVE_CURL soul.c el_runtime.c -lcurl
Binary: 805K arm64, smoke test passes (port in use = expected).
REPRODUCED: in the non-agentic path (Tools off / 'Just chat'), asking for
tool-work makes the model role-play tool use — it emits a fake ```json {...}```
'tool call' and says 'let me search/query/pull your sessions' while NOTHING
runs. Reads as a broken/lying app. (The agentic path is fine: verified it
calls search_memory and reports honestly.)
Root cause: build_system_prompt (handle_chat, the tool-less path) never told
the model it has no tools this turn, so it fabricated.
Fix: add a NO-TOOLS directive to the non-agentic system prompt — never emit
tool calls / JSON tool blocks / 'let me pull...' narration; answer from context
only; if a tool is truly needed, say so in one sentence and tell the user to
turn Tools on. Applied to chat.el (source) AND dist/soul.c (the curated TU the
CI compiles), so the CI-built binary carries it.
Verified the FABRICATION repro on the live local soul; could not verify the
patched binary locally (no matching el-runtime version on this machine — a
hand-link against origin/main runtime 404s on all routes). Builds correctly via
CI, which links soul.c against the pinned runtime.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Incorporates PRs #22/#23/#24:
- agentic_tools_all dedup fix (no duplicate web_search tool)
- workspace scope functions (agent_workspace_root, path_within_root, resolve_in_root)
- updated dispatch_tool with workspace confinement
- canonical-self bridge (ensure_self_canonical_bridge)
Also incorporates CI link fix from PR #26 (soul.c is self-contained, no
other dist/*.c needed). Fixes the CI build step which was compiling the
old June-16 soul.c that predated all these changes.
Genesis boot previously seeded a fresh identity and saved it over snapshot.json
whenever the in-memory graph looked empty. Replace the fixed node-count threshold
with a ratio guard: refuse to seed when the on-disk snapshot is large
(>200KB) but the loaded graph is sparse (< disk/16000 nodes).
KNOWN LIMITATION: this gates only the seed/pre-serve-save path. The deeper cause
is a non-atomic engram_save (fopen wb truncates to 0 before writing 47MB), which
creates a window where a concurrent load reads an empty file -> genesis -> and if
guard_disk is read in that same window the guard passes. The real fix is an
atomic engram_save (temp + fsync + rename) in el_runtime.c, tracked separately.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- sessions.el: new sessions module with session management and approval gate
- routes.el: wire /api/sessions routes (list, get, create, approve, tool_result)
- chat.el: thread-aware activation — short messages anchor to last reply
before engram compilation so follow-ups stay on-topic
- chat.el: agentic path tracks per-session history (session_hist_{id})
instead of shared conv_history, seeding each turn with prior context
- chat.el: add call_neuron_mcp, dispatch_tool, is_builtin_tool, next_bridge_id
agentic_loop, bridge_save, agentic_resume, handle_tool_result
- dist/soul: rebuild with all of the above
On startup, prefer the local engram snapshot if it has >50 nodes.
HTTP Engram is only used on first boot (no snapshot yet). This means
sessions, conversation history, and in-process state survive daemon
restarts.
awareness.el: sync source with compiled binary (periodic mem_save
on heartbeat was already in the binary but not in source).
Rebuilds soul.c with the new startup logic and ships updated binary.
neuron-api.el is a new first-class El module that implements all Neuron
cognitive API handlers natively — no HTTP round-trips, no MCP wrapper,
direct engram builtin calls. All capabilities that previously lived in
the MCP wrapper adapter now live here in the soul.
Handlers: begin_session, compile_ctx, remember, recall, search_knowledge,
browse_knowledge, capture_knowledge, evolve_knowledge, promote_knowledge,
browse_processes, define_process, log_state_event, list_state_events,
inspect_config, tune_config, inspect_graph, link_entities, list_typed,
consolidate.
Routes wired in routes.el under /api/neuron/* (GET + POST).
Also compiles all loop-1/loop-2 .el source changes into dist/*.c and
rebuilds the binary. memory.elh and neuron-api.elh updated with new exports.
Adds handle_dharma_room_turn_agentic to chat.el — same full tool loop
as handle_chat_agentic but reads transcript directly (not message), and
returns {response, cgi_id, tools_used} to match the dharma room shape.
Registers dharma_room_turn_agentic as a new event type in routes.el so
the studio can dispatch @neuron turns through this dedicated path.
- soul.el: SOUL_CGI_ID, SOUL_ENGRAM_PATH, SOUL_IDENTITY env vars;
state_set("soul_snapshot_path") so callers can find it; only call
init_soul_edges() when cgi_id == "ntn-genesis"
- chat.el: handle_dharma_room_turn — soul builds its own context from its
own engram, assembles system prompt, calls LLM, persists episodic memory;
also fix is_new_tool scoping bug in handle_chat_agentic (use has_tool)
- routes.el: wire dharma_room_turn event type before chat_as_soul branch
- rebuild dist/neuron: handle_dharma_room_turn now compiled in
Replace scan-by-offset fallback with engram_get_node_json calls for the
known high-salience identity nodes (family, origin). Offset-based scanning
is order-dependent and unreliable; direct ID fetch is stable regardless of
snapshot position. Ensures biographical context (Fox, Bobby, etc.) is
always in the system prompt when vector search returns nothing.
Routes a new event_type "chat_as_soul" through dharma/recv. The Studio
preassembles the system_prompt + transcript and dispatches per-speaker;
the soul-binary just performs the LLM call as the requested speaker_slug.
No engram_compile here — each soul has its own engram (88xx) and the
Studio queries it before composing the prompt.
Also: track the previously-untracked split source modules (chat, routes,
memory, awareness, studio) and add build.sh so the binary can be rebuilt
without the studio’s concat trick. elb resolves the import graph and
emits one .c per .el; we link them together with cc. dist/soul-el now
points at dist/neuron via symlink (matching the launchctl plist).
will.anderson
Tim Lingo