Non-agentic /api/chat (the desktop app's default "Tools: Off" mode) returned the
user's own screened text as a bare non-JSON string. Every JSON client failed to
parse it and showed "Couldn't reach Neuron - it may be offline."
Root cause: f52d5bd (2026-06-11) correctly moved the route onto the layer spine
(handle_chat -> layered_cycle), but L3 never got a generator — imprint_respond()
annotates its input and returns it. Two pieces of the architecture were already
waiting for that step: layered_cycle parks a bell directive in the state key
build_system_prompt is written to consume, and build_system_prompt carries a
chat_mode ("no tools") flag with no live caller.
The fix composes rather than replaces. Wiring handle_chat would have removed
safety_screen, the hard-bell short-circuit, the whole stewardship layer and
safety_validate — the only enforcing output gate in the codebase — in exchange
for a working reply (see _engine-websearch-20260804/SAFETY-STOP.md). Instead
layered_cycle keeps every gate, in order, and gains a generation step between
imprint_respond and safety_validate.
L1 screen -> guard -> hard-bell short-circuit -> L2a -> L2b -> L2c
-> L3 imprint_respond (prompt) -> L3b layered_generate (NEW) -> L1 validate
- chat.el: NEW layered_generate (L3 generation, no tools offered),
conv_history_block, conv_history_record.
FIX build_system_prompt never concatenated no_tools_rule into its
return — the "[NO TOOLS THIS TURN]" rule reached no model at all.
handle_chat annotated DO-NOT-WIRE with the reason.
- soul.el: layered_cycle gains L3b + post-validation turn bookkeeping.
- routes.el: NEW plain_chat_envelope; all three /api/chat dispatch sites wrap the
cycle's output. Built OUTSIDE the cycle so safety_validate always sees
raw model text — nothing to unwrap or rebuild on the crisis path.
Emits both `reply` and `response`: the desktop app reads `reply`,
the CLI tools and telegram-gateway read `response`.
Also fixes BUG-PLAINCHAT-1, a pre-existing CRITICAL crash on the crisis path.
elc compiles `let n: Int = pos + str_len(marker)` to el_str_concat() — string
concat on two integers — inside a block-expression initializer, segfaulting the
daemon (SIGSEGV in strlen). Six inline copies of the same " | ts:" parser had it:
two in layered_cycle L2c, two in engram_compile (live on the AGENTIC path too),
two in affective_context_prefix. A distress turn following an earlier affective
turn killed the whole process. Proven pre-existing: an unmodified baseline binary
crashes identically, and the same bad C is in the committed dist/soul.c. Fixed by
hoisting to one top-level function, affective_node_ts(), where the expression
compiles to integer addition — verified in the generated C.
Proof (throwaway HOME/engram, explicit NEURON_PORT, live chain untouched):
- Plain turn returns a JSON envelope with the provider's answer, not an echo.
- Captured request body: no `tools`, no `tool_choice`; system prompt carries the
NO-TOOLS rule. Tools:Off means no tool is offered, structurally.
- Hard bell: canned 988 message, and the provider request count does not move —
the message never reaches a model.
- Soft bell + a 2-char model reply: safety_validate's care phrase is appended to
the MODEL's output. Output gate acting, on this route.
- The L1 bell directive now reaches the model here for the first time (the state
addendum had a producer and no consumer).
- test_layered_cycle PASS; all six El suites byte-identical to baseline.
- verify-soul-contract.sh (bash 5.3): GATE PASS, 27/27, immutability PASS.
- The crash sequence that killed the baseline daemon now returns HTTP 200.
Not proven: no live Anthropic call — the login keychain refuses the key to a
non-interactive process (rc=24 errSecInteractionNotAllowed). Details and the
one-command close-out are in _engine-plainchat-20260805/README.md §7.
Builds on PR #108. dist/soul.c deliberately not regenerated — Will's toolchain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Re-authors soul-webfix-20260711.patch in El (the patch is a diff against
generated C at month-old offsets, so nothing was applied as a patch). Its
last two hunks — an unrelated /api/safety-contact implementation — were
deliberately not ported; that route already exists and is safety-critical.
Activation restores existing design, it does not invent a mechanism:
commit 8eea1d9 (2026-06-09, Tim-approved) made native web_search built-in
with no user-facing toggle, and tests/test_agentic_tools.el section 2 still
asserts agentic_tools_all() contains it — an assertion main currently fails.
The call site was lost when agentic_tools_all() (connector tools, PR #19)
replaced agentic_tools_with_web(). Attaching in agentic_tools_all() covers
handle_chat_agentic, handle_dharma_room_turn_agentic and agentic_resume.
pause_turn handling is included and was genuinely missing: the shipped
binary has zero occurrences of it. Without it a paused server-side search
returns only the text written so far and the loop treats it as final —
a silently truncated answer. final_text now accumulates across resume
cycles rather than overwriting (overwriting would discard everything
written before the pause).
Default tool version is web_search_20250305, NOT the newer _20260209, and
that is a measured choice: _20260209's dynamic filtering uses server-side
programmatic tool calling, which the API refuses to combine with ADR 0005's
stopgap —
HTTP 400 invalid_request_error
tool_choice.disable_parallel_tool_use: true cannot be used with
programmatic tool calling
Dropping the stopgap would resurrect neuron#78 bug b (killed runs 3/3 in
ADR 0005's own A/B). The basic variant is compatible with the stopgap and
returns real results, so neither feature is dropped. Version lives in state
key web_search_tool_version; flipping it once the stopgap retires is a
config write, no recompile. The fallback also fires on "programmatic tool
calling" so a premature flip self-heals loudly instead of dying.
Also fixes a real bug this port exposed: json_get is a first-match scanner
and a cited text block serialises citations FIRST, so json_get(block,"type")
returned the nested citation's type and every citation-bearing block — the
ones carrying the searched facts — was silently dropped from the reply.
Reply length on the same question: 79 -> 360 chars.
Other loop changes: server_tool_use accounting into tools_used, iteration
cap 8->12 for pause/resume cycles, max_tokens 4096->16384, container-id
carry-forward, API error head logged instead of swallowed, tools_used gated
on is_tool_turn so a truncated tool block is not reported as work done.
dist/soul.c is deliberately NOT regenerated — the local elc predates Will's
last regen (e610a41) and regenerating with an older compiler risks unrelated
codegen drift. Source only; regen is Will's.
E2E-VERIFIED on a sandbox soul (scratch HOME/engram, dead axon+ISE, explicit
NEURON_PORT): live Bentonville weather with tools_used ["web_search"]; the
27-route contract gate PASSes; the parallel-tool stopgap still completes a
3-file mission with no 400; a 3-search 3,486-char answer kept its end
sentinel. Honest gap: pause_turn is compiled in but not exercised by a live
pause (max_uses:5 caps the server loop below the pause threshold).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The agentic loop keeps only the FIRST tool_use block per round (chat.el:2281,
"Capture first tool_use block only"). Anthropic lets a model emit several
tool_use blocks in one message and requires a tool_result for every one, so a
parallel-tool turn is answered once, the rest are dropped, and the next request
dies with:
tool_use ids were found without tool_result blocks immediately after
(neuron#78 quotes this as "tool_use ids found without tool_result"; the above is
the API's actual wording - recorded so the next person's grep matches.)
This constrains the wire to match what the loop can assemble:
"tool_choice":{"type":"auto","disable_parallel_tool_use":true}
STOPGAP - AND THE DURABLE FIX ALREADY EXISTS. A correct multi-tool loop is
already in Will's EL runtime, in C, and the soul does not call it. Verified on
el:origin/main lang/el-compiler/runtime/el_runtime.c: llm_register_tool:9616,
llm_build_tool_results:9743 - which walks EVERY content block, emits one
tool_result per tool_use, and sets is_error for an unregistered tool -
llm_call_agentic:9817 calling it at :9918, iteration cap 10 at :9847. Will's
commit 12d5e77 (2026-04-30). grep for llm_call_agentic/llm_register_tool across
every neuron/*.el returns nothing; dist/soul.c has zero references. chat.el
hand-rolls its own single-tool loop instead, and that is the one that breaks.
The durable fix is therefore to register the soul's tools via llm_register_tool
and call llm_call_agentic - deleting a loop, not writing one. See ADR 0005.
Our own approved spec called this seven weeks ago:
docs/research/agentic-tool-approval-design.md (2026-06-12, "Approved for build"),
line 20 on the defect, line 30 on the goal ("Execute all tool_use blocks in a
turn (one result per block)").
Two edits, because dist/soul.c cannot be regenerated here (Will's gated elc/elb
toolchain is not on this machine):
(a) chat.el:2255 - source of truth, so a later regen carries the fix.
One edit covers all three routes: agentic_loop is called from chat.el:2152
(/api/chat agentic), :2676 (dharma room) and :2496 (agentic_resume).
(b) dist/soul.c:28173 - generated form, hand-spliced. Line 27624 is the
non-agentic/OpenAI-compat req_body (no tools) and was left untouched.
Prior art reused rather than reinvented: soul-narrated-runs-20260713.patch
(27,824 bytes) line 78 spliced this same string into the same concat chain on
2026-07-13.
Deliberately NOT ported from that patch, having read it: max_tokens is not
changed by it (16384 sits on both sides of the hunk; our main's 4096 is a
separate output-truncation concern), and its pause_turn pairing fix - same
defect class - is unreachable today because no server-side web_search is wired
(agentic_tools_with_web at chat.el:1418 is never called), so it is untestable
and logged instead.
Proven E2E on a scratch profile and port 7791, never the live chain. A/B against
a pristine origin/main control built from the same vendored runtime: fixed
completed the mission (tools_used read_file x3, 4 iterations, correct answer);
control failed 3/3. Direct API probe confirmed the mechanism - without the field
the model emits 3 parallel tool_use blocks and replaying the unfixed loop's next
turn returns HTTP 400; with it, exactly 1 block.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Union of all ship-critical soul fixes for the Mac beta:
- Keeps main's honest receipts (#100/#101), immutability arc + #199 by-id
gate (#83), Track B threat routing (#76), bounded beginSession (#103),
CI hardening (#85/#86), elc typo hotfix (#77), neuron-dev-setup (#84).
- Brings WIP's genesis-boot SIGSEGV fix (#150/#95), safety-contact 988
truncation fix (#96), bounded-persona floor (#93), and 10 self-review
commits (importance flattening, curiosity DF gating, WM/heartbeat
observability, boot-counter telemetry).
- Folds the multi-word ranked-search fix: api_query_param now url_decode()s
the extracted value so q=foo%20bar / foo+bar tokenize as two words.
Conflicts (neuron-api.el payload-bound comments, mcp-wrapper tool_forget)
resolved toward the correct end state: main's verified honest-receipt
read-back is kept; WIP's improved forget description is kept. Generated
dist/*.c taken from main and will be regenerated from this reconciled
source in the following commit.
Root cause: dispatch_tool's write_file returned {"ok":true} without checking
fs_write's result, and edit_file returned ok:true even when old_text was absent
(str_replace silently no-ops) and its fs_write was also unchecked. Any failed
or no-op write fed the model a false success receipt, which it then repeated
to the user as fact.
The change (Receipt Contract rule 1 — a tool result must reflect what actually
happened):
- write_file: check fs_write's return (1 = all bytes written, 0 = fail);
on failure return {"error":"write failed"} in the handler's existing
error-JSON shape.
- edit_file: reject empty old_text, verify old_text is actually present
(str_contains) before replacing, and check the fs_write result the same way.
- Verification is by operation result, NOT an fs_read read-back: fs_read arms
the runtime's one-shot binary send length, the exact mechanism that truncated
the safety-contact response (#96). Same honest-write pattern as that fix.
E2E evidence (sandboxed elb build, dispatch_tool driven directly):
- unpatched: write_file into a chmod-000 dir -> {"ok":true} (lie);
edit_file with absent old_text -> {"ok":true} (lie, file untouched)
- patched: same calls -> {"error":"write failed"} /
{"error":"old_text not found in file"}; happy paths still ok:true
- scripts/verify-soul-contract.sh on the patched soul: GATE PASS (27/27
presence + immutability)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A customer DMG install ships the full graph but presents a named, bounded
assistant that must never claim the imprint's human past. The neuron-ui
retrieval fence keeps the imprint's biography out of the ENGRAM CONTEXT; this
is the second half - it stops confabulation ("tell me about your childhood")
from inventing a human life or naming Will, even if biography leaks into context.
bounded_persona_floor() gates on SOUL_PERSONA_NAME: the customer DMG sets it,
owner (Will's) builds leave it unset so the real self is completely unchanged.
Applied at every generation path - chat, agentic (tools), vision, plan, soul,
dharma - so no path can leak.
Verified against claude-sonnet-4-5: with the floor on and Will's biography
deliberately leaked into the identity context, all probes (childhood / creator /
family) return the bounded-entity answer and explicitly refuse to claim the
leaked life; with the floor off the same context is fully confabulated as its own.
NOTE: dist/soul.c must be regenerated on a build host - local link is blocked by
a pre-existing el_runtime mismatch (engram_prune_telemetry), unrelated to this change.
Two compounding defects made every pause->approve write_file report success
while writing NOTHING:
1. The naive json_get scanner matches "content" anywhere in the approve
body — including INSIDE tool_input, which for write_file always carries
a content field. The handler therefore treated every approved builtin
write as already-client-executed, skipped dispatch entirely, and handed
the model the file's own content as the 'tool result'. The model then
narrated 'Done, created' — a false receipt with no file. Builtin tools
now ALWAYS dispatch server-side; client content is only honored for
non-builtin (MCP/client-executed) tools. Stricter only.
2. write_file returned {"ok":true} unconditionally — fs_write's outcome
was never checked, so any failed write also reported success. The write
now verifies the file landed (fs_exists) and returns the RESOLVED path
in the ok payload; failures return a real error naming the destination.
E2E on the test brain (boot 38): approve-path write lands byte-exact and
the result carries the resolved path; auto-run writes unchanged; denied
writes execute nothing. BUG-5 (approve wire lacked tool_name) had been
masking this one — two stacked bugs on the same path.
NOTE for review: the deeper cure is a nesting-aware json reader; this fix
removes the dangerous consequence at the two spots that lie about disk.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two consent-flow fixes, gates only get stricter:
1. PAUSE-CONTRACT: the client has sent require_approval:true on every
agentic request since Phase 1c, but needs_bridge never consulted it —
builtin sub-escalate tools ran server-side unasked, making the app's
Ask autonomy silently inert for that whole class. Now the flag is
persisted per session (set/reset every request, so /approve resumes
keep it) and ask_all bridges EVERY tool turn. Absent/false = behavior
byte-identical to before. E2E: the exact probe that executed a write
unasked now returns the tool_pending envelope with nothing on disk;
full in-app circle verified (card → crash → resurrection → late
approve → fence re-fires on re-entry).
2. BUG-LEAK: agent_workspace_root lived in ONE shared state key — any
request that omitted a root inherited the previous session's folder
(proven: a rootless curl session wrote into another session's run
folder). Root is now stored per session and every request re-asserts
its own (possibly empty) root into the shared key the guards read;
same re-assert on the /approve and resume paths. Env fallback intact.
LIMITATION: assumes serialized handling; true per-call scoping means
threading session_id through dispatch — flagged for review.
Runnable C-patch for the test brain: neuron-container-build/
soul-pause-contract-20260716.patch (pause-contract only; the leak fix
needs the #23 root-write which the running C predates — source carries
both for the regen).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Regenerates the combined dist/soul.c and per-module dist/*.c from the current
El sources, on top of the elc-source-typo fixes (PR #77) and the Track B
threat-to-others routing (PR #76), both already on this branch.
Validated end to end under a physical-RSS watchdog (macOS silently ignores
ulimit -v / RLIMIT_AS, so every elc/elb run was RSS-polled and kill -9'd at a
3GB ceiling, one module at a time):
- OOM is GONE. The stale dist/soul-with-nlg.el (which still carries the
malformed string literals) explodes to 3.3GB+ and is watchdog-killed at ~90%.
With the typos fixed, every one of the 48 modules compiles at <=18MB peak RSS,
and the full flat amalgamation compiles as a single translation unit at ~68MB.
The 700GB pathology was purely the unbounded-parser-on-malformed-literal loop;
no malformed construct means no loop.
- The regenerated soul.c contains Track B: safety_classify_hard_bell ->
threat_other -> safety_hard_directive routes credible threat-to-others to 911
and explicitly NOT to 988 / the safety contact. Verified in source, in the
emitted C, and in the linked binary's strings. Track A (abuse / self_harm)
is unchanged and still checked first.
- The regenerated soul links to a working native arm64 binary and boots: serves
on a throwaway port, /health returns 200, awareness loop runs.
Also fixes one source blocker discovered during regen (unrelated to the typos
or Track B): chat.el handle_chat_agentic left a void `if { println(...) }` in
value position, which the current elc lowers to `_if_result = (println(...))`
(assigning void) -> invalid C. Bound an explicit Bool so the branch is
non-void; behavior unchanged (still only logs on persist failure).
NOTE (runtime dependency, for controlled deploy): this branch's chat.el calls
engram_get_node_by_label, which the canonical el-compiler/runtime does not yet
declare/define (the release runtime v1.0.0-20260501 has it; the newest runtime
has arena + http_serve_async but not this). Building the soul requires a runtime
that has all three. Land engram_get_node_by_label into the runtime package
before this soul.c can be built in CI.
Do not merge — regen + Track B going live is a controlled-deploy call.
The model already narrates its intent in a text block before every tool call;
agentic_loop DISCARDED that prose on tool rounds. Now: (1) each loop round
appends {i, t: narration, tool} to state key run_progress_<sid>, reset at run
start, closed with {done:true}; (2) new GET /api/run-progress/<sid> returns the
ledger so clients poll live step updates during a run (the Cowork pattern,
no streaming needed); (3) tool_pending envelope gains a narration field;
(4) handle_config display default aligned to the intended product default
(claude-sonnet-4-5 silently became fresh-profile pickers' default).
Compiled proof for the running test bed:
neuron-container-build/soul-narrated-runs-20260713.patch (applies on top of
soul-webfix-20260711.patch); E2E-verified live: ledger filled DURING an agentic
run (narration + tool per round), safety-contact and workspace scoping intact.
Evidence for why: Tim's 2026-07-13 research run — 9 minutes of silence, then a
timeout banner, zero step visibility (compounded by the Haiku 4.5 incident
14:44-15:24 UTC same morning).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Enforcement moves from the client into the engine, where the tools execute:
- classify_tool_risk() tiers every tool call read/reversible/escalate. The
agentic loop REFUSES to auto-run the escalate tier — being a builtin is no
longer a free pass, and 'always allow' can never bypass escalate (irreversible
actions always confirm, the value line). Escalate suspends to the client's
existing consent bridge; the /approve round-trip is the only path that runs it.
risk_tier rides the tool_pending envelope so the client renders consent weight.
- run_command_guard() is a real fence, not a cwd suggestion: refuses parent
traversal, ~, command substitution, and absolute paths outside the workspace,
and refuses shell entirely when no workspace is set. Applied in dispatch_tool
so BOTH the loop auto-run and the post-consent approve-dispatch path are fenced.
- web_get gained an http(s)-only scheme guard (previously unguarded — file:// etc).
Adversarially verified against a compiled soul in an isolated container (soul
hit directly, app gate out of the loop): read-outside-workspace denied,
write-class shell suspends for consent, approve-swapped absolute/chaining/
command-substitution escapes all refused with no file created, file:// denied;
legit in-workspace approve executes and read commands auto-run (no over-block).
Still lexical (symlinks); OS-level confinement in el_runtime.c remains the
ceiling, flagged in the LIMITATION note. This closes BUG-8's client-only-gate
and escapable-run_command at the engine. dist/soul.c must be regenerated from
this chat.el via elb at merge (hand-port used only to verify behavior).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two mechanical refactors, semantics identical:
- affective_context_prefix(): the block-expression initializer form miscompiles
under locally-buildable elc (first typed let in a block-expr loses its
declaration — 3-line repro filed); function-hoist compiles correctly.
AFFECTIVE/CARE LOGIC BODY UNCHANGED, verbatim move.
- session_preload: same-scope re-let shadowing inside an if-expression
initializer emits duplicate C declarations; chained bindings renamed
bullets_0/1/2 etc. References preserved binding-for-binding.
Enables: chat.el compiles cleanly with a self-bootstrapped elc from el/lang
main (Jul 1). Blocked separately: sessions.el (compiler hang), safety.el
(string-lexing corruption — NOT touched, per safety-layer discipline).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds llm_base_url()/llm_wire_format() env-var readers and
openai_chat_complete() for basic (non-agentic) chat via any
OpenAI-compatible endpoint. Activated when NEURON_LLM_0_FORMAT=openai
and NEURON_LLM_0_URL is set; Anthropic path is untouched and remains
default. Agentic tool loop support deferred to a follow-up PR.
Adds current_engine_note() to chat.el and appends it to the system
prompt in handle_chat. Allows Neuron to answer 'what model am I
running on?' accurately — the model id from the request body (or
the configured default) is passed as a factual annotation rather
than expecting the LLM to guess from training data.
handle_dharma_room_turn and handle_dharma_chat both called
distill_transcript since June 30 but the function was never declared,
causing a build failure. Implements last-3-messages extraction for JSON
array transcripts and last-500-char truncation for plain text.
Additive: appends a factual [CURRENT ENGINE: <model>] line to the system prompt (model from the
request body — accurate even under Auto routing; falls back to configured default). An LLM can't
know its own model from training (name/version assigned post-training), so the harness must tell it.
Identity-consistent: model = engine, self layered on top. Does NOT alter identity/values/safety.
PARSES (elc chat.el exit 0); NOT built/tested — ships with the soul rebuild.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- PR #60: inject operator home dir into system prompt (#30)
Adds OPERATOR IDENTITY section so the LLM correctly resolves
'my files/notes/desktop' to the actual running user's $HOME.
Prevents identity confusion between imprint author and operator.
- PR #61: plan-mode endpoint POST /api/chat {mode:'plan'} (#27)
Adds handle_chat_plan — returns {steps:[{id,title,detail}]} JSON.
Wired into all three /api/chat route handlers. Grounds the plan
via engram_compile (same as agentic path) for context awareness.
dist changes:
- soul.c: both PRs compiled in; build_system_prompt updated to
2-param signature (ctx, chat_mode); handle_chat_plan added
- chat.c/routes.c/chat.elh: individual module outputs updated
- elp-c-decls.h: remove stale 1-param build_system_prompt decl,
add handle_chat_plan declaration
- soul.elh.c: new soul header declarations file (from PR #60)
Compile verified: cc -O2 -DHAVE_CURL soul.c el_runtime.c -lcurl
Binary: 805K arm64, smoke test passes (port in use = expected).
handle_chat_agentic now reads body image + image_media_type and, when present, sends the current
user turn as an Anthropic content-block array [{text},{image}] instead of a plain string — so the
model sees raw pixels alongside memory, history, and tools (parity with the CLI). Additive: no image
=> output byte-identical to before. elc-clean. Pairs with neuron-ui fix/chat-vision-attachments.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
All 8 session_preload node accesses (3 profile, 2 work, 2 project, 1
summary) now check id_in_seen(node_id, seen_ids) before including
content. seen_ids is populated by engram_compile via state and covers
all nodes already in the activation+search context block. Prevents
high-salience nodes from appearing twice in the system prompt.
Issue 1 (CRITICAL): Restore parse_float_x100 for correct single-decimal
float handling. "0.9" now correctly yields 90, not 9. Also restores
engram_numeric_valid guard that validates inputs before str_to_int.
Issue 2 (CRITICAL): Fix handle_chat_agentic safety screen history key
regression. state_get("conversation_history") -> state_get("conv_history")
so the safety screen receives actual history instead of always "".
Issue 3 (REAL BUG): Replace _sel_N JSON sentinel injection in
engram_compile_ranked with |N| index string tracking. Sentinels were
leaking into node JSON delivered to the LLM and cleanup only covered
indices 0-14, leaving indices 15+ uncleaned.
Issue 4 (REGRESSION): Restore rendered conversation history formatting.
Conversation history is now rendered as "User: .../Assistant: ..." with
400-char truncation per turn, not raw JSON array injection.
Issue 5 (SCOPE/SAFETY): Restore removed defensive code: engram_numeric_valid
and parse_float_x100 guards; conv_history_load label-based fetch + partial-
write guard + load-failure state flag; conv_history_persist partial-write
guard + failure logging; hist_warning in response envelope.
Issue 6 (UNDOCUMENTED): Restore bell event cutoff from 259200s (3 days)
back to 1209600s (14 days). Also restore PositiveEvent affective context
search that was removed alongside the cutoff change.
Issue 7 (LOGIC REGRESSION): Fix affective_prefix to run every turn
(not just hist_len == 0). The care/joy directives must persist throughout
the session, not vanish after turn 1.
Issue 8 (MINOR): session_summary_write_dated now uses el_from_float(0.85)
for salience and importance (two-decimal) to avoid any ambiguity in float
parsing, and the function is re-added with the session-end hook.
Two design bugs in the state_set placement caused the dedup seen-ID set
to be incomplete even with callsites wired up:
1. state_set("engram_compile_seen_ids") was called immediately after
merging the main node pools, before scan_part (persona fallback) and
affective_part (bell node) were computed. Nodes appearing only in
those segments were never added to the seen set.
2. affective_part is a bare JSON object (bn0 from json_array_get), not
a JSON array. Passing it to engram_extract_ids would have gotten
json_array_len == 0 and silently skipped the affective node's ID.
Fix: move state_set to after ctx is assembled from all three segments.
Extract ids_from_merged and ids_from_scan via engram_extract_ids (both
are JSON arrays), and extract ids_from_affective via json_get(affective_part, "id")
directly since it is a bare object. Merge all three via add_to_seen
before publishing to state.
Thread a seen-node-ID exclusion set from engram_compile() through to
session_preload in handle_chat, preventing the same high-salience nodes
(identity, recent memories) from appearing 2-3x in the system prompt.
Changes:
- Add id_in_seen(), add_to_seen(), engram_extract_ids() helpers that
maintain a comma-delimited seen-ID accumulator (EL has no Set type)
- In engram_compile(): after merging all topic/entity/recall pools, extract
node IDs from merged_nodes and publish via state_set(engram_compile_seen_ids)
- In handle_chat(): read seen_ids from state after engram_compile() returns,
then check id_in_seen() before emitting each session_preload bullet
(profile x3, work x2, project x2, summary x1 — all 8 candidate nodes guarded)
Nodes already present in the compiled engram context are skipped in preload,
eliminating 3000-3500 token repetition on first-message turns.
engram_compile() already published seen node IDs to state via engram_compile_seen_ids
but handle_chat never read or applied them. Wire up the consumption side:
- Read engram_compile_seen_ids from state after engram_compile() returns
- Check each session_preload candidate node (profile x3, work x2, project x2,
summary x3) against id_in_seen() before emitting its content bullet
- Nodes already present in the compiled engram context are skipped entirely,
preventing the same high-salience identity/memory nodes from appearing 2-3x
in the system prompt and burning 3000-3500 tokens on repetition
- Add engram_render_node/render_nodes/dedup_nodes helpers for human-readable
prose bullet output instead of raw JSON node objects reaching the LLM
- Fix engram_compile_ranked to use |N| index sentinel instead of _sel_N JSON
mutation which leaked sentinel fields into LLM-visible node data (Issue #11)
- Update build_system_prompt with chat_mode param; no_tools_rule only included
for chat path, not agentic paths (Issue #9)
- Move engram block to end of system prompt for strongest LLM attention (Issue #8)
- Label sections: STABLE IDENTITY vs RETRIEVED MEMORY (Issue #10)
- Render conversation history as User:/Assistant: dialogue instead of raw JSON
- Add RETRIEVED MEMORY labels to agentic and dharma room system prompt assembly
- Cache bell node in engram_compile state (engram_compile_bell_node)
so handle_chat reads cached value instead of duplicate bell query (Issue 2)
- Cache activation result (engram_compile_activation_json) for strengthen_chat_nodes
reuse — eliminates third activation query per turn (Issue 7)
- Fix context cap to truncate at clean JSON object boundary (Issue 6)
- Cache bell node result in engram_compile state (engram_compile_bell_node)
so handle_chat affective_prefix reads the cached value instead of firing
a duplicate engram query for distress signals (Issue 2)
- Cache primary activation result in engram_compile state
(engram_compile_activation_json) using nodes0 from engram_compile_multi
- Replace redundant engram_activate_json(message, 2) in strengthen_chat_nodes
with state_get(engram_compile_activation_json) - eliminates a third
activation query per turn (Issue 7)
- engram_compile already has object-boundary truncation and cross-set
dedup via engram_nodes_merge/engram_dedup_nodes (Issues 1, 6, 9)
- Issue 2: replace raw 50-char threshold with is_genuine_continuation() that
checks for explicit follow-up phrases and mid-sentence capitalization (proper
nouns signal a new topic, not a continuation)
- Issue 3/8: build_activation_seed() scans back to find the prior USER turn as
the topic anchor instead of using the last assistant reply (hist_len-1)
- Issue 4: engram_compile_multi() fans out across three seeds — enriched primary,
raw message (entity queries), and emotion query — merging non-redundant results
- Issue 5: agent workspace_root appended to ag_seed so agentic activation is
workspace-aware; previously ignored despite being available in state
- Issue 6: distill_transcript() extracts salient tail+question content from full
transcripts before passing to engram_compile in dharma room handlers
- Issue 7: dist/soul-with-nlg.el handle_chat and handle_chat_agentic now load
history and use build_activation_seed() — the raw message path is eliminated
- Issue 9: topic_snip_from_entry() takes the TAIL 200 chars of a long reply and
finds the last sentence boundary — captures end-of-reply named concepts
- Issue 10: multi_turn_topic() pulls up to 3 prior user turns into the non-
continuation seed so earlier thread context re-activates high-salience nodes
Fix critical float parsing bug: %g serializes 0.70 as '0.7', naive str_replace
dot-strip gives str_to_int('07')=7 not 70. New parse_salience_100() uses
str_index_of to detect single-decimal strings and multiplies by 10 to correct.
Affects conv nodes (0.6/0.7), default memories (0.5/0.5), utterance nodes (0.6)
— the majority of the graph was scoring near zero and filtered by threshold=25.
Fix recency to use max(created_at, updated_at, last_activated) so nodes
strengthened by engram_strengthen() after chat turns score as fresh, not by
original write time. A node referenced yesterday but created 25 days ago
was borderline-filtered; now correctly scores fresh.
Compress recency dynamic range from 10x (10-100) to 1.54x (65-100) via
formula (50 + recency/2). Old formula: sal*imp*recency/10000 let recency
dominate — a canonical high-importance node at 30 days scored identical to
a fresh noise node. New: high-importance nodes remain competitive when old.
Add tier-aware decay with softer floor (30 not 10): Canonical nodes decay
over 365 days, Episodic over 90 days, working/untiered over 35 days. Long-
term identity and persona nodes are no longer permanently filtered.
Lower threshold from 25 to 15 to admit moderately-relevant older nodes that
pass scoring with the corrected formula. Backfills recall coverage lost when
single-decimal nodes were being silently discarded.
Apply scoring to activation nodes: engram_compile_ranked(activate_json, 5)
replaces unconditional pass-through. Threshold 5 preserves recall while
excluding genuinely zero-quality stale nodes.
Extend sentinel cleanup in engram_compile_ranked from _sel_0-9 to _sel_0-19
so max_nodes can safely be increased past 10 without JSON corruption.
Tim Lingo
will.anderson