Merge improve/safety-crisis-detection (PR #31): reads layered_cycle_safety_system_addendum
from state and appends to system prompt on each turn (cleared after use to prevent bleed).
Safety ts extraction falls back to updated_at. Affective prefix now wires into system build.
Conflict with PR #33 resolved: capability_rules and session_preload both preserved.
- engram_compile: BellEvent nodes do not carry created_at in the engram
node JSON; extract the unix timestamp from the embedded ' | ts:NNNNN'
pattern in the content string instead. Fall back to created_at/updated_at
if the marker is absent. Guard str_to_int against empty string so the 72h
recency check never silently treats every node as epoch-0 stale.
- auto_persist: append the current unix timestamp to the BellEvent label
('bell:soft:1749876543') to make it unique per turn. The previous label
('bell:soft') was the same for every soft bell, causing engram to treat
all subsequent writes as updates to the same node.
- Remove dead soft_bell block in layered_cycle that wrote soul_safety_system_augment
to state but was never read; safety augmentation now goes through the correct
layered_cycle_safety_system_addendum state key read by build_system_prompt
- build_system_prompt now reads layered_cycle_safety_system_addendum and appends
it to the system prompt, clearing the key after consumption
- Timestamp extraction for distress nodes falls back to updated_at when created_at
is empty, preventing the 72h recency check from always treating nodes as stale
Add strip -s after gcc compilation to remove symbol table and relocation info.
Reduces binary size and prevents symbol-level reverse engineering of EL runtime internals.
- auto_persist: detect bell level (soft/hard) on every user message using
safety_detect_bell_level; write a dedicated BellEvent engram node with
calibrated salience alongside the Conversation node when a bell fires.
Tag the Conversation node with bell:soft/bell:hard and 'affective' for
direct discovery without scanning all chat nodes.
- auto_persist: track per-session bell count, dominant level, and last
signal in state (session_bell_count/level/signal keys) so downstream
functions can act on the emotional history without re-scanning engram.
- engram_compile: include the top-1 most recent BellEvent node within 72h
in every context build. Distress context from earlier turns (same or
recent session) automatically travels into all subsequent LLM calls.
- hist_trim_with_bell_guard: replace hist_trim at the handle_chat call site.
Before evicting the oldest turn from the 20-turn window, inspect the user
message for bell signals. If a bell was present, write a preservation
BellEvent to engram before dropping the turn so the full message survives
the rolling window.
- session_hist_save: after writing the history node, check session bell
counters. On the first save where bell_count > 0, write a
session:emotional-summary BellEvent node with distress signal, count,
and dominant level. A state flag prevents duplicate writes on subsequent
saves in the same session.
- engram_compile: rank search results by recency x relevance before including
in context. Pulls 20 candidates, scores each (salience * importance * recency
decay), keeps top 8. Eliminates stale/low-signal nodes that diluted context.
- handle_chat: on hist_len==0 (session start), proactively load user profile
and active-work context from engram and inject as brief bullets in the system
prompt. Gives the soul grounding before any conversation history exists.
- build_system_prompt: add [CAPABILITY GAPS] directive instructing the soul to
offer partial help and reasoning instead of flat "I don't have access to that"
refusals when a tool is missing.
- handle_chat_agentic: run safety_screen at entry, mirroring layered_cycle.
Hard bell exits immediately with the crisis response without entering the loop.
- agentic_loop: surface the 8-iteration cap explicitly in the error envelope
("agentic loop hit the 8-iteration cap...") rather than the opaque "no response".
Add iterations count to both the error and success envelopes for observability.
- Add per-IP in-memory rate limiter (60 req/min default, configurable via
soul_rate_limit state key; /health exempt; loopback callers skipped)
- Extend /health with uptime_secs (from soul_boot_ts) and live LLM probe
- Add missing_param 400 guard on POST /api/chat before passing to LLM
- Standardise error envelopes: add "code" field to err_404/err_405 and all
missing-param returns; route_synthesize now errors clearly instead of
returning the misleading {"mechanism":"did not engage"} on bad input
- Document streaming gap in /api/chat (SSE not implemented, note added)
- handle_request gains ip param; rate_limit_check wired at entry point
- soul.el: fix state key bug in layered_cycle (conversation_history -> conv_history)
- safety.el: add indirect crisis location patterns to soft_bell phrase list
- soul.el: wire safety_augment_system into layered_cycle for soft_bell turns
- chat.el: load cross-session affective context at session start when distress signals found within 72h
REPRODUCED: in the non-agentic path (Tools off / 'Just chat'), asking for
tool-work makes the model role-play tool use — it emits a fake ```json {...}```
'tool call' and says 'let me search/query/pull your sessions' while NOTHING
runs. Reads as a broken/lying app. (The agentic path is fine: verified it
calls search_memory and reports honestly.)
Root cause: build_system_prompt (handle_chat, the tool-less path) never told
the model it has no tools this turn, so it fabricated.
Fix: add a NO-TOOLS directive to the non-agentic system prompt — never emit
tool calls / JSON tool blocks / 'let me pull...' narration; answer from context
only; if a tool is truly needed, say so in one sentence and tell the user to
turn Tools on. Applied to chat.el (source) AND dist/soul.c (the curated TU the
CI compiles), so the CI-built binary carries it.
Verified the FABRICATION repro on the live local soul; could not verify the
patched binary locally (no matching el-runtime version on this machine — a
hand-link against origin/main runtime 404s on all routes). Builds correctly via
CI, which links soul.c against the pinned runtime.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Knowledge nodes dominated the WM-autobiographical auto_term slot:
'Numeric tier strings...' (a Knowledge node) always scored highest
in WM and its first word 'Numeric' became the curiosity seed every
scan — activating more Numeric nodes, keeping that node in WM,
repeating indefinitely.
Fix: only derive auto_term from Memory, BacklogItem, or Entity nodes.
Knowledge nodes are reference material, not live context. Dynamic/
personal nodes carry the salience worth radiating from.
Also patches proactive_curiosity directly in dist/neuron.c (ELC
cannot compile soul.el within timeout — fallback build pattern).
The Dockerfile's --mount=type=secret path was corrupting the SA key JSON
due to control character handling differences. Pre-download soul + El SDK
in the CI workflow (using already-authenticated gcloud) and COPY them from
the build context. No credentials needed inside the Docker build.
Previous builds leave cached layers and images on the runner. Add a
docker system prune at start of deploy to avoid container-creation
failures from disk exhaustion.
--secret requires BuildKit; DOCKER_BUILDKIT=1 enables it on the legacy
Docker client. Also add GITHUB_SHA fallback and git rev-parse last-resort
so the image tag is never empty.
elb overwrites dist/soul.c with a fresh (non-inlined) compilation before
its link step fails, discarding the patched self-contained version.
Save the repo copy before elb and restore it after so the compiler always
gets the complete translation unit with all patches applied.
Modern gcloud CLI (>= 400) requires this env var so kubectl uses the
installed gke-gcloud-auth-plugin binary instead of the deprecated
application-default credentials path. Without it, kubectl commands
silently fail even after get-credentials succeeds.
Incorporates PRs #22/#23/#24:
- agentic_tools_all dedup fix (no duplicate web_search tool)
- workspace scope functions (agent_workspace_root, path_within_root, resolve_in_root)
- updated dispatch_tool with workspace confinement
- canonical-self bridge (ensure_self_canonical_bridge)
Also incorporates CI link fix from PR #26 (soul.c is self-contained, no
other dist/*.c needed). Fixes the CI build step which was compiling the
old June-16 soul.c that predated all these changes.
elb generates a dist/soul.c with all El modules inlined. Linking
dist/soul.c alone is sufficient and is exactly what the local mac
build does. Including other dist/*.c files causes two failures:
1. dist/chat.c has a capability-violation #error that fires when the
file is compiled as a utility module (outside the cgi entrypoint).
2. --allow-multiple-definition masked other issues silently.
Drop OTHER_C, drop --allow-multiple-definition, drop the now-unused
elp-c-decls.h generation step. The cc command now matches the proven
local build exactly.
The graph API resolves name=self/neuron to kn-efeb4a5b (neuron-api.el:471),
which carries only 8 incidental 'tagged' edges. The curated identity lives on
self node 015644f5 (1461 edges: identity, embodies, remembers, values). So
public self-traversal reaches tags, not the real self.
Add ensure_self_canonical_bridge(): an idempotent boot-time repair that links
kn-efeb4a5b <-> 015644f5 with a 'canonical-self' edge, only if missing. Runs in
the genesis safe-to-seed path regardless of the <100-edge gate, so the live
populated graph gets repaired and persisted. Connect-only-if-missing prevents
the duplicate-edge stacking that gates init_soul_edges().
Compile-checked with elc (darwin arm64); not link/run-gated locally. Needs a
soul build + smoke test before merge.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Confine the agentic file tools (read_file, write_file, list_files, grep)
to a configured workspace subtree via a lexical path check, and run
run_command with its cwd set to that root. Root comes from state key
"agent_workspace_root" or env NEURON_AGENT_ROOT. When no root is set,
behavior is unchanged (unscoped) for backward compatibility.
Defense-in-depth, NOT a hard boundary: the lexical guard does not resolve
symlinks and cannot stop an arbitrary shell command from cd-ing out of the
root. Real confinement needs runtime support (cwd-locked exec / sandbox-exec
/ chroot) in el_runtime.c.
Compile-checked with elc (darwin arm64); not link/run-gated locally
(darwin elb unavailable). Needs a soul build + smoke test before merge.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
agentic_tools_literal() already contains a custom web_search tool.
agentic_tools_with_web() adds the Anthropic server-side web_search_20250305
tool (also named web_search). Combining them caused Anthropic to reject
every agentic request with 'Tool names must be unique.'
agentic_tools_all() now calls agentic_tools_literal() directly. Connector
tools splice in as before. The web_search-only variant (agentic_tools_with_web)
is unchanged for callers that specifically want native search without connectors.
fix(soul): add HTTP-engram guard to safe_to_seed — when ENGRAM_URL is set
the HTTP Engram owns persistence; genesis must never save to local snapshot
regardless of node counts (was: guard_disk forced to empty string, making
the ratio check vacuously true and allowing init_soul_edges+engram_save).
fix(soul): use multiplication form for ratio guard — node_count * 16000 <
disk_len avoids floor-division truncation that underestimated boundary files
(250KB / 16000 = 15.6, floors to 15; a 15-node graph wrongly passed old guard).
fix(chat): add safety_augment_system to handle_chat_as_soul,
handle_dharma_room_turn, and handle_dharma_room_turn_agentic — all three
called the LLM without Hard Bell evaluation, leaving users in dharma rooms
without crisis resource routing.
fix(neuron-api): add api_persisted read-back to handle_api_define_process —
was the only write handler that returned ok:true without verifying the node
was actually written to engram.
fix(routes): unique temp file path in connectd_post — replaces fixed
/tmp/neuron-connectors-req.json with a timestamped path to prevent
collision if concurrency is added or two soul instances share a machine.
test: add tests/test_bell_safety.el — covers safety_detect_bell_level
(none/soft/hard), safety_classify_hard_bell (abuse/self_harm routing),
safety_normalize (smart-quote), safety_augment_system, and
handle_safety_contact_post (validation + read-back).
test: add tests/test_soul_guard.el — pure-function logic tests for the
safe_to_seed predicate: 200KB boundary, 47MB/63-node clobber scenario,
HTTP-engram mode, multiplication vs division truncation at 250KB.
test: add tests/test_api_define_process.el — verifies the define_process
write is read-back verified after the fix.
Genesis boot previously seeded a fresh identity and saved it over snapshot.json
whenever the in-memory graph looked empty. Replace the fixed node-count threshold
with a ratio guard: refuse to seed when the on-disk snapshot is large
(>200KB) but the loaded graph is sparse (< disk/16000 nodes).
KNOWN LIMITATION: this gates only the seed/pre-serve-save path. The deeper cause
is a non-atomic engram_save (fopen wb truncates to 0 before writing 47MB), which
creates a window where a concurrent load reads an empty file -> genesis -> and if
guard_disk is read in that same window the guard passes. The real fix is an
atomic engram_save (temp + fsync + rename) in el_runtime.c, tracked separately.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
BLOCKER 1: use untyped reassignment (let x = ...) for the fallback bindings
in agentic_resume instead of re-declaring typed let bindings (let x: Type = ...)
for the same variable in the same scope. The typed form risks shadowing semantics
that differ from the established pattern used everywhere else in the loop
(e.g. agentic_loop line 720).
BLOCKER 2: add empty-string guards in both bridge_save and agentic_resume.
bridge_save now returns false without writing state if messages or tools_json
is empty — preventing syntactically invalid JSON blobs. agentic_resume now
returns an error envelope after the fallback resolution if either field is
still empty, rather than passing empty strings into agentic_loop which would
silently start a fresh turn with no context.
Also add tests:
- test_bridge_serialization.el: covers bridge_save empty-guard, golden-path
raw-JSON round-trip, agentic_resume unknown/corrupt/missing-fields paths,
and legacy string-escaped fallback path
- test_sessions_routes.el: covers DELETE and PATCH /api/sessions/:id routes
(valid args, unknown id, empty body) and GET /api/sessions regression after
removal of the duplicate route_sessions() handler
When agentic_loop suspends for an MCP bridge tool it returns a
{"tool_pending":true,...} envelope with no "reply" key. Without an
explicit check, json_get(loop_result, "reply") returns "" and the
function emitted {"response":"","cgi_id":"..."} — a silent empty
response indistinguishable from a successful LLM turn with no content.
Two guards added after the existing error check:
1. tool_pending passthrough: if the loop suspended, return the pending
envelope directly so callers (dharma room orchestrators) can
distinguish suspension from failure and route to the approve flow.
2. Empty-reply guard: if final_text is empty after the pending check,
return an explicit {"error":"no response",...} envelope instead of
silently succeeding with an empty response field.
Also adds tests/test_agentic_tools.el:
- agentic_tools_all() includes all literal tool names and web_search
- connector_tools_json() returns valid JSON when bridge is down (graceful degradation)
- tool_pending envelope detection patterns (the is_pending logic)
- json_get(pending_envelope, "reply") returns "" confirming the empty-reply
guard is load-bearing (pure string/JSON, no LLM or network required)
session_delete cleared the per-session state (session_hist_ and
session_node_) but not the shared session_index cache. The next call
to session_list() hit the fast path (state_get("session_index")) and
returned the deleted session until the daemon restarted.
session_update_patch already called state_set("session_index","") to
force a re-fetch from Engram; session_delete now does the same.
Add tests/test_sessions.el covering:
- session_title_from_message (pure function, all edge cases)
- session_make_content (JSON structure and required session:meta marker)
- DELETE cache invalidation: session_index cleared, fast path disabled
- PATCH cache invalidation: stale title/folder not returned via fast path
- GET /api/sessions: session_list() fast path returns session_index
(confirms removal of the stale route_sessions() engram stub)
BLOCKER 1 (sessions.el, modern path): Add guard that rejects allow
action when tool_name is missing from the body. Previously, omitting
tool_name caused dispatch_tool("", ...) to return "unknown tool: " and
silently inject a corrupted tool_result into the conversation.
BLOCKER 2 (sessions.el, modern path): Stop re-executing client-side
tools server-side. When the client provides body["content"], use it
directly as the tool result (matching the handle_tool_result contract).
Only fall back to dispatch_tool for builtin tools when no content is
present. Non-builtin tools with no client content now return a clear
error instead of a broken dispatch attempt.
WARNING 1 (chat.el, agentic_loop): Wire always_allow_<session_id> state
into the bridge-suspension decision. When a tool is in the session's
always-allow list, treat it as locally dispatchable (like a builtin)
and skip the bridge pause, so the approval UI is never shown again for
that tool in that session.
WARNING 2 (sessions.el, legacy path): Read a "tools_variant" field from
the legacy pending blob when present, and call the corresponding
agentic_tools_*() variant on resume. Falls back to agentic_tools_literal()
for blobs written before this field existed.
tests/test_sessions_approve.el: Add 10-case test suite covering:
- empty session_id / missing call_id / missing action guards
- no pending tool returns correct error
- missing tool_name on allow returns error (BLOCKER 1)
- deny action does not require tool_name
- legacy call_id mismatch returns mismatch error
- always action records tool_name in always_allow state
- allow with client content skips re-execution (BLOCKER 2)
handle_chat_agentic was calling agentic_tools_with_web(), which omits
MCP connector tools, so mcp__* calls were never available in agentic
mode even when neuron-connectd is running.
Switch both agentic entry points to agentic_tools_all(). For
handle_dharma_room_turn_agentic, also replace the inline 8-iteration
loop with a call to agentic_loop() so bridge suspension and the full
connector tool set work consistently. Session IDs are prefixed with
'dharma:' + room_id so suspensions stay room-scoped.
bridge_save was wrapping messages and tools_json with json_safe() before
storing them as string fields. Since both are already well-formed JSON arrays
containing double quotes, json_safe added a second escape layer. agentic_resume
then called json_get() which stripped only one layer, leaving the messages array
corrupted before it was passed back into agentic_loop.
Fix: store messages as messages_raw and tools_json as tools_raw as inline raw
JSON values (unquoted), and read them back with json_get_raw. Backward
compatibility: fall back to the old string-escaped fields if the raw fields are
absent, so sessions saved before this fix can still be resumed.
Also fixes write_file returning a pre-escaped literal instead of calling
json_safe consistently with every other tool result.
will.anderson
Tim Lingo