From 8cdd1512d1fb4c41b745669feaf30ab244455429 Mon Sep 17 00:00:00 2001 From: Tim Lingo <1timlingo@gmail.com> Date: Mon, 13 Jul 2026 13:40:07 -0500 Subject: [PATCH 1/5] =?UTF-8?q?docs(narrated-runs):=20engine=20notes=20for?= =?UTF-8?q?=20the=20regen=20=E2=80=94=20compiled-form=20fixes=20+=20debts?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 --- docs/NARRATED-RUNS-ENGINE-NOTES-20260713.md | 34 +++++++++++++++++++++ 1 file changed, 34 insertions(+) create mode 100644 docs/NARRATED-RUNS-ENGINE-NOTES-20260713.md diff --git a/docs/NARRATED-RUNS-ENGINE-NOTES-20260713.md b/docs/NARRATED-RUNS-ENGINE-NOTES-20260713.md new file mode 100644 index 0000000..a5a64cf --- /dev/null +++ b/docs/NARRATED-RUNS-ENGINE-NOTES-20260713.md @@ -0,0 +1,34 @@ +# Narrated runs — engine notes for Will (2026-07-13) + +Source half: commit aa67f86 on feat/agent-phase1-soul (run-progress ledger, +`/api/run-progress/` route, narration on the pause envelope, config display +default). E2E-verified via the compiled test bed on Tim's clean profile. + +Compiled-form-only fixes (in `neuron-container-build/soul-narrated-runs-20260713.patch`, +applies ON TOP of `soul-webfix-20260711.patch` — these need porting to chat.el when the +webfix itself is ported): + +1. **pause_turn + tool_use interleave**: a pause_turn response can ALSO carry a client + tool_use; resuming verbatim leaves it unpaired → Anthropic 400 "tool_use ids were + found without tool_result". Fix: tool-bearing pause rounds are tool turns + (dispatch + pair); verbatim resume only when the round has no client tool. +2. **Agentic toolset scope**: agentic_tools_all() fed EVERY connector/MCP tool (Notion, + code-execution…) into the loop. Code-execution flips the API into programmatic + tool calling, whose pairing protocol the single-tool manual loop does not speak — + source of the dangling-pair 400s AND the bash_code_execution workspace-dodge. + Fix: handle_chat_agentic declares builtins + ONE server web_search only. + Connector tools return when the loop gains real multi-tool/programmatic support. +3. **disable_parallel_tool_use: true** on agentic requests — the loop captures only the + first tool_use per round; Opus-class models parallel-call. Enforce the invariant. +4. **web_search server-tool default variant → web_search_20250305 (GA)**. The 20260209 + variant couples to code-execution ⇒ programmatic mode (see #2, and the June note: + "inert unless code-execution attached"). +5. **Homegrown web_search removed** from the tool catalog (server-side is the one tool). + +Known engine debts this work surfaced (not fixed): + +- **Poisoned session history**: a failed run persists the malformed assistant turn; every + later turn in that session replays it and 400s. Needs history sanitation on load. +- **Huge-history invalid-escape 400** (~346KB request) — likely the same poisoned blob. +- **macOS note**: replacing a binary in place invalidates its ad-hoc signature (instant + silent SIGKILL, looks like exit 0). `rm + cp + codesign -f -s -` is the swap ritual. From 9a6014d65b2b7496acd2d2763a4b992be98a99da Mon Sep 17 00:00:00 2001 From: Tim Lingo <1timlingo@gmail.com> Date: Thu, 16 Jul 2026 22:26:34 -0500 Subject: [PATCH 2/5] =?UTF-8?q?fix(engine):=20honor=20require=5Fapproval?= =?UTF-8?q?=20=E2=80=94=20the=20pause=20contract,=20implemented=20(PAUSE-C?= =?UTF-8?q?ONTRACT=20+=20BUG-LEAK=20source=20fixes)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two consent-flow fixes, gates only get stricter: 1. PAUSE-CONTRACT: the client has sent require_approval:true on every agentic request since Phase 1c, but needs_bridge never consulted it — builtin sub-escalate tools ran server-side unasked, making the app's Ask autonomy silently inert for that whole class. Now the flag is persisted per session (set/reset every request, so /approve resumes keep it) and ask_all bridges EVERY tool turn. Absent/false = behavior byte-identical to before. E2E: the exact probe that executed a write unasked now returns the tool_pending envelope with nothing on disk; full in-app circle verified (card → crash → resurrection → late approve → fence re-fires on re-entry). 2. BUG-LEAK: agent_workspace_root lived in ONE shared state key — any request that omitted a root inherited the previous session's folder (proven: a rootless curl session wrote into another session's run folder). Root is now stored per session and every request re-asserts its own (possibly empty) root into the shared key the guards read; same re-assert on the /approve and resume paths. Env fallback intact. LIMITATION: assumes serialized handling; true per-call scoping means threading session_id through dispatch — flagged for review. Runnable C-patch for the test brain: neuron-container-build/ soul-pause-contract-20260716.patch (pause-contract only; the leak fix needs the #23 root-write which the running C predates — source carries both for the regen). Co-Authored-By: Claude Fable 5 --- chat.el | 39 ++++++++++++++++++++++++++++++++++++++- sessions.el | 5 +++++ 2 files changed, 43 insertions(+), 1 deletion(-) diff --git a/chat.el b/chat.el index 1fc12ed..23ec5f0 100644 --- a/chat.el +++ b/chat.el @@ -1935,8 +1935,24 @@ fn handle_chat_agentic(body: String) -> String { // no root (or cleared the field), and we must not overwrite a server-configured root // from NEURON_AGENT_ROOT with an empty string, which would silently un-scope the agent. let ws_root: String = json_get(body, "agent_workspace_root") + // BUG-LEAK fix (2026-07-16): the root used to live ONLY in the shared key, so any + // request that omitted it INHERITED the previous session's folder (proven: a rootless + // curl session wrote into another session's run folder). Now each session keeps its + // own copy, and every request RE-ASSERTS its own root (possibly empty) into the shared + // key the tool guards read — no session can ever act under another session's root. + // Empty state still falls through to env NEURON_AGENT_ROOT inside + // agent_workspace_root(), so a server-configured root survives unchanged. + // LIMITATION (for review): assumes serialized request handling; true per-call scoping + // means threading session_id through dispatch_tool/classify — deeper change, Will's call. + let sess_for_root: String = json_get(body, "session_id") if !str_eq(ws_root, "") { + if !str_eq(sess_for_root, "") { + state_set("agent_workspace_root_" + sess_for_root, ws_root) + } state_set("agent_workspace_root", ws_root) + } else { + let own_root: String = if str_eq(sess_for_root, "") { "" } else { state_get("agent_workspace_root_" + sess_for_root) } + state_set("agent_workspace_root", own_root) } // L1 safety screen — agentic path must pass the same gate as layered_cycle. @@ -2066,6 +2082,14 @@ fn handle_chat_agentic(body: String) -> String { // Use caller-supplied session_id if provided, otherwise generate a bridge id. let session_id: String = if str_eq(req_session, "") { next_bridge_id() } else { req_session } + // PAUSE-CONTRACT fix (2026-07-16): honor the client's require_approval field — the + // Phase 1c contract ("the soul pauses on EVERY tool; the client's tier gate decides + // what actually prompts") was never implemented engine-side, which made the client's + // Ask autonomy silently inert for builtin sub-escalate tools. Persisted per session + // (set/reset on every request) so the /approve resume path keeps the same behavior + // for the rest of the run. Absent/false = behavior identical to before this fix. + let req_ask_all: String = json_get(body, "require_approval") + state_set("require_approval_" + session_id, if str_eq(req_ask_all, "true") { "true" } else { "" }) // Provider fork: OpenAI-compatible providers (Ollama/OpenAI/Grok/Gemini) take the plain-completion // path (v1, no tools); everything else stays on the Anthropic agentic loop (the default). let use_openai: Bool = !str_eq(llm_base_url(), "") && str_eq(llm_wire_format(), "openai") @@ -2134,6 +2158,12 @@ fn handle_chat_agentic(body: String) -> String { fn agentic_loop(session_id: String, model: String, safe_sys: String, tools_json: String, messages_in: String, h: Map, tools_log_in: String) -> String { let api_url: String = "https://api.anthropic.com/v1/messages" + // PAUSE-CONTRACT fix (2026-07-16): when the client asked to approve every action + // (require_approval on the request, persisted per session), EVERY tool turn bridges — + // the client's tier gate decides what actually prompts vs auto-continues. Read from + // session state so the /approve resume re-entry keeps the same behavior mid-run. + let ask_all: Bool = !str_eq(session_id, "") && str_eq(state_get("require_approval_" + session_id), "true") + let messages: String = messages_in let final_text: String = "" let tools_log: String = tools_log_in @@ -2220,7 +2250,10 @@ fn agentic_loop(session_id: String, model: String, safe_sys: String, tools_json: // confirm). Escalated calls suspend to the client's consent flow; the // /approve round-trip is the only path that executes them. let risk_tier: String = if is_tool_turn { classify_tool_risk(tool_name, tool_input) } else { "" } - let needs_bridge: Bool = is_tool_turn && (str_eq(risk_tier, "escalate") || (!is_builtin_tool(tool_name) && !is_always_allowed)) + // PAUSE-CONTRACT fix (2026-07-16): ask_all bridges EVERYTHING — stricter only. + // Escalate keeps its unconditional bridge; "always allow" shortcuts never apply + // under ask_all (the client owns its own standing grants at its tier gate). + let needs_bridge: Bool = is_tool_turn && (ask_all || str_eq(risk_tier, "escalate") || (!is_builtin_tool(tool_name) && !is_always_allowed)) // Built-in tools dispatch locally; bridged tools yield "" (never sent upstream). let tool_result_raw: String = if is_tool_turn && !needs_bridge { dispatch_tool(tool_name, tool_input) } else { "" } @@ -2360,6 +2393,10 @@ fn agentic_resume(session_id: String, tool_use_id: String, content: String) -> S if str_eq(blob, "") { return "{\"error\":\"unknown session_id\",\"reply\":\"\"}" } + // BUG-LEAK fix (2026-07-16): re-assert THIS session's own workspace root before the + // loop continues — a resume must never run under whatever root the last unrelated + // request happened to leave in the shared key. + state_set("agent_workspace_root", state_get("agent_workspace_root_" + session_id)) let model: String = json_get(blob, "model") let safe_sys: String = json_get(blob, "safe_sys") diff --git a/sessions.el b/sessions.el index c5c57e9..1ccc0f6 100644 --- a/sessions.el +++ b/sessions.el @@ -677,6 +677,11 @@ fn handle_session_approve(session_id: String, body: String) -> String { // path for all sessions created through handle_chat_agentic / agentic_loop. let bridge_blob: String = state_get("mcp_bridge:" + session_id) if !str_eq(bridge_blob, "") { + // BUG-LEAK fix (2026-07-16): the approved tool executes below via dispatch_tool, + // whose path/command guards read the shared workspace-root key. Re-assert THIS + // session's own root first — an approval must never execute under whatever root + // the last unrelated request left behind. + state_set("agent_workspace_root", state_get("agent_workspace_root_" + session_id)) // For "always": record tool_name in the always-allow list before resuming. // The tool_name is not stored in the bridge blob (only tool_use_id is). // Accept it from the body so the client can pass it along. From 4171aadfffdaeb0fcc188f72806e7c65000a8ed3 Mon Sep 17 00:00:00 2001 From: Tim Lingo <1timlingo@gmail.com> Date: Thu, 16 Jul 2026 23:02:07 -0500 Subject: [PATCH 3/5] =?UTF-8?q?fix(engine):=20BUG-6=20=E2=80=94=20approved?= =?UTF-8?q?=20writes=20must=20land,=20and=20say=20where=20(false-receipt?= =?UTF-8?q?=20kill)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two compounding defects made every pause->approve write_file report success while writing NOTHING: 1. The naive json_get scanner matches "content" anywhere in the approve body — including INSIDE tool_input, which for write_file always carries a content field. The handler therefore treated every approved builtin write as already-client-executed, skipped dispatch entirely, and handed the model the file's own content as the 'tool result'. The model then narrated 'Done, created' — a false receipt with no file. Builtin tools now ALWAYS dispatch server-side; client content is only honored for non-builtin (MCP/client-executed) tools. Stricter only. 2. write_file returned {"ok":true} unconditionally — fs_write's outcome was never checked, so any failed write also reported success. The write now verifies the file landed (fs_exists) and returns the RESOLVED path in the ok payload; failures return a real error naming the destination. E2E on the test brain (boot 38): approve-path write lands byte-exact and the result carries the resolved path; auto-run writes unchanged; denied writes execute nothing. BUG-5 (approve wire lacked tool_name) had been masking this one — two stacked bugs on the same path. NOTE for review: the deeper cure is a nesting-aware json reader; this fix removes the dangerous consequence at the two spots that lie about disk. Co-Authored-By: Claude Fable 5 --- chat.el | 12 ++++++++++-- sessions.el | 8 +++++++- 2 files changed, 17 insertions(+), 3 deletions(-) diff --git a/chat.el b/chat.el index 23ec5f0..f29ee92 100644 --- a/chat.el +++ b/chat.el @@ -1680,8 +1680,16 @@ fn dispatch_tool(tool_name: String, tool_input: String) -> String { if !path_within_root(path, root) { return json_safe("denied: path is outside the agent workspace root") } - fs_write(resolve_in_root(path, root), content) - return json_safe("{\"ok\":true}") + // BUG-6 fix (2026-07-17): never claim ok without disk truth. fs_write's result was + // never checked, so a failed write reported ok — the exact false-receipt failure + // the run guards exist to kill. Verify the file landed and return the RESOLVED + // path so callers and the model can only narrate what is really on disk. + let dest: String = resolve_in_root(path, root) + fs_write(dest, content) + if !fs_exists(dest) { + return json_safe("{\"error\":\"write failed - nothing landed at " + dest + "\"}") + } + return json_safe("{\"ok\":true,\"path\":\"" + dest + "\"}") } if str_eq(tool_name, "web_get") { let url: String = json_get(tool_input, "url") diff --git a/sessions.el b/sessions.el index 1ccc0f6..6432b81 100644 --- a/sessions.el +++ b/sessions.el @@ -713,7 +713,13 @@ fn handle_session_approve(session_id: String, body: String) -> String { // For builtin tools with no client-provided content: fall back to // dispatch_tool so those tools still execute correctly. let client_content: String = json_get(body, "content") - let use_client_content: Bool = !str_eq(client_content, "") + // BUG-6 fix (2026-07-17): the naive json_get scanner matches "content" ANYWHERE + // in the body — including INSIDE tool_input — so every approved write_file (whose + // input always carries a content field) was mistaken for client-executed, never + // dispatched, and narrated as done: a false receipt with no file on disk. Builtin + // tools now ALWAYS dispatch server-side; client content is only honored for + // non-builtin (MCP/client-executed) tools. Stricter only. + let use_client_content: Bool = !str_eq(client_content, "") && !is_builtin_tool(approve_tool_name) let use_dispatch: Bool = is_builtin_tool(approve_tool_name) && !use_client_content let raw_input: String = json_get_raw(body, "tool_input") let eff_input: String = if str_eq(raw_input, "") { "{}" } else { raw_input } From f0454650a20bdc3bc6653d557506e5233745a981 Mon Sep 17 00:00:00 2001 From: Will Anderson Date: Sat, 18 Jul 2026 13:59:02 -0500 Subject: [PATCH 4/5] ci: harden soul-contract-gate boot for the Linux runner MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The gate booted the soul with `env -i`, which strips the ambient environment — including the library path the dynamically-linked soul needs on the GCE CI runner. The soul never came up there, so the gate failed the build (run 3384) even though the soul is correct (the gate passes locally against the exact published CI runtime). Switch to preserving the ambient env and instead UNSET only the live-service vars (ENGRAM_URL/API keys/ identity) while pointing HOME + snapshot at throwaway paths and axon at a dead port. Isolation is unchanged (verified: no touch of the live soul/engram); the soul now boots on the runner. --- scripts/verify-soul-contract.sh | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/scripts/verify-soul-contract.sh b/scripts/verify-soul-contract.sh index c120920..815df39 100755 --- a/scripts/verify-soul-contract.sh +++ b/scripts/verify-soul-contract.sh @@ -113,9 +113,16 @@ KNOWN_PENDING=( ) # --- boot the soul ----------------------------------------------------------- +# Preserve the ambient environment (PATH, LD_LIBRARY_PATH, TMPDIR) so the +# dynamically-linked soul finds its libs on any runner — using `env -i` here +# stripped the library path on the Linux CI runner and the soul never booted. +# Isolation is still guaranteed by UNSETTING the live-service vars (so it can +# never reach the real engram/axon or make an LLM call) and by pointing HOME + +# the snapshot at throwaway paths and the axon at a dead port. echo "== booting soul: $SOUL on port $PORT (throwaway HOME=$THROW_HOME) ==" -env -i \ - PATH="/usr/bin:/bin:/usr/sbin:/sbin" \ +env \ + -u ENGRAM_URL -u ENGRAM_API_KEY -u SOUL_ENGRAM_URL \ + -u ANTHROPIC_API_KEY -u NEURON_LLM_API_KEY -u SOUL_IDENTITY \ HOME="$THROW_HOME" \ NEURON_PORT="$PORT" \ SOUL_CGI_ID="ntn-contract-$$" \ From 192b277229bc482f82e18dcc7c77065bcc4b60d9 Mon Sep 17 00:00:00 2001 From: Will Anderson Date: Sat, 18 Jul 2026 14:10:48 -0500 Subject: [PATCH 5/5] ci: link the Linux soul with -rdynamic so its http handler resolves MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Run 3388's gate failed with every route returning "el-runtime: no http handler registered". The runtime resolves handle_request (and the tool handlers) by name via dlsym(RTLD_DEFAULT, ...). On glibc/Linux a symbol is only visible to dlsym if it is in the dynamic symbol table, so the stripped CI binary booted but served nothing. macOS exports these freely, which is why the local build passed and masked it. Add -rdynamic to the cc link (mirrors the Windows build's --export-all-symbols). strip -s keeps .dynsym, so the handler still resolves after stripping. This fixes both the gate AND the actual deployed soul — without it the Linux/GKE soul is a server that answers nothing. --- .gitea/workflows/ci.yaml | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/.gitea/workflows/ci.yaml b/.gitea/workflows/ci.yaml index 5e13896..96a804d 100644 --- a/.gitea/workflows/ci.yaml +++ b/.gitea/workflows/ci.yaml @@ -94,16 +94,23 @@ jobs: # entirely: elb on Linux would OOM the runner (elc uses 24GB+ virtual memory # on a 16GB host) and we always restore from the repo's soul.c anyway. mkdir -p dist - cc -O2 -DHAVE_CURL \ + # -rdynamic: the el runtime resolves the HTTP request handler (and the + # tool handlers) by NAME via dlsym(RTLD_DEFAULT, "handle_request"). + # macOS exports these symbols freely, but glibc/Linux only makes symbols + # visible to dlsym if they are in the dynamic symbol table — so without + # -rdynamic the stripped Linux binary boots but returns "el-runtime: no + # http handler registered" for EVERY route (i.e. a soul that serves + # nothing). Same reason the Windows build links -Wl,--export-all-symbols. + cc -O2 -DHAVE_CURL -rdynamic \ -I$RUNTIME \ dist/soul.c \ $RUNTIME/el_runtime.c \ -lssl -lcrypto -lcurl -lpthread -lm \ -o dist/neuron - # Strip debug symbols and non-essential symbol table entries. - # -s removes the symbol table + relocation info (max size reduction). - # Keeps the binary functional; debuggability is preserved via source + CI logs. + # -s strips .symtab + debug for size. .dynsym (which -rdynamic populated + # with the dlsym-resolved handlers) is preserved, so the handler still + # resolves after stripping. strip -s dist/neuron ls -lh dist/neuron