Compare commits

..

6 Commits

Author SHA1 Message Date
Tim Lingo 8f3a478771 fix(engine): excise the receipt, do not truncate at it — a leading receipt was erasing whole answers
CAUGHT BY A/B, AND ONLY BY A/B. The previous commit's receipt_strip assumed the receipt is
always TERMINAL and cut everything from the marker onward. It is not always terminal: once
receipt_rule told the model what [[RECEIPT ...]] means, the model sometimes LED with one and
wrote the answer underneath. Cutting at the marker then deleted the entire answer and the
turn returned {"error":"no response"}.

MEASURED, same prompt (two web searches, cited prose), fresh session each run:
    round-7 brain   4 / 4 answered   (471, 473, 473, 544 chars)
    round-8 brain   2 / 7 answered   (five {"error":"no response"})
This looked exactly like a flaky model. It was not — it was mine. Running the two brains
side by side on the same prompt is the only reason it was found, and it is the reason the
A/B is now part of how this class gets tested.

AFTER THE FIX, same protocol:
    round-7 brain   4 / 4   (473, 473, 473, 544)
    round-8 brain   4 / 4   (657, 657, 657, 673)

THE FIX: remove the [[...]] span and keep BOTH sides, instead of truncating at the marker.
An unterminated marker at position 0 is left completely alone — no rule about receipts is
worth erasing an answer over. Bounded four-pass loop rather than a conditional exit, because
rebinding the counter inside an if-expression is the block-expression shape that miscompiles
integer arithmetic under this elc (BUG-PLAINCHAT-1). Verified in the generated C:
    str_slice(rest, (e + 2), str_len(rest))   <- integer addition, correct
    el_str_concat(head, tail)                 <- string concat, correct
and zero el_str_concat(<ident>, str_len(...)) sites across all 49 modules.

SEAM PROOF (FIX C) rides on the same runs — a real two-search cited answer, inspected byte
by byte, in BOTH failure directions:
  missing separator (the round-7 "to.Good", bytes 77 2e 47): 0 hits. Sentence boundaries
    measure 2e 20 4d — "." SPACE "M".
  over-separation (a cited sentence shattered across paragraphs): 0 hits. The answer is one
    continuous paragraph with its sentences intact, which is the direction a blanket
    separator would have broken.

BUILT: sha256 77115f2733e794c5bc4ad1f55b1acaf658f8f4a91cccd423a2d633d94a726cbc

Refs neuron#109

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 23:25:48 -05:00
Tim Lingo 9ea41eed78 fix(engine): the model was signing its own answers with our receipt — strip it
FOUND BY E2E, NOT BY REASONING. The previous commit's design note asserted the provenance
receipt "never reaches the user: it is appended to the history copy, not the reply." That
was FALSE, and only running the thing showed it. On the first live run against the built
DMG brain, two agentic turns out of two came back with

  "Your favourite colour is chartreuse and your project is called Perihelion.

   [[RECEIPT - recorded by the soul, not written by the model: no tools ran on this turn.]]"

— the receipt in the user-visible reply.

MECHANISM: the receipt is stored inside the assistant turn, and the agentic path replays
history VERBATIM as Anthropic message objects. So the model sees its own previous answers
ending in [[RECEIPT ...]] and does the obvious thing — it imitates the format and signs the
next answer the same way. The plain path did NOT leak, which is the tell: there, history is
rendered into the SYSTEM prompt as labelled lines rather than replayed as assistant turns,
and a model imitates its own turns far more readily than a transcript.

FIX, two layers, because one of them is not a guarantee:
  - receipt_rule() names the marker in both system prompts (plain and agentic): these lines
    are written by the system, read them as evidence, never write one. Reduces occurrence.
  - receipt_strip() truncates any [[RECEIPT ...]] out of model output before it becomes the
    reply — plain path in layered_generate, agentic path on final_text in agentic_loop.
    Deterministic. A guard that depends on the model choosing to obey is exactly the class of
    thing round 8 exists to stop shipping, so the instruction is the optimisation and the
    strip is the guarantee.
Placed ABOVE agentic_loop's empty-check on purpose: a turn whose entire output was an
imitated receipt has produced no answer, and must be reported as no answer.

The receipt stays in HISTORY, which is the whole point and is proven to work: asked "What
source did you use for that?" one turn after a live web_search, this brain answered
"I used Weather Underground (https://www.wunderground.com/weather/is/reykjav%C3%ADk) for the
current temperature in Reykjavik" — a real source, no apology. That is the false confession
dead, and it is dead BECAUSE the model can read the receipt.

BUILT: 887,112 bytes, sha256 54a2eff84d4fa44f8d2db6781dcf225df4b5075a8ec5f1058018bd40cc1af10b
BUG-PLAINCHAT-1 miscompile guard: zero sites.

Refs neuron#109

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 23:10:53 -05:00
Tim Lingo ff421d39f6 fix(engine): history keeps its provenance and its session — the false confession and the blank stare
DESIGN FIT: three of round 7's five defects share ONE root — the conversation-history
layer persists only {role, content}, discarding tool provenance, session scoping, and the
distinction between a real user turn and an internal utility call. Fixes A and B RESTORE
Will's design rather than extend it: his agentic path already scopes history per session,
the plain path never got it, and his own source carries the TODO admitting the resulting
race (chat.el, handle_chat: "process-global key; concurrent /api/chat requests without
session_id race on this read-append-write"). Fix C repairs one join Will wrote that was
correct for a year and one we added last week. E1/E2 are ours.

FIX A — tool provenance in history (kills the FALSE CONFESSION)
  Root cause, EXECUTED-verified: handle_chat_agentic recorded turns via hist_append, which
  emits {"role","content"} only. server_tool_use blocks, web_search_tool_result blocks and
  every citation were discarded, then replayed as text. On the next turn the model saw a
  data-rich answer with zero evidence a search had happened, and its own permanent rule
  ("never describe a search you did not perform") left one conclusion available: that it
  had fabricated the data. It apologised for a search it HAD run — four independent lines
  of evidence confirm the search was real. The defect is not the model's honesty. It is
  that we deleted the evidence and then asked it to account for itself.
  Change: agentic_loop accumulates the source URLs it already walks past (citations and
  web_search_tool_result content) and returns them as "sources"; handle_chat_agentic folds
  tools_used + sources into a receipt line stored WITH the assistant turn. Receipts are
  unconditional — a negative receipt ("no tools ran") is the other half of the guarantee,
  because "no evidence of a tool" and "evidence of no tool" were previously identical in
  the transcript. conv_history_block splits the receipt off before snipping so a long
  answer cannot truncate away the evidence. The user never sees it: it is appended to the
  history copy, not the reply.

FIX B — one history key for both paths (kills the BLANK STARE)
  Root cause, EXECUTED-verified: the agentic path keyed history on session_hist_<id>; the
  plain path was hard-wired to the process-global conv_history and never read session_id.
  One conversation, two buckets. Proven in the guest engram: the scoped node held exactly
  two turns starting at "Try again" while the earlier exchanges sat unscoped.
  Change: conv_hist_key/conv_hist_label are now the single definition, used by BOTH paths;
  session_id is threaded route -> layered_cycle -> layered_generate / conv_history_record.
  The 2-line fallback (plain path reads the agentic key) was REJECTED: it keeps the
  process-global bucket as a live write target, which is the bleed the TODO describes.
  Also found and closed while threading: layered_cycle read session_id from the state key
  "current_session_id", which is read here and WRITTEN NOWHERE in the entire source. It
  was unconditionally "", so TODO(reliability #4) — per-session steward continuity — was
  dead code that could never fire. It fires now.
  LAZY SESSION, decided explicitly: we create the session EAGERLY at the door (app half,
  ui#223) rather than migrating orphaned turns. Migration would copy the CONTENTS of a
  process-global bucket, possibly another conversation's, into a named session — the bleed,
  performed deliberately. Eager creation makes the situation impossible instead. Migration
  is deliberately not implemented and must not be added without solving provenance first.

FIX C — the two text-join seams ("to.Good", byte-verified 0x77 0x2e 0x47)
  Two bare `+` joins, written a year apart, had drifted into two answers to one question:
  within-response block joins (Will's, 2026-05-03, latent until server-side web_search
  began interleaving non-text blocks) and across-round joins (ours, 62af564).
  Change: one named rule, text_join_sep, at both sites. NOT a blanket separator — a cited
  answer splits MID-SENTENCE ("The current temperature is " + "86°F" + ", with "), so a
  blanket separator shatters every sourced sentence. The rule takes the one bit that
  distinguishes the cases: whether a NON-TEXT block intervened. Hoisting it also makes the
  fix verifiable in the shipped binary, which an inline `+` is not.

FIX E1 — utility generations stay out of the transcript
  Title generation ("Write a 3-6 word title...") and insight passes ran down the same plain
  door as a real message and were recorded as if the user had typed them; the same calls are
  the "model":"unknown" rows in usage.jsonl. is_utility_request reads an explicit utility
  flag from the app, with the __title__/__insight__ id prefixes as a fallback for older
  clients. Answered normally, never recorded.

FIX E2 — OPERATOR IDENTITY is scoped to tool-capable turns
  The block (env USER/HOME, closing "This is a hard rule") was prepended to EVERY system
  prompt including chat mode. On a Tools:Off turn there is no filesystem in reach, so it
  governed nothing and merely supplied the loudest fact in the prompt — which is why the
  model opened a fresh conversation with "You're test, on your machine at /Users/test".
  Hoisted to operator_identity_block() and gated on !chat_mode. Unchanged wherever a file
  or command tool can actually be reached.

ALSO: agentic_loop's per-session history persist had a second hand-rolled copy of
conv_history_persist with a different label expression, different salience scores and
different tags for the same node. Since both now derive the label from conv_hist_label and
engram_node_full upserts by label, two score policies were writing one node. Collapsed to
one writer.

BUILD NOTE: dist/elp-c-decls.h is force-included by the documented link recipe and carried
the OLD C arities, so it is updated here. This is the build-support header, NOT the stale
generated dist/soul.c — no dist/*.c was read or edited; all engine changes are .el source.
chat.elh/soul.elh are committed because a first-pass build against the old signatures FAILS
(measured); the other regenerated headers are reverted as unrelated churn.

BUILT: 887,000 bytes, sha256 d632b061ad75269d6adeb52578d030eaf49e895d91289d7f946b19c08450d728
Zero el_str_concat(<int>, str_len(...)) sites (the BUG-PLAINCHAT-1 miscompile guard).
web_search_20250305 and disable_parallel_tool_use both still present — PR #108's web search
and the ADR-0005 stopgap are intact.

Refs neuron#109 (builds on it), neuron#78 (Receipt Contract — the real fix A is a stopgap for)

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 22:59:54 -05:00
Tim Lingo 635f6febe4 feat(engine): plain chat generates at L3 — inside the safety cycle, not around it
Neuron Soul CI / build (pull_request) Failing after 12m8s
Neuron Soul CI / deploy (pull_request) Has been skipped
Non-agentic /api/chat (the desktop app's default "Tools: Off" mode) returned the
user's own screened text as a bare non-JSON string. Every JSON client failed to
parse it and showed "Couldn't reach Neuron - it may be offline."

Root cause: f52d5bd (2026-06-11) correctly moved the route onto the layer spine
(handle_chat -> layered_cycle), but L3 never got a generator — imprint_respond()
annotates its input and returns it. Two pieces of the architecture were already
waiting for that step: layered_cycle parks a bell directive in the state key
build_system_prompt is written to consume, and build_system_prompt carries a
chat_mode ("no tools") flag with no live caller.

The fix composes rather than replaces. Wiring handle_chat would have removed
safety_screen, the hard-bell short-circuit, the whole stewardship layer and
safety_validate — the only enforcing output gate in the codebase — in exchange
for a working reply (see _engine-websearch-20260804/SAFETY-STOP.md). Instead
layered_cycle keeps every gate, in order, and gains a generation step between
imprint_respond and safety_validate.

  L1 screen -> guard -> hard-bell short-circuit -> L2a -> L2b -> L2c
    -> L3 imprint_respond (prompt) -> L3b layered_generate (NEW) -> L1 validate

- chat.el:  NEW layered_generate (L3 generation, no tools offered),
            conv_history_block, conv_history_record.
            FIX build_system_prompt never concatenated no_tools_rule into its
            return — the "[NO TOOLS THIS TURN]" rule reached no model at all.
            handle_chat annotated DO-NOT-WIRE with the reason.
- soul.el:  layered_cycle gains L3b + post-validation turn bookkeeping.
- routes.el: NEW plain_chat_envelope; all three /api/chat dispatch sites wrap the
            cycle's output. Built OUTSIDE the cycle so safety_validate always sees
            raw model text — nothing to unwrap or rebuild on the crisis path.
            Emits both `reply` and `response`: the desktop app reads `reply`,
            the CLI tools and telegram-gateway read `response`.

Also fixes BUG-PLAINCHAT-1, a pre-existing CRITICAL crash on the crisis path.
elc compiles `let n: Int = pos + str_len(marker)` to el_str_concat() — string
concat on two integers — inside a block-expression initializer, segfaulting the
daemon (SIGSEGV in strlen). Six inline copies of the same " | ts:" parser had it:
two in layered_cycle L2c, two in engram_compile (live on the AGENTIC path too),
two in affective_context_prefix. A distress turn following an earlier affective
turn killed the whole process. Proven pre-existing: an unmodified baseline binary
crashes identically, and the same bad C is in the committed dist/soul.c. Fixed by
hoisting to one top-level function, affective_node_ts(), where the expression
compiles to integer addition — verified in the generated C.

Proof (throwaway HOME/engram, explicit NEURON_PORT, live chain untouched):
- Plain turn returns a JSON envelope with the provider's answer, not an echo.
- Captured request body: no `tools`, no `tool_choice`; system prompt carries the
  NO-TOOLS rule. Tools:Off means no tool is offered, structurally.
- Hard bell: canned 988 message, and the provider request count does not move —
  the message never reaches a model.
- Soft bell + a 2-char model reply: safety_validate's care phrase is appended to
  the MODEL's output. Output gate acting, on this route.
- The L1 bell directive now reaches the model here for the first time (the state
  addendum had a producer and no consumer).
- test_layered_cycle PASS; all six El suites byte-identical to baseline.
- verify-soul-contract.sh (bash 5.3): GATE PASS, 27/27, immutability PASS.
- The crash sequence that killed the baseline daemon now returns HTTP 200.

Not proven: no live Anthropic call — the login keychain refuses the key to a
non-interactive process (rc=24 errSecInteractionNotAllowed). Details and the
one-command close-out are in _engine-plainchat-20260805/README.md §7.

Builds on PR #108. dist/soul.c deliberately not regenerated — Will's toolchain.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 09:11:03 -05:00
Tim Lingo 62af5649fe feat(engine): port Anthropic server-side web_search into the agentic loop
Neuron Soul CI / build (pull_request) Failing after 10m45s
Neuron Soul CI / deploy (pull_request) Has been skipped
Re-authors soul-webfix-20260711.patch in El (the patch is a diff against
generated C at month-old offsets, so nothing was applied as a patch). Its
last two hunks — an unrelated /api/safety-contact implementation — were
deliberately not ported; that route already exists and is safety-critical.

Activation restores existing design, it does not invent a mechanism:
commit 8eea1d9 (2026-06-09, Tim-approved) made native web_search built-in
with no user-facing toggle, and tests/test_agentic_tools.el section 2 still
asserts agentic_tools_all() contains it — an assertion main currently fails.
The call site was lost when agentic_tools_all() (connector tools, PR #19)
replaced agentic_tools_with_web(). Attaching in agentic_tools_all() covers
handle_chat_agentic, handle_dharma_room_turn_agentic and agentic_resume.

pause_turn handling is included and was genuinely missing: the shipped
binary has zero occurrences of it. Without it a paused server-side search
returns only the text written so far and the loop treats it as final —
a silently truncated answer. final_text now accumulates across resume
cycles rather than overwriting (overwriting would discard everything
written before the pause).

Default tool version is web_search_20250305, NOT the newer _20260209, and
that is a measured choice: _20260209's dynamic filtering uses server-side
programmatic tool calling, which the API refuses to combine with ADR 0005's
stopgap —

  HTTP 400 invalid_request_error
  tool_choice.disable_parallel_tool_use: true cannot be used with
  programmatic tool calling

Dropping the stopgap would resurrect neuron#78 bug b (killed runs 3/3 in
ADR 0005's own A/B). The basic variant is compatible with the stopgap and
returns real results, so neither feature is dropped. Version lives in state
key web_search_tool_version; flipping it once the stopgap retires is a
config write, no recompile. The fallback also fires on "programmatic tool
calling" so a premature flip self-heals loudly instead of dying.

Also fixes a real bug this port exposed: json_get is a first-match scanner
and a cited text block serialises citations FIRST, so json_get(block,"type")
returned the nested citation's type and every citation-bearing block — the
ones carrying the searched facts — was silently dropped from the reply.
Reply length on the same question: 79 -> 360 chars.

Other loop changes: server_tool_use accounting into tools_used, iteration
cap 8->12 for pause/resume cycles, max_tokens 4096->16384, container-id
carry-forward, API error head logged instead of swallowed, tools_used gated
on is_tool_turn so a truncated tool block is not reported as work done.

dist/soul.c is deliberately NOT regenerated — the local elc predates Will's
last regen (e610a41) and regenerating with an older compiler risks unrelated
codegen drift. Source only; regen is Will's.

E2E-VERIFIED on a sandbox soul (scratch HOME/engram, dead axon+ISE, explicit
NEURON_PORT): live Bentonville weather with tools_used ["web_search"]; the
27-route contract gate PASSes; the parallel-tool stopgap still completes a
3-file mission with no 400; a 3-search 3,486-char answer kept its end
sentinel. Honest gap: pause_turn is compiled in but not exercised by a live
pause (max_uses:5 caps the server loop below the pause threshold).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-04 22:57:59 -05:00
Tim Lingo 710761e2d5 fix(engine): send tool_choice.disable_parallel_tool_use on agentic loop (STOPGAP)
The agentic loop keeps only the FIRST tool_use block per round (chat.el:2281,
"Capture first tool_use block only"). Anthropic lets a model emit several
tool_use blocks in one message and requires a tool_result for every one, so a
parallel-tool turn is answered once, the rest are dropped, and the next request
dies with:

  tool_use ids were found without tool_result blocks immediately after

(neuron#78 quotes this as "tool_use ids found without tool_result"; the above is
the API's actual wording - recorded so the next person's grep matches.)

This constrains the wire to match what the loop can assemble:
  "tool_choice":{"type":"auto","disable_parallel_tool_use":true}

STOPGAP - AND THE DURABLE FIX ALREADY EXISTS. A correct multi-tool loop is
already in Will's EL runtime, in C, and the soul does not call it. Verified on
el:origin/main lang/el-compiler/runtime/el_runtime.c: llm_register_tool:9616,
llm_build_tool_results:9743 - which walks EVERY content block, emits one
tool_result per tool_use, and sets is_error for an unregistered tool -
llm_call_agentic:9817 calling it at :9918, iteration cap 10 at :9847. Will's
commit 12d5e77 (2026-04-30). grep for llm_call_agentic/llm_register_tool across
every neuron/*.el returns nothing; dist/soul.c has zero references. chat.el
hand-rolls its own single-tool loop instead, and that is the one that breaks.

The durable fix is therefore to register the soul's tools via llm_register_tool
and call llm_call_agentic - deleting a loop, not writing one. See ADR 0005.

Our own approved spec called this seven weeks ago:
docs/research/agentic-tool-approval-design.md (2026-06-12, "Approved for build"),
line 20 on the defect, line 30 on the goal ("Execute all tool_use blocks in a
turn (one result per block)").

Two edits, because dist/soul.c cannot be regenerated here (Will's gated elc/elb
toolchain is not on this machine):
  (a) chat.el:2255 - source of truth, so a later regen carries the fix.
      One edit covers all three routes: agentic_loop is called from chat.el:2152
      (/api/chat agentic), :2676 (dharma room) and :2496 (agentic_resume).
  (b) dist/soul.c:28173 - generated form, hand-spliced. Line 27624 is the
      non-agentic/OpenAI-compat req_body (no tools) and was left untouched.
Prior art reused rather than reinvented: soul-narrated-runs-20260713.patch
(27,824 bytes) line 78 spliced this same string into the same concat chain on
2026-07-13.

Deliberately NOT ported from that patch, having read it: max_tokens is not
changed by it (16384 sits on both sides of the hunk; our main's 4096 is a
separate output-truncation concern), and its pause_turn pairing fix - same
defect class - is unreachable today because no server-side web_search is wired
(agentic_tools_with_web at chat.el:1418 is never called), so it is untestable
and logged instead.

Proven E2E on a scratch profile and port 7791, never the live chain. A/B against
a pristine origin/main control built from the same vendored runtime: fixed
completed the mission (tools_used read_file x3, 4 iterations, correct answer);
control failed 3/3. Direct API probe confirmed the mechanism - without the field
the model emits 3 parallel tool_use blocks and replaying the unfixed loop's next
turn returns HTTP 400; with it, exactly 1 block.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-03 17:07:04 -05:00
8 changed files with 1079 additions and 177 deletions
+892 -128
View File
File diff suppressed because it is too large Load Diff
+26 -3
View File
@@ -1,4 +1,4 @@
// auto-generated by elc --emit-header do not edit
// auto-generated by elc --emit-header - do not edit
extern fn chat_default_model() -> String
extern fn engram_numeric_valid(s: String) -> Bool
extern fn parse_float_x100(s: String) -> Int
@@ -16,18 +16,35 @@ extern fn engram_nodes_merge(a: String, b: String) -> String
extern fn id_in_seen(node_id: String, seen: String) -> Bool
extern fn add_to_seen(seen: String, node_id: String) -> String
extern fn engram_extract_ids(nodes_json: String) -> String
extern fn affective_node_ts(node_json: String) -> Int
extern fn engram_compile(intent: String) -> String
extern fn distill_transcript(transcript: String) -> String
extern fn json_safe(s: String) -> String
extern fn current_engine_note(model: String) -> String
extern fn bounded_persona_floor() -> String
extern fn operator_identity_block() -> String
extern fn build_system_prompt(ctx: String, chat_mode: Bool) -> String
extern fn hist_append(hist: String, role: String, content: String) -> String
extern fn conv_hist_key(session_id: String) -> String
extern fn conv_hist_label(session_id: String) -> String
extern fn is_utility_request(body: String, session_id: String) -> Bool
extern fn provenance_scan_urls(arr: String, acc: String) -> String
extern fn provenance_add_sources(block: String, btype: String, has_cit: Bool, cit_raw: String, acc: String) -> String
extern fn provenance_names(tools_used: String) -> String
extern fn text_join_sep(accumulated: String, incoming: String, after_interruption: Bool) -> String
extern fn receipt_rule() -> String
extern fn receipt_strip(s: String) -> String
extern fn tool_receipt(tools_used: String, sources: String) -> String
extern fn hist_trim(hist: String) -> String
extern fn hist_trim_with_bell_guard(hist: String) -> String
extern fn clean_llm_response(s: String) -> String
extern fn conv_history_persist(hist: String) -> Void
extern fn conv_history_load() -> String
extern fn conv_history_persist(session_id: String, hist: String) -> Void
extern fn conv_history_load(session_id: String) -> String
extern fn conv_history_record(session_id: String, user_msg: String, assistant_msg: String, receipt: String) -> Void
extern fn conv_history_block(session_id: String) -> String
extern fn layered_generate(prompt: String, imprint_id: String, session_id: String) -> String
extern fn session_preload_bullets(nodes: String, max_bullets: Int, snip_len: Int) -> String
extern fn affective_context_prefix() -> String
extern fn handle_chat(body: String) -> String
extern fn handle_see(body: String) -> String
extern fn studio_tools_json() -> String
@@ -37,6 +54,8 @@ extern fn llm_wire_format() -> String
extern fn json_escape(s: String) -> String
extern fn openai_chat_complete(model: String, base_url: String, api_key: String, safe_sys: String, messages_json: String) -> String
extern fn agentic_tools_literal() -> String
extern fn web_search_tool_json() -> String
extern fn strip_client_web_search(tools_inner: String) -> String
extern fn agentic_tools_with_web() -> String
extern fn connector_tools_json() -> String
extern fn agentic_tools_all() -> String
@@ -46,6 +65,10 @@ extern fn call_neuron_mcp(tool_name: String, args: String) -> String
extern fn agent_workspace_root() -> String
extern fn path_within_root(path: String, root: String) -> Bool
extern fn resolve_in_root(path: String, root: String) -> String
extern fn run_command_is_readonly(cmd: String) -> Bool
extern fn cmd_abs_escape_at(cmd: String, root: String, needle: String) -> Bool
extern fn run_command_guard(cmd: String, root: String) -> String
extern fn classify_tool_risk(tool_name: String, tool_input: String) -> String
extern fn dispatch_tool(tool_name: String, tool_input: String) -> String
extern fn is_builtin_tool(tool_name: String) -> Bool
extern fn next_bridge_id() -> String
Generated Vendored
+17 -3
View File
@@ -5,6 +5,15 @@ el_val_t add_punct(el_val_t s, el_val_t intent);
el_val_t add_to_seen(el_val_t seen, el_val_t node_id);
el_val_t aff_try_slot(el_val_t slot_json, el_val_t aff_7d_ts, el_val_t acc_key);
el_val_t affective_context_prefix(void);
el_val_t is_utility_request(el_val_t body, el_val_t session_id);
el_val_t operator_identity_block(void);
el_val_t provenance_add_sources(el_val_t block, el_val_t btype, el_val_t has_cit, el_val_t cit_raw, el_val_t acc);
el_val_t provenance_names(el_val_t tools_used);
el_val_t provenance_scan_urls(el_val_t arr, el_val_t acc);
el_val_t text_join_sep(el_val_t accumulated, el_val_t incoming, el_val_t after_interruption);
el_val_t receipt_rule(void);
el_val_t receipt_strip(el_val_t s);
el_val_t tool_receipt(el_val_t tools_used, el_val_t sources);
el_val_t agent_number(el_val_t agent);
el_val_t agent_person(el_val_t agent);
el_val_t agent_workspace_root(void);
@@ -151,8 +160,12 @@ el_val_t cmd_abs_escape_at(el_val_t cmd, el_val_t root, el_val_t needle);
el_val_t connectd_get(el_val_t suffix);
el_val_t connectd_post(el_val_t suffix, el_val_t body);
el_val_t connector_tools_json(void);
el_val_t conv_history_load(void);
el_val_t conv_history_persist(el_val_t hist);
el_val_t conv_hist_key(el_val_t session_id);
el_val_t conv_hist_label(el_val_t session_id);
el_val_t conv_history_block(el_val_t session_id);
el_val_t conv_history_load(el_val_t session_id);
el_val_t conv_history_persist(el_val_t session_id, el_val_t hist);
el_val_t conv_history_record(el_val_t session_id, el_val_t user_msg, el_val_t assistant_msg, el_val_t receipt);
el_val_t cop_article(el_val_t gender, el_val_t number, el_val_t definite);
el_val_t cop_bwk_future(el_val_t prefix);
el_val_t cop_bwk_perfect(el_val_t prefix);
@@ -782,7 +795,8 @@ el_val_t lang_profile_uga(void);
el_val_t lang_profile_zh(void);
el_val_t lang_profile(el_val_t code, el_val_t word_order, el_val_t morph_type, el_val_t has_case, el_val_t has_gender, el_val_t script_dir, el_val_t agreement, el_val_t null_subject);
el_val_t lang_word_order(el_val_t profile);
el_val_t layered_cycle(el_val_t raw_input);
el_val_t layered_cycle(el_val_t raw_input, el_val_t session_id, el_val_t utility);
el_val_t layered_generate(el_val_t prompt, el_val_t imprint_id, el_val_t session_id);
el_val_t lex_class(el_val_t entry);
el_val_t lex_form(el_val_t entry, el_val_t idx);
el_val_t lex_pos(el_val_t entry);
Generated Vendored
+15
View File
@@ -1,3 +1,18 @@
//
// STALE BUNDLE DO NOT BUILD. UNSAFE CHAT PATH.
//
// This concatenated bundle is a snapshot, not a source of truth, and it is stale in
// a way that matters for safety: it wires /api/chat straight to handle_chat and
// contains NO layered_cycle at all (verified: zero occurrences in the bundled code
// the only textual hit in this file is this banner). A binary built from
// this file would run chat with no enforcing input gate (no safety_screen, no
// hard-bell short-circuit) and no enforcing output gate (no safety_validate).
//
// Build from the .el sources via manifest.el (entry soul.el), or from dist/soul.c.
// Nothing in the repo references this file. It is kept only as a historical artifact
// and should be deleted once Will confirms nothing external depends on it.
// (Flagged 2026-08-04 in _engine-websearch-20260804/SAFETY-STOP.md; banner added
// 2026-08-05 with the plain-chat generation fix.)
// language-profile.el - Language profile data and accessors.
//
// A language profile is a slot map ([String] key-value list) describing the
Generated Vendored
+1 -1
View File
@@ -28170,7 +28170,7 @@ el_val_t agentic_loop(el_val_t session_id, el_val_t model, el_val_t safe_sys, el
state_set(el_str_concat(EL_STR("run_progress_"), session_id), EL_STR(""));
}
while (keep_going && (iteration < 8)) {
el_val_t req_body = el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(EL_STR("{\"model\":\""), model), EL_STR("\"")), EL_STR(",\"max_tokens\":4096")), EL_STR(",\"system\":\"")), safe_sys), EL_STR("\"")), EL_STR(",\"tools\":")), tools_json), EL_STR(",\"messages\":")), messages), EL_STR("}"));
el_val_t req_body = el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(EL_STR("{\"model\":\""), model), EL_STR("\"")), EL_STR(",\"max_tokens\":4096,\"tool_choice\":{\"type\":\"auto\",\"disable_parallel_tool_use\":true}")), EL_STR(",\"system\":\"")), safe_sys), EL_STR("\"")), EL_STR(",\"tools\":")), tools_json), EL_STR(",\"messages\":")), messages), EL_STR("}"));
el_val_t raw_resp = http_post_with_headers(api_url, req_body, h);
el_val_t is_error = ((str_starts_with(raw_resp, EL_STR("{\"error\"")) || str_starts_with(raw_resp, EL_STR("{\"type\":\"error\""))) || str_contains(raw_resp, EL_STR("authentication_error")));
if (is_error) {
+54 -6
View File
@@ -15,6 +15,40 @@ fn flag_true(body: String, key: String) -> Bool {
return json_get_bool(body, key) || json_get_int(body, key) > 0
}
// ---------------------------------------------------------------------------
// plain_chat_envelope the JSON response contract for a non-agentic ("Tools: Off")
// chat turn. Every /api/chat dispatch that calls layered_cycle goes through here, so
// the three call sites cannot drift apart.
//
// WHY THE ENVELOPE IS BUILT HERE AND NOT INSIDE layered_cycle:
// layered_cycle returns the user-facing text AFTER safety_validate has acted on it.
// Keeping the JSON out of the cycle means the output gate always sees raw model text
// and never an escaped blob there is nothing to unwrap and re-wrap on the crisis
// path, which is exactly the failure mode that made wiring handle_chat unsafe.
// Escaping is the last thing that happens, strictly after the gate.
//
// FIELDS: `reply` and `response` carry the same validated text. Both are required by
// live clients the desktop app reads `reply` first (DaemonClient.parseChatResponse),
// while the CLI tools and the Telegram gateway read `response` (the gateway reads only
// `response`). Emitting one would break the other.
//
// EMPTY MEANS FAILURE, NOT AN EMPTY ANSWER: a hard bell returns the fixed crisis
// message and a soft bell is padded to non-empty by safety_validate, so the only way
// an empty string leaves the cycle is a failed model call. It is reported as an error
// rather than dressed up as a successful blank reply.
// ---------------------------------------------------------------------------
fn plain_chat_envelope(validated: String, model: String) -> String {
if str_eq(validated, "") {
return "{\"error\":\"llm unavailable\",\"reply\":\"\",\"response\":\"\",\"agentic\":false,\"tools_used\":[]}"
}
let safe: String = json_safe(validated)
return "{\"reply\":\"" + safe + "\""
+ ",\"response\":\"" + safe + "\""
+ ",\"model\":\"" + json_safe(model) + "\""
+ ",\"agentic\":false"
+ ",\"tools_used\":[]}"
}
// ---------------------------------------------------------------------------
// Rate limiting simple in-memory per-IP sliding window counter.
//
@@ -243,8 +277,14 @@ fn handle_dharma_recv(body: String) -> String {
} else if agentic_flag {
handle_chat_agentic(chat_body)
} else {
let screened_reply: String = layered_cycle(raw_msg)
screened_reply
// Non-agentic ("Tools: Off"): the full L1L2L3L1 cycle, which now generates
// at L3 instead of echoing. Envelope built outside the cycle see
// plain_chat_envelope.
// FIX B/E1 (2026-08-05): the cycle is told which conversation it is in, and
// whether this generation is conversation at all. Same two arguments at all
// three dispatch sites.
let screened_reply: String = layered_cycle(raw_msg, json_get(chat_body, "session_id"), is_utility_request(chat_body, json_get(chat_body, "session_id")))
plain_chat_envelope(screened_reply, chat_default_model())
}
auto_persist(chat_body, reply)
return reply
@@ -416,8 +456,11 @@ fn handle_request(method: String, path: String, body: String) -> String {
} else if agentic_flag {
handle_chat_agentic(body)
} else {
let screened_reply: String = layered_cycle(eff_msg)
screened_reply
// Non-agentic ("Tools: Off") same cycle and same envelope as POST.
// FIX B/E1: same threading. A GET probe usually carries no session_id, which
// resolves to the anonymous window the documented behaviour for this door.
let screened_reply: String = layered_cycle(eff_msg, json_get(body, "session_id"), is_utility_request(body, json_get(body, "session_id")))
plain_chat_envelope(screened_reply, chat_default_model())
}
auto_persist(body, reply)
return reply
@@ -580,8 +623,13 @@ fn handle_request(method: String, path: String, body: String) -> String {
} else if agentic_flag {
handle_chat_agentic(body)
} else {
let screened_reply: String = layered_cycle(raw_msg)
screened_reply
// Non-agentic ("Tools: Off") the app's DEFAULT mode (AgentMode.NEVER).
// Full L1L2L3L1 cycle with real generation at L3; envelope built
// outside the cycle so safety_validate always sees raw text.
// FIX B/E1: same threading. This is the app's main plain-chat door, so this
// is the site that ends the blank stare in practice.
let screened_reply: String = layered_cycle(raw_msg, json_get(body, "session_id"), is_utility_request(body, json_get(body, "session_id")))
plain_chat_envelope(screened_reply, chat_default_model())
}
auto_persist(body, reply)
return reply
+71 -35
View File
@@ -379,9 +379,23 @@ fn emit_session_start_event() -> Void {
// layered_cycle routes user-facing requests through the 4-layer consciousness stack.
// L0 (core) L1 (safety screen) L2a (continuity + behavioral profiling) L2b (mission alignment) L3 (imprint) L1 (safety validate)
// Internal cognition (heartbeat, proactive, memory ops) bypasses layers use one_cycle directly.
fn layered_cycle(raw_input: String) -> String {
let history: String = state_get("conv_history")
let session_id: String = state_get("current_session_id")
//
// FIX B (2026-08-05) the cycle now knows which conversation it is in.
//
// session_id: the caller's session, threaded from the route. Was previously read from the
// state key "current_session_id", which is read HERE and written NOWHERE in the entire
// source verified across every .el file. So this value was unconditionally "", and every
// downstream consumer of it silently fell back to a process-global bucket: conversation
// history, and the steward's continuity tracking (TODO reliability #4, below, describes the
// cross-session bleed this caused; threading the real id closes it). The plain path's blank
// stare and the agentic path's scoped history were the same defect seen from two sides.
//
// utility: true when the generation is not part of the user's conversation the app's
// title and insight passes. Answered normally, never recorded. See is_utility_request.
fn layered_cycle(raw_input: String, session_id: String, utility: Bool) -> String {
// Safety-screen history amplification now reads the SAME window the turn will be
// recorded into, so a session's own escalation pattern is what gets scored.
let history: String = state_get(conv_hist_key(session_id))
// L1 in: safety screen
let screen_result: String = safety_screen(raw_input, history)
@@ -423,8 +437,10 @@ fn layered_cycle(raw_input: String) -> String {
let cont_action: String = json_get(continuity, "action")
// Store continuity status so imprint can adjust its response register.
// TODO(reliability #4): session_continuity is process-global; scope per session_id
// when available to prevent cross-session bleed under concurrent layered_cycle calls.
// TODO(reliability #4) CLOSED 2026-08-05: this line was already written to scope per
// session it just never received a session id, because the only source was a state key
// nothing wrote. It is now threaded from the route, so named sessions genuinely get their
// own continuity state and only anonymous callers share the global one.
let cont_key: String = if str_eq(session_id, "") { "session_continuity" } else { "session_continuity:" + session_id }
state_set(cont_key, cont_status)
@@ -453,40 +469,24 @@ fn layered_cycle(raw_input: String) -> String {
let lc_aff_cutoff: Int = time_now() - 259200
let lc_bell_nodes: String = engram_search_json("bell:soft bell:hard BellEvent affective", 2)
let lc_has_bell: Bool = !str_eq(lc_bell_nodes, "") && !str_eq(lc_bell_nodes, "[]")
// CRASH FIX 2026-08-05 (BUG-PLAINCHAT-1): the " | ts:" parser used to be inline here.
// Inside this block-expression initializer elc compiled `lbmp + str_len(lbm)` to
// el_str_concat() on two integers, which segfaulted the whole daemon the moment a
// distress turn followed an earlier affective turn i.e. exactly on the crisis path.
// Verified against the unmodified baseline binary AND present in the committed
// dist/soul.c. affective_node_ts() is a top-level function, where the same expression
// compiles to integer addition. Do not inline it back.
let lc_bell_note: String = if lc_has_bell {
let lb0: String = json_array_get(lc_bell_nodes, 0)
let lb_c: String = json_get(lb0, "content")
let lbm: String = " | ts:"
let lbmp: Int = str_index_of(lb_c, lbm)
let lb_ts_raw: String = if lbmp >= 0 {
let lbs: Int = lbmp + str_len(lbm)
let lbr: String = str_slice(lb_c, lbs, str_len(lb_c))
let lbn: Int = str_index_of(lbr, " | ")
if lbn < 0 { lbr } else { str_slice(lbr, 0, lbn) }
} else {
let lbca: String = json_get(lb0, "created_at")
if str_eq(lbca, "") { json_get(lb0, "updated_at") } else { lbca }
}
let lb_ts: Int = if str_eq(lb_ts_raw, "") { 0 } else { str_to_int(lb_ts_raw) }
let lb_ts: Int = affective_node_ts(lb0)
if lb_ts > lc_aff_cutoff { "[AFFECTIVE NOTE: User was in distress in a recent session.]" } else { "" }
} else { "" }
let lc_pos_nodes: String = engram_search_json("PositiveEvent joy:high joy:low affective", 2)
let lc_has_pos: Bool = !str_eq(lc_pos_nodes, "") && !str_eq(lc_pos_nodes, "[]")
// Same crash fix as the bell note above (BUG-PLAINCHAT-1).
let lc_pos_note: String = if lc_has_pos && str_eq(lc_bell_note, "") {
let lp0: String = json_array_get(lc_pos_nodes, 0)
let lp_c: String = json_get(lp0, "content")
let lpm: String = " | ts:"
let lpmp: Int = str_index_of(lp_c, lpm)
let lp_ts_raw: String = if lpmp >= 0 {
let lps: Int = lpmp + str_len(lpm)
let lpr: String = str_slice(lp_c, lps, str_len(lp_c))
let lpn: Int = str_index_of(lpr, " | ")
if lpn < 0 { lpr } else { str_slice(lpr, 0, lpn) }
} else {
let lpca: String = json_get(lp0, "created_at")
if str_eq(lpca, "") { json_get(lp0, "updated_at") } else { lpca }
}
let lp_ts: Int = if str_eq(lp_ts_raw, "") { 0 } else { str_to_int(lp_ts_raw) }
let lp_ts: Int = affective_node_ts(lp0)
if lp_ts > lc_aff_cutoff { "[AFFECTIVE NOTE: User shared positive news in a recent session.]" } else { "" }
} else { "" }
let lc_affective_note: String = if !str_eq(lc_bell_note, "") { lc_bell_note } else { lc_pos_note }
@@ -498,11 +498,47 @@ fn layered_cycle(raw_input: String) -> String {
}
state_set("layered_cycle_safety_system_addendum", augmented_addendum)
// L3: imprint responds
let output: String = imprint_respond(aligned, imprint_id)
// L3: imprint responds applies the active imprint's voice/domain annotation to the
// steward-aligned input. This produces the PROMPT, not the answer.
let prompt: String = imprint_respond(aligned, imprint_id)
// L1 out: validate output before delivery
return safety_validate(output, screen_action)
// L3b: the imprint SPEAKS (added 2026-08-05).
//
// Until now the cycle stopped at the annotation above, so /api/chat with agentic:false
// handed the user's own screened text back as the "reply" every gate ran, but nothing
// ever generated. The generation is placed HERE, inside the cycle, rather than by
// pointing the route at handle_chat(): handle_chat has no enforcing input gate and no
// enforcing output gate, so calling it instead of this cycle would have traded the whole
// safety pipeline for a working reply. Composing keeps both.
//
// Order is deliberate and must not be rearranged: this call sits strictly AFTER the L1
// screen, the safe-mode guard, the hard-bell short-circuit and the L2 stewardship layers,
// and strictly BEFORE the L1 output gate. A hard bell never reaches a model the branch
// above returns first. Tools are not offered on this turn; see layered_generate.
let output: String = layered_generate(prompt, imprint_id, session_id)
// L1 out: validate output before delivery. Still the terminal gate nothing below this
// line can change the string this function returns.
let validated: String = safety_validate(output, screen_action)
// Turn bookkeeping. Records the VALIDATED text, never the raw model output, and is only
// reachable on the non-bell path: both bell branches above return before this point, so
// bell turns still never enter conversation history. Pure state side effect it cannot
// alter what is returned.
//
// FIX A: the receipt is unconditional and always negative on this path, because on this
// path it is structurally true layered_generate offers no tools at all (build_system_prompt
// chat mode + a request body with no "tools" key). Recording "no tools ran" is not padding:
// it is the only thing that distinguishes "nothing ran" from "we forgot to write down what
// ran", and that ambiguity is what made the model confess to a search it had performed.
//
// FIX E1: a utility generation is answered but not recorded. Guarded here rather than at
// the route so every /api/chat dispatch site inherits it from one place.
let receipt: String = tool_receipt("", "")
if !utility {
conv_history_record(session_id, raw_input, validated, receipt)
}
return validated
}
let soul_cgi_id_raw: String = env("SOUL_CGI_ID")
+3 -1
View File
@@ -1,6 +1,8 @@
// auto-generated by elc --emit-header - do not edit
extern fn init_soul_edges() -> Void
extern fn ensure_self_canonical_bridge() -> Void
extern fn aff_try_slot(slot_json: String, aff_7d_ts: Int, acc_key: String) -> Void
extern fn load_identity_context() -> Void
extern fn seed_persona_from_env() -> Void
extern fn emit_session_start_event() -> Void
extern fn layered_cycle(raw_input: String) -> String
extern fn layered_cycle(raw_input: String, session_id: String, utility: Bool) -> String