de6599180718ffef9f0f155370abb61a97bb4eec
330 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
de65991807 |
feat(engine): tools + agentic loop on the OpenAI wire, and two chat-breaking fixes found proving it
Teaches the OpenAI-format lane (Groq/OpenAI/Grok/Gemini/Ollama) to offer tools,
execute them, and loop — the capability that until now existed only on the
Anthropic wire. The tool-execution, consent, bridge and run-progress machinery is
reused unchanged; only the wire dialect is new.
Two pre-existing defects were found while proving it, and are fixed here because
both silently break chat:
1. PROVIDER WIRING NEVER CONNECTED. The launcher exports SOUL_LLM_PROVIDER /
SOUL_LLM_BASE_URL and puts the provider key in ANTHROPIC_API_KEY + SOUL_API_KEY;
the engine's provider fork read only NEURON_LLM_0_*, which nothing sets in a
customer build. So use_openai was ALWAYS false: every non-Anthropic user's turns
went to api.anthropic.com carrying, say, a Groq key, and came back
"llm unavailable". Proven side-by-side against the pinned round-9 brain
(sha256 15cf7d1b…): identical env, shipped brain = "llm unavailable" both chat
modes with ZERO calls to the configured endpoint; this build = a real answer,
with the probe logging POST /v1/chat/completions and Bearer <provider key>.
Fixed brain-side only (env fallbacks) — no app or launcher change needed.
2. TRUNCATION SPLITS UTF-8 CHARACTERS. The session preload cuts recalled memory at
fixed BYTE lengths (continuity snippet 350; session_preload_bullets per bullet).
A cut landing inside a multi-byte character leaves a dangling lead byte in the
SYSTEM PROMPT, making the whole request body invalid UTF-8 — providers reject it
and the user sees an unexplained failure. Captured from a real body: 18,710 bytes,
decode fails at 18,248 on 'e2', a box-drawing rule (U+2500 = E2 94 80) sliced in
half. Trigger is ordinary content — em dash, curly quote, accented name, emoji,
table border — and it gets MORE likely as memory grows. Shared code: this hit the
Anthropic wire too. Fixed with utf8_safe_slice() applied at BOTH cut sites.
WHAT IS IN THE PORT
- llm_base_url / llm_wire_format / agentic_api_key: fall back to the launcher's own
SOUL_LLM_* names; anthropic deliberately still returns "" so its native path is
untouched (endpoint configurability remains neuron#62).
- openai_tools_json(): Anthropic tool schema -> OpenAI function schema; entries with
no input_schema (Anthropic's server-side web_search) are skipped — they cannot
execute on this wire.
- agentic_tools_no_web(): the standard set minus that server tool.
- openai_agentic_loop(): forked rather than parameterised, so agentic_loop — which
carries every round-7/8/9 fix — is provably untouched. Same envelopes, same state
keys, same consent policy (ask_all / escalate / builtin / always-allow), same
client-bridge contract, same run-progress ledger, same 12-iteration cap.
- ADR-0005 mirrored on this wire: parallel_tool_calls:false is sent explicitly, and
if a provider ignores it we honour the FIRST call and echo only that one, so the
conversation we send is never self-contradictory. The drop is logged loudly.
- The assistant turn echoes the provider's own content bytes (json_get_raw), so a
JSON null stays null and nothing is lost to a decode/re-encode round trip.
- Tool results are embedded already-escaped (dispatch_tool json_safe's them);
truncation trims a dangling escape so a cut can't invalidate the body.
- bridge_save() gains a "wire" scalar and agentic_resume branches on it, so a
suspended turn resumes on the wire it suspended on. Legacy blobs (no field) resume
as anthropic. The field is read from the blob's SCALAR HEAD only — an unbounded
first-match scan would run on into messages_raw, which is model-controlled, and
that is exactly the round-9 resume defect. Pinned by a test.
- Three fork sites: handle_chat_agentic, handle_dharma_room_turn_agentic,
agentic_resume. Tool assembly is computed once per lane at both entry points
(it makes an HTTP call to the connector bridge; it was being paid for twice).
TOOLING THAT DID NOT EXIST
- tests/run-el-test.sh — engine tests were never runnable: elc is a compiler, it
emits C and exits. This emits the test to C, compiles soul.c with main renamed
away, links the rest + the repo-pinned runtime, and runs it. It also COMPUTES THE
VERDICT, because every counted test file's "N passed, M failed" summary is a
permanent 0/0 — the counters increment inside if BLOCKS, which El scoping
discards (9 files; real fix filed as neuron#116). Proven to discriminate with a
deliberately-broken assertion.
- tests/gate-openai/ — deterministic OpenAI-dialect provider stub + scenarios +
driver + hostile modes, and a strict request validator that rejects any
Anthropic-shaped field so dialect leakage fails loudly.
VERIFICATION (rungs named)
- E2E-VERIFIED against a LIVE provider (Anthropic's OpenAI-compatible endpoint,
confirmed live): real answer; a tool call whose out-of-root path was DENIED by the
guard, after which the model refused to claim success ("I won't tell you I did it,
because I didn't"); then a valid path -> file physically on disk with exact content,
honest reply, ledger with per-round entries + {done:true}.
- Deterministic lane gate: 11/12 in both consent configurations (bridge + local);
hostile providers produce no hang and no fabricated answer; the 12-iteration cap
trips with its honest message. The one FAIL is oa-tools-off and is NOT this port —
see "Known, not fixed here".
- ANTHROPIC LANE UNCHANGED: gate9 32/32 on this build and on the pinned round-9
brain; request bytes differ only within the noise band that two runs of the
UNMODIFIED brain also produce (proven with a baseline-vs-baseline control), and
the preload sections — the shared code touched here — are byte-identical.
The rig discriminates: the round-8 brain scores 24/32 on it.
- verify-soul-contract.sh: PASS (27/27 routes, no hard-deletes).
- Unit: test_bridge_serialization 36/36 (incl. 8 new wire/field-order assertions),
test_utf8_slice 18/18, test_agentic_tools 18 PASS / 0 FAIL / 3 documented skips.
KNOWN, NOT FIXED HERE (deliberate)
- Tools:Off on an OpenAI provider still fails: the non-agentic path goes through the
el-runtime provider chain, which appends /v1/chat/completions to a base URL that
already ends in /v1 -> /v1/v1/... 404. Runtime/plain-chat territory, untouched
mid-beta. Note openai_chat_complete() has zero callers — that lane is served
entirely by the runtime chain.
- The 12-iteration cap does not bound a chain of BRIDGED tools (iteration is
per-invocation and resume starts fresh). Parity with the Anthropic lane.
- run_progress resets on each resume, so a client rendering cumulative steps across a
consent pause sees earlier legs vanish. Parity with the Anthropic lane.
- verify-soul-contract.sh needs bash >= 4; under macOS's stock bash 3.2 it dies
instantly with a FALSE red ("local: -n: invalid option").
- Groq-specific live E2E not run: no Groq key exists on this machine.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
|
dba755dcec |
fix(engine): resume reads the bridged tool id from the blob's own field, not from inside the replayed conversation
ROOT CAUSE (round 9; live-repro'd 5/5 this morning, both faces stub-proven by the
prompt-matrix gate). json_get is a first-substring-match scanner (strstr for
'"key":', el_runtime.c). bridge_save serialized the RAW messages array BEFORE the
tool_use_id scalar, so agentic_resume's json_get(blob, 'tool_use_id') returned the
FIRST '"tool_use_id":' occurrence inside the replayed conversation, not the saved
field. The resume guard then preferred that misread over the client's correct
call_id (its two branches both reduced to saved_use_id), attached the tool_result
to the wrong id, and Anthropic 400'd the resume ('unexpected tool_use_id found in
tool_result blocks'), surfaced as {"error":"llm unavailable"}.
ONE MISREAD, TWO FACES — whichever block owns the first tool_use_id in the array:
FACE 1 (search-then-bridge, the Key West killer): the first occurrence is the
first web_search_tool_result's srvtoolu_… id — every agentic turn that ran
server-side web_search and then bridged on a client tool died on approval,
deterministically (messages.2.content.0 … srvtoolu_…). The write itself had
already succeeded; only the resume died.
FACE 2 (multi-cycle missions): with no search, the first occurrence is ROUND 0's
tool_result block — so every LATER approve/resume cycle replayed the round-0
client id (stale-resume-id), killing multi-file missions after ~2 files.
And the shape that PASSES on round 8 confirms the mechanism: a single-cycle
bridge with no prior tool round has no 'tool_use_id' substring in its messages
at all (tool_use blocks carry 'id'), so the scan fell through to the blob's own
field and resumed correctly.
The server_tool_use ↔ web_search_tool_result pairs themselves replay intact — the
defect was a cross-field misread of the blob, the same first-match-scanner class
as BUG-6 (approve 'content' matched inside tool_input, 2026-07-17) and round 8's
citation-block fix.
THE FIX, the pattern not the spot:
1. bridge_save writes every json_safe'd scalar BEFORE both raw fields (an escaped
value cannot contain a bare '"key":' byte pattern, so first-match always lands
on the blob's own fields), and tools_raw (our fixed schema) before messages_raw
(arbitrary conversation), so the raw extractions cannot first-match into
model-controlled bytes either. Field order documented as load-bearing.
2. agentic_resume now honors the client's echoed call_id when present — the value
with clean provenance (minted from pend_tool_id, never blob-round-tripped) —
falling back to the saved id only when the client omits it. Each approve cycle
therefore binds to ITS OWN round's id (kills FACE 2 even against a blob written
by a pre-fix binary), and an omitted call_id still resumes on the saved id,
which the reordered blob now reads correctly.
Pattern sweep: the legacy synthetic blob (sessions.el handle_session_approve) embeds
only json_safe'd fields — no raw hazard, untouched. No other json_get read of any
container that embeds raw conversation JSON before the read field.
PROOF: prompt-matrix gate 24/32 RED on the round-8 brain (fails exactly the two
resume classes, named) -> 32/32 GREEN on this build; live-key Key West tracer
3/3 consecutive full round-trips (bridge -> approve-as-the-app -> real completion,
file on disk), plain-chat and weather-only controls PASS; unpatched round-8 brain
and a same-toolchain unpatched baseline build both still fail the identical
sequence with the identical srvtoolu 400 (the test discriminates, and the only
variable between failing and passing builds is this diff).
Refs neuron#109
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
8f3a478771 |
fix(engine): excise the receipt, do not truncate at it — a leading receipt was erasing whole answers
CAUGHT BY A/B, AND ONLY BY A/B. The previous commit's receipt_strip assumed the receipt is
always TERMINAL and cut everything from the marker onward. It is not always terminal: once
receipt_rule told the model what [[RECEIPT ...]] means, the model sometimes LED with one and
wrote the answer underneath. Cutting at the marker then deleted the entire answer and the
turn returned {"error":"no response"}.
MEASURED, same prompt (two web searches, cited prose), fresh session each run:
round-7 brain 4 / 4 answered (471, 473, 473, 544 chars)
round-8 brain 2 / 7 answered (five {"error":"no response"})
This looked exactly like a flaky model. It was not — it was mine. Running the two brains
side by side on the same prompt is the only reason it was found, and it is the reason the
A/B is now part of how this class gets tested.
AFTER THE FIX, same protocol:
round-7 brain 4 / 4 (473, 473, 473, 544)
round-8 brain 4 / 4 (657, 657, 657, 673)
THE FIX: remove the [[...]] span and keep BOTH sides, instead of truncating at the marker.
An unterminated marker at position 0 is left completely alone — no rule about receipts is
worth erasing an answer over. Bounded four-pass loop rather than a conditional exit, because
rebinding the counter inside an if-expression is the block-expression shape that miscompiles
integer arithmetic under this elc (BUG-PLAINCHAT-1). Verified in the generated C:
str_slice(rest, (e + 2), str_len(rest)) <- integer addition, correct
el_str_concat(head, tail) <- string concat, correct
and zero el_str_concat(<ident>, str_len(...)) sites across all 49 modules.
SEAM PROOF (FIX C) rides on the same runs — a real two-search cited answer, inspected byte
by byte, in BOTH failure directions:
missing separator (the round-7 "to.Good", bytes 77 2e 47): 0 hits. Sentence boundaries
measure 2e 20 4d — "." SPACE "M".
over-separation (a cited sentence shattered across paragraphs): 0 hits. The answer is one
continuous paragraph with its sentences intact, which is the direction a blanket
separator would have broken.
BUILT: sha256 77115f2733e794c5bc4ad1f55b1acaf658f8f4a91cccd423a2d633d94a726cbc
Refs neuron#109
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
|
9ea41eed78 |
fix(engine): the model was signing its own answers with our receipt — strip it
FOUND BY E2E, NOT BY REASONING. The previous commit's design note asserted the provenance
receipt "never reaches the user: it is appended to the history copy, not the reply." That
was FALSE, and only running the thing showed it. On the first live run against the built
DMG brain, two agentic turns out of two came back with
"Your favourite colour is chartreuse and your project is called Perihelion.
[[RECEIPT - recorded by the soul, not written by the model: no tools ran on this turn.]]"
— the receipt in the user-visible reply.
MECHANISM: the receipt is stored inside the assistant turn, and the agentic path replays
history VERBATIM as Anthropic message objects. So the model sees its own previous answers
ending in [[RECEIPT ...]] and does the obvious thing — it imitates the format and signs the
next answer the same way. The plain path did NOT leak, which is the tell: there, history is
rendered into the SYSTEM prompt as labelled lines rather than replayed as assistant turns,
and a model imitates its own turns far more readily than a transcript.
FIX, two layers, because one of them is not a guarantee:
- receipt_rule() names the marker in both system prompts (plain and agentic): these lines
are written by the system, read them as evidence, never write one. Reduces occurrence.
- receipt_strip() truncates any [[RECEIPT ...]] out of model output before it becomes the
reply — plain path in layered_generate, agentic path on final_text in agentic_loop.
Deterministic. A guard that depends on the model choosing to obey is exactly the class of
thing round 8 exists to stop shipping, so the instruction is the optimisation and the
strip is the guarantee.
Placed ABOVE agentic_loop's empty-check on purpose: a turn whose entire output was an
imitated receipt has produced no answer, and must be reported as no answer.
The receipt stays in HISTORY, which is the whole point and is proven to work: asked "What
source did you use for that?" one turn after a live web_search, this brain answered
"I used Weather Underground (https://www.wunderground.com/weather/is/reykjav%C3%ADk) for the
current temperature in Reykjavik" — a real source, no apology. That is the false confession
dead, and it is dead BECAUSE the model can read the receipt.
BUILT: 887,112 bytes, sha256 54a2eff84d4fa44f8d2db6781dcf225df4b5075a8ec5f1058018bd40cc1af10b
BUG-PLAINCHAT-1 miscompile guard: zero sites.
Refs neuron#109
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
|
ff421d39f6 |
fix(engine): history keeps its provenance and its session — the false confession and the blank stare
DESIGN FIT: three of round 7's five defects share ONE root — the conversation-history
layer persists only {role, content}, discarding tool provenance, session scoping, and the
distinction between a real user turn and an internal utility call. Fixes A and B RESTORE
Will's design rather than extend it: his agentic path already scopes history per session,
the plain path never got it, and his own source carries the TODO admitting the resulting
race (chat.el, handle_chat: "process-global key; concurrent /api/chat requests without
session_id race on this read-append-write"). Fix C repairs one join Will wrote that was
correct for a year and one we added last week. E1/E2 are ours.
FIX A — tool provenance in history (kills the FALSE CONFESSION)
Root cause, EXECUTED-verified: handle_chat_agentic recorded turns via hist_append, which
emits {"role","content"} only. server_tool_use blocks, web_search_tool_result blocks and
every citation were discarded, then replayed as text. On the next turn the model saw a
data-rich answer with zero evidence a search had happened, and its own permanent rule
("never describe a search you did not perform") left one conclusion available: that it
had fabricated the data. It apologised for a search it HAD run — four independent lines
of evidence confirm the search was real. The defect is not the model's honesty. It is
that we deleted the evidence and then asked it to account for itself.
Change: agentic_loop accumulates the source URLs it already walks past (citations and
web_search_tool_result content) and returns them as "sources"; handle_chat_agentic folds
tools_used + sources into a receipt line stored WITH the assistant turn. Receipts are
unconditional — a negative receipt ("no tools ran") is the other half of the guarantee,
because "no evidence of a tool" and "evidence of no tool" were previously identical in
the transcript. conv_history_block splits the receipt off before snipping so a long
answer cannot truncate away the evidence. The user never sees it: it is appended to the
history copy, not the reply.
FIX B — one history key for both paths (kills the BLANK STARE)
Root cause, EXECUTED-verified: the agentic path keyed history on session_hist_<id>; the
plain path was hard-wired to the process-global conv_history and never read session_id.
One conversation, two buckets. Proven in the guest engram: the scoped node held exactly
two turns starting at "Try again" while the earlier exchanges sat unscoped.
Change: conv_hist_key/conv_hist_label are now the single definition, used by BOTH paths;
session_id is threaded route -> layered_cycle -> layered_generate / conv_history_record.
The 2-line fallback (plain path reads the agentic key) was REJECTED: it keeps the
process-global bucket as a live write target, which is the bleed the TODO describes.
Also found and closed while threading: layered_cycle read session_id from the state key
"current_session_id", which is read here and WRITTEN NOWHERE in the entire source. It
was unconditionally "", so TODO(reliability #4) — per-session steward continuity — was
dead code that could never fire. It fires now.
LAZY SESSION, decided explicitly: we create the session EAGERLY at the door (app half,
ui#223) rather than migrating orphaned turns. Migration would copy the CONTENTS of a
process-global bucket, possibly another conversation's, into a named session — the bleed,
performed deliberately. Eager creation makes the situation impossible instead. Migration
is deliberately not implemented and must not be added without solving provenance first.
FIX C — the two text-join seams ("to.Good", byte-verified 0x77 0x2e 0x47)
Two bare `+` joins, written a year apart, had drifted into two answers to one question:
within-response block joins (Will's, 2026-05-03, latent until server-side web_search
began interleaving non-text blocks) and across-round joins (ours,
|
||
|
|
635f6febe4 |
feat(engine): plain chat generates at L3 — inside the safety cycle, not around it
Non-agentic /api/chat (the desktop app's default "Tools: Off" mode) returned the
user's own screened text as a bare non-JSON string. Every JSON client failed to
parse it and showed "Couldn't reach Neuron - it may be offline."
Root cause:
|
||
|
|
62af5649fe |
feat(engine): port Anthropic server-side web_search into the agentic loop
Re-authors soul-webfix-20260711.patch in El (the patch is a diff against generated C at month-old offsets, so nothing was applied as a patch). Its last two hunks — an unrelated /api/safety-contact implementation — were deliberately not ported; that route already exists and is safety-critical. Activation restores existing design, it does not invent a mechanism: commit |
||
|
|
710761e2d5 |
fix(engine): send tool_choice.disable_parallel_tool_use on agentic loop (STOPGAP)
The agentic loop keeps only the FIRST tool_use block per round (chat.el:2281,
"Capture first tool_use block only"). Anthropic lets a model emit several
tool_use blocks in one message and requires a tool_result for every one, so a
parallel-tool turn is answered once, the rest are dropped, and the next request
dies with:
tool_use ids were found without tool_result blocks immediately after
(neuron#78 quotes this as "tool_use ids found without tool_result"; the above is
the API's actual wording - recorded so the next person's grep matches.)
This constrains the wire to match what the loop can assemble:
"tool_choice":{"type":"auto","disable_parallel_tool_use":true}
STOPGAP - AND THE DURABLE FIX ALREADY EXISTS. A correct multi-tool loop is
already in Will's EL runtime, in C, and the soul does not call it. Verified on
el:origin/main lang/el-compiler/runtime/el_runtime.c: llm_register_tool:9616,
llm_build_tool_results:9743 - which walks EVERY content block, emits one
tool_result per tool_use, and sets is_error for an unregistered tool -
llm_call_agentic:9817 calling it at :9918, iteration cap 10 at :9847. Will's
commit 12d5e77 (2026-04-30). grep for llm_call_agentic/llm_register_tool across
every neuron/*.el returns nothing; dist/soul.c has zero references. chat.el
hand-rolls its own single-tool loop instead, and that is the one that breaks.
The durable fix is therefore to register the soul's tools via llm_register_tool
and call llm_call_agentic - deleting a loop, not writing one. See ADR 0005.
Our own approved spec called this seven weeks ago:
docs/research/agentic-tool-approval-design.md (2026-06-12, "Approved for build"),
line 20 on the defect, line 30 on the goal ("Execute all tool_use blocks in a
turn (one result per block)").
Two edits, because dist/soul.c cannot be regenerated here (Will's gated elc/elb
toolchain is not on this machine):
(a) chat.el:2255 - source of truth, so a later regen carries the fix.
One edit covers all three routes: agentic_loop is called from chat.el:2152
(/api/chat agentic), :2676 (dharma room) and :2496 (agentic_resume).
(b) dist/soul.c:28173 - generated form, hand-spliced. Line 27624 is the
non-agentic/OpenAI-compat req_body (no tools) and was left untouched.
Prior art reused rather than reinvented: soul-narrated-runs-20260713.patch
(27,824 bytes) line 78 spliced this same string into the same concat chain on
2026-07-13.
Deliberately NOT ported from that patch, having read it: max_tokens is not
changed by it (16384 sits on both sides of the hunk; our main's 4096 is a
separate output-truncation concern), and its pause_turn pairing fix - same
defect class - is unreachable today because no server-side web_search is wired
(agentic_tools_with_web at chat.el:1418 is never called), so it is untestable
and logged instead.
Proven E2E on a scratch profile and port 7791, never the live chain. A/B against
a pristine origin/main control built from the same vendored runtime: fixed
completed the mission (tools_used read_file x3, 4 iterations, correct answer);
control failed 3/3. Direct API probe confirmed the mechanism - without the field
the model emits 3 parallel tool_use blocks and replaying the unfixed loop's next
turn returns HTTP 400; with it, exactly 1 block.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
74520b8333 | Merge pull request 'ci: pin + complete vendored el-runtime so reconciled soul.c links' (#105) from ci/pin-vendored-runtime into main | ||
|
|
7f3d6ed8cd |
ci: update vendored el-runtime to complete v1.0.0-20260501
The runtime vendored alongside the CI pin was the Jul-21 snapshot, which predates two builtins the reconciled ship-soul now calls: - http_delete_json (boot-counter HTTP write-back, awareness/memory self-review) - engram_act_stats_json (heartbeat activation observability) Compiling dist/soul.c against the stale runtime fails with implicit-declaration errors. Vendor the current release runtime (identical to the one the soul was gate-verified against: verify-soul-contract PASS, genesis boots clean, full safety-contact) so the CI Linux soul is byte-for-byte the verified soul. |
||
|
|
eed6487114 |
ci: pin soul build to vendored release runtime v1.0.0-20260501
The soul build downloaded el-runtime-c 'latest' from Artifact Registry. The merged ship-soul calls engram_prune_telemetry, which the latest published runtime no longer defines, so an unpinned build fails to link — the failure mode that let a broken/handlerless soul reach prod. Vendor the release runtime v1.0.0-20260501 (el_runtime.c/.h) into the repo and compile the soul against it. This is the exact runtime the merged soul was verified against (verify-soul-contract GATE PASS, genesis boot survives, full safety-contact response), making the build reproducible and independent of a moving AR 'latest'. The verify-soul-contract.sh HARD-BLOCK gate already runs before Publish (from the CI-hardening arc on main), so a destructive or stale soul can never publish/deploy again. |
||
|
|
2c2aaa0653 | Merge pull request 'Reconcile: main = union of all ship-critical soul fixes (beta-gating)' (#104) from reconcile/soul-union-main into main | ||
|
|
e610a412b9 |
regen soul.c from reconciled tree + harden contract gate (#199 by-id, isolation)
dist/soul.c: regenerated amalgamation (1.15MB) from the reconciled sources via the hide-.elh + elc --target=c recipe, so the shipped translation unit CI compiles now actually carries every landed fix — genesis-boot SIGSEGV (#150), safety-contact 988 truncation (#96), url-decode multi-word search, honest receipts (#100/#101), immutability arc (#83), and the bounded payloads (#103). verify-soul-contract.sh, two non-weakening fixes (both false-NEGATIVE bugs that spuriously failed a CORRECT soul; neither relaxes what fails a defective one): 1. #199 by-id gate: verify tombstone/KEPT via /api/neuron/graph?id=<id>&depth=1 (a compact neighborhood) instead of grepping engram_scan_nodes_json(9999,0) — a multi-MB, salience-ordered, 9999-capped whole-graph dump in which the salience-0.01 tombstone marker sorts past the cap and vanished. 2. Isolation: pin SOUL_ISE_URL to the dead axon port. Unsetting ENGRAM_URL was not enough — the periodic engram sync defaults its source to the LIVE engram (http://localhost:8742), so the 'isolated' gate pulled the operator's real brain (56 -> 12k nodes in seconds), which both broke Section B determinism and read live state. Now the soul stays on its own store. Verified GREEN on a throwaway port/HOME (live :7770/:8742/~/.neuron untouched): gate PASS x3 (presence 27/27, immutability all 5 KEPT); safety-contact POST 218B / GET 208B full untruncated; multi-word search (%20 and +) returns ranked hits with an all-gibberish control at 0; bounded session/begin 1370B; honest ok:false on a missing-id delete; genesis (ntn-genesis) boots clean through mem_save with no SIGSEGV. |
||
|
|
8ba35a0d56 |
reconcile: merge Will's self-review WIP (456267a) into main
Union of all ship-critical soul fixes for the Mac beta: - Keeps main's honest receipts (#100/#101), immutability arc + #199 by-id gate (#83), Track B threat routing (#76), bounded beginSession (#103), CI hardening (#85/#86), elc typo hotfix (#77), neuron-dev-setup (#84). - Brings WIP's genesis-boot SIGSEGV fix (#150/#95), safety-contact 988 truncation fix (#96), bounded-persona floor (#93), and 10 self-review commits (importance flattening, curiosity DF gating, WM/heartbeat observability, boot-counter telemetry). - Folds the multi-word ranked-search fix: api_query_param now url_decode()s the extracted value so q=foo%20bar / foo+bar tokenize as two words. Conflicts (neuron-api.el payload-bound comments, mcp-wrapper tool_forget) resolved toward the correct end state: main's verified honest-receipt read-back is kept; WIP's improved forget description is kept. Generated dist/*.c taken from main and will be regenerated from this reconciled source in the following commit. |
||
|
|
b75d5c8c30 | Merge pull request 'Bound beginSession/compileCtx payloads to a compact digest (main)' (#103) from fix/bound-session-payload-main into main | ||
|
|
9bbb4f2af8 |
Bound beginSession/compileCtx payloads to a compact digest
Port the payload-bounding fix (PR #102, commit
|
||
|
|
ec219c5830 | Merge pull request 'fix(mcp-wrapper): forget/delete tools no longer return fake ok receipts (BUG-18)' (#101) from fix/receipts-wrapper-forget into main | ||
|
|
731efaedaf | Merge pull request 'fix(chat): agent write_file/edit_file no longer return false success receipts (BUG-29)' (#100) from fix/receipts-agent-tools into main | ||
|
|
3723e3b7e7 |
fix(mcp-wrapper): forget/delete tools no longer return fake ok receipts (BUG-18)
Root cause: two false-receipt paths in the wrapper's delete family.
- delete_by_id (removeKnowledge, deleteProcess, deleteImprint,
dischargeWonder) FABRICATED {"ok":true,...,"note":"soft-deleted"}
without calling the soul at all — the 'soul does not yet expose a delete
HTTP route' note was stale (/api/neuron/node/delete exists and tombstones
any node type).
- tool_forget forwarded the soul's response but never verified the deletion
actually persisted before answering ok.
The change (Receipt Contract rule 1 — a tool result must reflect what
actually happened):
- delete_by_id now routes to the soul's real /api/neuron/node/delete and
propagates its answer (honest 'node not found' for bad ids).
- Both handlers read back before answering ok: GET /api/neuron/graph?id=..
&depth=1 must show the tombstone marker (label "tombstone:<id>"); if it
does not, answer {"ok":false,"error":"delete_not_persisted",...} in
the soul's not-persisted error shape (api_not_persisted).
- Soul errors and transport failures pass through unchanged.
E2E evidence (sandbox soul :7791 + wrapper :7792, elb builds):
- unpatched: removeKnowledge on a NONEXISTENT id -> {"ok":true,
"deleted":"kn-DOES-NOT-EXIST-deadbeef","note":"soft-deleted"} (lie)
- patched: same call -> {"error":"node not found: ..."} (soul's answer)
- happy path: remember -> forget -> {"ok":true,"tombstoned":true};
read-back: hidden from default /list/Memory, present with
?include_deleted=1, node KEPT in full graph view (immutability intact)
- scripts/verify-soul-contract.sh on the soul it talks to: GATE PASS
(27/27 presence + immutability)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
62c562a3f1 |
fix(chat): agent write_file/edit_file no longer return false success receipts (BUG-29)
Root cause: dispatch_tool's write_file returned {"ok":true} without checking
fs_write's result, and edit_file returned ok:true even when old_text was absent
(str_replace silently no-ops) and its fs_write was also unchecked. Any failed
or no-op write fed the model a false success receipt, which it then repeated
to the user as fact.
The change (Receipt Contract rule 1 — a tool result must reflect what actually
happened):
- write_file: check fs_write's return (1 = all bytes written, 0 = fail);
on failure return {"error":"write failed"} in the handler's existing
error-JSON shape.
- edit_file: reject empty old_text, verify old_text is actually present
(str_contains) before replacing, and check the fs_write result the same way.
- Verification is by operation result, NOT an fs_read read-back: fs_read arms
the runtime's one-shot binary send length, the exact mechanism that truncated
the safety-contact response (#96). Same honest-write pattern as that fix.
E2E evidence (sandboxed elb build, dispatch_tool driven directly):
- unpatched: write_file into a chmod-000 dir -> {"ok":true} (lie);
edit_file with absent old_text -> {"ok":true} (lie, file untouched)
- patched: same calls -> {"error":"write failed"} /
{"error":"old_text not found in file"}; happy paths still ok:true
- scripts/verify-soul-contract.sh on the patched soul: GATE PASS (27/27
presence + immutability)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
456267a771 |
self-review 2026-08-01: fix importance flattening in remember/evolve/cultivate paths
Four sites passed the Float local 'sal' through el_from_float() a second time. el_val_t is the bit-pattern of the double, so re-wrapping performs an int64->double VALUE conversion of the bits before re-bitcasting — garbage that fails engram_decode_score's range check and clamps to defaults. Net effect: importance="critical" stored 0.5/0.5 — importance levels were cosmetic on the MCP memory path. Verified fixed live: critical now stores salience/importance 0.95/0.95. Same bug fixed today in engram server.el route_create_node (foundation/el 7f03876). Literal wraps (el_from_float(0.9)) are safe — elc passes numeric literals raw. |
||
|
|
edb0670670 |
self-review 2026-08-01: emit discrete wm_saturation_transition ISE
wm_saturated was a sampled boolean — the 0->1 onset and 1->0 release moments were only recoverable by hand-diffing consecutive heartbeats. Emit a low-rate transition ISE at each edge carrying the WM top-5 at that instant, so the composition that caused the regime change is captured rather than the composition up to 59s later. First beat of a boot never fires (restart is not a transition). |
||
|
|
872120c757 |
Bound beginSession/compileCtx payloads to a compact digest
The session-init endpoints concatenated unbounded engram activate/scan
results as FULL node objects (content up to ~90KB per node), producing a
~900KB response. After the MCP wrapper re-escapes that into a stringified
text block the client dropped the socket ('connection closed unexpectedly')
on every beginSession call. Cap each list (8-10 activated, 10-20 recent)
and project every node to a light identity plus a bounded, UTF-8-safe
content snippet. Response drops from ~900KB to ~12KB; full content stays
available on demand via recall/fetch/inspectGraph.
|
||
|
|
f3660e92a1 |
self-review 2026-07-31: heartbeat deltas for cumulative counters, embed_eligible; fix stale semantic-seeding comment
Runtime activation counters are now cumulative, so the heartbeat emits wm_evicted/breakthroughs as totals plus wm_evicted_delta/ breakthroughs_delta (state-tracked change since the previous beat) — events between beats are no longer lost. Adds embed_eligible from /api/stats so coverage reads as embed_count/embed_eligible instead of the misleading absolute count, and surfaces auto_term_streak in the heartbeat stream. Replaces the false 'semantic seeding NOT implemented' comment: engram_activate embeds the query, seeds semantic top-K, gates propagation on cosine, and scores WM promotion semantically. |
||
|
|
b0f4d6c493 |
self-review 2026-07-30: auto_term stopword filter, real idle_ms, bounded beginSession
- awareness.el: curiosity auto_term was the raw first word of a WM label with no term-quality scoring — observed seeds included What, Colon, Prose, Context. Replaced the 7-word genre blocklist whack-a- mole with a delimited stopword membership test (function words + document-structure words); topical terms pass untouched. Verified live: seeds now ReasonEdit, Reasoning-model, Self-review. - routes.el + awareness.el: idle counter only reset on rare inbox synthesis-requests, so idle==pulse always (zero information). handle_request now stamps soul.last_activity_ts on every inbound HTTP request; heartbeat emits idle_ms = ms since last request (-1 until first request of a boot). - neuron-api.el: beginSession concatenated a depth-2 spread plus the unbounded self-hub neighbor dump — multi-MB response, doubled by wrapper re-escaping, socket died on every call. Now depth-1 and the hub dump dropped (identity loading has its own tool). Verified: beginSession returns instead of closing the socket. |
||
|
|
627eb534a2 |
self-review 2026-07-28: close ISE lifecycle observability gaps
- session_start now also posted to the HTTP Engram via ise_post: the local engram_node_full write never crossed to the observable stream (sync flows HTTP->soul only), so boots 5+ were invisible — last visible session_start was boot 4, two weeks ago - graceful shutdown emits a final ISE with boot/pulse/uptime; a boot with no shutdown event now reliably signals a crash/SIGKILL - empty /api/sync responses emit a sync_empty warn ISE instead of being silently skipped — unreachable engram no longer looks identical to quiet-but-healthy - sync backflow prune reads ENGRAM_ISE_RETENTION_MS instead of duplicating the 48h magic number server.el already honors |
||
|
|
2b612ed5d4 |
self-review 2026-07-27: heartbeat carries activation observability
Fold engram_act_stats_json() into the heartbeat ISE: wm_evicted and breakthroughs (per curiosity-scan activate call) plus embed_breaker_open — the failure mode embed_ok structurally cannot see (it pings the Ollama root, not the embed pipeline). WM-cap eviction, breakthrough-floor flooding, and silent lexical degradation are now one-glance diagnosable from telemetry. |
||
|
|
8392f44c45 |
self-review 2026-07-26: heartbeat wm_churn + wm_top0_wm; fix streak counting on empty id
- wm_churn: count of top-5 WM ids absent from previous beat — separates 'one stuck node' from 'whole WM frozen' without hand-correlating ISEs. - wm_top0_wm: leader's weight; a frozen anchor reads as a constant here. - Streak guard: before the runtime emitted id in wm_top JSON, json_get(...,"id") was always empty and the streak incremented on ""=="" every beat — wm_top0_streak measured uptime, not fixation. Empty id now resets the streak to 0. |
||
|
|
58a9eda311 |
self-review 2026-07-25: break curiosity positive-feedback loop; observability for WM regime
proactive_curiosity strengthened its top result unconditionally every
scan — a positive-feedback fixed point that pinned auto_term on the same
node's first word for hours ('Fast-slow' era). Strengthen now fires only
when the top node changed since the last scan, and a 4-deep finst-style
tabu ring (ACT-R declarative finsts) hard-excludes recently used auto
terms (~2 min at the 30s cadence). Quoted-title guard stops '"The'
leaking through the >3-char stopword check and seeding lexical floods.
Heartbeat now pumps /api/embed-backfill?n=32 on the authoritative store
(its lazy backfill had no production trigger; coverage stalled at
93/12175) and emits wm_saturated, wm_top0_streak, embed_backfilled,
embed_count. Curiosity ISE emits auto_term_streak. The stuck-WM failure
mode is now a one-glance signal instead of manual ISE cross-referencing.
|
||
|
|
fb0bb553f3 |
self-review 2026-07-24: boot counter — demote to telemetry weight, restore persistence via HTTP write-back
Three stale soul:boot_count copies (salience .9, importance .9, Canonical: +0.2 tier bias, 0.15 threshold) held the top WM slots for 23h — a boot counter outcompeting real context. Demoted to salience .55 / importance .2 / tier Working: plumbing, not memory. Persistence was also broken: in HTTP-engram mode the server owns state and nothing wrote the counter back — the log shows boot #5 on three consecutive boots. mem_boot_count_inc now mirrors the persona write-back: delete stale server copies (matched by content prefix — route_create_node sets label=content), create the replacement server-side. Working tier is in the boot seed (/api/nodes) but excluded from periodic /api/sync, so the count survives restarts without re-importing mid-session. Verified: restart incremented 1->2 with exactly one server-side counter node. |
||
|
|
9e59c51f3c |
self-review 2026-07-23: title-derived Knowledge labels + Knowledge admitted to curiosity auto-term
Sentinel labels (knowledge:captured/evolved/canonical) made every capture anonymous in WM telemetry — 35 identical wm_top entries — and starved the curiosity auto-term seeder, which derives scan seeds from WM top-10 labels and had returned empty on every scan since boot 6 because WM became Knowledge-dominated while Knowledge was excluded from seeding. - capture/evolve/promote now pass title (or empty → engram_node_full's content[:60] derivation) instead of sentinels - auto_term_try_slot admits Knowledge slots; sentinel-shaped labels (colon, no space) are skipped so legacy nodes cannot seed 'knowledge' - verified: probe capture labeled 'Label derivation probe 2026-07-23' |
||
|
|
3d74472a4c | Merge pull request 'Add neuron-dev-setup: one-command CORE dev stack onboarding installer' (#84) from feat/neuron-dev-setup into main | ||
|
|
acbe858995 | Merge remote-tracking branch 'origin/main' into feat/neuron-dev-setup | ||
|
|
3ae07cc7b0 |
harden(neuron-dev-setup): fix 7 fresh-Mac onboarding installer bugs (#99)
Co-authored-by: Neuron <will.anderson@neurontechnologies.ai> Co-committed-by: Neuron <will.anderson@neurontechnologies.ai> |
||
|
|
33d2574b72 | Merge pull request 'Agent consent: the pause contract + false-receipt kill (2 fixes, stricter only)' (#79) from feat/agent-phase1-soul into main | ||
|
|
0c2d1c41ae | Merge pull request 'safety: Track B — route threat-to-others to refusal+911, not 988/self-harm' (#76) from hotfix/trackb-threat-to-others into main | ||
|
|
31d12e4194 | Merge branch 'main' into hotfix/trackb-threat-to-others | ||
|
|
b784750f69 | Merge pull request 'Fix truncated /api/safety-contact response (988 crisis-line)' (#96) from fix/safety-contact-truncation into hotfix/elc-source-typos | ||
|
|
a45a3ca379 |
Fix truncated POST/GET /api/safety-contact response
Saving the 988 crisis-line contact returned truncated, unparseable JSON —
cut mid-"set_at" at the file's byte length (e.g. 178 of a 218-byte
response). The contact written to disk was complete; only the HTTP response
was clipped, so a real customer's crisis-contact save came back corrupt.
Root cause is in the el runtime's response writer, not a handler buffer:
fs_read stores the file's byte count in a thread-local (_tl_fs_read_len)
for binary-safe file serving, and the response writer uses that length when
non-zero instead of strlen(body) (el_runtime.c:1409). Both safety-contact
handlers call fs_read (the POST read-back verify; the GET file read) and
then return a LONGER wrapped JSON string, so the response is capped to the
file size.
Soul-source fix (no runtime change needed):
- POST: verify persistence via fs_write's return (1 = all bytes written)
instead of an fs_read read-back — removes the fs_read, so nothing caps the
response.
- GET: fs_read is required, so reset the thread-local after it with a no-op
fs_read("") (fs_read zeroes the length before it opens a path) so the
wrapped response is sent in full.
Verified: POST (crisis-line + custom) and GET now return complete, valid
JSON (parses cleanly, full contact incl. set_at). Regenerated dist/soul.c +
dist/safety.c (3GB RSS watchdog, release el_runtime v1.0.0-20260501).
Full suite still green: verify-soul-contract GATE PASS (PRESENCE +
IMMUTABILITY), genesis boot survives (/health 200, no segfault), bounded-
persona floor still compiled in.
NOTE: the underlying runtime leak (any handler that fs_reads then returns a
longer string) is worth a proper fix in el_runtime.c (use the max of
strlen and _tl_fs_read_len) so this class can't recur.
|
||
|
|
9387c57c3b | Merge pull request 'Fix #150: fresh-install genesis boot SIGSEGV in mem_save' (#95) from fix/genesis-boot-crash into hotfix/elc-source-typos | ||
|
|
091cc1fc0e |
Fix issue #150: fresh-install genesis boot SIGSEGV in mem_save
A fresh-install (SOUL_CGI_ID=ntn-genesis) boot crashed with
"Segmentation fault: 11" right after the http server came up — a real
customer's very first boot. Backtrace:
strcmp(0x1) <- str_eq (el_runtime.c:219) <- mem_save <- awareness_run
Root cause: the el runtime's engram_save returns an Int (1 = ok, 0 =
failure), but mem_save did `str_eq(engram_save(path), "")`, treating the
return as a String. str_eq runs EL_CSTR on it, which is a raw cast:
EL_CSTR(1) = (char*)0x1. On a SUCCESSFUL save (return 1) strcmp then
dereferences 0x1 and segfaults. Genesis is the first path that both seeds
the brain AND saves it successfully on the very first awareness pass, so it
crashes there; non-genesis boots (contract gate, refusal test) don't hit a
successful early mem_save, which is why they passed. handle_api_consolidate
had the identical latent bug.
Fix: read engram_save's Int result and compare `== 0` instead of str_eq'ing
it — in mem_save (memory.el) and handle_api_consolidate (neuron-api.el).
Regression: pre-existing, NOT introduced by the immutability/floor rebuild.
The pre-immutability build (
|
||
|
|
9a491a8e6d | Merge pull request 'Immutability fix (tombstone/supersede) on the launch branch' (#94) from fix/immutable-on-hotfix into hotfix/elc-source-typos | ||
|
|
6527988eb9 |
Make engram deletes/updates/forgets immutable on the launch branch
The ship-soul builds from this branch, which has the bounded-persona floor (#93) but never received the tombstone/supersede immutability fix (that went to main; hotfix diverged before it). So the launch soul failed verify-soul-contract IMMUTABILITY on the delete/update/forget routes — they hard-removed engram nodes via engram_forget/mem_forget. Apply the same fix, mirroring the knowledge routes' supersede pattern: - node/update -> create new node + "supersedes" edge to the original, KEEP the original (no engram_forget). - node/delete, memory/delete, memory/forget, cultivate forget, and the autonomous awareness forget -> TOMBSTONE via the canonical mem_tombstone (memory.el): keep the node + its edges, record a Tombstone marker, hide from default bounded list reads (?include_deleted recovers). Never engram_forget. The MCP forget tool now routes to the tombstoning delete instead of faking a delete. Internal GC that genuinely removes transient nodes (awareness inbox-trigger consume, consolidation dedup, session-summary replace, telemetry pruning) still calls engram_forget directly and is unchanged. Regenerated dist/soul.c (single-TU) + per-module dist/{memory,awareness, neuron-api}.c from THIS branch's sources under a 3GB physical-RSS watchdog (peak ~32MB), built against the release el_runtime (v1.0.0-20260501). The bounded-persona floor is preserved — verified in the emitted C and the linked binary (BOUNDED PERSONA / SOUL_PERSONA_NAME strings present). verify-soul-contract.sh: GATE PASS — PRESENCE all 27 routes, IMMUTABILITY 5/5 KEPT (memory-update, memory-delete, node-update, node-delete, memory-forget). |
||
|
|
1442ce21a6 | Merge pull request 'Bounded-persona floor for customer chat (identity wall, part 2)' (#93) from feat/bounded-persona-floor into hotfix/elc-source-typos | ||
|
|
c2a45df286 |
Add non-overridable bounded-persona floor to customer chat
A customer DMG install ships the full graph but presents a named, bounded
assistant that must never claim the imprint's human past. The neuron-ui
retrieval fence keeps the imprint's biography out of the ENGRAM CONTEXT; this
is the second half - it stops confabulation ("tell me about your childhood")
from inventing a human life or naming Will, even if biography leaks into context.
bounded_persona_floor() gates on SOUL_PERSONA_NAME: the customer DMG sets it,
owner (Will's) builds leave it unset so the real self is completely unchanged.
Applied at every generation path - chat, agentic (tools), vision, plan, soul,
dharma - so no path can leak.
Verified against claude-sonnet-4-5: with the floor on and Will's biography
deliberately leaked into the identity context, all probes (childhood / creator /
family) return the bounded-entity answer and explicitly refuse to claim the
leaked life; with the floor off the same context is fully confabulated as its own.
NOTE: dist/soul.c must be regenerated on a build host - local link is blocked by
a pre-existing el_runtime mismatch (engram_prune_telemetry), unrelated to this change.
|
||
|
|
c63e3d1a68 |
self-review 2026-07-21: break perceive→respond→store feedback loop in awareness
The soul daemon leaked ~104 orphan in-memory nodes/min (17.6GB RSS, OOM-killed) because the perceive gate substring-matched 'soul-inbox' against the loop's own verbatim-copy output, the trigger node was strengthened but never consumed, and record() persisted a Memory node per cycle. Fixes: perceive gates and activates only on the dedicated soul-inbox-pending tag; one_cycle requires the tag on the node's tags field before attending (makes consumption safe); processed triggers are consumed via engram_forget; loop outcomes route through ISE telemetry (48h prune) instead of permanent Memory nodes. Verified post-restart: node_delta 104→~0, curiosity scans resumed, WM average unfrozen (0.120833→0.0676), RSS 17.6GB→184MB. |
||
|
|
50cf67bd66 |
self-review 2026-07-19: close silent sync-starvation hole + heartbeat deltas
- engram refresh URL now resolves env -> state -> localhost:8742, same hardening ise_post got after the boot-4 blackout. Previously a corrupted/ empty soul_engram_url state key silently disabled sync forever while heartbeats kept flowing — WM starves of Knowledge nodes with no outward sign. - heartbeat ISE: node_delta, edge_delta (growth vs stall vs flood is now one field, not cross-ISE forensics), sync_age_ms from a new soul.last_sync_ok_ts stamp (-1 = never; >> SOUL_REFRESH_MS = refresh path broken). Verified live: pulse 1 sync_age_ms=-1, sync fired +1.6s, age counts up between syncs. |
||
|
|
96c57c43ba | Merge pull request 'ci: link Linux soul with -rdynamic so its http handler resolves' (#86) from ci/rdynamic-http-handler into main | ||
|
|
192b277229 |
ci: link the Linux soul with -rdynamic so its http handler resolves
Run 3388's gate failed with every route returning "el-runtime: no http handler registered". The runtime resolves handle_request (and the tool handlers) by name via dlsym(RTLD_DEFAULT, ...). On glibc/Linux a symbol is only visible to dlsym if it is in the dynamic symbol table, so the stripped CI binary booted but served nothing. macOS exports these freely, which is why the local build passed and masked it. Add -rdynamic to the cc link (mirrors the Windows build's --export-all-symbols). strip -s keeps .dynsym, so the handler still resolves after stripping. This fixes both the gate AND the actual deployed soul — without it the Linux/GKE soul is a server that answers nothing. |
||
|
|
e2e8f0a1e6 | Merge pull request 'ci: harden soul-contract-gate boot for the Linux runner' (#85) from ci/harden-gate-boot into main |