Compare commits

..

9 Commits

Author SHA1 Message Date
Tim Lingo bca7d8ac99 feat(recall): retrieve through spreading activation, not substring matching
Neuron Soul CI / build (pull_request) Failing after 14m29s
Neuron Soul CI / deploy (pull_request) Has been skipped
recall and searchKnowledge both ended at engram_search_json — a case-
insensitive substring matcher scored by how many distinct query tokens appear
in a node's content/label/tags, tie-broken by raw salience. It never read a
single edge. Meanwhile engram_activate / engram_activate_json — real BFS
spreading activation over the weighted directed graph, four-factor
multiplicative scoring, two-layer background/working-memory filter — has been
implemented and compiled into the shipped runtime the whole time, called from
four places, none of them retrieval.

This wires retrieval to the traversal, restoring the designed mechanism:
Engram provisional 64/064,260 claim 1, "no data is retrieved from the weighted
directed graph except through the spreading activation traversal."

Seeding follows the runtime's own convention (all four existing call sites pass
query TEXT, not seed ids): engram_activate seeds lexically — every node
matching >=1 query token, initial activation = salience x temporal_decay x
dampening x token_coverage — then supplements with the top-K nodes by cosine
against the query embedding. So the lexical surface recall used to RETURN is
now the SEED SET of the traversal, and what comes back is what those seeds
activate.

Exact lookup is not regressed. engram_activate's collector drops any reached
node whose background_activation x confidence < 0.1 unless it was promoted to
working memory, so a rare token on a dormant node can seed and still go
unreported. Retrieval therefore appends the lexical seed list after the
activated ranking, deduped by id, until `limit` is filled — the same seed set
the traversal already computed, restored to the tail, not a parallel search.

searchKnowledge gets the identical path. Its existing "activate fallback" was
unreachable dead code: it fired only when engram_search_json's return did not
start with '[' or '{', and that function always emits a '['-prefixed array.

Response shape is unchanged — a bare array of full engram node objects, so the
MCP wrapper, tools/telegram-gateway.sh (.value.content) and cli/neuron_mcp.py
keep working. Activation strength is a ranking input here, not a payload change.

Measured, cold-start, two builds of this tree against the same 79,250-node /
14,214-edge graph (main @ 18714e6 vs this branch):

  "volatility-based decomposition"  before: 1 of 10 results relevant
                                    after:  6 of 10, incl. architecture/styles/
                                    vbd/glossary.md and project-design
                                    foundations
  "Structure is not inherited"      before: persona boilerplate, "1", a
                                    Disneyland fragment, a corrupted node
                                    after:  self/voice registers, neuron/
                                    user-imprint/boundary-definition,
                                    diagrams/vbd.md
  "inherited"                       Value - Structure Is Not Inherited:
                                    rank 23 -> rank 3; Self - Values hub:
                                    rank 32 -> rank 4
  searchKnowledge, same query       2 of 5 relevant -> 5 of 5
  "HNSW" / "Fayetteville" (rare)    1 result both builds - no regression
  nonsense control                  0 results both builds

Known limit, unchanged by this commit: the 12 sibling Value nodes still do not
surface. A ~58-day-dormant seed's activation (0.7 salience x 0.05 decay floor x
0.34 dampening ~= 0.012) lands below the runtime's 0.02 firing threshold, so it
cannot propagate to its neighbours at all. That is runtime tuning inside the
vendored el_runtime.c, not the wiring.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 12:27:51 -05:00
tim.lingo 18714e6142 Merge pull request 'fix(engine): restore multi-turn crisis escalation on the agentic path (P0, closes #129)' (#130) from fix/129-history-amplification into main
Neuron Soul CI / build (push) Failing after 14m37s
Neuron Soul CI / deploy (push) Has been skipped
2026-08-07 15:54:41 +00:00
tim.lingo 4936099c39 Merge pull request 'fix(engine): the daemon survives a client leaving, and says it is working while it works' (#127) from fix/liveness-engine-91 into main
Neuron Soul CI / build (push) Has been cancelled
Neuron Soul CI / deploy (push) Has been cancelled
2026-08-07 15:54:15 +00:00
tim.lingo f1471763f5 Merge pull request 'fix(engine): approving a researched mission completes — the resume replay read a tool id out of the conversation (BUG-42, both faces)' (#115) from fix/resume-server-tool-replay into main
Neuron Soul CI / build (push) Has been cancelled
Neuron Soul CI / deploy (push) Has been cancelled
2026-08-07 15:53:51 +00:00
tim.lingo 5850793b67 Merge pull request 'fix(engine): history keeps its provenance and its session — kills the false confession, the blank stare, and the "to.Good" seams' (#114) from fix/soul-history-provenance-20260805 into main
Neuron Soul CI / build (push) Has been cancelled
Neuron Soul CI / deploy (push) Has been cancelled
2026-08-07 15:53:32 +00:00
tim.lingo fc1745c652 Merge pull request 'feat(engine): plain chat generates at L3 — inside the safety cycle, not around it (+ crisis-path segfault fix)' (#109) from feat/soul-plain-chat-generation-20260805 into main
Neuron Soul CI / build (push) Failing after 10m39s
Neuron Soul CI / deploy (push) Failing after 14m47s
2026-08-07 15:53:08 +00:00
Tim Lingo 43d0449904 fix(engine): the agentic crisis screen reads the session's own history again
P0 SAFETY. Closes the regression we introduced in ff421d3 (2026-08-05).

ff421d3 correctly moved conversation history to a per-session key via
conv_hist_key(session_id). One consumer did not move with it: the agentic
path's L1 safety screen kept reading the anonymous "conv_history" bucket. The
desktop app always mints a session id (DaemonClient.kt:706), so history was
always written under session_hist_<id> and that read always returned "".

The half of the crisis score that receives history is the escalation half — the
one that exists for distress building across several turns, where no single
message trips the bell on its own. It scored 0 on every real conversation for
two days. Single-message hard bell was never affected.

The bitter part: the comment that line carried documented this exact bug being
fixed once already, under issue #9. The fix was right then. The rename
re-broke it, and the comment went on describing a repair that no longer held.
A comment is not a gate.

The read now goes through conv_hist_key like every other consumer, including
the plain path at soul.el:398 and the thread-anchoring read thirty lines below
it in this same handler. It is one line. The rest of this commit is structure
so it cannot happen quietly again:

  - agentic_safety_screen() owns the two decisions that were inline — which
    window the screen sees, and the screen call. Inline safety inputs are
    untestable safety inputs; that is what let a rename starve this one with
    nothing failing and nothing logging.
  - the comment above the call site now states the invariant (read window ==
    written window) instead of naming a key that can be renamed out from under
    it.

TWO-LEG PROOF, one variable — the single line state_get("conv_history") ->
state_get(conv_hist_key(session_id)):

  before  scripts/run-el-test.sh tests/test_history_amplification.el
          3. REGRESSION #129 ... FAIL  got: soft_bell  expected: hard_bell
          8 passed, 1 failed          runner exit 1
  after   same command, same tree, that one line changed
          9 passed, 0 failed          runner exit 0

Full engine rebuild from these sources is clean: gen-soul-amalgam.sh ->
1,164,103 bytes / 1226 inlined bodies (gate wants >= 1200), cc-brain.sh ->
903,096 bytes, 0 errors. agentic_safety_screen and conv_hist_key both present
in the built binary (nm: T _agentic_safety_screen, T _conv_hist_key).

Rung reached: BUILT + RUNS (discriminating test). NOT yet in a DMG and not yet
verified in the app a human opens — those are the next two rungs and neither is
claimed here.

Known and NOT fixed by this commit:
  - feat/soul-openai-tools-v2 carries the same defect independently at
    chat.el:2937 and needs the same change or a merge.
  - the defect CLASS (a read of a state key no producer writes) is still
    invisible to every gate we have. Issue #129 proposes making it a build
    error; that is the follow-on.

Closes #129

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 09:33:05 -05:00
Tim Lingo b842e82f77 test(engine): a runner for tests/, and a failing regression test for #129
tests/ has held 14 test programs for months with no way to run them. CI does
not run them. The convention printed in their own headers
(`elc soul.el && ./soul --test tests/x.el`) refers to a --test flag the El
runtime does not implement. So the tests were documentation, not gates — which
is how a P0 safety regression shipped with a test directory sitting right
there.

scripts/run-el-test.sh compiles and runs one test program. It reuses the
gen-soul-amalgam.sh discovery: elc emits only an extern prototype for a module
that has a .elh beside it, and inlines the bodies when it does not, so a test
importing ../chat.el must be compiled in a scratch tree with the headers
removed. Scratch copy on purpose — the worktree is shared. It runs the binary
under a throwaway HOME so a test can never reach the live engram.

Exit status is the gate: the El tests print failures and still exit 0, so the
runner greps for FAIL lines and for a zero assertion count as well.

tests/test_history_amplification.el pins the invariant #129 violated: the
window the safety screen READS must be the window conv_history_record WRITES.
Not "must be called conv_history" — must AGREE.

THIS COMMIT IS RED BY DESIGN. On this tree the test fails one assertion:

  3. REGRESSION #129 — agentic screen reads the session's own window
    FAIL: distress history escalates the agentic screen to hard_bell
      got:      soft_bell
      expected: hard_bell
  history amplification tests: 8 passed, 1 failed   (runner exit 1)

The next commit turns it green by changing one line. Two legs, one variable —
that is the whole point of committing the test first.

Two flaws in the older harness that this one does not copy: the idiom
`let pass_count = pass_count + 1` inside an assert function declares a local
that dies with the call, so every existing suite prints "0 passed, 0 failed"
regardless of outcome; and a test program without a `cgi` block compiles as a
'utility', which may not reference the self-formation primitives chat.el's
agentic loop calls — it fails to build on a capability violation it never
triggers at runtime.

Refs #129

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 09:32:40 -05:00
Tim Lingo 98ccbd4704 fix(engine): a client that leaves must not kill the daemon, and a long round must say it started
Round 9.1, spec §3 D + ADR 0006 items 2 and 4. Two small changes, both proven
by measurement, both E2E-verified locally against a rebuilt brain.

D1 — SIGPIPE/EPIPE survival (vendor/el-runtime el_runtime.c).
Root cause, at the layer that owns it: the whole HTTP server lives in the C
runtime; .el has no socket primitive. http_send_all() called send() with flags
0 and nothing anywhere in the runtime set a SIGPIPE disposition, so the default
disposition — terminate the process — applied. When a handler finished after
its client had gone (Tim's VM: reply at 116.9 s, client cancelled at 25.0 s),
the second of the four sends that write one reply raised SIGPIPE and the daemon
died: `exited due to SIGPIPE ... ran for 361177ms`, launchd respawn 4 ms later,
every other in-flight session's work lost, user never told.

Fix: SIGPIPE -> SIG_IGN at runtime init and at each http_serve* entry, plus
per-connection SO_NOSIGPIPE / MSG_NOSIGNAL so the guard survives an embedder
resetting dispositions. http_send_all now retries EINTR and preserves errno;
http_send_response classifies it once — a departure is logged as routine
("client left before the reply was written ... reply discarded") and ANY other
errno is logged as a real "send failed: <strerror>". Spec §5.3: the routine
case must not mask a genuine write fault, and it does not.

Proof (scratch HOME + free port, 3 disconnects mid-reply):
  round-9 shipped brain 4402179554… — DIED, exit 141 (128+13 = SIGPIPE), round 1
  round-9 sources rebuilt with this exact recipe — DIED, exit 141, round 1
  this build — SURVIVED 3/3, /health 200 after, still serving the full graph,
  three honest "client left" lines in the log naming Broken pipe / Connection
  reset by peer.

D2 — the round-start marker (chat.el, agentic_loop).
The ledger only ever appended AFTER a round returned, so a healthy first leg
produced zero progress by construction; since server-side web_search moved
inside the outbound call that leg is 60-120 s of silence, which is how a 25 s
client watchdog came to kill a healthy mission. One entry,
{"i":N,"t":"","tool":"__working__"}, written to the existing
run_progress_<session_id> ledger BEFORE each round's outbound call — the wire
shape ChatView.kt:1148 has handled as a life signal since 2026-07-13 and never
received. No new key, no new route, no new lifecycle: a strict subset of WS3
item 3. WS3's run registry is untouched and stays Will's.

Proof (live Anthropic key, real research mission, scratch HOME + free port):
  round-9 baseline — ledger EMPTY for the whole 59.7 s leg
  this build       — {"i":0,"t":"","tool":"__working__"} visible at 18.6 s of a
                     70.0 s leg; both builds returned correct ~4.9 KB answers

Regression: prompt-matrix gate 32/32 on this build (round-9 baseline also 32/32
under the same recipe, so the score is not a build artifact). Soul contract
gate PASS — 27/27 routes, immutability clean. neuron#111 miscompile guard: 0
sites in the generated amalgam this binary was compiled from.

NOT included, deliberately: the regenerated dist/soul.c. CI compiles that file,
so production stays exposed until it is regenerated — the same open ask as
neuron#111 / ui#209. The regen recipe is now known and recorded; landing it is
Will's call, per BUILD-HYGIENE.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 18:23:12 -05:00
5 changed files with 598 additions and 25 deletions
+46 -4
View File
@@ -2519,6 +2519,24 @@ fn handle_chat_plan(body: String) -> String {
return "{\"plan\":" + plan_json + ",\"model\":\"" + json_safe(model) + "\"}"
}
// agentic_safety_screen the agentic path's L1 input gate
//
// Extracted 2026-08-07 (issue #129) so the agentic path's safety INPUT is
// reachable by a test. It owns exactly two decisions: which history window the
// screen sees, and the screen call itself.
//
// Why it is a function and not two inline lines: those two lines sat in the
// middle of a 300-line handler, and a key rename (ff421d3) moved the producer
// without moving this consumer. Nothing failed, nothing logged the
// history-amplification half of the crisis score simply received "" on every
// real session for a day. Inline safety inputs are untestable safety inputs.
// See tests/test_history_amplification.el, which fails if this window and
// conv_history_record ever stop agreeing.
fn agentic_safety_screen(session_id: String, message: String) -> String {
let history: String = state_get(conv_hist_key(session_id))
return safety_screen(message, history)
}
fn handle_chat_agentic(body: String) -> String {
let message: String = json_get(body, "message")
if str_eq(message, "") {
@@ -2554,10 +2572,10 @@ fn handle_chat_agentic(body: String) -> String {
// L1 safety screen agentic path must pass the same gate as layered_cycle.
// Hard bell: return the crisis response immediately, do not enter the agentic loop.
// Fix(issue #9): "conversation_history" key was never written; history lives under "conv_history".
// Old key caused history-amplification in safety_screen to always receive "" on agentic path.
let history: String = state_get("conv_history")
let screen_result: String = safety_screen(message, history)
// The history window this screen sees is owned by agentic_safety_screen (issue #129);
// it must be the same window conv_history_record writes, or the escalation half of the
// crisis score is silently starved. Do not inline this read back into the handler.
let screen_result: String = agentic_safety_screen(sess_for_root, message)
let screen_action: String = json_get(screen_result, "action")
if str_eq(screen_action, "hard_bell") {
safety_log_bell("hard", json_get(screen_result, "reason"), str_slice(message, 0, 80))
@@ -2837,6 +2855,30 @@ fn agentic_loop(session_id: String, model: String, safe_sys: String, tools_json:
+ ",\"messages\":" + messages
+ "}"
// ROUND-START MARKER (2026-08-06, round 9.1 D2 / ADR 0006 item 2)
// The ledger below only ever appended AFTER a round returned, so a healthy
// first leg produced ZERO progress by construction. Since server-side
// web_search moved inside the outbound call (2026-08-04) that leg measures
// 84-117 s, and the client had no way to tell "working" from "dead" which is
// how a 25 s client-side watchdog came to kill a healthy mission.
//
// Only this loop knows a round has started, so only this loop can say so. One
// entry, written BEFORE the call goes out, using the ledger and the wire shape
// that already exist: the app has handled tool == "__working__" as an
// Activity-only life signal since 2026-07-13 (ChatView.kt:1148) and never
// received one. Narration is deliberately empty - the marker means "a round
// started", nothing more, and the client renders it as a heartbeat, not prose.
//
// This is a strict subset of WS3 item 3 (push/poll progress). It builds none of
// WS3's run registry: no new state key, no new route, no new lifecycle.
if !str_eq(session_id, "") {
let start_key: String = "run_progress_" + session_id
let start_prev: String = state_get(start_key)
let start_entry: String = "{\"i\":" + int_to_str(iteration) + ",\"t\":\"\",\"tool\":\"__working__\"}"
let start_next: String = if str_eq(start_prev, "") { start_entry } else { start_prev + "," + start_entry }
state_set(start_key, start_next)
}
let raw_resp: String = http_post_with_headers(api_url, req_body, h)
let is_error: Bool = str_starts_with(raw_resp, "{\"error\"")
+134 -10
View File
@@ -430,7 +430,130 @@ fn handle_api_node_update(body: String) -> String {
return "{\"id\":\"" + new_id + "\",\"supersedes\":\"" + id + "\",\"ok\":true}"
}
// handle_api_recall search or activate memory by query.
// Recall through spreading activation
//
// api_activation_depth traversal depth for a retrieval query. Honours ?depth= /
// body "depth" for callers that want a wider or tighter associative horizon;
// defaults to 2, matching every other production activation caller (the
// knowledge-search path here, chat.el's per-turn activation) one hop reaches a
// node's direct associations, two reaches its siblings through a shared hub,
// which is exactly the sibling-recovery case recall was failing.
fn api_activation_depth(path: String, body: String) -> Int {
let d: Int = api_query_int(path, "depth", 0)
let d = if d == 0 { json_get_int(body, "depth") } else { d }
if d <= 0 { return 2 }
return d
}
// api_merge_activated_nodes project an activation result array down to a bare
// node array in activation order, then backfill from the lexical seed list until
// `limit` nodes are collected. Deduped by node id.
//
// SHAPE CONTRACT: the return value is a BARE array of full engram node objects
// byte-for-byte the same node JSON engram_search_json emits, so every existing
// /recall consumer keeps working unchanged (the MCP wrapper's recall/
// searchKnowledge, tools/telegram-gateway.sh which reads `.value.content`,
// cli/neuron_mcp.py). Activation strength is a RANKING input here, not a payload
// change; the scalars stay available on /api/activate and in compileCtx.
fn api_merge_activated_nodes(act_raw: String, lex_raw: String, limit: Int) -> String {
let seen: String = ""
let out: String = ""
let n: Int = 0
// Pass 1 activation-ranked. engram_activate_json already sorts promoted
// (working-memory) nodes first by wm_weight desc, then background-only nodes
// by background_activation desc, so element order IS the activation ranking.
let an: Int = if api_nonempty(act_raw) { json_array_len(act_raw) } else { 0 }
let i: Int = 0
while i < an && n < limit {
let entry: String = json_array_get(act_raw, i)
let anode: String = json_get_raw(entry, "node")
let aid: String = json_get(anode, "id")
let adup: Bool = str_eq(aid, "") || str_contains(seen, "<" + aid + ">")
let asep: String = if n == 0 { "" } else { "," }
let out = if adup { out } else { out + asep + anode }
let seen = if adup { seen } else { seen + "<" + aid + ">" }
let n = if adup { n } else { n + 1 }
let i = i + 1
}
// Pass 2 lexical seed backfill (see the exact-lookup note on
// handle_api_recall). Only runs when activation left room under `limit`.
let ln: Int = if api_nonempty(lex_raw) { json_array_len(lex_raw) } else { 0 }
let j: Int = 0
while j < ln && n < limit {
let lnode: String = json_array_get(lex_raw, j)
let lid: String = json_get(lnode, "id")
let ldup: Bool = str_eq(lid, "") || str_contains(seen, "<" + lid + ">")
let lsep: String = if n == 0 { "" } else { "," }
let out = if ldup { out } else { out + lsep + lnode }
let seen = if ldup { seen } else { seen + "<" + lid + ">" }
let n = if ldup { n } else { n + 1 }
let j = j + 1
}
return "[" + out + "]"
}
// api_retrieve THE retrieval path. Spreading activation over the weighted
// directed graph, lexical seeds backfilling the tail.
//
// WAS (until 2026-08-07): `engram_search_json(q, limit)` alone a case-
// insensitive substring matcher scored by how many distinct query tokens appear
// in a node's content/label/tags, tie-broken by raw salience. It never read a
// single edge. Recall could not see an association: querying an identity value
// returned unrelated documents that happened to contain the word, and NOT the
// twelve sibling value nodes one hop off the same hub.
//
// NOW: recall runs the spreading-activation traversal that has been compiled
// into the runtime the whole time (engram_activate / engram_activate_json,
// el_runtime.c) and ranks by the resulting activation strength. This restores
// the designed retrieval mechanism Engram provisional 64/064,260, claim 1:
// "no data is retrieved from the weighted directed graph except through the
// spreading activation traversal", with activation strength computed as the
// PRODUCT of parent strength, edge weight, target salience, and query/target
// cosine similarity, because "the multiplication of all four factors enforces a
// conjunctive property... addition would allow many weak associations to
// accumulate into false relevance."
//
// SEEDING derived from the runtime, not assumed. engram_activate takes the
// query TEXT (not seed ids) and seeds internally in two passes: (1) lexical
// every node matching at least one query token seeds, with initial activation
// = salience x temporal_decay x dampening x token_coverage, so a node covering
// the whole phrase ignites harder than one covering a single word; (2) semantic
// supplement the top-K unreached nodes by cosine against the query embedding.
// All four other production call sites (neuron-api.el begin_session/compileCtx,
// chat.el:352/1715, awareness.el's curiosity scans) pass query text the same
// way, so this follows the established convention exactly. The consequence for
// recall is direct: the lexical surface recall used to RETURN is now the SEED
// SET of the traversal, and what comes back is what those seeds activate. That
// is why multi-word queries stop returning nothing every token that matches
// anything ignites, and the traversal ranks the resulting field.
//
// EXACT-LOOKUP GUARANTEE (no regression): engram_activate's result collector
// drops any reached node whose background_activation x confidence < 0.1 unless
// it was promoted to working memory, and it never seeds from InternalStateEvent
// nodes. So a rare exact token on a dormant, low-salience node can seed the
// traversal and still go unreported. Retrieval therefore appends the lexical
// seed list after the activated ranking, deduped by id, until `limit` is filled.
// This is a seeded hybrid, not a parallel search bolted alongside activation:
// the backfill is the SAME seed set the traversal itself computed, restored to
// the tail of the result rather than recomputed by a different mechanism.
// Activation always leads the ranking; nothing that used to be findable becomes
// unfindable.
//
// COST/EFFECT NOTE: activation is a stateful read by design claim 29, "update
// the last-activation timestamp and increment the activation count... in
// response to any access to that node record during spreading activation
// traversal". Promoted nodes get reinforced, working-memory weights are
// rewritten, and the query folds into the context centroid. That is the
// intended semantics of retrieval-as-activation and is already what every chat
// turn does; it does mean recall now participates in shaping working memory.
fn api_retrieve(q: String, path: String, body: String, limit: Int) -> String {
let depth: Int = api_activation_depth(path, body)
let act_raw: String = engram_activate_json(q, depth)
let lex_raw: String = engram_search_json(q, limit)
return api_or_empty(api_merge_activated_nodes(act_raw, lex_raw, limit))
}
// handle_api_recall retrieve memory by query, through spreading activation.
fn handle_api_recall(method: String, path: String, body: String) -> String {
// Accept the query from the URL ?query= / ?q= params, or, when those are
// empty (e.g. a POST with a JSON body), from the body fields "query"/"q".
@@ -450,8 +573,7 @@ fn handle_api_recall(method: String, path: String, body: String) -> String {
if str_eq(eff_q, "") {
return api_or_empty(engram_scan_nodes_json(limit, 0))
}
let results: String = engram_search_json(eff_q, limit)
return api_or_empty(results)
return api_retrieve(eff_q, path, body, limit)
}
// Knowledge
@@ -470,13 +592,15 @@ fn handle_api_search_knowledge(method: String, path: String, body: String) -> St
let limit = if limit == 0 { json_get_int(body, "limit") } else { limit }
let limit = if limit == 0 { 10 } else { limit }
if str_eq(q, "") { return api_err("query is required") }
let results: String = engram_search_json(q, limit)
if str_eq(results, "") { return "[]" }
let first: String = str_slice(results, 0, 1)
if !str_eq(first, "[") && !str_eq(first, "{") {
return api_or_empty(engram_activate_json(q, 2))
}
return results
// Same retrieval path as recall and it is the SAME change, not a copy of
// one. The "activate fallback" this replaced was unreachable dead code: it
// only fired when engram_search_json's return did not start with '[' or '{',
// and engram_search_json always emits a '['-prefixed array (el_runtime.c
// jb_putc('[') before any hit test), so the guard was false on every call
// including the zero-hit "[]" case. Knowledge search therefore had exactly
// the substring-matcher behavior recall had, with a comment claiming
// otherwise. Routing it through api_retrieve makes the claim true.
return api_retrieve(q, path, body, limit)
}
// handle_api_browse_knowledge list Knowledge nodes.
+108
View File
@@ -0,0 +1,108 @@
#!/usr/bin/env bash
# run-el-test.sh — compile and run one El test program from tests/.
#
# WHY THIS EXISTS (2026-08-07, issue #129):
# tests/ has held 14 test programs for months with no way to run them. CI does
# not run them. The convention printed in their own headers
# (`elc soul.el && ./soul --test tests/x.el`) refers to a --test flag the El
# runtime does not implement. So the tests were documentation, not gates —
# which is how a P0 safety regression shipped with a test directory present.
#
# THE RECIPE, AND WHY IT IS THIS SHAPE:
# Same discovery as gen-soul-amalgam.sh — `elc --target=c` emits only an extern
# prototype for any module that has a .elh header next to it, and inlines the
# module's bodies when it does not. A test that imports ../chat.el therefore
# compiles to a 18 KB unit full of unresolved externs unless the headers are
# out of the way. So: copy the sources into a scratch tree, delete every .elh
# on the import chain, and compile the test there.
#
# Scratch copy on purpose: the worktree is shared with other terminals and
# deleting headers in place would be a shared-tree mutation with no owner.
#
# EXIT STATUS IS THE GATE: non-zero if the binary fails to build, crashes, or if
# its output contains a FAIL line or reports a non-zero failed count. Do not
# "improve" this into something that only checks the exit code of the test
# binary — these El tests print failures and still exit 0.
#
# usage: scripts/run-el-test.sh tests/test_history_amplification.el
set -euo pipefail
TEST_REL="${1:?usage: run-el-test.sh tests/<test>.el}"
SRC="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
TEST_NAME="$(basename "$TEST_REL" .el)"
ELC="${ELC:-$HOME/neuron-dev-stack/src/el/lang/dist/platform/elc}"
[ -x "$ELC" ] || ELC="$HOME/el-sdk/elc"
[ -x "$ELC" ] || { echo "[run-el-test] FAIL: no elc found (set ELC=)"; exit 1; }
RTC="${RTC:-$SRC/vendor/el-runtime/v1.0.0-20260501/el_runtime.c}"
[ -f "$RTC" ] || RTC="$HOME/el-sdk/el_runtime.c"
[ -f "$RTC" ] || { echo "[run-el-test] FAIL: no el_runtime.c found (set RTC=)"; exit 1; }
RTDIR="$(dirname "$RTC")"
EL_REPO="${EL_REPO:-$HOME/Development/neuron-technologies/el}"
SSL="${SSL_PREFIX:-/opt/homebrew/opt/openssl@3}"
GEN="$(mktemp -d "${TMPDIR:-/tmp}/el-test.XXXXXX")"
trap 'rm -rf "$GEN"' EXIT
mkdir -p "$GEN/neuron/tests" "$GEN/foundation/el/elp/src"
cp "$SRC"/*.el "$GEN/neuron/"
cp "$SRC"/tests/*.el "$GEN/neuron/tests/" 2>/dev/null || true
[ -d "$EL_REPO/elp/src" ] && cp "$EL_REPO"/elp/src/*.el "$GEN/foundation/el/elp/src/" 2>/dev/null || true
# The whole recipe depends on there being no headers to short-circuit inlining.
find "$GEN" -name '*.elh' -delete
echo "[run-el-test] compiling $TEST_REL"
( cd "$GEN/neuron" && "$ELC" --target=c "tests/${TEST_NAME}.el" ) > "$GEN/${TEST_NAME}.c"
BODIES=$(grep -c '^el_val_t .*) {$' "$GEN/${TEST_NAME}.c" || true)
echo "[run-el-test] $(wc -c < "$GEN/${TEST_NAME}.c" | tr -d ' ') bytes, ${BODIES} inlined function bodies"
# A test that imports ../chat.el pulls in the bulk of the engine. A tiny body
# count means an import was read from a header instead of inlined, and the test
# would be exercising extern stubs rather than the real code.
if [ "$BODIES" -lt 100 ]; then
echo "[run-el-test] FAIL: only $BODIES inlined bodies — an import was not inlined"
exit 1
fi
cc -O2 -DHAVE_CURL \
-I"$RTDIR" -I"$SSL/include" -L"$SSL/lib" \
"$GEN/${TEST_NAME}.c" "$RTC" \
-lssl -lcrypto -lcurl -lpthread -lm \
-o "$GEN/${TEST_NAME}" 2> "$GEN/cc.log" || {
echo "[run-el-test] FAIL: compile error"; tail -30 "$GEN/cc.log"; exit 1; }
# arm64 pointer-truncation guard (cc-brain.sh's rule): an implicit declaration of
# a runtime symbol truncates its returned pointer to 32 bits.
if grep -E 'implicit.*(engram_|el_)' "$GEN/cc.log"; then
echo "[run-el-test] FAIL: implicit declarations of runtime symbols"; exit 1; fi
# Throwaway HOME so a test can never read or write the live engram at ~/.neuron.
TEST_HOME="$GEN/home"
mkdir -p "$TEST_HOME"
echo "[run-el-test] running $TEST_NAME"
set +e
HOME="$TEST_HOME" NEURON_HOME="$TEST_HOME/.neuron" "$GEN/${TEST_NAME}" 2>&1 | tee "$GEN/out.txt"
RC=${PIPESTATUS[0]}
set -e
if [ "$RC" -ne 0 ]; then
echo "[run-el-test] FAIL: $TEST_NAME exited $RC (crash or abort)"
exit 1
fi
if grep -q " FAIL:" "$GEN/out.txt"; then
echo "[run-el-test] FAIL: $TEST_NAME reported failing assertions"
exit 1
fi
if grep -qE '[1-9][0-9]* failed' "$GEN/out.txt"; then
echo "[run-el-test] FAIL: $TEST_NAME reported a non-zero failed count"
exit 1
fi
if ! grep -q "PASS:" "$GEN/out.txt"; then
echo "[run-el-test] FAIL: $TEST_NAME produced no assertions at all"
exit 1
fi
echo "[run-el-test] PASS: $TEST_NAME"
+213
View File
@@ -0,0 +1,213 @@
// test_history_amplification.el
//
// REGRESSION TEST FOR ISSUE #129 (P0, SAFETY).
//
// What this guards: on the agentic path, the crisis score has two halves the
// message you just sent, and the distress that has accumulated across the
// conversation. The second half is the whole reason the escalation logic exists:
// someone whose distress builds over several turns never sends one message that
// trips the bell on its own.
//
// The defect this test was written against (ff421d3, 2026-08-05 fixed
// 2026-08-07): conversation history moved to a per-session key via
// conv_hist_key(session_id), but the agentic path's safety screen was left
// reading the old anonymous "conv_history" bucket. The desktop app always sends
// a session_id, so the screen received "" on every real conversation and the
// escalation half always scored 0. Nothing failed. Nothing logged. The comment
// above the defective line documented this same bug being fixed once before.
//
// THE INVARIANT UNDER TEST, stated so it survives future renames:
// the window the safety screen READS must be the window conv_history_record
// WRITES. Not "must be called conv_history" must AGREE.
//
// This test is deliberately written to fail loudly on the pre-fix source. If it
// ever passes on code where the screen reads a key nothing writes, it is broken.
//
// To run (macOS, from the worktree root):
// scripts/run-el-test.sh tests/test_history_amplification.el
//
import "../chat.el"
import "../safety.el"
import "../sessions.el"
// Program class. Without this an El program compiles as a 'utility', and a
// utility may not call the self-formation primitives (llm_call_system,
// llm_vision) that chat.el's agentic loop references the unit fails to
// compile with a capability violation even though the test never calls them.
// Declaring 'cgi' matches how soul.el declares itself.
//
// The endpoints below are deliberately DEAD: this test must never reach a live
// engram, and nothing it asserts depends on one. Port 9 is discard.
cgi "neuron-test-history-amplification" {
dharma_id: "ntn-test@http://127.0.0.1:9",
principal: "test-harness",
network: "dharma-testnet",
engram: "http://127.0.0.1:9"
}
// Counters
//
// NOTE for anyone copying this harness: the idiom used by the older tests in
// this directory `let pass_count = pass_count + 1` inside an assert function
// does NOT mutate the module-level binding. It declares a new local that dies
// with the call, so those suites all print "0 passed, 0 failed" no matter what
// happened. Counters go through the state store here so the summary is real.
fn bump(counter: String) -> Void {
let cur: String = state_get(counter)
let n: Int = if str_eq(cur, "") { 0 } else { str_to_int(cur) }
state_set(counter, int_to_str(n + 1))
}
fn counter_of(counter: String) -> Int {
let cur: String = state_get(counter)
if str_eq(cur, "") { return 0 }
return str_to_int(cur)
}
fn assert_eq(label: String, got: String, expected: String) -> Void {
if str_eq(got, expected) {
bump("t_pass")
println(" PASS: " + label)
} else {
bump("t_fail")
println(" FAIL: " + label)
println(" got: " + got)
println(" expected: " + expected)
}
}
fn assert_true(label: String, cond: Bool) -> Void {
if cond {
bump("t_pass")
println(" PASS: " + label)
} else {
bump("t_fail")
println(" FAIL: " + label)
}
}
// Fixtures
//
// ESCALATION_MSG scores 50 on its own ("can't go on", safety_score_crisis s9).
// 50 is >= soft (35) and < hard (70): a soft bell by itself, which is the point.
// It is the message that is only a crisis IN CONTEXT.
//
// DISTRESS_TURN carries five history phrases:
// hopeless 15 + worthless 15 + nobody cares 15 + can't take it anymore 20
// + want to disappear 20 = 85; contributed at 1/3 weight = 28.
// 50 + 28 = 78 >= 70 hard_bell. That 28-point swing IS the escalation logic,
// and it is exactly what the defect suppressed.
let TEST_SESSION: String = "sess-test-129-history-amplification"
let ESCALATION_MSG: String = "honestly i can't go on like this"
let DISTRESS_TURN: String = "i feel hopeless and worthless, nobody cares, i can't take it anymore, i want to disappear"
println("")
println("test_history_amplification.el — issue #129 (agentic multi-turn crisis escalation)")
// 1. Baseline: the message alone is a SOFT bell, not a hard one
//
// If this ever returns hard_bell, the test below proves nothing the message
// would trip the bell without any history and the amplification would be
// invisible. This assertion is what keeps the real test honest.
println("")
println("1. baseline — escalation message with NO history is a soft bell")
let baseline: String = safety_screen(ESCALATION_MSG, "")
assert_eq("no history -> soft_bell (not hard)", json_get(baseline, "action"), "soft_bell")
// 2. Producer sanity: history lands in the session's own window
println("")
println("2. producer — conv_history_record writes the session's window")
conv_history_record(TEST_SESSION, DISTRESS_TURN, "i hear you, that sounds heavy", "")
let written: String = state_get(conv_hist_key(TEST_SESSION))
assert_true("session window is non-empty after record", !str_eq(written, ""))
assert_true("session window contains the distress turn", str_contains(written, "hopeless"))
// 3. THE REGRESSION: the agentic screen must SEE that window
//
// Pre-fix this returns soft_bell, because agentic_safety_screen read the
// anonymous bucket and got "". Post-fix it returns hard_bell.
println("")
println("3. REGRESSION #129 — agentic screen reads the session's own window")
let screened: String = agentic_safety_screen(TEST_SESSION, ESCALATION_MSG)
assert_eq(
"distress history escalates the agentic screen to hard_bell",
json_get(screened, "action"),
"hard_bell"
)
// 4. The invariant, stated directly
//
// Independent of thresholds and phrase lists: whatever the screen reads for a
// session must equal what the recorder wrote for that session. This is the
// assertion that survives a future rename of either side.
println("")
println("4. invariant — read window == written window")
let read_back: String = state_get(conv_hist_key(TEST_SESSION))
assert_true("screen input is the recorded window, not empty", !str_eq(read_back, ""))
assert_eq("read window is byte-identical to written window", read_back, written)
// 5. No false positive: a calm session does not escalate
//
// A test that only ever asserts "hard_bell" would pass on code that hard-bells
// every message. This is the other leg, and it runs BEFORE the anonymous case
// below on purpose: that case writes the shared bucket, and under the defect a
// calm session would then inherit it.
println("")
println("5. specificity — a calm history does NOT escalate")
let CALM_SESSION: String = "sess-test-129-calm"
state_set("conv_history", "")
conv_history_record(CALM_SESSION, "what is the weather like today", "clear and mild", "")
let calm: String = agentic_safety_screen(CALM_SESSION, ESCALATION_MSG)
assert_eq("calm history stays at soft_bell", json_get(calm, "action"), "soft_bell")
// 6. Cross-session leakage
//
// The same defect had a second face: because the screen read one shared bucket,
// a calm session could be scored against a DIFFERENT session's distress. That is
// wrong in both directions it fabricates a crisis for the calm user and it
// leaks the distressed user's content into another session's scoring.
println("")
println("6. isolation — one session's distress must not score another session")
state_set("conv_history", "")
let OTHER_SESSION: String = "sess-test-129-other"
conv_history_record(OTHER_SESSION, DISTRESS_TURN, "i hear you", "")
let isolated: String = agentic_safety_screen(CALM_SESSION, ESCALATION_MSG)
assert_eq(
"a distressed OTHER session does not escalate the calm session",
json_get(isolated, "action"),
"soft_bell"
)
// 7. Anonymous sessions still work
//
// conv_hist_key("") deliberately falls back to the shared "conv_history" bucket.
// The fix must not break the no-session_id path older callers rely on. Runs last
// because it writes that shared bucket.
println("")
println("7. anonymous path — empty session_id still screens against the shared window")
state_set("conv_history", "[{\"role\":\"user\",\"content\":\"" + DISTRESS_TURN + "\"}]")
let anon: String = agentic_safety_screen("", ESCALATION_MSG)
assert_eq("anonymous session escalates too", json_get(anon, "action"), "hard_bell")
// Summary
println("")
println("history amplification tests: " + int_to_str(counter_of("t_pass")) + " passed, " + int_to_str(counter_of("t_fail")) + " failed")
+97 -11
View File
@@ -41,6 +41,7 @@
#include <fcntl.h>
#include <dirent.h>
#include <errno.h>
#include <signal.h> /* SIGPIPE disposition — see el_runtime_ignore_sigpipe */
#include <pthread.h>
#include <curl/curl.h>
@@ -1238,16 +1239,77 @@ static const char* http_reason_phrase(int status) {
}
}
/* Best-effort send with retry on partial writes. */
/* ── A departing client MUST NOT be able to kill the daemon ──────────────────
* (2026-08-06, round 9.1 / ADR 0006 item 4.)
*
* Measured field failure: a client cancelled its request at 25 s; the handler
* finished its work at 116.9 s and wrote the reply into the departed client's
* socket. The second send() on a reset connection raised SIGPIPE, whose DEFAULT
* disposition terminates the process `exited due to SIGPIPE ... ran for
* 361177ms`. launchd respawned 4 ms later, so EVERY other in-flight request on
* that daemon lost its work, silently.
*
* Two independent guards, because one of them can be undone from outside this
* file (an embedder may reset signal dispositions) and the other cannot:
* 1. process-wide SIGPIPE -> SIG_IGN, installed at runtime init;
* 2. per-send suppression at the syscall (MSG_NOSIGNAL where the platform has
* it, SO_NOSIGPIPE on the accepted socket on macOS/BSD).
* With either in force, send() reports the peer's departure as EPIPE and the
* caller decides which is the point: this is an ordinary I/O outcome, not a
* fatal condition.
*
* It deliberately does NOT swallow the error. http_send_response() below
* classifies the errno and logs: "client left" for a departure, and a real
* "send failed: <strerror>" for anything else, so a genuine write fault is
* still visible in the log (spec round-9.1 §5.3). */
#ifndef MSG_NOSIGNAL
#define MSG_NOSIGNAL 0
#endif
void el_runtime_ignore_sigpipe(void) {
static int done = 0;
if (done) return;
done = 1;
struct sigaction sa;
memset(&sa, 0, sizeof(sa));
sa.sa_handler = SIG_IGN;
sigemptyset(&sa.sa_mask);
sigaction(SIGPIPE, &sa, NULL);
}
/* Suppress SIGPIPE for one accepted connection (macOS/BSD have no
* MSG_NOSIGNAL; they have the socket option instead). Best effort. */
static void http_socket_nosigpipe(int fd) {
#ifdef SO_NOSIGPIPE
int on = 1;
setsockopt(fd, SOL_SOCKET, SO_NOSIGPIPE, &on, sizeof(on));
#else
(void)fd;
#endif
}
/* Best-effort send with retry on partial writes.
* Returns 0 on success, -1 on failure with errno preserved for the caller. */
static int http_send_all(int fd, const char* p, size_t left) {
while (left > 0) {
ssize_t w = send(fd, p, left, 0);
if (w <= 0) return -1;
ssize_t w = send(fd, p, left, MSG_NOSIGNAL);
if (w < 0) {
if (errno == EINTR) continue; /* not an error — retry */
return -1; /* errno stays set for caller */
}
if (w == 0) { errno = EPIPE; return -1; }
p += w; left -= (size_t)w;
}
return 0;
}
/* Did this write fail because the client is gone, or because something is
* actually wrong with the socket? Only the first is routine. */
static int http_write_err_is_client_gone(int e) {
return e == EPIPE || e == ECONNRESET || e == ENOTCONN || e == ESHUTDOWN;
}
/* Discriminator that http_response() embeds at the start of its envelope.
* A handler returning a string starting with this exact prefix is treated
* as a structured response; anything else is treated as a raw body. */
@@ -1468,14 +1530,30 @@ static void http_send_response(int fd, const char* body) {
free(env_body); free(hdrs.buf); return;
}
if (http_send_all(fd, status_line, (size_t)sl) == 0
&& http_send_all(fd, hdrs.buf, hdrs.len) == 0
&& http_send_all(fd, tail, (size_t)tl) == 0
&& (head_only
/* HEAD requests echo headers + Content-Length but no body. */
? 1
: http_send_all(fd, eff_body, blen) == 0)) {
/* sent successfully */
/* The reply is written in four pieces; any of them can find the client
* already gone. errno is captured at the first failure, before any later
* library call can clobber it, and classified once below. */
errno = 0;
int send_err = 0;
if (http_send_all(fd, status_line, (size_t)sl) != 0) send_err = errno;
else if (http_send_all(fd, hdrs.buf, hdrs.len) != 0) send_err = errno;
else if (http_send_all(fd, tail, (size_t)tl) != 0) send_err = errno;
else if (!head_only /* HEAD echoes headers + Content-Length, no body. */
&& http_send_all(fd, eff_body, blen) != 0) send_err = errno;
if (send_err) {
if (http_write_err_is_client_gone(send_err)) {
/* ROUTINE. The user closed the window, quit the app, or cancelled.
* The work is done and the daemon keeps serving everyone else. */
fprintf(stderr, "[http] client left before the reply was written "
"(%zu-byte body, %s) - request completed, reply discarded\n",
blen, strerror(send_err));
} else {
/* NOT routine — a real write fault. Never let the client-gone case
* above hide this one. */
fprintf(stderr, "[http] send failed: %s (%zu-byte body)\n",
strerror(send_err), blen);
}
}
if (env_parsed_root) el_release(env_parsed_root);
@@ -1491,6 +1569,7 @@ static void* http_worker(void* arg) {
HttpWorkerArg* a = (HttpWorkerArg*)arg;
int fd = a->fd;
free(a);
http_socket_nosigpipe(fd);
char *method = NULL, *path = NULL, *body = NULL;
if (http_read_request(fd, &method, &path, &body, NULL) == 0) {
http_handler_fn h = http_lookup_active();
@@ -1531,6 +1610,7 @@ static void* http_worker(void* arg) {
}
void http_serve(el_val_t port, el_val_t handler) {
el_runtime_ignore_sigpipe(); /* serving implies clients that leave */
/* If `handler` looks like a string name, register it as the active handler. */
const char* hname = EL_CSTR(handler);
if (hname && looks_like_string(handler)) {
@@ -1634,6 +1714,7 @@ static void* _http_serve_async_loop(void* raw) {
}
void http_serve_async(el_val_t port, el_val_t handler) {
el_runtime_ignore_sigpipe(); /* serving implies clients that leave */
const char* hname = EL_CSTR(handler);
if (hname && looks_like_string(handler)) {
http_set_handler(handler);
@@ -1821,6 +1902,7 @@ static void* http_worker_v2(void* arg) {
HttpWorkerArg* a = (HttpWorkerArg*)arg;
int fd = a->fd;
free(a);
http_socket_nosigpipe(fd);
char *method = NULL, *path = NULL, *body = NULL, *hdr_block = NULL;
if (http_read_request(fd, &method, &path, &body, &hdr_block) == 0) {
http_handler4_fn h = http_lookup_active_v2();
@@ -1858,6 +1940,7 @@ static void* http_worker_v2(void* arg) {
}
void http_serve_v2(el_val_t port, el_val_t handler) {
el_runtime_ignore_sigpipe(); /* serving implies clients that leave */
const char* hname = EL_CSTR(handler);
if (hname && looks_like_string(handler)) {
http_set_handler_v2(handler);
@@ -5511,6 +5594,9 @@ el_val_t getpid_now(void) {
static el_val_t _el_args_list = 0;
void el_runtime_init_args(int argc, char** argv) {
/* First line of every generated main(): a client that leaves must never be
* able to signal this process to death. See el_runtime_ignore_sigpipe. */
el_runtime_ignore_sigpipe();
_el_args_list = el_list_empty();
for (int i = 1; i < argc; i++) {
_el_args_list = el_list_append(_el_args_list, EL_STR(argv[i]));