Commit Graph

9 Commits

Author SHA1 Message Date
will.anderson c3762ec352 Merge remote-tracking branch 'origin/pr/149' into HEAD
Neuron Soul CI / build (pull_request) Failing after 13m22s
Neuron Soul CI / deploy (pull_request) Has been skipped
# Conflicts:
#	chat.elh
#	dist/soul.c
#	mcp-wrapper/src/main.el
#	routes.el
2026-08-15 18:29:47 -05:00
will.anderson 86e269fa91 self-review 2026-08-07: push what was learned; stop a read route writing the canonical store
Two fixes, one found by making the other.

1. HEBBIAN WRITE-BACK. This daemon learned 1,198 associations in 23h48m and
kept none of them: it syncs FROM the engram server and never pushes, and
mem_save() is unreachable in HTTP mode by design (soul.el only sets
soul_snapshot_path inside `is_genesis && safe_to_seed`, false whenever
ENGRAM_URL is set, because the server owns persistence). So the one process
that runs idle cognition -- where essentially all co-activation happens -- was
the one process that could not remember what it learned.

hebb_consolidate() now drains the runtime's write-back queue on every heartbeat
and POSTs it as ONE batch to /api/edges/batch. One request, one durable write,
not one 60MB snapshot per edge. Also drains on clean shutdown, so an exit
between beats doesn't take the last 8 minutes of learning with it.

_auth is required and its absence is silent: check_auth_ok exempts GET and
/api/neuron/state-events (which is why ise_post works keyless) but gates every
other mutation on "_auth" in the BODY -- http_serve surfaces no headers, so
there is no Bearer path. An unauthorized reply is NON-EMPTY, so the obvious
`if resp == "" return 0` check would have reported delivery of edges that were
refused, after the drain had already destroyed them. Caught before it shipped.
Gauges hebb_wb_pending/_drained/_dropped/_sent go into the heartbeat so a
consolidation path that stops delivering is visible in the stream.

2. A READ ROUTE MUST NEVER WRITE THE CANONICAL SNAPSHOT. GET /api/graph/edges
serialized this process's graph straight over $HOME/.neuron/engram/snapshot.json
-- the engram SERVER's durable store -- and read the edges back out of it. I
triggered it myself this morning fetching edges for the census above:
snapshot.json went from the server's 41,213 edges to the soul's 42,431, and the
next engram restart loaded the soul's graph as canonical. It happened to be a
superset (Knowledge 1198->1218, Memory 1238->1242, no durable type down), so
nothing was lost. That was luck. Had the soul been running a partial load --
the exact failure soul.el's safe_to_seed guard exists to catch -- one GET would
have destroyed the store, with no write-side guard able to see it coming.

The engram server fixed this same class of bug on 2026-07-21 by routing exports
to a dotted sidecar; the soul kept the original pattern. Same fix: exports go to
.soul-edges-export.json. Also stops a 60MB serialize-and-reread per GET.

Verified: boot 26 loaded 42,432 edges with hebb_max 0.4941 carried across the
restart -- the first time this daemon has ever started knowing what it learned.
2026-08-07 08:46:53 -05:00
Tim Lingo ff421d39f6 fix(engine): history keeps its provenance and its session — the false confession and the blank stare
DESIGN FIT: three of round 7's five defects share ONE root — the conversation-history
layer persists only {role, content}, discarding tool provenance, session scoping, and the
distinction between a real user turn and an internal utility call. Fixes A and B RESTORE
Will's design rather than extend it: his agentic path already scopes history per session,
the plain path never got it, and his own source carries the TODO admitting the resulting
race (chat.el, handle_chat: "process-global key; concurrent /api/chat requests without
session_id race on this read-append-write"). Fix C repairs one join Will wrote that was
correct for a year and one we added last week. E1/E2 are ours.

FIX A — tool provenance in history (kills the FALSE CONFESSION)
  Root cause, EXECUTED-verified: handle_chat_agentic recorded turns via hist_append, which
  emits {"role","content"} only. server_tool_use blocks, web_search_tool_result blocks and
  every citation were discarded, then replayed as text. On the next turn the model saw a
  data-rich answer with zero evidence a search had happened, and its own permanent rule
  ("never describe a search you did not perform") left one conclusion available: that it
  had fabricated the data. It apologised for a search it HAD run — four independent lines
  of evidence confirm the search was real. The defect is not the model's honesty. It is
  that we deleted the evidence and then asked it to account for itself.
  Change: agentic_loop accumulates the source URLs it already walks past (citations and
  web_search_tool_result content) and returns them as "sources"; handle_chat_agentic folds
  tools_used + sources into a receipt line stored WITH the assistant turn. Receipts are
  unconditional — a negative receipt ("no tools ran") is the other half of the guarantee,
  because "no evidence of a tool" and "evidence of no tool" were previously identical in
  the transcript. conv_history_block splits the receipt off before snipping so a long
  answer cannot truncate away the evidence. The user never sees it: it is appended to the
  history copy, not the reply.

FIX B — one history key for both paths (kills the BLANK STARE)
  Root cause, EXECUTED-verified: the agentic path keyed history on session_hist_<id>; the
  plain path was hard-wired to the process-global conv_history and never read session_id.
  One conversation, two buckets. Proven in the guest engram: the scoped node held exactly
  two turns starting at "Try again" while the earlier exchanges sat unscoped.
  Change: conv_hist_key/conv_hist_label are now the single definition, used by BOTH paths;
  session_id is threaded route -> layered_cycle -> layered_generate / conv_history_record.
  The 2-line fallback (plain path reads the agentic key) was REJECTED: it keeps the
  process-global bucket as a live write target, which is the bleed the TODO describes.
  Also found and closed while threading: layered_cycle read session_id from the state key
  "current_session_id", which is read here and WRITTEN NOWHERE in the entire source. It
  was unconditionally "", so TODO(reliability #4) — per-session steward continuity — was
  dead code that could never fire. It fires now.
  LAZY SESSION, decided explicitly: we create the session EAGERLY at the door (app half,
  ui#223) rather than migrating orphaned turns. Migration would copy the CONTENTS of a
  process-global bucket, possibly another conversation's, into a named session — the bleed,
  performed deliberately. Eager creation makes the situation impossible instead. Migration
  is deliberately not implemented and must not be added without solving provenance first.

FIX C — the two text-join seams ("to.Good", byte-verified 0x77 0x2e 0x47)
  Two bare `+` joins, written a year apart, had drifted into two answers to one question:
  within-response block joins (Will's, 2026-05-03, latent until server-side web_search
  began interleaving non-text blocks) and across-round joins (ours, 62af564).
  Change: one named rule, text_join_sep, at both sites. NOT a blanket separator — a cited
  answer splits MID-SENTENCE ("The current temperature is " + "86°F" + ", with "), so a
  blanket separator shatters every sourced sentence. The rule takes the one bit that
  distinguishes the cases: whether a NON-TEXT block intervened. Hoisting it also makes the
  fix verifiable in the shipped binary, which an inline `+` is not.

FIX E1 — utility generations stay out of the transcript
  Title generation ("Write a 3-6 word title...") and insight passes ran down the same plain
  door as a real message and were recorded as if the user had typed them; the same calls are
  the "model":"unknown" rows in usage.jsonl. is_utility_request reads an explicit utility
  flag from the app, with the __title__/__insight__ id prefixes as a fallback for older
  clients. Answered normally, never recorded.

FIX E2 — OPERATOR IDENTITY is scoped to tool-capable turns
  The block (env USER/HOME, closing "This is a hard rule") was prepended to EVERY system
  prompt including chat mode. On a Tools:Off turn there is no filesystem in reach, so it
  governed nothing and merely supplied the loudest fact in the prompt — which is why the
  model opened a fresh conversation with "You're test, on your machine at /Users/test".
  Hoisted to operator_identity_block() and gated on !chat_mode. Unchanged wherever a file
  or command tool can actually be reached.

ALSO: agentic_loop's per-session history persist had a second hand-rolled copy of
conv_history_persist with a different label expression, different salience scores and
different tags for the same node. Since both now derive the label from conv_hist_label and
engram_node_full upserts by label, two score policies were writing one node. Collapsed to
one writer.

BUILD NOTE: dist/elp-c-decls.h is force-included by the documented link recipe and carried
the OLD C arities, so it is updated here. This is the build-support header, NOT the stale
generated dist/soul.c — no dist/*.c was read or edited; all engine changes are .el source.
chat.elh/soul.elh are committed because a first-pass build against the old signatures FAILS
(measured); the other regenerated headers are reverted as unrelated churn.

BUILT: 887,000 bytes, sha256 d632b061ad75269d6adeb52578d030eaf49e895d91289d7f946b19c08450d728
Zero el_str_concat(<int>, str_len(...)) sites (the BUG-PLAINCHAT-1 miscompile guard).
web_search_20250305 and disable_parallel_tool_use both still present — PR #108's web search
and the ADR-0005 stopgap are intact.

Refs neuron#109 (builds on it), neuron#78 (Receipt Contract — the real fix A is a stopgap for)

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 22:59:54 -05:00
Tim Lingo 6d8a992716 feat(soul): add safety module, expand connectors API, memory-recall bug notes
- safety.el/.elh: new safety module
- neuron-api.el, routes.el, soul.el, chat.el: connectors API expansion
- regenerated dist/ C artifacts
- MEMORY_RECALL_BUG.md: investigation notes

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 13:18:35 -05:00
will.anderson 00f15b094b feat(soul): add sessions layer, MCP connectors, conversation continuity fix
Deploy Soul to GKE / deploy (push) Failing after 12m39s
Neuron Soul CI / build (push) Failing after 12m49s
- sessions.el: new sessions module with session management and approval gate
- routes.el: wire /api/sessions routes (list, get, create, approve, tool_result)
- chat.el: thread-aware activation — short messages anchor to last reply
  before engram compilation so follow-ups stay on-topic
- chat.el: agentic path tracks per-session history (session_hist_{id})
  instead of shared conv_history, seeding each turn with prior context
- chat.el: add call_neuron_mcp, dispatch_tool, is_builtin_tool, next_bridge_id
  agentic_loop, bridge_save, agentic_resume, handle_tool_result
- dist/soul: rebuild with all of the above
2026-06-15 12:40:47 -05:00
will.anderson bebf1f8c86 fix(soul): address review issues in feat/layer-composition
Neuron Soul CI / build (pull_request) Failing after 6m5s
- Add stub implementations of safety.el, stewardship.el, and imprint.el
  with their .elh headers so the branch compiles without the dependency
  branches (feat/layer-safety, feat/layer-stewardship, feat/layer-imprint).
  Each stub documents the layer contract it must satisfy when replaced.

- Fix GET /api/chat bypass: update the GET branch in handle_request to
  call layered_cycle() consistently with the POST branch, rather than
  calling handle_chat() directly and skipping the consciousness stack.

- Export layered_cycle() from soul.elh (and dist/soul.elh) so routes.el
  can resolve the symbol via the header import.

- Fix steward_action else branch: add explicit handling for "block"
  (returns safe refusal immediately, skips L3) and "redirect" (uses
  redirect_to field). Unknown actions now log a warning and fall back to
  the screened input rather than silently passing an empty string to
  imprint_respond().

- Document hard_bell path: clarify that omitting auto_persist/history
  update is intentional security isolation, and document the safety_validate
  second-param sentinel contract ("hard_bell" vs screen_action).
2026-06-11 11:47:45 -05:00
will.anderson 48ecd83421 fix: restore elb build — import paths, morphology deps, C master declarations header
- Fix wrong ELP import paths in soul.el, elp-input.el, studio.el
  (../foundation/elp/src → ../foundation/el/elp/src)
- Add missing import "morphology.el" to all 29 language morphology modules
- Recompile all affected dist/*.c with correct cross-module declarations
- Add dist/elp-c-decls.h: C-level master forward declarations for ELP package
  (enables elb --force-include to resolve undeclared cross-module calls)
2026-05-08 19:43:57 -05:00
Will Anderson 2665810962 soul: parameterize CGI ID + add dharma_room_turn handler
- soul.el: SOUL_CGI_ID, SOUL_ENGRAM_PATH, SOUL_IDENTITY env vars;
  state_set("soul_snapshot_path") so callers can find it; only call
  init_soul_edges() when cgi_id == "ntn-genesis"
- chat.el: handle_dharma_room_turn — soul builds its own context from its
  own engram, assembles system prompt, calls LLM, persists episodic memory;
  also fix is_new_tool scoping bug in handle_chat_agentic (use has_tool)
- routes.el: wire dharma_room_turn event type before chat_as_soul branch
- rebuild dist/neuron: handle_dharma_room_turn now compiled in
2026-05-03 17:55:37 -05:00
Will Anderson 2622bb04bd ELP: two-layer activation pipeline (activate → suppress → reason → generate)
elp-input.el: replace broken engram_search_json with engram_activate_json
as Layer 1. Layer 2 suppress/filter keeps nodes with non-zero salience/
importance. Reason step extracts patient from top activated node content.
ELP grammar realizes the response via generate().

routes.el: add 'elp' event_type to handle_dharma_recv so the studio can
route ELP requests through dharma.
2026-05-03 11:31:04 -05:00