neuron-technologies/neuron
3
0
Fork 0
Code Issues 44 Pull Requests 7 Actions Packages Projects Releases Wiki Activity

Bound beginSession/compileCtx payloads to a compact digest #102

Closed
will.anderson wants to merge 9 commits from fix/bound-session-payload into hotfix/elc-source-typos
pull from: fix/bound-session-payload
merge into: :hotfix/elc-source-typos
Conversation 1 Commits 9 Files Changed 11
+923 -241
will.anderson commented 2026-07-31 19:59:38 +00:00
Owner
Copy Link
Copy Source

Session-init endpoints returned unbounded engram nodes (~900KB), closing the MCP client socket on every beginSession. Cap the lists and project nodes to identity + UTF-8-safe content snippets. Response drops from ~900KB to ~12KB; verified end-to-end through the proxy (HTTP 200, valid JSON-RPC). Graph intact (node_count ~12762).

Session-init endpoints returned unbounded engram nodes (~900KB), closing the MCP client socket on every beginSession. Cap the lists and project nodes to identity + UTF-8-safe content snippets. Response drops from ~900KB to ~12KB; verified end-to-end through the proxy (HTTP 200, valid JSON-RPC). Graph intact (node_count ~12762).
will.anderson added 9 commits 2026-07-31 19:59:38 +00:00
self-review 2026-07-23: title-derived Knowledge labels + Knowledge admitted to curiosity auto-term 9e59c51f3c 
Sentinel labels (knowledge:captured/evolved/canonical) made every capture
anonymous in WM telemetry — 35 identical wm_top entries — and starved the
curiosity auto-term seeder, which derives scan seeds from WM top-10 labels
and had returned empty on every scan since boot 6 because WM became
Knowledge-dominated while Knowledge was excluded from seeding.

- capture/evolve/promote now pass title (or empty → engram_node_full's
  content[:60] derivation) instead of sentinels
- auto_term_try_slot admits Knowledge slots; sentinel-shaped labels
  (colon, no space) are skipped so legacy nodes cannot seed 'knowledge'
- verified: probe capture labeled 'Label derivation probe 2026-07-23'
self-review 2026-07-24: boot counter — demote to telemetry weight, restore persistence via HTTP write-back fb0bb553f3 
Three stale soul:boot_count copies (salience .9, importance .9, Canonical:
+0.2 tier bias, 0.15 threshold) held the top WM slots for 23h — a boot
counter outcompeting real context. Demoted to salience .55 / importance .2 /
tier Working: plumbing, not memory.

Persistence was also broken: in HTTP-engram mode the server owns state and
nothing wrote the counter back — the log shows boot #5 on three consecutive
boots. mem_boot_count_inc now mirrors the persona write-back: delete stale
server copies (matched by content prefix — route_create_node sets
label=content), create the replacement server-side. Working tier is in the
boot seed (/api/nodes) but excluded from periodic /api/sync, so the count
survives restarts without re-importing mid-session. Verified: restart
incremented 1->2 with exactly one server-side counter node.
self-review 2026-07-25: break curiosity positive-feedback loop; observability for WM regime 58a9eda311 
proactive_curiosity strengthened its top result unconditionally every
scan — a positive-feedback fixed point that pinned auto_term on the same
node's first word for hours ('Fast-slow' era). Strengthen now fires only
when the top node changed since the last scan, and a 4-deep finst-style
tabu ring (ACT-R declarative finsts) hard-excludes recently used auto
terms (~2 min at the 30s cadence). Quoted-title guard stops '"The'
leaking through the >3-char stopword check and seeding lexical floods.

Heartbeat now pumps /api/embed-backfill?n=32 on the authoritative store
(its lazy backfill had no production trigger; coverage stalled at
93/12175) and emits wm_saturated, wm_top0_streak, embed_backfilled,
embed_count. Curiosity ISE emits auto_term_streak. The stuck-WM failure
mode is now a one-glance signal instead of manual ISE cross-referencing.
self-review 2026-07-26: heartbeat wm_churn + wm_top0_wm; fix streak counting on empty id 8392f44c45 
- wm_churn: count of top-5 WM ids absent from previous beat — separates
  'one stuck node' from 'whole WM frozen' without hand-correlating ISEs.
- wm_top0_wm: leader's weight; a frozen anchor reads as a constant here.
- Streak guard: before the runtime emitted id in wm_top JSON,
  json_get(...,"id") was always empty and the streak incremented on
  ""=="" every beat — wm_top0_streak measured uptime, not fixation.
  Empty id now resets the streak to 0.
self-review 2026-07-27: heartbeat carries activation observability 2b612ed5d4 
Fold engram_act_stats_json() into the heartbeat ISE: wm_evicted and
breakthroughs (per curiosity-scan activate call) plus embed_breaker_open —
the failure mode embed_ok structurally cannot see (it pings the Ollama
root, not the embed pipeline). WM-cap eviction, breakthrough-floor
flooding, and silent lexical degradation are now one-glance diagnosable
from telemetry.
self-review 2026-07-28: close ISE lifecycle observability gaps 627eb534a2 
- session_start now also posted to the HTTP Engram via ise_post: the
  local engram_node_full write never crossed to the observable stream
  (sync flows HTTP->soul only), so boots 5+ were invisible — last
  visible session_start was boot 4, two weeks ago
- graceful shutdown emits a final ISE with boot/pulse/uptime; a boot
  with no shutdown event now reliably signals a crash/SIGKILL
- empty /api/sync responses emit a sync_empty warn ISE instead of
  being silently skipped — unreachable engram no longer looks
  identical to quiet-but-healthy
- sync backflow prune reads ENGRAM_ISE_RETENTION_MS instead of
  duplicating the 48h magic number server.el already honors
self-review 2026-07-30: auto_term stopword filter, real idle_ms, bounded beginSession b0f4d6c493 
- awareness.el: curiosity auto_term was the raw first word of a WM
  label with no term-quality scoring — observed seeds included What,
  Colon, Prose, Context. Replaced the 7-word genre blocklist whack-a-
  mole with a delimited stopword membership test (function words +
  document-structure words); topical terms pass untouched. Verified
  live: seeds now ReasonEdit, Reasoning-model, Self-review.
- routes.el + awareness.el: idle counter only reset on rare inbox
  synthesis-requests, so idle==pulse always (zero information).
  handle_request now stamps soul.last_activity_ts on every inbound
  HTTP request; heartbeat emits idle_ms = ms since last request
  (-1 until first request of a boot).
- neuron-api.el: beginSession concatenated a depth-2 spread plus the
  unbounded self-hub neighbor dump — multi-MB response, doubled by
  wrapper re-escaping, socket died on every call. Now depth-1 and
  the hub dump dropped (identity loading has its own tool). Verified:
  beginSession returns instead of closing the socket.
self-review 2026-07-31: heartbeat deltas for cumulative counters, embed_eligible; fix stale semantic-seeding comment f3660e92a1 
Runtime activation counters are now cumulative, so the heartbeat emits
wm_evicted/breakthroughs as totals plus wm_evicted_delta/
breakthroughs_delta (state-tracked change since the previous beat) —
events between beats are no longer lost. Adds embed_eligible from
/api/stats so coverage reads as embed_count/embed_eligible instead of
the misleading absolute count, and surfaces auto_term_streak in the
heartbeat stream. Replaces the false 'semantic seeding NOT implemented'
comment: engram_activate embeds the query, seeds semantic top-K, gates
propagation on cosine, and scores WM promotion semantically.
Bound beginSession/compileCtx payloads to a compact digest 872120c757 
The session-init endpoints concatenated unbounded engram activate/scan
results as FULL node objects (content up to ~90KB per node), producing a
~900KB response. After the MCP wrapper re-escapes that into a stringified
text block the client dropped the socket ('connection closed unexpectedly')
on every beginSession call. Cap each list (8-10 activated, 10-20 recent)
and project every node to a light identity plus a bounded, UTF-8-safe
content snippet. Response drops from ~900KB to ~12KB; full content stays
available on demand via recall/fetch/inspectGraph.
will.anderson commented 2026-08-01 16:42:32 +00:00
Author
Owner
Copy Link
Copy Source

Superseded by #103, which landed this payload-bounding fix directly on main (merge commit b75d5c8). #102 targeted hotfix/elc-source-typos and carried hotfix-compiled generated C; #103 ports the neuron-api.el source onto current main and regenerates dist/soul.c (the CI-built amalgamation) + dist/neuron-api.c from main's tree. Verified: clean CI build + verify-soul-contract.sh PRESENCE/IMMUTABILITY PASS. Closing.

Superseded by #103, which landed this payload-bounding fix directly on `main` (merge commit b75d5c8). #102 targeted `hotfix/elc-source-typos` and carried hotfix-compiled generated C; #103 ports the neuron-api.el source onto current main and regenerates dist/soul.c (the CI-built amalgamation) + dist/neuron-api.c from main's tree. Verified: clean CI build + verify-soul-contract.sh PRESENCE/IMMUTABILITY PASS. Closing.
will.anderson closed this pull request 2026-08-01 16:42:33 +00:00

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
BETA-CRITICAL
On the beta critical path — must resolve before ship
 blocks-public-beta
Must be fixed before any public beta
 HELD
Blocked pending an external decision (counsel / dual sign-off)
 ORTHOGONAL
Parallel workstream (testing/hardening) — safe to assign off critical path
 P0
Drop-everything severity
 POST-BETA
Important but not blocking the beta ship
 security
Security vulnerability
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
andre.almeida (Andre Almeida) tim.lingo (Tim Lingo) will.anderson (Will Anderson)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: neuron-technologies/neuron#102
Delete Branch "fix/bound-session-payload"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?