|
|
|
@@ -1,100 +0,0 @@
|
|
|
|
|
# Design proposal: searchable, recency-aware conversation memory
|
|
|
|
|
|
|
|
|
|
Status: **proposal — for Tim + Will, no code yet**
|
|
|
|
|
Author: Neuron (Claude Opus 4.8), 2026-06-21
|
|
|
|
|
Trigger: "Summarize the key themes across my recent conversations" returns nothing useful.
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
## TL;DR
|
|
|
|
|
|
|
|
|
|
Conversations **are** being persisted — `auto_persist` writes every turn as a
|
|
|
|
|
timestamped `Conversation`/`Episodic` node. The failure is **retrieval**, not
|
|
|
|
|
storage. Two gaps:
|
|
|
|
|
|
|
|
|
|
1. **No recency-ordered retrieval.** There is no way to ask "give me my last N
|
|
|
|
|
conversation turns by time." Search is keyword-ranked only.
|
|
|
|
|
2. **Lexical-only search.** `search_memory` → `engram_search_json` is BM25/lexical.
|
|
|
|
|
A semantic/thematic query ("themes across recent conversations") doesn't share
|
|
|
|
|
keywords with the actual topic content, so it misses.
|
|
|
|
|
|
|
|
|
|
The model literally tried to express the missing capability in the fake tool call
|
|
|
|
|
it hallucinated: `"recency_weight": 0.8`, `"sort_by": "recency"`,
|
|
|
|
|
`node_type: "ConversationTurn"`. It wanted a recency-windowed conversation fetch
|
|
|
|
|
that doesn't exist.
|
|
|
|
|
|
|
|
|
|
## What exists today (verified)
|
|
|
|
|
|
|
|
|
|
- `auto_persist(req, resp)` (chat.el): after each non-agentic turn, stores
|
|
|
|
|
`{"q","a","created_at","source":"chat","label":"chat:<ts>"}` as
|
|
|
|
|
`engram_node_full(... "Conversation" ... "Episodic" ...)`, tags
|
|
|
|
|
`["Conversation","chat","timestamped"]`.
|
|
|
|
|
- `conv_history_persist` (chat.el): a **single overwriting** `conv:history`
|
|
|
|
|
Episodic node holding the rolling JSON history (continuity across restarts) —
|
|
|
|
|
not per-turn, not individually searchable.
|
|
|
|
|
- Live engram (founder instance): **5,113 nodes, 59 conversation nodes** — a mix
|
|
|
|
|
of `chat:<ts>`, several `conv:history` copies, and older `Q:/A:` nodes.
|
|
|
|
|
- Retrieval surface for the agentic loop: `search_memory`, `recall`,
|
|
|
|
|
`neuron_search_knowledge`, `neuron_recall` — all **query-keyword** based.
|
|
|
|
|
None is "most recent N by time," none is embedding/semantic.
|
|
|
|
|
|
|
|
|
|
## The gap, precisely
|
|
|
|
|
|
|
|
|
|
| User intent | Needs | Have today |
|
|
|
|
|
|---|---|---|
|
|
|
|
|
| "summarize my recent conversations" | last-N-by-time fetch | ✗ (keyword only) |
|
|
|
|
|
| "what did we discuss about X" | semantic match on topic | ~ (lexical only; misses paraphrase) |
|
|
|
|
|
| "themes across everything" | semantic cluster over corpus | ✗ |
|
|
|
|
|
|
|
|
|
|
`auto_persist` only fires on the **non-agentic** path (`handle_chat`). Worth
|
|
|
|
|
confirming the **agentic** path (`handle_chat_agentic`) persists turns too — if
|
|
|
|
|
not, agentic conversations never get stored, a second (smaller) gap.
|
|
|
|
|
|
|
|
|
|
## Proposal
|
|
|
|
|
|
|
|
|
|
Three layers, smallest-first. (1) alone fixes the headline use case.
|
|
|
|
|
|
|
|
|
|
### 1. Recency-windowed conversation retrieval (the high-value, low-cost win)
|
|
|
|
|
A runtime/engram primitive + an agentic tool:
|
|
|
|
|
|
|
|
|
|
- **Engram**: `engram_recent_by_type(node_type, limit, since_ts?)` → newest-first
|
|
|
|
|
by `created_at`. (Conversation nodes already carry `created_at`.)
|
|
|
|
|
- **Agentic tool**: `recent_conversations(limit=20, since?)` →
|
|
|
|
|
`[{q,a,created_at}, …]`, newest first. Exposed in `agentic_tools_all`.
|
|
|
|
|
- **System-prompt hint**: for "recent / lately / this week / summarize our
|
|
|
|
|
conversations," prefer `recent_conversations` over `search_memory`.
|
|
|
|
|
|
|
|
|
|
This directly answers "summarize my recent conversations" — fetch last N, hand
|
|
|
|
|
the model the actual turns, let it cluster themes. No embeddings required.
|
|
|
|
|
|
|
|
|
|
### 2. Stable per-session threading
|
|
|
|
|
Today each turn is an independent `chat:<ts>` node; there's no session grouping.
|
|
|
|
|
Add `session_id` + a monotonic turn index to the persisted content (the UI already
|
|
|
|
|
sends `session_id`). Enables "summarize *this* conversation" and per-session recall,
|
|
|
|
|
and lets retrieval return coherent threads instead of loose turns.
|
|
|
|
|
|
|
|
|
|
### 3. Semantic retrieval (the real fix for thematic queries)
|
|
|
|
|
Lexical BM25 can't do "themes." Options, in order of effort:
|
|
|
|
|
- **a.** Embeddings on Conversation nodes + a vector search tool
|
|
|
|
|
(`semantic_search`). Biggest lift; also fixes knowledge recall broadly.
|
|
|
|
|
- **b.** Interim: a two-pass "map-reduce" — `recent_conversations` to pull the
|
|
|
|
|
window, then let the model cluster. Cheap, ships with (1), no infra.
|
|
|
|
|
|
|
|
|
|
Recommend **(1) + (2) now, (3b) as the interim thematic answer, (3a) as the
|
|
|
|
|
roadmap item** once embeddings land (this dovetails with the GraphRAG/embedding
|
|
|
|
|
work already noted in memory: substring 1.7% P@5 vs BM25 55% vs graph 21.7%).
|
|
|
|
|
|
|
|
|
|
## Open questions for Will
|
|
|
|
|
1. ~~Does the agentic path persist turns?~~ **Resolved: yes** — the dispatcher
|
|
|
|
|
calls `auto_persist` after both the agentic and non-agentic branches
|
|
|
|
|
(`routes.el` lines 156/298). Both paths store per-turn nodes.
|
|
|
|
|
2. `conv:history` is accumulating duplicate overwriting nodes (saw several in the
|
|
|
|
|
live engram) — intended, or should it truly overwrite/dedupe?
|
|
|
|
|
3. Is there appetite for the `engram_recent_by_type` primitive in the runtime, or
|
|
|
|
|
should recency be done in `.el` by scanning + sorting (fine at 59 nodes, weak
|
|
|
|
|
at scale)?
|
|
|
|
|
4. Embeddings (3a): on the roadmap timeline, or defer and ship (1)+(2)+(3b)?
|
|
|
|
|
|
|
|
|
|
## Not in scope
|
|
|
|
|
Persistence itself (it works), and the separate **confabulation** fix (model
|
|
|
|
|
faking tool calls in Just-chat mode) — that's `neuron` PR #29.
|
will.anderson