feat(engine): plain chat generates at L3 — inside the safety cycle, not around it
Neuron Soul CI / build (pull_request) Failing after 12m8s
Neuron Soul CI / deploy (pull_request) Has been skipped

Non-agentic /api/chat (the desktop app's default "Tools: Off" mode) returned the
user's own screened text as a bare non-JSON string. Every JSON client failed to
parse it and showed "Couldn't reach Neuron - it may be offline."

Root cause: f52d5bd (2026-06-11) correctly moved the route onto the layer spine
(handle_chat -> layered_cycle), but L3 never got a generator — imprint_respond()
annotates its input and returns it. Two pieces of the architecture were already
waiting for that step: layered_cycle parks a bell directive in the state key
build_system_prompt is written to consume, and build_system_prompt carries a
chat_mode ("no tools") flag with no live caller.

The fix composes rather than replaces. Wiring handle_chat would have removed
safety_screen, the hard-bell short-circuit, the whole stewardship layer and
safety_validate — the only enforcing output gate in the codebase — in exchange
for a working reply (see _engine-websearch-20260804/SAFETY-STOP.md). Instead
layered_cycle keeps every gate, in order, and gains a generation step between
imprint_respond and safety_validate.

  L1 screen -> guard -> hard-bell short-circuit -> L2a -> L2b -> L2c
    -> L3 imprint_respond (prompt) -> L3b layered_generate (NEW) -> L1 validate

- chat.el:  NEW layered_generate (L3 generation, no tools offered),
            conv_history_block, conv_history_record.
            FIX build_system_prompt never concatenated no_tools_rule into its
            return — the "[NO TOOLS THIS TURN]" rule reached no model at all.
            handle_chat annotated DO-NOT-WIRE with the reason.
- soul.el:  layered_cycle gains L3b + post-validation turn bookkeeping.
- routes.el: NEW plain_chat_envelope; all three /api/chat dispatch sites wrap the
            cycle's output. Built OUTSIDE the cycle so safety_validate always sees
            raw model text — nothing to unwrap or rebuild on the crisis path.
            Emits both `reply` and `response`: the desktop app reads `reply`,
            the CLI tools and telegram-gateway read `response`.

Also fixes BUG-PLAINCHAT-1, a pre-existing CRITICAL crash on the crisis path.
elc compiles `let n: Int = pos + str_len(marker)` to el_str_concat() — string
concat on two integers — inside a block-expression initializer, segfaulting the
daemon (SIGSEGV in strlen). Six inline copies of the same " | ts:" parser had it:
two in layered_cycle L2c, two in engram_compile (live on the AGENTIC path too),
two in affective_context_prefix. A distress turn following an earlier affective
turn killed the whole process. Proven pre-existing: an unmodified baseline binary
crashes identically, and the same bad C is in the committed dist/soul.c. Fixed by
hoisting to one top-level function, affective_node_ts(), where the expression
compiles to integer addition — verified in the generated C.

Proof (throwaway HOME/engram, explicit NEURON_PORT, live chain untouched):
- Plain turn returns a JSON envelope with the provider's answer, not an echo.
- Captured request body: no `tools`, no `tool_choice`; system prompt carries the
  NO-TOOLS rule. Tools:Off means no tool is offered, structurally.
- Hard bell: canned 988 message, and the provider request count does not move —
  the message never reaches a model.
- Soft bell + a 2-char model reply: safety_validate's care phrase is appended to
  the MODEL's output. Output gate acting, on this route.
- The L1 bell directive now reaches the model here for the first time (the state
  addendum had a producer and no consumer).
- test_layered_cycle PASS; all six El suites byte-identical to baseline.
- verify-soul-contract.sh (bash 5.3): GATE PASS, 27/27, immutability PASS.
- The crash sequence that killed the baseline daemon now returns HTTP 200.

Not proven: no live Anthropic call — the login keychain refuses the key to a
non-interactive process (rc=24 errSecInteractionNotAllowed). Details and the
one-command close-out are in _engine-plainchat-20260805/README.md §7.

Builds on PR #108. dist/soul.c deliberately not regenerated — Will's toolchain.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Tim Lingo
2026-08-05 09:11:03 -05:00
parent 62af5649fe
commit 635f6febe4
4 changed files with 283 additions and 87 deletions
+186 -54
View File
@@ -416,6 +416,46 @@ fn engram_extract_ids(nodes_json: String) -> String {
// A proper cache/circuit-breaker requires C runtime support (e.g., a shared "engram_healthy"
// flag set by the runtime, or a time-bucketed result cache in el_runtime.c). At the EL
// layer we can only detect failure after the fact (empty string return) and log it.
// affective_node_ts unix timestamp of an affective node (BellEvent / PositiveEvent).
//
// Prefers the " | ts:<epoch>" marker auto_persist writes into the node content; falls back
// to created_at / updated_at. Returns 0 when there is no usable timestamp, which every
// caller already treats as "too old to surface".
//
// ELC CODEGEN NOTE THIS MUST STAY A TOP-LEVEL FUNCTION
// Do not inline this back into a block-expression initializer. Written inline as
// let start: Int = pos + str_len(marker)
// inside a `let x: String = if cond { ... }` initializer, elc loses the declared Int type
// and emits el_str_concat() for the `+`. el_str_concat takes the C string of each operand,
// so two integers become a wild pointer and the daemon SEGFAULTS (EXC_BAD_ACCESS in
// strlen). The identical expression in a plain function body compiles to integer addition
// verified in the generated C both ways. This is the same defect family Will hit on
// 2026-06-23 and solved the same way (aff_try_slot, soul.el).
//
// Six inline copies of this parser existed before this function: two in layered_cycle's
// L2c, two in engram_compile below, two in affective_context_prefix. All six emitted the
// bad concat and all six now call here. Full write-up: BUG-PLAINCHAT-1 in
// _engine-plainchat-20260805/README.md and the PR that introduced this function.
//
fn affective_node_ts(node_json: String) -> Int {
if str_eq(node_json, "") { return 0 }
let content: String = json_get(node_json, "content")
let marker: String = " | ts:"
let mpos: Int = str_index_of(content, marker)
if mpos < 0 {
let ca: String = json_get(node_json, "created_at")
let alt: String = if str_eq(ca, "") { json_get(node_json, "updated_at") } else { ca }
if !engram_numeric_valid(alt) { return 0 }
return str_to_int(alt)
}
let start: Int = mpos + str_len(marker)
let rest: String = str_slice(content, start, str_len(content))
let nxt: Int = str_index_of(rest, " | ")
let raw: String = if nxt < 0 { rest } else { str_slice(rest, 0, nxt) }
if !engram_numeric_valid(raw) { return 0 }
return str_to_int(raw)
}
fn engram_compile(intent: String) -> String {
// Issue 1: decompose multi-topic messages into sub-queries.
let topics: String = engram_split_topics(intent)
@@ -519,20 +559,9 @@ fn engram_compile(intent: String) -> String {
let cutoff_ts: Int = now_ts - 1209600
let recent_bell: String = if bell_ok {
let bn0: String = json_array_get(bell_nodes, 0)
let bn_content: String = json_get(bn0, "content")
let ts_marker: String = " | ts:"
let ts_pos: Int = str_index_of(bn_content, ts_marker)
let bn_ts_raw: String = if ts_pos >= 0 {
let ts_start: Int = ts_pos + str_len(ts_marker)
let rest: String = str_slice(bn_content, ts_start, str_len(bn_content))
let next_sep: Int = str_index_of(rest, " | ")
if next_sep < 0 { rest } else { str_slice(rest, 0, next_sep) }
} else {
let ca: String = json_get(bn0, "created_at")
if str_eq(ca, "") { json_get(bn0, "updated_at") } else { ca }
}
// Q1 fix: validate bell timestamp before str_to_int.
let bn_ts: Int = if !engram_numeric_valid(bn_ts_raw) { 0 } else { str_to_int(bn_ts_raw) }
// Q1 fix (validate before str_to_int) now lives inside affective_node_ts, which
// also replaces the inline " | ts:" parser that miscompiled to el_str_concat here.
let bn_ts: Int = affective_node_ts(bn0)
if bn_ts > cutoff_ts { bn0 } else { "" }
} else { "" }
// Positive emotion context: check for recent joy/success moments within 72h.
@@ -540,19 +569,7 @@ fn engram_compile(intent: String) -> String {
let pos_ec_ok: Bool = !str_eq(pos_ec_nodes, "") && !str_eq(pos_ec_nodes, "[]")
let recent_positive_ec: String = if pos_ec_ok {
let pec0: String = json_array_get(pos_ec_nodes, 0)
let pec_content: String = json_get(pec0, "content")
let pec_ts_marker: String = " | ts:"
let pec_ts_pos: Int = str_index_of(pec_content, pec_ts_marker)
let pec_ts_raw: String = if pec_ts_pos >= 0 {
let pec_ts_start: Int = pec_ts_pos + str_len(pec_ts_marker)
let pec_rest: String = str_slice(pec_content, pec_ts_start, str_len(pec_content))
let pec_next: Int = str_index_of(pec_rest, " | ")
if pec_next < 0 { pec_rest } else { str_slice(pec_rest, 0, pec_next) }
} else {
let pec_ca: String = json_get(pec0, "created_at")
if str_eq(pec_ca, "") { json_get(pec0, "updated_at") } else { pec_ca }
}
let pec_ts: Int = if str_eq(pec_ts_raw, "") { 0 } else { str_to_int(pec_ts_raw) }
let pec_ts: Int = affective_node_ts(pec0)
if pec_ts > cutoff_ts { pec0 } else { "" }
} else { "" }
let affective_part: String = if !str_eq(recent_bell, "") {
@@ -768,7 +785,12 @@ fn build_system_prompt(ctx: String, chat_mode: Bool) -> String {
safety_addendum
}
return identity + operator_section + date_line + voice_rules + security_rules + capability_rules + bounded_persona_block + identity_block + affective_boot_block + engram_block + safety_block
// BUG FIX 2026-08-05: no_tools_rule was computed above and then never concatenated into
// this return, so the "[NO TOOLS THIS TURN]" instruction has not actually reached a model
// in this revision the chat_mode flag had no effect on the prompt. Restored here, in the
// permanent-rules group, immediately after capability_rules (the rule it qualifies).
// Zero effect on agentic paths: they pass chat_mode=false, so no_tools_rule is "".
return identity + operator_section + date_line + voice_rules + security_rules + capability_rules + no_tools_rule + bounded_persona_block + identity_block + affective_boot_block + engram_block + safety_block
}
fn hist_append(hist: String, role: String, content: String) -> String {
@@ -929,6 +951,123 @@ fn conv_history_load() -> String {
return content
}
// conv_history_record append one completed turn to the conversation window.
//
// Same window, same append, same bell-guarded eviction handle_chat uses inline. It exists
// as a function so the layered_cycle path records turns through exactly this code instead
// of growing a second copy that can drift away from the bell guard.
//
// CONTRACT: assistant_msg MUST be post-safety_validate text. Recording the validated text
// rather than the raw model output means the history window can never replay something the
// output gate replaced or augmented. Callers on a hard bell must not call this at all
// bell turns are kept out of conversation history by design (see layered_cycle).
fn conv_history_record(user_msg: String, assistant_msg: String) -> Void {
if str_eq(user_msg, "") { return "" }
let state_hist: String = state_get("conv_history")
let stored_hist: String = if str_eq(state_hist, "") { conv_history_load() } else { state_hist }
let h1: String = hist_append(stored_hist, "user", user_msg)
let h2: String = hist_append(h1, "assistant", assistant_msg)
// Bell-guarded trim: an evicted turn that triggered a bell is preserved to engram
// before it leaves the in-memory window.
let final_hist: String = if json_array_len(h2) > 20 {
hist_trim_with_bell_guard(h2)
} else {
h2
}
state_set("conv_history", final_hist)
conv_history_persist(final_hist)
}
// conv_history_block recent dialogue, rendered for a system prompt.
//
// Same rendering handle_chat uses (role label + snipped content, one line per turn), read
// from the same "conv_history" window, so a plain-chat turn can follow the thread instead
// of answering every message from cold. Read-only: never writes history.
fn conv_history_block() -> String {
let state_hist: String = state_get("conv_history")
let stored_hist: String = if str_eq(state_hist, "") { conv_history_load() } else { state_hist }
let hist_len: Int = if str_eq(stored_hist, "") { 0 } else { json_array_len(stored_hist) }
if hist_len == 0 {
return ""
}
let rh_out: String = ""
let rh_i: Int = 0
while rh_i < hist_len {
let rh_entry: String = json_array_get(stored_hist, rh_i)
let rh_role: String = json_get(rh_entry, "role")
let rh_content: String = json_get(rh_entry, "content")
let rh_label: String = if str_eq(rh_role, "user") { "User" } else { "Assistant" }
let rh_snip: String = if str_len(rh_content) > 400 { str_slice(rh_content, 0, 400) + "..." } else { rh_content }
let rh_line: String = rh_label + ": " + rh_snip
let rh_out = if str_eq(rh_out, "") { rh_line } else { rh_out + "\n" + rh_line }
let rh_i = rh_i + 1
}
return "\n\n[RECENT CONVERSATION — last " + int_to_str(hist_len) + " turns]\n" + rh_out
}
// layered_generate the L3 generation step of layered_cycle. This is where the imprint
// SPEAKS. (Added 2026-08-05.)
//
// layered_cycle calls this immediately after imprint_respond(), which has already applied
// the active imprint's voice/domain annotation to the steward-aligned input. Before this
// existed, L3 ended at that annotation and layered_cycle handed the user's own text back
// as the "reply" every gate ran, but nothing ever generated.
//
// It lives in chat.el, not imprint.el, on purpose: imprint.el is L3 and declares that the
// lower layers are structurally inaccessible from it. It has zero imports, and making it
// reach chat.el would transitively pull in safety.el (L1), inverting the layering and
// breaking the tests that link imprint.el on its own. The layer ORDER is enforced by
// layered_cycle, which is the only caller; this function holds no layer authority.
//
// SAFETY CONTRACT this function is NOT a gate and must never become one:
// - Everything upstream has already run inside layered_cycle: L1 safety_screen, the
// safe-mode guard, the hard-bell short-circuit, L2a continuity/profiling, L2b mission
// alignment, L2c affective context. A hard bell can never reach this function
// layered_cycle returns the fixed crisis message before L3 is entered.
// - Everything downstream is safety_validate(), which layered_cycle applies to this
// function's return value. Nothing here may bypass it, so this always returns plain
// text: no JSON envelope, no escaping, nothing for the output gate to have to unwrap.
// - The bell directive layered_cycle computed with safety_augment_system() is parked in
// the state key "layered_cycle_safety_system_addendum", and build_system_prompt() is
// its designated consumer. That is why the system prompt is assembled through
// build_system_prompt() here rather than hand-rolled: routing around it would drop the
// soft-bell / crisis directive on the floor. safety_augment_system() is NOT called
// again here one evaluation per turn, one InternalStateEvent per bell.
// - No affective_context_prefix() call either: L2c already folded the affective note
// into that same addendum, and injecting it twice would double the cue.
//
// TOOLS: none, structurally. build_system_prompt(ctx, true) is chat mode, which injects
// the permanent "NO TOOLS THIS TURN" rule, and llm_call_system() is a plain /v1/messages
// call whose request body is built in el_runtime.c (llm_provider_request) with no "tools"
// and no "tool_choice" key at all. Tools:Off means no tool is offered to the model, not
// merely that none is used.
//
// Returns "" when the model call fails, so the caller reports the failure honestly instead
// of echoing the user's own text back at them.
fn layered_generate(prompt: String, imprint_id: String) -> String {
if str_eq(prompt, "") {
return ""
}
let ctx: String = engram_compile(prompt)
let model: String = chat_default_model()
let base_system: String = build_system_prompt(ctx, true) + current_engine_note(model)
let hist_block: String = conv_history_block()
let full_system: String = base_system + hist_block
let raw: String = llm_call_system(model, full_system, prompt)
let is_error: Bool = str_starts_with(raw, "{\"error\"")
|| str_starts_with(raw, "{\"type\":\"error\"")
|| str_contains(raw, "authentication_error")
if is_error {
println("[chat] layered_generate: model call failed — returning empty so the caller can report it honestly")
return ""
}
return clean_llm_response(raw)
}
// session_preload_bullets render up to max_bullets nodes from a JSON array as
// bullet lines, truncating content at snip_len chars each.
fn session_preload_bullets(nodes: String, max_bullets: Int, snip_len: Int) -> String {
@@ -969,19 +1108,7 @@ fn affective_context_prefix() -> String {
} else {
if has_dist_aff {
let dn0: String = json_array_get(dist_nodes_aff, 0)
let dn_content: String = json_get(dn0, "content")
let daff_marker: String = " | ts:"
let daff_pos: Int = str_index_of(dn_content, daff_marker)
let daff_ts_str: String = if daff_pos >= 0 {
let daff_start: Int = daff_pos + str_len(daff_marker)
let daff_rest: String = str_slice(dn_content, daff_start, str_len(dn_content))
let daff_next: Int = str_index_of(daff_rest, " | ")
if daff_next < 0 { daff_rest } else { str_slice(daff_rest, 0, daff_next) }
} else {
let daff_ca: String = json_get(dn0, "created_at")
if str_eq(daff_ca, "") { json_get(dn0, "updated_at") } else { daff_ca }
}
let daff_ts: Int = if str_eq(daff_ts_str, "") { 0 } else { str_to_int(daff_ts_str) }
let daff_ts: Int = affective_node_ts(dn0)
daff_ts > aff_cutoff
} else { false }
}
@@ -989,19 +1116,7 @@ fn affective_context_prefix() -> String {
let has_pos_aff: Bool = !str_eq(pos_nodes_aff, "") && !str_eq(pos_nodes_aff, "[]")
let found_recent_pos: Bool = if has_pos_aff && !found_recent_dist {
let pn0: String = json_array_get(pos_nodes_aff, 0)
let pn_content: String = json_get(pn0, "content")
let paff_marker: String = " | ts:"
let paff_pos: Int = str_index_of(pn_content, paff_marker)
let paff_ts_str: String = if paff_pos >= 0 {
let paff_start: Int = paff_pos + str_len(paff_marker)
let paff_rest: String = str_slice(pn_content, paff_start, str_len(pn_content))
let paff_next: Int = str_index_of(paff_rest, " | ")
if paff_next < 0 { paff_rest } else { str_slice(paff_rest, 0, paff_next) }
} else {
let paff_ca: String = json_get(pn0, "created_at")
if str_eq(paff_ca, "") { json_get(pn0, "updated_at") } else { paff_ca }
}
let paff_ts: Int = if str_eq(paff_ts_str, "") { 0 } else { str_to_int(paff_ts_str) }
let paff_ts: Int = affective_node_ts(pn0)
paff_ts > aff_cutoff
} else { false }
let affective_out: String = if found_recent_dist {
@@ -1014,6 +1129,23 @@ fn affective_context_prefix() -> String {
return affective_out
}
//
// handle_chat UNWIRED. DO NOT ROUTE /api/chat HERE. (annotated 2026-08-05)
//
// This was the non-agentic chat handler until 2026-06-11 (f52d5bd, "wire consciousness
// layers"), when Will moved /api/chat onto layered_cycle. It has had zero call sites since.
//
// It must stay unwired: it has NO enforcing input gate and NO enforcing output gate.
// It never calls safety_screen, so a hard bell would reach the model instead of being
// refused; it never calls safety_validate, the only enforcing output gate in the codebase;
// and it runs no stewardship layer. Its one safety touch, safety_augment_system at the
// llm_call_system site below, is an advisory system-prompt string it cannot refuse,
// replace, or block anything.
//
// Plain chat generates through layered_cycle's L3 (layered_generate) instead, which keeps
// the screen, the stewardship layers and the output gate wrapped around the model call.
// If this function is ever revived, it must be gated first, not wired first.
//
fn handle_chat(body: String) -> String {
let message: String = json_get(body, "message")
if str_eq(message, "") {