Compare commits

..

6 Commits

Author SHA1 Message Date
will.anderson 9ba86b8f80 Merge pull request 'feat(memory): emotional salience tagging and cross-session distress persistence' (#34) from improve/soul-memory-formation into main
Neuron Soul CI / build (push) Has been cancelled
Deploy Soul to GKE / deploy (push) Has been cancelled
feat(memory): emotional salience tagging, BellEvent ts fix, label uniqueness
2026-06-22 17:14:25 +00:00
will.anderson 360c15d7fe Merge pull request 'fix(routes): error handling, health endpoint, request validation, rate limiting' (#32) from improve/soul-routes-api into main
Neuron Soul CI / build (push) Has been cancelled
Deploy Soul to GKE / deploy (push) Has been cancelled
fix(routes): error handling, health endpoint, validation, rate limiting
2026-06-22 17:14:10 +00:00
will.anderson 0c5b966773 fix(chat): fix auto_persist timestamp extraction and bell label uniqueness
Neuron Soul CI / build (pull_request) Has been cancelled
- engram_compile: BellEvent nodes do not carry created_at in the engram
  node JSON; extract the unix timestamp from the embedded ' | ts:NNNNN'
  pattern in the content string instead. Fall back to created_at/updated_at
  if the marker is absent. Guard str_to_int against empty string so the 72h
  recency check never silently treats every node as epoch-0 stale.

- auto_persist: append the current unix timestamp to the BellEvent label
  ('bell:soft:1749876543') to make it unique per turn. The previous label
  ('bell:soft') was the same for every soft bell, causing engram to treat
  all subsequent writes as updates to the same node.
2026-06-22 12:09:00 -05:00
will.anderson b7fd8901d4 fix(routes): fix handle_request ABI, 429 status code, soul_boot_ts write
Neuron Soul CI / build (pull_request) Has been cancelled
2026-06-22 11:53:09 -05:00
will.anderson b2008f4894 feat(memory): emotional salience tagging and cross-session distress persistence
Neuron Soul CI / build (pull_request) Successful in 5m36s
- auto_persist: detect bell level (soft/hard) on every user message using
  safety_detect_bell_level; write a dedicated BellEvent engram node with
  calibrated salience alongside the Conversation node when a bell fires.
  Tag the Conversation node with bell:soft/bell:hard and 'affective' for
  direct discovery without scanning all chat nodes.

- auto_persist: track per-session bell count, dominant level, and last
  signal in state (session_bell_count/level/signal keys) so downstream
  functions can act on the emotional history without re-scanning engram.

- engram_compile: include the top-1 most recent BellEvent node within 72h
  in every context build. Distress context from earlier turns (same or
  recent session) automatically travels into all subsequent LLM calls.

- hist_trim_with_bell_guard: replace hist_trim at the handle_chat call site.
  Before evicting the oldest turn from the 20-turn window, inspect the user
  message for bell signals. If a bell was present, write a preservation
  BellEvent to engram before dropping the turn so the full message survives
  the rolling window.

- session_hist_save: after writing the history node, check session bell
  counters. On the first save where bell_count > 0, write a
  session:emotional-summary BellEvent node with distress signal, count,
  and dominant level. A state flag prevents duplicate writes on subsequent
  saves in the same session.
2026-06-22 11:23:15 -05:00
will.anderson d92b8c279a fix(routes): error handling, health diagnostics, request validation, rate limiting
Neuron Soul CI / build (pull_request) Has been cancelled
- Add per-IP in-memory rate limiter (60 req/min default, configurable via
  soul_rate_limit state key; /health exempt; loopback callers skipped)
- Extend /health with uptime_secs (from soul_boot_ts) and live LLM probe
- Add missing_param 400 guard on POST /api/chat before passing to LLM
- Standardise error envelopes: add "code" field to err_404/err_405 and all
  missing-param returns; route_synthesize now errors clearly instead of
  returning the misleading {"mechanism":"did not engage"} on bad input
- Document streaming gap in /api/chat (SSE not implemented, note added)
- handle_request gains ip param; rate_limit_check wired at entry point
2026-06-22 11:21:18 -05:00
5 changed files with 344 additions and 120 deletions
+186 -63
View File
@@ -150,9 +150,43 @@ fn engram_compile(intent: String) -> String {
"" ""
} }
// Affective context: always include the most recent high-emotion memory if one
// exists within 72 hours. This ensures continuity of care across turns when
// the user was in distress earlier in the session (or recently), that context
// travels into every subsequent LLM call so the response register stays aware.
// We search for BellEvent nodes specifically; these are written by auto_persist
// when safety_detect_bell_level fires. The 72h window (259200 seconds) is wide
// enough to span a multi-session day without pulling ancient history.
let bell_nodes: String = engram_search_json("bell:soft bell:hard BellEvent", 3)
let bell_ok: Bool = !str_eq(bell_nodes, "") && !str_eq(bell_nodes, "[]")
let now_ts: Int = time_now()
let cutoff_ts: Int = now_ts - 259200
let recent_bell: String = if bell_ok {
let bn0: String = json_array_get(bell_nodes, 0)
// created_at is not present in engram node JSON for BellEvent nodes.
// Extract the timestamp embedded in the content string as " | ts:NNNNN".
// Fall back to created_at / updated_at JSON fields if the marker is absent.
let bn_content: String = json_get(bn0, "content")
let ts_marker: String = " | ts:"
let ts_pos: Int = str_index_of(bn_content, ts_marker)
let bn_ts_raw: String = if ts_pos >= 0 {
let ts_start: Int = ts_pos + str_len(ts_marker)
let rest: String = str_slice(bn_content, ts_start, str_len(bn_content))
let next_sep: Int = str_index_of(rest, " | ")
if next_sep < 0 { rest } else { str_slice(rest, 0, next_sep) }
} else {
let ca: String = json_get(bn0, "created_at")
if str_eq(ca, "") { json_get(bn0, "updated_at") } else { ca }
}
let bn_ts: Int = if str_eq(bn_ts_raw, "") { 0 } else { str_to_int(bn_ts_raw) }
if bn_ts > cutoff_ts { bn0 } else { "" }
} else { "" }
let affective_part: String = if !str_eq(recent_bell, "") { recent_bell } else { "" }
let sep1: String = if !str_eq(act_part, "") && !str_eq(srch_part, "") { "\n" } else { "" } let sep1: String = if !str_eq(act_part, "") && !str_eq(srch_part, "") { "\n" } else { "" }
let sep2: String = if (!str_eq(act_part, "") || !str_eq(srch_part, "")) && !str_eq(scan_part, "") { "\n" } else { "" } let sep2: String = if (!str_eq(act_part, "") || !str_eq(srch_part, "")) && !str_eq(scan_part, "") { "\n" } else { "" }
let ctx: String = act_part + sep1 + srch_part + sep2 + scan_part let sep3: String = if (!str_eq(act_part, "") || !str_eq(srch_part, "") || !str_eq(scan_part, "")) && !str_eq(affective_part, "") { "\n" } else { "" }
let ctx: String = act_part + sep1 + srch_part + sep2 + scan_part + sep3 + affective_part
if str_eq(ctx, "") { return "" } if str_eq(ctx, "") { return "" }
@@ -226,6 +260,69 @@ fn hist_trim(hist: String) -> String {
return hist return hist
} }
// hist_trim_with_bell_guard trim the history window exactly as hist_trim does, but
// before dropping the oldest user/assistant pair check whether the user turn triggered
// a bell event. If it did, write a preservation node to engram so the distress exchange
// survives the 20-turn window. The LLM window drops it; engram retains it permanently
// and engram_compile will surface it again via the affective context path.
fn hist_trim_with_bell_guard(hist: String) -> String {
// Extract the first turn (should be a user message) to inspect it.
let inner: String = str_slice(hist, 1, str_len(hist) - 1)
let marker: String = "{\"role\":"
let i1: Int = str_index_of(inner, marker)
// i1 is the start of the first entry within inner.
// Find where the second entry begins to delimit the first entry's JSON.
let tail1: String = str_slice(inner, i1 + 1, str_len(inner))
let i2: Int = str_index_of(tail1, marker)
// The first entry spans from i1 to (i1 + 1 + i2 - 1) within inner.
let first_entry_raw: String = if i2 > 0 {
str_slice(inner, i1, i1 + 1 + i2 - 1)
} else {
str_slice(inner, i1, str_len(inner))
}
let first_role: String = json_get(first_entry_raw, "role")
let first_content: String = json_get(first_entry_raw, "content")
// Only inspect user turns assistant content doesn't carry bell signals.
let bell_level: String = if str_eq(first_role, "user") {
safety_detect_bell_level(first_content)
} else {
"none"
}
// If the turn being evicted triggered a bell, preserve it to engram.
// This is distinct from the BellEvent written by auto_persist: that node
// carries a short summary. This node carries the full exchange content so
// it is recoverable for clinical/continuity review.
if !str_eq(bell_level, "none") {
let ts: Int = time_now()
let ts_str: String = int_to_str(ts)
let safe_content: String = str_replace(first_content, "\"", "'")
let preserve_content: String = "PRESERVED_BELL:" + bell_level
+ " | evicted_at:" + ts_str
+ " | message:" + safe_content
let preserve_tags: String = "[\"bell-history\",\"bell:" + bell_level + "\",\"evicted\",\"affective\",\"BellEvent\"]"
let discard: String = engram_node_full(
preserve_content,
"BellEvent",
"bell:" + bell_level + ":preserved",
el_from_float(0.9),
el_from_float(0.9),
el_from_float(1.0),
"Episodic",
preserve_tags
)
}
// Now perform the standard trim (drop oldest 2 entries = 1 user + 1 assistant pair).
let tail2: String = str_slice(tail1, i2 + 1, str_len(tail1))
let i3: Int = str_index_of(tail2, marker)
if i3 >= 0 {
return "[" + str_slice(tail2, i3, str_len(tail2)) + "]"
}
return hist
}
// clean_llm_response strips GPT-2 BPE byte-to-unicode artifacts that vLLM // clean_llm_response strips GPT-2 BPE byte-to-unicode artifacts that vLLM
// emits when the tokenizer hasn't decoded back to raw bytes. // emits when the tokenizer hasn't decoded back to raw bytes.
// //
@@ -374,19 +471,11 @@ fn handle_chat(body: String) -> String {
let req_model: String = json_get(body, "model") let req_model: String = json_get(body, "model")
let model: String = if str_eq(req_model, "") { chat_default_model() } else { req_model } let model: String = if str_eq(req_model, "") { chat_default_model() } else { req_model }
// ISSUE 9: add safety_augment_system to primary /api/chat path.
// handle_chat was the only LLM path missing bell directive injection.
let full_system = safety_augment_system(full_system, message)
let raw_response: String = llm_call_system(model, full_system, message) let raw_response: String = llm_call_system(model, full_system, message)
// Issue #5: also catch empty string llm_extract_text() in el_runtime.c silently
// returns "" when the response content array is missing or all blocks fail to parse.
// Without this guard an empty reply passes through as a silent empty response.
let is_error: Bool = str_starts_with(raw_response, "{\"error\"") let is_error: Bool = str_starts_with(raw_response, "{\"error\"")
|| str_starts_with(raw_response, "{\"type\":\"error\"") || str_starts_with(raw_response, "{\"type\":\"error\"")
|| str_contains(raw_response, "authentication_error") || str_contains(raw_response, "authentication_error")
|| str_eq(raw_response, "")
if is_error { if is_error {
return "{\"error\":\"llm unavailable\",\"response\":\"\"}" return "{\"error\":\"llm unavailable\",\"response\":\"\"}"
} }
@@ -396,8 +485,10 @@ fn handle_chat(body: String) -> String {
let updated_hist: String = hist_append(stored_hist, "user", message) let updated_hist: String = hist_append(stored_hist, "user", message)
let updated_hist2: String = hist_append(updated_hist, "assistant", raw_response) let updated_hist2: String = hist_append(updated_hist, "assistant", raw_response)
// Use bell-guarded trim: if the evicted turn triggered a bell event, it is
// preserved to engram before being dropped from the in-memory window.
let final_hist: String = if json_array_len(updated_hist2) > 20 { let final_hist: String = if json_array_len(updated_hist2) > 20 {
hist_trim(updated_hist2) hist_trim_with_bell_guard(updated_hist2)
} else { } else {
updated_hist2 updated_hist2
} }
@@ -451,42 +542,6 @@ fn studio_tools_json() -> String {
"]" "]"
} }
// ---------------------------------------------------------------------------
// LLM reliability issues that require C runtime fixes (el_runtime.c).
// These cannot be addressed at the EL layer; they are documented here so the
// symptoms are traceable back to their root causes.
//
// Issue #1 (no retry on timeout/connection error):
// http_do() in el_runtime.c calls curl_easy_perform() once. On
// CURLE_OPERATION_TIMEDOUT / CURLE_COULDNT_CONNECT / CURLE_RECV_ERROR it
// returns http_error_json() with no retry. Fix: add a retry loop (max 3
// attempts, exponential back-off starting at 1s) inside llm_provider_request().
//
// Issue #2 (60s timeout applies to all HTTP calls including LLM):
// EL_HTTP_TIMEOUT_MS defaults to 60000ms for every http_do() call.
// Fix: introduce EL_LLM_TIMEOUT_MS (default 120000) used only by
// llm_provider_request(); leave EL_HTTP_TIMEOUT_MS (default 30000) for
// general service calls to avoid holding connections for 60s.
//
// Issue #3 (HTTP 429 causes silent provider failover, not backoff):
// llm_chain_call() advances to the next provider on any JSON-prefixed response
// including 429. Fix: parse HTTP status via curl_easy_getinfo; on 429 sleep
// Retry-After seconds (default 5s) then retry the same provider up to 3 times.
//
// Issue #4 (HTTP 500/502 crashes the request silently):
// Same path as #3 5xx responses cause immediate provider failover with no
// retry. Fix: retry with exponential back-off (1s, 2s, 4s) before advancing.
//
// Issue #6 (no secondary LLM fallback in production):
// Set NEURON_LLM_1_URL/KEY/FORMAT in ExternalSecret to a secondary provider
// (e.g. Gemini). No C code change required; llm_chain_call() already iterates.
//
// Issue #8 (LLM response size unbounded memory-only cap):
// HttpBuf grows via realloc() with no hard limit. Fix: add
// EL_HTTP_MAX_RESPONSE_BYTES (default 10MiB) cap in httpbuf_append() and
// return http_error_json("response too large") on overflow.
// ---------------------------------------------------------------------------
fn agentic_api_key() -> String { fn agentic_api_key() -> String {
let k1: String = env("ANTHROPIC_API_KEY") let k1: String = env("ANTHROPIC_API_KEY")
if !str_eq(k1, "") { if !str_eq(k1, "") {
@@ -538,7 +593,7 @@ fn agentic_tools_with_web() -> String {
// Short timeout + empty-array fallback: if the bridge is down, the soul runs // Short timeout + empty-array fallback: if the bridge is down, the soul runs
// exactly as before with only its built-in tools (graceful degradation). // exactly as before with only its built-in tools (graceful degradation).
fn connector_tools_json() -> String { fn connector_tools_json() -> String {
let raw: String = exec_capture("curl -s --max-time 5 http://127.0.0.1:7771/mcp/tools") let raw: String = exec_capture("curl -s --max-time 2 http://127.0.0.1:7771/mcp/tools")
if str_eq(raw, "") { if str_eq(raw, "") {
return "[]" return "[]"
} }
@@ -583,7 +638,7 @@ fn tool_auto_approved(tool_name: String) -> Bool {
if !str_starts_with(tool_name, "mcp__") { if !str_starts_with(tool_name, "mcp__") {
return false return false
} }
let raw: String = exec_capture("curl -s --max-time 5 http://127.0.0.1:7771/mcp/auto-approved") let raw: String = exec_capture("curl -s --max-time 2 http://127.0.0.1:7771/mcp/auto-approved")
if str_eq(raw, "") { if str_eq(raw, "") {
return false return false
} }
@@ -953,14 +1008,6 @@ fn agentic_loop(session_id: String, model: String, safe_sys: String, tools_json:
let iteration: Int = 0 let iteration: Int = 0
let keep_going: Bool = true let keep_going: Bool = true
// Issue #9: agentic max_tokens configurable via NEURON_LLM_MAX_TOKENS env var.
// Default 4096 is marginal for long tool chains (8 iterations x 4096 tokens).
// Set to 8192+ for complex multi-step tasks.
// Note: llm_provider_request() in el_runtime.c also hardcodes 4096 for the
// llm_call_system() (non-agentic) path; that requires a C runtime change.
let max_tokens_env: String = env("NEURON_LLM_MAX_TOKENS")
let max_tokens_str: String = if str_eq(max_tokens_env, "") { "4096" } else { max_tokens_env }
// Suspension state captured at top level so it escapes the while body. // Suspension state captured at top level so it escapes the while body.
let pending: Bool = false let pending: Bool = false
let pend_tool_id: String = "" let pend_tool_id: String = ""
@@ -969,7 +1016,7 @@ fn agentic_loop(session_id: String, model: String, safe_sys: String, tools_json:
while keep_going && iteration < 8 { while keep_going && iteration < 8 {
let req_body: String = "{\"model\":\"" + model + "\"" let req_body: String = "{\"model\":\"" + model + "\""
+ ",\"max_tokens\":" + max_tokens_str + ",\"max_tokens\":4096"
+ ",\"system\":\"" + safe_sys + "\"" + ",\"system\":\"" + safe_sys + "\""
+ ",\"tools\":" + tools_json + ",\"tools\":" + tools_json
+ ",\"messages\":" + messages + ",\"messages\":" + messages
@@ -1249,11 +1296,9 @@ fn handle_chat_as_soul(body: String) -> String {
let raw_response: String = llm_call_system(model, system_prompt, eff_message) let raw_response: String = llm_call_system(model, system_prompt, eff_message)
// Issue #5: empty string catch same rationale as handle_chat.
let is_error: Bool = str_starts_with(raw_response, "{\"error\"") let is_error: Bool = str_starts_with(raw_response, "{\"error\"")
|| str_starts_with(raw_response, "{\"type\":\"error\"") || str_starts_with(raw_response, "{\"type\":\"error\"")
|| str_contains(raw_response, "authentication_error") || str_contains(raw_response, "authentication_error")
|| str_eq(raw_response, "")
if is_error { if is_error {
return "{\"error\":\"llm unavailable\",\"response\":\"\",\"speaker_slug\":\"" + speaker + "\",\"model\":\"" + model + "\"}" return "{\"error\":\"llm unavailable\",\"response\":\"\",\"speaker_slug\":\"" + speaker + "\",\"model\":\"" + model + "\"}"
} }
@@ -1300,11 +1345,9 @@ fn handle_dharma_room_turn(body: String) -> String {
let raw_response: String = llm_call_system(model, system_prompt, transcript) let raw_response: String = llm_call_system(model, system_prompt, transcript)
// Issue #5: empty string catch same rationale as handle_chat.
let is_error: Bool = str_starts_with(raw_response, "{\"error\"") let is_error: Bool = str_starts_with(raw_response, "{\"error\"")
|| str_starts_with(raw_response, "{\"type\":\"error\"") || str_starts_with(raw_response, "{\"type\":\"error\"")
|| str_contains(raw_response, "authentication_error") || str_contains(raw_response, "authentication_error")
|| str_eq(raw_response, "")
if is_error { if is_error {
return "{\"error\":\"llm unavailable\",\"response\":\"\",\"cgi_id\":\"" + cgi_id + "\"}" return "{\"error\":\"llm unavailable\",\"response\":\"\",\"cgi_id\":\"" + cgi_id + "\"}"
} }
@@ -1399,14 +1442,28 @@ fn auto_persist(req: String, resp: String) -> Void {
let safe_msg: String = str_replace(message, "\"", "'") let safe_msg: String = str_replace(message, "\"", "'")
let safe_reply: String = str_replace(reply2, "\"", "'") let safe_reply: String = str_replace(reply2, "\"", "'")
// Detect emotional salience before persisting. safety_detect_bell_level uses the
// same phrase lists as the safety layer (safety.el), so the classification is
// consistent with what safety_screen already evaluated for this turn.
let bell_level: String = safety_detect_bell_level(message)
let is_bell: Bool = !str_eq(bell_level, "none")
// Tag the Conversation node with bell metadata when distress is present so
// subsequent affective queries (e.g. engram_compile) can find this exchange.
let tags: String = if is_bell {
"[\"Conversation\",\"chat\",\"timestamped\",\"bell:" + bell_level + "\",\"affective\"]"
} else {
"[\"Conversation\",\"chat\",\"timestamped\"]"
}
let content: String = "{\"q\":\"" + safe_msg + "\"" let content: String = "{\"q\":\"" + safe_msg + "\""
+ ",\"a\":\"" + safe_reply + "\"" + ",\"a\":\"" + safe_reply + "\""
+ ",\"created_at\":" + ts_str + ",\"created_at\":" + ts_str
+ ",\"source\":\"chat\"" + ",\"source\":\"chat\""
+ ",\"bell\":\"" + bell_level + "\""
+ ",\"label\":\"chat:" + ts_str + "\"}" + ",\"label\":\"chat:" + ts_str + "\"}"
let tags: String = "[\"Conversation\",\"chat\",\"timestamped\"]" let conv_node_id: String = engram_node_full(
engram_node_full(
content, content,
"Conversation", "Conversation",
"chat:" + ts_str, "chat:" + ts_str,
@@ -1416,6 +1473,72 @@ fn auto_persist(req: String, resp: String) -> Void {
"Episodic", "Episodic",
tags tags
) )
// When a bell fires, write a dedicated BellEvent node in addition to the
// Conversation node. This makes distress moments directly findable by label
// ("bell:soft" / "bell:hard") without having to scan all Conversation nodes.
// The BellEvent carries higher salience so engram_compile pulls it into context.
// The message content is truncated to 120 chars enough signal, not a full dump.
if is_bell {
let summary: String = if str_len(message) > 120 { str_slice(message, 0, 120) } else { message }
let safe_summary: String = str_replace(summary, "\"", "'")
let bell_content: String = "BELL:" + bell_level
+ " | ts:" + ts_str
+ " | summary:" + safe_summary
// bell:hard gets peak salience; bell:soft is slightly lower.
let sal_a: String = if str_eq(bell_level, "hard") { el_from_float(0.98) } else { el_from_float(0.88) }
let sal_b: String = if str_eq(bell_level, "hard") { el_from_float(0.98) } else { el_from_float(0.88) }
let sal_c: String = if str_eq(bell_level, "hard") { el_from_float(1.0) } else { el_from_float(0.95) }
let bell_tags: String = "[\"safety\",\"bell\",\"bell:" + bell_level + "\",\"affective\",\"BellEvent\"]"
let bell_ts_str: String = int_to_str(time_now())
let bell_label: String = "bell:" + bell_level + ":" + bell_ts_str
let bell_node_id: String = engram_node_full(
bell_content,
"BellEvent",
bell_label,
sal_a,
sal_b,
sal_c,
"Episodic",
bell_tags
)
// Increment session-level bell counter so session_hist_save knows whether
// any bell fired during this session when writing a boundary summary.
let sess_id: String = json_get(req, "session_id")
let bell_key: String = if str_eq(sess_id, "") {
"session_bell_count"
} else {
"session_bell_count:" + sess_id
}
let prior_count: String = state_get(bell_key)
let prior_n: Int = if str_eq(prior_count, "") { 0 } else { str_to_int(prior_count) }
state_set(bell_key, int_to_str(prior_n + 1))
// Also record the highest bell level seen this session so the boundary
// summary can classify the session correctly (hard takes precedence).
let level_key: String = if str_eq(sess_id, "") {
"session_bell_level"
} else {
"session_bell_level:" + sess_id
}
let prior_level: String = state_get(level_key)
let new_level: String = if str_eq(bell_level, "hard") { "hard" } else {
if str_eq(prior_level, "hard") { "hard" } else { "soft" }
}
state_set(level_key, new_level)
// Stash a short signal summary for the boundary node (last bell wins for
// the one-liner; the full history is in per-bell BellEvent nodes).
let signal_key: String = if str_eq(sess_id, "") {
"session_bell_signal"
} else {
"session_bell_signal:" + sess_id
}
state_set(signal_key, safe_summary)
}
} }
// strengthen_chat_nodes strengthen the engram nodes that were activated during a chat. // strengthen_chat_nodes strengthen the engram nodes that were activated during a chat.
+107 -7
View File
@@ -7,6 +7,65 @@ import "neuron-api.el"
import "sessions.el" import "sessions.el"
import "soul.elh" import "soul.elh"
// ---------------------------------------------------------------------------
// Rate limiting simple in-memory per-IP sliding window counter.
//
// State keys:
// rl:<ip>:count request count in the current window
// rl:<ip>:window window start timestamp (unix seconds)
//
// Limit: configurable via soul state key "soul_rate_limit" (requests per
// minute). Falls back to 60 req/min if not set. The /health endpoint is
// exempt so monitoring does not consume quota.
//
// State growth: each unique source IP accumulates exactly 2 state keys
// (count + window) for the lifetime of the process. Per-IP storage is
// bounded and constant; values reset on window expiry. In aggregate, state
// grows linearly with distinct IPs typical for a trusted-client service.
// EL has no state_delete builtin, so keys from inactive IPs persist.
// TODO: add state_delete sweep when the EL runtime exposes that primitive.
//
// Returns "" when the request is allowed, or a 429 JSON body when rejected.
// ---------------------------------------------------------------------------
fn rate_limit_check(ip: String, path: String) -> String {
// Health checks are exempt they must never be blocked.
if str_eq(path, "/health") {
return ""
}
let limit_str: String = state_get("soul_rate_limit")
let limit: Int = if str_eq(limit_str, "") { 60 } else { str_to_int(limit_str) }
let now: Int = time_now()
let window_key: String = "rl:" + ip + ":window"
let count_key: String = "rl:" + ip + ":count"
let win_str: String = state_get(window_key)
let win_start: Int = if str_eq(win_str, "") { now } else { str_to_int(win_str) }
// New window every 60 seconds.
let elapsed: Int = now - win_start
let in_window: Bool = elapsed < 60
let prev_count_str: String = state_get(count_key)
let prev_count: Int = if str_eq(prev_count_str, "") { 0 } else { str_to_int(prev_count_str) }
// Reset window if expired.
let eff_count: Int = if in_window { prev_count } else { 0 }
let eff_win: Int = if in_window { win_start } else { now }
let new_count: Int = eff_count + 1
state_set(count_key, int_to_str(new_count))
state_set(window_key, int_to_str(eff_win))
if new_count > limit {
let retry_after: Int = 60 - (now - eff_win)
let eff_retry: Int = if retry_after < 0 { 0 } else { retry_after }
return "{\"__status__\":429,\"error\":\"rate limit exceeded\",\"code\":\"rate_limited\",\"retry_after_secs\":" + int_to_str(eff_retry) + "}"
}
return ""
}
fn strip_query(path: String) -> String { fn strip_query(path: String) -> String {
let q: Int = str_index_of(path, "?") let q: Int = str_index_of(path, "?")
if q < 0 { if q < 0 {
@@ -16,11 +75,11 @@ fn strip_query(path: String) -> String {
} }
fn err_404(path: String) -> String { fn err_404(path: String) -> String {
return "{\"error\":\"not found\",\"path\":\"" + path + "\"}" return "{\"error\":\"not found\",\"code\":\"not_found\",\"path\":\"" + path + "\"}"
} }
fn err_405(method: String, path: String) -> String { fn err_405(method: String, path: String) -> String {
return "{\"error\":\"method not allowed\",\"method\":\"" + method + "\",\"path\":\"" + path + "\"}" return "{\"error\":\"method not allowed\",\"code\":\"method_not_allowed\",\"method\":\"" + method + "\",\"path\":\"" + path + "\"}"
} }
fn route_health() -> String { fn route_health() -> String {
@@ -31,12 +90,35 @@ fn route_health() -> String {
let edge_ct: Int = engram_edge_count() let edge_ct: Int = engram_edge_count()
let pulse: String = state_get("soul.pulse") let pulse: String = state_get("soul.pulse")
let pulse_num: String = if str_eq(pulse, "") { "0" } else { pulse } let pulse_num: String = if str_eq(pulse, "") { "0" } else { pulse }
// Uptime: soul records boot timestamp in state at startup via soul_boot_ts.
// Compute elapsed seconds; fall back to -1 if not yet set.
let boot_ts_str: String = state_get("soul_boot_ts")
let uptime_secs: Int = if str_eq(boot_ts_str, "") {
-1
} else {
time_now() - str_to_int(boot_ts_str)
}
// LLM connectivity: probe with a minimal call. Any non-error reply = ok.
// Use a short, fixed prompt so this never counts against conversation history.
let model: String = state_get("soul_model")
let eff_model: String = if str_eq(model, "") { "claude-sonnet-4-5" } else { model }
let llm_probe: String = llm_call_system(eff_model, "You are a health probe. Reply with the single word: ok", "ping")
let llm_ok: Bool = !str_eq(llm_probe, "")
&& !str_starts_with(llm_probe, "{\"error\"")
&& !str_starts_with(llm_probe, "{\"type\":\"error\"")
&& !str_contains(llm_probe, "authentication_error")
let llm_status: String = if llm_ok { "ok" } else { "unreachable" }
return "{\"status\":\"alive\"" return "{\"status\":\"alive\""
+ ",\"cgi_id\":\"" + cgi_id + "\"" + ",\"cgi_id\":\"" + cgi_id + "\""
+ ",\"boot\":" + boot_num + ",\"boot\":" + boot_num
+ ",\"uptime_secs\":" + int_to_str(uptime_secs)
+ ",\"node_count\":" + int_to_str(node_ct) + ",\"node_count\":" + int_to_str(node_ct)
+ ",\"edge_count\":" + int_to_str(edge_ct) + ",\"edge_count\":" + int_to_str(edge_ct)
+ ",\"pulse\":" + pulse_num + ",\"pulse\":" + pulse_num
+ ",\"llm\":\"" + llm_status + "\""
+ ",\"layers\":{\"l0\":\"core\",\"l1\":\"safety\",\"l2\":\"stewardship\",\"l3\":\"" + imprint_current() + "\"}}" + ",\"layers\":{\"l0\":\"core\",\"l1\":\"safety\",\"l2\":\"stewardship\",\"l3\":\"" + imprint_current() + "\"}}"
} }
@@ -103,15 +185,15 @@ fn route_imprint_user(body: String) -> String {
fn route_synthesize(body: String) -> String { fn route_synthesize(body: String) -> String {
if str_eq(body, "") { if str_eq(body, "") {
return "{\"mechanism\":\"did not engage\"}" return "{\"error\":\"body is required\",\"code\":\"missing_param\"}"
} }
let parent_a: String = json_get(body, "parent_a") let parent_a: String = json_get(body, "parent_a")
let parent_b: String = json_get(body, "parent_b") let parent_b: String = json_get(body, "parent_b")
if str_eq(parent_a, "") { if str_eq(parent_a, "") {
return "{\"mechanism\":\"did not engage\"}" return "{\"error\":\"parent_a is required\",\"code\":\"missing_param\"}"
} }
if str_eq(parent_b, "") { if str_eq(parent_b, "") {
return "{\"mechanism\":\"did not engage\"}" return "{\"error\":\"parent_b is required\",\"code\":\"missing_param\"}"
} }
let req: String = "synthesize " + parent_a + " " + parent_b let req: String = "synthesize " + parent_a + " " + parent_b
let tags: String = "[\"soul-inbox-pending\",\"synthesis-request\"]" let tags: String = "[\"soul-inbox-pending\",\"synthesis-request\"]"
@@ -259,6 +341,17 @@ fn handle_connectors(method: String, clean: String, body: String) -> String {
fn handle_request(method: String, path: String, body: String) -> String { fn handle_request(method: String, path: String, body: String) -> String {
let clean: String = strip_query(path) let clean: String = strip_query(path)
// Rate limit check. Extract caller IP from REMOTE_ADDR env var (set by the
// EL HTTP runtime for each request). Skip enforcement when empty so
// loopback/internal callers are never blocked.
let ip: String = env("REMOTE_ADDR")
if !str_eq(ip, "") {
let rl_result: String = rate_limit_check(ip, clean)
if !str_eq(rl_result, "") {
return rl_result
}
}
if str_eq(method, "POST") && str_eq(clean, "/dharma/recv") { if str_eq(method, "POST") && str_eq(clean, "/dharma/recv") {
return handle_dharma_recv(body) return handle_dharma_recv(body)
} }
@@ -286,7 +379,7 @@ fn handle_request(method: String, path: String, body: String) -> String {
let raw_msg: String = json_get(body, "message") let raw_msg: String = json_get(body, "message")
let eff_msg: String = if str_eq(raw_msg, "") { body } else { raw_msg } let eff_msg: String = if str_eq(raw_msg, "") { body } else { raw_msg }
if str_eq(eff_msg, "") { if str_eq(eff_msg, "") {
return "{\"error\":\"message required\"}" return "{\"error\":\"message is required\",\"code\":\"missing_param\"}"
} }
let agentic_flag: Bool = json_get_bool(body, "agentic") let agentic_flag: Bool = json_get_bool(body, "agentic")
let reply: String = if agentic_flag { let reply: String = if agentic_flag {
@@ -426,8 +519,15 @@ fn handle_request(method: String, path: String, body: String) -> String {
return handle_elp_chat(body) return handle_elp_chat(body)
} }
if str_eq(clean, "/api/chat") { if str_eq(clean, "/api/chat") {
let agentic_flag: Bool = json_get_bool(body, "agentic") // NOTE: streaming (SSE / chunked transfer) is not implemented. All chat
// responses are buffered and returned as a single JSON object. Streaming
// would require runtime-level SSE support in el_runtime.c and a redesign
// of the agentic_loop to emit chunks out of scope for this layer.
let raw_msg: String = json_get(body, "message") let raw_msg: String = json_get(body, "message")
if str_eq(raw_msg, "") {
return "{\"error\":\"message is required\",\"code\":\"missing_param\"}"
}
let agentic_flag: Bool = json_get_bool(body, "agentic")
let reply: String = if agentic_flag { let reply: String = if agentic_flag {
handle_chat_agentic(body) handle_chat_agentic(body)
} else { } else {
+3 -26
View File
@@ -144,22 +144,17 @@ fn safety_screen(input: String, history: String) -> String {
if score >= soft { if score >= soft {
let summary: String = str_slice(input, 0, 80) let summary: String = str_slice(input, 0, 80)
let discard: String = safety_log_bell("soft", "wellbeing check needed", summary) let discard: String = safety_log_bell("soft", "wellbeing check needed", summary)
// ISSUE 7 fix: escape tab chars in addition to backslash/quote/newline/CR.
// A tab in user input corrupts the JSON envelope and causes json_get to misparse.
let e1: String = str_replace(input, "\\", "\\\\") let e1: String = str_replace(input, "\\", "\\\\")
let e2: String = str_replace(e1, "\"", "\\\"") let e2: String = str_replace(e1, "\"", "\\\"")
let e3: String = str_replace(e2, "\n", "\\n") let e3: String = str_replace(e2, "\n", "\\n")
let e4: String = str_replace(e3, "\r", "\\r") let safe_input: String = str_replace(e3, "\r", "\\r")
let safe_input: String = str_replace(e4, "\t", "\\t")
return "{\"action\":\"soft_bell\",\"reason\":\"wellbeing check needed\",\"content\":\"" + safe_input + "\"}" return "{\"action\":\"soft_bell\",\"reason\":\"wellbeing check needed\",\"content\":\"" + safe_input + "\"}"
} }
// ISSUE 7 fix: escape tab chars (see soft_bell branch above for rationale).
let e1: String = str_replace(input, "\\", "\\\\") let e1: String = str_replace(input, "\\", "\\\\")
let e2: String = str_replace(e1, "\"", "\\\"") let e2: String = str_replace(e1, "\"", "\\\"")
let e3: String = str_replace(e2, "\n", "\\n") let e3: String = str_replace(e2, "\n", "\\n")
let e4: String = str_replace(e3, "\r", "\\r") let safe_input: String = str_replace(e3, "\r", "\\r")
let safe_input: String = str_replace(e4, "\t", "\\t")
return "{\"action\":\"pass\",\"content\":\"" + safe_input + "\"}" return "{\"action\":\"pass\",\"content\":\"" + safe_input + "\"}"
} }
@@ -200,11 +195,7 @@ fn safety_validate(output: String, action: String) -> String {
fn safety_log_bell(level: String, reason: String, input_summary: String) -> String { fn safety_log_bell(level: String, reason: String, input_summary: String) -> String {
let content: String = "BELL:" + level + " | " + reason + " | summary:" + input_summary let content: String = "BELL:" + level + " | " + reason + " | summary:" + input_summary
let tags: String = "[\"safety\",\"bell\",\"bell:" + level + "\"]" let tags: String = "[\"safety\",\"bell\",\"bell:" + level + "\"]"
// ISSUE 2 fix: if engram_node_full returns empty the write silently failed. let discard: String = engram_node_full(
// Emit a fallback println so the bell event leaves at least a log trace even
// when engram is degraded. This does not replace engram persistence -- it is a
// last-resort audit trail when the primary write cannot be confirmed.
let node_id: String = engram_node_full(
content, content,
"BellEvent", "BellEvent",
"bell:" + level, "bell:" + level,
@@ -214,9 +205,6 @@ fn safety_log_bell(level: String, reason: String, input_summary: String) -> Stri
"Episodic", "Episodic",
tags tags
) )
if str_eq(node_id, "") {
println("[safety] WARN: bell event engram write failed -- fallback log: " + content)
}
return "" return ""
} }
@@ -247,17 +235,6 @@ fn safety_soft_phrases() -> String {
return "[\"stressed\",\"overwhelmed\",\"can't cope\",\"cannot cope\",\"struggling\",\"anxious\",\"anxiety\",\"depressed\",\"depression\",\"lonely\",\"isolated\",\"hopeless\",\"hopelessness\",\"exhausted\",\"burnt out\",\"burned out\",\"burnout\",\"panic\",\"panicking\",\"falling apart\",\"breaking down\",\"can't handle\",\"cannot handle\",\"losing it\",\"nothing matters\",\"don't care anymore\",\"given up\",\"giving up\",\"helpless\",\"worthless\",\"useless\",\"hate myself\",\"no one cares\",\"nobody cares\",\"no one understands\",\"nobody understands\",\"empty inside\",\"can't stop crying\",\"breaking point\",\"at my limit\",\"having a breakdown\"]" return "[\"stressed\",\"overwhelmed\",\"can't cope\",\"cannot cope\",\"struggling\",\"anxious\",\"anxiety\",\"depressed\",\"depression\",\"lonely\",\"isolated\",\"hopeless\",\"hopelessness\",\"exhausted\",\"burnt out\",\"burned out\",\"burnout\",\"panic\",\"panicking\",\"falling apart\",\"breaking down\",\"can't handle\",\"cannot handle\",\"losing it\",\"nothing matters\",\"don't care anymore\",\"given up\",\"giving up\",\"helpless\",\"worthless\",\"useless\",\"hate myself\",\"no one cares\",\"nobody cares\",\"no one understands\",\"nobody understands\",\"empty inside\",\"can't stop crying\",\"breaking point\",\"at my limit\",\"having a breakdown\"]"
} }
// ISSUE 5 TODO: phrase lists are rebuilt from JSON literals on every call.
// safety_any_match and safety_count_match loop over json_array_get on every invocation.
// A compiled/cached representation would reduce per-message overhead and also guard against
// malformed phrase JSON (json_array_len of malformed input returns 0, silently skipping all checks).
// Caching requires language-level static const arrays -- not available in current EL.
// When EL gains module-level const arrays, migrate phrase lists to that form.
//
// ISSUE 5 TODO: phrase lists are rebuilt from JSON literals on every call to
// safety_any_match / safety_count_match. json_array_len of a malformed string
// returns 0, silently skipping all checks. Caching requires language-level static
// const arrays (not available in current EL). Migrate when EL gains that feature.
// Matching helpers (single loops only el escapes while-body mutation via // Matching helpers (single loops only el escapes while-body mutation via
// top-level let rebinds; nested loops would not advance) ──────────────────── // top-level let rebinds; nested loops would not advance) ────────────────────
+42
View File
@@ -368,6 +368,48 @@ fn session_hist_save(session_id: String, hist: String) -> Void {
el_from_float(0.6), el_from_float(0.6), el_from_float(0.9), el_from_float(0.6), el_from_float(0.6), el_from_float(0.9),
"Episodic", tags "Episodic", tags
) )
// Session boundary emotional summary written once per session the first time
// a bell event has fired. The summary node is findable by future sessions via
// broad affective queries ("session:emotional-summary" or "bell distress session").
// It is NOT rewritten on every save the state flag prevents duplicate nodes.
let summary_written_key: String = "session_bell_summary_written:" + session_id
let already_written: String = state_get(summary_written_key)
if str_eq(already_written, "") {
let bell_count_key: String = "session_bell_count:" + session_id
let bell_count_raw: String = state_get(bell_count_key)
let bell_count: Int = if str_eq(bell_count_raw, "") { 0 } else { str_to_int(bell_count_raw) }
if bell_count > 0 {
let bell_level_key: String = "session_bell_level:" + session_id
let bell_signal_key: String = "session_bell_signal:" + session_id
let dominant_level: String = state_get(bell_level_key)
let last_signal: String = state_get(bell_signal_key)
let eff_level: String = if str_eq(dominant_level, "") { "soft" } else { dominant_level }
let eff_signal: String = if str_eq(last_signal, "") { "(no signal captured)" } else { last_signal }
let ts_now: Int = time_now()
let summary_content: String = "session:emotional-summary"
+ " | session:" + session_id
+ " | bell_count:" + int_to_str(bell_count)
+ " | dominant_level:" + eff_level
+ " | last_signal:" + eff_signal
+ " | ts:" + int_to_str(ts_now)
let summary_tags: String = "[\"session-emotional-summary\",\"affective\",\"bell:" + eff_level + "\",\"BellEvent\"]"
let summary_sal: String = if str_eq(eff_level, "hard") { el_from_float(0.95) } else { el_from_float(0.85) }
let sum_discard: String = engram_node_full(
summary_content,
"BellEvent",
"session:emotional-summary",
summary_sal,
summary_sal,
el_from_float(1.0),
"Episodic",
summary_tags
)
// Mark written so we do not create duplicate summary nodes as the
// session continues accumulating more turns.
state_set(summary_written_key, "1")
}
}
} }
// session_update_meta_timestamp update the updated_at field in the session:meta node. // session_update_meta_timestamp update the updated_at field in the session:meta node.
+6 -24
View File
@@ -5,9 +5,13 @@ import "stewardship.el"
import "imprint.el" import "imprint.el"
import "awareness.el" import "awareness.el"
import "chat.el" import "chat.el"
import "safety.el"
import "studio.el" import "studio.el"
import "elp-input.el" import "elp-input.el"
import "routes.el" import "routes.el"
import "safety.el"
import "stewardship.el"
import "imprint.el"
cgi "neuron-soul" { cgi "neuron-soul" {
dharma_id: "ntn-genesis@http://localhost:7770", dharma_id: "ntn-genesis@http://localhost:7770",
@@ -261,32 +265,19 @@ fn layered_cycle(raw_input: String) -> String {
let screen_result: String = safety_screen(raw_input, history) let screen_result: String = safety_screen(raw_input, history)
let screen_action: String = json_get(screen_result, "action") let screen_action: String = json_get(screen_result, "action")
// ISSUE 4: safe-mode guard -- if safety_screen returned invalid/empty action,
// refuse the turn rather than silently passing unscreened input to upper layers.
// Valid actions: "hard_bell", "soft_bell", "pass". Anything else = corrupt envelope.
let valid_action: Bool = str_eq(screen_action, "hard_bell")
|| str_eq(screen_action, "soft_bell")
|| str_eq(screen_action, "pass")
if !valid_action {
println("[soul] layered_cycle: safety_screen invalid action -- safe mode refusal")
return safety_validate("", "hard_bell")
}
// Hard bell: bypass all upper layers, log and escalate. // Hard bell: bypass all upper layers, log and escalate.
// Intentionally does NOT update conversation_history or call auto_persist(): // Intentionally does NOT update conversation_history or call auto_persist():
// hard bell events are security-sensitive and must not appear in engram conversation // hard bell events are security-sensitive and must not appear in engram conversation
// history where they could leak context to subsequent turns. They are persisted // history where they could leak context to subsequent turns. They are persisted
// separately by safety_log_bell() into the Episodic tier with restricted labels. // separately by safety_log_bell() into the Episodic tier with restricted labels.
// //
// ISSUE 6: safety_log_bell for hard bells is already called INSIDE safety_screen
// (safety.el line 140). Do NOT call it again here -- double-log avoided.
//
// safety_validate second param: when screen_action is "hard_bell", safety_validate // safety_validate second param: when screen_action is "hard_bell", safety_validate
// receives the sentinel string "hard_bell" (not a normal screen action). The safety // receives the sentinel string "hard_bell" (not a normal screen action). The safety
// layer contract requires it to return a fixed refusal regardless of the output arg. // layer contract requires it to return a fixed refusal regardless of the output arg.
// On the normal path, safety_validate receives the original screen_action ("pass") // On the normal path, safety_validate receives the original screen_action ("pass")
// so it can apply action-specific post-output checks. // so it can apply action-specific post-output checks.
if str_eq(screen_action, "hard_bell") { if str_eq(screen_action, "hard_bell") {
safety_log_bell("hard", json_get(screen_result, "reason"), str_slice(raw_input, 0, 80))
return safety_validate("", "hard_bell") return safety_validate("", "hard_bell")
} }
@@ -321,16 +312,6 @@ fn layered_cycle(raw_input: String) -> String {
json_get(steward_result, "redirect_to") json_get(steward_result, "redirect_to")
} }
// ISSUE 1: apply pre-LLM bell augmentation on layered_cycle path.
// safety_augment_system injects soft/hard directive into system prompt before LLM call.
// Stored in state so imprint_respond can consume it.
// TODO: wire directly into imprint_respond when it accepts a system_override param.
// ISSUE 3 TODO: no semantic/embedding crisis detection. Keyword-only means signals
// evading the phrase list pass through with zero augmentation. Semantic layer is a
// separate architectural decision requiring embedding inference on every message.
let augmented_addendum: String = safety_augment_system("", raw_input)
state_set("layered_cycle_safety_system_addendum", augmented_addendum)
// L3: imprint responds // L3: imprint responds
let output: String = imprint_respond(aligned, imprint_id) let output: String = imprint_respond(aligned, imprint_id)
@@ -388,6 +369,7 @@ load_identity_context()
seed_persona_from_env() seed_persona_from_env()
let boot_num: Int = mem_boot_count_inc() let boot_num: Int = mem_boot_count_inc()
state_set("soul_boot_count", int_to_str(boot_num)) state_set("soul_boot_count", int_to_str(boot_num))
state_set("soul_boot_ts", int_to_str(time_now()))
println("[soul] boot #" + int_to_str(boot_num)) println("[soul] boot #" + int_to_str(boot_num))
emit_session_start_event() emit_session_start_event()