runtime: HTTP replies truncated to fs_read's file length — Content-Length lied, onboarding trapped #78

Closed
tim.lingo wants to merge 2 commits from fix/http-fs-read-len into main
Member

What customers hit: every fresh install trapped at 'Set your safety contact' with 'Couldn't save… check your connection', and configured users saw the gate re-appear every launch. Deterministic on every machine.

Root cause: the binary-safe fs_read length hint (_tl_fs_read_len) is consumed by the HTTP send path for ANY body on the same request. A handler that fs_reads a file and wraps it into a larger reply (safety-contact GET/POST) gets its reply truncated to the file's length — Content-Length lies AND the send stops short, cutting the JSON mid-field. The app correctly refuses the mangled reply. Also latent: a stale hint LARGER than a later body over-reads heap memory out the socket (privacy-grade risk).

Fix (f34270d): pair the hint with the exact buffer pointer it describes; the send path honors it only when the response IS that buffer (PNG/binary serving keeps working — the hint follows the worker's copy-out); both reset at request start. Also ports engram_get_node_by_label (from releases/v1.0.0) — needed by soul.el session continuity in local mode; not-found returns "" ("{}" flips the truthiness check in emit_session_start_event). Note: neuron repo's dist/elp-c-decls.h needs the prototype or arm64 implicit declaration truncates the returned pointer (added there, uncommitted; the decls generator should emit it).

Related: memory.el still string-compares engram_save's int return (#80) — re-applied as build fixups; imprint_respond echo placeholder (#81).

Verified: genesis boot + byte-math E2E — safety-contact GET/POST/GET all Content-Length==body and JSON-parse clean; /health + /api/config regressions match; 30s heartbeat survival. neuron-ui packaging now runs a response-integrity gate that boots a genesis brain and fails on any truncated/unparseable probe.

🤖 Generated with Claude Code

**What customers hit:** every fresh install trapped at 'Set your safety contact' with 'Couldn't save… check your connection', and configured users saw the gate re-appear every launch. Deterministic on every machine. **Root cause:** the binary-safe fs_read length hint (`_tl_fs_read_len`) is consumed by the HTTP send path for ANY body on the same request. A handler that fs_reads a file and wraps it into a larger reply (safety-contact GET/POST) gets its reply truncated to the file's length — Content-Length lies AND the send stops short, cutting the JSON mid-field. The app correctly refuses the mangled reply. Also latent: a stale hint LARGER than a later body over-reads heap memory out the socket (privacy-grade risk). **Fix (f34270d):** pair the hint with the exact buffer pointer it describes; the send path honors it only when the response IS that buffer (PNG/binary serving keeps working — the hint follows the worker's copy-out); both reset at request start. Also ports `engram_get_node_by_label` (from releases/v1.0.0) — needed by soul.el session continuity in local mode; not-found returns "" ("{}" flips the truthiness check in emit_session_start_event). Note: neuron repo's dist/elp-c-decls.h needs the prototype or arm64 implicit declaration truncates the returned pointer (added there, uncommitted; the decls generator should emit it). **Related:** memory.el still string-compares engram_save's int return (#80) — re-applied as build fixups; imprint_respond echo placeholder (#81). **Verified:** genesis boot + byte-math E2E — safety-contact GET/POST/GET all Content-Length==body and JSON-parse clean; /health + /api/config regressions match; 30s heartbeat survival. neuron-ui packaging now runs a response-integrity gate that boots a genesis brain and fails on any truncated/unparseable probe. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
tim.lingo added 2 commits 2026-07-17 23:29:36 +00:00
Measured on the live container mind (pinned 40-query eval, judged): substring
2/40=5% hit@5 -> ranked 35/40=88%. Multi-word queries stop returning zero; new
memories stop losing to storage order (created_at tiebreak). Transparent-layer
identity filter preserved in both passes; jb_finish (#64) tail preserved.
query_param now url_decode()s values - %XX arrived literal before (pre-existing
GET defect, masked while multi-word substring returned nothing anyway).
E2E-verified in Tim's container deployment 2026-07-14/15; eval harness:
docs repo research-archive/p0-prototypes/eval_pinned_40q_20260715.py.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
runtime: fs_read length hint must be paired with its buffer — fixes truncated HTTP responses
El SDK Release / build-and-release (pull_request) Failing after 25s
f34270d63d
The binary-safe fs_read length (_tl_fs_read_len) was consumed by the HTTP
response path for ANY body, even when the handler wrapped the file into a
larger reply. Content-Length then lied AND the send stopped short: the
safety-contact routes returned 178 of 208/218 bytes, cut mid-'set_at' —
unparseable JSON. The desktop app read that as failure: fresh installs
trapped at 'Set your safety contact' (POST reply mangled) and configured
users saw the gate re-appear every launch (GET reply mangled). Worse, a
stale hint LARGER than a later body would over-read heap memory out the
socket.

Fix: pair the hint with the exact buffer pointer it describes; consume it
only when the response IS that buffer (binary file serving keeps working,
the hint follows the worker's copy); reset both at request start. Also
ports engram_get_node_by_label (from releases/v1.0.0) needed by soul.el
session continuity in local mode — not-found returns "" (matches shipped
behavior; '{}' flips the truthiness check upstream).

Verified: genesis boot + byte-math E2E on :7797 sandbox — safety-contact
GET/POST/GET all Content-Length==body, json-parse clean; /health,
/api/config regressions match.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Owner

Superseded by #79 (durable pointer-identity truncation fix, both runtimes), landing on main via the dev->stage->main chain (PR #80/#81). #78 only touched the mainline runtime and was 21 commits behind main; #79 also fixes the v1.0.0 release runtime the desktop souls compile against — which is what kept the Windows brain truncating. Closing as superseded.

Superseded by #79 (durable pointer-identity truncation fix, both runtimes), landing on main via the dev->stage->main chain (PR #80/#81). #78 only touched the mainline runtime and was 21 commits behind main; #79 also fixes the v1.0.0 release runtime the desktop souls compile against — which is what kept the Windows brain truncating. Closing as superseded.
will.anderson closed this pull request 2026-07-22 21:07:45 +00:00

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
No Reviewers
No labels
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: neuron-technologies/el#78