nsbx: fail loud on daemon-not-ready + el_seed.c standalone compile #118
Merged
will.anderson
merged 5 commits from 2026-08-15 23:06:02 +00:00
fix/nsbx-tooling-hardening into dev
5 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 bigmerge
|
3718bf0380 |
runtime: port missing __channel_* primitives into el_seed.c
El SDK CI - dev / build-and-test (pull_request) Failing after 3m44s
runtime/channel.el has always called __channel_new/__channel_send/ __channel_recv/__channel_try_recv/__channel_close, but these were only ever implemented in the pre-restructure lang/el-compiler/runtime/el_runtime.c. When the canonical runtime was consolidated onto the release copy (lang/runtime/el_runtime.c) and el_seed.c became the sole C dependency, the channel implementation was never carried forward — __mutex_new made the move, __channel_* did not. Any El program using Go-style channels currently fails to link on dev. Ported the working buffered-MPMC-channel implementation (mutex+condvar+ circular buffer, bounded and unbounded modes) from the old el_runtime.c verbatim, adapted only to el_seed.c's arena API (seed_arena_track in place of el_arena_track). Declared in el_seed.h alongside the existing mutex primitives. |
||
|
bigmerge
|
9d40f87926 |
ingest: unify transduce_prose/transduce_structured into one transduce()
transduce() is now THE single mechanism: one function, no content-type branch inside it. It never asks whether `source` is prose, JSON, or raw/opaque bytes (audio, etc.) — it runs one algorithm unconditionally: split on "\n\n" as a universal boundary-marker check, and if that finds no boundary, fall back to fixed 4096-char windows. Same node/edge wiring (root -contains-> chunk, chunk -precedes-> next, "#"-prefixed chunk gets a heading/section_of link) regardless of what's inside a chunk. Dedup is the existing find_existing_by_content path via merge_manifold, applied uniformly. The old transduce_structured JSON dataset/records/feature-node interpretation is deleted outright, not just unused — a JSON file now gets chunked and deduped like anything else, with no pre-computed structure. All five ingest_* entry points still exist unchanged in name and role; ingest_file/ingest_dir/ingest_url/ingest_llm now call the one transduce() (ingest_stream builds its own turn-nodes directly and never called either old function, so it's untouched). This unlocks raw/opaque content (audio, or anything else with no natural text/JSON shape) without any DSP, LLM call, or external API: transduce() chunks it exactly like it chunks anything else. There is zero semantic understanding of audio (or any payload) claimed or built here — any meaning is expected to emerge later from Neuron's own existing mechanisms (embedding, spreading activation, dedup) acting on this real geometry over time. Two small C builtins added to el_runtime.c/h (fs_size, fs_read_b64_chunk) because El strings are NUL-unsafe under strlen-based ops and fs_read()'s result silently truncates at the first embedded NUL, which is routine in real binary/audio bytes. ingest_file compares fs_read()'s string length against a real fs_size() stat() count; on mismatch it rebuilds the payload as base64-encoded fixed 3072-byte windows read directly off disk (binary-safe in C, verbatim, no invention), joined with the same "\n\n" marker transduce()'s boundary scan already looks for. This is a mechanical fidelity fix, not interpretation of content — transduce() never learns a fallback happened. Registered both builtins' arity in codegen.el; did not rebuild the elc compiler binary itself (unrelated, pre-existing gap: self-hosting elc via el_seed.c fails on this worktree independent of this change, reproduced with codegen.el reverted) — the existing elc binary compiles calls to unregistered builtins via its already-existing arity=-1 passthrough, confirmed by an actual clean `elc ingest.el` + `cc` build against the modified el_runtime.c. INGEST_KIND keeps existing only as an acquisition-mechanism selector (dir/file/url/llm/stream — which RPC to use to fetch bytes), not as a content-type flag; the redundant "structured" value (an alias for "file" that hinted the now-deleted JSON branch) is removed. ingest_dir drops its file-extension filter for the same reason: transduce() takes anything now. Verification: local manifold construction confirmed correct against a real captured audio file (will_clean.wav, 304288 bytes, and a 12288-byte real prefix slice) — exact expected node/edge counts both times (101 nodes/199 edges full file; 5 nodes/7 edges for the slice, matching ceil(bytes/3072)+1 nodes and 2n-1 edges), with real, verbatim base64 content confirmed decoding back to the actual WAV header bytes. Compiles clean via the real elc + the modified el_runtime.c/engram_*.c (built and booted an actual sandbox engram off this exact source with `nsbx create --branch`). NOT verified this session, disclosed rather than papered over: end-to-end server-confirmed persistence (a real before/after /api/stats delta, and a fetched node by id) for the audio, prose, and JSON-fixture cases. Every local nsbx sandbox engram tried tonight (two stock pre-#109 binaries hitting the known O(N*D) brute-force scan bug, then a fresh #109/HNSW binary built from current dev) took minutes-to indefinitely long on the final /api/load-merge write's embedding step and hit the client's 60s HTTP timeout before responding, even for a 5-node write. This is confirmed as real (if slow) forward progress, not a hang: the sandbox's WAL file was observed growing steadily across every attempt. The code's own pre-existing HONESTY GATE correctly refused to report success in every case, returning "load-merge failed: ..." with a "nothing below this manifold was confirmed persisted by the server" note instead — exactly as designed. This is an environment/infrastructure limitation, not a defect introduced by this change: the engram server binary itself is untouched by this commit. |
||
|
bigmerge
|
90d3f0bc76 |
engram: port PR #105's 3 genuine wins onto dev's existing cosq/e_eff semantic layer
Reconciles PR #105 ("fix: engram search latency — pin embed model, cache query embeddings, bound activate BFS") with dev's ACTUAL current engram_activate, rather than the ancient pre-restructure snapshot #105 was built against. WHY THIS NEEDED RECONCILIATION, NOT A DIRECT PORT: #105's single commit ( |
||
|
bigmerge
|
2d0aef4ef8 |
lang: declare the el_runtime.c symbols el_seed.c's wrappers call
El SDK CI - dev / build-and-test (pull_request) Failing after 3m55s
runtime/el_seed.c does not compile standalone via the exact command
tools/install.sh uses (`cc -std=c11 -O2 -I runtime -c runtime/el_seed.c`):
51 of its __-prefixed wrapper functions (http serving, JSON access, key-val
state, URL/HTML escaping, and the whole engram_* node/edge/layer/search
surface) call unprefixed counterparts that are implemented in el_runtime.c,
not in el_seed.c itself, and el_seed.c never declared them -- a toolchain
that treats an implicit function declaration as a hard error under C11
fails the compile outright.
install.sh already compiles el_seed.c and el_runtime.c as separate objects
and archives both into libel.a, so the symbols are always present at link
time; el_seed.c alone was just missing the prototypes.
A plain `#include "el_runtime.h"` was tried first and rejected: it redefines
el_to_float/el_from_float, which el_seed.h already provides -- a real
compile error, not a style preference. Added narrow prototypes instead,
copied verbatim from el_runtime.h, for exactly the 51 symbols el_seed.c's
wrappers reference and nothing else.
Verified clean:
- `cc -std=c11 -O2 -I runtime -c runtime/el_seed.c` (install.sh's exact
per-file compile) -- 0 errors, 0 warnings, even with -ferror-limit=0.
- full `tools/install.sh` run -- compiles both objects and archives them
into libel.a successfully.
Separately (not fixed here, out of scope): AGENTS.md's documented compiler
self-rebuild command links elc-new.c against el_seed.c, but elc-new.c's own
generated `#include "el_runtime.h"` line and 3 undeclared symbols
(el_mem_check, stdout_to_file, stdout_restore -- present in neither
el_runtime.c nor el_seed.c) mean that command fails regardless of which
runtime file it's linked against; and install.sh's libel.a only archives
el_seed.o + el_runtime.o, so any program that calls into the engram_*
surface fails to link against it (el_runtime.c's engram_* wrappers need
engram_store.c/engram_geometry.c/engram_reason.c/engram_cognition.c/
engram_vindex.c, none of which install.sh compiles in). Both are real,
pre-existing, and independent of this fix -- worth their own look.
|
||
|
bigmerge
|
979e820f68 |
nsbx: fail loud on daemon-not-ready instead of printing a false success banner
Three confirmed-live bugs tonight: - `nsbx up` printed "daemon did not become ready" immediately followed by a green "your sandbox is ready" banner and exited 0, because the existing- sandbox restart path (`daemon_alive || start_daemon`) never checked start_daemon's return code. `cmd_build` had the identical unguarded pattern, plus `cmd_run`/`cmd_validate`'s own start-if-dead calls. All four now `|| die` with a message pointing at daemon.log. - `nsbx status`/`nsbx list` reported bare "state: running" for a process that's alive (passes kill -0) but not actually answering /api/stats -- pegged, hung, or mid-boot. Added daemon_health(), which does the real stats fetch and distinguishes stopped/running/unresponsive; both commands now say "running but NOT RESPONDING" with a next-step hint instead of silently going quiet on the stats field. Reproduced live against another agent's actively-running (CPU-pinned, non-responsive) sandbox tonight, and again via a deliberate SIGSTOP on a throwaway sandbox. - Sandboxes carried no visible signal that their binary predated a relevant fix. `status`/`list` now show the binary's sha + real build timestamp (mtime survives `cp -p`), plus a best-effort staleness note: for stock-prod clones, compare against the currently-configured live binary; for source/branch builds, compare the recorded source commit against local origin/dev via merge-base --is-ancestor. Also, found live while verifying the above: - A cold boot under concurrent sandbox/CPU load can legitimately take past the old hardcoded 15s readiness window. Made it configurable (NSBX_READY_TIMEOUT_SECS) rather than just widening the default blindly. - cmd_create's post-boot baseline capture could silently record sbx_baseline as 0/0 when the stats fetch came back empty right after the auto-remerge step -- which would make every future `nsbx validate` zero-loss/reboot- prove check trivially PASS regardless of real data loss. Added a bounded retry and a loud warning if it still comes back empty. - Sharpened a handful of "no such sandbox" / missing-binary errors to name the next command instead of just stating the failure. |