neuron-technologies/el
3
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases 5 Wiki Activity

fix(runtime): UTF-8 guard, engram sync route, native platform backends, UI vessels #61

Closed
will.anderson wants to merge 0 commits from fix/windows-rusage-guard into main
pull from: fix/windows-rusage-guard
merge into: :main
Conversation 0 Commits - Files Changed -
will.anderson commented 2026-06-29 17:48:37 +00:00
Owner
Copy Link
Copy Source

Summary

Six commits landing a mix of fixes and new platform work:

Fixes

  • engram_node_full rejects invalid UTF-8 before any persist call — silent corrupt writes were garbling JSON snapshots
  • println flushes stdout after every write — launchd-redirected streams were holding startup log output indefinitely
  • Knowledge node activation threshold set to 0.20 (below generic 0.40) — consistent with ACT-R base-level for repeatedly-activated nodes

Engram

  • GET /api/sync route — soul daemon periodic pull of non-ISE knowledge into in-process graph; previously returned 404, causing the soul to attempt an empty merge every 10 min
  • ELP type headers updated: Any → [String] for generate_frame, generate_frame_lang, build_form_from_json

Native platform backends (9,699 lines)

  • el_native_target.h — shared target abstraction
  • el_appkit.m, el_uikit.m, el_android.c, el_gtk4.c, el_sdl2.c, el_lvgl.c, el_win32.c, el_runtime_win32.c

Platform tooling

  • ElBridge.java — Android JNI companion
  • PLATFORM_BRIDGE_SPEC.md — authoritative bridge contract
  • detect-platforms, new-platform — probe + scaffold scripts

UI

  • el-native vessel (wraps __widget_* builtins)
  • el-html/main.elh update
  • native-hello examples (AppKit, GTK4, SDL2, Win32, Pi, Docker, Android, iOS)
  • profile-card manifest, UI tools / native codegen pass

Test plan

  • engram_node_full with invalid UTF-8 content logs rejection and returns without persisting
  • GET /api/sync returns snapshot JSON; soul daemon sync loop no longer errors every 10 min
  • println output appears immediately in launchd log files (no buffering delay)
  • native-hello builds on at least one target (AppKit or GTK4)
## Summary Six commits landing a mix of fixes and new platform work: **Fixes** - `engram_node_full` rejects invalid UTF-8 before any persist call — silent corrupt writes were garbling JSON snapshots - `println` flushes stdout after every write — launchd-redirected streams were holding startup log output indefinitely - Knowledge node activation threshold set to 0.20 (below generic 0.40) — consistent with ACT-R base-level for repeatedly-activated nodes **Engram** - `GET /api/sync` route — soul daemon periodic pull of non-ISE knowledge into in-process graph; previously returned 404, causing the soul to attempt an empty merge every 10 min - ELP type headers updated: `Any` → `[String]` for `generate_frame`, `generate_frame_lang`, `build_form_from_json` **Native platform backends** (9,699 lines) - `el_native_target.h` — shared target abstraction - `el_appkit.m`, `el_uikit.m`, `el_android.c`, `el_gtk4.c`, `el_sdl2.c`, `el_lvgl.c`, `el_win32.c`, `el_runtime_win32.c` **Platform tooling** - `ElBridge.java` — Android JNI companion - `PLATFORM_BRIDGE_SPEC.md` — authoritative bridge contract - `detect-platforms`, `new-platform` — probe + scaffold scripts **UI** - `el-native` vessel (wraps `__widget_*` builtins) - `el-html/main.elh` update - native-hello examples (AppKit, GTK4, SDL2, Win32, Pi, Docker, Android, iOS) - profile-card manifest, UI tools / native codegen pass ## Test plan - [ ] `engram_node_full` with invalid UTF-8 content logs rejection and returns without persisting - [ ] `GET /api/sync` returns snapshot JSON; soul daemon sync loop no longer errors every 10 min - [ ] `println` output appears immediately in launchd log files (no buffering delay) - [ ] native-hello builds on at least one target (AppKit or GTK4)
will.anderson added 17 commits 2026-06-29 17:48:41 +00:00
feat(el-runtime): native Windows port of el_runtime.c (winsock/dlsym/CreateProcess) 28ef43264a 
Compiles for Windows x64 via mingw-w64 and still compiles clean on POSIX
(darwin/linux) — all Windows code is behind #ifdef _WIN32, POSIX path unchanged.

- el_platform_win.h (new): winsock2 + auto WSAStartup, el_closesocket(),
  dlsym->GetProcAddress, popen/_popen, mkdir/_mkdir, setenv/_putenv_s,
  timegm/_mkgmtime, localtime_r/gmtime_r. Threading unchanged — mingw
  winpthreads supplies <pthread.h> + -lpthread.
- el_runtime.c: include block guarded; 10 socket-close sites -> el_closesocket();
  setsockopt arg4 cast; tm_zone guarded; exec_bg fork/exec -> CreateProcess.

Part of feat/windows-port. Core-el change, for Will's review.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
fix(el-runtime): promote http_handler typedefs to el_runtime.h (cross-module + Windows)
El SDK Release / build-and-release (pull_request) Failing after 13m0s
Details
a36a62ca14 
http_handler_fn / http_handler4_fn were defined only inside el_runtime.c, so soul
modules (routes/chat/...) that reference them via cross-module forward declarations
couldn't see the types — which broke the Windows link of every module. Moving the
public function-pointer types to the shared header is the correct home and unblocks
the build on all platforms (identical typedef, C11-safe redefinition in el_runtime.c).

With this, the soul links into a native Windows neuron.exe (mingw, static) that boots
and serves HTTP on :7770 — verified /health → 200 {"status":"alive",...} in a Win11 VM.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
fix(runtime): reconcile live data-integrity fixes onto main (UAF + atomic engram_save)
El SDK Release / build-and-release (pull_request) Failing after 17s
Details
2dec76c87a 
Ports the fixes that until now lived only in the un-versioned el-sdk source the live
macOS soul was hand-built from (captured in the [DO NOT MERGE] live-darwin-runtime
snapshot) FORWARD onto main, faithfully and minimally — without dragging in the
snapshot's deletions of main's newer engram_wm_/engram_load_merge/http_serve_async.

1. UAF (hallucinated/lost-saves root cause): engram_new_id + engram_node_full now use
   el_strdup_persist, NOT el_strdup. el_strdup tracks into the per-request arena that
   el_request_end() frees when the creating HTTP request completes — leaving stored
   nodes with dangling pointers (corrupted ids, 'saved but never listed'). Transplanted
   verbatim from the live runtime; el_strdup_persist sites 19->27, matching live.

2. Atomic engram_save: write <path>.tmp, fflush+fsync, rename() over target (atomic on
   POSIX) so a booting soul's engram_load never reads a truncated/0-byte snapshot — the
   genesis -> nodes=1 -> 63-node-clobber loop. Plus a sparse-write floor: refuse to
   overwrite a >200KB snapshot with one < 1/16 its size. (Validated in isolation:
   harness 11/11; rebuilt+booted the darwin soul, round-tripped 5113 nodes, no clobber.)

The response-truncation fix is already on main (_tl_fs_read_len binary-safe length).
Compiles clean. For Will to build through CI/elb and deploy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
feat(engram): add accumulation layer (layer 5) — new nodes default to it, not core-identity
El SDK Release / build-and-release (pull_request) Failing after 12m23s
Details
18e1ab6db1 
Implements the accumulation layer from the Layered Consciousness architecture
(provisional 64/064,262) and answers the deferred design question. Per the spec
and Will's design: new user-facing nodes (memories, knowledge, conversations) are
created in an accumulation layer at the TOP of the consciousness stack — the engram
the user sees — while the layers below (safety, core-identity, domain, imprint,
suit) shape behavior but are hidden from the user.

- Adds ENGRAM_LAYER_ACCUMULATION (5) + the layer record in engram_init_layers
  (activation_priority 50, suppressible, not injectable, transparent=0).
- engram_node and engram_node_full now assign new nodes to ENGRAM_LAYER_ACCUMULATION.
- ENGRAM_LAYER_DEFAULT stays CORE_IDENTITY ON PURPOSE: it is the fallback for LEGACY
  nodes loaded from snapshots without a layer_id, so existing data (the originator
  corpus) is NEVER migrated. New-nodes-only — the immutable-originator rule.

This is the foundation for fixing the identity-bleed / customer-isolation issue
(user data was landing in Neuron's core-identity layer). The retrieval-side
provenance filter (introspection should compile from accumulation, not the
originator corpus — Persona 64/036,574) is a follow-on, pending the batch-2
Layered Consciousness + Engram spec docs for exact semantics. Compiles clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Merge pull request 'Reconcile live runtime data-integrity fixes onto main (UAF + atomic engram_save)' (#58) from fix/runtime-integrity-reconcile into stage
El SDK CI - stage / build-and-test (push) Failing after 9m32s
Details
b6187501fd
Merge pull request 'feat(engram): accumulation layer — new nodes to top of stack, not core-identity' (#59) from feat/accumulation-layer into stage
El SDK CI - stage / build-and-test (push) Failing after 8m50s
Details
718a2e0c06
fix(windows): resolve PR blockers — nanosleep shim, unsetenv, duplicate typedefs, SOCKET type, el_closesocket
El SDK CI - stage / build-and-test (pull_request) Failing after 22s
Details
c087b97093
Merge branch 'stage' into feat/windows-el-runtime
El SDK CI - stage / build-and-test (pull_request) Failing after 15s
Details
99b113ea9d 
Resolve el_runtime.c conflict: include both sys/resource.h (from stage)
and el_closesocket POSIX shim (from Windows port) within the #else block.
feat(windows): native Windows port of el_runtime.c — fix all blockers
El SDK CI - stage / build-and-test (push) Failing after 7m45s
Details
2d751890ea
fix(windows): guard el_mem_check with _WIN32 — rusage is POSIX-only
El SDK CI - stage / build-and-test (pull_request) Failing after 11m3s
Details
93d36fddb1
self-review 2026-06-26: WM cap, breakthrough floor 0.25→0.10, ISE WM exclusion, /api/neuron/state-events route f7bd99ae45 
Three improvements from today's self-review:

1. ENGRAM_BREAKTHROUGH_WEIGHT 0.25→0.10
   Live data showed 524/525 WM nodes at breakthrough floor (0.25). Knowledge
   nodes promoted at 0.21 decayed to 0.147 in one call, fell below the old
   0.25 floor, and were immediately evicted for fresh breakthrough candidates.
   Natural promotion was invisible. Invariant maintained: 0.10 < all
   per-type thresholds (min=0.15 Canonical).

2. ENGRAM_WM_CAP=24 with Pass 4 (per-call) + Pass 5 (global) enforcement
   Without a cap, broad queries like 'knowledge' promote 525+ nodes
   simultaneously. WM is now bounded to 24 nodes. Algorithm: qsort on
   promoted weights, keep top-24 by cutoff, evict the rest. Global pass
   enforces cap across nodes that were promoted in prior calls and persist
   via working_memory_weight. Validated: WM promoted goes 525→24.
   Cognitive basis: Cowan (2001) WM ~4 chunks; 24 gives richer multi-topic
   context while preventing flooding.

3. ISE exclusion from WM + /api/neuron/state-events route
   InternalStateEvent nodes were reaching WM via breakthrough (5 suppression
   cycles) because their content (curiosity seed JSON with 'knowledge',
   'memory', etc.) triggered lexical seeding. ISEs are observability-only
   and must never surface in context. Fix: guard in Pass 2 clears
   suppression_count and skips to wm_weights[i]=0.0.
   Also added POST /api/neuron/state-events route to server.el (auth-exempt,
   internal endpoint). The main soul daemon posts ISEs here but the route
   was missing — all ise_post() calls were silently returning 'not found'.

Research: SYNAPSE (arXiv 2601.02744) validates spreading factor 0.8 (our
0.7), top-M WM cap design, and cosine similarity seeding. Next priority:
implement cosine similarity initial seeding from the other branch.
prevent engram corruption: add UTF-8 validation in engram_node_full e7c2dc7734 
Reject content containing invalid UTF-8 bytes before persisting — silently
writing invalid UTF-8 garbles JSON snapshots and corrupts node reads.
feat(engram): /api/sync route for soul daemon periodic pull; update ELP type headers 192241c7c1
fix(releases/v1.0.0): println stdout flush for launchd; Knowledge node activation threshold 3da9181deb
feat(runtime): native platform backends (AppKit, UIKit, Android, GTK4, SDL2, LVGL, Win32) 6271cb42b2 
Add seven platform bridge implementations and the shared native target header:
el_native_target.h, el_appkit.m, el_uikit.m, el_android.c, el_gtk4.c,
el_sdl2.c, el_lvgl.c, el_win32.c, el_runtime_win32.c. Each bridge implements
the 33 __widget_* C builtins declared in el_native_target.h for its platform
toolkit. el_runtime_win32.c provides a POSIX-free runtime stub for cross-compiled
Win32 targets.
feat(runtime): Java platform bridge and platform detection tooling edff25180e 
ElBridge.java: Android Java companion to el_android.c — all public methods are
static, dispatches View mutations to the UI thread via runOnUiThread/CountDownLatch,
and exposes native callbacks (nativeOnClick, nativeOnChange, nativeOnSubmit).

PLATFORM_BRIDGE_SPEC.md: authoritative spec for implementing new platform bridges
(slot table contract, required __* functions, callback dispatch pattern).

detect-platforms: shell script that probes for available bridge toolchains and
prints what can be built on the current machine.

new-platform: scaffold generator that creates a new el_<name>.c with all 33
required stubs wired up.
feat(ui): native vessel, HTML vessel update, native hello examples, profile card, UI tools
El SDK Release / build-and-release (pull_request) Failing after 17s
Details
58753a88d7 
el-native vessel: El-level wrappers around __widget_* C builtins, exposing
vstack, label, button, text_field, etc. as clean El functions for application code.

el-html/main.elh: updated extern declarations for the HTML vessel's codegen API.

native-hello: cross-platform desktop example (AppKit/GTK4/Win32/SDL2) with
build scripts, Dockerfiles for Linux/Pi, and Win32 cross-compile support.

native-hello-android: Gradle project with ElBridge integration and build script.

native-hello-ios: Xcode project for the iOS UIKit target.

profile-card: manifest.el for a styling/layout/i18n example app that exercises
el-style, el-layout, el-i18n, el-config, and el-secrets vessels.

ui/tools/native-codegen: Python codegen pass (el_ui_native_codegen.py) that
lowers el-ui component DSL to el-native vessel calls, plus build script and
test fixtures.
will.anderson closed this pull request 2026-07-01 16:38:35 +00:00

Pull request closed

This pull request cannot be reopened because the branch was deleted.
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
BETA-CRITICAL
On the beta critical path — must resolve before ship
 HELD
Blocked pending an external decision (counsel / dual sign-off)
 ORTHOGONAL
Parallel workstream (testing/hardening) — safe to assign off critical path
 POST-BETA
Important but not blocking the beta ship
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
andre.almeida (Andre Almeida) tim.lingo (Tim Lingo) will.anderson (Will Anderson)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: neuron-technologies/el#61
Delete Branch "fix/windows-rusage-guard"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?