Addind readme.md file to start documenting the repo

README.md

@@ -0,0 +1,154 @@
+# El
+
+**A self-hosting, statically-typed language that compiles to C — built around a graph-native runtime instead of a database driver.**
+
+El is the execution substrate for the Neuron agent runtime, the DHARMA network, and the Engram knowledge graph. This repository is the monorepo for the whole stack: the language itself, the graph memory engine it's built to talk to natively, and the tools (package manager, IDE, UI framework, diagramming) built on top of it.
+
+---
+
+## Why El exists
+
+Every other language treats persistent, associative state as something you reach for through a driver — a SQL client, an ORM, a Redis library bolted on from outside. El inverts that: graph operations (`engram_*`) are runtime primitives, on the same footing as string or list operations. There is no separate database driver because the database is not separate.
+
+El has four defining properties:
+
+1. **Self-hosting compiler.** The compiler (`lexer.el`, `parser.el`, `codegen.el`, `compiler.el`) is written in El. It compiles El source to C, which `cc` compiles against a fixed runtime into a native binary. A Rust genesis compiler bootstrapped the first iteration; the self-hosted binary at `lang/dist/platform/elc` has been the canonical compiler ever since — every binary in `dist/platform/` was produced by an earlier version of itself compiling `el-compiler/src/`. The chain is auditable: source is the ground truth, not the binary. See [lang/BOOTSTRAP.md](lang/BOOTSTRAP.md) for the full recovery path if that binary is ever lost.
+2. **C compilation target.** Every compiled program is plain C11. Every El value is `el_val_t` (`int64_t`); strings are heap pointers cast through it. Functions become C functions; top-level statements become `main()`.
+3. **Graph-native runtime.** The runtime provides first-class graph operations over an in-process Engram store — no separate DB driver, no ORM.
+4. **DHARMA-aware identity.** A `cgi` block declares a program's DHARMA identity at compile time. The runtime resolves identity before user code runs, so `dharma_*` calls have a stable principal and channel surface throughout.
+
+---
+
+## Architecture map
+
+```
+                     ┌─────────────┐
+                     │    lang     │   El compiler + C runtime
+                     │ (El itself) │   everything below is written in it,
+                     └──────┬──────┘   or compiles down through it
+                            │
+              ┌─────────────┼─────────────┐
+              │             │             │
+       ┌──────▼─────┐ ┌─────▼─────┐ ┌─────▼─────┐
+       │   engram   │ │    epm    │ │    ide    │
+       │ graph/mem  │ │  package  │ │  editor + │
+       │  substrate │ │  manager  │ │    LSP    │
+       └──────┬─────┘ └───────────┘ └───────────┘
+              │
+      ┌───────┼────────────────┬─────────────────────┐
+      │       │                │                     │
+┌─────▼───┐ ┌─▼──────────┐  ┌──▼──────────┐    ┌─────▼──────┐
+│   elp   │ │ ql         │  │  ui         │    │   arbor    │
+│  NLG /  │ │engram-el.  │  |spreading-   │    |arbor       │
+│ 31 langs│ │studio+tests│  |activation UI│    |diagram lang│
+└─────────┘ └────────────┘  └─────────────┘    └────────────┘
+```
+
+`lang` is the foundation — the compiler and C runtime everything else builds on. `engram` is the graph-native memory/state engine that gives El its identity (property 3 above). Everything else is either a tool for working with El (`epm`, `ide`) or a system built on top of Engram's graph model (`elp`, `ql`, `ui`, `arbor`).
+
+---
+
+## Repository layout
+
+### [lang/](lang/) — the El language
+
+The compiler and runtime. Self-hosting: `elc-cli.el` → `compiler.el` → `lexer.el` / `parser.el` / `codegen.el` / `codegen-js.el`, textually inlined and compiled in one pass. Compiles to C11 and links against `el-compiler/runtime/el_seed.c`, a hand-maintained OS-boundary layer (libcurl HTTP, pthreads, filesystem, arena allocation) — everything else in the runtime is native El (`runtime/*.el`).
+
+Two layers to know: **El programs** (`.el` files — where nearly all work belongs) and **the C seed** (`el_seed.c` — edit only for genuine OS-level access; never re-implement what El can already express).
+
+Current status (single source of truth: [lang/spec/language.md](lang/spec/language.md)): lexer/parser/codegen and the C runtime's core (I/O, strings, math, lists, maps, filesystem, args) are implemented. In flight: `%` operator, match-statement codegen, `?` nil-propagation, `cgi` block parsing + DHARMA identity resolution, VBD role enforcement (`@manager`/`@engine`/`@accessor`), the real `engram_*` and `dharma_*` runtimes (currently stubs), and libcurl-backed `http_get`/`http_post`/`http_serve`. Bitwise operators, `??`, and `as` casts are explicitly **not** in this language.
+
+Key docs: [AGENTS.md](lang/AGENTS.md) (agent-facing orientation), [BOOTSTRAP.md](lang/BOOTSTRAP.md) (compiler recovery from scratch), [spec/language.md](lang/spec/language.md), [spec/codegen-js.md](lang/spec/codegen-js.md).
+
+### [engram/](engram/) — graph intelligence substrate
+
+**A local-first memory substrate for accumulating intelligence**, and the reason El's runtime doesn't need a database driver. Rust core (`engram-core`, `engram-ffi`) exposed to El and other languages (Kotlin, TypeScript/WASM, Go bindings).
+
+The model: retrieval is **spreading activation**, not query. You name seed nodes and a query embedding; activation propagates outward through weighted edges, attenuating multiplicatively per hop (`strength = parent_strength × edge_weight × target_salience × cosine_sim`), gets pruned below a threshold, and the top-N nodes by activation strength come back. Storage and retrieval are the same structure — the way long-term potentiation works in biological memory, not the way a relational or vector database works.
+
+Nodes live in four tiers (Working / Episodic / Semantic / Procedural, mirroring prefrontal / hippocampal / neocortical / cerebellar memory) and migrate between them based on **salience decay** — `importance × recency-decay × log(activation_count)`. Forgetting is adaptive pruning, not a bug: unreinforced memories stop competing for attention without being deleted.
+
+Backed by `sled` (embedded, local-first, no daemon) with flat cosine scan for vector search — deliberately simple until scale demands an HNSW layer. Full API and design rationale in [engram/README.md](engram/README.md).
+
+### [elp/](elp/) — Engram Language Protocol
+
+Bidirectional engine mapping between Engram semantic forms and natural-language surface text, across **31 languages** — from Spanish and Japanese through historical/liturgical languages (Old Norse, Sanskrit, Sumerian, Coptic, Akkadian, Ge'ez). Compilation order runs `language-profile` + `vocabulary` → per-language `morphology-*` → `grammar` → `realizer` → `semantics` → `elp`. This is what lets an Engram graph node round-trip to and from readable text in any of those languages.
+
+### [epm/](epm/) — El Package Manager
+
+Manages **vessels** (El's package unit): publish, install, resolve dependencies. Vessels are stored in Engram as graph nodes, not files in a registry index — `epm` reads the local `manifest.el`, talks to Engram over HTTP, and writes resolved vessels to `.epm/vessels/`. Source: `registry.el`, `install.el`, `update.el`, `manifest.el`.
+
+### [ide/](ide/) — El IDE
+
+Three vessels: **el-ide-server** (HTTP backend — file ops, build/run, LSP bridge, plugin host, settings), **el-lsp** (the language server — completion, hover, diagnostics, outline, format, type graph), and **el-plugin-host** (first-party plugin lifecycle: install/remove/enable/disable). `ide/projects/` and `ide/examples/` hold sample projects, including the canonical `hello-friends` first-program walkthrough.
+
+### [ql/](ql/) — engram-el
+
+The El-native integration layer for a *live* Engram server — not a library (no importable modules, no build artifact), a set of standalone `.el` programs run directly via `el run-file`. Three components: **Studio** (`studio/studio.el`, a full terminal graph explorer), a **Hebbian field-model** proof of concept, and El builtin / LLM-builtin smoke test suites. This is the reference for correct patterns when an El program uses Engram as its substrate. Spec: [ql/spec/elql.md](ql/spec/elql.md).
+
+### [ui/](ui/) — el-ui
+
+A frontend framework where **component state is an Engram graph and reactivity is spreading activation** — not virtual-DOM diffing (React), Proxy-based dependency tracking (Vue), or compile-time analysis (Svelte). Re-renders are activated and propagated the same way associative memory retrieval works in `engram/`.
+
+~15 vessels covering the full frontend surface: `el-platform` (env/fs/network/clock abstraction), `el-config`, `el-html` (SSR emit primitives), `el-layout`, `el-style` (design tokens/themes), `el-i18n`, `el-auth` / `el-identity` (JWT, sessions, OAuth PKCE — Engram-native), `el-services` (REST/gRPC/WebSocket bindings), `el-aop` (`@authenticate`/`@authorize`/`@cache`/`@rate_limit` decorators), `el-secrets`, `el-graph` (graph rendering/editor), `el-publish` (App Store / Play Store automation), and `el-ui-compiler` (El→JS component compiler; currently a stub pending a JS backend in `elc`). Spec: [ui/spec/framework.md](ui/spec/framework.md).
+
+### [arbor/](arbor/) — diagram language
+
+A `.arbor` diagram language and toolchain: `arbor-core` (NodeId/shape/edge-kind types), `arbor-parse` (recursive-descent parser), `arbor-diagram` (IR + Mermaid serializer + architecture-diagram builders), `arbor-layout` (hierarchical layout — rank assignment, positioning, group bounds), `arbor-render` (SVG renderer), `arbor-cli`. (The architecture map above is the kind of diagram this is for.)
+
+---
+
+## Getting started
+
+Install the El SDK from the latest release:
+
+```bash
+bash lang/install.sh
+# EL_VERSION=v1.0.0   bash lang/install.sh   # pin a specific release tag
+# EL_PREFIX=/opt/el   bash lang/install.sh   # custom install prefix
+```
+
+Or build the compiler from source and verify the self-hosting chain:
+
+```bash
+cd lang
+./dist/platform/elc elc-cli.el > elc-new.c
+cc -std=c11 -I el-compiler/runtime -lcurl -lpthread \
+   -o dist/platform/elc-new \
+   elc-new.c el-compiler/runtime/el_seed.c
+
+# Confirm the new binary reproduces itself exactly
+./dist/platform/elc-new elc-cli.el > elc-verify.c
+diff elc-new.c elc-verify.c   # should be identical
+
+mv dist/platform/elc-new dist/platform/elc
+```
+
+Run your first program:
+
+```bash
+./lang/dist/platform/elc lang/examples/hello.el > hello.c
+cc -std=c11 -I lang/el-compiler/runtime -lcurl -lpthread \
+   -o hello hello.c lang/el-compiler/runtime/el_seed.c
+./hello
+```
+
+More examples in [lang/examples/](lang/examples/), including a full starter project at `lang/examples/hello-project/`.
+
+If the compiler binary is ever lost or corrupted, [lang/BOOTSTRAP.md](lang/BOOTSTRAP.md) is the authoritative recovery path.
+
+---
+
+## Development workflow
+
+Branching follows `dev → stage → main`: work lands on `dev`, promotes to `stage` for integration testing, and is promoted to `main` for release (visible directly in the git history of this repo). CI is defined per-subproject under `.gitea/workflows/` — `lang`/`epm`/`ide` share the root pipeline; `engram` and `ql` carry their own (`ci-dev`, `ci-stage`, and a release workflow each).
+
+- Language/runtime specs live at `*/spec/*.md` (`lang/spec/`, `ql/spec/`, `ui/spec/`) and are the single source of truth for implemented-vs-planned status — code and docs are expected to agree with the spec's status markers, not the other way around.
+- Agent-facing orientation guides live at `*/AGENTS.md` (currently `lang/AGENTS.md`); more subprojects may grow their own as they need agent-specific conventions documented.
+- Tagged releases live under `lang/releases/`, each with its own `RELEASE.md`.
+
+---
+
+## Status
+
+This is an actively developed, internal monorepo — not yet published under an open license. Treat everything here as proprietary to Neuron Technologies unless told otherwise.

lang/el-compiler/src/codegen.el

@@ -3627,24 +3627,6 @@ fn codegen_streaming(tokens: [Any], sigs: [Map<String, Any>], source: String) ->
     let pos: Int = 0
     let el_main_body: [Map<String, Any>] = native_list_empty()
     let toplevel_exec_stmts: [Map<String, Any>] = native_list_empty()
-    // CGI IDENTITY CAPTURE (2026-08-09). A cgi block is a top-level DECLARATION, so
-    // the classifier below correctly excludes it from toplevel_exec_stmts and calls
-    // el_release on it. The identity emission further down then searched
-    // toplevel_exec_stmts for it — a list that structurally can never contain it —
-    // found nothing, and emitted nothing, silently. Measured: that search sees only
-    // [Let, Expr] for a program whose first statement is a cgi block.
-    // Fix: copy the values out BEFORE the release (strings, so no dangling reference)
-    // and emit from these. No search, so the failure mode is removed rather than moved.
-    let cgi_have: Bool = false
-    let cgi_name_v: String = ""
-    let cgi_did_v: String = ""
-    let cgi_prin_v: String = ""
-    let cgi_net_v: String = ""
-    let cgi_eng_v: String = ""
-    let cgi_has_did: Bool = false
-    let cgi_has_prin: Bool = false
-    let cgi_has_net: Bool = false
-    let cgi_has_eng: Bool = false
     let has_toplevel_exec: Bool = false
 
     let stream_running: Bool = true
@@ -3755,20 +3737,6 @@ fn codegen_streaming(tokens: [Any], sigs: [Map<String, Any>], source: String) ->
                         if is_top_level_decl(stmt) {
                             // Import, TypeDef, EnumDef, CgiBlock, ServiceBlock, ExternFn
                             // These are no-ops in codegen (forward decls already emitted)
-                            // — except a CgiBlock, whose declared identity must survive
-                            // this release to be emitted as a compiled constant.
-                            if str_eq(sk, "CgiBlock") {
-                                let cgi_have = true
-                                let cgi_name_v = stmt["name"]
-                                let cgi_did_v = stmt["dharma_id"]
-                                let cgi_prin_v = stmt["principal"]
-                                let cgi_net_v = stmt["network"]
-                                let cgi_eng_v = stmt["engram"]
-                                let cgi_has_did = stmt["has_dharma_id"]
-                                let cgi_has_prin = stmt["has_principal"]
-                                let cgi_has_net = stmt["has_network"]
-                                let cgi_has_eng = stmt["has_engram"]
-                            }
                             el_release(stmt)
                         } else {
                             if str_eq(sk, "Let") {
@@ -3847,17 +3815,33 @@ fn codegen_streaming(tokens: [Any], sigs: [Map<String, Any>], source: String) ->
         let sig2 = native_list_get(sigs, si2)
         let sk3: String = sig2["kind"]
         if str_eq(sk3, "cgi_block") {
-            // Emit from the values captured before the declaration was released.
+            // We need the full cgi_block data — it was parsed by scan_fn_sigs
-            // The previous implementation searched toplevel_exec_stmts, which by
+            // but scan only stored the name. For cgi_init we need dharma_id etc.
-            // construction never contains a declaration — so it emitted nothing and
+            // Since cgi blocks are rare and small, they end up in toplevel_exec_stmts.
-            // said nothing. See the capture block near toplevel_exec_stmts init.
+            // Find the CgiBlock in toplevel_exec_stmts.
-            if cgi_have {
+            let tes_n: Int = native_list_len(toplevel_exec_stmts)
-                let arg_name2: String = "EL_STR(" + c_str_lit(cgi_name_v) + ")"
+            let tes_i: Int = 0
-                let arg_did2: String = cgi_arg(cgi_did_v, cgi_has_did)
+            while tes_i < tes_n {
-                let arg_prin2: String = cgi_arg(cgi_prin_v, cgi_has_prin)
+                let tes = native_list_get(toplevel_exec_stmts, tes_i)
-                let arg_net2: String = cgi_arg(cgi_net_v, cgi_has_net)
+                let tes_k: String = tes["stmt"]
-                let arg_eng2: String = cgi_arg(cgi_eng_v, cgi_has_eng)
+                if str_eq(tes_k, "CgiBlock") {
-                emit_line("  el_cgi_init(" + arg_name2 + ", " + arg_did2 + ", " + arg_prin2 + ", " + arg_net2 + ", " + arg_eng2 + ");")
+                    let cname2: String = tes["name"]
+                    let cdid2: String = tes["dharma_id"]
+                    let cprin2: String = tes["principal"]
+                    let cnet2: String = tes["network"]
+                    let ceng2: String = tes["engram"]
+                    let has_did2: Bool = tes["has_dharma_id"]
+                    let has_prin2: Bool = tes["has_principal"]
+                    let has_net2: Bool = tes["has_network"]
+                    let has_eng2: Bool = tes["has_engram"]
+                    let arg_name2: String = "EL_STR(" + c_str_lit(cname2) + ")"
+                    let arg_did2: String = cgi_arg(cdid2, has_did2)
+                    let arg_prin2: String = cgi_arg(cprin2, has_prin2)
+                    let arg_net2: String = cgi_arg(cnet2, has_net2)
+                    let arg_eng2: String = cgi_arg(ceng2, has_eng2)
+                    emit_line("  el_cgi_init(" + arg_name2 + ", " + arg_did2 + ", " + arg_prin2 + ", " + arg_net2 + ", " + arg_eng2 + ");")
+                }
+                let tes_i = tes_i + 1
             }
         }
         let si2 = si2 + 1