el/lang/AGENTS.md

# El Language — Agent Guide

El is a self-hosting, statically-typed language that compiles to C. This file orients agents that work on El itself or on programs written in El.

---

## What El Is

El compiles `.el` source → C → native binary. Every El value is `el_val_t` (int64_t). Strings are heap pointers cast through int64_t. The compiler is written in El (self-hosting).

**The compiler pipeline:**
```
elc-cli.el
  └─ imports: compiler.el
       └─ imports: lexer.el, parser.el, codegen.el, codegen-js.el
```

The canonical compiler binary is `dist/platform/elc`. It was produced by running an earlier version of itself on `elc-cli.el`.

---

## The Two Layers — Know Which One You're In

### Layer 1: El programs (`.el` files)

This is where almost all work belongs. El programs are source files that get compiled by `elc`. New library functions, application logic, and language-level utilities all go here as `.el` files.

**Do not add C code when El can express it.** If functionality can be built from existing El primitives (string ops, `exec`, `fs_read/write`, `http_post`, etc.), write it in El.

### Layer 2: The C seed (`el-compiler/runtime/el_seed.c`)

This is the self-contained C OS-boundary layer. It provides the `__`-prefixed primitives that compiled El programs call: libcurl HTTP, pthreads, filesystem I/O, arena allocation, etc. It is **not generated** — it is maintained by hand.

The runtime is native El (`runtime/*.el`) over a C OS-boundary. **Status (verified 2026-08-15):** the migration to a seed-only boundary is *in progress, not done*. Two files exist:
- `el-compiler/runtime/el_runtime.c` (~516 KB) — **LIVE**. Holds the engram store (`EngramStore engram_global`) plus the `http_*`/`json_*`/`state_*`/`engram_*` impls. It is the authoritative single-file link target for the compiler, and `tools/install.sh` compiles it into `libel.a`. This is where a new C builtin's *implementation* must currently live to be linkable.
- `el-compiler/runtime/el_seed.c` — the intended hand-maintained `__`-prefixed seed (thin wrappers over the above). It is compiled alongside `el_runtime.c` by `tools/install.sh`, but does **not** compile standalone yet (see the build-path caveat under "Rebuilding the Compiler").
- `el-compiler/runtime/legacy/el_runtime.c` (~419 KB) — **DEAD**. Archived duplicate; no build script references it.

**Only edit these when you genuinely need OS-level access** (raw sockets, GPU calls, new libcurl features, a new engram store op). For everything else, write El.

When you add a C builtin (verbatim-emit recipe — the El name is emitted as the exact C symbol; `builtin_arity` is an arity guard only, not a dispatch table):
1. Implement the C function in `el_runtime.c` (and declare it in `el_runtime.h`).
2. Add a `__`-prefixed thin wrapper in `el_seed.c` and declare it in `el_seed.h`.
3. Add the name to `builtin_arity` in `el-compiler/src/codegen.el` — add **both** the plain and `__`-prefixed spellings.
4. Rebuild the elc binary (see below) and confirm the self-host fixpoint is byte-identical.

Worked example: the `engram_assert_json` (op_assert seam) and `engram_node_full_in`/`engram_connect_in` (purview write-side) primitives added 2026-08-15 follow exactly this recipe.

---

## Rebuilding the Compiler

After changing any `.el` source in `el-compiler/src/` (run from the `lang/` dir):

```bash
# 1. Stage2: current elc compiles the (modified) compiler to C
./dist/platform/elc elc-cli.el > elc-new.c
# 2. Build the new compiler. The C link target is el_runtime.c — it holds the
#    engram store + http/json/state impls the compiler output calls. el_runtime.c
#    self-hosts elc on its own; el_seed.c is the (aspirational) seed layer and does
#    NOT compile standalone under clang (missing prototypes for the el_runtime.c
#    symbols it wraps — see caveat below), so link el_runtime.c here.
cc -std=c11 -I el-compiler/runtime -lcurl -lpthread \
   -o dist/platform/elc-new \
   elc-new.c el-compiler/runtime/el_runtime.c
# 3. Verify self-hosting FIXPOINT (stage3 == stage2 output, byte-identical):
./dist/platform/elc-new elc-cli.el > elc-verify.c
diff elc-new.c elc-verify.c   # must be identical
mv dist/platform/elc-new dist/platform/elc
```

> **Build-path caveat (verified 2026-08-15).** `el_seed.c` is the intended hand-maintained OS-boundary seed, but it does **not** compile standalone under modern clang: it wraps ~16 unprefixed `el_runtime.c` symbols (`http_serve`, `json_*`, `state_*`, `http_response`) without prototypes, and clang treats implicit declarations as errors (C99+). The productionised install (`tools/install.sh`) builds `libel.a` from **both** `el_seed.o` + `el_runtime.o` together, which is why linking succeeds there. To make `el_seed.c` build on its own, add prototypes for those symbols (or `#include "el_runtime.h"`, reconciling the `__http_serve` return-type mismatch first). Until then, `el_runtime.c` is the authoritative single-file link target for the compiler.

After changing `el_seed.c` only (no El source changes), rebuild downstream programs but do NOT need to rebuild the compiler binary itself — the seed is linked at the application level, not the compiler level.

---

## How El Programs Are Built

Each El application has a `build.sh` that:
1. Concatenates all `.el` source files (stripping `import` lines)
2. Runs `elc` to produce a `.c` file
3. Runs `cc` linking against `el_seed.c`

Example (cgi-studio daemon):
```bash
cd products/cgi-studio/el-daemon
./build.sh
```

When you add a new `.el` file to an application, add it to that application's `build.sh` concat list.

---

## Parallelism in El

El is single-threaded at the application level. Parallelism is achieved through subprocess fan-out:

```el
// Pattern: write payloads to temp files, exec bash script with & and wait,
// read results back from temp files.
fn http_post_parallel(urls: [String], bodies: [String]) -> [String] {
    // ... bash fan-out via exec() ...
}
```

Use `exec()` (blocking) or `exec_bg()` (fire-and-forget) with shell scripts to run concurrent work. There is no goroutine or async/await — parallelism goes through the OS process layer.

---

## Key Files

| Path | What it is |
|------|-----------|
| `dist/platform/elc` | Canonical compiler binary (arm64 Mac) |
| `el-compiler/src/codegen.el` | Code generator — builtin arity table lives here |
| `el-compiler/src/lexer.el` | Lexer |
| `el-compiler/src/parser.el` | Parser |
| `el-compiler/runtime/el_seed.c` | Self-contained C OS-boundary layer (replaces el_runtime.c) |
| `el-compiler/runtime/el_seed.h` | Seed header (C function declarations) |
| `spec/language.md` | Language specification |
| `BOOTSTRAP.md` | How to recover the compiler from scratch |
| `elc-cli.el` | Compiler entry point |
| `elc-combined.el` | Pre-merged single-file compiler (used during early bootstrap) |

---

## HTTP Timeout

The El HTTP client (libcurl) defaults to **60 seconds**. Override per-process via `EL_HTTP_TIMEOUT_MS` env var. Set it before spawning any subprocess that makes long API calls:

```el
exec("EL_HTTP_TIMEOUT_MS=300000 " + SOME_BIN + " " + args + " 2>&1")
```

---

## Rules

- New library functions → write in El
- New OS/hardware primitives → write in C and register in `codegen.el` arity table
- Never edit `dist/platform/elc` directly — always rebuild from source
- Never modify `el_seed.c` to add functionality that El can express