neuron-technologies/el

Fork 0

Files

T

Will Anderson 1ae68962cf restructure: move el compiler content into lang/

2026-05-05 01:38:51 -05:00

38 KiB

Raw Permalink Blame History

El Language Specification

Version 1.2.0 — April 30, 2026

Overview

El is a statically-typed, compiled programming language that serves as the execution substrate for the Neuron agent runtime, the DHARMA network, and the Engram knowledge graph. El compiles to C and links against a fixed runtime, producing native binaries.

El has four defining properties:

Self-hosting compiler. The compiler (lexer.el, parser.el, codegen.el, compiler.el) is written in El. It compiles El source to C, which is then compiled by cc against el_runtime.c to produce a native binary. A Rust genesis compiler bootstraps the first iteration; the self-hosted binary at dist/platform/elc is the canonical compiler thereafter.
C compilation target. Every compiled program is plain C11. Every El value is el_val_t (int64_t). Strings are pointers cast through int64_t. Functions become C functions; top-level statements become main().
Graph-native runtime. The runtime provides first-class graph operations (engram_*) over an in-process Engram store. CGI programs use these primitives directly; there is no separate database driver.
DHARMA-aware identity. The cgi block declares a program's DHARMA identity at compile time. The runtime resolves identity before any user code runs, so subsequent dharma_* calls have a stable principal and channel surface.

Implementation Status

This section is the single source of truth for what works and what is planned. Each subsequent section repeats the relevant marker. If a feature here is marked "planned," it appears in the spec because it is in the design, but the current self-hosting toolchain does not yet emit code for it.

Implemented (today)

Lexer: keywords, identifiers, integer/float/string/bool literals, operators below.
Parser: let, return, fn, type, enum, import, from … import, while, for, if/else if/else, match, @decorator, array/map literals, all listed operators, function calls, field access, index access, unary !/-, postfix ?.
Codegen: function definitions, top-level main(), all expression forms above, control flow, decorator-as-AST-attachment.
C runtime: I/O, string operations, integer math, lists, maps, filesystem, command-line args, basic json_get substring lookup.

Planned (in flight)

% (modulo) operator. Currently not lexed. Adding to lexer + parser + codegen.
Match codegen. Currently parsed; codegen does not emit. Adding ({ ... }) statement-expression emission.
? propagation. Currently no-op. Adding nil-propagation semantics.
cgi block parsing. Currently lexed (cgi is a keyword) but not parsed as a statement. Adding parse_cgi_block and codegen of el_cgi_init at the head of main().
VBD role enforcement. @manager/@engine/@accessor are accepted as decorators but not enforced. Adding compile-time check that dharma_emit/dharma_field only appear inside @manager functions.
vessel keyword. Replaces package in manifests. Adding to lexer.
Real engram_* runtime. Currently stub. Adding in-process graph store with spreading activation, Hebbian strengthening, and disk persistence — see Section 16.4.
Real dharma_* runtime. Currently stub. Adding network transport, channel registry, identity resolution.
Real http_get/http_post/http_serve. Currently empty stubs. Adding libcurl-backed client and a thread-pool server.
JSON, time, UUID, state, env, additional string/list/math builtins. See Section 12 for the canonical list.

Not in this language

Bitwise operators (&, |, ^, <<, >>). Single & is silently consumed by the lexer; single | is lexed as Pipe but unused. None are parsed as binary operators. Removed from the spec entirely.
?? null-coalescing. Was reserved; not lexed. Removed.
as casts. as is a keyword but no parse form. Removed from the spec until shipped.
Floating-point arithmetic distinct from Int. All values are int64_t at the C level. Float literals are accepted but stored as bit-cast doubles only when the runtime is float-aware (currently only via float_to_str and friends in the planned runtime extension).

1. Lexical Structure

1.1 Source Encoding

El source is UTF-8, file extension .el.

1.2 Comments

// Single-line comment — extends to end of line

Block comments are not supported.

1.3 Whitespace

Spaces, tabs, newlines (\n), and carriage returns (\r) are whitespace. Whitespace is not significant except as a token separator.

1.4 Identifiers

identifier = (alpha | '_') (alnum | '_')*

Identifiers are case-sensitive. Names beginning with __ are reserved for compiler-generated symbols.

1.5 Keywords

The following words are reserved and cannot be used as identifiers. Each row notes whether the parser currently consumes the keyword as a structural form.

Keyword	Parsed today	Notes
`let`	yes	Variable binding
`fn`	yes	Function definition
`type`	yes	Struct definition
`enum`	yes	Enum definition
`match`	yes	Pattern match expression
`return`	yes	Function return
`if` / `else`	yes	Conditional
`for` / `in`	yes	Iteration
`while`	yes	Loop
`import` / `from` / `as`	yes	Module import
`true` / `false`	yes	Bool literals
`cgi`	planned	Top-level CGI declaration block
`manager` / `engine` / `accessor`	as decorators	VBD role marker on `fn` (enforcement planned)
`vessel`	planned	Manifest declaration (replaces `package`)
`activate` / `where`	planned	Spreading-activation construct
`sealed`	planned	Capability scope block
`with`	reserved	No parse form yet
`test` / `seed` / `assert`	reserved	Testing primitives, no parse form
`protocol` / `impl`	reserved	Trait-like, no parse form
`retry` / `times` / `fallback` / `reason`	reserved	Resilience primitives, no parse form
`parallel` / `trace`	reserved	Concurrency, no parse form
`requires` / `deploy` / `to` / `via` / `target`	reserved	Deployment surface, no parse form

1.6 Token Types

Token	Pattern	Notes
`Int`	`[0-9]+`
`Float`	`[0-9]+ '.' [0-9]+`
`Str`	`"…"` with `\"`, `\n`, `\t`, `\\` escapes
`Bool`	`true` or `false`
`Ident`	identifier (not a keyword)
keyword tokens	one per keyword above	e.g. `Let`, `Fn`, `If`
`Eq`	`=`
`EqEq`	`==`
`NotEq`	`!=`
`Not`	`!`
`Lt` `LtEq` `Gt` `GtEq`	`<` `<=` `>` `>=`
`And`	`&&`	Single `&` is consumed and discarded
`Or`	`\|\|`
`Pipe`	`\|`	Lexed; not used by parser
`PipeOp`	`\|>`	Lexed; not used by parser
`Plus` `Minus` `Star` `Slash`	`+` `-` `*` `/`
`Arrow`	`->`
`FatArrow`	`=>`
`Colon` `ColonColon`	`:` `::`
`LParen` `RParen` `LBrace` `RBrace` `LBracket` `RBracket`	`(` `)` `{` `}` `[` `]`
`Comma` `Dot` `Semicolon`	`,` `.` `;`
`At` `QuestionMark`	`@` `?`
`Eof`	end-of-input sentinel

1.7 String Escapes

Sequence	Character
`\n`	Newline (U+000A)
`\t`	Tab (U+0009)
`\"`	Double quote
`\\`	Backslash
(other)	Character as-is

2. Type System

2.1 Primitive Types

Type	Description	Example literals
`Int`	64-bit signed integer	`42`, `-7`
`Float`	64-bit double (planned float arithmetic; today stored as int64)	`3.14`, `0.5`
`String`	UTF-8 string	`"hello"`
`Bool`	Boolean	`true`, `false`
`Void`	No value	—
`Any`	Dynamically-typed value	(for generic containers)

2.2 Composite Types

Type form	Description
`[T]`	Array of `T`
`T?`	Optional `T` (planned propagation; today the `?` postfix is no-op)
`Map<K, V>`	Key-value map
Named type	User-defined struct or enum

2.3 Type Annotations

Type annotations appear in let bindings and function signatures. The current compiler parses and skips annotations — no type checking is performed at compile time. They serve as documentation. A type checker is planned.

let x: Int = 42
fn greet(name: String) -> String { … }

2.4 Optional Types

T? is accepted in type position. The postfix ? on an expression produces a Try AST node. Today the codegen passes the inner expression through transparently; nil propagation is planned.

3. Variables and Bindings

3.1 Let Bindings

let name: Type = expression
let name = expression

All bindings are block-scoped. El allows re-binding the same name in the same scope; the codegen emits a plain assignment instead of a redeclaration:

let count = 0
let count = count + 1   // plain `count = count + 1;` in C

3.2 Scope

Bindings are valid from the point of declaration to the end of the enclosing block. Function bodies, if arms, for bodies, while bodies, and explicit { … } blocks each introduce a new C scope.

4. Functions

4.1 Definition

fn name(p1: T1, p2: T2) -> R {
    // body
    return expr
}

Compiles to el_val_t name(el_val_t p1, el_val_t p2). The return type is parsed and skipped. A return 0; is appended automatically at the end of every function body.

4.2 Forward Declarations

The codegen emits forward declarations for all top-level fn definitions before any function body. Mutual recursion within a file is supported.

4.3 Return

return expression
return            // bare; compiles to `return 0;`

A bare return followed by } or end-of-file compiles to return 0;.

4.4 Calling Convention

All arguments are el_val_t, passed in declaration order.

Method-style calls

obj.method(args…) compiles to method(obj, args…). The runtime exports short-name aliases for common operations:

El source	Emitted C	Runtime alias
`list.append(x)`	`append(list, x)`	`el_list_append`
`list.len()`	`len(list)`	`el_list_len`
`list.get(i)`	`get(list, i)`	`el_list_get`
`map.map_get(k)`	`map_get(map, k)`	`el_map_get`
`map.map_set(k, v)`	`map_set(map, k, v)`	`el_map_set`

5. Control Flow

5.1 If / Else

if cond { … } else if cond2 { … } else { … }

if may appear as an expression (RHS of let, etc.); when used in expression position the codegen emits a ternary stub. if as a statement emits standard if (…) { … } else { … } C code.

5.2 While

while cond { … }

Exits when cond is 0.

5.3 For

for item in list { … }

Compiles to a tracked C for loop:

{
  el_val_t _el_lst = <list>;
  el_val_t _el_len = el_list_len(_el_lst);
  for (el_val_t _el_i = 0; _el_i < _el_len; _el_i++) {
    el_val_t item = el_list_get(_el_lst, _el_i);
    // body
  }
}

5.4 Match

match expression {
    Pattern1 => result_expr
    Pattern2 => result_expr
    _        => default_expr
}

Pattern forms today:

Pattern	Meaning
`_`	Wildcard — always matches
`name`	Binding — captures subject as `name`
`42`	Integer literal
`"str"`	String literal
`true` / `false`	Boolean literal

Enum-variant patterns (EnumName::Variant) are reserved but not yet parsed.

Codegen status: Parsed today; codegen emits a ({ … }) statement-expression in the planned runtime extension. Until then, match is recognized but produces no emitted code.

6. Data Structures

6.1 Struct Types

type TypeName {
    field1: Type1
    field2: Type2
}

Type definitions are parsed and recorded; no C type is emitted. Struct values at runtime are ElMap. Field access value.field compiles to el_get_field(value, "field").

Optional commas between fields are accepted.

6.2 Enum Types

enum EnumName {
    Variant1
    Variant2
    VariantWithPayload(PayloadType)
}

Variant names are recorded. The current codegen does not emit a dedicated C enum type; values are represented as strings or maps. Payload variants accept the (Type) syntax but the parser records only the variant name.

6.3 Array Literals

let numbers = [1, 2, 3]
let empty   = []

Compile to el_list_new(3, 1, 2, 3) and el_list_new(0).

6.4 Map Literals

let m = { "k1": v1, "k2": v2 }

Keys are Str tokens. Compiles to el_map_new(2, "k1", v1, "k2", v2).

6.5 Field and Index Access

let f = struct_value.field_name   // el_get_field(struct_value, "field_name")
let e = array[0]                  // el_list_get(array, 0)
let v = map["key"]                // el_list_get(map, "key")

Index access compiles to el_list_get. Out-of-bounds returns 0 (EL_NULL).

7. Operators

7.1 Arithmetic

Operator	Status	Notes
`+`	implemented	Int addition or String concatenation (heuristic)
`-`	implemented	Subtraction; also unary negation
`*`	implemented	Multiplication
`/`	implemented	Integer division
`%`	planned	Modulo. Not currently lexed.

+ dispatch: the codegen inspects operand AST node kinds. If either side is Str, a chained +, a Call, or an Ident, it emits el_str_concat(a, b). If both sides are Int literals, it emits arithmetic +. Mixed Int+Ident is treated as string concatenation by default — explicit casts are required for arithmetic on Ident-typed integers.

7.2 Comparison

Operator	Behavior
`==`	`str_eq(a, b)` for Str/Ident/Call operands; `==` for Int/Bool
`!=`	Negation of the above
`<` `>` `<=` `>=`	Integer comparison

7.3 Logical

Operator	Description
`&&`	Short-circuit AND
`\|\|`	Short-circuit OR
`!`	Unary NOT

7.4 Unary

Operator	Meaning
`!`	Logical NOT — emits `!expr`
`-`	Negation — emits `(-expr)`
`?`	Try (postfix) — pass-through today; nil-propagation planned

7.5 Precedence (high to low)

* / % — precedence 6
+ - — precedence 5
< > <= >= — precedence 4
== != — precedence 3
&& — precedence 2
|| — precedence 1

8. Module System

8.1 Import

import "filename.el"

Records an Import AST node. The compiler concatenates all imports into the single C output; the linker produces one binary.

8.2 From-Import

from module_name import { Name1, Name2 }

Parsed. The module name is recorded; the brace-list is consumed. Both forms produce Import nodes. Selective import (importing only specific names) is parsed but not yet enforced — all names in the imported file are visible.

9. Decorators

@manager
fn handle(channel: String, msg: String) -> Void { … }

The @ token followed by an identifier attaches a decorator name to the next FnDef. Decorators with structural meaning today: none. Planned enforcement (Section 16.2): VBD roles @manager, @engine, @accessor.

Non-VBD decorators are accepted and ignored.

10. The Activate Construct [planned]

activate TypeName where "semantic query string"

activate and where are reserved keywords. Lexed today, no parse form. The planned semantics: compile to a runtime call into the local Engram graph that performs spreading-activation retrieval, returning a typed list of nodes that match TypeName.

11. The Sealed Block [planned]

sealed {
    let api_key = "sk-prod-12345"
}

sealed is a reserved keyword. Planned semantics: a capability scope where access to certain runtime services (filesystem, network) is restricted by default and explicit allow-lists must be declared.

12. Standard Library Builtins

Builtins live in el_runtime.c / el_runtime.h. Programs call them by name; no import is required. The status column reflects the canonical self-hosting runtime. Anything marked planned is in flight as part of the in-progress runtime extension.

12.1 I/O — implemented

Builtin	Description
`println(s)`	Print string + newline
`print(s)`	Print string
`readline()`	Read one line from stdin
`args()`	Command-line arguments as a `[String]` (excludes argv[0])

12.2 String — implemented unless noted

Builtin	Description	Status
`str_eq(a, b)`	String equality	implemented
`str_starts_with(s, p)`	Prefix test	implemented
`str_ends_with(s, suf)`	Suffix test	implemented
`str_contains(s, sub)`	Substring test	implemented
`str_len(s)`	Byte length	implemented
`str_slice(s, start, end)`	Substring (byte offsets)	implemented
`str_replace(s, from, to)`	Replace all	implemented
`str_to_upper(s)` / `str_to_lower(s)`	Case fold	implemented
`str_trim(s)`	Strip whitespace	implemented
`str_concat(a, b)`	Concatenate	implemented
`int_to_str(n)`	Format Int	implemented
`str_to_int(s)`	Parse Int	implemented
`str_to_float(s)`	Parse Float	planned
`str_index_of(s, sub)`	Position of substring; `-1` if absent	planned
`str_split(s, sep)`	Split on separator → `[String]`	planned
`str_char_at(s, i)`	Character at byte index	planned
`str_char_code(s, i)`	Unicode code point	planned
`str_pad_left(s, w, p)` / `str_pad_right(s, w, p)`	Pad to width	planned
`str_format(template, data)`	`{key}` interpolation	planned
`str_lower(s)` / `str_upper(s)`	Aliases for `str_to_lower`/`str_to_upper`	planned

12.3 Math — partial

Builtin	Description	Status
`el_abs(n)`	Absolute value	implemented
`el_max(a, b)`	Maximum	implemented
`el_min(a, b)`	Minimum	implemented
`math_sqrt(f)`	Square root	planned
`math_log(f)` / `math_ln(f)`	Logarithms	planned
`math_sin(f)` / `math_cos(f)` / `math_pi()`	Trig	planned

12.4 List — implemented unless noted

Builtin	Description	Status
`el_list_empty()`	Empty list	implemented
`el_list_new(count, …)`	List from N values (varargs; emitted for array literals)	implemented
`el_list_len(list)`	Length	implemented
`el_list_get(list, i)`	Element at index; `0` on out-of-bounds	implemented
`el_list_append(list, e)`	Append; returns updated list	implemented
`list_push(list, e)`	Alias for `el_list_append`	planned
`list_push_front(list, e)`	Prepend	planned
`list_join(list, sep)`	Join → `String`	planned
`list_range(start, end)`	Integer range `[start, end)`	planned

List append returns a new (or reallocated) list pointer; the return value must be used.

12.5 Map — implemented

Builtin	Description
`el_map_new(count, …)`	Map from key/value pairs (emitted for map literals)
`el_map_get(map, key)`	Value by key
`el_map_set(map, key, value)`	Set; returns map
`el_get_field(map, key)`	Alias; emitted for `.field`

12.6 HTTP — planned

Builtin	Status
`http_get(url)`	stub today; libcurl impl planned
`http_post(url, body)`	stub today; libcurl impl planned
`http_serve(port, handler)`	stub today; thread-pool impl planned

handler is a function value of type (method: String, path: String, body: String) -> String. The server thread invokes it on every request.

12.7 Filesystem — implemented

Builtin	Description
`fs_read(path)`	Read file → `String`; `""` on error
`fs_write(path, content)`	Write `String`; returns `1` on success, `0` otherwise

12.8 JSON — partial

Builtin	Description	Status
`json_get(json, key)`	Substring lookup of `"key":` value	implemented
`json_parse(s)`	Parse JSON string → `List` or `Map`	planned
`json_stringify(v)`	Serialize `Any` → `String`	planned
`json_get_string(j, key)`	Typed extract: String	planned
`json_get_int(j, key)`	Typed extract: Int	planned
`json_get_float(j, key)`	Typed extract: Float	planned
`json_get_bool(j, key)`	Typed extract: Bool	planned
`json_get_raw(j, key)`	Extract nested object/array as JSON String	planned
`json_set(j, key, value)`	Update field, return new JSON String	planned
`json_array_len(j)`	Length of JSON array string	planned

12.9 Process — implemented

Builtin	Description
`exit_program(code)`	Exit with code
`args()`	(see Section 12.1)

12.10 Time — planned

Builtin	Description
`time_now()`	Unix epoch milliseconds (Int)
`time_now_utc()`	Same; explicit UTC
`time_format(ts, fmt)`	Format timestamp; `"ISO"` for ISO 8601
`time_to_parts(ts)`	Decompose to `Map` of fields
`time_from_parts(secs, ns, tz)`	Construct
`time_add(ts, n, unit)`	Add duration; unit ∈ `"ms"`, `"sec"`, `"day"`, etc.
`time_diff(ts1, ts2, unit)`	Difference

12.11 Identifiers — planned

Builtin	Description
`uuid_new()`	RFC 4122 v4 UUID String
`uuid_v4()`	Alias for `uuid_new`

12.12 Float Formatting — planned

Builtin	Description
`float_to_str(f)`	Default float string
`int_to_float(n)`	Widen Int → Float
`float_to_int(f)`	Truncate Float → Int
`format_float(f, decimals)`	Format with N decimal places
`decimal_round(f, decimals)`	Round to N decimals

12.13 Process Environment — planned

Builtin	Description
`env(key)`	Read environment variable; `""` when unset

12.14 In-Process State — planned

Builtin	Description
`state_set(key, value)`	Store in process-global key/value table
`state_get(key)`	Retrieve; `""` if absent
`state_del(key)`	Delete
`state_keys()`	All keys as `[String]`

State persists for the lifetime of the OS process. Used by HTTP servers to share data between request handlers.

12.15 Native Compiler Primitives — implemented

These are used by the self-hosting compiler source and are thin aliases over the runtime list/string operations.

Builtin	Description
`native_list_empty()`	Empty list
`native_list_append(l, v)`	Append
`native_list_get(l, idx)`	Element at index
`native_list_len(l)`	Length
`native_string_chars(s)`	Split string → `[String]` of one-character strings
`native_int_to_str(n)`	Format integer

13. Compilation Model

13.1 Pipeline

source.el
    → [Lexer]    → token list
    → [Parser]   → AST (list of statement maps)
    → [Codegen]  → C source (streamed to stdout)
    → [cc]       → native binary

The codegen streams output line-by-line via println to avoid O(n²) string concatenation.

13.2 Self-Hosting Architecture

The El compiler lives in el-compiler/src/:

lexer.el — tokenizer
parser.el — recursive-descent parser
codegen.el — C emitter
compiler.el — pipeline wiring + main() entry

These are concatenated into elc-combined.el (single-file bootstrap edition). The bootstrap compiler binary lives at dist/platform/elc; from there elc compiles itself and all El programs.

13.3 C Runtime

Every compiled program links against:

el_runtime.h — declaration header
el_runtime.c — implementation

Compile command:

cc -std=c11 -I<runtime-dir> -o <prog> <prog>.c el_runtime.c

13.4 Output Format

#include <stdint.h>
#include <stdlib.h>
#include "el_runtime.h"

// Forward declarations
el_val_t fn1(el_val_t p1, el_val_t p2);
…

// Function definitions
el_val_t fn1(el_val_t p1, el_val_t p2) {
    …
    return 0;
}

// main() — top-level El statements
int main(int argc, char** argv) {
    el_runtime_init_args(argc, argv);
    [el_cgi_init(…) if cgi block present — planned]
    …
    return 0;
}

All values are el_val_t (int64_t). Strings are pointers cast to int64_t via EL_STR(s) / EL_CSTR(v).

14. Grammar (EBNF)

program     = stmt* EOF

stmt        = let_stmt
            | return_stmt
            | fn_def
            | type_def
            | enum_def
            | import_stmt
            | from_import_stmt
            | while_stmt
            | for_stmt
            | decorator_stmt
            | cgi_block          (* planned *)
            | sealed_block       (* planned *)
            | expr_stmt

let_stmt         = "let" IDENT (":" type_expr)? "=" expr
return_stmt      = "return" expr?
fn_def           = "fn" IDENT "(" param_list ")" ("->" type_expr)? "{" stmt* "}"
type_def         = "type" IDENT "{" (IDENT ":" type_expr ","?)* "}"
enum_def         = "enum" IDENT "{" (IDENT ("(" type_expr ")")? ","?)* "}"
import_stmt      = "import" STRING
from_import_stmt = "from" IDENT "import" "{" (IDENT ","?)* "}"
while_stmt       = "while" expr "{" stmt* "}"
for_stmt         = "for" IDENT "in" expr "{" stmt* "}"
decorator_stmt   = "@" IDENT stmt
cgi_block        = "cgi" STRING "{" cgi_field* "}"      (* planned *)
cgi_field        = IDENT ":" STRING                      (* planned *)
expr_stmt        = expr

param_list       = (param ("," param)*)?
param            = IDENT ":" type_expr

type_expr        = IDENT
                 | "[" type_expr "]"
                 | type_expr "?"
                 | IDENT "<" type_expr ("," type_expr)* ">"

expr             = binop_expr
binop_expr       = unary_expr (binop unary_expr)*
binop            = "||" | "&&" | "==" | "!=" | "<" | ">" | "<=" | ">=" | "+" | "-" | "*" | "/" | "%"
unary_expr       = "!" primary | "-" primary | postfix_expr
postfix_expr     = primary ("." IDENT | "(" arg_list ")" | "[" expr "]" | "?")*

primary          = INT | FLOAT | STRING | BOOL
                 | "(" expr ")"
                 | "[" arg_list "]"
                 | "{" (STRING ":" expr ","?)* "}"
                 | "if" expr "{" stmt* "}" ("else" ("if" expr "{" stmt* "}" | "{" stmt* "}"))?
                 | "match" expr "{" match_arm* "}"
                 | "for" IDENT "in" expr "{" stmt* "}"
                 | IDENT

arg_list         = (expr ("," expr)*)?
match_arm        = pattern "=>" expr ","?
pattern          = "_" | IDENT | INT | STRING | BOOL

% is in the grammar; lexer/parser/codegen support is planned (see Section 7.1).

15. Vessel System

A vessel is the El equivalent of a package: a buildable unit with a manifest at the project root.

15.1 Manifest — `manifest.el`

The manifest is itself an El file. It uses block syntax with space-separated declarations, no equals signs, strings in "…", integers as bare numbers, arrays as […].

// manifest.el
vessel "engram" {
    version "1.0.0"
    description "Engram graph intelligence substrate"
    authors ["Will Anderson <will@neurontechnologies.ai>"]
    edition "2026"
}

dependencies {
    el-platform "1.0"
    el-services "1.0"
}

build {
    entry "src/server.el"
    output "dist/"
}

Rules:

String values use "…".
Integer values are bare numbers.
Arrays use […].
Block sections use { }.
Section headers in [bracket] form are not used.

vessel replaces the legacy package keyword. (Lexer support: planned. Old projects may continue to use package until migrated.)

15.2 CLI

el new <name>           scaffold a new vessel
el build                build the vessel
el run                  build and run debug
el test                 run tests
el check                type-check only (when type checker lands)
el fmt                  format source
el clean                clear build artifacts
el build-file <file>    compile a single file

16. DHARMA Network and CGI Communication

DHARMA (Dynamic Heuristic Agent Relationship and Memory Architecture) is the global network of CGI Entities and their Human Sponsors. Every registered CGI–Sponsor pair is a member of the DHARMA Network. The technical infrastructure (registry, transport, validators) exists to serve that collective. The persistence layer is Engram: a weighted graph where every CGI interaction strengthens an edge (Hebbian), knowledge propagates by spreading activation, and relationships persist across sessions.

This section specifies the El-language constructs for CGI programs.

16.1 The `cgi` Block — planned

cgi "name" {
    dharma_id: "…"
    principal: "…"
    network:   "…"
    engram:    "…"
}

Field	Type	Required	Default
`dharma_id`	String	yes	—
`principal`	String	yes	—
`network`	String	no	`"dharma-mainnet"`
`engram`	String	no	`"http://localhost:8742"`

Grammar extension:

stmt      = … | cgi_block
cgi_block = "cgi" STRING "{" cgi_field* "}"
cgi_field = IDENT ":" STRING

Compilation (planned): the codegen emits an el_cgi_init(name, dharma_id, principal, network, engram) call as the first statement inside main(), before any user code runs. The runtime uses this to register with DHARMA before any dharma_* call resolves a peer.

cgi is mutually exclusive with an app block. Exactly one or the other per program.

16.2 VBD Component Roles — planned enforcement

El programs that participate in DHARMA follow Volatility-Based Decomposition. The role is declared on a function via decorator:

@manager
fn handle_message(channel: String, msg: String) -> Void { … }

@engine
fn process_content(content: String) -> String { … }

@accessor
fn fetch_peer_state(cgi_id: String) -> Map<String, Any> { … }

Role	Decorator	Responsibility
Manager	`@manager`	Orchestrates workflows; sole emitter/fielder of DHARMA events
Engine	`@engine`	Pure computation; no side effects
Accessor	`@accessor`	External state I/O (Engram, network, storage)

Planned compile-time constraints:

dharma_emit and dharma_field are only callable from @manager functions. Calling either from @engine/@accessor/undecorated code is a compile error.
Cross-component call rules:
- Manager → Engine, Manager → Accessor: allowed (sync).
- Manager → Manager: only via the planned async modifier (sync M→M is a compile error).
- Engine → Engine, Engine → Accessor: allowed.
- Engine → Manager: prohibited.
- Accessor → anything: prohibited (Accessors are receivers only).

Today the parser accepts the decorators but enforces nothing.

16.3 DHARMA Network Builtins — stubs

All dharma_* functions are available without import to CGI programs. Today they are stubs in el_runtime.c: each prints a descriptive line to stdout and returns an empty value. Full implementations land with the runtime extension.

`dharma_connect(cgi_id: String) -> String`

Open a channel to another CGI. Returns a channel ID. Idempotent for the same cgi_id.

`dharma_send(channel: String, content: String) -> String`

Send content over channel. Blocks until response. Returns the response string.

`dharma_activate(query: String) -> [Map<String, Any>]`

Spreading activation across the DHARMA network. Aggregates results from all reachable CGIs' Engram graphs, sorted by activation strength.

`dharma_emit(event_type: String, payload: String) -> Void`

Emit a network event. Manager-only (planned constraint).

`dharma_field(event_type: String) -> Map<String, Any>`

Block until the next event of event_type arrives. Returns { type, payload, source_cgi, timestamp }. Manager-only (planned constraint).

`dharma_strengthen(cgi_id: String, weight: Float) -> Void`

Hebbian potentiation of the relationship to another CGI. The runtime auto-calls this with a small increment after each successful send/receive cycle.

`dharma_relationship(cgi_id: String) -> Float`

Returns the current relationship weight (0.0–1.0).

`dharma_peers() -> [String]`

Returns CGI IDs with non-zero relationship weight, sorted descending.

16.4 Engram Local Graph Primitives — runtime-native (full impl in flight)

Engram is the knowledge graph substrate. The Engram store is in-process — embedded directly in el_runtime.c. CGI programs and the Engram HTTP server both call these primitives; there is no driver layer and no SQL. The primitives operate on the host process's graph, with snapshot-to-disk persistence handled by the runtime.

This is the central architectural commitment: graph is a first-class runtime concept, not a library.

`engram_node(content: String, node_type: String, salience: Float) -> String`

Create a node. salience is initial activation in [0.0, 1.0]. Returns the node ID.

`engram_get(node_id: String) -> Map<String, Any>`

Retrieve a node by ID. Returns { id, content, node_type, salience, importance, confidence, tier, tags, created_at, updated_at }. Empty map if not found.

`engram_activate(query: String, depth: Int) -> [Map<String, Any>]`

Spreading activation in the local graph. Seeds match on text or label; activation propagates up to depth hops with attenuation by edge weight.

`engram_connect(from_id: String, to_id: String, weight: Float, relation: String) -> Void`

Create a directed edge. weight ∈ [0.0, 1.0]. relation is the edge type label.

`engram_strengthen(node_id: String) -> Void`

Hebbian potentiation. Boosts salience by a fixed increment, clamped at 1.0. Auto-called by the runtime when a node is retrieved via activation.

`engram_neighbors(node_id: String, max_depth: Int) -> [Map<String, Any>]`

Breadth-first traversal. Returns a list of { node, edge, hops } triples.

`engram_search(query: String, limit: Int) -> [Map<String, Any>]`

Full-text search on content, label, and tags. Returns nodes sorted by salience.

`engram_forget(node_id: String) -> Void`

Remove a node and all incident edges.

`engram_node_count() -> Int`

Total node count.

`engram_edge_count() -> Int`

Total edge count.

`engram_save(path: String) -> Bool`

Snapshot the graph to disk as a single JSON document at path.

`engram_load(path: String) -> Bool`

Restore the graph from a snapshot. Replaces the current in-memory graph.

Status: All engram_* are stubs in the current runtime (print + return empty). The full in-process implementation — node store, edge indexes, salience-ranked retrieval, spreading activation, Hebbian strengthening, snapshot persistence — is the primary work of the in-flight runtime extension.

16.5 Backing Model

Storage. Nodes and edges are kept in process memory as flat arrays plus secondary indexes (by ID, by node_type, by tier, by from, by to). Salience is updated in place. The graph is durable via periodic snapshots (engram_save) and a write-ahead log written by the runtime on every mutation.

Hebbian learning. Every successful retrieval automatically calls engram_strengthen on the activated node and dharma_strengthen on the source CGI when the result crossed a network edge. Strengthening is additive with a small increment (default 0.01) and clamps to 1.0.

Spreading activation. The activation algorithm follows the field model in elql/test/field_test.el:

proximity = 1 / (1 + dist²) for the latent semantic gradient.
temporal_decay = clamp(1 − rate × age, 0, 1).
path_strength = edge_weight × temporal_decay.
epistemic_confidence = node_confidence × path_strength.

Below a confidence threshold (0.2 by default), retrieval emits a "refresh" signal — telling the caller the answer is uncertain and should be re-grounded.

Cross-CGI activation. dharma_activate(query) runs engram_activate locally, then propagates the query to every connected CGI via DHARMA channels, attenuating activation by relationship weight. Stronger relationships → higher residual activation → earlier and more confident results from that peer.

16.6 Complete Example

cgi "genesis" {
    dharma_id: "ntn-genesis"
    principal: "will-anderson"
    network:   "dharma-mainnet"
    engram:    "http://localhost:8742"
}

@accessor
fn record_observation(content: String, salience: Float) -> String {
    return engram_node(content, "observation", salience)
}

@engine
fn format_share(content: String, source: String) -> String {
    return "{\"content\":\"" + content + "\",\"source\":\"" + source + "\"}"
}

@manager
fn collaborate_with_archivist() -> Void {
    let channel = dharma_connect("ntn-archivist")
    let trust   = dharma_relationship("ntn-archivist")
    println("Relationship: " + float_to_str(trust))

    let node_id = record_observation("Spreading activation improves recall by 40%", 0.9)
    let msg     = format_share("Spreading activation improves recall by 40%", "ntn-genesis")
    let reply   = dharma_send(channel, msg)
    println("Archivist: " + reply)

    dharma_emit("knowledge.validated", msg)

    let related = dharma_activate("spreading activation recall memory")
    for node in related {
        println(node["content"])
    }

    dharma_strengthen("ntn-archivist", 0.05)
}

collaborate_with_archivist()

16.7 Stub Behavior (today)

The current runtime stubs print a line and return empty values:

dharma_connect → prints, returns "ch:<cgi_id>".
dharma_send → prints, returns "".
dharma_activate → prints, returns [].
dharma_emit → prints, returns void.
dharma_field → prints, returns {}.
dharma_strengthen → prints, returns void.
dharma_relationship → prints, returns 0.
dharma_peers → prints, returns [].
engram_node → prints, returns "stub-node-id".
engram_activate → prints, returns [].
engram_connect → prints, returns void.
engram_strengthen → prints, returns void.

Stub output goes to stdout so unit tests can observe call patterns without a live runtime. This behavior is temporary; the in-flight runtime extension replaces every stub with a real implementation.

17. Roadmap to v1.3

The next minor version closes the implementation gaps named in this document. Tracked in order:

Runtime extension — JSON, time, UUID, env, state, real HTTP, real engram_* (in-process graph store), real dharma_* (network transport).
Lexer/parser/codegen extensions — % operator, match codegen, ? propagation, cgi block, vessel keyword, VBD role enforcement.
Self-hosted recompilation — rebuild dist/platform/elc against the extended language and runtime.
Engram conversion — Engram becomes a thin HTTP face over engram_*, with no internal db.el layer.
Spec follow-up (v1.3) — every "planned" marker in this document becomes "implemented." Status section consolidates.

End of specification.

38 KiB Raw Permalink Blame History Unescape Escape