Files
el/ql/spec/elql.md
T

22 KiB
Raw Blame History

engram-el Specification

Version 1.0.0 — April 30, 2026


Overview

engram-el is the El-native integration layer for the Engram graph engine. It is a collection of El programs that operate on a live Engram server, demonstrating the correct patterns for El programs that use the Engram knowledge graph as their substrate.

The repository contains three components:

Component Path Description
Studio studio/studio.el Full-featured terminal graph explorer
Field model test/field_test.el Hebbian field model proof of concept
Language tests test/language_features_test.el El builtin coverage tests
LLM tests test/llm_test.el LLM builtin smoke tests

engram-el is not a library. It contains no importable modules and no compilation artifact. It is a set of standalone .el programs run directly with el run-file.


1. Architecture

1.1 Integration Model

All Engram access uses El's native HTTP builtins over the Engram HTTP API. There is no SDK, no special driver, and no additional compilation step. The integration is:

El program → http_get / http_post → Engram HTTP API → JSON response → json_parse

This is intentional. The HTTP API is the canonical external interface to Engram. Programs that need tight runtime integration (the Neuron daemon itself) use the lower-level graph_compile and graph_traverse VM builtins. External tools use HTTP.

1.2 Configuration

All programs read server configuration from environment variables at startup:

Variable Default Description
ENGRAM_URL http://localhost:8340 Engram server base URL
ENGRAM_REPORT /tmp/engram-studio-report.txt Studio report output path

Example:

# Default local server
el run-file studio/studio.el

# Remote server
ENGRAM_URL=http://engram.example.com el run-file studio/studio.el

# Custom report path
ENGRAM_REPORT=/var/log/studio.txt el run-file studio/studio.el

1.3 Error Handling Convention

All API wrappers follow this convention: if the response begins with {"error", return empty string. All callers check for empty string and emit a placeholder rather than crashing. Programs run to completion even when the server is unreachable.

fn api_get(path: String) -> String {
    let url: String = get_base_url() + path
    let resp: String = http_get(url)
    if str_starts_with(resp, "{\"error\"") {
        return ""
    }
    return resp
}

2. Studio Application

studio/studio.el is a 788-line terminal graph explorer. It connects to Engram, fetches data across all graph dimensions, renders a formatted report to the terminal, and writes a text report to disk.

2.1 Sections Rendered

The studio renders these sections sequentially:

Section Engram Endpoint Description
Database Statistics GET /api/stats Node count, edge count, avg salience, DB size
Recent Nodes GET /api/nodes?limit=N Most recently created nodes
Top by Salience GET /api/nodes?limit=N&min_salience=0.0 Highest-salience nodes
Memory Nodes GET /api/nodes?node_type=Memory&limit=N Nodes of type Memory
Concept Nodes GET /api/nodes?node_type=Concept&limit=N Nodes of type Concept
Event Nodes GET /api/nodes?node_type=Event&limit=N Nodes of type Event
Entity Nodes GET /api/nodes?node_type=Entity&limit=N Nodes of type Entity
Process Nodes GET /api/nodes?node_type=Process&limit=N Nodes of type Process
Semantic Tier GET /api/nodes?tier=Semantic&limit=N Nodes in Semantic memory tier
Episodic Tier GET /api/nodes?tier=Episodic&limit=N Nodes in Episodic memory tier
Knowledge Browser GET /api/nodes?node_type=Concept&limit=50 Concept nodes as domain anchors
Text Search GET /api/search?q=<query>&limit=N Full-text search results
Edge Explorer GET /api/edges?limit=N Sample of edges with weights
Interactive Menu (local) Menu of available commands

2.2 Engram Node Fields Consumed

The studio reads these fields from each node JSON object:

Field Type Usage
id String (UUID) Display (first 8 chars) and API lookups
label String Primary display text
node_type String Type badge and section filtering
tier String Tier badge and section filtering
salience Float Salience bar visualization and sorting
importance Float Node detail display
confidence Float Node detail display
created_at Int (ms) Timestamp display
updated_at Int (ms) Timestamp display

2.3 Node Display

Each node in list views renders as a single row:

   1. 9685fc7a  label text here                    Memory      0.750

Format: index. short_id label_padded_to_36 type_badge_padded_to_10 salience

Salience is also rendered as a colored block bar (10 segments) in the Top by Salience section:

  • >= 0.7 — green (████████░░)
  • >= 0.4 — yellow
  • < 0.4 — dim

2.4 Spreading Activation

The studio includes a spreading activation section (requires a specific seed node ID):

fn show_activation(seed_id: String, limit: Int, report: String) -> String {
    let path: String = "/api/activate?seeds=" + seed_id
        + "&limit=" + int_to_str(limit) + "&depth=3"
    let json_str: String = api_get(path)
    let results_raw: String = json_get_raw(json_str, "results")
    let results: List = json_parse(results_raw)
    // renders each result: node label, activation_strength, hops
}

Each activation result has fields: node (nested object), activation_strength (Float), hops (Int).

2.5 Neighbor Traversal

Node detail view fetches BFS neighbors:

let nb_path: String = "/api/neighbors/" + node_id + "?depth=2"
let nb_json: String = api_get(nb_path)
let neighbors: List = json_parse(nb_json)

Each neighbor object has fields: node (nested object), edge (nested object), hops (Int). Nested objects are extracted with json_get_raw.

2.6 Report Export

The studio accumulates a plain-text report string as each section renders. On completion it writes the report to ENGRAM_REPORT using fs_write:

fn export_report(content: String, path: String) -> String {
    let header: String = "ENGRAM DATA STUDIO — Report\n"
        + "Generated: " + time_format(time_now_utc(), "ISO") + "\n"
        + "Server: " + get_base_url() + "\n"
        + repeat_str("=", 60) + "\n"
    let ok: Bool = fs_write(path, header + content)
    // ...
}

fs_write returns Bool — true on success.


3. Field Model

test/field_test.el is a pure in-memory demonstration of a Hebbian field model. It does not connect to Engram. It uses El arithmetic to simulate the mechanics of the Engram activation engine.

3.1 Purpose

The field test documents the mathematical model underlying Engram's spreading activation and confidence-qualified retrieval. It is both a test that the math is correct and a specification of what the numbers mean.

3.2 Core Functions

Proximity (latent semantic gradient):

fn proximity(ax: Float, ay: Float, bx: Float, by: Float) -> Float {
    let d: Float = dist_sq(ax, ay, bx, by)
    return 1.0 / (1.0 + d)
}

Returns a value in (0, 1]. Nodes closer in 2D semantic space have higher proximity.

Temporal decay:

fn temporal_decay(age: Float, decay_rate: Float) -> Float {
    let d: Float = 1.0 - decay_rate * age
    return clamp_f(d, 0.0, 1.0)
}

Linear decay from 1.0 at age=0 toward 0.0. Clamped to [0, 1].

Hebbian weight update:

fn hebbian_update(weight: Float, act_i: Float, act_j: Float, lr: Float) -> Float {
    let delta: Float = lr * act_i * act_j
    return clamp_f(weight + delta, 0.0, 1.0)
}

When two nodes co-activate, their edge weight increases by learning_rate × activation_i × activation_j. Clamped to [0, 1].

Edge decay (between activations):

fn edge_decay(weight: Float, decay_rate: Float) -> Float {
    return clamp_f(weight * (1.0 - decay_rate), 0.0, 1.0)
}

Path strength:

fn path_strength(edge_weight: Float, node_age: Float, decay_rate: Float) -> Float {
    let td: Float = temporal_decay(node_age, decay_rate)
    return edge_weight * td
}

Combined confidence qualifier on a retrieval path. Strong edge × recently activated = high path strength.

Epistemic confidence:

fn epistemic_confidence(node_confidence: Float, ps: Float) -> Float {
    return node_confidence * ps
}

Final confidence on a retrieved node: the node's stored confidence multiplied by the path strength through which it was reached.

3.3 Simulation Scenario

The test models a clinical scenario with four nodes:

Node Label Semantic position Temporal coordinate
A patient has fever (0.2, 0.4) t=100
B influenza (0.3, 0.5) t=90
C drug interaction warning (0.6, 0.3) t=10 (old)
D ibuprofen (0.65, 0.3) t=10 (old)

The semantic axes are: x = concrete↔abstract, y = negative↔positive valence.

Three co-activation events between A and B (fever/flu diagnosis) strengthen the A→B edge via Hebbian learning. After three events with learning rate 0.3 and full activation strength:

w_ab: 0.0 → 0.3 → 0.51 → 0.657

C and D are never co-activated with A, so their edges remain at latent proximity only.

Retrieval: Activating A (fever) propagates to B with high confidence. C (drug interaction) is reached only via latent proximity gradient — weak signal, old node — producing low epistemic confidence below threshold 0.2. This triggers a "refresh" signal. After the doctor looks up drug interactions, A, C, and D co-activate; edges strengthen; subsequent retrieval finds C and D with high confidence.

The scenario demonstrates: perfect storage, calibrated confidence, and self-correcting retrieval.


4. Language Feature Tests

test/language_features_test.el tests El builtins exercised by engram-el programs. These are functional smoke tests — each prints its result for visual verification.

4.1 Arithmetic Operators Tested

Operator Example
% (modulo) 17 % 52
& (bitwise AND) 10 & 128
^ (bitwise XOR) 10 ^ 126
<< (left shift) 1 << 38
>> (right shift) 16 >> 24

4.2 Math Builtins

Builtin Signature Description
math_sin(f) Float → Float Sine
math_cos(f) Float → Float Cosine
math_pi() () → Float Pi constant

4.3 String Builtins

Builtin Signature Description
str_pad_left(s, width, pad) String, Int, String → String Pad to width on left
str_pad_right(s, width, pad) String, Int, String → String Pad to width on right
str_format(template, data) String, Map → String {key} template interpolation
str_len(s) String → Int Character count
str_slice(s, start, end) String, Int, Int → String Substring by index
str_starts_with(s, prefix) String, String → Bool Prefix test
str_char_at(s, i) String, Int → String Single character at index
str_char_code(s, i) String, Int → Int Unicode code point at index
str_from_char_code(n) Int → String Character from code point

str_format uses {key} syntax. The data argument is a map literal:

let tmpl = "Hello {name}, you are {age} years old!"
let data = {"name": "Will", "age": "30"}
let formatted = str_format(tmpl, data)
//  "Hello Will, you are 30 years old!"

4.4 Float Builtins

Builtin Signature Description
format_float(f, decimals) Float, Int → String Format to N decimal places
float_to_str(f) Float → String Default float string
float_to_int(f) Float → Int Truncate (not round)
int_to_float(n) Int → Float Widen to float
decimal_round(f, decimals) Float, Int → Float Round to N decimal places

4.5 Time Builtins

Builtin Signature Description
time_now_utc() () → Int Current time as Unix milliseconds
time_format(ts, fmt) Int, String → String Format timestamp; "ISO" for ISO 8601
time_to_parts(ts) Int → Map Decompose timestamp
time_from_parts(secs, ns, tz) Int, Int, String → Int Construct timestamp
time_add(ts, n, unit) Int, Int, String → Int Add duration; units: "day", etc.
time_diff(ts1, ts2, unit) Int, Int, String → Int Difference in units

4.6 List Builtins

Builtin Signature Description
list_new() () → List Empty list
list_len(lst) List → Int Element count
list_get(lst, i) List, Int → Any Element at index
list_push(lst, v) List, Any → List Append element (returns new list)
list_peek_last(lst) List → Any Last element without removing
list_range(start, end) Int, Int → List Integer range [start, end)
list_join(lst, sep) List, String → String Join as string with separator

4.7 Stack Builtins

Builtin Signature Description
stack_new() () → Stack Empty stack
stack_push(s, v) Stack, Any → Stack Push value (returns new stack)
stack_pop(s) Stack → Stack Pop top (returns new stack)
stack_peek(s) Stack → Any Top element without removing

4.8 Queue Builtins

Builtin Signature Description
queue_new() () → Queue Empty queue
queue_enqueue(q, v) Queue, Any → Queue Enqueue (returns new queue)
queue_dequeue(q) Queue → Queue Dequeue front (returns new queue)
queue_peek(q) Queue → Any Front element without removing

4.9 JSON Builtins

Builtin Signature Description
json_parse(s) String → List or Map Parse JSON array or object
json_stringify(v) Any → String Serialize value to JSON
json_get_string(json, key) String, String → String Extract string field
json_get_int(json, key) String, String → Int Extract integer field
json_get_float(json, key) String, String → Float Extract float field
json_get_raw(json, key) String, String → String Extract nested object as raw JSON string

json_get_raw is required when a JSON field is itself an object or array that needs to be parsed as a list. Example:

let results_raw: String = json_get_raw(json_str, "results")
let results: List = json_parse(results_raw)

4.10 Nil Builtins

Builtin Signature Description
is_nil(v) Any → Bool True if value is nil
unwrap_or(v, default) Any, Any → Any Return v if not nil, else default

4.11 Type Conversion

Builtin Signature Description
int_to_str(n) Int → String Integer to decimal string
bool_to_str(b) Bool → String "true" or "false"
int_to_float(n) Int → Float Widen
float_to_int(f) Float → Int Truncate toward zero

4.12 Terminal Color Builtins

Used by the studio for terminal output formatting:

Builtin Description
color_bold(s) Bold text
color_dim(s) Dim/gray text
color_green(s) Green text
color_yellow(s) Yellow text
color_cyan(s) Cyan text
color_red(s) Red text

4.13 HTTP Builtins

Builtin Signature Description
http_get(url) String → String HTTP GET, returns response body
http_post(url, body) String, String → String HTTP POST with body, returns response body

4.14 File System Builtins

Builtin Signature Description
fs_write(path, content) String, String → Bool Write string to file; true on success

4.15 Environment and LLM Builtins

Builtin Signature Description
env(key) String → String Read environment variable; empty string if unset
llm_models() () → List List available LLM models
llm_call(model, prompt) String, String → String Call an LLM; returns response string or error

llm_call requires ANTHROPIC_API_KEY in the environment. Returns an error string (not a crash) when the key is missing.


5. Engram HTTP API Reference

The API surface consumed by engram-el programs:

5.1 Endpoints

Method Path Parameters Returns
GET /api/stats {node_count, edge_count, avg_salience, db_size_bytes}
GET /api/nodes node_type, tier, limit, min_salience JSON array of node objects
GET /api/nodes/{id} Single node object
GET /api/edges limit JSON array of edge objects
GET /api/search q, limit JSON array of node objects
GET /api/neighbors/{id} depth JSON array of {node, edge, hops} objects
GET /api/activate seeds, limit, depth {results: [{node, activation_strength, hops}]}
POST /api/nodes JSON body Created node object with id

5.2 Node Object Schema

{
    "id": "uuid-string",
    "label": "human-readable label",
    "node_type": "Memory|Concept|Event|Entity|Process|InternalState",
    "tier": "Working|Episodic|Semantic|Procedural",
    "salience": 0.75,
    "importance": 0.5,
    "confidence": 0.9,
    "created_at": 1714500000000,
    "updated_at": 1714500000000
}

5.3 Edge Object Schema

{
    "id": "uuid-string",
    "from_id": "uuid-string",
    "to_id": "uuid-string",
    "relation": "relation-type-name",
    "weight": 0.8,
    "confidence": 0.9
}

5.4 Activation Result Schema

The /api/activate response wraps results in a top-level results key:

{
    "results": [
        {
            "node": { /* node object */ },
            "activation_strength": 0.62,
            "hops": 2
        }
    ]
}

Access with json_get_raw(response, "results") then json_parse the extracted string.


6. Integration Patterns

6.1 Read Nodes

fn get_nodes_of_type(node_type: String, limit: Int) -> List {
    let path: String = "/api/nodes?node_type=" + node_type
        + "&limit=" + int_to_str(limit)
    let json_str: String = http_get(env("ENGRAM_URL") + path)
    if json_str == "" {
        return list_new()
    }
    return json_parse(json_str)
}

6.2 Write a Node

fn create_node(label: String, content: String, node_type: String, tier: String) -> String {
    let body: String = "{\"label\":\"" + label
        + "\",\"content\":\"" + content
        + "\",\"node_type\":\"" + node_type
        + "\",\"tier\":\"" + tier
        + "\",\"importance\":0.5}"
    let resp: String = http_post(env("ENGRAM_URL") + "/api/nodes", body)
    return json_get_string(resp, "id")
}
fn search_graph(query: String, limit: Int) -> List {
    let path: String = "/api/search?q=" + query + "&limit=" + int_to_str(limit)
    let json_str: String = http_get(env("ENGRAM_URL") + path)
    if json_str == "" {
        return list_new()
    }
    return json_parse(json_str)
}

6.4 Spreading Activation

fn activate_from_node(node_id: String, depth: Int, limit: Int) -> List {
    let path: String = "/api/activate?seeds=" + node_id
        + "&depth=" + int_to_str(depth)
        + "&limit=" + int_to_str(limit)
    let resp_str: String = http_get(env("ENGRAM_URL") + path)
    if resp_str == "" {
        return list_new()
    }
    let results_raw: String = json_get_raw(resp_str, "results")
    return json_parse(results_raw)
}

6.5 Extract Nested Objects

When a JSON field is itself an object (e.g., node inside a neighbor result), use json_get_raw to extract it as a string before reading its fields:

let neighbor_json: String = json_stringify(list_get(neighbors, i))
let node_obj: String = json_get_raw(neighbor_json, "node")
let edge_obj: String = json_get_raw(neighbor_json, "edge")
let label: String = json_get_string(node_obj, "label")
let relation: String = json_get_string(edge_obj, "relation")

7. Design Decisions

7.1 Stateless Programs

All engram-el programs are stateless. They read from Engram on each run and write nothing back (the studio is read-only). Exploration tools observe the graph; they do not mutate it.

7.2 No Imports

El programs in this repository use zero imports. All capabilities come from El's built-in dispatch layer. This is correct El style for programs that do not need cross-module sharing — El is not a library ecosystem.

7.3 Immutable Update Style

Stack, queue, and list operations return new values rather than mutating in place. This is how El builtins are designed:

let s0 = stack_new()
let s1 = stack_push(s0, 10)   // s0 is unchanged
let s2 = stack_push(s1, 20)
let top = stack_peek(s2)       // 20
let s3 = stack_pop(s2)         // s3 has only 10

The idiomatic pattern is to shadow the binding with the new value when the old one is no longer needed:

let stack = stack_new()
let stack = stack_push(stack, 10)
let stack = stack_push(stack, 20)

7.4 String Accumulation Pattern

The studio builds its text report by threading a String accumulator through each section function. Each section function receives report: String and returns report + new_content. This is the correct pattern in El, which has no mutable references:

let report: String = ""
let report: String = show_stats(report)
let report: String = show_recent(20, report)
// ...
export_report(report, report_path)