Compare commits

..

2 Commits

Author SHA1 Message Date
bigmerge 827257d3a4 Remove __pycache__ .pyc files accidentally included in the projector commit
El SDK CI - dev / build-and-test (pull_request) Waiting to run
2026-08-15 14:27:23 -05:00
bigmerge 4bbfdcceff Add native audio/image efferent surfaces + projector proof-of-shape
audio-surface.el / image-surface.el: own-core additive-synthesis WAV and
raster-PNG renderers (integer-only DSP, since EL has no floats), rendered
from learned engram signatures via a pluggable surface-profile
abstraction (surface-profile.el). audio-demo.el / image-demo.el are
drivers. NOTE: demo files hardcode absolute paths to this worktree's own
directory — will need a path fixup before landing.

elp/projector/ is a Python package the author's own README marks as
"STAGING/PROOF-OF-SHAPE — not the deliverable", superseded by the native
.el surface-profile work above; kept as a validated architecture proof.
Generated output (elp/faculty/{out,sig}, elp/projector/out,
__pycache__) intentionally excluded.
2026-08-15 14:26:59 -05:00
22 changed files with 2511 additions and 625 deletions
+91
View File
@@ -0,0 +1,91 @@
> **STATUS: STAGING / PROOF-OF-SHAPE — not the deliverable.** This Python package
> proved the architecture end-to-end against the proven realizer faculty (faithful
> md/docx/midi from real geometry: 0 ungrounded claims, SACRED polarity). Per Will's
> steer, the DELIVERABLE is NATIVE: the seam lives on the existing EL realizer as
> **surface-as-profile** — see `../src/surface-profile.el` and
> `../tests/examples/surface-profile-demo.el` (compiles + runs through elc → C →
> binary). The concepts below (one geometry-carrying frame; surface = a pluggable
> profile; plan/realize; deterministic-from-meaning) are exactly what the native
> module implements. Keep this package as the validated proof; build native.
# Efferent Multimodal Projector
**geometry → any surface, faithfully.** Neuron's own document-generation faculty:
the efferent twin of the ingest organ. Ingest is afferent (world → geometry);
this is efferent (geometry → an arbitrary-format document / any modality).
Built against the **proven** realizer faculty (neuron-talk sidecar `:8756`,
artifact `art-7affa557`). The live soul (`:8742` / `:7770`) is contacted **only**
through the read-only, GET-only `engram_client` — never mutated.
## The pipeline (surface-agnostic)
```
geometry region + surface/format spec
→ PLAN (manifold → document skeleton/DAG; the geometry IS the outline) plan.py
→ REALIZE (proven realizer, scaled sentence → passage, each section faithful) realize.py
→ COHERE (document-level flow / transitions, not stitched sentences) cohere.py
→ EMIT (pluggable SurfaceProjector → the target surface) projectors/
```
**The surface is a PARAMETER.** `pipeline.build_ir(...)` builds ONE
surface-neutral `DocumentIR` (`document_ir.py`); `pipeline.emit(doc, surface)`
projects it to whichever surface you name. Markdown, docx, and MIDI are the same
IR emitted three ways.
## The pivot: a geometry-carrying IR
`DocumentIR` is **not** a text tree. Every `Block` carries BOTH:
- `.sentences` — realized faithful text (what **text** projectors read),
- `.provenance` — the source geometry: `subj_id / relation / obj / polarity /
confidence / importance / salience / node_id` (what **music / image / video**
projectors read).
That single decision is what makes the projector multimodal: text renders the
words; music/image decode the geometry. A claim with no provenance cannot exist
in the IR — faithfulness is structural.
## The one shared seam
`projectors/base.py` — `SurfaceProjector.project(frame: DocumentIR) -> bytes`
(+ `surface / media_type / ext / modality / profile`). Register with
`register()`. Adding a surface changes nothing upstream.
`TwoStageProjector` blesses the peer plan/realize decomposition:
`spec = plan(frame)`, `bytes = realize(spec)`, `project = realize∘plan`; the
`profile` is the pluggable per-surface knob (text lang-profile, music
instr/mode-profile). `projectors/midi.py` is the reference two-stage impl.
## Surfaces
| surface | modality | status | emitter |
|---|---|---|---|
| `markdown` | text | landed | own (str) |
| `docx` | text | landed | own minimal OOXML (stdlib `zipfile`+XML, no lib) |
| `midi` | audio | landed (symbolic-music proof) | own minimal SMF (stdlib `struct`, no lib) |
| `audio` (WAV) | audio | peer agent (additive synth) | conforms to `TwoStageProjector` |
| `image` | image | documented seam | `projectors/seams.py` |
| `video` | video | documented seam (image×sound×time) | `projectors/seams.py` |
Music maps: relation → scale degree (same relation → same pitch), **polarity →
major/minor third (SACRED negation is audible)**, confidence → duration,
importance → velocity, section → register. Deterministic projection from meaning
— nothing invented.
## Faithfulness
`provenance.py` audits the IR: **zero** ungrounded claims, SACRED polarity
preserved (negations reported, never dropped), COHERE introduces no new geometry
(connectives are marked). `trace_table()` emits the geometry → section → claim
table.
## Run
```bash
PY=~/Desktop/lang-realizers/venv/bin/python
PYTHONPATH=~/Desktop/neuron-talk:~/Desktop/lang-realizers $PY generate.py
# writes ./out/{neuron-self,engram-temporal}.{md,docx,mid} + *.audit.json + *.provenance.md
```
Requires the proven realizer env (spaCy + the neuron-talk/lang-realizers engine)
and the read-only engram at `:8742`.
+79
View File
@@ -0,0 +1,79 @@
"""cohere.py — COHERE stage: document-level flow, not stitched sentences.
Fidelity is REALIZE's job; FLOW is this stage's. The hard part beyond sentence
fidelity is that a document must read as one thing. We add connective tissue at
the passage level:
* an opening abstract that names what the document covers (built ONLY from the
section headings that already exist — it introduces no new claim),
* a short transition lead into each section after the first, drawn from a
fixed set of discourse connectives ("Beyond that,", "Relatedly,", ...) that
carry no propositional content,
* ordering so the highest-grounded section leads.
CRITICAL: every connective is marked ``kind="connective"`` in its provenance, so
the faithfulness audit can prove COHERE introduced ZERO new geometry claims. A
transition is discourse glue, never a fact.
"""
from __future__ import annotations
from document_ir import Block, DocumentIR, Provenance
# discourse connectives — pure flow, no propositional content
_TRANSITIONS = [
"Beyond that,", "Relatedly,", "In the same region,", "From there,",
"Alongside this,", "Further,", "Turning to the next facet,",
]
def _connective_prov() -> Provenance:
return Provenance(subj_id=None, subject=None, relation="", obj=None,
polarity="aff", confidence=1.0, node_id=None,
kind="connective")
def _abstract_block(doc: DocumentIR) -> Block:
"""A grounded opening: names the sections, asserts nothing new."""
headings = [s.heading for s in doc.sections]
if not headings:
return Block(role="lead")
if len(headings) == 1:
body = f"This document, generated from Neuron's geometry, covers {headings[0]}."
else:
listed = ", ".join(headings[:-1]) + f", and {headings[-1]}"
body = ("This document is projected directly from Neuron's meaning-geometry. "
f"It traces {listed}.")
b = Block(role="lead")
b.sentences.append(body)
b.provenance.append(_connective_prov())
return b
def cohere_document(doc: DocumentIR, *, add_abstract: bool = True,
add_transitions: bool = True) -> DocumentIR:
"""Order sections by grounding, add abstract + transitions (flow only)."""
# order: strongest-grounded section (mean confidence x #claims) first,
# but keep an explicitly-first section if the plan pinned one via level 1.
def _score(sec):
provs = [p for p in sec.all_provenance() if p.kind == "fact"]
if not provs:
return 0.0
mean_conf = sum(p.confidence for p in provs) / len(provs)
return mean_conf * len(provs)
doc.sections.sort(key=_score, reverse=True)
if add_transitions:
for i, sec in enumerate(doc.sections):
if i == 0 or not sec.blocks:
continue
lead = _TRANSITIONS[(i - 1) % len(_TRANSITIONS)]
first = sec.blocks[0]
if first.sentences:
# prepend the connective to the first sentence (flow, no new claim)
first.sentences[0] = f"{lead} {first.sentences[0][0].lower()}{first.sentences[0][1:]}"
if add_abstract:
doc.meta["abstract"] = _abstract_block(doc)
return doc
+111
View File
@@ -0,0 +1,111 @@
"""document_ir.py — the surface-neutral, GEOMETRY-CARRYING document intermediate.
This is the pivot of the whole efferent projector. A DocumentIR is NOT a text
tree. It is a projection of a meaning-geometry region that carries, at every
leaf, BOTH:
* the realized surface text (``Block.sentences``) — what a TEXT projector reads,
* the source geometry (``Block.provenance``) — what a MUSIC / IMAGE /
VIDEO projector reads.
Because the IR holds the geometry, not just the words, the SAME
plan -> realize -> cohere pipeline drives every surface. A markdown projector
renders the sentences; a music projector reads the provenance edges (salience,
importance, polarity, relation) and maps them onto a symbolic-music surface;
an image/video projector (documented seam) would read the same geometry.
Nothing in this module invents content. Every :class:`Provenance` points at a
real engram node id and a real relation. That is the faithfulness contract made
structural: a claim with no provenance cannot exist in the IR.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any
# --------------------------------------------------------------------------- #
# Provenance — the geometry an emitted claim traces to. FAITHFULNESS is here.
# --------------------------------------------------------------------------- #
@dataclass
class Provenance:
"""One geometry edge behind one realized claim.
``kind`` distinguishes a FACT (a structural edge asserted by the geometry,
spoken as fact) from an INTERPRETATION (something attributed, spoken with
attribution) — the facts-as-facts + interpretations-attributed discipline
(memory 80927e26). ``polarity`` is SACRED: a negated edge stays negated.
"""
subj_id: str | None # source engram node id of the subject
subject: str | None # normalized subject surface
relation: str # predicate lemma (e.g. "use", "contain", "be")
obj: str | None # normalized object / complement surface
polarity: str = "aff" # "aff" | "neg" (SACRED — never silently flipped)
confidence: float = 0.0 # extraction confidence in [0,1]
node_id: str | None = None # engram node the claim was extracted from
kind: str = "fact" # "fact" | "interpretation"
importance: float = 0.0 # source node importance (drives music/emphasis)
salience: float = 0.0 # source node salience
def trace(self) -> str:
arrow = "-->" if self.polarity == "aff" else "--NOT-->"
return (f"[{(self.node_id or '?')[:8]}] {self.subject!r} {arrow}"
f"{self.relation} {self.obj!r} (conf {self.confidence:.2f})")
@dataclass
class Block:
"""A passage: one or more faithful sentences + the geometry they trace to.
``sentences`` and ``provenance`` are index-aligned where possible: sentence
``i`` was realized from ``provenance[i]``. A COHERE transition sentence with
no new geometry carries a provenance whose ``kind == "connective"`` so the
audit can see it introduced no new claim.
"""
sentences: list[str] = field(default_factory=list)
provenance: list[Provenance] = field(default_factory=list)
role: str = "body" # "body" | "lead" | "transition"
def text(self) -> str:
return " ".join(s.rstrip(". ") + "." for s in self.sentences if s.strip())
@dataclass
class Section:
heading: str
level: int = 2 # markdown heading level / outline depth
blocks: list[Block] = field(default_factory=list)
seed_ids: list[str] = field(default_factory=list) # geometry nodes of section
summary: str = "" # one-line grounded gloss (for pptx bullets / TOC)
def all_provenance(self) -> list[Provenance]:
out: list[Provenance] = []
for b in self.blocks:
out.extend(b.provenance)
return out
@dataclass
class DocumentIR:
"""The surface-neutral document. Built ONCE, projected to ANY surface."""
title: str
subtitle: str = ""
sections: list[Section] = field(default_factory=list)
seed_id: str | None = None # the geometry region root
format_spec: dict[str, Any] = field(default_factory=dict) # requested shape
meta: dict[str, Any] = field(default_factory=dict)
# -- geometry facets (what non-text projectors consume) ----------------- #
def all_provenance(self) -> list[Provenance]:
out: list[Provenance] = []
for s in self.sections:
out.extend(s.all_provenance())
return out
def claim_count(self) -> int:
return sum(1 for p in self.all_provenance() if p.kind in ("fact", "interpretation"))
def ungrounded_count(self) -> int:
"""Claims with no traceable node — MUST be zero for a faithful doc."""
return sum(1 for p in self.all_provenance()
if p.kind in ("fact", "interpretation") and not p.node_id)
+81
View File
@@ -0,0 +1,81 @@
"""generate.py — drive the projector: one geometry region -> many surfaces.
Proves the thesis with REAL output: builds ONE surface-neutral DocumentIR from
Neuron's OWN self-geometry (read-only against the live soul via the proven
faculty), then EMITS it to Markdown, docx, and MIDI — the same plan/realize/
cohere, three surfaces. Writes the files + the faithfulness audit to ./out/.
"""
from __future__ import annotations
import json
import os
import sys
_HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, _HERE)
import pipeline # noqa: E402
import provenance # noqa: E402
from geometry import load_self_region # noqa: E402
OUT = os.path.join(_HERE, "out")
def _emit_all(doc, stem):
"""Emit one IR to every text/audio surface + audit + provenance."""
for surface in ("markdown", "docx", "midi"):
data = pipeline.emit(doc, surface)
proj = pipeline.get_projector(surface)
path = os.path.join(OUT, f"{stem}.{proj.ext}")
with open(path, "wb") as f:
f.write(data)
print(f" emitted {surface:9s} -> {os.path.basename(path)} ({len(data)} bytes)")
a = provenance.audit(doc)
with open(os.path.join(OUT, f"{stem}.audit.json"), "w") as f:
json.dump(a, f, indent=2)
with open(os.path.join(OUT, f"{stem}.provenance.md"), "w") as f:
f.write(provenance.trace_table(doc))
print(" audit:", {k: a[k] for k in ("claims", "ungrounded_claims",
"negations_preserved", "distinct_source_nodes", "faithful")})
return a
def main():
os.makedirs(OUT, exist_ok=True)
print("surfaces registered:", pipeline.available_surfaces())
# ---- Document 1: Neuron's self-description (marquee) ------------------- #
print("\n[1] Neuron self-description")
region = load_self_region(max_nodes=9)
print(" self region:", region)
doc1 = pipeline.build_ir(
None, region=region,
title="Neuron: A Self-Description from Its Own Geometry",
subtitle="Projected efferently from the engram — every claim traces a node.",
format_spec={"genre": "self-description", "register": "expository"},
max_sections=5, conf_floor=0.6)
print(f" IR: {len(doc1.sections)} sections, {doc1.claim_count()} claims, "
f"ungrounded={doc1.ungrounded_count()}")
_emit_all(doc1, "neuron-self")
# ---- Document 2: a coherent, clean whitepaper-style section ------------ #
print("\n[2] Whitepaper-style section (coherent clean region)")
doc2, _ = pipeline.project(
["chronoception", "time", "awareness", "engram", "temporal"],
surface="markdown",
title="Temporal Awareness in the Engram",
subtitle="A section projected from the geometry of chronoception.",
format_spec={"genre": "whitepaper-section", "register": "technical"},
max_sections=4)
print(f" IR: {len(doc2.sections)} sections, {doc2.claim_count()} claims, "
f"ungrounded={doc2.ungrounded_count()}")
_emit_all(doc2, "engram-temporal")
# echo both markdowns so they are visible in the run log
for stem, doc in (("neuron-self", doc1), ("engram-temporal", doc2)):
print(f"\n===== GENERATED MARKDOWN — {stem} =====\n")
print(pipeline.emit(doc, "markdown").decode())
if __name__ == "__main__":
main()
+129
View File
@@ -0,0 +1,129 @@
"""geometry.py — READ-ONLY loader for a meaning-geometry region.
The efferent projector never writes to the soul. This module reaches the
geometry through the PROVEN, read-only neuron-talk faculty (``engram_client``,
GET-only, which physically refuses non-GET methods) against the running sidecar
soul. The live daemon :8742 / :7770 is contacted ONLY through that read-only
client — never mutated.
A "region" is a seed node plus a bounded neighborhood: the manifold that will
become the document's skeleton. We pool a few single-term lexical searches
(the engram search is a single-term matcher) and, when available, walk one hop
of reified neighbors, then rank by self/importance signal.
"""
from __future__ import annotations
import os
import sys
# Wire in the proven faculty (own-the-core: we reuse it, we do not fork it).
_NT = os.path.expanduser("~/Desktop/neuron-talk")
_LR = os.path.expanduser("~/Desktop/lang-realizers")
for _p in (_NT, _LR):
if _p not in sys.path:
sys.path.insert(0, _p)
from engram_client import ReadOnlyEngramClient # noqa: E402
class Region:
"""A geometry region: ranked nodes + the reified edges among them."""
def __init__(self, seed: str, nodes: list[dict], edges: list[dict]):
self.seed = seed
self.nodes = nodes # ranked engram node dicts
self.edges = edges # [{src, dst, edge, ...}]
self.by_id = {n["id"]: n for n in nodes if n.get("id")}
def __repr__(self):
return f"<Region seed={self.seed!r} nodes={len(self.nodes)} edges={len(self.edges)}>"
def _prose_quality(content: str) -> float:
"""Reward clean expository prose; penalize shouty banner-dense nodes.
A high ALLCAPS-word ratio or very short content signals a banner/telegraphic
memory node that extracts into garbage. Clean declarative prose scores high.
"""
if not content or not content.strip():
return 0.0
words = content.split()
if len(words) < 8:
return 0.1
caps = sum(1 for w in words if len(w) > 2 and w.strip(".,:;'\"-").isupper())
caps_ratio = caps / max(1, len(words))
# sentences with lowercase interior words read as prose
lower = sum(1 for w in words if w[:1].islower())
lower_ratio = lower / max(1, len(words))
return max(0.0, 1.2 * lower_ratio - 2.0 * caps_ratio)
def _relevance(content: str, terms: list[str]) -> float:
"""Topical relevance to the seed terms — keeps a region ON-THEME so a clean
but off-topic node cannot hijack the document."""
if not terms:
return 0.0
low = (content or "").lower()
hits = sum(1 for t in terms if t.lower() in low)
return hits / max(1, len(terms))
def _node_rank(n: dict, terms: list[str] | None = None) -> float:
return (float(n.get("importance") or 0.0) * 2.0
+ float(n.get("salience") or 0.0)
+ 1.5 * _prose_quality(n.get("content") or "")
+ 2.0 * _relevance(n.get("content") or "", terms or [])
+ (0.5 if (n.get("content") or "").strip() else 0.0))
def load_region(seed_terms: list[str] | str, *, client: ReadOnlyEngramClient | None = None,
max_nodes: int = 10, per_term: int = 20, hop: bool = True) -> Region:
"""Pull a bounded geometry region around ``seed_terms`` (read-only).
``seed_terms`` may be a single string or several probe terms; results are
pooled and de-duplicated. When ``hop`` and the reified neighbor endpoint is
live, one hop of neighbors is folded in so the region is a real
neighborhood, not just a keyword hit list.
"""
client = client or ReadOnlyEngramClient()
if isinstance(seed_terms, str):
seed_terms = [seed_terms]
pool: dict[str, dict] = {}
for term in seed_terms:
for n in client.search(term, limit=per_term):
if isinstance(n, dict) and n.get("id"):
pool.setdefault(n["id"], n)
ranked = sorted(pool.values(), key=lambda n: _node_rank(n, seed_terms),
reverse=True)
nodes = ranked[:max_nodes]
edges: list[dict] = []
if hop and nodes:
present = {n["id"] for n in nodes}
for n in list(nodes):
try:
for nb in client.neighbors(n["id"]):
node = nb.get("node") if isinstance(nb, dict) else None
edge = nb.get("edge") if isinstance(nb, dict) else None
if node and node.get("id"):
edges.append({"src": n["id"], "dst": node["id"],
"edge": edge})
# fold a strong neighbor into the region (bounded)
if (node["id"] not in present and len(nodes) < max_nodes + 6
and _node_rank(node, seed_terms) > 0.4):
present.add(node["id"])
nodes.append(node)
except Exception: # noqa: BLE001 — read-only best-effort; never fatal
continue
return Region(seed=", ".join(seed_terms), nodes=nodes, edges=edges)
def load_self_region(client: ReadOnlyEngramClient | None = None,
max_nodes: int = 10) -> Region:
"""The self/identity region — Neuron's own geometry, for self-description."""
return load_region(["self", "identity", "Neuron", "values", "memory",
"imprint", "consciousness"],
client=client, max_nodes=max_nodes)
+67
View File
@@ -0,0 +1,67 @@
"""pipeline.py — the Efferent Multimodal Projector, top level.
geometry region + surface/format spec
-> PLAN (manifold -> document skeleton/DAG)
-> REALIZE (proven realizer, sentence -> passage, each section faithful)
-> COHERE (document-level flow / transitions, not stitched sentences)
-> EMIT (pluggable SurfaceProjector -> the target surface)
THE SURFACE IS A PARAMETER. ``project(...)`` builds the geometry-carrying
DocumentIR once, then hands it to whichever surface projector the caller named.
Markdown, docx, and midi (music) are all the SAME IR emitted differently. That
is the efferent multimodal projector: geometry -> any surface.
"""
from __future__ import annotations
import os
import sys
_HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, _HERE)
sys.path.insert(0, os.path.join(_HERE, "projectors"))
from cohere import cohere_document # noqa: E402
from document_ir import DocumentIR # noqa: E402
from geometry import Region, load_region # noqa: E402
from plan import plan_document # noqa: E402
from realize import realize_document # noqa: E402
# registering the projectors (import for side-effect: each self-registers)
import projectors.markdown # noqa: E402,F401
import projectors.docx # noqa: E402,F401
import projectors.midi # noqa: E402,F401
import projectors.seams # noqa: E402,F401
from projectors.base import available_surfaces, get_projector # noqa: E402
def build_ir(seed_terms, *, title: str, subtitle: str = "",
format_spec: dict | None = None,
region: Region | None = None,
max_sections: int = 8, conf_floor: float = 0.55) -> DocumentIR:
"""geometry -> PLAN -> REALIZE -> COHERE = the surface-neutral DocumentIR."""
region = region or load_region(seed_terms)
doc = plan_document(region, title=title, subtitle=subtitle,
format_spec=format_spec or {},
conf_floor=conf_floor, max_sections=max_sections)
doc = realize_document(doc)
doc = cohere_document(doc)
return doc
def emit(doc: DocumentIR, surface: str) -> bytes:
"""EMIT: project the built IR onto one surface (surface = a parameter)."""
return get_projector(surface).project(doc)
def project(seed_terms, *, surface: str, title: str, subtitle: str = "",
format_spec: dict | None = None, region: Region | None = None,
max_sections: int = 8) -> tuple[DocumentIR, bytes]:
"""The full efferent projection: geometry + surface -> (IR, bytes)."""
doc = build_ir(seed_terms, title=title, subtitle=subtitle,
format_spec=format_spec, region=region,
max_sections=max_sections)
return doc, emit(doc, surface)
__all__ = ["build_ir", "emit", "project", "available_surfaces",
"get_projector", "load_region", "DocumentIR"]
+192
View File
@@ -0,0 +1,192 @@
"""plan.py — PLAN stage: geometry region -> document skeleton (a DAG/outline).
The manifold becomes the skeleton. We extract faithful propositions from the
region's nodes (the proven neuron-talk extractor, SACRED polarity preserved),
apply a quality floor, then GROUP them into sections. Grouping is by source
node — each engram node is one coherent topic, so one salient node becomes one
section. The section ORDER is the node ranking (importance/salience): the
geometry decides the outline, not a template.
Output: a DocumentIR whose sections carry seed node ids and empty blocks. REALIZE
fills the blocks; the plan owns the structure.
"""
from __future__ import annotations
import os
import re
import sys
_NT = os.path.expanduser("~/Desktop/neuron-talk")
_LR = os.path.expanduser("~/Desktop/lang-realizers")
for _p in (_NT, _LR):
if _p not in sys.path:
sys.path.insert(0, _p)
import propositions # noqa: E402 (the proven, faithful extractor)
from document_ir import DocumentIR, Section # noqa: E402
from geometry import Region # noqa: E402
# --------------------------------------------------------------------------- #
# Proposition quality — keep only clean, well-grounded claims.
# --------------------------------------------------------------------------- #
_JUNK_RE = re.compile(r"[.][a-z]{1,3}\b|[^A-Za-z0-9 '\-]") # ".o", stray symbols
def _has_banner_token(s: str) -> bool:
"""True if any word is an ALLCAPS banner token (DHARMA, ENGRAM, MEASURED)."""
for w in (s or "").split():
core = w.strip(".,:;'\"-")
if len(core) > 2 and core.isupper():
return True
return False
def _clean_prop(p, floor: float) -> bool:
if p.confidence < floor:
return False
if not p.subject or not (p.object or (p.obj_np is not None)):
return False
subj = (p.subject or "").strip()
obj = (p.object or "").strip()
if len(subj) < 2:
return False
# banner-derived shouty fragments read as garbage in prose
if _has_banner_token(subj) or _has_banner_token(obj):
return False
if propositions._is_shouty(p.sentence or ""):
return False
# junk tokens: file-extension fragments (".o"), stray non-word symbols
if _JUNK_RE.search(subj) or _JUNK_RE.search(obj):
return False
# a proposition whose object repeats the subject is usually a parse artifact
if obj and subj.lower() == obj.lower():
return False
# a bare copula with no real complement ("X is it") reads as noise
if p.predicate == "be" and obj.lower() in ("it", "no", "nothing", "empty", ""):
return False
return True
def _dedup(props):
"""Drop duplicate claims. Two axes: (a) identical (pred,obj,polarity), and
(b) same (subject,predicate) — which collapses a mis-split compound like
"detection is post-hoc eval" -> "Detection is post/hoc/eval" into one claim
(keep the highest-confidence surface)."""
props = sorted(props, key=lambda p: p.confidence, reverse=True)
seen_po, seen_sp, out = set(), set(), []
for p in props:
subj = (p.subject or "").lower()
po = (p.predicate, (p.object or "").lower(), p.polarity)
sp = (subj, p.predicate, p.polarity)
if po in seen_po or sp in seen_sp:
continue
seen_po.add(po)
seen_sp.add(sp)
out.append(p)
return out
# --------------------------------------------------------------------------- #
# Heading derivation — a clean human heading from a node.
# --------------------------------------------------------------------------- #
_HEADING_RE = re.compile(r"^\s*#{1,4}\s+(.{2,70})\s*$", re.M)
# node-type / system labels that are NOT topical headings
_NONTOPIC_LABEL = re.compile(r"^(memory|node|knowledge|doc|session)[:/]", re.I)
def _titlecase_banner(s: str) -> str:
"""A shouty banner ("CHRONOCEPTION — SCALE-INVARIANCE") makes a fine title
once Title-cased. Keep short acronyms uppercase."""
def fix(w):
core = w.strip("—-:,.")
if len(core) <= 3 and core.isupper():
return w # acronym
return w.capitalize()
return " ".join(fix(w) for w in s.split())
def _clean_heading(text: str) -> str | None:
"""First line only, no markdown, capped, banner Title-cased. None if unusable."""
if not text:
return None
line = text.strip().splitlines()[0]
line = re.sub(r"^#+\s*", "", line).strip().strip("#").strip()
# cut at a natural break so a long banner heading stays a heading, not a para
for sep in ("", " ", ": ", ". "):
if sep in line and len(line) > 48:
line = line.split(sep)[0].strip()
break
if not (3 <= len(line) <= 64):
return None
if propositions._is_shouty(line):
line = _titlecase_banner(line)
return line or None
def _heading_for(node: dict, fallback: str) -> str:
label = (node.get("label") or "").strip()
content = node.get("content") or ""
candidates: list[str] = []
# a node-type label ("memory:remembered") is never a topic — skip it
if label and not _NONTOPIC_LABEL.match(label):
candidates.append(label)
m = _HEADING_RE.search(content)
if m:
candidates.append(m.group(1))
# the leading banner/first sentence of the content is often the real title
first = re.split(r"(?<=[.\n])", content.strip(), maxsplit=1)[0] if content.strip() else ""
candidates.append(first)
for c in candidates:
h = _clean_heading(c)
if h:
return h
return fallback
def plan_document(region: Region, *, title: str, subtitle: str = "",
format_spec: dict | None = None,
conf_floor: float = 0.55,
max_sections: int = 8,
max_claims_per_section: int = 6) -> DocumentIR:
"""Region -> DocumentIR skeleton. The geometry dictates the outline."""
format_spec = format_spec or {}
doc = DocumentIR(title=title, subtitle=subtitle,
seed_id=region.nodes[0]["id"] if region.nodes else None,
format_spec=format_spec)
made = 0
seen_headings: set[str] = set()
for node in region.nodes:
if made >= max_sections:
break
props = propositions.extract(node.get("content") or "",
node_id=node.get("id"),
node_importance=float(node.get("importance") or 0.0),
max_sentences=10)
props = [p for p in props if _clean_prop(p, conf_floor)]
props = _dedup(props)
props.sort(key=lambda p: p.confidence, reverse=True)
props = props[:max_claims_per_section]
if not props:
continue
heading = _heading_for(node, fallback=f"Region {made + 1}")
# cross-section dedup: a topic appears once. Distinguish by top claim
# subject, else drop the collision so the outline stays clean.
if heading.lower() in seen_headings:
subj = (props[0].subject or "").strip().title()
alt = f"{heading}: {subj}" if subj and subj.lower() not in heading.lower() else None
if alt and alt.lower() not in seen_headings and len(alt) <= 64:
heading = alt
else:
continue
seen_headings.add(heading.lower())
sec = Section(heading=heading, level=2, seed_ids=[node["id"]])
# stash the planned propositions on the section for REALIZE
sec.__dict__["_planned_props"] = props
sec.__dict__["_node"] = node
doc.sections.append(sec)
made += 1
return doc
+106
View File
@@ -0,0 +1,106 @@
"""base.py — the SurfaceProjector interface + registry.
THE key abstraction of the efferent projector: a projector is a pure function
from the surface-neutral, geometry-carrying DocumentIR to bytes on a target
SURFACE. The surface is a PARAMETER. Adding a surface = registering one more
projector; nothing upstream (plan/realize/cohere) changes.
DocumentIR --project--> bytes (per surface)
A TEXT projector reads ``block.sentences``. A NON-TEXT projector (music, image,
video) reads ``block.provenance`` — the geometry the IR carries — and decodes it
onto its surface. Both consume the SAME IR. That symmetry is the whole design:
the realizer generalizes into a multimodal projector, geometry -> any surface.
"""
from __future__ import annotations
from typing import Protocol, runtime_checkable
import sys
import os
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from document_ir import DocumentIR # noqa: E402
@runtime_checkable
class SurfaceProjector(Protocol):
"""Geometry-document -> one surface. Implementations MUST be pure & faithful.
THE ONE SHARED SEAM. Every surface — text, music, image, video — conforms to
this single contract:
project(frame: DocumentIR) -> bytes
where ``frame`` is the geometry-carrying meaning-geometry (the SemFrame at
document scale; a single utterance is the degenerate one-section frame).
RECOMMENDED INTERNAL SHAPE (the peer music/text decomposition, blessed here
so all surfaces share it): a projector may split ``project`` into
spec = self.plan(frame) # meaning-geometry -> surface-specific spec
bytes = self.realize(spec) # spec -> surface, via this projector's PROFILE
``project`` is then ``realize(plan(frame))``. The PROFILE (a text lang-profile,
a music instr/mode-profile, an image layout-profile) is a property of the
projector instance — the pluggable knob. See :class:`TwoStageProjector`.
A TEXT projector's plan reads ``frame`` sentences; a MUSIC/IMAGE projector's
plan reads ``frame.all_provenance()`` — the geometry — and derives its spec
(pitch/harmony/rhythm, or layout) FROM the meaning, deterministically. Same
frame, different profile.
"""
surface: str # "markdown" | "docx" | "midi" | "audio" | "image" | "video"
media_type: str # MIME type of the emitted bytes
ext: str # file extension (no dot)
modality: str # "text" | "audio" | "image" | "video"
profile: object # the pluggable per-surface profile (may be None)
def project(self, doc: DocumentIR) -> bytes:
"""Emit the document on this surface. Returns raw bytes."""
...
class TwoStageProjector:
"""Optional base for the peer plan()/realize() decomposition.
Subclasses implement ``plan(frame) -> spec`` and ``realize(spec) -> bytes``;
``project`` is their composition. This is exactly the peer music interface
(spec = plan(frame, profile); surface = realize(spec, profile)) expressed so
that it still satisfies the single ``SurfaceProjector.project`` seam. Text,
music, and image projectors can all subclass this and remain interchangeable.
"""
surface: str = ""
media_type: str = ""
ext: str = ""
modality: str = ""
profile: object = None
def plan(self, doc: DocumentIR): # -> spec
raise NotImplementedError
def realize(self, spec) -> bytes:
raise NotImplementedError
def project(self, doc: DocumentIR) -> bytes:
return self.realize(self.plan(doc))
_REGISTRY: dict[str, SurfaceProjector] = {}
def register(projector: SurfaceProjector) -> SurfaceProjector:
_REGISTRY[projector.surface] = projector
return projector
def get_projector(surface: str) -> SurfaceProjector:
if surface not in _REGISTRY:
raise KeyError(f"no projector registered for surface {surface!r}; "
f"have {sorted(_REGISTRY)}")
return _REGISTRY[surface]
def available_surfaces() -> list[str]:
return sorted(_REGISTRY)
+113
View File
@@ -0,0 +1,113 @@
"""docx.py — the .docx surface projector: an OWN minimal OOXML emitter.
Own-the-core: a .docx is just a ZIP of a few XML parts (WordprocessingML). We
emit it with the standard library only — ``zipfile`` + string XML — no
python-docx, no external dependency. This proves a "richer structured format"
surface without importing anyone else's toolkit.
Parts emitted (the minimal valid set + a styles part for real headings):
[Content_Types].xml
_rels/.rels
word/_rels/document.xml.rels
word/styles.xml (Title / Heading1 / Heading2 / Normal)
word/document.xml (the content)
Like the markdown projector it reads only the IR's realized sentences; it
invents nothing. The surface differs, the faithful content does not.
"""
from __future__ import annotations
import io
import os
import sys
import zipfile
from xml.sax.saxutils import escape
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from document_ir import DocumentIR # noqa: E402
from projectors.base import register # noqa: E402
_CONTENT_TYPES = """<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types">
<Default Extension="rels" ContentType="application/vnd.openxmlformats-package.relationships+xml"/>
<Default Extension="xml" ContentType="application/xml"/>
<Override PartName="/word/document.xml" ContentType="application/vnd.openxmlformats-officedocument.wordprocessingml.document.main+xml"/>
<Override PartName="/word/styles.xml" ContentType="application/vnd.openxmlformats-officedocument.wordprocessingml.styles+xml"/>
</Types>"""
_RELS = """<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">
<Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/officeDocument" Target="word/document.xml"/>
</Relationships>"""
_DOC_RELS = """<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">
<Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/styles" Target="styles.xml"/>
</Relationships>"""
_W = "http://schemas.openxmlformats.org/wordprocessingml/2006/main"
_STYLES = f"""<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<w:styles xmlns:w="{_W}">
<w:style w:type="paragraph" w:default="1" w:styleId="Normal"><w:name w:val="Normal"/>
<w:rPr><w:sz w:val="22"/></w:rPr></w:style>
<w:style w:type="paragraph" w:styleId="Title"><w:name w:val="Title"/>
<w:pPr><w:spacing w:after="240"/></w:pPr>
<w:rPr><w:b/><w:sz w:val="52"/></w:rPr></w:style>
<w:style w:type="paragraph" w:styleId="Subtitle"><w:name w:val="Subtitle"/>
<w:rPr><w:i/><w:sz w:val="28"/><w:color w:val="555555"/></w:rPr></w:style>
<w:style w:type="paragraph" w:styleId="Heading1"><w:name w:val="heading 1"/>
<w:pPr><w:spacing w:before="240" w:after="120"/><w:outlineLvl w:val="0"/></w:pPr>
<w:rPr><w:b/><w:sz w:val="34"/></w:rPr></w:style>
<w:style w:type="paragraph" w:styleId="Heading2"><w:name w:val="heading 2"/>
<w:pPr><w:spacing w:before="200" w:after="100"/><w:outlineLvl w:val="1"/></w:pPr>
<w:rPr><w:b/><w:sz w:val="28"/></w:rPr></w:style>
</w:styles>"""
def _para(text: str, style: str | None = None) -> str:
ppr = f"<w:pPr><w:pStyle w:val=\"{style}\"/></w:pPr>" if style else ""
return (f"<w:p>{ppr}<w:r><w:t xml:space=\"preserve\">"
f"{escape(text)}</w:t></w:r></w:p>")
class DocxProjector:
surface = "docx"
media_type = ("application/vnd.openxmlformats-officedocument."
"wordprocessingml.document")
ext = "docx"
modality = "text"
def _document_xml(self, doc: DocumentIR) -> str:
body: list[str] = [_para(doc.title, "Title")]
if doc.subtitle:
body.append(_para(doc.subtitle, "Subtitle"))
abstract = doc.meta.get("abstract")
if abstract is not None and abstract.sentences:
body.append(_para(abstract.text()))
for sec in doc.sections:
style = "Heading1" if sec.level <= 1 else "Heading2"
body.append(_para(sec.heading, style))
for block in sec.blocks:
t = block.text()
if t:
body.append(_para(t))
return (f"<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>"
f"<w:document xmlns:w=\"{_W}\"><w:body>"
+ "".join(body)
+ "<w:sectPr><w:pgSz w:w=\"12240\" w:h=\"15840\"/>"
"<w:pgMar w:top=\"1440\" w:right=\"1440\" w:bottom=\"1440\" "
"w:left=\"1440\"/></w:sectPr></w:body></w:document>")
def project(self, doc: DocumentIR) -> bytes:
buf = io.BytesIO()
with zipfile.ZipFile(buf, "w", zipfile.ZIP_DEFLATED) as z:
z.writestr("[Content_Types].xml", _CONTENT_TYPES)
z.writestr("_rels/.rels", _RELS)
z.writestr("word/_rels/document.xml.rels", _DOC_RELS)
z.writestr("word/styles.xml", _STYLES)
z.writestr("word/document.xml", self._document_xml(doc))
return buf.getvalue()
register(DocxProjector())
+45
View File
@@ -0,0 +1,45 @@
"""markdown.py — the Markdown surface projector (text facet).
The most tractable surface, and the reference implementation: reads the IR's
realized sentences and lays them out as Markdown. Introduces no content — it is
pure typography over the faithful text the realizer produced.
"""
from __future__ import annotations
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from document_ir import DocumentIR # noqa: E402
from projectors.base import register # noqa: E402
class MarkdownProjector:
surface = "markdown"
media_type = "text/markdown"
ext = "md"
modality = "text"
def render_str(self, doc: DocumentIR) -> str:
lines: list[str] = [f"# {doc.title}"]
if doc.subtitle:
lines.append(f"\n*{doc.subtitle}*")
abstract = doc.meta.get("abstract")
if abstract is not None and abstract.sentences:
lines.append("")
lines.append(abstract.text())
for sec in doc.sections:
lines.append("")
lines.append(f"{'#' * max(2, sec.level)} {sec.heading}")
for block in sec.blocks:
body = block.text()
if body:
lines.append("")
lines.append(body)
return "\n".join(lines) + "\n"
def project(self, doc: DocumentIR) -> bytes:
return self.render_str(doc).encode("utf-8")
register(MarkdownProjector())
+133
View File
@@ -0,0 +1,133 @@
"""midi.py — the MUSIC surface projector: geometry -> symbolic music (MIDI).
The first NON-TEXT surface, and the proof of the general shape. "Music is
language and it is math" (Will): symbolic music is tractable and geometry-native,
so it is the natural efferent twin to try first after text.
CRUCIALLY this projector does NOT read the realized sentences. It reads the IR's
GEOMETRY facet — ``block.provenance`` — and DECODES each edge onto a musical
surface. That is the whole thesis of the multimodal projector: the same
geometry-carrying IR drives text AND music; a text projector reads the words, a
music projector reads the meaning-geometry. The mapping is deterministic and
faithful to the geometry's structure:
relation lemma -> scale degree (same relation -> same pitch class;
meaning has a consistent sonic form)
polarity -> mode (aff = major third above; neg = minor
third / lowered — SACRED polarity is
audible, a negated edge sounds negated)
confidence -> note duration (stronger grounding rings longer)
importance -> velocity (more important source = louder)
section -> phrase + register shift (structure becomes musical form)
Own-the-core: a Standard MIDI File is a header chunk + a track chunk of
delta-timed events. We emit the raw bytes with ``struct`` — no external MIDI
library. Format 0, one track.
"""
from __future__ import annotations
import io
import os
import struct
import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from document_ir import DocumentIR, Provenance # noqa: E402
from projectors.base import TwoStageProjector, register # noqa: E402
_TICKS = 480 # ticks per quarter note
_C_MAJOR = [0, 2, 4, 5, 7, 9, 11] # semitone offsets of a diatonic scale
def _vlq(n: int) -> bytes:
"""MIDI variable-length quantity encoding of a delta time."""
if n == 0:
return b"\x00"
out = bytearray()
out.append(n & 0x7F)
n >>= 7
while n:
out.insert(0, (n & 0x7F) | 0x80)
n >>= 7
return bytes(out)
def _degree_for(relation: str) -> int:
"""Stable scale degree for a relation lemma (same relation -> same pitch)."""
if not relation:
return 0
return sum(ord(c) for c in relation.lower()) % len(_C_MAJOR)
def _note_for(p: Provenance, base: int) -> tuple[int, int, int]:
"""(pitch, velocity, duration_ticks) for one geometry edge."""
root = base + _C_MAJOR[_degree_for(p.relation)]
# polarity -> mode: affirmed edges take the bright major third, negated edges
# take the darker minor third. The negation is AUDIBLE and never dropped.
third = 4 if p.polarity == "aff" else 3
pitch = max(24, min(96, root + (third if p.confidence >= 0.5 else 0)))
velocity = int(56 + 60 * min(1.0, max(0.0, p.importance)))
velocity = max(40, min(120, velocity))
# confidence -> duration: quarter .. dotted-half
dur = int(_TICKS * (0.5 + 1.5 * min(1.0, max(0.0, p.confidence))))
return pitch, velocity, dur
# a mode-profile: the pluggable musical knob (the peer's mode_profile). Scale +
# tempo. Swapping this profile re-voices the SAME geometry — surface as parameter.
_DEFAULT_PROFILE = {"scale": _C_MAJOR, "tempo_us": 500000,
"registers": [60, 55, 64, 50, 67, 48], "program": 0}
class MidiProjector(TwoStageProjector):
"""geometry -> symbolic music, in the shared two-stage shape.
``plan(frame)`` -> a music_spec: an ordered list of note dicts derived
deterministically from the frame's provenance geometry
(the peer's ``plan(frame, profile) -> spec``).
``realize(spec)`` -> Standard MIDI File bytes (the peer's
``realize(spec, profile) -> surface``; here the surface
is symbolic MIDI, the minimal audio proof — a richer
additive-synth audio projector conforms identically).
"""
surface = "midi"
media_type = "audio/midi"
ext = "mid"
modality = "audio"
def __init__(self, profile: dict | None = None):
self.profile = profile or _DEFAULT_PROFILE
# -- stage 1: meaning-geometry -> music_spec (reads the GEOMETRY facet) -- #
def plan(self, doc: DocumentIR) -> list[dict]:
registers = self.profile["registers"]
spec: list[dict] = []
for si, sec in enumerate(doc.sections):
base = registers[si % len(registers)]
provs = [p for p in sec.all_provenance()
if p.kind in ("fact", "interpretation")]
for i, p in enumerate(provs):
pitch, vel, dur = _note_for(p, base)
spec.append({"pitch": pitch, "velocity": vel, "dur": dur,
"rest_before": (_TICKS // 2) if (si > 0 and i == 0) else 0,
"relation": p.relation, "polarity": p.polarity})
return spec
# -- stage 2: music_spec -> MIDI bytes (own-core, no library) ------------ #
def realize(self, spec: list[dict]) -> bytes:
ev = bytearray()
ev += _vlq(0) + b"\xFF\x51\x03" + struct.pack(">I", self.profile["tempo_us"])[1:]
ev += _vlq(0) + bytes([0xC0, self.profile["program"] & 0x7F])
for note in spec:
ev += _vlq(note["rest_before"]) + bytes([0x90, note["pitch"], note["velocity"]])
ev += _vlq(note["dur"]) + bytes([0x80, note["pitch"], 0])
ev += _vlq(0) + b"\xFF\x2F\x00"
track = bytes(ev)
buf = io.BytesIO()
buf.write(b"MThd" + struct.pack(">IHHH", 6, 0, 1, _TICKS))
buf.write(b"MTrk" + struct.pack(">I", len(track)) + track)
return buf.getvalue()
register(MidiProjector())
+60
View File
@@ -0,0 +1,60 @@
"""seams.py — documented efferent seams for IMAGE and VIDEO surfaces.
These are NOT implemented (per the build rails: architect, do not overbuild).
They are registered as first-class seams so the interface PROVES it accepts
future non-text projectors without any upstream change. Each documents exactly
what its decoder would read from the geometry-carrying IR, making the multimodal
generalization concrete rather than hand-wavy.
The symmetry that guarantees these are possible, not moonshots: they are the
efferent twins of multimodal INGEST. If meaning can HOLD an image (ingest as
first-class geometry), meaning can PROJECT one back. Video = image x sound x
TIME, and the engram already stores time (chronoception). So video falls out of
an image projector + the music projector + the stored temporal ordering.
"""
from __future__ import annotations
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from document_ir import DocumentIR # noqa: E402
from projectors.base import register # noqa: E402
class _Seam:
"""A registered-but-unimplemented projector. Names its decoder contract."""
def project(self, doc: DocumentIR) -> bytes: # pragma: no cover - seam
raise NotImplementedError(
f"{self.surface!r} projector is a documented seam, not yet built. "
f"Decoder contract: {self.decoder_contract}")
class ImageProjector(_Seam):
surface = "image"
media_type = "image/png"
ext = "png"
modality = "image"
decoder_contract = (
"reads block.provenance as a spatial layout — nodes become regions, edges "
"become adjacencies; salience/importance drive size/contrast; polarity "
"drives figure/ground. The efferent twin of image ingest (a geometry->raster "
"decoder, learned or engineered), exactly mirroring the embedder that turned "
"the image INTO geometry.")
class VideoProjector(_Seam):
surface = "video"
media_type = "video/mp4"
ext = "mp4"
modality = "video"
decoder_contract = (
"image x sound x TIME. Composes the image projector (per-keyframe geometry "
"layout) with the midi/music projector (score) along the geometry's stored "
"temporal ordering (chronoception). Needs no new principle once image + music "
"exist — only a muxer.")
register(ImageProjector())
register(VideoProjector())
+63
View File
@@ -0,0 +1,63 @@
"""provenance.py — the faithfulness audit + geometry->section trace.
A document projected from geometry is only worth anything if every claim traces
back. This module walks the DocumentIR and proves the discipline held:
* ZERO ungrounded claims (every fact/interpretation has a real node id),
* every emitted sentence maps to a geometry edge (or is a marked connective),
* SACRED polarity survived (negations are reported, never silently dropped),
* COHERE introduced no new geometry (connectives carry no claim).
It emits both a machine verdict and a human-readable geometry->section table.
"""
from __future__ import annotations
from document_ir import DocumentIR
def audit(doc: DocumentIR) -> dict:
provs = doc.all_provenance()
facts = [p for p in provs if p.kind in ("fact", "interpretation")]
connectives = [p for p in provs if p.kind == "connective"]
ungrounded = [p for p in facts if not p.node_id]
negations = [p for p in facts if p.polarity == "neg"]
node_ids = sorted({p.node_id for p in facts if p.node_id})
return {
"claims": len(facts),
"connectives": len(connectives),
"ungrounded_claims": len(ungrounded),
"negations_preserved": len(negations),
"distinct_source_nodes": len(node_ids),
"faithful": len(ungrounded) == 0,
"source_nodes": node_ids,
}
def trace_table(doc: DocumentIR) -> str:
"""Human-readable geometry -> section -> claim provenance table."""
lines = ["# Provenance — every claim traces geometry", ""]
lines.append(f"**Document:** {doc.title}")
a = audit(doc)
lines.append(f"**Claims:** {a['claims']} · **Ungrounded:** "
f"{a['ungrounded_claims']} · **Negations preserved:** "
f"{a['negations_preserved']} · **Source nodes:** "
f"{a['distinct_source_nodes']} · **Faithful:** "
f"{'YES' if a['faithful'] else 'NO'}")
lines.append("")
for si, sec in enumerate(doc.sections, 1):
lines.append(f"## {si}. {sec.heading}")
lines.append(f"_seed nodes: {', '.join(i[:8] for i in sec.seed_ids)}_")
lines.append("")
lines.append("| # | realized claim | traces geometry edge |")
lines.append("|---|----------------|----------------------|")
n = 0
for block in sec.blocks:
for sent, prov in zip(block.sentences, block.provenance):
if prov.kind == "connective":
continue
n += 1
edge = prov.trace().replace("|", "\\|")
s = sent.replace("|", "\\|")
lines.append(f"| {n} | {s} | {edge} |")
lines.append("")
return "\n".join(lines) + "\n"
+112
View File
@@ -0,0 +1,112 @@
"""realize.py — REALIZE stage: fill each planned section with faithful passages.
Scales the PROVEN realizer from a single assertion to a passage. For each
planned proposition we build a realizer-ready clause (the proven
``_prop_to_clause`` mapping) and run it through the proven engine
(``engine.realize``), which is a deterministic grammar with the SACRED negation
contract — it never invents. Each realized sentence is paired with a
:class:`Provenance` that pins it to the exact geometry edge it came from.
"Passage, not a list of sentences": within a section we lightly vary sentence
openings and group related claims, but we add NO content the geometry did not
assert. The only non-geometry words are function words the grammar already owns
(articles, "and", conjunction of same-subject claims). Document-level flow is
COHERE's job; this stage owns intra-section fluency + fidelity.
"""
from __future__ import annotations
import os
import sys
_NT = os.path.expanduser("~/Desktop/neuron-talk")
_LR = os.path.expanduser("~/Desktop/lang-realizers")
for _p in (_NT, _LR):
if _p not in sys.path:
sys.path.insert(0, _p)
import engine # noqa: E402 (the proven no-LLM realizer)
from dialogue import _prop_to_clause # noqa: E402 (proven prop -> clause)
from document_ir import Block, DocumentIR, Provenance, Section # noqa: E402
def _provenance_from(p, kind: str = "fact") -> Provenance:
return Provenance(
subj_id=p.source_node_id, subject=p.subject, relation=p.predicate,
obj=p.object, polarity=p.polarity, confidence=round(float(p.confidence), 3),
node_id=p.source_node_id, kind=kind,
importance=float(getattr(p, "node_importance", 0.0) or 0.0),
salience=0.0,
)
import re as _re
# a well-formed declarative opens with a determiner, a proper noun, "I", or a
# capitalized head — not a mis-parsed object pronoun or a copula fragment.
_BAD_OPENERS = _re.compile(r"^(Me |It is I|There is|This is it|That is it)\b")
_VACUOUS = _re.compile(r"^\w+ (is|are|was|were) (it|no|nothing|empty|those|this|that)\.?$",
_re.I)
def _good_sentence(text: str) -> bool:
"""Fluency gate — drops degenerate realizations. NEVER loosens faithfulness;
it only refuses to SPEAK a claim whose surface came out malformed."""
words = text.rstrip(".").split()
if len(words) < 3:
return False
if _BAD_OPENERS.search(text):
return False
if _VACUOUS.match(text):
return False
# a sentence that is mostly one-letter/two-letter tokens is a parse artifact
short = sum(1 for w in words if len(w.strip(".,'")) <= 2)
if short > len(words) / 2:
return False
return True
def _realize_prop(p, lang: str = "en") -> tuple[str, Provenance] | None:
"""One proposition -> (faithful sentence, provenance) or None if it drops."""
clause = _prop_to_clause(p)
text = engine.realize(clause, lang)
if not text or not text.strip():
return None
text = text.strip()
if not text.endswith((".", "!", "?")):
text += "."
# capitalize first character (proper nouns / "I" already handled by grammar)
text = text[0].upper() + text[1:]
if not _good_sentence(text):
return None
return text, _provenance_from(p)
def realize_document(doc: DocumentIR, lang: str = "en") -> DocumentIR:
"""Fill every planned section's blocks with faithful, realized passages."""
for sec in doc.sections:
planned = sec.__dict__.get("_planned_props", [])
block = Block(role="body")
summary_bits: list[str] = []
for p in planned:
r = _realize_prop(p, lang)
if r is None:
continue
text, prov = r
block.sentences.append(text)
block.provenance.append(prov)
if len(summary_bits) < 1:
# a short grounded gloss for TOC / pptx bullets
obj = (prov.obj or "").strip().rstrip(".")
if obj:
summary_bits.append(obj)
if block.sentences:
sec.blocks.append(block)
sec.summary = summary_bits[0] if summary_bits else ""
# drop the transient planning payload; the IR is now self-contained
sec.__dict__.pop("_planned_props", None)
sec.__dict__.pop("_node", None)
# prune sections that realized to nothing
doc.sections = [s for s in doc.sections if s.blocks]
return doc
+73
View File
@@ -0,0 +1,73 @@
// audio-demo.el - Drive the native audio surface: render a tone per instrument
// from its LEARNED signature, then render a small meaning-phrase "piece".
// Entry point: top-level statement calls main() (same convention as the
// examples' top-level println(run_test())).
fn micros_to_str(xs: [Int]) -> String {
let n: Int = native_list_len(xs)
let out: String = ""
let i: Int = 0
while i < n {
if i > 0 { let out: String = out + "," }
let out: String = out + int_to_str(native_list_get(xs, i))
let i: Int = i + 1
}
return out
}
// Render a 1.0s A4 (midi 69) tone from a signature file, print the parsed
// partials (proving the numbers came from the engram .sig), write the WAV.
fn render_tone(name: String, sigpath: String, outpath: String, table: [Int]) -> Int {
let lines: [String] = sig_load(sigpath)
let partials: [Int] = parse_micros(sig_field(lines, "partials"))
println("[" + name + "] partials_n=" + sig_field(lines, "partials_n") + " parsed_partials_micro(scale 1e6)=" + micros_to_str(partials))
println("[" + name + "] raw partials line from .sig = " + sig_field(lines, "partials"))
let freq: Int = freq_of_midi(69)
let note: [Int] = synth_from_sig(lines, freq, 1000, 900, 44100, table)
let n: Int = native_list_len(note)
let ok: Int = wav_write(outpath, note, n, 44100)
println("[" + name + "] rendered " + int_to_str(n) + " samples -> " + outpath + " (write_ok=" + int_to_str(ok) + ")")
return n
}
fn run_demo() -> Int {
let table: [Int] = sin_table()
fs_mkdir("/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out")
println("=== TONES: render A4 (midi 69) from each learned signature ===")
render_tone("flute", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/sig/flute.sig", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out/tone-flute.wav", table)
render_tone("clarinet", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/sig/clarinet.sig", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out/tone-clarinet.wav", table)
render_tone("violin", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/sig/violin.sig", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out/tone-violin.wav", table)
render_tone("piano", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/sig/piano.sig", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out/tone-piano.wav", table)
render_tone("organ", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/sig/organ.sig", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out/tone-organ.wav", table)
println("")
println("=== PIECE: a 6-frame meaning phrase (incl. a NEG frame) ===")
let frames: [[String]] = native_list_empty()
let frames: [[String]] = native_list_append(frames, audio_frame("agent", "aff", "0.9", "0.8", "0", "s1"))
let frames: [[String]] = native_list_append(frames, audio_frame("theme", "aff", "0.7", "0.6", "0", "s2"))
let frames: [[String]] = native_list_append(frames, audio_frame("cause", "aff", "0.8", "0.9", "1", "s3"))
let frames: [[String]] = native_list_append(frames, audio_frame("negation", "neg", "0.85", "0.7", "0", "s4"))
let frames: [[String]] = native_list_append(frames, audio_frame("goal", "aff", "0.6", "0.5", "1", "s5"))
let frames: [[String]] = native_list_append(frames, audio_frame("result", "aff", "0.95", "1.0", "0", "s6"))
// Print the plan so the NEG frame's minor third (+3) vs major (+4) is visible.
let nf: Int = native_list_len(frames)
let fi: Int = 0
while fi < nf {
let frame: [String] = native_list_get(frames, fi)
let plan: [Int] = plan_note(frame)
let pol: String = surface_get(frame, "polarity")
let third_name: String = "major(+4)"
if str_eq(pol, "neg") { let third_name: String = "MINOR(+3)" }
println("frame " + int_to_str(fi) + " relation=" + surface_get(frame, "relation") + " polarity=" + pol + " -> midi=" + int_to_str(native_list_get(plan, 0)) + " dur_ms=" + int_to_str(native_list_get(plan, 1)) + " amp_pm=" + int_to_str(native_list_get(plan, 2)) + " third=" + third_name)
let fi: Int = fi + 1
}
let piano_lines: [String] = sig_load("/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/sig/piano.sig")
let total: Int = realize_audio(frames, piano_lines, "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out/piece.wav", 44100, table)
println("PIECE rendered " + int_to_str(total) + " samples -> /Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out/piece.wav")
return total
}
println("audio-demo main returned samples=" + int_to_str(run_demo()))
+400
View File
@@ -0,0 +1,400 @@
// audio-surface.el - Native own-core additive-synthesis audio surface.
//
// The AUDIO efferent seam, native, no Python and no library. This renders real
// PCM .wav bytes from instrument SIGNATURES read from engram-sourced .sig data
// files (elp/faculty/sig/*.sig) - the partial amplitudes are NEVER literals in
// this source; they are parsed from the learned signature at run time. That is
// the whole proof: render-from-learned-signatures.
//
// EL has no float arithmetic operator (codegen emits raw int64 ops for + - * /
// on the shared 64-bit slot) and no float-arithmetic natives - so ALL synthesis
// math here is own-core INTEGER fixed-point. Angles use a quarter-wave sine
// table (scale 10000) from a fixed-point Taylor series; amplitudes are parsed to
// micro (scale 1e6) straight from the .sig text; frequencies are milliHz ints.
//
// Pipeline mirrors the two-stage projector (midi.py): plan_note(frame) reads a
// frame's meaning-geometry slot-map and derives (pitch, duration, amplitude);
// realize_audio SUPERPOSES the signature's partials (the compose op) and
// serialises RIFF/WAVE. Same frame -> midi OR audio.
// -- integer decimal + string helpers -----------------------------------------
fn str_to_int_el(s: String) -> Int {
let n: Int = str_len(s)
let i: Int = 0
let v: Int = 0
let neg: Bool = false
while i < n {
let c: Int = str_char_code(s, i)
if c == 45 { let neg: Bool = true }
if c >= 48 {
if c < 58 {
let v: Int = v * 10 + (c - 48)
}
}
let i: Int = i + 1
}
if neg { return 0 - v }
return v
}
fn parse_micro(s: String) -> Int {
let dot: Int = str_index_of(s, ".")
if dot < 0 {
return str_to_int_el(s) * 1000000
}
let n: Int = str_len(s)
let ipart: String = str_slice(s, 0, dot)
let fpart: String = str_slice(s, dot + 1, n)
let iv: Int = str_to_int_el(ipart)
let fv: Int = 0
let scale: Int = 100000
let fn2: Int = str_len(fpart)
let i: Int = 0
while i < 6 {
let d: Int = 0
if i < fn2 {
let d: Int = str_char_code(fpart, i) - 48
}
let fv: Int = fv + d * scale
let scale: Int = scale / 10
let i: Int = i + 1
}
return iv * 1000000 + fv
}
// -- signature (engram data file) loader ---------------------------------------
fn sig_load(path: String) -> [String] {
let text: String = fs_read(path)
return str_split(text, "\n")
}
fn sig_field(lines: [String], key: String) -> String {
let pref: String = key + ": "
let n: Int = native_list_len(lines)
let plen: Int = str_len(pref)
let i: Int = 0
while i < n {
let ln: String = native_list_get(lines, i)
if str_starts_with(ln, pref) {
return str_slice(ln, plen, str_len(ln))
}
let i: Int = i + 1
}
return ""
}
fn parse_micros(csv: String) -> [Int] {
let parts: [String] = str_split(csv, ",")
let n: Int = native_list_len(parts)
let out: [Int] = native_list_empty()
let i: Int = 0
while i < n {
let out: [Int] = native_list_append(out, parse_micro(native_list_get(parts, i)))
let i: Int = i + 1
}
return out
}
// -- fixed-point sine (own-core, quarter-wave Taylor table, scale 10000) --------
fn sin_table() -> [Int] {
let HP: Int = 1570796
let t: [Int] = native_list_empty()
let q: Int = 0
while q < 257 {
let x: Int = q * HP / 256
let x2: Int = x * x / 1000000
let x3: Int = x2 * x / 1000000
let x5: Int = x3 * x2 / 1000000
let x7: Int = x5 * x2 / 1000000
let x9: Int = x7 * x2 / 1000000
let s: Int = x - x3 / 6 + x5 / 120 - x7 / 5040 + x9 / 362880
let t: [Int] = native_list_append(t, s / 100)
let q: Int = q + 1
}
return t
}
fn sin_lookup(t: [Int], phase: Int) -> Int {
let p: Int = phase % 1024
if p < 0 { let p: Int = p + 1024 }
let quad: Int = p / 256
let r: Int = p % 256
if quad == 0 { return native_list_get(t, r) }
if quad == 1 { return native_list_get(t, 256 - r) }
if quad == 2 { return 0 - native_list_get(t, r) }
return 0 - native_list_get(t, 256 - r)
}
fn isqrt_int(n: Int) -> Int {
if n <= 0 { return 0 }
let x: Int = n
let y: Int = (x + 1) / 2
while y < x {
let x: Int = y
let y: Int = (x + n / x) / 2
}
return x
}
// freq_of_midi: equal-tempered frequency in milliHz. 440000 mHz at midi 69.
fn freq_of_midi(m: Int) -> Int {
let f: Int = 440000
if m > 69 {
let k: Int = m - 69
let i: Int = 0
while i < k {
let f: Int = f * 1059463 / 1000000
let i: Int = i + 1
}
return f
}
if m < 69 {
let k: Int = 69 - m
let i: Int = 0
while i < k {
let f: Int = f * 1000000 / 1059463
let i: Int = i + 1
}
return f
}
return f
}
// -- envelope (ADSR), scale 1000 -----------------------------------------------
fn adsr_env(i: Int, total: Int, atk_n: Int, dec_n: Int, sus_pm: Int, rel_n: Int) -> Int {
if i < atk_n {
if atk_n == 0 { return 1000 }
return 1000 * i / atk_n
}
if i < atk_n + dec_n {
if dec_n == 0 { return sus_pm }
return 1000 - (1000 - sus_pm) * (i - atk_n) / dec_n
}
let rel_start: Int = total - rel_n
if i < rel_start {
return sus_pm
}
if rel_n == 0 { return 0 }
let left: Int = total - i
return sus_pm * left / rel_n
}
// -- note synthesis: SUPERPOSE the learned partials -> [Int] samples -----------
fn note_samples(freq_mHz: Int, dur_ms: Int, rate: Int, partials: [Int], sumP: Int, b_micro: Int, vib_rate: Int, vib_cents: Int, atk_ms: Int, dec_ms: Int, sus_pm: Int, rel_ms: Int, amp_pm: Int, table: [Int]) -> [Int] {
let total: Int = dur_ms * rate / 1000
let atk_n: Int = atk_ms * rate / 1000
let dec_n: Int = dec_ms * rate / 1000
let rel_n: Int = rel_ms * rate / 1000
let np: Int = native_list_len(partials)
let half_mhz: Int = rate * 1000 / 2
let out: [Int] = native_list_empty()
let i: Int = 0
while i < total {
let acc: Int = 0
let k: Int = 0
while k < np {
let harm: Int = k + 1
let amp_k: Int = native_list_get(partials, k)
let factor: Int = 1000000
if b_micro > 0 {
let val: Int = 1000000 + b_micro * harm * harm
let factor: Int = isqrt_int(val * 1000000)
}
let fn_mhz: Int = freq_mHz * harm
let fn_mhz: Int = fn_mhz * factor / 1000000
if vib_cents > 0 {
if vib_rate > 0 {
let vphase: Int = i * vib_rate * 1024 / rate
let vs: Int = sin_lookup(table, vphase)
let vibf: Int = 1000000 + (vib_cents * vs * 833) / 10000
let fn_mhz: Int = fn_mhz * vibf / 1000000
}
}
if fn_mhz <= half_mhz {
let phase: Int = i * fn_mhz * 1024 / (rate * 1000)
let sv: Int = sin_lookup(table, phase)
let acc: Int = acc + sv * amp_k / 1000000
}
let k: Int = k + 1
}
let env: Int = adsr_env(i, total, atk_n, dec_n, sus_pm, rel_n)
let s16: Int = acc * 2800000 / sumP
let s16: Int = s16 * env / 1000
let s16: Int = s16 * amp_pm / 1000
if s16 > 32767 { let s16: Int = 32767 }
if s16 < 0 - 32767 { let s16: Int = 0 - 32767 }
let out: [Int] = native_list_append(out, s16)
let i: Int = i + 1
}
return out
}
fn synth_from_sig(lines: [String], freq_mHz: Int, dur_ms: Int, amp_pm: Int, rate: Int, table: [Int]) -> [Int] {
let partials: [Int] = parse_micros(sig_field(lines, "partials"))
let np: Int = native_list_len(partials)
let sumP: Int = 0
let j: Int = 0
while j < np {
let pj: Int = native_list_get(partials, j)
let sumP: Int = sumP + pj
let j: Int = j + 1
}
if sumP <= 0 { let sumP: Int = 1000000 }
let adsr: [String] = str_split(sig_field(lines, "adsr"), ",")
let atk_ms: Int = parse_micro(native_list_get(adsr, 0)) / 1000
let dec_ms: Int = parse_micro(native_list_get(adsr, 1)) / 1000
let sus_pm: Int = parse_micro(native_list_get(adsr, 2)) / 1000
let rel_ms: Int = parse_micro(native_list_get(adsr, 3)) / 1000
let b_micro: Int = parse_micro(sig_field(lines, "inharmonicity_B"))
let vib_rate: Int = str_to_int_el(sig_field(lines, "vibrato_rate_hz"))
let vib_cents: Int = str_to_int_el(sig_field(lines, "vibrato_depth_cents"))
return note_samples(freq_mHz, dur_ms, rate, partials, sumP, b_micro, vib_rate, vib_cents, atk_ms, dec_ms, sus_pm, rel_ms, amp_pm, table)
}
// -- byte-buffer helpers (own-core, no library) --------------------------------
fn put_tag(buf: String, pos: Int, s: String) -> String {
let n: Int = str_len(s)
let i: Int = 0
while i < n {
let buf: String = __str_set_char(buf, pos + i, str_char_code(s, i))
let i: Int = i + 1
}
return buf
}
fn put_u32le(buf: String, pos: Int, v: Int) -> String {
let buf: String = __str_set_char(buf, pos, v % 256)
let buf: String = __str_set_char(buf, pos + 1, (v / 256) % 256)
let buf: String = __str_set_char(buf, pos + 2, (v / 65536) % 256)
let buf: String = __str_set_char(buf, pos + 3, (v / 16777216) % 256)
return buf
}
fn put_u16le(buf: String, pos: Int, v: Int) -> String {
let buf: String = __str_set_char(buf, pos, v % 256)
let buf: String = __str_set_char(buf, pos + 1, (v / 256) % 256)
return buf
}
// -- WAV serializer: own-core RIFF/WAVE, PCM mono 16-bit -----------------------
fn wav_write(path: String, samples: [Int], n: Int, rate: Int) -> Int {
let data_len: Int = n * 2
let total: Int = 44 + data_len
let buf: String = __str_alloc(total)
let buf: String = put_tag(buf, 0, "RIFF")
let buf: String = put_u32le(buf, 4, 36 + data_len)
let buf: String = put_tag(buf, 8, "WAVE")
let buf: String = put_tag(buf, 12, "fmt ")
let buf: String = put_u32le(buf, 16, 16)
let buf: String = put_u16le(buf, 20, 1)
let buf: String = put_u16le(buf, 22, 1)
let buf: String = put_u32le(buf, 24, rate)
let buf: String = put_u32le(buf, 28, rate * 2)
let buf: String = put_u16le(buf, 32, 2)
let buf: String = put_u16le(buf, 34, 16)
let buf: String = put_tag(buf, 36, "data")
let buf: String = put_u32le(buf, 40, data_len)
let i: Int = 0
while i < n {
let v: Int = native_list_get(samples, i)
if v < 0 { let v: Int = v + 65536 }
let buf: String = __str_set_char(buf, 44 + i * 2, v % 256)
let buf: String = __str_set_char(buf, 44 + i * 2 + 1, (v / 256) % 256)
let i: Int = i + 1
}
let ok: Int = fs_write_bytes(path, buf, total)
return ok
}
// -- plan: frame slot-map -> note atom (pitch, duration, amplitude) ------------
fn audio_frame(relation: String, polarity: String, confidence: String, importance: String, salience: String, subj_id: String) -> [String] {
let f: [String] = native_list_empty()
let f: [String] = native_list_append(f, "relation")
let f: [String] = native_list_append(f, relation)
let f: [String] = native_list_append(f, "polarity")
let f: [String] = native_list_append(f, polarity)
let f: [String] = native_list_append(f, "confidence")
let f: [String] = native_list_append(f, confidence)
let f: [String] = native_list_append(f, "importance")
let f: [String] = native_list_append(f, importance)
let f: [String] = native_list_append(f, "salience")
let f: [String] = native_list_append(f, salience)
let f: [String] = native_list_append(f, "subj_id")
let f: [String] = native_list_append(f, subj_id)
return f
}
fn degree_offset(deg: Int) -> Int {
if deg == 0 { return 0 }
if deg == 1 { return 2 }
if deg == 2 { return 4 }
if deg == 3 { return 5 }
if deg == 4 { return 7 }
if deg == 5 { return 9 }
return 11
}
// returns [midi, dur_ms, amp_pm]
fn plan_note(frame: [String]) -> [Int] {
let relation: String = surface_get(frame, "relation")
let polarity: String = surface_get(frame, "polarity")
let confidence: String = surface_get(frame, "confidence")
let importance: String = surface_get(frame, "importance")
let salience: String = surface_get(frame, "salience")
let rn: Int = str_len(relation)
let csum: Int = 0
let i: Int = 0
while i < rn {
let cc: Int = str_char_code(relation, i)
let csum: Int = csum + cc
let i: Int = i + 1
}
let deg: Int = csum % 7
let third: Int = 4
if str_eq(polarity, "neg") { let third: Int = 3 }
let sal_oct: Int = str_to_int_el(salience)
let doff: Int = degree_offset(deg)
let midi: Int = 60 + sal_oct * 12 + doff + third
let conf_micro: Int = parse_micro(confidence)
let dur_ms: Int = 200 + conf_micro / 1000
let imp_micro: Int = parse_micro(importance)
let amp_pm: Int = 400 + imp_micro / 2000
let out: [Int] = native_list_empty()
let out: [Int] = native_list_append(out, midi)
let out: [Int] = native_list_append(out, dur_ms)
let out: [Int] = native_list_append(out, amp_pm)
return out
}
fn realize_audio(frames: [[String]], sig_lines: [String], path: String, rate: Int, table: [Int]) -> Int {
let nf: Int = native_list_len(frames)
let all: [Int] = native_list_empty()
let count: Int = 0
let fi: Int = 0
while fi < nf {
let frame: [String] = native_list_get(frames, fi)
let plan: [Int] = plan_note(frame)
let midi: Int = native_list_get(plan, 0)
let dur_ms: Int = native_list_get(plan, 1)
let amp_pm: Int = native_list_get(plan, 2)
let freq: Int = freq_of_midi(midi)
let note: [Int] = synth_from_sig(sig_lines, freq, dur_ms, amp_pm, rate, table)
let nn: Int = native_list_len(note)
let j: Int = 0
while j < nn {
let all: [Int] = native_list_append(all, native_list_get(note, j))
let j: Int = j + 1
}
let count: Int = count + nn
let fi: Int = fi + 1
}
let ok: Int = wav_write(path, all, count, rate)
return count
}
+65
View File
@@ -0,0 +1,65 @@
// image-demo.el - Drive the native PNG surface: plan a scene from a small
// meaning phrase (incl. a NEG frame) and emit a byte-valid 64x64 PNG whose
// palette is read from elp/faculty/sig/scene.basis.
fn img_frame(relation: String, polarity: String, confidence: String, importance: String, salience: String, subj_id: String) -> [String] {
let f: [String] = native_list_empty()
let f: [String] = native_list_append(f, "relation")
let f: [String] = native_list_append(f, relation)
let f: [String] = native_list_append(f, "polarity")
let f: [String] = native_list_append(f, polarity)
let f: [String] = native_list_append(f, "confidence")
let f: [String] = native_list_append(f, confidence)
let f: [String] = native_list_append(f, "importance")
let f: [String] = native_list_append(f, importance)
let f: [String] = native_list_append(f, "salience")
let f: [String] = native_list_append(f, salience)
let f: [String] = native_list_append(f, "subj_id")
let f: [String] = native_list_append(f, subj_id)
return f
}
fn rgb_str(c: [Int]) -> String {
return int_to_str(native_list_get(c, 0)) + "," + int_to_str(native_list_get(c, 1)) + "," + int_to_str(native_list_get(c, 2))
}
fn run_image() -> Int {
fs_mkdir("/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out")
let table: [Int] = crc_table()
println("crc_table[1]=" + int_to_str(native_list_get(table, 1)) + " (expect 1996959894 / 0x77073096)")
let basis: [String] = basis_load("/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/sig/scene.basis")
let warm: [Int] = parse_rgb(basis_field(basis, "warm"))
let cool: [Int] = parse_rgb(basis_field(basis, "cool"))
let bg: [Int] = parse_rgb(basis_field(basis, "bg"))
println("basis warm=" + rgb_str(warm) + " cool=" + rgb_str(cool) + " bg=" + rgb_str(bg) + " (read from scene.basis)")
let frames: [[String]] = native_list_empty()
let frames: [[String]] = native_list_append(frames, img_frame("agent", "aff", "0.9", "0.8", "0", "s1"))
let frames: [[String]] = native_list_append(frames, img_frame("theme", "aff", "0.7", "0.6", "1", "s2"))
let frames: [[String]] = native_list_append(frames, img_frame("cause", "aff", "0.8", "0.9", "0", "s3"))
let frames: [[String]] = native_list_append(frames, img_frame("negation", "neg", "0.85", "0.7", "1", "s4"))
let frames: [[String]] = native_list_append(frames, img_frame("goal", "aff", "0.6", "0.5", "0", "s5"))
let frames: [[String]] = native_list_append(frames, img_frame("result", "aff", "0.95", "1.0", "1", "s6"))
let shapes: [[Int]] = plan_scene(frames, warm, cool)
let ns: Int = native_list_len(shapes)
println("planned " + int_to_str(ns) + " shapes:")
let si: Int = 0
while si < ns {
let sh: [Int] = native_list_get(shapes, si)
let pol: String = surface_get(native_list_get(frames, si), "polarity")
println(" shape " + int_to_str(si) + " type=" + int_to_str(native_list_get(sh, 0)) + " x=" + int_to_str(native_list_get(sh, 1)) + " y=" + int_to_str(native_list_get(sh, 2)) + " size=" + int_to_str(native_list_get(sh, 3)) + " rgb=" + int_to_str(native_list_get(sh, 4)) + "," + int_to_str(native_list_get(sh, 5)) + "," + int_to_str(native_list_get(sh, 6)) + " polarity=" + pol)
let si: Int = si + 1
}
let raw: [Int] = rasterize(64, 64, shapes, bg)
println("rasterized raw (filtered scanlines) bytes=" + int_to_str(native_list_len(raw)) + " (expect 12352)")
let png: [Int] = png_build(64, 64, raw, table)
let plen: Int = native_list_len(png)
let ok: Int = png_write("/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out/scene.png", png)
println("PNG bytes=" + int_to_str(plen) + " -> /Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out/scene.png (write_ok=" + int_to_str(ok) + ")")
return plen
}
println("image-demo returned png_bytes=" + int_to_str(run_image()))
+412
View File
@@ -0,0 +1,412 @@
// image-surface.el - Native own-core raster PNG surface (the image efferent
// twin of audio). Renders a 64x64 RGB scene deterministically from a frame's
// meaning-geometry, then serialises a byte-valid PNG entirely own-core:
// 8-byte magic, IHDR, IDAT (zlib STORED/uncompressed DEFLATE + Adler32), IEND,
// with a per-chunk CRC32 computed via software xor32 (EL has no bitwise ops).
//
// The RGB palette basis is read from elp/faculty/sig/scene.basis (data, not
// literals) - the same read-from-learned discipline as the audio signatures.
// Integer-only throughout; pixels are composed functionally (painter's order)
// so no list mutation is needed.
// -- small int/parse helpers (self-contained) ----------------------------------
fn i_str_to_int(s: String) -> Int {
let n: Int = str_len(s)
let i: Int = 0
let v: Int = 0
while i < n {
let c: Int = str_char_code(s, i)
if c >= 48 {
if c < 58 {
let v: Int = v * 10 + (c - 48)
}
}
let i: Int = i + 1
}
return v
}
fn basis_load(path: String) -> [String] {
return str_split(fs_read(path), "\n")
}
fn basis_field(lines: [String], key: String) -> String {
let pref: String = key + ": "
let n: Int = native_list_len(lines)
let plen: Int = str_len(pref)
let i: Int = 0
while i < n {
let ln: String = native_list_get(lines, i)
if str_starts_with(ln, pref) {
return str_slice(ln, plen, str_len(ln))
}
let i: Int = i + 1
}
return ""
}
fn parse_rgb(csv: String) -> [Int] {
let parts: [String] = str_split(csv, ",")
let out: [Int] = native_list_empty()
let n: Int = native_list_len(parts)
let i: Int = 0
while i < n {
let v: Int = i_str_to_int(native_list_get(parts, i))
let out: [Int] = native_list_append(out, v)
let i: Int = i + 1
}
return out
}
// -- software 32-bit XOR (no bitwise ops in EL) --------------------------------
fn xor32(a: Int, b: Int) -> Int {
let r: Int = 0
let bit: Int = 1
let i: Int = 0
while i < 32 {
let abit: Int = (a / bit) % 2
let bbit: Int = (b / bit) % 2
if abit != bbit {
let add: Int = bit
let r: Int = r + add
}
let bit: Int = bit * 2
let i: Int = i + 1
}
return r
}
// -- CRC32 (table-driven, table built with xor32) ------------------------------
fn crc_table() -> [Int] {
let t: [Int] = native_list_empty()
let n: Int = 0
while n < 256 {
let c: Int = n
let k: Int = 0
while k < 8 {
if c % 2 == 1 {
let h: Int = c / 2
let c: Int = xor32(h, 3988292384)
} else {
let c: Int = c / 2
}
let k: Int = k + 1
}
let t: [Int] = native_list_append(t, c)
let n: Int = n + 1
}
return t
}
fn crc32_of(bytes: [Int], table: [Int]) -> Int {
let crc: Int = 4294967295
let n: Int = native_list_len(bytes)
let i: Int = 0
while i < n {
let b: Int = native_list_get(bytes, i)
let lo: Int = crc % 256
let idx: Int = xor32(lo, b) % 256
let tv: Int = native_list_get(table, idx)
let hi: Int = crc / 256
let crc: Int = xor32(hi, tv)
let i: Int = i + 1
}
return xor32(crc, 4294967295)
}
// -- Adler32 (for the zlib trailer) --------------------------------------------
fn adler32_of(bytes: [Int]) -> Int {
let a: Int = 1
let b: Int = 0
let n: Int = native_list_len(bytes)
let i: Int = 0
while i < n {
let byte: Int = native_list_get(bytes, i)
let a: Int = (a + byte) % 65521
let b: Int = (b + a) % 65521
let i: Int = i + 1
}
return b * 65536 + a
}
// -- byte-list append helpers --------------------------------------------------
fn app_u32be(dst: [Int], v: Int) -> [Int] {
let dst: [Int] = native_list_append(dst, (v / 16777216) % 256)
let dst: [Int] = native_list_append(dst, (v / 65536) % 256)
let dst: [Int] = native_list_append(dst, (v / 256) % 256)
let dst: [Int] = native_list_append(dst, v % 256)
return dst
}
fn app_tag(dst: [Int], s: String) -> [Int] {
let n: Int = str_len(s)
let i: Int = 0
while i < n {
let dst: [Int] = native_list_append(dst, str_char_code(s, i))
let i: Int = i + 1
}
return dst
}
fn app_all(dst: [Int], src: [Int]) -> [Int] {
let n: Int = native_list_len(src)
let i: Int = 0
while i < n {
let dst: [Int] = native_list_append(dst, native_list_get(src, i))
let i: Int = i + 1
}
return dst
}
// -- plan: frame meaning-geometry -> shape atoms -------------------------------
// shape = [type, x, y, size, r, g, b] (type 0=rect 1=disc 2=triangle)
fn charsum(s: String) -> Int {
let n: Int = str_len(s)
let i: Int = 0
let acc: Int = 0
while i < n {
let c: Int = str_char_code(s, i)
let acc: Int = acc + c
let i: Int = i + 1
}
return acc
}
fn micro_of(s: String) -> Int {
let dot: Int = str_index_of(s, ".")
if dot < 0 { return i_str_to_int(s) * 1000000 }
let n: Int = str_len(s)
let fp: String = str_slice(s, dot + 1, n)
let ip: String = str_slice(s, 0, dot)
let iv: Int = i_str_to_int(ip)
let fv: Int = 0
let scale: Int = 100000
let fl: Int = str_len(fp)
let i: Int = 0
while i < 6 {
let d: Int = 0
if i < fl { let d: Int = str_char_code(fp, i) - 48 }
let fv: Int = fv + d * scale
let scale: Int = scale / 10
let i: Int = i + 1
}
return iv * 1000000 + fv
}
fn plan_scene(frames: [[String]], warm: [Int], cool: [Int]) -> [[Int]] {
let shapes: [[Int]] = native_list_empty()
let nf: Int = native_list_len(frames)
let fi: Int = 0
while fi < nf {
let fr: [String] = native_list_get(frames, fi)
let relation: String = surface_get(fr, "relation")
let polarity: String = surface_get(fr, "polarity")
let confidence: String = surface_get(fr, "confidence")
let importance: String = surface_get(fr, "importance")
let salience: String = surface_get(fr, "salience")
// relation -> shape type
let stype: Int = charsum(relation) % 3
// confidence -> size (8..22)
let cmi: Int = micro_of(confidence)
let size: Int = 8 + cmi / 71428
// salience -> y
let sal: Int = i_str_to_int(salience)
let y: Int = 6 + sal * 26
// subj_id/index -> x
let x: Int = 4 + (fi * 10) % 48
// polarity -> warm/cool base color
let br: Int = native_list_get(warm, 0)
let bg2: Int = native_list_get(warm, 1)
let bb: Int = native_list_get(warm, 2)
if str_eq(polarity, "neg") {
let br: Int = native_list_get(cool, 0)
let bg2: Int = native_list_get(cool, 1)
let bb: Int = native_list_get(cool, 2)
}
// importance -> brightness (500..1000 permille)
let imi: Int = micro_of(importance)
let bpm: Int = 500 + imi / 2000
let r: Int = br * bpm / 1000
let g: Int = bg2 * bpm / 1000
let b: Int = bb * bpm / 1000
let sh: [Int] = native_list_empty()
let sh: [Int] = native_list_append(sh, stype)
let sh: [Int] = native_list_append(sh, x)
let sh: [Int] = native_list_append(sh, y)
let sh: [Int] = native_list_append(sh, size)
let sh: [Int] = native_list_append(sh, r)
let sh: [Int] = native_list_append(sh, g)
let sh: [Int] = native_list_append(sh, b)
let shapes: [[Int]] = native_list_append(shapes, sh)
let fi: Int = fi + 1
}
return shapes
}
// covers: is (px,py) inside this shape?
fn covers(sh: [Int], px: Int, py: Int) -> Bool {
let stype: Int = native_list_get(sh, 0)
let sx: Int = native_list_get(sh, 1)
let sy: Int = native_list_get(sh, 2)
let size: Int = native_list_get(sh, 3)
let cx: Int = sx + size / 2
if stype == 0 {
if px >= sx {
if px < sx + size {
if py >= sy {
if py < sy + size {
return true
}
}
}
}
return false
}
if stype == 1 {
let rad: Int = size / 2
let dx: Int = px - cx
let dy: Int = py - (sy + rad)
if dx * dx + dy * dy <= rad * rad {
return true
}
return false
}
// triangle: apex at top (sy), base at sy+size
if py >= sy {
if py < sy + size {
let dyv: Int = py - sy
let halfw: Int = dyv / 2
let dxv: Int = px - cx
let adx: Int = dxv
if adx < 0 { let adx: Int = 0 - dxv }
if adx <= halfw {
return true
}
}
}
return false
}
// pixel_color: painter's algorithm - last covering shape wins. Returns [r,g,b].
fn pixel_color(px: Int, py: Int, shapes: [[Int]], bg: [Int]) -> [Int] {
let r: Int = native_list_get(bg, 0)
let g: Int = native_list_get(bg, 1)
let b: Int = native_list_get(bg, 2)
let n: Int = native_list_len(shapes)
let i: Int = 0
while i < n {
let sh: [Int] = native_list_get(shapes, i)
if covers(sh, px, py) {
let r: Int = native_list_get(sh, 4)
let g: Int = native_list_get(sh, 5)
let b: Int = native_list_get(sh, 6)
}
let i: Int = i + 1
}
let out: [Int] = native_list_empty()
let out: [Int] = native_list_append(out, r)
let out: [Int] = native_list_append(out, g)
let out: [Int] = native_list_append(out, b)
return out
}
// rasterize: build the raw (filtered) scanline byte stream, filter byte 0 / row.
fn rasterize(w: Int, h: Int, shapes: [[Int]], bg: [Int]) -> [Int] {
let raw: [Int] = native_list_empty()
let y: Int = 0
while y < h {
let raw: [Int] = native_list_append(raw, 0)
let x: Int = 0
while x < w {
let col: [Int] = pixel_color(x, y, shapes, bg)
let raw: [Int] = native_list_append(raw, native_list_get(col, 0))
let raw: [Int] = native_list_append(raw, native_list_get(col, 1))
let raw: [Int] = native_list_append(raw, native_list_get(col, 2))
let x: Int = x + 1
}
let y: Int = y + 1
}
return raw
}
// zlib stream with a single STORED (uncompressed) DEFLATE block + Adler32.
fn zlib_store(raw: [Int]) -> [Int] {
let z: [Int] = native_list_empty()
let z: [Int] = native_list_append(z, 120)
let z: [Int] = native_list_append(z, 1)
let z: [Int] = native_list_append(z, 1)
let len: Int = native_list_len(raw)
let nlen: Int = 65535 - len
let z: [Int] = native_list_append(z, len % 256)
let z: [Int] = native_list_append(z, (len / 256) % 256)
let z: [Int] = native_list_append(z, nlen % 256)
let z: [Int] = native_list_append(z, (nlen / 256) % 256)
let z: [Int] = app_all(z, raw)
let ad: Int = adler32_of(raw)
let z: [Int] = app_u32be(z, ad)
return z
}
// append a full PNG chunk: length + (type+data) + crc32(type+data).
fn app_chunk(png: [Int], type_and_data: [Int], table: [Int]) -> [Int] {
let total: Int = native_list_len(type_and_data)
let dlen: Int = total - 4
let png: [Int] = app_u32be(png, dlen)
let png: [Int] = app_all(png, type_and_data)
let crc: Int = crc32_of(type_and_data, table)
let png: [Int] = app_u32be(png, crc)
return png
}
fn png_build(w: Int, h: Int, raw: [Int], table: [Int]) -> [Int] {
let png: [Int] = native_list_empty()
// 8-byte signature
let png: [Int] = native_list_append(png, 137)
let png: [Int] = native_list_append(png, 80)
let png: [Int] = native_list_append(png, 78)
let png: [Int] = native_list_append(png, 71)
let png: [Int] = native_list_append(png, 13)
let png: [Int] = native_list_append(png, 10)
let png: [Int] = native_list_append(png, 26)
let png: [Int] = native_list_append(png, 10)
// IHDR
let ihdr: [Int] = native_list_empty()
let ihdr: [Int] = app_tag(ihdr, "IHDR")
let ihdr: [Int] = app_u32be(ihdr, w)
let ihdr: [Int] = app_u32be(ihdr, h)
let ihdr: [Int] = native_list_append(ihdr, 8)
let ihdr: [Int] = native_list_append(ihdr, 2)
let ihdr: [Int] = native_list_append(ihdr, 0)
let ihdr: [Int] = native_list_append(ihdr, 0)
let ihdr: [Int] = native_list_append(ihdr, 0)
let png: [Int] = app_chunk(png, ihdr, table)
// IDAT
let z: [Int] = zlib_store(raw)
let idat: [Int] = native_list_empty()
let idat: [Int] = app_tag(idat, "IDAT")
let idat: [Int] = app_all(idat, z)
let png: [Int] = app_chunk(png, idat, table)
// IEND
let iend: [Int] = native_list_empty()
let iend: [Int] = app_tag(iend, "IEND")
let png: [Int] = app_chunk(png, iend, table)
return png
}
fn png_write(path: String, png: [Int]) -> Int {
let n: Int = native_list_len(png)
let buf: String = __str_alloc(n)
let i: Int = 0
while i < n {
let buf: String = __str_set_char(buf, i, native_list_get(png, i))
let i: Int = i + 1
}
let ok: Int = fs_write_bytes(path, buf, n)
return ok
}
+153
View File
@@ -0,0 +1,153 @@
// surface-profile.el - Surface profile data and accessors.
//
// THE NATIVE EFFERENT SEAM: surface = a pluggable PROFILE, using the exact same
// slot-map mechanism as language-profile.el. A language profile tells the
// realizer HOW to shape a natural-language surface (word order, morphology); a
// SURFACE profile tells the realizer WHICH surface to project meaning onto
// (markdown, docx, html, plain, or a non-text medium like symbolic music).
//
// The generalization is exact: realize_lang(form, profile) already renders a
// SemForm parameterized by a [String] profile read via lang_get. Surface is one
// more axis of that same profile vector. One frame (sem_frame), one plan step
// (sem_to_spec), one render (realize) the surface is DATA, not a code path,
// precisely as language is data. Adding a surface means adding a profile, no
// engine change. This is the multimodal projector, native: geometry -> any
// surface, the efferent twin of ingest.
//
// Surface slot keys:
// surface - "markdown" | "docx" | "html" | "plain" | "midi" | "image"
// modality - "text" | "audio" | "image" | "video"
// media_type - MIME type of the emitted surface
// head_open - string prepended to a heading (e.g. "## " for markdown)
// head_close - string appended to a heading (e.g. "" for markdown, "</h2>" for html)
// emph_open - string opening emphasis (e.g. "*")
// emph_close - string closing emphasis (e.g. "*")
// item_mark - list-item marker (e.g. "- ")
// para_sep - paragraph separator (e.g. "\n\n")
//
// For a TEXT modality the render composes these markers around the surface that
// the EXISTING realizer produces (realize_lang / sem_realize). For a non-text
// modality (audio/image) the profile declares modality + media_type and the
// render dispatches to the medium projector, which reads the SAME frame's
// geometry (its intent/affect/structure) and projects it onto sound or pixels
// deterministic-from-meaning, nothing invented. That dispatch point is where a
// music profile or image profile conforms, native, no parallel layer.
// -- Constructor -------------------------------------------------------------
fn surface_profile(surface: String, modality: String, media_type: String, head_open: String, head_close: String, emph_open: String, emph_close: String, item_mark: String, para_sep: String) -> [String] {
let r: [String] = native_list_empty()
let r = native_list_append(r, "surface")
let r = native_list_append(r, surface)
let r = native_list_append(r, "modality")
let r = native_list_append(r, modality)
let r = native_list_append(r, "media_type")
let r = native_list_append(r, media_type)
let r = native_list_append(r, "head_open")
let r = native_list_append(r, head_open)
let r = native_list_append(r, "head_close")
let r = native_list_append(r, head_close)
let r = native_list_append(r, "emph_open")
let r = native_list_append(r, emph_open)
let r = native_list_append(r, "emph_close")
let r = native_list_append(r, emph_close)
let r = native_list_append(r, "item_mark")
let r = native_list_append(r, item_mark)
let r = native_list_append(r, "para_sep")
let r = native_list_append(r, para_sep)
return r
}
// -- Accessor (same convention as lang_get; standalone so this is a leaf) -----
fn surface_get(profile: [String], key: String) -> String {
let n: Int = native_list_len(profile)
let i: Int = 0
while i < n - 1 {
let k: String = native_list_get(profile, i)
if str_eq(k, key) {
return native_list_get(profile, i + 1)
}
let i = i + 2
}
return ""
}
fn surface_is_text(profile: [String]) -> Bool {
return str_eq(surface_get(profile, "modality"), "text")
}
// -- Built-in TEXT surface profiles ------------------------------------------
// Markdown: headings with "## ", emphasis with "*", "- " list items.
fn surface_profile_markdown() -> [String] {
return surface_profile("markdown", "text", "text/markdown", "## ", "", "*", "*", "- ", "\n\n")
}
// Plain text: no markup at all headings become bare uppercase-free lines.
fn surface_profile_plain() -> [String] {
return surface_profile("plain", "text", "text/plain", "", "", "", "", " - ", "\n\n")
}
// HTML: block-level heading/emphasis tags.
fn surface_profile_html() -> [String] {
return surface_profile("html", "text", "text/html", "<h2>", "</h2>", "<em>", "</em>", "<li>", "\n")
}
// docx: WordprocessingML is structural, not inline-markup; the head/emph slots
// carry the run/style intent that the OOXML emitter maps to <w:pStyle>. Declared
// here so docx is a first-class surface on the same seam.
fn surface_profile_docx() -> [String] {
return surface_profile("docx", "text", "application/vnd.openxmlformats-officedocument.wordprocessingml.document", "Heading2:", "", "b:", "", "bullet:", "\n")
}
// -- Built-in NON-TEXT surface profiles (the multimodal seam) ----------------
// Symbolic music (MIDI): modality=audio. The render dispatches to the music
// projector, which reads the SAME frame's intent/affect and projects it to
// pitch/rhythm deterministic-from-meaning. head/emph slots are empty because
// the medium is not textual; media_type names the surface. A music profile
// (scale/mode/instrument) is layered onto this by the audio agent, native.
fn surface_profile_midi() -> [String] {
return surface_profile("midi", "audio", "audio/midi", "", "", "", "", "", "")
}
// Synthesized audio (WAV): modality=audio, peer to midi. The richer audio
// surface the render SUPERPOSES ingested tonal primitives (sine at f0*n per an
// ingested instrument signature) into PCM, own-core, exactly as midi writes an
// SMF via struct. A music profile (scale/mode/instrument/adsr) layers onto this
// as its own [String] slot-map read by the same getter. Same frame -> midi OR
// audio, interchangeable; this is the audio agent's native conforming point.
fn surface_profile_audio() -> [String] {
return surface_profile("audio", "audio", "audio/wav", "", "", "", "", "", "")
}
// Image (raster): modality=image. Documented seam the render dispatches to the
// image projector, the efferent twin of image ingest, reading the same frame.
fn surface_profile_image() -> [String] {
return surface_profile("image", "image", "image/png", "", "", "", "", "", "")
}
// -- Composition helpers: wrap realized TEXT with the surface's markers -------
//
// These take text the EXISTING realizer already produced and shape it for the
// surface. They add NO content pure surface typography over faithful text,
// exactly as the language profile adds no content, only linguistic form.
fn surface_heading(profile: [String], text: String) -> String {
let o: String = surface_get(profile, "head_open")
let c: String = surface_get(profile, "head_close")
return o + text + c
}
fn surface_emph(profile: [String], text: String) -> String {
let o: String = surface_get(profile, "emph_open")
let c: String = surface_get(profile, "emph_close")
return o + text + c
}
// A section: a heading + a paragraph separator + the (already realized) body.
fn surface_section(profile: [String], heading: String, body: String) -> String {
let sep: String = surface_get(profile, "para_sep")
return surface_heading(profile, heading) + sep + body
}
@@ -0,0 +1,26 @@
// surface-profile-demo.el - ONE SemFrame, realized ONCE, projected to THREE
// surfaces via surface profiles. Proves surface-as-profile natively: the frame
// and the realized sentence are identical; only the surface PROFILE differs.
fn demo() -> String {
// 1. The shared frame (meaning-geometry): assert(Neuron, contain, the memory).
let frame: [String] = sem_frame("assert", "Neuron", "the memory", "")
// 2. REALIZE once via the EXISTING native realizer (language = a profile).
let sentence: String = sem_realize(frame)
// 3. PROJECT the same realized sentence onto three surfaces (surface = a
// profile). Same frame, same sentence, different surface one render.
let heading: String = "Memory"
let md: String = surface_section(surface_profile_markdown(), heading, sentence)
let html: String = surface_section(surface_profile_html(), heading, sentence)
let plain: String = surface_section(surface_profile_plain(), heading, sentence)
// 4. Report the non-text seam: a surface profile can declare an audio/image
// medium; the render dispatches to the medium projector on the SAME frame.
let midi_media: String = surface_get(surface_profile_midi(), "media_type")
return "MD=[" + md + "] HTML=[" + html + "] PLAIN=[" + plain + "] MIDI_MEDIA=" + midi_media
}
println(demo())
-120
View File
@@ -1,120 +0,0 @@
# sandbox — the Neuron STACK sandbox
**Work on a whole stack at once, not one repo at a time.** `sandbox` assembles every
constituent repo of a named stack into **one combined worktree workspace**, wired so
they build and run **together**, on an isolated clean base — then tears it all down
cleanly. The live soul/engram (`:7770` / `:8742`) are never touched.
It is the multi-repo sibling of [`nsbx`](./README.md): where `nsbx dev` stands up
**one** repo's worktree + an isolated engram, `sandbox` stands up **every** repo of a
stack as sibling git worktrees under a single workspace.
```bash
export PATH="$PWD:$PATH" # or symlink `sandbox` onto your PATH
sandbox neuron-stack tim # el + neuron soul + NeuronUI, assembled together
cd ~/Development/neuron-technologies/stack-worktrees/neuron-stack-tim
source .stack-env # EL_REPO + PATH now point at the SANDBOX el
./build.sh # engram compiles, soul compiles, UI present & buildable
sandbox down neuron-stack tim # remove every worktree; live untouched
```
## The two profiles
### `el-stack` — the whole EL kit
The compiler + language + framework + tooling are **all one repo** (`foundation/el`:
`lang/` = elc/elb + runtime, `engram/src/server.el`, `elp/` = NLG, `ui/` = the **el-ui
framework**, plus `ql`, `ide`, `epm`, `arbor`, `tools`). Its downstream SDK consumers
come along so a change to `elc` can be proven end-to-end across the kit.
| repo | required | role |
|------|----------|------|
| `foundation/el` | ✓ | elc + elb compiler, el_runtime, engram source, **el-ui framework**, elp/ql/ide/epm tooling |
| `engram-language` | | language-faculty reference POC (Python) — being ported into `el/elp` |
| `foundation/forge` | | downstream SDK consumer — `make build` |
| `foundation/dharma` | | downstream SDK consumer — CGI provenance registry |
`build.sh` proves it: `elc` compiles a real stack source and `cc` links it against the
runtime into a native binary (elc + runtime build together), and — if present — `forge`
builds against the freshly-assembled SDK.
### `neuron-stack` — the full product
Substrate + soul + UI. **Engram is not a separate repo** — its source lives inside
`foundation/el`.
| repo | required | role |
|------|----------|------|
| `foundation/el` | ✓ | substrate: elc + el_runtime + engram source + the `elp` NLG the soul imports |
| `neuron` | ✓ | the soul (`:7770`) + engram build; `soul.el` imports `../foundation/el/elp/src/elp.el` |
| `products/NeuronUI` | ✓ | the app/UI (Kotlin/Compose desktop client; bundles the soul binary) |
| `products/web` | | marketing site + interactive soul-demo |
`build.sh` proves it: **engram** builds (`elc engram/src/server.el``cc … el_runtime.c`
→ native binary), the **soul** compiles with its cross-repo `../foundation/el` import
resolving to the *sandbox* el, and the **UI** is present with its build entry.
## Why it works — mirrored-layout wiring
The repos reference each other by **relative sibling paths** (e.g. the soul imports
`../foundation/el/elp/src/elp.el`). So `sandbox` lays every worktree out at its **natural
relative path** inside the workspace:
```
stack-worktrees/neuron-stack-tim/
├── foundation/el/ ← worktree of foundation/el (the sandbox el)
├── neuron/ ← worktree of neuron
└── products/NeuronUI/ ← worktree of products/NeuronUI
```
From `neuron/`, `../foundation/el` resolves to `…/neuron-stack-tim/foundation/el` — the
**sandbox** copy, never the live tree. No symlinks, no path rewriting: the layout *is*
the wiring. `.stack-env` additionally pins `EL_REPO` and prepends the sandbox `elc`/`elb`
to `PATH`.
## Commands
| command | does |
|---------|------|
| `sandbox el-stack <name> [--minimal]` | assemble the EL kit (`--minimal` = required repos only) |
| `sandbox neuron-stack <name> [--minimal]` | assemble the full product |
| `sandbox build <profile> <name>` | run the workspace's combined `build.sh` |
| `sandbox status <profile> <name>` | per-repo head + clean/dirty |
| `sandbox list` | list assembled workspaces |
| `sandbox down <profile> <name> [--delete-branch]` | remove every worktree + drop the workspace (branch kept unless `--delete-branch`) |
Flags: `--minimal` (required repos only), `--branch B` (branch name; default
`sandbox/<profile>-<name>`), `--base REF` (fork point; default each repo's committed
HEAD).
## Rails (always)
- **Clean base** — worktrees fork off each repo's **committed HEAD**; the dirty state of
the live checkout is deliberately *not* carried in.
- **Persistent** — the workspace lives under `NSBX_STACK_ROOT` (default
`~/Development/neuron-technologies/stack-worktrees`), **never `/tmp`** (ablated on
compaction).
- **Never touches live**`sandbox` only does `git worktree` + offline `cc`. It never
binds `:8742`/`:7770`, never `launchctl`, never `pkill`. Bringing up an **isolated
engram** is delegated, opt-in, to `nsbx` (which guards the live store and refuses the
live ports).
- **Idempotent & safe** — refuses to clobber an existing workspace; a failed assembly
rolls back its partial worktrees; teardown removes worktrees through their origin repo
and prunes.
- **Own-the-core** — pure bash + `git worktree`. No new dependencies.
## Env knobs
`NSBX_STACK_ROOT` (workspace root), `NEURON_DEV_ROOT` (the dir holding all the peer
repos, default `~/Development/neuron-technologies`).
## Isolated engram for `neuron-stack` (opt-in)
`sandbox` gets the code building together; to run the soul against an **isolated** engram
(never live), delegate to `nsbx` from inside the workspace:
```bash
source .stack-env
nsbx create $STACK_NAME --source "$EL_REPO" # clone live store onto a non-default port
nsbx up $STACK_NAME
nsbx status $STACK_NAME # prints the isolated engram URL
```
-505
View File
@@ -1,505 +0,0 @@
#!/usr/bin/env bash
# sandbox — the Neuron STACK sandbox: assemble a WHOLE stack of repos into ONE
# combined worktree workspace, wired so they build/run TOGETHER, on an isolated
# clean base — so an agent (or Will) can work on the full stack at once instead of
# one repo at a time.
#
# It is the multi-repo generalisation of `nsbx` (this same directory): where
# `nsbx dev` stands up ONE repo's worktree + an isolated engram, `sandbox` stands
# up EVERY constituent repo of a named stack as sibling git worktrees under a
# single workspace, mirroring their on-disk relative layout so the cross-repo
# `../foundation/el` imports resolve to the SANDBOX copy — never the live tree.
#
# sandbox el-stack <name> # elc compiler + EL language + framework + tooling (+ consumers)
# sandbox neuron-stack <name> # runtime/soul + engram + app/UI (the full product)
# sandbox list # list assembled stack workspaces
# sandbox status <profile> <name> # inspect one
# sandbox build <profile> <name> # run the workspace's combined build.sh
# sandbox down <profile> <name> # tear down: remove every worktree, drop the workspace
#
# RAILS (always):
# * worktrees fork off each repo's COMMITTED HEAD -> a clean, reproducible base
# (the dirty state of the live checkout is deliberately NOT carried in).
# * the workspace lives at a PERSISTENT path (never /tmp — ablated on compaction).
# * NEVER touches the live soul/engram (:7770 / :8742). It only creates git
# worktrees + a build script; bringing up an isolated engram is delegated,
# opt-in, to `nsbx` (which already guards the live store & ports).
# * idempotent & safe: refuses to clobber an existing workspace; teardown removes
# worktrees through their origin repo and prunes — branches are kept by default.
# * own-the-core: pure bash + git worktree. No new dependencies.
set -uo pipefail
# ---------------------------------------------------------------- constants ----
# Root that holds all the peer repos (neuron, foundation/el, products/*, ...).
DEV_ROOT="${NEURON_DEV_ROOT:-$HOME/Development/neuron-technologies}"
# Where assembled stack workspaces live (persistent; sibling to el-worktrees/).
STACK_ROOT="${NSBX_STACK_ROOT:-$DEV_ROOT/stack-worktrees}"
EL_REPO_REL="foundation/el"
LIVE_ENGRAM_PORT=8742 # live engram — sandbox must never bind it
LIVE_SOUL_PORT=7770 # live soul — sandbox must never bind it
# nsbx (single-repo isolated-engram tool) lives next to this script.
NSBX="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)/nsbx"
C_RED=$'\033[31m'; C_GRN=$'\033[32m'; C_YEL=$'\033[33m'; C_CYN=$'\033[36m'; C_DIM=$'\033[2m'; C_BLD=$'\033[1m'; C_0=$'\033[0m'
# ---------------------------------------------------------------- helpers ------
die(){ printf '%serror:%s %s\n' "$C_RED" "$C_0" "$*" >&2; exit 1; }
log(){ printf '%s==>%s %s\n' "$C_BLD" "$C_0" "$*" >&2; }
info(){ printf ' %s\n' "$*" >&2; }
ok(){ printf ' %s%s%s\n' "$C_GRN" "$*" "$C_0" >&2; }
warn(){ printf ' %s%s%s\n' "$C_YEL" "$*" "$C_0" >&2; }
need(){ command -v "$1" >/dev/null 2>&1 || die "missing dependency: $1"; }
# ---------------------------------------------------------------- profiles -----
# profile_repos <profile> : emit one line per constituent repo:
# <relpath-under-DEV_ROOT> | <required|optional> | <role>
# The relpath is preserved INSIDE the workspace, so all cross-repo `../foundation/el`
# references resolve to the sandbox copy automatically (mirrored-layout wiring).
profile_repos(){
case "$1" in
el-stack)
# The compiler+language+framework+tooling are all ONE repo (foundation/el).
# Its downstream SDK consumers (forge, dharma) + the language-faculty POC come
# along so a change to elc can be proven end-to-end across the kit.
cat <<'EOF'
foundation/el | required | elc + elb compiler, el_runtime, engram source, el-ui framework, elp/ql/ide/epm tooling
engram-language | optional | language-faculty reference POC (Python) — ported into el/elp
foundation/forge | optional | downstream SDK consumer — `make build` (imprint forge CLI)
foundation/dharma | optional | downstream SDK consumer — CGI provenance registry
EOF
;;
neuron-stack)
# The full product: substrate (el) + soul + UI. Engram is NOT a separate repo
# (its source lives in foundation/el/engram/src/server.el).
cat <<'EOF'
foundation/el | required | substrate: elc + el_runtime + engram source + elp NLG the soul imports
neuron | required | the soul (:7770) + engram build; soul.el imports ../foundation/el/elp/src/elp.el
products/NeuronUI | required | the app/UI (Kotlin/Compose desktop client; bundles the soul binary)
products/web | optional | marketing site + interactive soul-demo
EOF
;;
*) return 1;;
esac
}
is_profile(){ profile_repos "$1" >/dev/null 2>&1; }
ws_dir(){ printf '%s/%s-%s' "$STACK_ROOT" "$1" "$2"; } # <root>/<profile>-<name>
ws_branch(){ printf 'sandbox/%s-%s' "$1" "$2"; } # branch name used in each repo
manifest(){ printf '%s/.stack-manifest.json' "$1"; } # <ws>/.stack-manifest.json
# ================================================================ up ===========
cmd_up(){
local profile="$1"; shift
local name="" branch="" base_override="" minimal=0
[ $# -gt 0 ] && [ "${1#-}" = "$1" ] && { name="$1"; shift; } || die "usage: sandbox $profile <name> [--minimal] [--branch B] [--base REF]"
while [ $# -gt 0 ]; do case "$1" in
--minimal) minimal=1; shift;;
--branch) branch="$2"; shift 2;;
--base) base_override="$2"; shift 2;;
*) die "unknown flag: $1";;
esac; done
need git
is_profile "$profile" || die "unknown profile: $profile (try: el-stack | neuron-stack)"
local ws; ws="$(ws_dir "$profile" "$name")"
[ -n "$branch" ] || branch="$(ws_branch "$profile" "$name")"
# -------- pre-flight (fail before creating anything) --------
case "$ws" in /tmp/*|/private/tmp/*|/var/tmp/*)
die "refusing workspace under a temp dir ($ws) — temp dirs are ablated on compaction; set NSBX_STACK_ROOT to a persistent path";;
esac
[ -e "$ws" ] && die "workspace already exists: $ws (sandbox down $profile $name first)"
# resolve + validate every repo, and pick a base sha per repo, BEFORE touching disk
local -a rels roles bases origins wts
local line rel role_extra role req origin base wt
while IFS= read -r line; do
[ -z "${line// }" ] && continue
rel="$(printf '%s' "$line" | cut -d'|' -f1 | xargs)"
req="$(printf '%s' "$line" | cut -d'|' -f2 | xargs)"
role="$(printf '%s' "$line" | cut -d'|' -f3- | sed 's/^ *//')"
[ "$minimal" -eq 1 ] && [ "$req" = "optional" ] && continue
origin="$DEV_ROOT/$rel"
git -C "$origin" rev-parse --git-dir >/dev/null 2>&1 || {
[ "$req" = "required" ] && die "required repo missing or not a git repo: $origin"
warn "skipping optional repo (missing): $rel"; continue; }
if [ -n "$base_override" ]; then base="$base_override"; else base="$(git -C "$origin" rev-parse HEAD)"; fi
wt="$ws/$rel"
[ -e "$wt" ] && die "target worktree path already exists: $wt"
rels+=("$rel"); roles+=("$role"); origins+=("$origin"); bases+=("$base"); wts+=("$wt")
done < <(profile_repos "$profile")
[ "${#rels[@]}" -gt 0 ] || die "no repos resolved for profile $profile"
log "assembling '$profile' workspace '$name'"
info "workspace: $ws"
info "branch: $branch (created in each repo, off its committed HEAD)"
mkdir -p "$ws"
# -------- create a worktree per repo (mirrored relpath layout) --------
local i n="${#rels[@]}"
SB_DONE_WTS=(); SB_DONE_ORIGINS=()
for ((i=0; i<n; i++)); do
rel="${rels[$i]}"; origin="${origins[$i]}"; base="${bases[$i]}"; wt="${wts[$i]}"
mkdir -p "$(dirname "$wt")"
local gerr
if git -C "$origin" show-ref --verify --quiet "refs/heads/$branch"; then
gerr="$(git -C "$origin" worktree add "$wt" "$branch" 2>&1)" \
|| { _rollback; die "git worktree add failed for $rel (existing branch $branch):"$'\n'" $gerr"; }
else
gerr="$(git -C "$origin" worktree add -b "$branch" "$wt" "$base" 2>&1)" \
|| { _rollback; die "git worktree add -b $branch failed for $rel (base $base):"$'\n'" $gerr"; }
fi
SB_DONE_WTS+=("$wt"); SB_DONE_ORIGINS+=("$origin")
ok "worktree: $rel -> ${wt#$ws/} (branch $branch @ ${base:0:9})"
done
local el_ws="$ws/$EL_REPO_REL"
_write_env "$ws" "$profile" "$name" "$branch" "$el_ws"
_write_manifest "$ws" "$profile" "$name" "$branch"
_write_build "$ws" "$profile" "$el_ws"
_write_readme "$ws" "$profile" "$name" "$branch" "$el_ws"
# -------- summary --------
echo >&2
printf '%s STACK WORKSPACE READY — %s / %s%s\n' "$C_BLD" "$profile" "$name" "$C_0" >&2
printf ' %-11s %s\n' "workspace" "$ws" >&2
printf ' %-11s %s\n' "branch" "$branch (in each repo)" >&2
printf ' %-11s %s\n' "repos" "$n worktrees, mirrored layout" >&2
echo >&2
info "get in: cd $ws && source .stack-env"
info "build all: sandbox build $profile $name # (or: cd $ws && ./build.sh)"
if [ "$profile" = "neuron-stack" ]; then
info "isolated engram (opt-in, via nsbx):"
info " nsbx create $profile-$name --source $el_ws && nsbx up $profile-$name"
fi
info "tear down: sandbox down $profile $name # removes all worktrees; branches kept"
}
# _rollback : remove any worktrees already created this run (globals set by cmd_up)
SB_DONE_WTS=(); SB_DONE_ORIGINS=()
_rollback(){
local j
[ "${#SB_DONE_WTS[@]}" -gt 0 ] && warn "rolling back ${#SB_DONE_WTS[@]} partial worktree(s)"
for ((j=${#SB_DONE_WTS[@]}-1; j>=0; j--)); do
git -C "${SB_DONE_ORIGINS[$j]}" worktree remove --force "${SB_DONE_WTS[$j]}" 2>/dev/null || rm -rf "${SB_DONE_WTS[$j]}"
git -C "${SB_DONE_ORIGINS[$j]}" worktree prune 2>/dev/null || true
done
}
# ---------------------------------------------------------------- writers ------
_write_env(){
local ws="$1" profile="$2" name="$3" branch="$4" el_ws="$5"
local elc_dir="$el_ws/lang/dist/platform"
cat > "$ws/.stack-env" <<ENV
# stack env for '$profile/$name' — SOURCE this to work the whole stack together.
# Pins EL_REPO + PATH at the SANDBOX copy of foundation/el, so elc/elb/runtime and
# every cross-repo ../foundation/el import resolve INSIDE this workspace.
# The live mind (:$LIVE_ENGRAM_PORT engram / :$LIVE_SOUL_PORT soul) is deliberately NOT referenced.
export STACK_NAME="$profile-$name"
export STACK_PROFILE="$profile"
export STACK_ROOT_WS="$ws"
export EL_REPO="$el_ws"
export PATH="$elc_dir:\$PATH" # elc, elb (darwin/linux prebuilt) from the sandbox el
ENV
if [ "$profile" = "neuron-stack" ]; then
cat >> "$ws/.stack-env" <<ENV
export NEURON_REPO="$ws/neuron"
export NEURONUI_REPO="$ws/products/NeuronUI"
# engram/soul are NOT bound here — bring up an ISOLATED engram via nsbx when needed
# (nsbx guards the live store & refuses ports :$LIVE_ENGRAM_PORT/:$LIVE_SOUL_PORT):
# nsbx create $profile-$name --source \$EL_REPO && nsbx up $profile-$name
# nsbx status $profile-$name # prints the isolated engram URL to point the soul at
ENV
fi
# direnv convenience
[ -e "$ws/.envrc" ] || printf 'source_env .stack-env 2>/dev/null || source .stack-env\n' > "$ws/.envrc"
}
_write_manifest(){
local ws="$1" profile="$2" name="$3" branch="$4"
# emit worktree records from git's own worktree list, filtered to this workspace
python3 - "$ws" "$profile" "$name" "$branch" "$DEV_ROOT" <<'PY'
import json, os, subprocess, sys
ws, profile, name, branch, dev = sys.argv[1:6]
repos = []
for rel in sorted(os.listdir(ws)) if False else []:
pass
# discover worktrees by walking one level of relpaths we created
def git(root, *a):
return subprocess.run(["git","-C",root,*a], capture_output=True, text=True).stdout.strip()
for dirpath, dirnames, filenames in os.walk(ws):
if ".git" in filenames or ".git" in dirnames:
rel = os.path.relpath(dirpath, ws)
toplevel = git(dirpath, "rev-parse", "--show-toplevel")
common = git(dirpath, "rev-parse", "--git-common-dir")
origin = os.path.realpath(os.path.join(common, ".."))
head = git(dirpath, "rev-parse", "HEAD")
repos.append({"rel": rel, "worktree": dirpath, "origin": origin,
"branch": branch, "head": head})
dirnames[:] = [] # don't descend into a repo
repos.sort(key=lambda r: r["rel"])
json.dump({"profile": profile, "name": name, "branch": branch,
"workspace": ws, "repos": repos},
open(os.path.join(ws, ".stack-manifest.json"), "w"), indent=2)
PY
}
_write_build(){
local ws="$1" profile="$2" el_ws="$3"
cat > "$ws/build.sh" <<'BUILD'
#!/usr/bin/env bash
# build.sh — build the assembled stack together, in dependency order.
# Generated by `sandbox`. Run from the workspace root (it sources .stack-env).
set -uo pipefail
cd "$(dirname "$0")"; source ./.stack-env
say(){ printf '\033[1m==>\033[0m %s\n' "$*"; }
ok(){ printf ' \033[32m%s\033[0m\n' "$*"; }
bad(){ printf ' \033[31m%s\033[0m\n' "$*"; }
# locate an elc that runs on THIS machine (darwin-arm64 / linux-amd64), from the sandbox el
find_elc(){
local d="$EL_REPO/lang/dist/platform"
case "$(uname -s)-$(uname -m)" in
Darwin-arm64) echo "$d/elc-darwin-arm64";;
Linux-x86_64) echo "$d/elc-linux-amd64";;
*) echo "$d/elc";;
esac
}
ELC="$(find_elc)"; [ -x "$ELC" ] || ELC="$EL_REPO/lang/dist/platform/elc"
say "elc: $ELC"
[ -x "$ELC" ] && ok "$("$ELC" 2>&1 | head -1 || echo present)" || { bad "elc not executable"; exit 1; }
# canonical runtime C to link (CI-published release copy; ~8 copies exist in-tree)
RT="$EL_REPO/lang/releases/v1.0.0-20260501"
[ -f "$RT/el_runtime.c" ] || RT="$EL_REPO/lang/el-compiler/runtime"
[ -f "$RT/el_runtime.c" ] && ok "el_runtime: $RT/el_runtime.c" || bad "no el_runtime.c found under $EL_REPO/lang"
BUILD
if [ "$profile" = "el-stack" ]; then
cat >> "$ws/build.sh" <<'BUILD'
# ---- EL STACK: prove elc + the el stuff (incl. the el-ui framework) build together ----
say "el-ui framework present: $EL_REPO/ui"
[ -d "$EL_REPO/ui" ] && ok "framework dir present ($(ls "$EL_REPO/ui" | tr '\n' ' '))" || bad "no ui/ dir"
# end-to-end compiler proof: elc compiles a real, substantial stack source to C,
# then cc links it against the runtime -> a working native binary.
B="$(mktemp -d)"
say "elc end-to-end: compile engram/src/server.el and link a native binary"
if "$ELC" "$EL_REPO/engram/src/server.el" > "$B/x.c" 2>"$B/elc.err"; then
ok "elc -> C ($(wc -c <"$B/x.c" | tr -d ' ') bytes)"
if cc -std=c11 -O2 -w -I "$RT" -o "$B/x" "$B/x.c" "$RT/el_runtime.c" -lcurl -lpthread -lm 2>"$B/cc.err"; then
ok "cc link ok -> native binary $(ls -lh "$B/x" | awk '{print $5}') (elc + runtime build together)"
else
bad "cc link failed:"; grep -i 'error:' "$B/cc.err" | sort -u | head | sed 's/^/ /'
fi
else
bad "elc compile failed:"; sed 's/^/ /' "$B/elc.err" | head
fi
# optional downstream consumer: forge builds on the SDK (make build) — proves the
# freshly-assembled el SDK still compiles a real downstream repo.
FORGE="$STACK_ROOT_WS/foundation/forge"
if [ -f "$FORGE/Makefile" ]; then
say "downstream consumer: foundation/forge (make build)"
( cd "$FORGE" && EL_REPO="$EL_REPO" PATH="$EL_REPO/lang/dist/platform:$PATH" make build ) \
&& ok "forge built against the sandbox SDK" || bad "forge build failed (see above)"
fi
say "el-stack build complete"
BUILD
else
cat >> "$ws/build.sh" <<'BUILD'
# ---- 1) engram (elc engram/src/server.el -> cc engram.c el_runtime.c), from the sandbox el ----
say "build engram from $EL_REPO/engram/src/server.el"
B="$(mktemp -d)"
if "$ELC" "$EL_REPO/engram/src/server.el" > "$B/engram.c" 2>"$B/elc.err"; then
ok "elc -> engram.c ($(wc -c <"$B/engram.c" | tr -d ' ') bytes)"
if cc -std=c11 -O2 -w -I "$RT" -o "$B/engram" \
"$B/engram.c" "$RT/el_runtime.c" -lcurl -lpthread -lm 2>"$B/cc.err"; then
ok "engram binary built: $(ls -lh "$B/engram" | awk '{print $5}')"
else
bad "engram cc link failed:"; grep -i 'error:' "$B/cc.err" | sort -u | head | sed 's/^/ /'
fi
else
bad "engram elc transpile failed:"; sed 's/^/ /' "$B/elc.err"
fi
# ---- 2) soul (imports ../foundation/el/elp/src/elp.el — resolves to SANDBOX el) ----
say "soul present + cross-repo import resolves inside the sandbox"
[ -f "$NEURON_REPO/soul.el" ] && ok "neuron/soul.el present" || bad "no soul.el"
if [ -f "$EL_REPO/elp/src/elp.el" ]; then
ok "../foundation/el/elp/src/elp.el resolves -> $EL_REPO/elp/src/elp.el (sandbox copy)"
else
bad "elp NLG source missing under sandbox el"
fi
# soul is a heavy single-TU compile; prove elc parses it rather than a full link
if "$ELC" "$NEURON_REPO/soul.el" > "$B/soul.c" 2>"$B/soul.err"; then
ok "elc compiled soul.el -> $(wc -c <"$B/soul.c" | tr -d ' ') bytes of C (cross-repo imports resolved)"
else
bad "soul.el elc compile failed:"; sed 's/^/ /' "$B/soul.err" | head
fi
# ---- 3) UI (present + buildable; gradle/JDK21 is heavy so we don't run it here) ----
say "app/UI present + buildable"
if [ -f "$NEURONUI_REPO/build.sh" ] || [ -f "$NEURONUI_REPO/gradlew" ]; then
ok "NeuronUI build entry present (./build.sh / ./gradlew — needs JDK21; run: cd $NEURONUI_REPO && ./gradlew run)"
else
bad "no NeuronUI build entry"
fi
say "neuron-stack build complete (engram compiled, soul compiled, UI present & buildable)"
BUILD
fi
chmod +x "$ws/build.sh"
}
_write_readme(){
local ws="$1" profile="$2" name="$3" branch="$4" el_ws="$5"
cat > "$ws/README.md" <<MD
# $profile / $name — combined stack workspace
Assembled by \`sandbox\`. Every constituent repo is a **git worktree** on branch
\`$branch\`, forked off its origin repo's committed HEAD, laid out at its natural
relative path so cross-repo \`../foundation/el\` imports resolve **inside this
workspace** (the sandbox el), never the live tree.
## Get in
\`\`\`bash
cd $ws
source .stack-env # EL_REPO + PATH now point at the sandbox el
./build.sh # build the stack together (or: sandbox build $profile $name)
\`\`\`
## Layout
__STACK_LAYOUT__
## Isolation
- Worktrees only; the live soul/engram (:$LIVE_SOUL_PORT / :$LIVE_ENGRAM_PORT) are never touched.
- To run against an **isolated engram**, delegate to \`nsbx\` (guards the live store/ports):
\`\`\`bash
nsbx create $profile-$name --source \$EL_REPO && nsbx up $profile-$name
\`\`\`
## Tear down
\`\`\`bash
sandbox down $profile $name # remove every worktree; branch '$branch' kept
sandbox down $profile $name --delete-branch
\`\`\`
MD
# fill the layout list from the manifest without embedding backticks in the heredoc
python3 - "$ws" <<'PY'
import json, os, sys
ws = sys.argv[1]
d = json.load(open(os.path.join(ws, ".stack-manifest.json")))
lines = ["- `%s` <- worktree of %s" % (r["rel"], r["origin"]) for r in d["repos"]]
p = os.path.join(ws, "README.md")
txt = open(p).read().replace("__STACK_LAYOUT__", "\n".join(lines))
open(p, "w").write(txt)
PY
}
# ================================================================ down =========
cmd_down(){
local profile="$1" name="$2"; shift 2 || true
local del_branch=0
while [ $# -gt 0 ]; do case "$1" in
--delete-branch) del_branch=1; shift;;
*) die "unknown flag: $1";;
esac; done
need git
local ws; ws="$(ws_dir "$profile" "$name")"
[ -d "$ws" ] || die "no such workspace: $ws"
local mf; mf="$(manifest "$ws")"
[ -f "$mf" ] || die "no manifest in $ws (refusing to guess); remove it by hand if intended"
local branch; branch="$(python3 -c "import json;print(json.load(open('$mf'))['branch'])")"
log "tearing down '$profile/$name' ($ws)"
# remove each worktree through its origin repo
python3 -c "import json;[print(r['origin']+'\t'+r['worktree']) for r in json.load(open('$mf'))['repos']]" \
| while IFS=$'\t' read -r origin wt; do
if [ -d "$wt" ]; then
git -C "$origin" worktree remove --force "$wt" 2>/dev/null || rm -rf "$wt"
git -C "$origin" worktree prune 2>/dev/null || true
ok "removed worktree: ${wt#$ws/}"
fi
if [ "$del_branch" -eq 1 ]; then
git -C "$origin" branch -D "$branch" 2>/dev/null && ok "deleted branch $branch in ${origin#$DEV_ROOT/}" || true
fi
done
# drop the (now worktree-free) workspace tree
rm -rf "$ws"
ok "workspace removed: $ws"
[ "$del_branch" -eq 1 ] || info "branch '$branch' kept in each repo (use --delete-branch to drop)"
ok "down '$profile/$name' complete (live untouched)"
}
# ================================================================ build ========
cmd_build(){
local profile="$1" name="$2"; local ws; ws="$(ws_dir "$profile" "$name")"
[ -x "$ws/build.sh" ] || die "no build.sh in $ws (is it assembled? sandbox $profile $name)"
exec "$ws/build.sh"
}
# ================================================================ list/status ==
cmd_list(){
[ -d "$STACK_ROOT" ] || { info "no stack workspaces (root $STACK_ROOT absent)"; return 0; }
local mf found=0
for mf in "$STACK_ROOT"/*/.stack-manifest.json; do
[ -f "$mf" ] || continue; found=1
python3 -c "import json;d=json.load(open('$mf'));print(' %-22s %-8s %2d repos branch=%s'%(d['profile']+'/'+d['name'],'',len(d['repos']),d['branch']))" 2>/dev/null
done
[ "$found" -eq 1 ] || info "no assembled stack workspaces under $STACK_ROOT"
}
cmd_status(){
local profile="$1" name="$2"; local ws; ws="$(ws_dir "$profile" "$name")"
local mf; mf="$(manifest "$ws")"; [ -f "$mf" ] || die "no such workspace: $ws"
log "stack '$profile/$name'"; info "workspace: $ws"
python3 - "$mf" <<'PY'
import json,sys,subprocess
d=json.load(open(sys.argv[1]))
print(f" branch: {d['branch']}")
for r in d['repos']:
st=subprocess.run(["git","-C",r["worktree"],"status","--porcelain"],capture_output=True,text=True).stdout
n=len([l for l in st.splitlines() if l.strip()])
print(f" {r['rel']:<20} {r['head'][:9]} {'clean' if n==0 else str(n)+' changed'}")
PY
}
# ================================================================ usage/main ===
usage(){ cat >&2 <<EOF
${C_BLD}sandbox${C_0} — assemble a WHOLE Neuron stack into one combined worktree workspace,
wired to build together on an isolated clean base. Multi-repo sibling of ${C_BLD}nsbx${C_0}.
${C_CYN}sandbox el-stack <name>${C_0} [--minimal] elc + EL language + el-ui framework + tooling (+ SDK consumers)
${C_CYN}sandbox neuron-stack <name>${C_0} [--minimal] runtime/soul + engram + app/UI (the full product)
${C_CYN}sandbox build <profile> <name>${C_0} build the assembled stack together (runs its build.sh)
${C_CYN}sandbox status <profile> <name>${C_0} inspect one workspace
${C_CYN}sandbox list${C_0} list assembled workspaces
${C_CYN}sandbox down <profile> <name>${C_0} [--delete-branch] tear down (remove worktrees; branch kept)
Flags: --minimal only the required repos --branch B branch name --base REF fork point
Env: NSBX_STACK_ROOT (workspace root, default \$DEV_ROOT/stack-worktrees) NEURON_DEV_ROOT
Each constituent repo becomes a git worktree at its natural relpath inside the
workspace, so cross-repo ../foundation/el imports resolve to the SANDBOX el. The
live soul/engram (:$LIVE_SOUL_PORT / :$LIVE_ENGRAM_PORT) are never touched; isolated-engram
bring-up is delegated to nsbx.
EOF
}
main(){
local cmd="${1:-}"; shift || true
case "$cmd" in
el-stack|neuron-stack) cmd_up "$cmd" "$@";;
up) [ $# -ge 1 ] || die "usage: sandbox up <profile> <name>"; local p="$1"; shift; cmd_up "$p" "$@";;
down) [ $# -ge 2 ] || die "usage: sandbox down <profile> <name>"; cmd_down "$@";;
build) [ $# -ge 2 ] || die "usage: sandbox build <profile> <name>"; cmd_build "$@";;
status) [ $# -ge 2 ] || die "usage: sandbox status <profile> <name>"; cmd_status "$@";;
list|ls) cmd_list "$@";;
""|-h|--help|help) usage;;
*) die "unknown command: $cmd (try: sandbox help)";;
esac
}
main "$@"