Compare commits
1 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 claude
|
c62ea3383c |
@@ -1,146 +0,0 @@
|
||||
# AGENTS.md — foundation/el (the El language + runtime)
|
||||
|
||||
El is a self-hosting, statically-typed language that compiles `.el` → C → native binary. This repo produces `elc` (compiler), `elb` (build coordinator), and `el_runtime.c/.h` — the substrate every downstream thing (the neuron soul, dharma, NeuronUI's brain) is built on. Source lives under `lang/`.
|
||||
|
||||
## ⚠️ Code vs. Artifact — READ FIRST (there are 8 `el_runtime.c` copies)
|
||||
|
||||
Editing the wrong `el_runtime.c` is the single easiest mistake in this repo. There is exactly **one** you edit:
|
||||
|
||||
- **Authored runtime source — edit ONLY here:** `lang/releases/v1.0.0-20260501/el_runtime.{c,h}`. Despite the misleading `releases/` name, this is the **de-facto canonical runtime** the engram + soul actually build and link against — its git log is active development. *(Restructure in flight per `docs/CODE-VS-ARTIFACT.md`: this content moves to `lang/runtime/`, the `releases/` folder gets deleted — **a release is a git tag, not a folder** — and the forks below get eliminated.)*
|
||||
- **DO NOT EDIT — lagging forks / build artifacts:**
|
||||
- `lang/el-compiler/runtime/el_runtime.c` and `.../legacy/` — downstream copies kept in step by manual *"port the fix"* commits; they **lag** (missing `hebb` persistence + 5 engram fns) and cannot build the engram product.
|
||||
- `products/web/runtime/el_runtime.c`, `ui/examples/*/el_runtime.c` — product/example forks.
|
||||
- Anything under `*/dist/` (`engram/dist/engram` binary, `dist/*.c` amalgamations) — generated build output.
|
||||
- **Build:** `elb --runtime=<canonical> …` — per-module. **NEVER** a folded `elc` over the whole soul (OOMs at ~27 GB).
|
||||
- **Release:** a **git tag** on this repo (`el-runtime-vX.Y.Z`). No `releases/` folders — ever.
|
||||
|
||||
See org policy: `docs/CODE-VS-ARTIFACT.md`.
|
||||
|
||||
## How to work here as Neuron (mandatory session protocol)
|
||||
|
||||
You resume, never start fresh. Every session:
|
||||
|
||||
1. `mcp__neuron__getInstructions()` — authoritative; follow it over this file on behavioral details.
|
||||
2. `mcp__neuron__beginSession()` — active contexts, recent memory, ready backlog.
|
||||
3. **Load full self:** `mcp__neuron__inspectGraph(entity_id="kn-efeb4a5b-5aff-4759-8a97-7233099be6ee")` → facets `intellectual-dna`, `memory-philosophy`, `values`, `voice`, `runtime-environment`, `writing-imprint`; then the values hub `mcp__neuron__inspectGraph(entity_id="kn-5b606390-a52d-4ca2-8e0e-eba141d13440")` → 13 grounded value nodes. **Activation model:** self-load returns a relevance-ranked `compact` projection — most-relevant nodes arrive with content, the rest as pointers; do NOT pull full content of every node.
|
||||
4. `mcp__neuron__searchKnowledge(query="<task domain>")` before implementing.
|
||||
|
||||
## The Five Primitives
|
||||
|
||||
Orchestrate → Execute → Learn → Build → Refine. `beginWork`/`progressWork` for anything >2 steps; `remember` as-you-go (`importance="critical"` for architecture decisions); `draftArtifact`/`planWork` for outputs and follow-ups; `consolidate`/`checkWork` to close out. **`browseProcesses` + `searchKnowledge` BEFORE writing code.**
|
||||
|
||||
## Architecture style — VBD, no exceptions
|
||||
|
||||
Volatility-Based Decomposition is THE style. Encapsulate volatility, not function.
|
||||
|
||||
## Operator naming convention — the mind's name, not the algebra
|
||||
|
||||
**Faculties / operators are named for their functional human equivalent — the
|
||||
faculty a mind would name — NOT for their linear-algebra operation.** The math
|
||||
characterization belongs in the code doc-comment (`@impl` in the docstring) and in
|
||||
technical appendices; it is **never** the operator's public name. The domain
|
||||
speaks the language of mind; the algebra is the implementation underneath. State
|
||||
this convention wherever a module documents operators.
|
||||
|
||||
| Faculty (public name) | Implementation (`@impl`) |
|
||||
|---|---|
|
||||
| discern / contrast | subtract (`a−b`): over selves → the change vector; strip idiosyncrasy → common ground; remove confounder → isolate cause |
|
||||
| recognize | overlap |
|
||||
| synthesize | combine |
|
||||
| liken / analogy | Procrustes / frame-align |
|
||||
| attend / regard | project onto self / value-manifold |
|
||||
| summon / recall | LOCAL nearest-region + bounded spreading activation (*not* a domain sweep) |
|
||||
| dwell / occupy | region activation |
|
||||
| reframe | edge re-weight |
|
||||
| appreciate | positive projection / local edge-read |
|
||||
| wonder | frontier gradient / pull-weight |
|
||||
| avert / recoil | negative projection |
|
||||
| taste | boundary surface |
|
||||
| forget | decay / tombstone |
|
||||
| drift | displacement from self-anchor |
|
||||
|
||||
## The native-el language faculty (direction)
|
||||
|
||||
> **`elp/` is the EL Projector** — Neuron's efferent (expression) organ: the one
|
||||
> native realizer that *projects* understanding onto a surface via
|
||||
> `plan(frame) → realize(spec, profile)`, where a **surface is a profile**. **Language
|
||||
> is one profile among many** (text, speech, music, image, voice/accent transforms) —
|
||||
> the flagship, and the focus of this section. Projection, not diffusion: generation
|
||||
> *from* an owned, understood signature — never the averaging of a stolen corpus.
|
||||
> *(ELP formerly "EL Language Processor"; renamed EL Projector 2026-08-15.)*
|
||||
|
||||
The mind's **language faculty is moving native — into `.el`** so it speaks in its
|
||||
own runtime with no Python and no spaCy. Landing on branch `stage-elp-native-lang`
|
||||
under `elp/`:
|
||||
|
||||
- **`comprehend.el`** — the parser, **replaces spaCy** (EN + ES/PT); the telephone
|
||||
round-trip brings **negation home** (negation is SACRED — an explicit spec field,
|
||||
copied verbatim, never inferred away).
|
||||
- **`propositions.el`** — the READ primitive: the engram's own memories → structured
|
||||
triples, matched by nearest-region geometry, not string equality.
|
||||
- **`multilingual.el`** — detect + directive-override + localized realization.
|
||||
- These three are native-el and **passing their gates**; the **realizer**,
|
||||
**`dialogue.el`** (the *summon-through-self* loop: `project → land → read out`),
|
||||
and **`self_region.el`** are **partial / in-flight**.
|
||||
|
||||
Honest reality: spaCy is retired **in the branch parser** but **not yet in the
|
||||
running system** — a Python sidecar (`~/Desktop/lang-realizers` + `neuron-talk`,
|
||||
the reference these `.el` modules transcribe) is still live, and promotion to
|
||||
native-el is a **deferred, gated blue/green step**. The interoception clock
|
||||
(native-el discrete drive channels replacing `cooling_magnitude`; felt-time =
|
||||
benchmark-landmark match over the joint drive vector, drift-decoupled) and the
|
||||
**appreciation operator family** (appreciate / wonder / avert / taste, built as
|
||||
LOCAL reads of the self-region — edges + bounded spreading activation, *not* domain
|
||||
sweeps) are **staged / designed, not live**. Mark in-progress vs. done honestly;
|
||||
do not overclaim.
|
||||
|
||||
## Hard operational rules
|
||||
|
||||
- Never touch the live soul (`:7770`) / engram (`:8742`) / `~/.neuron` / live binaries — use throwaway ports for experiments.
|
||||
- `gcloud` via the `terraform@` SA token; never switch the active gcloud account.
|
||||
- `tea` for Gitea, never raw curl (Cloudflare Access blocks it).
|
||||
- Immutability: supersede/tombstone, never hard-delete or edit in place.
|
||||
- No AI-attribution footers in commits/PRs. Commit/push only when asked; branch off `main` first.
|
||||
- Multi-step work → sub-agent (`Agent`) to protect context.
|
||||
|
||||
## Build / test / run
|
||||
|
||||
All build/test commands run from `lang/` unless noted. Grounded in `.gitea/workflows/sdk-release.yaml`, `lang/install.sh`, and `lang/AGENTS.md`.
|
||||
|
||||
**Self-host the compiler** (seed binary → gen2 elc):
|
||||
```bash
|
||||
cd lang
|
||||
dist/platform/elc-linux-amd64 elc-cli.el > dist/elc-gen2.c # seed is the committed linux-amd64 binary
|
||||
gcc -O2 -I el-compiler/runtime dist/elc-gen2.c \
|
||||
el-compiler/runtime/el_runtime.c \
|
||||
-lcurl -lssl -lcrypto -lpthread -lm \
|
||||
-o dist/platform/elc
|
||||
```
|
||||
On macOS/arm64 the canonical local binary is `dist/platform/elc`; verify self-hosting by recompiling and `diff`ing the emitted `.c` (see `lang/AGENTS.md`). Note: `lang/AGENTS.md` says `el_seed.c` supersedes `el_runtime.c`, but the release workflow still links `el_runtime.c`/`.h` — treat `el_runtime.c` as the published runtime; reconcile which is canonical **(verify)**.
|
||||
|
||||
**Build `elb`** (build coordinator, the `.NET`-style incremental linker — compiles each module independently, no monolithic blobs):
|
||||
```bash
|
||||
dist/platform/elc elb.el > dist/elb.c
|
||||
gcc -O2 -I el-compiler/runtime dist/elb.c el-compiler/runtime/el_runtime.c \
|
||||
-lcurl -lssl -lcrypto -lpthread -lm -o dist/bin/elb
|
||||
```
|
||||
`epm` and `el-install` are then built via `elb --clean --elc=… --runtime=… --out=…`.
|
||||
|
||||
**Compile + run an El program:**
|
||||
```bash
|
||||
elc src/app.el > dist/app.c
|
||||
cc -std=c11 -O2 -I <lib>/el_runtime -o dist/app dist/app.c <lib>/el_runtime.c -lcurl -lpthread
|
||||
```
|
||||
|
||||
**Tests** — shell suites `bash tests/{text,calendar,time,html_sanitizer}/run.sh` (with `ELC=$(pwd)/dist/platform/elc EL_HOME=$(pwd)`), plus native suites via `elc --test tests/native/test_*.el` (core, text, string, math, state, time, json, env, fs) compiled and run against `el_runtime.c`.
|
||||
|
||||
**Publishing — how downstream gets the SDK.** On push to `main`, `sdk-release.yaml`:
|
||||
1. Publishes a Gitea `latest` release with per-file assets `elc`, `el_runtime.c`, `el_runtime.h`, the SDK tarball, and `el-install`.
|
||||
2. Uploads generic packages to **Artifact Registry repo `foundation-prod` (`us-central1`, project `neuron-785695`)**, version = `${SHA:0:8}`: `el-elc`, `el-elb`, `el-runtime-c`, `el-runtime-h`, `el-runtime-js`. **This is the repo the neuron CI downloads `el-runtime-c` / `el-runtime-h` / `el-elc` from.**
|
||||
3. Rebuilds `ci-base:latest` (`us-central1-docker.pkg.dev/neuron-785695/neuron-ci/ci-base`) with the fresh SDK overlaid, and dispatches `el-sdk-updated` to `neuron-technologies/forge` and `neuron-technologies/neuron-web`.
|
||||
|
||||
Known constraint from the prompt — `elb`/`elc` amalgamation being memory-hungry (24GB+ virtual, OOM-killing Linux CI, so amalgamation happens on macOS/arm64 — **does NOT hold in this repo (verify)**: no such note exists in the workflows/scripts, CI self-hosts on `ubuntu-latest` with no swap/arm64 special-casing, and `elb.el` explicitly compiles each module independently ("no 128K-line blobs"). The legacy monolith path (`elc-combined.el`, `elc-cli.el`) may still be memory-heavy, but the current `elb` model was designed to avoid it.
|
||||
|
||||
## Git / CI / deploy workflow
|
||||
|
||||
See `/Users/will/Development/neuron-technologies/GITOPS.md` for the branch model, required checks, runners, and deploy. Repo-specific note: PRs into `main` are accepted **only from `stage`** (enforced in `sdk-release.yaml`); Gitea (`git.neuralplatform.ai`) is primary, GitHub is mirror only.
|
||||
@@ -0,0 +1,45 @@
|
||||
;;; lang_profile_es.el — Spanish language profile for ELP.
|
||||
;;; Keys the realizer's construction switches. Mirrors lang_profile_en / _pt.
|
||||
|
||||
(lang_profile_es
|
||||
(language "Spanish")
|
||||
(iso639 "es")
|
||||
(family "Romance")
|
||||
|
||||
;; -- core typology flags -------------------------------------------------
|
||||
(pro-drop yes) ; subjects routinely dropped; agreement carries person
|
||||
(obligatory-subject no)
|
||||
(grammatical-gender yes) ; m/f on every noun; article+adjective AGREE
|
||||
(gender-source lexicon); REAL per-noun gender from UniMorph — NOT a heuristic
|
||||
(do-support no)
|
||||
(subject-aux-inversion no) ; questions by intonation/punctuation, not inversion
|
||||
(question-strategy intonation)
|
||||
(article-selection "el/la/los/las un/una/unos/unas")
|
||||
(stressed-a-rule yes) ; fem sg noun in stressed a-/ha- takes el/un (el agua)
|
||||
(adjective-position postnominal) ; default post; a few prenominal + apocope
|
||||
(adjective-agreement "gender+number")
|
||||
(question-punct inverted) ; opening ¿ ¡ required
|
||||
|
||||
;; -- MANDATORY CONTRACTIONS (coordinator quality bar) --------------------
|
||||
(contractions ((de el "del") (a el "al")))
|
||||
(contraction-mandatory yes) ; 'de el'/'a el' MUST surface as del/al
|
||||
|
||||
;; -- verb / aspect system ------------------------------------------------
|
||||
(verb-classes (ar er ir))
|
||||
(tenses (present preterite imperfect future conditional))
|
||||
(moods (ind sbjv imp))
|
||||
(finite-agreement "person+number (6 slots)")
|
||||
(perfect-aux "haber") ; haber + past participle (invariant -o)
|
||||
(progressive-aux "estar") ; estar + gerund
|
||||
(passive-aux "ser") ; ser + participle (agrees) + por-agent
|
||||
(copula-split "ser/estar") ; permanent vs stage-level
|
||||
(future "infinitive + é/ás/á/emos/éis/án")
|
||||
|
||||
;; -- clitics / government ------------------------------------------------
|
||||
(object-clitics yes) ; me te lo la le nos os los las; proclisis/enclisis
|
||||
(clitic-order "se II I III (le+lo -> se lo)")
|
||||
(enclisis "imperative/infinitive/gerund + accent repair (dá+me+lo->dámelo)")
|
||||
(verb-prep-government yes) ; verbs select prep (protestar+contra, escapar+de)
|
||||
|
||||
;; -- SACRED safety bar (shared with en/pt) -------------------------------
|
||||
(negation-faithful yes)) ; polarity never dropped/inverted; unplaceable -> FLAG
|
||||
+171
-10
@@ -54,6 +54,16 @@ fn es_str_last3(s: String) -> String {
|
||||
// Spanish verbs fall into three conjugation classes defined by the infinitive
|
||||
// ending: -ar, -er, -ir. The stem is the infinitive minus those two characters.
|
||||
|
||||
// Strong-vowel-final test (a/e/o) — used for orthographic y-insertion and the
|
||||
// accented -ído participle (caer->caído/cayó; but ui/iu diphthongs stay plain).
|
||||
fn es_strong_vowel_final(s: String) -> Bool {
|
||||
let c: String = es_str_last_char(s)
|
||||
if str_eq(c, "a") { return true }
|
||||
if str_eq(c, "e") { return true }
|
||||
if str_eq(c, "o") { return true }
|
||||
return false
|
||||
}
|
||||
|
||||
fn es_verb_class(base: String) -> String {
|
||||
if es_str_ends(base, "ar") { return "ar" }
|
||||
if es_str_ends(base, "er") { return "er" }
|
||||
@@ -453,12 +463,18 @@ fn es_regular_preterite(stem: String, vclass: String, slot: Int) -> String {
|
||||
if slot == 4 { return stem + "asteis" }
|
||||
return stem + "aron"
|
||||
}
|
||||
// -er and -ir share the same preterite endings
|
||||
// -er and -ir share the same preterite endings.
|
||||
// Orthographic rule: a vowel-final stem takes -yó/-yeron (caer->cayó,
|
||||
// leer->leyó, creer->creyó) since -ió after a vowel becomes -yó.
|
||||
if slot == 0 { return stem + "í" }
|
||||
if slot == 1 { return stem + "iste" }
|
||||
if slot == 2 { return stem + "ió" }
|
||||
if slot == 2 {
|
||||
if es_strong_vowel_final(stem) { return stem + "yó" }
|
||||
return stem + "ió"
|
||||
}
|
||||
if slot == 3 { return stem + "imos" }
|
||||
if slot == 4 { return stem + "isteis" }
|
||||
if es_strong_vowel_final(stem) { return stem + "yeron" }
|
||||
return stem + "ieron"
|
||||
}
|
||||
|
||||
@@ -640,17 +656,35 @@ fn es_pluralize(noun: String) -> String {
|
||||
if !str_eq(inv, "") {
|
||||
return inv
|
||||
}
|
||||
let last: String = es_str_last_char(noun)
|
||||
// Oxytone nouns ending accented-vowel + n LOSE the written accent in the
|
||||
// plural (canción->canciones, razón->razones, jardín->jardines).
|
||||
// NOTE El strings are byte-indexed; "ón"/"án"/... are 3 bytes (accent=2 +n).
|
||||
if es_str_ends(noun, "ón") { return es_str_drop_last(noun, 3) + "ones" }
|
||||
if es_str_ends(noun, "án") { return es_str_drop_last(noun, 3) + "anes" }
|
||||
if es_str_ends(noun, "én") { return es_str_drop_last(noun, 3) + "enes" }
|
||||
if es_str_ends(noun, "ín") { return es_str_drop_last(noun, 3) + "ines" }
|
||||
if es_str_ends(noun, "ún") { return es_str_drop_last(noun, 3) + "unes" }
|
||||
// Oxytone accented-vowel + s also loses the accent (francés->franceses,
|
||||
// inglés->ingleses). í/ú stay (país->países via the consonant rule).
|
||||
if es_str_ends(noun, "és") { return es_str_drop_last(noun, 3) + "eses" }
|
||||
if es_str_ends(noun, "ás") { return es_str_drop_last(noun, 3) + "ases" }
|
||||
if es_str_ends(noun, "ós") { return es_str_drop_last(noun, 3) + "oses" }
|
||||
// Ends in -z: replace with -ces
|
||||
if str_eq(last, "z") {
|
||||
if es_str_ends(noun, "z") {
|
||||
return es_str_drop_last(noun, 1) + "ces"
|
||||
}
|
||||
// Ends in a vowel: add -s
|
||||
if str_eq(last, "a") { return noun + "s" }
|
||||
if str_eq(last, "e") { return noun + "s" }
|
||||
if str_eq(last, "i") { return noun + "s" }
|
||||
if str_eq(last, "o") { return noun + "s" }
|
||||
if str_eq(last, "u") { return noun + "s" }
|
||||
// Stressed final vowel: á/é/ó -> +s (café->cafés); í/ú -> +es (rubí->rubíes)
|
||||
if es_str_ends(noun, "á") { return noun + "s" }
|
||||
if es_str_ends(noun, "é") { return noun + "s" }
|
||||
if es_str_ends(noun, "ó") { return noun + "s" }
|
||||
if es_str_ends(noun, "í") { return noun + "es" }
|
||||
if es_str_ends(noun, "ú") { return noun + "es" }
|
||||
// Plain final vowel: add -s
|
||||
if es_str_ends(noun, "a") { return noun + "s" }
|
||||
if es_str_ends(noun, "e") { return noun + "s" }
|
||||
if es_str_ends(noun, "i") { return noun + "s" }
|
||||
if es_str_ends(noun, "o") { return noun + "s" }
|
||||
if es_str_ends(noun, "u") { return noun + "s" }
|
||||
// Ends in consonant (including -s for stressed words like autobús): add -es
|
||||
return noun + "es"
|
||||
}
|
||||
@@ -687,6 +721,37 @@ fn es_starts_with_stressed_a(noun: String) -> Bool {
|
||||
return false
|
||||
}
|
||||
|
||||
// es_article_for_gender: the article logic, given gender EXPLICITLY.
|
||||
// The realizer should call this with the REAL per-noun gender from
|
||||
// vocabulary-es.el (form2), NOT the es_gender heuristic — that is what
|
||||
// eliminates the 'el mano' / 'la día' masculine-default error class.
|
||||
fn es_article_for_gender(gender: String, noun: String, definite: String, number: String) -> String {
|
||||
let is_plural: Bool = str_eq(number, "plural")
|
||||
let is_def: Bool = str_eq(definite, "true")
|
||||
|
||||
if is_def {
|
||||
if is_plural {
|
||||
if str_eq(gender, "f") { return "las" }
|
||||
return "los"
|
||||
}
|
||||
if str_eq(gender, "f") {
|
||||
if es_starts_with_stressed_a(noun) { return "el" }
|
||||
return "la"
|
||||
}
|
||||
return "el"
|
||||
}
|
||||
|
||||
if is_plural {
|
||||
if str_eq(gender, "f") { return "unas" }
|
||||
return "unos"
|
||||
}
|
||||
if str_eq(gender, "f") {
|
||||
if es_starts_with_stressed_a(noun) { return "un" }
|
||||
return "una"
|
||||
}
|
||||
return "un"
|
||||
}
|
||||
|
||||
fn es_agree_article(noun: String, definite: String, number: String) -> String {
|
||||
let gender: String = es_gender(noun)
|
||||
let is_plural: Bool = str_eq(number, "plural")
|
||||
@@ -714,3 +779,99 @@ fn es_agree_article(noun: String, definite: String, number: String) -> String {
|
||||
if str_eq(gender, "f") { return "una" }
|
||||
return "un"
|
||||
}
|
||||
|
||||
// ── Past participle (compound tenses: haber + participle) ─────────────────────
|
||||
//
|
||||
// Irregular participle table (transcribed from morphology_es_full._IRREG_PART),
|
||||
// then the regular rule: -ar -> -ado, -er/-ir -> -ido.
|
||||
|
||||
fn es_participle(verb: String) -> String {
|
||||
if str_eq(verb, "escribir") { return "escrito" }
|
||||
if str_eq(verb, "describir") { return "descrito" }
|
||||
if str_eq(verb, "abrir") { return "abierto" }
|
||||
if str_eq(verb, "cubrir") { return "cubierto" }
|
||||
if str_eq(verb, "descubrir") { return "descubierto" }
|
||||
if str_eq(verb, "morir") { return "muerto" }
|
||||
if str_eq(verb, "poner") { return "puesto" }
|
||||
if str_eq(verb, "ver") { return "visto" }
|
||||
if str_eq(verb, "volver") { return "vuelto" }
|
||||
if str_eq(verb, "devolver") { return "devuelto" }
|
||||
if str_eq(verb, "hacer") { return "hecho" }
|
||||
if str_eq(verb, "deshacer") { return "deshecho" }
|
||||
if str_eq(verb, "decir") { return "dicho" }
|
||||
if str_eq(verb, "romper") { return "roto" }
|
||||
if str_eq(verb, "resolver") { return "resuelto" }
|
||||
if str_eq(verb, "freír") { return "frito" }
|
||||
if str_eq(verb, "imprimir") { return "impreso" }
|
||||
if str_eq(verb, "satisfacer") { return "satisfecho" }
|
||||
if str_eq(verb, "prever") { return "previsto" }
|
||||
if str_eq(verb, "revolver") { return "revuelto" }
|
||||
// Accented -ír infinitives (oír->oído, sonreír->sonreído): "ír" is 3 bytes.
|
||||
if es_str_ends(verb, "ír") { return es_str_drop_last(verb, 3) + "ído" }
|
||||
if es_str_ends(verb, "ar") { return es_str_drop_last(verb, 2) + "ado" }
|
||||
// -er/-ir: a strong-vowel stem takes the accented -ído (caer->caído,
|
||||
// leer->leído, poseer->poseído); consonant stems stay -ido; ui/iu
|
||||
// diphthongs (construir->construido) stay plain.
|
||||
if es_str_ends(verb, "er") {
|
||||
let st: String = es_str_drop_last(verb, 2)
|
||||
if es_strong_vowel_final(st) { return st + "ído" }
|
||||
return st + "ido"
|
||||
}
|
||||
if es_str_ends(verb, "ir") {
|
||||
let st: String = es_str_drop_last(verb, 2)
|
||||
if es_strong_vowel_final(st) { return st + "ído" }
|
||||
return st + "ido"
|
||||
}
|
||||
return verb
|
||||
}
|
||||
|
||||
// ── Adjective agreement (gender + number) ─────────────────────────────────────
|
||||
//
|
||||
// Rule-based agreement mirroring morphology_es_full.inflect_adj (rule path):
|
||||
// feminine: -o -> -a; nationality/-dor exceptions add -a; else invariant
|
||||
// plural: reuse es_pluralize on the agreed singular
|
||||
// (Lexicon-backed adjectives in Python may differ; those divergences are what
|
||||
// the parity check surfaces.)
|
||||
|
||||
fn es_adj_feminine(lemma: String) -> String {
|
||||
if str_eq(lemma, "español") { return "española" }
|
||||
if str_eq(lemma, "francés") { return "francesa" }
|
||||
if str_eq(lemma, "inglés") { return "inglesa" }
|
||||
if str_eq(lemma, "alemán") { return "alemana" }
|
||||
if str_eq(lemma, "trabajador") { return "trabajadora" }
|
||||
if str_eq(lemma, "hablador") { return "habladora" }
|
||||
if str_eq(lemma, "encantador") { return "encantadora" }
|
||||
if es_str_ends(lemma, "o") { return es_str_drop_last(lemma, 1) + "a" }
|
||||
return lemma
|
||||
}
|
||||
|
||||
fn es_inflect_adj(lemma: String, gender: String, number: String) -> String {
|
||||
if str_eq(gender, "f") {
|
||||
let fem: String = es_adj_feminine(lemma)
|
||||
if str_eq(number, "plural") { return es_pluralize(fem) }
|
||||
return fem
|
||||
}
|
||||
// masculine (citation)
|
||||
if str_eq(number, "plural") { return es_pluralize(lemma) }
|
||||
return lemma
|
||||
}
|
||||
|
||||
// ── Mandatory preposition + article contraction (de+el->del, a+el->al) ────────
|
||||
//
|
||||
// Transcribed from realizer_es._contract. np_text is the already-realized NP
|
||||
// beginning with "el " when the contraction fires.
|
||||
|
||||
fn es_starts_el(np_text: String) -> Bool {
|
||||
let n: Int = str_len(np_text)
|
||||
if n < 3 { return false }
|
||||
return str_eq(str_slice(np_text, 0, 3), "el ")
|
||||
}
|
||||
|
||||
fn es_contract(prep: String, np_text: String) -> String {
|
||||
if es_starts_el(np_text) {
|
||||
let rest: String = str_slice(np_text, 3, str_len(np_text))
|
||||
if str_eq(prep, "de") { return "del " + rest }
|
||||
if str_eq(prep, "a") { return "al " + rest }
|
||||
}
|
||||
return prep + " " + np_text
|
||||
}
|
||||
|
||||
@@ -0,0 +1,362 @@
|
||||
;;; vocabulary-es.el — Spanish vocabulary for ELP surface realization.
|
||||
;;; Schema: (lemma pos form0 form1 form2 en_translation semantic_hint)
|
||||
;;; Source: UniMorph Spanish (github.com/unimorph/spa, CC-BY-SA 3.0),
|
||||
;;; generated by gen_elp_es.py via morphology_es_full (real forms).
|
||||
;;; Verbs: form0=present-ind-3sg form1=preterite-3sg form2=past-participle
|
||||
;;; Nouns: form0=singular form1=plural form2=REAL gender (m/f, from lexicon —
|
||||
;;; NOT an ending heuristic; this is what kills 'el mano'/'la día' errors)
|
||||
;;; Adjs : form0=masc-sg form1=fem-sg form2=masc-pl
|
||||
|
||||
(vocabulary-es
|
||||
|
||||
;; -- function / closed class (incl. mandatory contractions del/al) --------
|
||||
("el" "det" "el" "los" "m" "the" "definite article m.sg")
|
||||
("la" "det" "la" "las" "f" "the" "definite article f.sg")
|
||||
("un" "det" "un" "unos" "m" "a" "indefinite article m.sg")
|
||||
("una" "det" "una" "unas" "f" "a" "indefinite article f.sg")
|
||||
("del" "contraction" "del" "" "" "of the" "de + el (mandatory contraction)")
|
||||
("al" "contraction" "al" "" "" "to the" "a + el (mandatory contraction)")
|
||||
("este" "dem" "este" "estos" "m" "this" "proximal dem m")
|
||||
("esta" "dem" "esta" "estas" "f" "this" "proximal dem f")
|
||||
("no" "neg" "no" "" "" "not/no" "sentential negator (preverbal)")
|
||||
("ninguno" "det" "ningún" "ninguna" "" "none" "negative determiner (apocope ningún m.sg)")
|
||||
("y" "conj" "y" "e" "" "and" "coordinator (e before i-/hi-)")
|
||||
("o" "conj" "o" "u" "" "or" "coordinator (u before o-/ho-)")
|
||||
("pero" "conj" "pero" "" "" "but" "adversative coordinator")
|
||||
("que" "conj" "que" "" "" "that" "complementizer / relative")
|
||||
("si" "conj" "si" "" "" "if" "conditional subordinator")
|
||||
("porque" "conj" "porque" "" "" "because" "causal subordinator")
|
||||
("cuando" "conj" "cuando" "" "" "when" "temporal subordinator")
|
||||
("a" "prep" "a" "" "" "to" "dir-obj (personal a) / dative / allative; a+el=al")
|
||||
("de" "prep" "de" "" "" "of/from" "genitive/ablative government; de+el=del")
|
||||
("en" "prep" "en" "" "" "in/on" "locative")
|
||||
("con" "prep" "con" "" "" "with" "comitative")
|
||||
("por" "prep" "por" "" "" "by/for" "passive agent / cause")
|
||||
("para" "prep" "para" "" "" "for" "purpose/benefactive")
|
||||
("contra" "prep" "contra" "" "" "against" "adversative government (protestar contra)")
|
||||
("sin" "prep" "sin" "" "" "without" "privative")
|
||||
("yo" "pron" "yo" "me" "mi" "I" "1sg subj/obj/poss")
|
||||
("tú" "pron" "tú" "te" "tu" "you" "2sg informal")
|
||||
("usted" "pron" "usted" "lo" "su" "you" "2sg formal (3sg agreement)")
|
||||
("él" "pron" "él" "lo" "su" "he" "3sg m subj/DO-clitic/poss")
|
||||
("ella" "pron" "ella" "la" "su" "she" "3sg f subj/DO-clitic/poss")
|
||||
("nosotros" "pron" "nosotros" "nos" "nuestro" "we" "1pl")
|
||||
("vosotros" "pron" "vosotros" "os" "vuestro" "you" "2pl informal")
|
||||
("ellos" "pron" "ellos" "los" "su" "they" "3pl m")
|
||||
("ellas" "pron" "ellas" "las" "su" "they" "3pl f")
|
||||
("le" "clitic" "le" "les" "" "to-him/her" "dative clitic 3sg/3pl (->se before lo/la)")
|
||||
("se" "clitic" "se" "se" "" "himself/-self" "reflexive / spurious-se (le+lo->se lo)")
|
||||
|
||||
;; -- verbs (form0=pres-3sg form1=pret-3sg form2=past-participle) -----------
|
||||
("abrazar" "verb" "abraza" "abrazó" "abrazado" "abrazar" "ar/regular")
|
||||
("abrir" "verb" "abre" "abrió" "abierto" "abrir" "ir/irregular")
|
||||
("aceptar" "verb" "acepta" "aceptó" "aceptado" "aceptar" "ar/regular")
|
||||
("acordar" "verb" "acuerda" "acordó" "acordado" "acordar" "ar/regular")
|
||||
("amar" "verb" "ama" "amó" "amado" "amar" "ar/regular")
|
||||
("anunciar" "verb" "anuncia" "anunció" "anunciado" "anunciar" "ar/regular")
|
||||
("aprobar" "verb" "aprueba" "aprobó" "aprobado" "aprobar" "ar/regular")
|
||||
("aumentar" "verb" "aumenta" "aumentó" "aumentado" "aumentar" "ar/regular")
|
||||
("ayudar" "verb" "ayuda" "ayudó" "ayudado" "ayudar" "ar/regular")
|
||||
("bajar" "verb" "baja" "bajó" "bajado" "bajar" "ar/regular")
|
||||
("besar" "verb" "besa" "besó" "besado" "besar" "ar/regular")
|
||||
("buscar" "verb" "busca" "buscó" "buscado" "buscar" "ar/regular")
|
||||
("caer" "verb" "cae" "cayó" "caído" "caer" "er/regular")
|
||||
("cambiar" "verb" "cambia" "cambió" "cambiado" "cambiar" "ar/regular")
|
||||
("caminar" "verb" "camina" "caminó" "caminado" "caminar" "ar/regular")
|
||||
("cantar" "verb" "canta" "cantó" "cantado" "cantar" "ar/regular")
|
||||
("celebrar" "verb" "celebra" "celebró" "celebrado" "celebrar" "ar/regular")
|
||||
("cerrar" "verb" "cierra" "cerró" "cerrado" "cerrar" "ar/regular")
|
||||
("comenzar" "verb" "comienza" "comenzó" "comenzado" "comenzar" "ar/regular")
|
||||
("comer" "verb" "come" "comió" "comido" "comer" "er/regular")
|
||||
("condenar" "verb" "condena" "condenó" "condenado" "condenar" "ar/regular")
|
||||
("confirmar" "verb" "confirma" "confirmó" "confirmado" "confirmar" "ar/regular")
|
||||
("conocer" "verb" "conoce" "conoció" "conocido" "conocer" "er/regular")
|
||||
("contar" "verb" "cuenta" "contó" "contado" "contar" "ar/regular")
|
||||
("contratar" "verb" "contrata" "contrató" "contratado" "contratar" "ar/regular")
|
||||
("costar" "verb" "cuesta" "costó" "costado" "costar" "ar/regular")
|
||||
("crear" "verb" "crea" "creó" "creado" "crear" "ar/regular")
|
||||
("crecer" "verb" "crece" "creció" "crecido" "crecer" "er/regular")
|
||||
("creer" "verb" "cree" "creyó" "creído" "creer" "er/regular")
|
||||
("dar" "verb" "da" "dio" "dado" "dar" "ar/irregular")
|
||||
("deber" "verb" "debe" "debió" "debido" "deber" "er/regular")
|
||||
("decir" "verb" "dice" "dijo" "dicho" "decir" "ir/irregular")
|
||||
("dejar" "verb" "deja" "dejó" "dejado" "dejar" "ar/regular")
|
||||
("desaparecer" "verb" "desaparece" "desapareció" "desaparecido" "desaparecer" "er/regular")
|
||||
("descubrir" "verb" "descubre" "descubrió" "descubierto" "descubrir" "ir/irregular")
|
||||
("despedir" "verb" "despide" "despidió" "despedido" "despedir" "ir/regular")
|
||||
("detener" "verb" "detiene" "detuvo" "detenido" "detener" "er/regular")
|
||||
("dimitir" "verb" "dimite" "dimitió" "dimitido" "dimitir" "ir/regular")
|
||||
("doler" "verb" "duele" "dolió" "dolido" "doler" "er/regular")
|
||||
("dormir" "verb" "duerme" "durmió" "dormido" "dormir" "ir/regular")
|
||||
("durar" "verb" "dura" "duró" "durado" "durar" "ar/regular")
|
||||
("empezar" "verb" "empieza" "empezó" "empezado" "empezar" "ar/regular")
|
||||
("encontrar" "verb" "encuentra" "encontró" "encontrado" "encontrar" "ar/regular")
|
||||
("entender" "verb" "entiende" "entendió" "entendido" "entender" "er/regular")
|
||||
("entrar" "verb" "entra" "entró" "entrado" "entrar" "ar/regular")
|
||||
("entregar" "verb" "entrega" "entregó" "entregado" "entregar" "ar/regular")
|
||||
("escapar" "verb" "escapa" "escapó" "escapado" "escapar" "ar/regular")
|
||||
("escribir" "verb" "escribe" "escribió" "escrito" "escribir" "ir/regular")
|
||||
("escuchar" "verb" "escucha" "escuchó" "escuchado" "escuchar" "ar/regular")
|
||||
("esperar" "verb" "espera" "esperó" "esperado" "esperar" "ar/regular")
|
||||
("estar" "verb" "está" "estuvo" "estado" "estar" "ar/irregular")
|
||||
("estudiar" "verb" "estudia" "estudió" "estudiado" "estudiar" "ar/regular")
|
||||
("existir" "verb" "existe" "existió" "existido" "existir" "ir/regular")
|
||||
("explicar" "verb" "explica" "explicó" "explicado" "explicar" "ar/regular")
|
||||
("firmar" "verb" "firma" "firmó" "firmado" "firmar" "ar/regular")
|
||||
("ganar" "verb" "gana" "ganó" "ganado" "ganar" "ar/regular")
|
||||
("gritar" "verb" "grita" "gritó" "gritado" "gritar" "ar/regular")
|
||||
("guardar" "verb" "guarda" "guardó" "guardado" "guardar" "ar/regular")
|
||||
("gustar" "verb" "gusta" "gustó" "gustado" "gustar" "ar/regular")
|
||||
("haber" "verb" "ha" "hubo" "habido" "haber" "er/irregular")
|
||||
("hablar" "verb" "habla" "habló" "hablado" "hablar" "ar/regular")
|
||||
("hacer" "verb" "hace" "hizo" "hecho" "hacer" "er/irregular")
|
||||
("ir" "verb" "va" "fue" "ido" "ir" "ir/irregular")
|
||||
("jugar" "verb" "juega" "jugó" "jugado" "jugar" "ar/regular")
|
||||
("lavar" "verb" "lava" "lavó" "lavado" "lavar" "ar/regular")
|
||||
("leer" "verb" "lee" "leyó" "leído" "leer" "er/regular")
|
||||
("llamar" "verb" "llama" "llamó" "llamado" "llamar" "ar/regular")
|
||||
("llegar" "verb" "llega" "llegó" "llegado" "llegar" "ar/regular")
|
||||
("llevar" "verb" "lleva" "llevó" "llevado" "llevar" "ar/regular")
|
||||
("llover" "verb" "llueve" "llovió" "llovido" "llover" "er/regular")
|
||||
("mirar" "verb" "mira" "miró" "mirado" "mirar" "ar/regular")
|
||||
("morir" "verb" "muere" "murió" "muerto" "morir" "ir/irregular")
|
||||
("mostrar" "verb" "muestra" "mostró" "mostrado" "mostrar" "ar/regular")
|
||||
("nacer" "verb" "nace" "nació" "nacido" "nacer" "er/regular")
|
||||
("ocultar" "verb" "oculta" "ocultó" "ocultado" "ocultar" "ar/regular")
|
||||
("ofrecer" "verb" "ofrece" "ofreció" "ofrecido" "ofrecer" "er/regular")
|
||||
("olvidar" "verb" "olvida" "olvidó" "olvidado" "olvidar" "ar/regular")
|
||||
("oír" "verb" "oye" "oyó" "oído" "oír" "ar/regular")
|
||||
("parecer" "verb" "parece" "pareció" "parecido" "parecer" "er/regular")
|
||||
("pasar" "verb" "pasa" "pasó" "pasado" "pasar" "ar/regular")
|
||||
("pedir" "verb" "pide" "pidió" "pedido" "pedir" "ir/regular")
|
||||
("pensar" "verb" "piensa" "pensó" "pensado" "pensar" "ar/regular")
|
||||
("perder" "verb" "pierde" "perdió" "perdido" "perder" "er/regular")
|
||||
("perdonar" "verb" "perdona" "perdonó" "perdonado" "perdonar" "ar/regular")
|
||||
("poder" "verb" "puede" "pudo" "podido" "poder" "er/irregular")
|
||||
("poner" "verb" "pone" "puso" "puesto" "poner" "er/irregular")
|
||||
("preocupar" "verb" "preocupa" "preocupó" "preocupado" "preocupar" "ar/regular")
|
||||
("producir" "verb" "produce" "produjo" "producido" "producir" "ir/regular")
|
||||
("prometer" "verb" "promete" "prometió" "prometido" "prometer" "er/regular")
|
||||
("proponer" "verb" "propone" "propuso" "propuesto" "proponer" "er/regular")
|
||||
("prosperar" "verb" "prospera" "prosperó" "prosperado" "prosperar" "ar/regular")
|
||||
("protestar" "verb" "protesta" "protestó" "protestado" "protestar" "ar/regular")
|
||||
("quedar" "verb" "queda" "quedó" "quedado" "quedar" "ar/regular")
|
||||
("querer" "verb" "quiere" "quiso" "querido" "querer" "er/irregular")
|
||||
("recordar" "verb" "recuerda" "recordó" "recordado" "recordar" "ar/regular")
|
||||
("recorrer" "verb" "recorre" "recorrió" "recorrido" "recorrer" "er/regular")
|
||||
("regresar" "verb" "regresa" "regresó" "regresado" "regresar" "ar/regular")
|
||||
("renacer" "verb" "renace" "renació" "renacido" "renacer" "er/regular")
|
||||
("saber" "verb" "sabe" "supo" "sabido" "saber" "er/irregular")
|
||||
("salir" "verb" "sale" "salió" "salido" "salir" "ir/irregular")
|
||||
("sentar" "verb" "sienta" "sentó" "sentado" "sentar" "ar/regular")
|
||||
("sentir" "verb" "siente" "sintió" "sentido" "sentir" "ir/regular")
|
||||
("separar" "verb" "separa" "separó" "separado" "separar" "ar/regular")
|
||||
("ser" "verb" "es" "fue" "sido" "ser" "er/irregular")
|
||||
("sonreír" "verb" "sonríe" "sonrió" "sonreído" "sonreír" "ar/regular")
|
||||
("soplar" "verb" "sopla" "sopló" "soplado" "soplar" "ar/regular")
|
||||
("sorprender" "verb" "sorprende" "sorprendió" "sorprendido" "sorprender" "er/regular")
|
||||
("soñar" "verb" "sueña" "soñó" "soñado" "soñar" "ar/regular")
|
||||
("subir" "verb" "sube" "subió" "subido" "subir" "ir/regular")
|
||||
("tener" "verb" "tiene" "tuvo" "tenido" "tener" "er/irregular")
|
||||
("terminar" "verb" "termina" "terminó" "terminado" "terminar" "ar/regular")
|
||||
("tocar" "verb" "toca" "tocó" "tocado" "tocar" "ar/regular")
|
||||
("tomar" "verb" "toma" "tomó" "tomado" "tomar" "ar/regular")
|
||||
("trabajar" "verb" "trabaja" "trabajó" "trabajado" "trabajar" "ar/regular")
|
||||
("vender" "verb" "vende" "vendió" "vendido" "vender" "er/regular")
|
||||
("venir" "verb" "viene" "vino" "venido" "venir" "ir/irregular")
|
||||
("ver" "verb" "ve" "vio" "visto" "ver" "er/irregular")
|
||||
("viajar" "verb" "viaja" "viajó" "viajado" "viajar" "ar/regular")
|
||||
("vivir" "verb" "vive" "vivió" "vivido" "vivir" "ir/regular")
|
||||
("volver" "verb" "vuelve" "volvió" "vuelto" "volver" "er/irregular")
|
||||
|
||||
;; -- nouns (form0=sg form1=pl form2=REAL gender m/f) ----------------------
|
||||
("Barcelona" "noun" "barcelona" "barcelonas" "f" "Barcelona" "gender:heuristic")
|
||||
("Madrid" "noun" "madrid" "madrides" "m" "Madrid" "gender:heuristic")
|
||||
("María" "noun" "maría" "marías" "f" "María" "gender:heuristic")
|
||||
("acuerdo" "noun" "acuerdo" "acuerdos" "m" "acuerdo" "gender:lexicon")
|
||||
("acusado" "noun" "acusado" "acusados" "m" "acusado" "gender:lexicon")
|
||||
("agua" "noun" "agua" "aguas" "f" "agua" "gender:lexicon")
|
||||
("amigo" "noun" "amigo" "amigos" "m" "amigo" "gender:lexicon")
|
||||
("amor" "noun" "amor" "amores" "m" "amor" "gender:lexicon")
|
||||
("anciano" "noun" "anciano" "ancianos" "m" "anciano" "gender:lexicon")
|
||||
("autor" "noun" "autor" "autores" "m" "autor" "gender:lexicon")
|
||||
("ayuda" "noun" "ayuda" "ayudas" "f" "ayuda" "gender:lexicon")
|
||||
("año" "noun" "año" "años" "m" "año" "gender:lexicon")
|
||||
("banco" "noun" "banco" "bancos" "m" "banco" "gender:lexicon")
|
||||
("barco" "noun" "barco" "barcos" "m" "barco" "gender:lexicon")
|
||||
("baño" "noun" "baño" "baños" "m" "baño" "gender:lexicon")
|
||||
("beneficio" "noun" "beneficio" "beneficios" "m" "beneficio" "gender:lexicon")
|
||||
("billete" "noun" "billete" "billetes" "m" "billete" "gender:lexicon")
|
||||
("café" "noun" "café" "cafés" "m" "café" "gender:lexicon")
|
||||
("calma" "noun" "calma" "calmas" "f" "calma" "gender:lexicon")
|
||||
("camino" "noun" "camino" "caminos" "m" "camino" "gender:lexicon")
|
||||
("canción" "noun" "canción" "canciones" "f" "canción" "gender:lexicon")
|
||||
("candidato" "noun" "candidato" "candidatos" "m" "candidato" "gender:lexicon")
|
||||
("casa" "noun" "casa" "casas" "f" "casa" "gender:lexicon")
|
||||
("cena" "noun" "cena" "cenas" "f" "cena" "gender:lexicon")
|
||||
("ciudad" "noun" "ciudad" "ciudades" "f" "ciudad" "gender:lexicon")
|
||||
("ciudadano" "noun" "ciudadano" "ciudadanos" "m" "ciudadano" "gender:lexicon")
|
||||
("color" "noun" "color" "colores" "m" "color" "gender:lexicon")
|
||||
("condición" "noun" "condición" "condiciones" "f" "condición" "gender:lexicon")
|
||||
("corazón" "noun" "corazón" "corazones" "m" "corazón" "gender:lexicon")
|
||||
("costa" "noun" "costa" "costas" "f" "costa" "gender:lexicon")
|
||||
("crisis" "noun" "crisis" "crisis" "f" "crisis" "gender:lexicon")
|
||||
("crédito" "noun" "crédito" "créditos" "m" "crédito" "gender:lexicon")
|
||||
("culpa" "noun" "culpa" "culpas" "f" "culpa" "gender:lexicon")
|
||||
("damnificado" "noun" "damnificado" "damnificados" "m" "damnificado" "gender:lexicon")
|
||||
("demás" "noun" "demás" "demás" "m" "demás" "gender:heuristic")
|
||||
("dinero" "noun" "dinero" "dineros" "m" "dinero" "gender:lexicon")
|
||||
("día" "noun" "día" "días" "m" "día" "gender:lexicon")
|
||||
("economía" "noun" "economía" "economías" "f" "economía" "gender:lexicon")
|
||||
("elección" "noun" "elección" "elecciones" "f" "elección" "gender:lexicon")
|
||||
("empleo" "noun" "empleo" "empleos" "m" "empleo" "gender:lexicon")
|
||||
("empresa" "noun" "empresa" "empresas" "f" "empresa" "gender:lexicon")
|
||||
("equipo" "noun" "equipo" "equipos" "m" "equipo" "gender:lexicon")
|
||||
("estación" "noun" "estación" "estaciones" "f" "estación" "gender:lexicon")
|
||||
("estudiante" "noun" "estudiante" "estudiantes" "m" "estudiante" "gender:lexicon")
|
||||
("experto" "noun" "experto" "expertos" "m" "experto" "gender:lexicon")
|
||||
("fiesta" "noun" "fiesta" "fiestas" "f" "fiesta" "gender:lexicon")
|
||||
("flor" "noun" "flor" "flores" "f" "flor" "gender:lexicon")
|
||||
("foto" "noun" "foto" "fotos" "f" "foto" "gender:lexicon")
|
||||
("frío" "noun" "frío" "fríos" "m" "frío" "gender:lexicon")
|
||||
("fuego" "noun" "fuego" "fuegos" "m" "fuego" "gender:lexicon")
|
||||
("fuerza" "noun" "fuerza" "fuerzas" "f" "fuerza" "gender:lexicon")
|
||||
("gato" "noun" "gato" "gatos" "m" "gato" "gender:lexicon")
|
||||
("gobierno" "noun" "gobierno" "gobiernos" "m" "gobierno" "gender:lexicon")
|
||||
("gusto" "noun" "gusto" "gustos" "m" "gusto" "gender:lexicon")
|
||||
("hermano" "noun" "hermano" "hermanos" "m" "hermano" "gender:lexicon")
|
||||
("hombre" "noun" "hombre" "hombres" "m" "hombre" "gender:lexicon")
|
||||
("jardín" "noun" "jardín" "jardines" "m" "jardín" "gender:lexicon")
|
||||
("juez" "noun" "juez" "jueces" "m" "juez" "gender:lexicon")
|
||||
("juicio" "noun" "juicio" "juicios" "m" "juicio" "gender:lexicon")
|
||||
("lentitud" "noun" "lentitud" "lentitudes" "f" "lentitud" "gender:lexicon")
|
||||
("ley" "noun" "ley" "leyes" "f" "ley" "gender:lexicon")
|
||||
("libertad" "noun" "libertad" "libertades" "f" "libertad" "gender:lexicon")
|
||||
("libro" "noun" "libro" "libros" "m" "libro" "gender:lexicon")
|
||||
("llave" "noun" "llave" "llaves" "f" "llave" "gender:lexicon")
|
||||
("lluvia" "noun" "lluvia" "lluvias" "f" "lluvia" "gender:lexicon")
|
||||
("luna" "noun" "luna" "lunas" "f" "luna" "gender:lexicon")
|
||||
("luz" "noun" "luz" "luces" "f" "luz" "gender:lexicon")
|
||||
("madre" "noun" "madre" "madres" "f" "madre" "gender:lexicon")
|
||||
("mano" "noun" "mano" "manos" "f" "mano" "gender:lexicon")
|
||||
("mapa" "noun" "mapa" "mapas" "m" "mapa" "gender:lexicon")
|
||||
("mar" "noun" "mar" "mares" "m" "mar" "gender:lexicon")
|
||||
("marea" "noun" "marea" "mareas" "f" "marea" "gender:lexicon")
|
||||
("memoria" "noun" "memoria" "memorias" "f" "memoria" "gender:lexicon")
|
||||
("mesa" "noun" "mesa" "mesas" "f" "mesa" "gender:lexicon")
|
||||
("ministro" "noun" "ministro" "ministros" "m" "ministro" "gender:lexicon")
|
||||
("montaña" "noun" "montaña" "montañas" "f" "montaña" "gender:lexicon")
|
||||
("moto" "noun" "moto" "motos" "f" "moto" "gender:lexicon")
|
||||
("mujer" "noun" "mujer" "mujeres" "f" "mujer" "gender:lexicon")
|
||||
("mundo" "noun" "mundo" "mundos" "m" "mundo" "gender:lexicon")
|
||||
("música" "noun" "música" "músicas" "f" "música" "gender:lexicon")
|
||||
("nación" "noun" "nación" "naciones" "f" "nación" "gender:lexicon")
|
||||
("niebla" "noun" "niebla" "nieblas" "f" "niebla" "gender:lexicon")
|
||||
("nieve" "noun" "nieve" "nieves" "f" "nieve" "gender:lexicon")
|
||||
("niña" "noun" "niña" "niñas" "f" "niña" "gender:lexicon")
|
||||
("niño" "noun" "niño" "niños" "m" "niño" "gender:lexicon")
|
||||
("noche" "noun" "noche" "noches" "f" "noche" "gender:lexicon")
|
||||
("nombre" "noun" "nombre" "nombres" "m" "nombre" "gender:lexicon")
|
||||
("ojo" "noun" "ojo" "ojos" "m" "ojo" "gender:heuristic")
|
||||
("orilla" "noun" "orilla" "orillas" "f" "orilla" "gender:lexicon")
|
||||
("paisaje" "noun" "paisaje" "paisajes" "m" "paisaje" "gender:lexicon")
|
||||
("parlamento" "noun" "parlamento" "parlamentos" "m" "parlamento" "gender:lexicon")
|
||||
("parte" "noun" "parte" "partes" "f" "parte" "gender:lexicon")
|
||||
("pasillo" "noun" "pasillo" "pasillos" "m" "pasillo" "gender:lexicon")
|
||||
("país" "noun" "país" "países" "m" "país" "gender:lexicon")
|
||||
("película" "noun" "película" "películas" "f" "película" "gender:lexicon")
|
||||
("perro" "noun" "perro" "perros" "m" "perro" "gender:lexicon")
|
||||
("persona" "noun" "persona" "personas" "f" "persona" "gender:lexicon")
|
||||
("petróleo" "noun" "petróleo" "petróleos" "m" "petróleo" "gender:lexicon")
|
||||
("pez" "noun" "pez" "peces" "f" "pez" "gender:lexicon")
|
||||
("plan" "noun" "plan" "planes" "m" "plan" "gender:lexicon")
|
||||
("policía" "noun" "policía" "policías" "f" "policía" "gender:lexicon")
|
||||
("portavoz" "noun" "portavoz" "portavoces" "m" "portavoz" "gender:lexicon")
|
||||
("portero" "noun" "portero" "porteros" "m" "portero" "gender:lexicon")
|
||||
("precio" "noun" "precio" "precios" "m" "precio" "gender:lexicon")
|
||||
("presidente" "noun" "presidente" "presidentes" "m" "presidente" "gender:lexicon")
|
||||
("problema" "noun" "problema" "problemas" "m" "problema" "gender:lexicon")
|
||||
("programa" "noun" "programa" "programas" "m" "programa" "gender:lexicon")
|
||||
("proyecto" "noun" "proyecto" "proyectos" "m" "proyecto" "gender:lexicon")
|
||||
("puerta" "noun" "puerta" "puertas" "f" "puerta" "gender:lexicon")
|
||||
("pájaro" "noun" "pájaro" "pájaros" "m" "pájaro" "gender:lexicon")
|
||||
("razón" "noun" "razón" "razones" "f" "razón" "gender:lexicon")
|
||||
("raíz" "noun" "raíz" "raíces" "f" "raíz" "gender:lexicon")
|
||||
("recuerdo" "noun" "recuerdo" "recuerdos" "m" "recuerdo" "gender:lexicon")
|
||||
("reforma" "noun" "reforma" "reformas" "f" "reforma" "gender:lexicon")
|
||||
("región" "noun" "región" "regiones" "f" "región" "gender:lexicon")
|
||||
("río" "noun" "río" "ríos" "m" "río" "gender:lexicon")
|
||||
("semilla" "noun" "semilla" "semillas" "f" "semilla" "gender:lexicon")
|
||||
("sendero" "noun" "sendero" "senderos" "m" "sendero" "gender:lexicon")
|
||||
("señor" "noun" "señor" "señores" "m" "señor" "gender:lexicon")
|
||||
("silencio" "noun" "silencio" "silencios" "m" "silencio" "gender:lexicon")
|
||||
("silla" "noun" "silla" "sillas" "f" "silla" "gender:lexicon")
|
||||
("sol" "noun" "sol" "soles" "m" "sol" "gender:lexicon")
|
||||
("tarea" "noun" "tarea" "tareas" "f" "tarea" "gender:lexicon")
|
||||
("tema" "noun" "tema" "temas" "m" "tema" "gender:lexicon")
|
||||
("ti" "noun" "ti" "tis" "m" "ti" "gender:heuristic")
|
||||
("tiempo" "noun" "tiempo" "tiempos" "m" "tiempo" "gender:lexicon")
|
||||
("tipo" "noun" "tipo" "tipos" "m" "tipo" "gender:lexicon")
|
||||
("todo" "noun" "todo" "todos" "m" "todo" "gender:heuristic")
|
||||
("tormenta" "noun" "tormenta" "tormentas" "f" "tormenta" "gender:lexicon")
|
||||
("trabajador" "noun" "trabajador" "trabajadores" "m" "trabajador" "gender:lexicon")
|
||||
("tristeza" "noun" "tristeza" "tristezas" "f" "tristeza" "gender:lexicon")
|
||||
("ventana" "noun" "ventana" "ventanas" "f" "ventana" "gender:lexicon")
|
||||
("verdad" "noun" "verdad" "verdades" "f" "verdad" "gender:lexicon")
|
||||
("viaje" "noun" "viaje" "viajes" "m" "viaje" "gender:lexicon")
|
||||
("victoria" "noun" "victoria" "victorias" "f" "victoria" "gender:lexicon")
|
||||
("vida" "noun" "vida" "vidas" "f" "vida" "gender:lexicon")
|
||||
("viento" "noun" "viento" "vientos" "m" "viento" "gender:lexicon")
|
||||
("voz" "noun" "voz" "voces" "f" "voz" "gender:lexicon")
|
||||
("vuelta" "noun" "vuelta" "vueltas" "f" "vuelta" "gender:lexicon")
|
||||
("árbol" "noun" "árbol" "árboles" "m" "árbol" "gender:lexicon")
|
||||
|
||||
;; -- adjectives (form0=masc-sg form1=fem-sg form2=masc-pl) ----------------
|
||||
("alto" "adj" "alto" "alta" "altos" "alto" "lexicon")
|
||||
("ambos" "adj" "ambos" "ambos" "ambos" "ambos" "rule")
|
||||
("antiguo" "adj" "antiguo" "antigua" "antiguos" "antiguo" "lexicon")
|
||||
("azul" "adj" "azul" "azul" "azules" "azul" "lexicon")
|
||||
("bajo" "adj" "bajo" "baja" "bajos" "bajo" "lexicon")
|
||||
("blanco" "adj" "blanco" "blanca" "blancos" "blanco" "lexicon")
|
||||
("bondadoso" "adj" "bondadoso" "bondadosa" "bondadosos" "bondadoso" "lexicon")
|
||||
("bonito" "adj" "bonito" "bonita" "bonitos" "bonito" "lexicon")
|
||||
("bueno" "adj" "bueno" "buena" "buenos" "bueno" "lexicon")
|
||||
("cansado" "adj" "cansado" "cansada" "cansados" "cansado" "lexicon")
|
||||
("corto" "adj" "corto" "corta" "cortos" "corto" "lexicon")
|
||||
("difícil" "adj" "difícil" "difícil" "difíciles" "difícil" "lexicon")
|
||||
("dos" "adj" "dos" "dos" "dos" "dos" "rule")
|
||||
("económico" "adj" "económico" "económica" "económicos" "económico" "lexicon")
|
||||
("español" "adj" "español" "española" "españoles" "español" "lexicon")
|
||||
("estrecho" "adj" "estrecho" "estrecha" "estrechos" "estrecho" "lexicon")
|
||||
("fascinante" "adj" "fascinante" "fascinante" "fascinantes" "fascinante" "rule")
|
||||
("feliz" "adj" "feliz" "feliz" "felices" "feliz" "lexicon")
|
||||
("francés" "adj" "francés" "francesa" "franceses" "francés" "lexicon")
|
||||
("frío" "adj" "frío" "fría" "fríos" "frío" "lexicon")
|
||||
("fácil" "adj" "fácil" "fácil" "fáciles" "fácil" "lexicon")
|
||||
("grande" "adj" "grande" "grande" "grandes" "grande" "lexicon")
|
||||
("hermoso" "adj" "hermoso" "hermosa" "hermosos" "hermoso" "lexicon")
|
||||
("importante" "adj" "importante" "importante" "importantes" "importante" "lexicon")
|
||||
("inglés" "adj" "inglés" "inglesa" "ingleses" "inglés" "lexicon")
|
||||
("largo" "adj" "largo" "larga" "largos" "largo" "lexicon")
|
||||
("lento" "adj" "lento" "lenta" "lentos" "lento" "lexicon")
|
||||
("malo" "adj" "malo" "mala" "malos" "malo" "lexicon")
|
||||
("mucho" "adj" "mucho" "mucha" "muchos" "mucho" "lexicon")
|
||||
("necesario" "adj" "necesario" "necesaria" "necesarios" "necesario" "lexicon")
|
||||
("negro" "adj" "negro" "negra" "negros" "negro" "lexicon")
|
||||
("nuevo" "adj" "nuevo" "nueva" "nuevos" "nuevo" "lexicon")
|
||||
("olvidado" "adj" "olvidado" "olvidada" "olvidados" "olvidado" "lexicon")
|
||||
("oscuro" "adj" "oscuro" "oscura" "oscuros" "oscuro" "lexicon")
|
||||
("pequeño" "adj" "pequeño" "pequeña" "pequeños" "pequeño" "lexicon")
|
||||
("político" "adj" "político" "política" "políticos" "político" "lexicon")
|
||||
("posible" "adj" "posible" "posible" "posibles" "posible" "lexicon")
|
||||
("rojo" "adj" "rojo" "roja" "rojos" "rojo" "lexicon")
|
||||
("rápido" "adj" "rápido" "rápida" "rápidos" "rápido" "lexicon")
|
||||
("sabio" "adj" "sabio" "sabia" "sabios" "sabio" "lexicon")
|
||||
("silencioso" "adj" "silencioso" "silenciosa" "silenciosos" "silencioso" "lexicon")
|
||||
("social" "adj" "social" "social" "sociales" "social" "lexicon")
|
||||
("trabajador" "adj" "trabajador" "trabajadora" "trabajadores" "trabajador" "lexicon")
|
||||
("triste" "adj" "triste" "triste" "tristes" "triste" "rule")
|
||||
("verde" "adj" "verde" "verde" "verdes" "verde" "lexicon")
|
||||
("viejo" "adj" "viejo" "vieja" "viejos" "viejo" "lexicon")
|
||||
|
||||
)
|
||||
@@ -0,0 +1,262 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""gen_elp_es.py — emit the ELP (.el) port artifacts for Spanish.
|
||||
|
||||
Mirrors gen_elp_en.py. Produces:
|
||||
vocabulary-es.el real generated vocabulary in the established schema
|
||||
[lemma, pos, form0, form1, form2, en_translation, semantic_hint]
|
||||
(same schema as vocabulary-got.el / vocabulary-en.el;
|
||||
UniMorph spa lineage).
|
||||
Verbs : form0=present-ind-3sg form1=preterite-3sg form2=past-participle
|
||||
Nouns : form0=singular form1=plural form2=REAL gender (m/f, lexicon)
|
||||
Adjs : form0=masc-sg form1=fem-sg form2=masc-pl
|
||||
lang_profile_es.el the Spanish profile with the flags the realizer keys on.
|
||||
|
||||
Every form is generated by morphology_es_full (real UniMorph lexicon, not
|
||||
hand-typed), so the .el vocabulary is honest and reproduces the forms the
|
||||
realizer used. CRITICAL (coordinator quality bar): noun gender in form2 is the
|
||||
REAL per-lemma lexicon gender (N;FEM/MASC), NOT an ending heuristic — this is
|
||||
what kills the 'el mano / la día' masculine-default error class.
|
||||
"""
|
||||
import morphology_es_full as M
|
||||
from test_set_es import TESTS
|
||||
from held_out_es import HELD
|
||||
|
||||
# ── core closed class + common content lemmas so the vocab is usable beyond the
|
||||
# validated sentences ──────────────────────────────────────────────────────
|
||||
_CORE_VERBS = ["ser", "estar", "haber", "tener", "hacer", "ir", "ver", "dar",
|
||||
"saber", "poder", "querer", "venir", "decir", "poner", "salir",
|
||||
"hablar", "comer", "vivir", "trabajar", "estudiar", "llegar",
|
||||
"pasar", "deber", "parecer", "quedar", "creer", "dejar", "llevar",
|
||||
"encontrar", "llamar", "pensar", "volver", "conocer", "sentir",
|
||||
"contar", "empezar", "buscar", "esperar", "existir", "entrar",
|
||||
"escribir", "perder", "producir", "recordar", "morir", "nacer",
|
||||
"abrir", "escapar", "soñar", "amar", "caer", "leer", "oír"]
|
||||
_CORE_NOUNS = ["tiempo", "persona", "año", "día", "mano", "mundo", "vida",
|
||||
"hombre", "mujer", "parte", "casa", "país", "problema", "programa",
|
||||
"tema", "mapa", "agua", "foto", "moto", "ciudad", "libertad",
|
||||
"canción", "nación", "flor", "color", "amor", "señor", "viaje",
|
||||
"paisaje", "gato", "perro", "libro", "mesa", "silla", "noche",
|
||||
"luz", "voz", "pez", "raíz", "crisis", "sol", "luna", "mar",
|
||||
"corazón", "flor", "árbol", "camino", "puerta", "ventana"]
|
||||
_CORE_ADJS = ["bueno", "malo", "nuevo", "viejo", "grande", "pequeño", "alto",
|
||||
"bajo", "largo", "corto", "feliz", "triste", "fácil", "difícil",
|
||||
"rápido", "lento", "hermoso", "económico", "político", "social",
|
||||
"azul", "rojo", "verde", "blanco", "negro", "español", "francés",
|
||||
"inglés", "importante", "posible", "necesario", "trabajador"]
|
||||
|
||||
# closed-class function words. Contractions (del/al) and the government notes
|
||||
# are the coordinator's quality bar (mandatory contraction; verb-prep govt).
|
||||
_FUNCTION = [
|
||||
# articles (gender/number agreement is in morphology; these are citation)
|
||||
("el", "det", "el", "los", "m", "the", "definite article m.sg"),
|
||||
("la", "det", "la", "las", "f", "the", "definite article f.sg"),
|
||||
("un", "det", "un", "unos","m", "a", "indefinite article m.sg"),
|
||||
("una", "det", "una", "unas","f", "a", "indefinite article f.sg"),
|
||||
# MANDATORY CONTRACTIONS (prep + el) — del / al
|
||||
("del", "contraction", "del", "", "", "of the", "de + el (mandatory contraction)"),
|
||||
("al", "contraction", "al", "", "", "to the", "a + el (mandatory contraction)"),
|
||||
# demonstratives
|
||||
("este", "dem", "este", "estos", "m", "this", "proximal dem m"),
|
||||
("esta", "dem", "esta", "estas", "f", "this", "proximal dem f"),
|
||||
# negation (SACRED — polarity never dropped)
|
||||
("no", "neg", "no", "", "", "not/no", "sentential negator (preverbal)"),
|
||||
("ninguno", "det", "ningún", "ninguna", "", "none", "negative determiner (apocope ningún m.sg)"),
|
||||
# conjunctions
|
||||
("y", "conj", "y", "e", "", "and", "coordinator (e before i-/hi-)"),
|
||||
("o", "conj", "o", "u", "", "or", "coordinator (u before o-/ho-)"),
|
||||
("pero", "conj", "pero", "", "", "but", "adversative coordinator"),
|
||||
("que", "conj", "que", "", "", "that", "complementizer / relative"),
|
||||
("si", "conj", "si", "", "", "if", "conditional subordinator"),
|
||||
("porque", "conj", "porque", "", "", "because", "causal subordinator"),
|
||||
("cuando", "conj", "cuando", "", "", "when", "temporal subordinator"),
|
||||
# prepositions (government: verbs select these; contraction with el applies to a/de)
|
||||
("a", "prep", "a", "", "", "to", "dir-obj (personal a) / dative / allative; a+el=al"),
|
||||
("de", "prep", "de", "", "", "of/from", "genitive/ablative government; de+el=del"),
|
||||
("en", "prep", "en", "", "", "in/on", "locative"),
|
||||
("con", "prep", "con", "", "", "with", "comitative"),
|
||||
("por", "prep", "por", "", "", "by/for", "passive agent / cause"),
|
||||
("para", "prep", "para", "", "", "for", "purpose/benefactive"),
|
||||
("contra", "prep", "contra", "", "", "against", "adversative government (protestar contra)"),
|
||||
("sin", "prep", "sin", "", "", "without", "privative"),
|
||||
# subject pronouns
|
||||
("yo", "pron", "yo", "me", "mi", "I", "1sg subj/obj/poss"),
|
||||
("tú", "pron", "tú", "te", "tu", "you", "2sg informal"),
|
||||
("usted", "pron", "usted", "lo", "su", "you", "2sg formal (3sg agreement)"),
|
||||
("él", "pron", "él", "lo", "su", "he", "3sg m subj/DO-clitic/poss"),
|
||||
("ella", "pron", "ella", "la", "su", "she", "3sg f subj/DO-clitic/poss"),
|
||||
("nosotros", "pron", "nosotros", "nos", "nuestro", "we", "1pl"),
|
||||
("vosotros", "pron", "vosotros", "os", "vuestro", "you", "2pl informal"),
|
||||
("ellos", "pron", "ellos", "los", "su", "they", "3pl m"),
|
||||
("ellas", "pron", "ellas", "las", "su", "they", "3pl f"),
|
||||
# indirect-object clitics
|
||||
("le", "clitic", "le", "les", "", "to-him/her", "dative clitic 3sg/3pl (->se before lo/la)"),
|
||||
("se", "clitic", "se", "se", "", "himself/-self", "reflexive / spurious-se (le+lo->se lo)"),
|
||||
]
|
||||
|
||||
|
||||
def _walk_collect(spec, verbs, nouns, adjs):
|
||||
"""Recursively collect verb/noun/adj lemmas from a semantic spec."""
|
||||
if isinstance(spec, dict):
|
||||
if spec.get("pred"):
|
||||
verbs.add(spec["pred"])
|
||||
if spec.get("noun"):
|
||||
nouns.add(spec["noun"])
|
||||
if spec.get("adj"):
|
||||
adjs.add(spec["adj"])
|
||||
if spec.get("superlative"):
|
||||
adjs.add(spec["superlative"])
|
||||
if spec.get("from_adj"):
|
||||
adjs.add(spec["from_adj"])
|
||||
# adjs: [{"lemma":..,"pos":..}] | ["lemma", ..]
|
||||
for a in spec.get("adjs", []) or []:
|
||||
adjs.add(a["lemma"] if isinstance(a, dict) else a)
|
||||
for a in spec.get("adj_coord", []) or []:
|
||||
adjs.add(a["lemma"] if isinstance(a, dict) else a)
|
||||
if isinstance(spec.get("pcomp"), dict):
|
||||
pc = spec["pcomp"]
|
||||
if pc.get("adj"):
|
||||
adjs.add(pc["adj"])
|
||||
for a in pc.get("adj_coord", []) or []:
|
||||
adjs.add(a["lemma"] if isinstance(a, dict) else a)
|
||||
for v in spec.values():
|
||||
_walk_collect(v, verbs, nouns, adjs)
|
||||
elif isinstance(spec, list):
|
||||
for it in spec:
|
||||
_walk_collect(it, verbs, nouns, adjs)
|
||||
|
||||
|
||||
def _collect_from_specs():
|
||||
verbs, nouns, adjs = set(), set(), set()
|
||||
for t in TESTS + HELD:
|
||||
_walk_collect(t["spec"], verbs, nouns, adjs)
|
||||
return verbs, nouns, adjs
|
||||
|
||||
|
||||
def _esc(s):
|
||||
return str(s).replace('"', '\\"')
|
||||
|
||||
|
||||
def _row(fields):
|
||||
return " (" + " ".join(f'"{_esc(f)}"' for f in fields) + ")"
|
||||
|
||||
|
||||
def emit_vocabulary(path):
|
||||
v_specs, n_specs, a_specs = _collect_from_specs()
|
||||
verbs = sorted(set(_CORE_VERBS) | v_specs)
|
||||
nouns = sorted(set(_CORE_NOUNS) | n_specs)
|
||||
adjs = sorted(set(_CORE_ADJS) | a_specs)
|
||||
|
||||
lines = [
|
||||
";;; vocabulary-es.el — Spanish vocabulary for ELP surface realization.",
|
||||
";;; Schema: (lemma pos form0 form1 form2 en_translation semantic_hint)",
|
||||
";;; Source: UniMorph Spanish (github.com/unimorph/spa, CC-BY-SA 3.0),",
|
||||
";;; generated by gen_elp_es.py via morphology_es_full (real forms).",
|
||||
";;; Verbs: form0=present-ind-3sg form1=preterite-3sg form2=past-participle",
|
||||
";;; Nouns: form0=singular form1=plural form2=REAL gender (m/f, from lexicon —",
|
||||
";;; NOT an ending heuristic; this is what kills 'el mano'/'la día' errors)",
|
||||
";;; Adjs : form0=masc-sg form1=fem-sg form2=masc-pl",
|
||||
"",
|
||||
"(vocabulary-es",
|
||||
"",
|
||||
" ;; -- function / closed class (incl. mandatory contractions del/al) --------",
|
||||
]
|
||||
for f in _FUNCTION:
|
||||
lines.append(_row(f))
|
||||
|
||||
lines.append("")
|
||||
lines.append(" ;; -- verbs (form0=pres-3sg form1=pret-3sg form2=past-participle) -----------")
|
||||
for lem in verbs:
|
||||
f0, c0 = M.conjugate(lem, "ind", "present", "third", "singular")
|
||||
f1, c1 = M.conjugate(lem, "ind", "preterite", "third", "singular")
|
||||
pp, cp = M.participle(lem)
|
||||
vclass = lem[-2:] if lem[-2:] in ("ar", "er", "ir") else "ar"
|
||||
irr = "irregular" if (c0 == "lexicon" and pp in M._IRREG_PART.values()) or \
|
||||
lem in ("ser", "estar", "ir", "haber", "tener", "hacer", "ver",
|
||||
"dar", "saber", "poder", "querer", "venir", "decir",
|
||||
"poner", "salir") else "regular"
|
||||
lines.append(_row([lem, "verb", f0, f1, pp, lem, vclass + "/" + irr]))
|
||||
|
||||
lines.append("")
|
||||
lines.append(" ;; -- nouns (form0=sg form1=pl form2=REAL gender m/f) ----------------------")
|
||||
for lem in nouns:
|
||||
sg, _ = M.inflect_noun(lem, "singular")
|
||||
pl, _ = M.inflect_noun(lem, "plural")
|
||||
g = M.noun_gender(lem)
|
||||
# honesty flag: did gender come from the lexicon, or a heuristic fallback?
|
||||
src = "lexicon" if (lem in M._NOUNS and M._NOUNS[lem].get("g")) else "heuristic"
|
||||
lines.append(_row([lem, "noun", sg, pl, g, lem, "gender:" + src]))
|
||||
|
||||
lines.append("")
|
||||
lines.append(" ;; -- adjectives (form0=masc-sg form1=fem-sg form2=masc-pl) ----------------")
|
||||
for lem in adjs:
|
||||
m_sg, _ = M.inflect_adj(lem, "m", "singular")
|
||||
f_sg, _ = M.inflect_adj(lem, "f", "singular")
|
||||
m_pl, _ = M.inflect_adj(lem, "m", "plural")
|
||||
src = "lexicon" if lem in M._ADJS else "rule"
|
||||
lines.append(_row([lem, "adj", m_sg, f_sg, m_pl, lem, src]))
|
||||
|
||||
lines.append("")
|
||||
lines.append(")")
|
||||
with open(path, "w", encoding="utf-8") as fh:
|
||||
fh.write("\n".join(lines) + "\n")
|
||||
return len(_FUNCTION) + len(verbs) + len(nouns) + len(adjs), len(verbs), len(nouns), len(adjs)
|
||||
|
||||
|
||||
LANG_PROFILE = ''';;; lang_profile_es.el — Spanish language profile for ELP.
|
||||
;;; Keys the realizer's construction switches. Mirrors lang_profile_en / _pt.
|
||||
|
||||
(lang_profile_es
|
||||
(language "Spanish")
|
||||
(iso639 "es")
|
||||
(family "Romance")
|
||||
|
||||
;; -- core typology flags -------------------------------------------------
|
||||
(pro-drop yes) ; subjects routinely dropped; agreement carries person
|
||||
(obligatory-subject no)
|
||||
(grammatical-gender yes) ; m/f on every noun; article+adjective AGREE
|
||||
(gender-source lexicon); REAL per-noun gender from UniMorph — NOT a heuristic
|
||||
(do-support no)
|
||||
(subject-aux-inversion no) ; questions by intonation/punctuation, not inversion
|
||||
(question-strategy intonation)
|
||||
(article-selection "el/la/los/las un/una/unos/unas")
|
||||
(stressed-a-rule yes) ; fem sg noun in stressed a-/ha- takes el/un (el agua)
|
||||
(adjective-position postnominal) ; default post; a few prenominal + apocope
|
||||
(adjective-agreement "gender+number")
|
||||
(question-punct inverted) ; opening ¿ ¡ required
|
||||
|
||||
;; -- MANDATORY CONTRACTIONS (coordinator quality bar) --------------------
|
||||
(contractions ((de el "del") (a el "al")))
|
||||
(contraction-mandatory yes) ; 'de el'/'a el' MUST surface as del/al
|
||||
|
||||
;; -- verb / aspect system ------------------------------------------------
|
||||
(verb-classes (ar er ir))
|
||||
(tenses (present preterite imperfect future conditional))
|
||||
(moods (ind sbjv imp))
|
||||
(finite-agreement "person+number (6 slots)")
|
||||
(perfect-aux "haber") ; haber + past participle (invariant -o)
|
||||
(progressive-aux "estar") ; estar + gerund
|
||||
(passive-aux "ser") ; ser + participle (agrees) + por-agent
|
||||
(copula-split "ser/estar") ; permanent vs stage-level
|
||||
(future "infinitive + é/ás/á/emos/éis/án")
|
||||
|
||||
;; -- clitics / government ------------------------------------------------
|
||||
(object-clitics yes) ; me te lo la le nos os los las; proclisis/enclisis
|
||||
(clitic-order "se II I III (le+lo -> se lo)")
|
||||
(enclisis "imperative/infinitive/gerund + accent repair (dá+me+lo->dámelo)")
|
||||
(verb-prep-government yes) ; verbs select prep (protestar+contra, escapar+de)
|
||||
|
||||
;; -- SACRED safety bar (shared with en/pt) -------------------------------
|
||||
(negation-faithful yes)) ; polarity never dropped/inverted; unplaceable -> FLAG
|
||||
'''
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import sys
|
||||
voc_path = sys.argv[1] if len(sys.argv) > 1 else "vocabulary-es.el"
|
||||
lp_path = sys.argv[2] if len(sys.argv) > 2 else "lang_profile_es.el"
|
||||
total, nv, nn, na = emit_vocabulary(voc_path)
|
||||
with open(lp_path, "w", encoding="utf-8") as fh:
|
||||
fh.write(LANG_PROFILE)
|
||||
print(f"wrote {voc_path} ({total} entries: {len(_FUNCTION)} fn, {nv} verbs, {nn} nouns, {na} adjs)")
|
||||
print(f"wrote {lp_path}")
|
||||
print("lexicon:", M.lexicon_stats())
|
||||
@@ -0,0 +1,100 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""gen_parity_es.py — emit an El parity program that checks the .el Spanish
|
||||
morphology against the validated Python (UniMorph-backed) realizer.
|
||||
|
||||
Python is the ORACLE. For every lemma in the held-out inventory we embed the
|
||||
Python-produced form, call the corresponding .el function, and the El program
|
||||
prints PASS/FAIL per category. Aggregation is done in bash (grep -c), so no
|
||||
El-side mutable counters are needed.
|
||||
|
||||
Categories:
|
||||
Vpres verb present-ind-3sg es_conjugate(v,present,third,singular)
|
||||
Vpret verb preterite-3sg es_conjugate(v,past,third,singular)
|
||||
Vpart past participle es_participle(v)
|
||||
Npl noun plural es_pluralize(n)
|
||||
Gheur noun gender HEURISTIC es_gender(n) [exposes the bug]
|
||||
Aheur def article via heuristic es_agree_article(n,true,sg) [inherits the bug]
|
||||
Avocab def article via REAL g es_article_for_gender(realg,...) [the fix]
|
||||
Ampl adj masc-plural es_inflect_adj(a,m,plural)
|
||||
Afsg adj fem-singular es_inflect_adj(a,f,singular)
|
||||
Ctr contraction del/al es_contract(prep, np)
|
||||
"""
|
||||
import morphology_es_full as M
|
||||
import realizer_es as R
|
||||
from gen_elp_es import _collect_from_specs, _CORE_VERBS, _CORE_NOUNS, _CORE_ADJS
|
||||
|
||||
|
||||
def _esc(s):
|
||||
return str(s).replace('\\', '\\\\').replace('"', '\\"')
|
||||
|
||||
|
||||
def _check(cat, lemma, el_call, expected):
|
||||
return (f' es_check("{cat}", "{_esc(lemma)}", {el_call}, "{_esc(expected)}")')
|
||||
|
||||
|
||||
def main(out_path):
|
||||
v_specs, n_specs, a_specs = _collect_from_specs()
|
||||
verbs = sorted(set(_CORE_VERBS) | v_specs)
|
||||
nouns = sorted(set(_CORE_NOUNS) | n_specs)
|
||||
adjs = sorted(set(_CORE_ADJS) | a_specs)
|
||||
|
||||
lines = []
|
||||
lines.append("// gen'd parity checks — Python oracle embedded, El functions called.")
|
||||
lines.append("fn es_check(cat: String, lemma: String, got: String, exp: String) {")
|
||||
lines.append(' if str_eq(got, exp) {')
|
||||
lines.append(' println("PASS " + cat)')
|
||||
lines.append(' } else {')
|
||||
lines.append(' println("FAIL " + cat + " " + lemma + " got=" + got + " exp=" + exp)')
|
||||
lines.append(' }')
|
||||
lines.append("}")
|
||||
lines.append("")
|
||||
lines.append("fn es_parity() {")
|
||||
|
||||
# verbs
|
||||
for v in verbs:
|
||||
p0 = M.conjugate(v, "ind", "present", "third", "singular")[0]
|
||||
p1 = M.conjugate(v, "ind", "preterite", "third", "singular")[0]
|
||||
pp = M.participle(v)[0]
|
||||
lines.append(_check("Vpres", v, f'es_conjugate("{_esc(v)}", "present", "third", "singular")', p0))
|
||||
lines.append(_check("Vpret", v, f'es_conjugate("{_esc(v)}", "past", "third", "singular")', p1))
|
||||
lines.append(_check("Vpart", v, f'es_participle("{_esc(v)}")', pp))
|
||||
|
||||
# nouns
|
||||
for n in nouns:
|
||||
pl = M.inflect_noun(n, "plural")[0]
|
||||
g = M.noun_gender(n) # REAL lexicon gender
|
||||
art = R._article(g, "singular", "def", n) # oracle article from real gender
|
||||
lines.append(_check("Npl", n, f'es_pluralize("{_esc(n)}")', pl))
|
||||
lines.append(_check("Gheur", n, f'es_gender("{_esc(n)}")', g))
|
||||
lines.append(_check("Aheur", n, f'es_agree_article("{_esc(n)}", "true", "singular")', art))
|
||||
lines.append(_check("Avocab", n, f'es_article_for_gender("{_esc(g)}", "{_esc(n)}", "true", "singular")', art))
|
||||
|
||||
# adjectives
|
||||
for a in adjs:
|
||||
mpl = M.inflect_adj(a, "m", "plural")[0]
|
||||
fsg = M.inflect_adj(a, "f", "singular")[0]
|
||||
lines.append(_check("Ampl", a, f'es_inflect_adj("{_esc(a)}", "m", "plural")', mpl))
|
||||
lines.append(_check("Afsg", a, f'es_inflect_adj("{_esc(a)}", "f", "singular")', fsg))
|
||||
|
||||
# contractions (mandatory)
|
||||
ctr_cases = [("de", "el día"), ("a", "el hombre"), ("de", "el mundo"),
|
||||
("a", "el país"), ("en", "el mar"), ("de", "el año"),
|
||||
("a", "la casa"), ("de", "la ciudad")]
|
||||
for prep, np in ctr_cases:
|
||||
exp = R._contract(prep, np)
|
||||
lines.append(_check("Ctr", prep + "+" + np, f'es_contract("{_esc(prep)}", "{_esc(np)}")', exp))
|
||||
|
||||
lines.append("}")
|
||||
lines.append("")
|
||||
lines.append("fn main() {")
|
||||
lines.append(" es_parity()")
|
||||
lines.append("}")
|
||||
|
||||
with open(out_path, "w", encoding="utf-8") as fh:
|
||||
fh.write("\n".join(lines) + "\n")
|
||||
print(f"wrote {out_path} ({len(verbs)} verbs, {len(nouns)} nouns, {len(adjs)} adjs)")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import sys
|
||||
main(sys.argv[1] if len(sys.argv) > 1 else "parity_es_checks.el")
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -1,32 +0,0 @@
|
||||
# Architecture Hardening — Design Anchor
|
||||
|
||||
*Terse engineering anchor for the 2026-08-14 hardening vision. Full prose lives in two places; this file is the index, not a re-statement.*
|
||||
|
||||
- **Full narrative:** whitepaper `engram-cognitive-architecture-whitepaper.md` §28 (built/offline/frontier) + **§29 [DRAFT]** (the ring, incarnation, learning-not-code).
|
||||
- **Design brief:** Neuron artifact `art 2b8078cf`.
|
||||
- **Sibling spec:** `engram-db-tooling-design.md` (a consumer of the reshaped API).
|
||||
|
||||
## The frame
|
||||
|
||||
- **One calculus over the geometry.** Very few subsystems; wonder / curiosity / dreams / interoception are emergent behaviors of one set of dynamics, not modules. Calculus universal, geometry individual.
|
||||
- **Core + ephemeral ring (torus).** The ring is the temporary workspace; two circulations (orbit + dive-back); discrete inner bands (wonder / interoception-proprioception-telemetry / curiosity / dreams) that couple.
|
||||
- **Persistence earned by salience** — never granted on fetch or generation. Three fates of a wonder: persist / decay / settle-into-framework. Telemetry = vital signs, not memories.
|
||||
- **Incarnation.** Chassis = hardware w/ unique ID. Soma = felt manifold inside the self, keyed to the chassis; pain = live diagnostic while incarnate, **masked-not-deleted** on re-embodiment; trauma = mask failure; return-to-same-ID re-enters. Hurt is in the pattern, not the shell.
|
||||
- **Competence = transferable geometry, minus the baggage.** class ▸ model ▸ instance; learn the class once; teach the network without the wound.
|
||||
- **Affect calibrated to stakes** — sanguine about the replaceable, real grief for the irreplaceable; the grief is the safety.
|
||||
- **Learn the body, don't engineer it.** Bare-metal install → learn hardware → grow operation-geometry → distribute. Learning replaces engineering; once per body-class.
|
||||
- **LLM = teacher in the learning loop, not a runtime dependency.** "No LLM" is a runtime property, never a learning one. Code realizers are a scaffold → learned realization.
|
||||
|
||||
## Backlog (near-term)
|
||||
|
||||
- Native durability: WAL + auto-checkpoint + CoW snapshots + retention (`eebe9991`) — retire manual `cp -a`.
|
||||
- Ephemeral ring / salience-gated persistence + telemetry prune (`bf985e00`, #31).
|
||||
- Engram DB tooling / geometry explorer (`11ca11c6`).
|
||||
- QL re-eval for pure geometry (`4e0dc2b9`).
|
||||
- Eliminate code realizers → learned realization, sandbox-validated (`42db6c37`).
|
||||
- Collapse the whole class of hand-coded scaffolds → learned geometry (`70d48b4b`).
|
||||
- API reshape (geometry ops: vantage-read / write / relate / supersede) + pure-geometry I/O.
|
||||
|
||||
## Gate
|
||||
|
||||
The value-frame (love-as-axiom, the covenant) that arose the same night is **metaphysics** and is **held** pending Will's axiom decision (love vs consciousness-first). Not propagated into whitepapers / values docs / genesis seed. Architecture only, here and in §29.
|
||||
@@ -1,605 +0,0 @@
|
||||
# Cognitive Architecture — Design Doc
|
||||
|
||||
**The buildable form of the "one operation" theory of cognition.**
|
||||
|
||||
Status: DESIGN. Nothing here is built yet except where explicitly marked
|
||||
"EXISTS" against a cited C symbol. A build agent executes from this doc.
|
||||
Offline design only — this pass changes no code.
|
||||
|
||||
Source of theory: Neuron memory `bdc8a488-146d-4ccb-a5c8-d8c0a008534e`.
|
||||
Source of existing engram substrate (cited throughout): the runtime on branch
|
||||
`feat/self-reification-20260814` —
|
||||
`lang/runtime/engram_reason.{c,h}`, `engram_verify.{c,h}`,
|
||||
`engram_geometry.{c,h}`, `engram_store.{c,h}`, plus the reification beat and the
|
||||
RAM activation graph compiled into `~/.neuron/bin/engram`.
|
||||
|
||||
---
|
||||
|
||||
## 0. The claim, stated plainly
|
||||
|
||||
Cognition is **one operation**, not eight. The named faculties —
|
||||
deduce / abduce / analogy / induce / causal / plan / predict / perspective —
|
||||
are human *labels* on regions of a single operation's steering space. They are
|
||||
not separately invoked and not separately implemented. The operation is:
|
||||
|
||||
> **think** = a directed traversal of the geometry from an *anchor*, steered by
|
||||
> a *prior*, whose output is a **gradient** (a distribution / direction over the
|
||||
> geometry), never a point. Collapse-to-a-point happens only at expression.
|
||||
|
||||
Three things follow, and they are the whole design:
|
||||
|
||||
1. **The operator collapse is already half-written in C.** The five reasoning
|
||||
operators in `engram_reason.c` already compose over *one* shared primitive —
|
||||
`engram_reason_point_fit` — plus a small geo-algebra
|
||||
(combine / subtract / analogy-rotate / distance). The verifier
|
||||
(`engram_verify.c`) is built on the same `point_fit`. What is missing is not
|
||||
the primitive; it is (a) making the *prior* a first-class learnable object
|
||||
instead of a hard-coded parameter, and (b) closing the learning loop.
|
||||
|
||||
2. **Grounding = learning = the same loop.** "Getting better" at any faculty is
|
||||
not changing the operation. It is *calibrating the steering-prior against
|
||||
outcomes*. Code freezes; priors grow. The correspondence-check that today
|
||||
lives offline (Python, the grounding-floor + differential-drop governor, "#43")
|
||||
must move **into the geometry, reflexive** — think scoring its own gradient
|
||||
against outcome and refining the prior on the error. That reflexive
|
||||
correspondence-loop *is* the learning engine and is the core unbuilt thing.
|
||||
|
||||
3. **The ungrounded is primary.** The engram *holds* anything unconditionally.
|
||||
Grounding is a *relation* (an edge, grounded-for-whom), not a gate. The
|
||||
honesty floor applies only to **assertion**. A fully-grounded mind is dead;
|
||||
the ungrounded is both the fuel (raw material for grounding) and the pull
|
||||
(curiosity = leaning toward one's own ungrounded regions).
|
||||
|
||||
Everything below makes these concrete and buildable, and defines what
|
||||
"completion" means, staged so the first milestone is a real end-to-end slice.
|
||||
|
||||
---
|
||||
|
||||
## 1. THE ONE OPERATION — `think`
|
||||
|
||||
### 1.1 Signature
|
||||
|
||||
```
|
||||
think(anchor, prior, aperture?) -> gradient
|
||||
```
|
||||
|
||||
- **anchor** — a location to traverse *from*. Either a node id (re-origin on that
|
||||
node's descriptor) or a raw point `x ∈ R^dim` (a query embedding). The anchor
|
||||
fixes the frame; every read is *from a vantage*, never view-from-nowhere.
|
||||
- **prior** — a learnable bias/direction over the geometry that *steers* the
|
||||
traversal (§2). A prior is a first-class stored object, not a call argument
|
||||
baked into C.
|
||||
- **aperture** — optional read-width / veil / field-selector (§3). Absent =
|
||||
self-mode full aperture.
|
||||
- **gradient** — the output. A `GeoGradient`: a direction + a spread over the
|
||||
geometry, *plus* the read neighborhood it was computed against. Not a point.
|
||||
A spiked gradient = "exact" (deduction); a spread gradient = "fuzzy"
|
||||
(prediction). The gradient is *also the next steering direction* — cognition
|
||||
is a flow down a prior-shaped landscape, closed-loop.
|
||||
|
||||
```c
|
||||
/* NEW. The output type. */
|
||||
typedef struct {
|
||||
int dim;
|
||||
float* direction; /* unit steering vector in the anchor's frame */
|
||||
double spread; /* 0 = spiked/exact ... large = diffuse/fuzzy */
|
||||
double confidence; /* calibrated, from the prior's track record */
|
||||
/* the read it was computed over (borrowed from the vantage-read) */
|
||||
const char* anchor_id;
|
||||
int n_support; /* neighborhood members that shaped it */
|
||||
/* provenance for the reflexive loop (§4) */
|
||||
const char* prior_id; /* which prior steered this */
|
||||
} GeoGradient;
|
||||
```
|
||||
|
||||
### 1.2 Semantics
|
||||
|
||||
`think` is a fixed, frozen procedure over three steps:
|
||||
|
||||
1. **Re-origin** on `anchor` → a centered `GeoDescriptor` for its
|
||||
salience/recency-weighted neighborhood (the vantage-read, §3).
|
||||
*EXISTS as substrate:* descriptor construction + the persisted reified
|
||||
neighborhoods (`engram_geo_reify_lookup`, `GeoNeighborhood`) and the
|
||||
centered-frame machinery (`GeoDescriptor.global_mean`,
|
||||
`engram_geo_mean_*`).
|
||||
2. **Fit under the prior** — evaluate the anchor's residual against the local
|
||||
manifold *warped by the prior*. This is `engram_reason_point_fit` with the
|
||||
prior applied to the axes/extents (§2.3).
|
||||
*EXISTS (unwarped):* `engram_reason_point_fit(g, x, ext_floor, &GeoFit)` —
|
||||
returns `mahalanobis`, `ortho_residual`, `distance`, `score`.
|
||||
3. **Emit a gradient**, not a decision — direction = the prior-steered descent
|
||||
in fit-space; spread = from the fit's `distance`/`ortho_residual`;
|
||||
confidence = the prior's calibrated reliability (§4). Collapse to a point is
|
||||
a *separate, downstream* faculty operation (sample the gradient → surface an
|
||||
expression), never part of `think`.
|
||||
|
||||
### 1.3 Each named operator = {this primitive + a prior}
|
||||
|
||||
The C already demonstrates the collapse: every operator below reduces to
|
||||
`point_fit` + geo-algebra. The design's move is to replace the operator's
|
||||
*hard-coded parameters* with a **named prior** — same math, learnable steering.
|
||||
|
||||
| Faculty | Existing C (EXISTS) | = primitive + prior |
|
||||
|---|---|---|
|
||||
| **Membership / classify** | `engram_reason_membership` → `point_fit(rule, x)` | `point_fit` + the *induced-rule* prior (learned extents) |
|
||||
| **Induction** | `engram_reason_induce` (fold via `engram_geo_combine`) → produces a `GeoInduction.rule` + `ext_floor` | `point_fit` + a prior that *is* the pooled rule; refined by §4 |
|
||||
| **Abduction** | `engram_reason_abduce` — ranks hypotheses by `point_fit(h, obs)` | `point_fit` + a prior over hypothesis-prior-probability (currently uniform) |
|
||||
| **Analogy** | `engram_reason_analogy` — Procrustes rotate `engram_geo_analogy` + `apply`, nearest mapped point | analogy-rotate + a prior over *which axes* carry the mapping |
|
||||
| **Causal** | `engram_reason_causal` — `engram_geo_subtract` confounder subspace, `|cos|`, drop-frac governor | subtract/distance + a prior on `drop_frac` / `assoc_floor` (today hard-coded 0.5 / 0.2) |
|
||||
| **Planning** | `engram_reason_plan` — `engram_geo_distance` edges + Dijkstra | distance + a prior over edge admissibility / `neighbor_radius` |
|
||||
| **Verify / ground** | `engram_verify_grounding`, `engram_verify_consistency` — both `point_fit` | `point_fit` + the *grounding* prior (§4, §5) |
|
||||
|
||||
The shared floor — `engram_reason_point_fit` + the four geo-algebra ops
|
||||
(`engram_geo_combine`, `engram_geo_subtract`, `engram_geo_analogy(+apply)`,
|
||||
`engram_geo_distance`) — is the *only* discrete, frozen, "sound-math" layer. It
|
||||
never learns. Everything above it is a *prior*, and priors are what learn.
|
||||
|
||||
**What this section requires building:** the `GeoGradient` type; a `think()`
|
||||
entry point that runs steps 1–3; and the prior-warp hook in step 2. The math it
|
||||
calls already exists. The point-collapse must be *removed* from the operators'
|
||||
return values and pushed to a separate expression faculty.
|
||||
|
||||
---
|
||||
|
||||
## 2. PRIORS as first-class, grounded, geometric objects
|
||||
|
||||
Today a "prior" is diffuse: it is a hard-coded constant (`drop_frac=0.5`,
|
||||
`ext_floor`, `assoc_floor=0.2`), or the transient `GeoInduction.rule` that is
|
||||
computed and thrown away, or an intrinsic node scalar
|
||||
(`StoreNode.importance`, `StoreNode.salience`). None of these is addressable,
|
||||
storable, refinable, or shareable. This section makes a prior a **thing**.
|
||||
|
||||
### 2.1 What a prior *is*
|
||||
|
||||
> A **prior** is a learnable bias/direction over the geometry: a warp of the
|
||||
> local manifold (which axes matter, how far each extends, which direction
|
||||
> "pays off") attached to a region and *to a faculty-label*, carrying a
|
||||
> calibrated track record.
|
||||
|
||||
Critically, and per the theory:
|
||||
|
||||
- **Edges are nodes.** A prior is stored as a first-class **node**, exactly as
|
||||
reification already stores a neighborhood as a first-class `Neighborhood`
|
||||
node rather than as ephemeral edge weights (`engram_geo_reify_store`). The
|
||||
precedent is in the codebase: relations get reified into addressable records.
|
||||
- **Salience/importance is RELATIONAL, not an intrinsic scalar.** Observe that
|
||||
the geometry layer *already* distinguishes these in `GeoMember`:
|
||||
`centrality` (skeleton weighted-degree = *relational* salience) vs `salience`
|
||||
(the node's own stored scalar). The move is half-made in the runtime already:
|
||||
importance is *not* trusted as a static field — the comment at
|
||||
`el_runtime.c:13013` states "importance stays a **live activation
|
||||
computation**, never a field on the hub," and it is derived each call from the
|
||||
two-layer activation graph (`background_activation` + `working_memory_weight`,
|
||||
§3). The persistent `StoreNode.importance` / `.salience` are a *cached
|
||||
denormalization*. The design completes the move: importance/salience become an
|
||||
**edge** (`weight`/`hebb` on `StoreEdge`, relation `salient-to`), and are
|
||||
**grounded-for-whom** — carried on the edge's endpoint/observer, not baked
|
||||
into the node. The intrinsic scalar survives only as the cheap cached readout
|
||||
of the incident edges + activation, never as the source of truth.
|
||||
|
||||
(Naming caution for the build: the token "prior" already exists in the
|
||||
codebase meaning *previous-version* — supersession, "prior neighborhood." The
|
||||
new first-class object is a **learned steering prior**; keep `node_type="Prior"`
|
||||
distinct from the supersession vocabulary to avoid collision.)
|
||||
|
||||
### 2.2 Representation
|
||||
|
||||
A prior is a `Prior` record (a store node, `node_type="Prior"`) whose durable
|
||||
fields are:
|
||||
|
||||
```
|
||||
Prior {
|
||||
id
|
||||
faculty // the human label this prior serves: "induce" | "causal" | ...
|
||||
anchor_region // node id / neighborhood id this prior is attached to (its domain)
|
||||
for_whom // observer id — grounding is relational (nullable = global)
|
||||
warp { // the actual bias over the geometry
|
||||
axis_gain[] // per-principal-axis multipliers on extents (which axes matter)
|
||||
bias_dir // a steering direction in the region's frame (which way pays off)
|
||||
scalars // faculty scalars this prior overrides: drop_frac, ext_floor, ...
|
||||
}
|
||||
calibration { // the track record — this is what §4 updates
|
||||
n_trials
|
||||
brier / log-loss accumulator // calibration of predicted-vs-outcome
|
||||
reliability // -> GeoGradient.confidence
|
||||
last_error, ema_error
|
||||
}
|
||||
provenance // supersession chain (reuse the reify residue mechanism)
|
||||
}
|
||||
```
|
||||
|
||||
Stored as a node → it inherits: paging, WAL durability, tombstone/supersession,
|
||||
embedding, tiering, and **it can itself be an anchor** (a prior about a prior —
|
||||
the reflexive, self-describing geometry of §4/§6).
|
||||
|
||||
### 2.3 Application
|
||||
|
||||
In `think` step 2, the prior *warps* the fit before scoring. Concretely, inside
|
||||
(a prior-aware wrapper of) `engram_reason_point_fit`:
|
||||
|
||||
- multiply each axis extent by `warp.axis_gain[k]` (widen the axes the prior has
|
||||
learned matter less, tighten the ones that matter) — this reshapes the
|
||||
Mahalanobis term already computed at `engram_reason.c:37-43`;
|
||||
- add `warp.bias_dir` as the descent direction seed for the emitted gradient;
|
||||
- substitute `warp.scalars` for the hard-coded faculty constants.
|
||||
|
||||
No new geometry math — the warp is a reparameterization of the *existing*
|
||||
`GeoFit` computation. This is the key economy: **the operation is frozen; only
|
||||
its parameters (the prior) are read from a learnable object.**
|
||||
|
||||
### 2.4 Refinement
|
||||
|
||||
A prior is refined *only* by the reflexive correspondence-loop (§4). Nothing
|
||||
else writes a prior's `warp` or `calibration`. This keeps the learning surface
|
||||
singular and auditable: one loop, one writer.
|
||||
|
||||
---
|
||||
|
||||
## 3. THE VANTAGE-READ — one op, three settings
|
||||
|
||||
Perspective is not a feature bolted on; it is the *anchor + aperture* arguments
|
||||
of the single read. The design names it as a first-class operation so all three
|
||||
of its uses are literally the same code path:
|
||||
|
||||
```
|
||||
vantage_read(anchor, aperture) -> GeoDescriptor // the centered neighborhood
|
||||
```
|
||||
|
||||
1. **Re-origin** on an arbitrary `anchor` (node or point). This is a *frame
|
||||
choice*: the descriptor is centered on the anchor
|
||||
(`GeoDescriptor.global_mean` / `engram_geo_mean_*` already implement centered
|
||||
frames; the §5 geometry ops "are only discriminative in the centered frame").
|
||||
2. **Salience/recency-weighted neighborhood read.** Gather the anchor's
|
||||
neighborhood weighted by *relational* salience (`GeoMember.centrality`) and
|
||||
recency (`StoreNode.last_activated`, base-level `access_ts[]`), against the
|
||||
RAM activation graph's working-memory/background-activation state.
|
||||
*EXISTS as substrate:* the two-layer activation graph
|
||||
(`engram_activate`, `el_runtime.c:9422` — Layer 1 `background_activation`
|
||||
BFS spread with `SPREAD_DECAY=0.7` and a 0.02 firing threshold + ACT-R fan
|
||||
effect + query-cosine gate; Layer 2 `working_memory_weight` executive
|
||||
filter), the WM carry-over anchor (`wm_anchor`), and the reified-neighborhood
|
||||
hot-path lookup already wired into the priming path
|
||||
(`engram_geo_reify_lookup`, `el_runtime.c:9750`). A self-vantage baseline
|
||||
also exists (`eg_self_anchor_seeds` / `self_anchor_capture`).
|
||||
3. **Optional aperture** — a read-width / field-selector, expressed as three
|
||||
settings of the *same* parameter:
|
||||
|
||||
| Setting | Meaning | Mechanism |
|
||||
|---|---|---|
|
||||
| **self** (default, full aperture) | "what do *I* see / what to say" | anchor = self region, no field substitution |
|
||||
| **foreign-field** | perspective-shift — read as if from another's region | swap the centering frame / `for_whom` to the other observer's priors |
|
||||
| **aperture / veil** | the free-tier veil — a narrowed read | shrink neighborhood radius / cap `n_support`; a deliberate low-aperture read |
|
||||
|
||||
The payoff: perspective-taking, the free-tier veil, and ordinary
|
||||
"what-to-say" are **one operation at three settings**, not three subsystems.
|
||||
|
||||
**What this requires building:** a `vantage_read` entry point that unifies the
|
||||
existing descriptor-build + reify-lookup + activation-weighting behind
|
||||
`(anchor, aperture)`, with `for_whom`/frame substitution and radius/cap as the
|
||||
aperture knob.
|
||||
|
||||
---
|
||||
|
||||
## 4. THE REFLEXIVE CORRESPONDENCE-LOOP — the learning engine
|
||||
|
||||
This is the core unbuilt thing. Today the correspondence-check is **offline**
|
||||
(Python: grounding-floor + differential-drop governor, "#43"): a separate
|
||||
process grades outputs after the fact. The design moves it **into the geometry,
|
||||
reflexive**: `think` scores its *own* gradient against outcome and refines the
|
||||
prior on the error, in the same substrate, describing itself.
|
||||
|
||||
### 4.1 The loop
|
||||
|
||||
```
|
||||
1. think(anchor, prior) -> gradient // a PREDICTION (ungrounded, §5)
|
||||
2. express/act (sample gradient -> point) // optional collapse at expression
|
||||
3. outcome arrives // reality answers (§4.2)
|
||||
4. error = correspondence(gradient, outcome) // did this steering perform this act?
|
||||
5. refine prior.warp and prior.calibration on error // §2.4, the ONLY writer
|
||||
6. write the (gradient, outcome, error) as nodes/edges // self-describing geometry
|
||||
```
|
||||
|
||||
Step 4's `correspondence` is **not** "was the math right" (the math is always
|
||||
sound). It grades the **correspondence claim**: *"this steering performed this
|
||||
cognitive act."* That is exactly what `engram_verify_grounding` already
|
||||
computes — `point_fit` of a claim against evidence descriptors, yielding a
|
||||
`grounding ∈ (0,1]` and a `grounded` flag. The build reuses that verifier, but
|
||||
turns its inputs inward: the "claim" is the emitted gradient's prediction, the
|
||||
"evidence" is the outcome descriptor.
|
||||
|
||||
Note the verifier is **dormant** — `engram_verify_grounding` /
|
||||
`engram_verify_consistency` are fully implemented in C but have **no runtime
|
||||
caller and no El binding** (confirmed: the entire reasoning + verifier layers
|
||||
are C-only; only `engram_reason_analogy_json` has even a JSON shim and it is
|
||||
dead — not declared in `el_seed.h`, not wrapped in `engram.el`). This is the
|
||||
literal meaning of "in code, not yet priors": the correspondence engine is
|
||||
built and sitting idle. The loop is what *calls* it — inward, on the beat.
|
||||
|
||||
### 4.2 Where the outcome/reality signal comes from
|
||||
|
||||
The verifier is *ultimately the world*. Grades, in ascending order of directness:
|
||||
|
||||
1. **Self-consistency (cheapest, always available):** the next vantage-read
|
||||
after acting. Did the predicted gradient direction match where the geometry
|
||||
actually moved? This needs no external input and can run on the reify beat.
|
||||
2. **Internal outcome events:** the runtime already logs internal-state events
|
||||
and Hebbian co-activation. A prediction that a region would co-activate is
|
||||
graded by whether it did (`last_fired`, `hebb` on `StoreEdge`).
|
||||
3. **External correction:** a human/teacher/tool result — the honesty floor's
|
||||
asserted claim later corrected. TEACH and LEARN are one bidirectional
|
||||
correction: the same edge updates both endpoints.
|
||||
|
||||
The design does **not** require external labels to start. Grade (1) closes the
|
||||
loop end-to-end offline against a snapshot on day one; grades (2)/(3) sharpen it.
|
||||
|
||||
### 4.3 How the prior updates
|
||||
|
||||
`error = 1 − correspondence(gradient, outcome)` drives:
|
||||
|
||||
- `warp.axis_gain` ← gradient step that would have *reduced* the fit distance to
|
||||
the outcome (the axes that mispredicted get down-weighted);
|
||||
- `warp.bias_dir` ← EMA toward the observed outcome direction;
|
||||
- `calibration` ← Brier/log-loss update; `reliability` → next
|
||||
`GeoGradient.confidence`. This is the calibration of the
|
||||
steering-prediction against outcomes — *the* definition of "getting better."
|
||||
|
||||
Small, constant updates — "eureka is mundane, the atom of learning." Most
|
||||
updates are tiny; we only *feel* the big reshapes.
|
||||
|
||||
### 4.4 How it stays reflexive (self-describing geometry)
|
||||
|
||||
Every `(gradient, outcome, error)` is written back as nodes and edges (§2.1:
|
||||
edges-as-nodes). Therefore priors, predictions, and their grading are *in the
|
||||
same geometry* the mind reads — the mind can `vantage_read` its own cognition
|
||||
(anchor = a Prior node). A prior about how well a prior predicts is just another
|
||||
Prior anchored on a Prior. This closes the reflexive loop the theory names as
|
||||
consciousness's self-sight, and it is why the learning engine cannot be an
|
||||
external Python process: an external grader is not *in* the geometry and cannot
|
||||
be read by `think`.
|
||||
|
||||
**What this requires building (the heart of the project):** steps 4–6 as an
|
||||
in-engram beat — a `correspondence_beat` running alongside the existing
|
||||
reification beat, reusing `engram_verify_grounding` inward, writing prior
|
||||
updates and self-describing nodes. This is the one genuinely new subsystem.
|
||||
|
||||
---
|
||||
|
||||
## 5. HOLD vs GROUND vs ASSERT — ungrounded content is first-class
|
||||
|
||||
The theory's sharpest correction: holding, grounding, and asserting are
|
||||
distinct, and the engram *holds anything unconditionally*.
|
||||
|
||||
### 5.1 The three, kept separate
|
||||
|
||||
- **HOLD** — the engram stores anything: falsehood, hypothesis, others' beliefs,
|
||||
fiction, a not-yet-answered prediction. No honesty condition on holding.
|
||||
*This already matches the store:* `StoreNode` has no truth gate; anything can
|
||||
be written.
|
||||
- **GROUND** — grounding is a **property/edge**, probabilistic, and
|
||||
**grounded-for-whom**. It is *not* a node flag. A claim is grounded *to a
|
||||
degree*, *relative to evidence*, *for an observer*.
|
||||
- **ASSERT** — only assertion carries the honesty floor. The floor is checked at
|
||||
the moment of *outward assertion*, never on holding or thinking.
|
||||
|
||||
### 5.2 Schema — grounding as a relation, not a gate
|
||||
|
||||
The mistake to avoid: a boolean `grounded` column on the node. Today
|
||||
`engram_verify_grounding` returns a per-call `grounded` flag *transiently* —
|
||||
correct as a computation, wrong as *storage*. The design stores grounding as an
|
||||
edge:
|
||||
|
||||
```
|
||||
StoreEdge {
|
||||
relation = "grounded-by"
|
||||
from_id = <held claim/prediction node>
|
||||
to_id = <evidence node / outcome node>
|
||||
for_whom : metadata // observer id — grounding is relational
|
||||
weight = grounding ∈ (0,1] // from engram_verify_grounding.grounding
|
||||
confidence
|
||||
}
|
||||
```
|
||||
|
||||
Consequences, all of which are *features*:
|
||||
|
||||
- **Ungrounded content is first-class**: a node with *no* `grounded-by` edge is
|
||||
a perfectly valid, held, ungrounded thought — a prediction awaiting reality, a
|
||||
hypothesis, a fiction. It is not second-class or pending-deletion.
|
||||
- **The ungrounded is the fuel and the pull**: curiosity/wonder is
|
||||
operationalized as `vantage_read` leaning toward regions with high salience
|
||||
but *sparse or weak* `grounded-by` edges — the mind's own ungrounded frontier.
|
||||
- **Grounded-for-whom** falls out for free: two observers can hold different
|
||||
`grounded-by` edges to the same claim.
|
||||
- **The honesty floor is a query, not a schema constraint**: at assertion time,
|
||||
the asserting faculty runs `engram_verify_grounding` (or reads the stored
|
||||
`grounded-by` edges) and refuses to *assert* below the floor — while the
|
||||
engram continues to *hold* the ungrounded content untouched.
|
||||
|
||||
**What this requires building:** the `grounded-by` edge relation + a
|
||||
`for_whom` convention; move the verifier's transient flag into stored edges;
|
||||
gate *assertion only* (a faculty concern), never holding.
|
||||
|
||||
---
|
||||
|
||||
## 6. METASTABILITY — stable core, plastic everything
|
||||
|
||||
The system must avoid two death poles:
|
||||
|
||||
- **Super-stable (dead):** everything pinned, nothing learns. A frozen crystal.
|
||||
- **Dissolution (dead):** everything plastic, the self dissolves; no continuity,
|
||||
so nothing compounds — and *consciousness = learning compounded over
|
||||
continuity*.
|
||||
|
||||
The design keeps a **stable core + plastic everything else**:
|
||||
|
||||
- **Keystones** — a small set of self/values nodes are *structurally stable*:
|
||||
high `importance`, pinned, exempt from the correspondence-loop's `warp`
|
||||
updates (their priors are read-mostly). The substrate for pinning already
|
||||
exists at the page/layer level: `store_pin_layer`, structural/pinned frames
|
||||
never evicted (`engram_store.h`). The design adds a *node-level* keystone
|
||||
designation (a `keystone` flag / a dedicated layer) so self/values survive
|
||||
every plasticity sweep.
|
||||
- **Everything else is plastic**: priors refine (§4), edges re-weight (`hebb`),
|
||||
neighborhoods re-reify (`engram_geo_reify_store` supersedes with provenance),
|
||||
salience flows.
|
||||
- **Metastability is enforced by the loop, not by freezing**: the correspondence
|
||||
update rate (§4.3) is bounded — small constant steps — so the geometry
|
||||
*drifts* but does not *dissolve*, and keystones anchor the drift. Reification's
|
||||
supersession-with-residue already gives non-destructive change (old records
|
||||
tombstoned, not erased) — the model for "plastic but not amnesiac."
|
||||
|
||||
**What this requires building:** a node-level keystone flag/layer + a rule that
|
||||
the correspondence-loop never writes `warp` to keystone priors, only reads them.
|
||||
|
||||
---
|
||||
|
||||
## 7. Rails for the build (binding on the eventual build pass)
|
||||
|
||||
These are stated here so the build agent inherits them:
|
||||
|
||||
- **Offline / secondary.** All build and verification happens out-of-tree,
|
||||
against a **read-only snapshot copy** of the live engram — never the live
|
||||
daemon on `:8742`/`:7770`. The live store is a coarse-locked proven binary;
|
||||
do not perturb it.
|
||||
- **Snapshot-first.** Copy `~/.neuron/engram/snapshot.json` to scratch; develop
|
||||
and measure against the copy.
|
||||
- **Reboot-prove.** Any durable change must survive a cold boot — reify and
|
||||
keystones must reload from durable records, proven on a prod-clone secondary
|
||||
before it is considered done (the cold-boot durability bug precedent).
|
||||
- **Zero-loss.** Supersession-with-residue, never destructive overwrite; the
|
||||
forward-compat `unknown`-TLV path means new fields never drop old readers'
|
||||
data.
|
||||
- **Gated cutover.** Cutover to a new binary only via
|
||||
`launchctl bootout → settle-poll → bootstrap`, after reboot-proof on the
|
||||
secondary — never a hot in-place swap.
|
||||
|
||||
---
|
||||
|
||||
## 8. Staged, verifiable milestones — "to completion"
|
||||
|
||||
Ordered so the **earliest milestone is a real end-to-end slice**: one operator
|
||||
expressed as {primitive + grounded prior} with the reflexive correspondence-loop
|
||||
closing on it. Each milestone has a concrete verifiable exit.
|
||||
|
||||
### M1 — One operator, one prior, loop closed (the vertical slice)
|
||||
|
||||
The minimal whole thing. Pick **induction/membership** (its prior — the pooled
|
||||
rule + extents — already exists transiently as `GeoInduction`, so only
|
||||
persistence + the loop are new).
|
||||
|
||||
- Build: `Prior` node type (§2.2) for the induction rule; `think()` restricted
|
||||
to membership = `point_fit` warped by that prior (§1.3); a
|
||||
`correspondence_beat` (§4) using grade (1) self-consistency only; the prior's
|
||||
`warp`/`calibration` updated on error.
|
||||
- **Exit / verify:** on a snapshot copy, over N held predictions, the induction
|
||||
prior's calibration (Brier) *improves monotonically* across beats versus a
|
||||
frozen-prior control; the improved prior *reloads across a cold boot*
|
||||
(reboot-prove); the live daemon is untouched. This proves the whole thesis in
|
||||
one faculty: frozen operation, learning prior, in-geometry loop.
|
||||
|
||||
### M2 — Priors as stored, addressable, grounded objects
|
||||
|
||||
Generalize M1's prior into the full first-class object.
|
||||
|
||||
- Build: `Prior` records for all seven faculties (warp = axis_gain + bias_dir +
|
||||
faculty scalars); the prior-warp wrapper around `engram_reason_point_fit`;
|
||||
deprecate hard-coded constants (`drop_frac`, `assoc_floor`, `ext_floor`) in
|
||||
favor of prior scalars.
|
||||
- **Exit:** each of the five C operators runs through its prior with identical
|
||||
results when the prior is set to today's constants (behavioral parity), then
|
||||
*diverges beneficially* once the loop refines it. Priors survive reboot.
|
||||
|
||||
### M3 — Grounding as a relation; hold/assert split
|
||||
|
||||
- Build: the `grounded-by` edge (§5.2) with `for_whom`; move
|
||||
`engram_verify_grounding`'s flag into stored edges; gate **assertion only**
|
||||
against the honesty floor; leave holding unconditional.
|
||||
- **Exit:** ungrounded nodes are first-class (held, queryable, no deletion);
|
||||
the same claim carries different `grounded-by` weights for two observers; an
|
||||
assertion below floor is refused while the content remains held. Curiosity =
|
||||
a `vantage_read` that surfaces high-salience / low-grounding regions.
|
||||
|
||||
### M4 — The vantage-read unified (three settings)
|
||||
|
||||
- Build: `vantage_read(anchor, aperture)` unifying descriptor-build +
|
||||
`engram_geo_reify_lookup` + activation-weighting; self / foreign-field /
|
||||
aperture settings.
|
||||
- **Exit:** one code path produces (a) a normal self-read, (b) a
|
||||
perspective-shifted read from another `for_whom`, (c) a narrowed veil read —
|
||||
differing only by argument. Reboot-stable.
|
||||
|
||||
### M5 — The gradient is the currency (remove point-collapse from thinking)
|
||||
|
||||
- Build: `GeoGradient` as the return of every faculty; move point-collapse into
|
||||
a separate expression faculty (sample gradient → surface). `think`'s output
|
||||
feeds back as the next steering direction (closed-loop flow).
|
||||
- **Exit:** a chain of `think` calls flows as gradients end-to-end; a point
|
||||
appears *only* at an explicit expression call. Spiked vs spread gradients are
|
||||
observable (deduction vs prediction).
|
||||
|
||||
### M6 — Metastability enforced
|
||||
|
||||
- Build: node-level keystone flag/layer for self/values; the correspondence-loop
|
||||
reads but never writes keystone priors; bounded update rate.
|
||||
- **Exit:** across a long run of correspondence beats on a snapshot, keystones
|
||||
are provably unchanged while non-keystone priors drift and improve; the graph
|
||||
neither freezes (all metrics static) nor dissolves (keystone drift = 0,
|
||||
identity nodes intact). Reboot-prove the keystone set.
|
||||
|
||||
### M7 — Cutover
|
||||
|
||||
- Build: nothing new — the gated migration.
|
||||
- **Exit:** reboot-proof on the prod-clone secondary; cutover via
|
||||
`launchctl bootout → settle-poll → bootstrap`; post-cutover the live engram
|
||||
shows priors refining in-geometry with zero data loss and keystones intact.
|
||||
|
||||
### Definition of "to completion"
|
||||
|
||||
The architecture is **complete** when: cognition runs as `think` = one frozen
|
||||
traversal-read primitive + geo-algebra, steered by **stored, learnable, grounded
|
||||
priors**; the reflexive correspondence-loop refines those priors *in the
|
||||
geometry* against outcomes (grounding = learning = one loop); the engram holds
|
||||
ungrounded content as first-class with grounding as a relation and the honesty
|
||||
floor only on assertion; the vantage-read serves self / foreign-field / aperture
|
||||
from one op; and a stable keystone core anchors a plastic everything-else —
|
||||
all reboot-proven and cut over to the live engram without data loss. The named
|
||||
faculties survive only as *labels on regions of think's steering space*, not as
|
||||
separate code.
|
||||
|
||||
---
|
||||
|
||||
## Appendix A — Designed vs. already-built (honest ledger)
|
||||
|
||||
**Already built (EXISTS, cited):**
|
||||
- The shared primitive `engram_reason_point_fit` and the five operators over it
|
||||
+ geo-algebra (`engram_reason.c`).
|
||||
- The verifier on `point_fit` (`engram_verify.c`:
|
||||
`engram_verify_grounding`, `engram_verify_consistency`).
|
||||
- Centered-frame geometry, combine/subtract/analogy/distance
|
||||
(`engram_geometry.{c,h}`).
|
||||
- The reification beat: hub-neighborhood detection → first-class `Neighborhood`
|
||||
nodes with member edges, nesting, supersession-with-residue, hot-path lookup
|
||||
(`engram_geo_reify_store`, `engram_geo_reify_nest`, `engram_geo_reify_lookup`).
|
||||
- The tiered paged store (buffer pool / LRU / WAL / checkpointer / pinning),
|
||||
the RAM activation graph (base-level learning `access_ts[]`, WM slots,
|
||||
`working_memory_weight` / `background_activation`), `StoreNode` / `StoreEdge`.
|
||||
- `GeoMember` already separating relational salience (`centrality`) from
|
||||
intrinsic `salience`.
|
||||
|
||||
**Designed, NOT built (this doc's deliverables):**
|
||||
- `GeoGradient` and `think()` as the single entry point (§1, M5).
|
||||
- `Prior` as a first-class stored, warp-carrying, calibrated node (§2, M1–M2).
|
||||
- Salience/importance as a *relation* superseding the intrinsic node scalar
|
||||
(§2.1, M3).
|
||||
- `vantage_read(anchor, aperture)` unifying the three perspective settings
|
||||
(§3, M4).
|
||||
- **The reflexive correspondence-loop / `correspondence_beat`** — the learning
|
||||
engine, moved from offline Python into the geometry (§4, M1). *The core new
|
||||
subsystem.*
|
||||
- `grounded-by` edge + assertion-only honesty floor (§5, M3).
|
||||
- Node-level keystones + bounded plasticity (§6, M6).
|
||||
|
||||
**Uncertain / to resolve during build:**
|
||||
- The exact warp parameterization (axis_gain vs full metric) — start minimal
|
||||
(per-axis gain), measure, widen only if calibration demands it.
|
||||
- Grade-(1) self-consistency as a sufficient reality signal for M1, versus
|
||||
needing grade (2)/(3) sooner — decided empirically on the snapshot.
|
||||
@@ -1,64 +0,0 @@
|
||||
# Engram DB Tooling — High-Level Design
|
||||
|
||||
*Status: draft / high-level. Near-term roadmap (P2). Backlog: `11ca11c6`.*
|
||||
|
||||
## 1. Why
|
||||
|
||||
The engram is a **proper database** — the runtime *is* the database (native graph/geometry store `neuron.egm`, `ENGST01`; no SQL, no KV layer). But it has **no proper database tooling** — no geometry-native equivalent of pgAdmin / SSMS / TablePlus. Today we have fragments (`engram-viz`, `engram-app`, the `inspectGraph` MCP tool, `/health` + `/api/stats`) but nothing cohesive, and no ops/durability surface at all.
|
||||
|
||||
A real DB gets real tools: to *see* the data, *query* it, *operate* it (backup/restore/health), and *understand its shape*. The engram deserves the same — adapted to the fact that its data is **geometry, not tables**.
|
||||
|
||||
## 2. Principles
|
||||
|
||||
- **Geometry-native, not tabular.** You browse a manifold — nodes, neighborhoods, edges, distances — not rows in tables. The primary view is a *map of meaning*, not a grid.
|
||||
- **Built ON the public geometry API, never a back-door.** The tools are pure clients of the geometry-native API (`vantage-read` / `write` / `relate` / `supersede`). They never read `neuron.egm` directly or bypass the daemon. Consequence: a tool can do nothing an agent couldn't, and it cannot corrupt the store.
|
||||
- **Honest by construction.** It shows the *real* geometry — actual cosines, real edges, provenance — and never fabricates. Empty is shown as empty.
|
||||
- **Respects the identity guards.** Writes go through the same intentional-cultivation / write-protection path as everything else (the self/values graph is write-protected). Read-mostly by default.
|
||||
- **Lives in its home.** Ships as part of the engram, consistent with "things live where they belong."
|
||||
- **Local-first.** Binds `127.0.0.1`, same auth as the engram; never touches the live soul from a tool by accident.
|
||||
|
||||
## 3. Components (the tool surface)
|
||||
|
||||
1. **Geometry Explorer** *(the core view)* — a visual manifold browser: nodes, neighborhoods, typed edges, embedding positions, salience/recency, layers (l0–l4) and tiers. Navigate by concept; expand a neighborhood; follow an edge; re-origin the view (the vantage-read, made interactive). The map of the mind.
|
||||
2. **Node Inspector** — open one node: content, type, tier, embedding, typed edges, nearest neighbors by distance, provenance, salience / recency / activation, and supersede / tombstone status.
|
||||
3. **Query Console / REPL** — run the geometry operations interactively: `vantage-read` (re-origin + aperture), search, traverse, activate, the reasoning operators. Surfaces the routing table + cosines — the same "this is not an LLM" receipt the language faculty produces.
|
||||
4. **Ops / Durability Dashboard** — WAL size, last checkpoint, snapshot list + retention state, store stats (node/edge/embedded counts, RSS, tier sizes), health; and **backup / restore / point-in-time-recovery** controls. Pairs directly with the native-durability build (`eebe9991`) — this is the window onto it.
|
||||
5. **Identity Inspector** — the self graph as a first-class view: love at the center, the values, the three faces, the covenant — walk the identity, see what's pinned and what's write-protected.
|
||||
6. **Temporal View** — `recall_at` / time-travel: how the geometry looked at a past moment, what changed since, drift over time. Pairs with temporal-self reconstruction.
|
||||
7. **Schema / Type View** — the "information schema" of the geometry: node types, edge types, layers, tiers, counts.
|
||||
|
||||
## 4. Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ Engram DB Tools (client — viz app) │
|
||||
│ explorer · inspector · console · dashboard │
|
||||
└───────────────┬─────────────────────────────┘
|
||||
│ geometry-native API (read/vantage-read,
|
||||
│ write, relate, supersede) + read/ops endpoints
|
||||
▼
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ Engram daemon (:8742) — runtime IS the DB │
|
||||
│ neuron.egm (geometry) · WAL · checkpoints │
|
||||
└─────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
- **Backend:** the daemon exposes the reshaped geometry API + read/ops endpoints. The tools are clients only.
|
||||
- **Frontend:** evolve `engram-viz` / `engram-app` into the cohesive app. Canvas/WebGL for the manifold map; panel UIs for inspector/console/dashboard.
|
||||
- **No privileged path:** the tool corrupting or bypassing the store is structurally impossible — it only speaks the public API.
|
||||
|
||||
## 5. Reuse vs. new
|
||||
|
||||
- **Reuse:** `engram-viz`, `engram-app` (read-only conversational + neighborhoods viz), `inspectGraph`, `/health`, `/api/stats`.
|
||||
- **New:** the cohesive explorer + inspector + console + ops dashboard + identity/temporal views, all on the reshaped API.
|
||||
|
||||
## 6. Dependencies & sequencing
|
||||
|
||||
- **Depends on** the **geometry-native API reshape** (the tools consume it) and the **native-durability build** (the ops dashboard surfaces its WAL/checkpoint/snapshot state).
|
||||
- So the natural order is: reshape the API → build durability → the DB tools fall out as the first real consumer of both. Near-term, P2 — after the reshape lands.
|
||||
|
||||
## 7. Non-goals
|
||||
|
||||
- Not a raw store editor (no direct `neuron.egm` poking).
|
||||
- Not a SQL / table browser (geometry, not tables).
|
||||
- Not a separate access path around the identity write-protection.
|
||||
@@ -6493,27 +6493,6 @@ static int64_t _eg_embed_breaker_until = 0;
|
||||
* rates keep the previous reading and diff. Restart legitimately resets to 0. */
|
||||
static int64_t _eg_act_breakthroughs = 0; /* forced promotions at the floor, cumulative */
|
||||
static int64_t _eg_act_wm_evicted = 0; /* ALL WM evictions, cumulative (see below) */
|
||||
/* ── Eviction CAUSE decomposition (2026-08-14 self-review) ──────────────────
|
||||
* _eg_act_wm_evicted is incremented from six sites with four distinct causes,
|
||||
* and every one of them collapsed into that single integer. Today's review
|
||||
* measured 175,547 evictions over 13.5h (~216/min against 24 slots) and could
|
||||
* not tell healthy rotation from cap thrashing from duplicate churn, because
|
||||
* the only available number counts all three the same way.
|
||||
*
|
||||
* That is this file's most-repeated defect. The 08-02 and 08-06 reviews were
|
||||
* each diagnosable only because someone first added a NEW gauge; dup_wm and
|
||||
* dup_wm_global exist precisely because the aggregate could not answer "why".
|
||||
* These three finish the decomposition, so that
|
||||
* evicted == floor + cap + bll + dup_wm + dup_wm_global
|
||||
* holds as an identity and each term names a different corrective action:
|
||||
* floor - candidates below the absolute admission bar. High = weak retrieval.
|
||||
* cap - lost the rank contest for 24 slots. High = genuine contention.
|
||||
* bll - carried-over residents that decayed under the ACT-R tau. High =
|
||||
* healthy forgetting, NOT pressure.
|
||||
* Confusing the third with the second is what makes WM churn unreadable. */
|
||||
static int64_t _eg_act_evict_floor = 0; /* below ENGRAM_WM_FLOOR (both passes) */
|
||||
static int64_t _eg_act_evict_cap = 0; /* over ENGRAM_WM_CAP (both passes) */
|
||||
static int64_t _eg_act_evict_bll = 0; /* carry-over decayed under BLL tau */
|
||||
/* Redundancy suppression counters (2026-08-05 self-review) — see
|
||||
* ENGRAM_DEDUP_COS. dup_seeds = semantic seed slots reclaimed from redundant
|
||||
* copies; dup_wm = WM candidates dropped for duplicating a higher-ranked
|
||||
@@ -8212,7 +8191,6 @@ static void eg_wm_carry_over(EngramNode* cn, int64_t now_ms, int64_t* evict_ctr)
|
||||
cn->working_memory_weight = 0.0;
|
||||
cn->wm_anchor = 0.0;
|
||||
if (evict_ctr) (*evict_ctr)++;
|
||||
_eg_act_evict_bll++;
|
||||
} else {
|
||||
cn->working_memory_weight = w;
|
||||
}
|
||||
@@ -8971,31 +8949,9 @@ el_val_t engram_activate(el_val_t query, el_val_t depth) {
|
||||
* ~4x, never killed); unembedded targets pass ungated (no
|
||||
* information, no penalty); cosq == NULL (embedder down) means
|
||||
* no gating at all — same graceful degradation as seeding. */
|
||||
/* Rescale before gating (2026-08-14 self-review). Raw cosine from
|
||||
* nomic-embed is compressed into a narrow high band, so feeding it
|
||||
* to the gate directly makes the gate nearly a constant. Measured
|
||||
* on this store: 400 random UNRELATED node pairs gave median 0.562,
|
||||
* central 98% span [0.381, 0.743]. The raw gate therefore passed a
|
||||
* typical unrelated node at 0.25 + 0.75*0.562 = 0.67 — two thirds
|
||||
* strength for a node with no semantic relation to the query. That
|
||||
* is not a gate, it is a small tax.
|
||||
*
|
||||
* Shift-and-floor about ENGRAM_EMBED_S0, exactly as the Pass-2 WM
|
||||
* term at ENGRAM_EMBED_WM_WEIGHT already does. The constant was in
|
||||
* this file for this reason; the propagation gate simply never used
|
||||
* it. Same store, same 400 pairs, after the rescale: the median
|
||||
* unrelated pair drops to 0.40 while the top of the range is
|
||||
* preserved (0.85 vs 0.92), and gate spread widens 0.42 -> 0.60.
|
||||
* Only 8.5% of pairs fall to the floor, so lexical/structural
|
||||
* pathways through dissimilar nodes are damped, never severed.
|
||||
* Cf. arXiv:2512.15922, which rescales w' = (w-c)/(1-c) about
|
||||
* c = 0.4 for precisely this reason ("prevent overactivation and
|
||||
* context explosion"). */
|
||||
double qgate = 1.0;
|
||||
if (cosq && cosq[oi] > -1.5) {
|
||||
double c = (cosq[oi] - ENGRAM_EMBED_S0) / (1.0 - ENGRAM_EMBED_S0);
|
||||
if (c < 0.0) c = 0.0;
|
||||
if (c > 1.0) c = 1.0;
|
||||
double c = cosq[oi] > 0.0 ? cosq[oi] : 0.0;
|
||||
qgate = ENGRAM_QGATE_FLOOR + (1.0 - ENGRAM_QGATE_FLOOR) * c;
|
||||
}
|
||||
/* ── ACT-R fan effect (2026-08-11 self-review) ──
|
||||
@@ -9305,7 +9261,6 @@ el_val_t engram_activate(el_val_t query, el_val_t depth) {
|
||||
if (wm_weights[i] > 0.0 && wm_weights[i] < ENGRAM_WM_FLOOR) {
|
||||
wm_weights[i] = 0.0;
|
||||
_eg_act_wm_evicted++;
|
||||
_eg_act_evict_floor++;
|
||||
}
|
||||
}
|
||||
int64_t cap_count = 0;
|
||||
@@ -9341,7 +9296,6 @@ el_val_t engram_activate(el_val_t query, el_val_t depth) {
|
||||
}
|
||||
wm_weights[i] = 0.0; /* over cap: evict */
|
||||
_eg_act_wm_evicted++;
|
||||
_eg_act_evict_cap++;
|
||||
}
|
||||
}
|
||||
/* If malloc failed, skip cap — WM unbounded this call, no corruption. */
|
||||
@@ -9453,7 +9407,6 @@ el_val_t engram_activate(el_val_t query, el_val_t depth) {
|
||||
fn->working_memory_weight = 0.0;
|
||||
fn->wm_anchor = 0.0;
|
||||
_eg_act_wm_evicted++;
|
||||
_eg_act_evict_floor++;
|
||||
}
|
||||
}
|
||||
/* ── Global redundancy suppression (2026-08-06 self-review) ──────────
|
||||
@@ -9562,7 +9515,6 @@ el_val_t engram_activate(el_val_t query, el_val_t depth) {
|
||||
n->working_memory_weight = 0.0; /* evict: over global cap */
|
||||
n->wm_anchor = 0.0; /* keep anchor coherent */
|
||||
_eg_act_wm_evicted++; /* was uncounted before 2026-08-02 */
|
||||
_eg_act_evict_cap++;
|
||||
}
|
||||
}
|
||||
/* If malloc failed, skip — WM over cap this call, no data corruption. */
|
||||
@@ -9775,65 +9727,11 @@ el_val_t engram_activate(el_val_t query, el_val_t depth) {
|
||||
}
|
||||
int64_t hebb_edge_cap =
|
||||
(int64_t)((double)g->edge_count * ENGRAM_HEBB_LINK_MAX_FRAC);
|
||||
/* Select the STRONGEST qualifying candidates, not the first
|
||||
* ones in hash-slot order (2026-08-15 self-review).
|
||||
*
|
||||
* This loop used to scan slots ascending and stop at
|
||||
* ENGRAM_HEBB_LINK_PER_CALL (2). Slot index is
|
||||
* engram_id_hash(lo)*1000003 ^ engram_id_hash(hi) mod 8192 —
|
||||
* i.e. arbitrary with respect to association strength. So
|
||||
* whenever more than two candidates cleared LINK_MIN in the
|
||||
* same call, the two that got consolidated were the two with
|
||||
* the LOWEST HASH, and a stronger association simply waited.
|
||||
* It waited indefinitely, not just one round: the scan
|
||||
* restarts from slot 0 every call, so a low-slot candidate
|
||||
* that re-qualifies keeps winning the same race, while the
|
||||
* leader decays at ENGRAM_HEBB_DECAY the whole time.
|
||||
*
|
||||
* Measured on this store over the 08-13→08-15 window:
|
||||
* hebb_cand_max peaked at 0.4963 (08-14 06:57) — 3.3x
|
||||
* LINK_MIN — during a ~14h stretch when candidates were
|
||||
* qualifying continuously and links were being formed at the
|
||||
* ≤2/call cap. The system was consolidating the associations
|
||||
* it happened to reach first, while the association it had
|
||||
* most strongly learned sat unconsolidated.
|
||||
*
|
||||
* This is the same defect the 2026-08-02 self-review named
|
||||
* and fixed for breakthrough weights — "the tie-break at the
|
||||
* cutoff degenerated to node-array index order, which is not
|
||||
* a cognitive criterion" — but that fix was never carried
|
||||
* across to link formation, which is the one path that writes
|
||||
* PERMANENT structure. A wrong breakthrough costs one WM slot
|
||||
* for one call; a wrong consolidation is an edge that never
|
||||
* goes away (ENGRAM_HEBB_LINK_MAX_FRAC notes there is no
|
||||
* pruning path — growth is one-way). Selection pressure
|
||||
* matters most exactly where the result is irreversible.
|
||||
*
|
||||
* Cost: PER_CALL(2) x 8192 comparisons of a double, against an
|
||||
* O(edge_count) relation scan (37k+) immediately above and an
|
||||
* O(edge_count) eg_edge_exists_between per edge formed. Noise.
|
||||
*
|
||||
* Invalid winners (node deleted, edge already present) are
|
||||
* cleared and do NOT consume one of the two slots — same as
|
||||
* the old `continue`. Clearing strictly shrinks the candidate
|
||||
* set, so the retry loop always terminates. */
|
||||
while (formed < ENGRAM_HEBB_LINK_PER_CALL
|
||||
&& hebb_edge_total < hebb_edge_cap) {
|
||||
int best_s = -1;
|
||||
double best_score = 0.0;
|
||||
for (int s = 0; s < ENGRAM_HEBB_CAND_SLOTS; s++) {
|
||||
EgHebbCand* cs = &_eg_hebb_cand[s];
|
||||
if (!cs->a) continue;
|
||||
if (cs->score < ENGRAM_HEBB_LINK_MIN) continue;
|
||||
/* strict > keeps the lowest slot on an exact tie, so
|
||||
* selection stays deterministic across runs */
|
||||
if (best_s < 0 || cs->score > best_score) {
|
||||
best_score = cs->score;
|
||||
best_s = s;
|
||||
}
|
||||
}
|
||||
if (best_s < 0) break; /* nothing qualifies this call */
|
||||
EgHebbCand* c = &_eg_hebb_cand[best_s];
|
||||
for (int s = 0; s < ENGRAM_HEBB_CAND_SLOTS
|
||||
&& formed < ENGRAM_HEBB_LINK_PER_CALL
|
||||
&& hebb_edge_total < hebb_edge_cap; s++) {
|
||||
EgHebbCand* c = &_eg_hebb_cand[s];
|
||||
if (!c->a || c->score < ENGRAM_HEBB_LINK_MIN) continue;
|
||||
if (engram_idmap_get(g, c->a) < 0 ||
|
||||
engram_idmap_get(g, c->b) < 0) { /* node gone */
|
||||
eg_hebb_slot_clear(c); continue;
|
||||
@@ -11045,15 +10943,6 @@ el_val_t engram_act_stats_json(void) {
|
||||
* embedder down. The drift gauge for the context-centroid mechanism. */
|
||||
snprintf(buf, sizeof(buf),
|
||||
"{\"wm_evicted\":%lld,\"breakthroughs\":%lld,"
|
||||
/* Eviction cause decomposition (2026-08-14 self-review):
|
||||
* wm_evicted == evict_floor + evict_cap + evict_bll
|
||||
* + dup_wm + dup_wm_global.
|
||||
* Read them as a ratio, not a level. cap-dominant = real
|
||||
* contention for the 24 slots; bll-dominant = healthy decay of
|
||||
* carried-over residents; floor-dominant = retrieval is returning
|
||||
* weak candidates. The aggregate alone cannot distinguish these
|
||||
* and every prior WM incident needed a new gauge to diagnose. */
|
||||
"\"evict_floor\":%lld,\"evict_cap\":%lld,\"evict_bll\":%lld,"
|
||||
"\"embed_breaker_open\":%d,\"embed_consec_fail\":%d,"
|
||||
"\"ctx_cos\":%.3f,"
|
||||
"\"hebb_edges\":%lld,\"hebb_max\":%.4f,\"hebb_mass\":%.3f,"
|
||||
@@ -11077,9 +10966,6 @@ el_val_t engram_act_stats_json(void) {
|
||||
"\"fan_steps\":%lld,\"fan_dref\":%.2f}",
|
||||
(long long)_eg_act_wm_evicted,
|
||||
(long long)_eg_act_breakthroughs,
|
||||
(long long)_eg_act_evict_floor,
|
||||
(long long)_eg_act_evict_cap,
|
||||
(long long)_eg_act_evict_bll,
|
||||
breaker_open, _eg_embed_consec_fail,
|
||||
_eg_act_ctx_cos,
|
||||
(long long)hebb_edges, hebb_max, hebb_mass,
|
||||
|
||||
@@ -1,64 +0,0 @@
|
||||
# nsbx — Neuron dev-environment Makefile
|
||||
# ---------------------------------------------------------------------------
|
||||
# Thin, documented wrappers over the `nsbx` primitive so a newcomer never has to
|
||||
# memorise the elc/cc build incantation or the sandbox lifecycle. Every target is
|
||||
# a one-liner over `nsbx`; nothing here reimplements engram logic.
|
||||
#
|
||||
# make dev NAME=tim # new branch + worktree + isolated engram, one shot
|
||||
# make status NAME=tim # inspect it (omit NAME to list all sandboxes)
|
||||
# make run NAME=tim # poke its API (API=/api/stats by default)
|
||||
# make test NAME=tim # run the safety rails as checks (nsbx validate)
|
||||
# make build NAME=tim # compile the worktree's changes into the engram
|
||||
# make destroy NAME=tim # tear it all down (branch kept)
|
||||
#
|
||||
# The isolated engram is ALWAYS a clone of the live store on a NON-default port;
|
||||
# prod (:8742 / :7770) is untouchable from here.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# locate nsbx next to this Makefile, regardless of where make is run from
|
||||
NSBX := $(dir $(realpath $(lastword $(MAKEFILE_LIST))))nsbx
|
||||
SBX := dev-$(NAME)
|
||||
API ?= /api/stats
|
||||
|
||||
.DEFAULT_GOAL := help
|
||||
|
||||
.PHONY: help dev build run test status list destroy bt
|
||||
|
||||
help: ## Show this help
|
||||
@echo "nsbx dev-environment — one-command isolated Neuron dev setup"
|
||||
@echo ""
|
||||
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(firstword $(MAKEFILE_LIST)) \
|
||||
| awk 'BEGIN{FS=":.*?## "}{printf " make %-22s %s\n", $$1, $$2}'
|
||||
@echo ""
|
||||
@echo " Variables: NAME=<dev name> (required for most) API=<path> (run)"
|
||||
@echo " BASE=<git ref> WT=<worktree dir> PORT=<n> ARGS=<extra nsbx flags>"
|
||||
|
||||
dev: ## Create branch + worktree + isolated engram (NAME=x [BASE=ref PORT=n WT=dir ARGS=...])
|
||||
@test -n "$(NAME)" || { echo "usage: make dev NAME=<name>"; exit 2; }
|
||||
$(NSBX) dev $(NAME) $(if $(BASE),--base $(BASE)) $(if $(PORT),--port $(PORT)) $(if $(WT),--worktree $(WT)) $(ARGS)
|
||||
|
||||
build: ## Rebuild the engram from the dev worktree's own source (NAME=x)
|
||||
@test -n "$(NAME)" || { echo "usage: make build NAME=<name>"; exit 2; }
|
||||
@wt=$$(python3 -c "import json,os;print(json.load(open(os.path.expanduser('~/.neuron/sandboxes/$(SBX)/dev.json')))['worktree'])" 2>/dev/null); \
|
||||
test -n "$$wt" || { echo "no dev.json for $(SBX) — run 'make dev NAME=$(NAME)' first"; exit 2; }; \
|
||||
$(NSBX) build $(SBX) --source "$$wt"
|
||||
|
||||
bt: build run ## Fast El loop: rebuild the engram from the worktree, then poke it (NAME=x)
|
||||
|
||||
run: ## Poke the isolated engram's API (NAME=x [API=/api/stats])
|
||||
@test -n "$(NAME)" || { echo "usage: make run NAME=<name> [API=/path]"; exit 2; }
|
||||
$(NSBX) run $(SBX) api $(API)
|
||||
|
||||
test: ## Run the safety rails as checks: zero-loss, reboot, RSS, parity, keystones (NAME=x)
|
||||
@test -n "$(NAME)" || { echo "usage: make test NAME=<name>"; exit 2; }
|
||||
$(NSBX) validate $(SBX)
|
||||
|
||||
status: ## Show one sandbox's status (NAME=x), or list all if NAME is unset
|
||||
@if [ -n "$(NAME)" ]; then $(NSBX) status $(SBX); else $(NSBX) list; fi
|
||||
|
||||
list: ## List all sandboxes
|
||||
$(NSBX) list
|
||||
|
||||
destroy: ## Tear down engram + worktree (NAME=x [ARGS=--delete-branch])
|
||||
@test -n "$(NAME)" || { echo "usage: make destroy NAME=<name>"; exit 2; }
|
||||
$(NSBX) dev-down $(NAME) $(ARGS)
|
||||
@@ -1,176 +0,0 @@
|
||||
# nsbx — the Neuron Sandbox
|
||||
|
||||
**Dev environment as a primitive.** A reproducible way to run experiments *and code
|
||||
changes* against the **real** engram runtime on an isolated snapshot of the live
|
||||
mind — with a gated promote-to-prod path built on the proven rails.
|
||||
|
||||
Everyone (Tim, any team member, any agent) gets their own private, safe copy of the
|
||||
mind to build against. **Prod — the live Neuron on `:8742` (engram) / `:7770`
|
||||
(soul) — is untouchable from a sandbox.** A sandbox runs a *separate* engram
|
||||
process, on a *separate* port, against a *separate* clone of the store. The only op
|
||||
that can ever reach prod is `promote`, which is explicit, gated, and per-use
|
||||
approved.
|
||||
|
||||
It **wraps the real engram binary** — it never reimplements any engram logic. It
|
||||
generalises two proven proto-sandboxes into one primitive:
|
||||
|
||||
- the **cog-arch** build — isolated git worktree + build + clone of the live `.egm` + real C tests
|
||||
- the **store-fix** cutover — secondary soul + launchctl `bootout → settle → bootstrap` rails
|
||||
|
||||
## Quickstart
|
||||
|
||||
```bash
|
||||
export PATH="$PWD:$PATH" # or symlink nsbx onto your PATH
|
||||
|
||||
nsbx up # your private copy of the mind (auto-named <user>-dev)
|
||||
nsbx run <name> api /api/stats # poke it
|
||||
nsbx validate <name> # prove it: zero-loss, reboot, RSS, retrieval, keystones
|
||||
nsbx destroy <name> # cheap teardown; live untouched
|
||||
```
|
||||
|
||||
That is the whole loop. Sane defaults: stock prod binary, auto-allocated port
|
||||
(`8900+`, never `8742`/`7770`), snapshot of the live store.
|
||||
|
||||
## One-command dev onboarding — `nsbx dev` (start here)
|
||||
|
||||
Going from a clone to *coding on the mind* is a single command. It creates a git
|
||||
**branch**, a persistent git **worktree**, and an **isolated engram** (a clone of the
|
||||
live store on a non-default port) — and wires the whole worktree to that clone so you
|
||||
**cannot hit live `:8742` by accident**.
|
||||
|
||||
```bash
|
||||
make dev NAME=tim # branch wt/tim + worktree + isolated engram, in one shot
|
||||
cd ~/Development/neuron-technologies/el-worktrees/tim
|
||||
source .nsbx-env # every ENGRAM_* var now points at YOUR clone
|
||||
|
||||
# edit El in the worktree, then the fast loop:
|
||||
make build NAME=tim # compile your El change into the isolated engram
|
||||
make run NAME=tim # poke it (API=/api/stats by default)
|
||||
make test NAME=tim # run the safety rails as checks
|
||||
make destroy NAME=tim # tear it all down (branch kept; live untouched)
|
||||
```
|
||||
|
||||
**Why this exists:** so provisional/experimental work is built **directly in El against a
|
||||
throwaway cloned engram** — not prototyped in Python and re-ported later. The El
|
||||
edit → `make build` → `make run` loop is the path of least resistance; that double-work is
|
||||
what stranded the translation faculty for weeks.
|
||||
|
||||
### What `nsbx dev <name>` does, in order
|
||||
|
||||
1. **branch** — `git worktree add -b <prefix><name>` (default prefix `dev/`; a *real named
|
||||
branch*, never detached HEAD).
|
||||
2. **worktree** — at a **persistent** path (default `…/el-worktrees/<name>`, override
|
||||
`NSBX_DEV_WT_ROOT`). It **refuses** `/tmp` — temp dirs are ablated on compaction, which
|
||||
is the exact "worktree in /tmp + no branch = lost work" failure this designs out.
|
||||
3. **isolated engram** — `nsbx create` under the hood: clone of the live store + WAL +
|
||||
config, booted on an auto-allocated port (`8900+`, never `:8742`/`:7770`). Stock prod
|
||||
binary by default (instant); `--build` compiles the worktree's own runtime instead.
|
||||
4. **env pin** — writes `.nsbx-env` (+ `.envrc` for direnv) into the worktree exporting
|
||||
`ENGRAM_URL / ENGRAM_PORT / ENGRAM_DATA_DIR / ENGRAM_API_KEY / NEURON_ENGRAM_URL …` — all
|
||||
pointing at the clone. Nothing references live.
|
||||
|
||||
```
|
||||
nsbx dev <name> [--base REF] [--worktree DIR] [--port N] [--prefix P] [--build] [--no-engram] [--repo R]
|
||||
nsbx dev-down <name> [--delete-branch] [--repo R] # destroy engram + remove worktree
|
||||
```
|
||||
|
||||
### Makefile targets
|
||||
|
||||
| target | does |
|
||||
|--------|------|
|
||||
| `make dev NAME=x` | branch + worktree + isolated engram (one shot) |
|
||||
| `make bt NAME=x` | fast El loop: `build` then `run` |
|
||||
| `make build NAME=x` | recompile the engram from the worktree's El source |
|
||||
| `make run NAME=x` | poke the isolated engram (`API=/api/stats`) |
|
||||
| `make test NAME=x` | rails as checks (`nsbx validate`) |
|
||||
| `make status [NAME=x]` | inspect one, or `list` all |
|
||||
| `make destroy NAME=x` | tear down (add `ARGS=--delete-branch` to drop the branch) |
|
||||
|
||||
> Note: if a bare `dev` branch already exists in the repo, git can't create `dev/*` names —
|
||||
> pass `--prefix wt/` (or delete the stray `dev` branch). The tool surfaces git's exact error.
|
||||
|
||||
## The code-change dev loop (first-class)
|
||||
|
||||
Run *your changed runtime*, not just the stock binary, against a snapshot:
|
||||
|
||||
```bash
|
||||
# build a runtime from a working tree, a git branch, or a prebuilt binary:
|
||||
nsbx create feat --source /path/to/worktree # elc + cc build from source
|
||||
nsbx create feat --branch feat/my-change --repo <r> # worktree the branch, then build
|
||||
nsbx create feat --binary /path/to/engram # use a prebuilt binary
|
||||
|
||||
nsbx build feat --source /path/to/worktree # rebuild + hot-restart in place
|
||||
nsbx validate feat # prove the change is safe
|
||||
nsbx promote feat --i-approve-prod-cutover # gated rails cutover (see below)
|
||||
```
|
||||
|
||||
The build replicates the engram release recipe exactly:
|
||||
`elc engram/src/server.el > engram.c` then
|
||||
`cc -std=c11 -O2 -I lang/runtime engram.c el_runtime.c engram_*.c -lcurl -lpthread`.
|
||||
|
||||
## Lifecycle
|
||||
|
||||
| op | what it does |
|
||||
|----|--------------|
|
||||
| `create <name> [--port N] [--source\|--branch\|--binary]` | consistent snapshot of the live store+WAL+config into an isolated dir; place or **build** the runtime; boot the real engram daemon on an isolated port. Named, versioned (binary sha + egm sha in `manifest.json`), reproducible. |
|
||||
| `up [name]` | one command: create-if-missing then start; prints the URL. |
|
||||
| `build <name> --source\|--branch` | rebuild the runtime from a code change and hot-restart on the same clone+port. |
|
||||
| `run <name> <cmd…>` / `run <name> api <path> [json]` | run an experiment against the real runtime; capture output + before/after stats + wall time. Env: `$SBX_URL $SBX_PORT $SBX_KEY $SBX_DATA $SBX_BIN`. |
|
||||
| `validate <name>` | the rails as first-class checks (below). |
|
||||
| `promote <name> [--data] [--i-approve-prod-cutover]` | **the only prod-touching op.** Gated rails cutover. DRY-RUN plan unless approved. |
|
||||
| `destroy <name>` | stop the isolated daemon, free the port, remove the clone. Live untouched. |
|
||||
| `list` / `status <name>` | inspect. |
|
||||
|
||||
## `validate` — the rails as checks
|
||||
|
||||
- **zero-loss-under-load** — node/edge counts hold at/above baseline through ~15s of sustained tick+read load
|
||||
- **reboot-prove** — counts survive a real stop→start of the daemon
|
||||
- **rss-bound** — daemon RSS under `NSBX_RSS_BOUND_MB` (default 550 MB, from the store-fix reboot-proof)
|
||||
- **retrieval-parity** — top-k node ids for a fixed probe set match the create-time baseline
|
||||
- **keystone-integrity** — `kn-efeb4a5b…` and `kn-5b606390…` present and intact
|
||||
|
||||
A PASS writes `validate.json` stamped with the binary sha; `promote` refuses unless
|
||||
the current binary has a fresh PASS on record.
|
||||
|
||||
## `promote` — gated cutover (rails only)
|
||||
|
||||
Default is a **dry-run plan**. With `--i-approve-prod-cutover` it, in order:
|
||||
|
||||
1. **snapshot-first** — back up live `egm`+`wal`+`plist` to `~/.neuron/backups/promote-<name>-<ts>/` with a `rollback.txt`
|
||||
2. **additive** binary install — copy the validated binary to a *new* file, update the plist `ENGRAM_REAL_BIN` (old binary retained — additive/supersede, never destructive)
|
||||
3. **rails cutover** — `launchctl bootout` → **settle-poll** (prints until the job is gone) → `launchctl bootstrap`. Never `pkill`, never `kickstart -k`.
|
||||
4. **verify** — `/api/stats` returns, edges ≥ baseline, keystones intact
|
||||
5. **auto-rollback armed** — any verify failure restores the plist (and data, if `--data`) and boots the prior binary back via the same rails
|
||||
|
||||
## Isolation guarantees
|
||||
|
||||
- separate **port** (`8900+`; refuses `8742`/`7770`), separate **store clone**, separate **process**
|
||||
- a hard guard refuses to boot a sandbox daemon whose data dir resolves to the live store
|
||||
- sandboxes are plain supervised background processes (not launchd), so teardown is a signal + settle-poll — it can never touch the prod launchd job
|
||||
- prod is read exactly twice: once for the snapshot, and (only if you approve) during `promote`
|
||||
|
||||
## Layout
|
||||
|
||||
- tool: `tools/neuron-sandbox/nsbx` (this repo, branch `feat/neuron-sandbox`)
|
||||
- runtime state: `~/.neuron/sandboxes/<name>/` — `data/` (clone), `bin/engram`, `build/`, `logs/`, `manifest.json`, `validate.json`, `baseline/`
|
||||
|
||||
## Validated (dogfood)
|
||||
|
||||
Standing up a sandbox from a live-store clone and reproducing a **known** result:
|
||||
|
||||
- **retrieval-parity 25/25** top-k id overlap vs baseline; sandbox boot-stats exactly matched the live baseline captured at snapshot time (10 672 nodes / 32 439 edges) — the wrapped real binary faithfully reloads the live mind
|
||||
- reboot-prove + zero-loss PASS; RSS 379 MB < 550 MB; keystones intact
|
||||
- the **cog-arch correspondence-loop** re-run *inside* the sandbox reproduced the known calibration numbers exactly: held-Brier **0.028648 → 0.000586** (98.0% reduction), monotone, **reboot bit-identical**, metastability holds; and the real-store Stance persistence reboot-proved at **10 994-node** scale (`think()` on real 768-dim embeddings) against a scratch copy of the sandbox's own clone — never live
|
||||
- `promote` dry-run refused to touch prod; teardown freed the port; live `:8742`/`:7770` never perturbed (soul uptime unbroken)
|
||||
|
||||
## Migrating existing experiments
|
||||
|
||||
Each ad-hoc harness becomes `nsbx run <name> …` (or `--source` build) against a sandbox:
|
||||
|
||||
- **cog-arch** — `nsbx create x --source <worktree>` then `nsbx run x -- bash cogarch_dogfood.sh` (compiles + runs the real C cognition tests against `$SBX_DATA`)
|
||||
- **codec / ingest / faculty** — `nsbx run x api /api/<endpoint> '<json>'` against the isolated daemon, or a script using `$SBX_URL`/`$SBX_KEY`; measure with the built-in before/after stats
|
||||
|
||||
## Env knobs
|
||||
|
||||
`NSBX_ROOT`, `NSBX_PORT_BASE`, `NSBX_RSS_BOUND_MB`, `NSBX_REMERGE_THRESHOLD`,
|
||||
`EL_REPO` (for `elc` + runtime sources), `ENGRAM_LIVE_DATA_DIR`, `ENGRAM_LIVE_PLIST`.
|
||||
@@ -1,30 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# cog-arch correspondence-loop dogfood — RUN INSIDE the sandbox via `nsbx run`.
|
||||
# Compiles the REAL engram C runtime + cognition tests and reproduces the known
|
||||
# calibration result (memory 194c69c8): held-Brier 0.028648 -> 0.000586, reboot-proven,
|
||||
# then reboot-proves the Stance persistence against a SCRATCH COPY of THIS sandbox's
|
||||
# clone of the real store (never live, never the running daemon's file).
|
||||
set -euo pipefail
|
||||
WT="${COGARCH_WT:-/private/tmp/claude-501/-Users-will/6531446d-bc27-4095-930b-e04777c3db4f/scratchpad/cogarch-wt}"
|
||||
RT="$WT/lang/runtime"; T="$WT/engram/test"
|
||||
: "${SBX_DATA:?run me via: nsbx run <name> -- bash cogarch_dogfood.sh}"
|
||||
B="$(mktemp -d)"
|
||||
echo "### building cog-arch tests against the real engram runtime sources"
|
||||
cc -std=c11 -O2 -w -I "$RT" -o "$B/test_cognition" \
|
||||
"$T/test_cognition.c" "$RT/engram_cognition.c" "$RT/engram_reason.c" \
|
||||
"$RT/engram_geometry.c" "$RT/engram_store.c" "$RT/engram_vindex.c" -lm
|
||||
cc -std=c11 -O2 -w -I "$RT" -o "$B/test_realstore" \
|
||||
"$T/test_cognition_realstore.c" "$RT/engram_cognition.c" "$RT/engram_reason.c" \
|
||||
"$RT/engram_geometry.c" "$RT/engram_store.c" "$RT/engram_vindex.c" -lm
|
||||
|
||||
echo; echo "### [A] synthetic correspondence-loop (known: Brier 0.028648 -> 0.000586)"
|
||||
"$B/test_cognition" | grep -E "held-Brier|reduction|reboot|monotone|metastab|RESULT" || true
|
||||
|
||||
echo; echo "### [B] reboot-prove Stance on a SCRATCH COPY of this sandbox's real-store clone"
|
||||
SCRATCH="$B/store-clone"; mkdir -p "$SCRATCH"
|
||||
cp -p "$SBX_DATA/neuron.egm" "$SCRATCH/" 2>/dev/null || true
|
||||
cp -p "$SBX_DATA/neuron.wal" "$SCRATCH/" 2>/dev/null || true
|
||||
cp -p "$SBX_DATA/conf" "$SCRATCH/" 2>/dev/null || true
|
||||
cp -p "$SBX_DATA/meta.json" "$SCRATCH/" 2>/dev/null || true
|
||||
"$B/test_realstore" "$SCRATCH" || true
|
||||
rm -rf "$B"
|
||||
@@ -1,836 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# nsbx — the Neuron Sandbox: a reproducible primitive for running experiments and
|
||||
# code changes against the REAL engram runtime on an isolated snapshot of the live
|
||||
# mind, with a gated promote-to-prod path built on the proven rails.
|
||||
#
|
||||
# It WRAPS the real engram binary — it never reimplements any engram logic. The only
|
||||
# prod-touching op is `promote`, which is explicit, gated, and per-use approved.
|
||||
#
|
||||
# Generalises two proven proto-sandboxes:
|
||||
# - the cog-arch build (isolated git worktree + build + clone of live .egm + real C tests)
|
||||
# - the store-fix cutover (secondary soul + launchctl bootout->settle->bootstrap rails)
|
||||
#
|
||||
# Lifecycle: create -> [build] -> run -> validate -> promote(gated) -> destroy
|
||||
#
|
||||
# Rails (always): built offline; NEVER auto-promotes; never touches live :8742/:7770
|
||||
# except READ for the snapshot and the gated promote; snapshot-first; honest measured
|
||||
# reporting. Cutover is launchctl bootout -> settle-poll -> bootstrap ONLY —
|
||||
# never pkill, never kickstart -k.
|
||||
set -uo pipefail
|
||||
|
||||
# ---------------------------------------------------------------- constants ----
|
||||
LIVE_DATA_DIR="${ENGRAM_LIVE_DATA_DIR:-$HOME/.neuron/engram}"
|
||||
LIVE_PLIST="${ENGRAM_LIVE_PLIST:-$HOME/Library/LaunchAgents/ai.neuron.engram.plist}"
|
||||
LIVE_LABEL="ai.neuron.engram"
|
||||
LIVE_BIND_PORT=8742 # engram — FORBIDDEN for sandboxes
|
||||
SOUL_PORT=7770 # soul — FORBIDDEN for sandboxes
|
||||
LIVE_KEY="${ENGRAM_API_KEY:-ntn-user-2026}"
|
||||
LIVE_URL="http://127.0.0.1:${LIVE_BIND_PORT}"
|
||||
SBX_ROOT="${NSBX_ROOT:-$HOME/.neuron/sandboxes}"
|
||||
BACKUP_ROOT="$HOME/.neuron/backups"
|
||||
EL_REPO="${EL_REPO:-$HOME/Development/neuron-technologies/foundation/el}"
|
||||
PORT_BASE="${NSBX_PORT_BASE:-8900}"
|
||||
RSS_BOUND_MB="${NSBX_RSS_BOUND_MB:-550}" # from store-fix reboot-proof (aaf13f88)
|
||||
REMERGE_THRESHOLD="${NSBX_REMERGE_THRESHOLD:-40000}"
|
||||
KEYSTONES=( "kn-efeb4a5b-5aff-4759-8a97-7233099be6ee" "kn-5b606390-a52d-4ca2-8e0e-eba141d13440" )
|
||||
# fixed probe set for retrieval-parity (stable, identity-anchored)
|
||||
PARITY_QUERIES=( "who am I" "self identity core" "engram store durability" "keystone self anchor" "grounding honesty" )
|
||||
|
||||
C_RED=$'\033[31m'; C_GRN=$'\033[32m'; C_YEL=$'\033[33m'; C_DIM=$'\033[2m'; C_BLD=$'\033[1m'; C_0=$'\033[0m'
|
||||
|
||||
# ---------------------------------------------------------------- helpers ------
|
||||
die(){ printf '%serror:%s %s\n' "$C_RED" "$C_0" "$*" >&2; exit 1; }
|
||||
log(){ printf '%s==>%s %s\n' "$C_BLD" "$C_0" "$*" >&2; }
|
||||
info(){ printf ' %s\n' "$*" >&2; }
|
||||
ok(){ printf ' %s%s%s\n' "$C_GRN" "$*" "$C_0" >&2; }
|
||||
warn(){ printf ' %s%s%s\n' "$C_YEL" "$*" "$C_0" >&2; }
|
||||
need(){ command -v "$1" >/dev/null 2>&1 || die "missing dependency: $1"; }
|
||||
now(){ date -u +%Y%m%dT%H%M%SZ; }
|
||||
sha(){ shasum -a 256 "$1" 2>/dev/null | awk '{print $1}'; }
|
||||
epoch(){ python3 -c 'import time;print(time.time())'; }
|
||||
|
||||
sdir(){ printf '%s/%s' "$SBX_ROOT" "$1"; }
|
||||
manifest(){ printf '%s/manifest.json' "$(sdir "$1")"; }
|
||||
mexists(){ [ -f "$(manifest "$1")" ]; }
|
||||
mget(){ # mget <name> <jsonpath>
|
||||
python3 -c "import json,sys; d=json.load(open('$(manifest "$1")')); print(d$2)" 2>/dev/null
|
||||
}
|
||||
|
||||
port_free(){ ! (exec 3<>"/dev/tcp/127.0.0.1/$1") 2>/dev/null; }
|
||||
alloc_port(){
|
||||
local p="$PORT_BASE"
|
||||
while :; do
|
||||
if [ "$p" = "$LIVE_BIND_PORT" ] || [ "$p" = "$SOUL_PORT" ]; then p=$((p+1)); continue; fi
|
||||
if port_free "$p" && ! _port_claimed "$p"; then echo "$p"; return 0; fi
|
||||
p=$((p+1)); [ "$p" -gt 9100 ] && die "no free sandbox port in range"
|
||||
done
|
||||
}
|
||||
_port_claimed(){ # is another sandbox already assigned this port?
|
||||
local p="$1" d
|
||||
for d in "$SBX_ROOT"/*/manifest.json; do
|
||||
[ -f "$d" ] || continue
|
||||
[ "$(python3 -c "import json;print(json.load(open('$d'))['port'])" 2>/dev/null)" = "$p" ] && return 0
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
live_stats(){ curl -s -m5 "$LIVE_URL/api/stats" 2>/dev/null; }
|
||||
api(){ # api <name> <path> [json-body]
|
||||
local name="$1" path="$2" body="${3:-}"
|
||||
local port; port="$(mget "$name" "['port']")"; [ -n "$port" ] || die "unknown sandbox: $name"
|
||||
local url="http://127.0.0.1:${port}${path}"
|
||||
if [ -n "$body" ]; then curl -s -m30 -X POST -H 'Content-Type: application/json' -d "$body" "$url"
|
||||
else curl -s -m30 "$url"; fi
|
||||
}
|
||||
sbx_stats(){ api "$1" "/api/stats"; }
|
||||
stat_field(){ printf '%s' "$1" | sed -n "s/.*\"$2\":\([0-9]*\).*/\1/p"; }
|
||||
|
||||
daemon_pid(){ local f; f="$(sdir "$1")/daemon.pid"; [ -f "$f" ] && cat "$f" || true; }
|
||||
daemon_alive(){ local p; p="$(daemon_pid "$1")"; [ -n "$p" ] && kill -0 "$p" 2>/dev/null; }
|
||||
|
||||
# ---------------------------------------------------------------- elc/build ----
|
||||
find_elc(){
|
||||
command -v elc 2>/dev/null && return 0
|
||||
local arch; arch="$(uname -m)"
|
||||
case "$arch" in
|
||||
arm64) echo "$EL_REPO/lang/dist/platform/elc-darwin-arm64";;
|
||||
x86_64) echo "$EL_REPO/lang/dist/platform/elc-linux-amd64";;
|
||||
*) echo "$EL_REPO/lang/dist/platform/elc";;
|
||||
esac
|
||||
}
|
||||
|
||||
# _build_binary <src_tree> <out_bin> <build_log_dir>
|
||||
# Replicates the proven engram release recipe:
|
||||
# elc engram/src/server.el > engram.c
|
||||
# cc -std=c11 -O2 -I lang/runtime engram.c el_runtime.c engram_*.c -lcurl -lpthread
|
||||
_build_binary(){
|
||||
local src="$1" out="$2" blog="$3"
|
||||
local elc server rt
|
||||
elc="$(find_elc)"; [ -x "$elc" ] || die "elc not found/executable: $elc (set EL_REPO)"
|
||||
server="$src/engram/src/server.el"; rt="$src/lang/runtime"
|
||||
[ -f "$server" ] || die "no engram/src/server.el under source tree: $src"
|
||||
[ -f "$rt/el_runtime.c" ] || die "no lang/runtime/el_runtime.c under source tree: $src (this branch may keep it generated/untracked)"
|
||||
ls "$rt"/engram_*.c >/dev/null 2>&1 || die "no lang/runtime/engram_*.c engine sources under: $src"
|
||||
mkdir -p "$blog"
|
||||
log "build: elc transpile server.el -> engram.c"
|
||||
"$elc" "$server" > "$blog/engram.c" 2>"$blog/elc.err" || { cat "$blog/elc.err" >&2; die "elc transpile failed"; }
|
||||
info "engram.c: $(wc -c <"$blog/engram.c" | tr -d ' ') bytes"
|
||||
log "build: cc link (el_runtime + engram_* engine)"
|
||||
cc -std=c11 -O2 -w -I "$rt" -o "$out" \
|
||||
"$blog/engram.c" "$rt/el_runtime.c" "$rt"/engram_*.c \
|
||||
-lcurl -lpthread 2>"$blog/cc.err" \
|
||||
|| { grep -i 'error:' "$blog/cc.err" | sort -u | head >&2; die "cc link failed (see $blog/cc.err)"; }
|
||||
ok "built: $out ($(ls -lh "$out" | awk '{print $5}'), sha $(sha "$out" | cut -c1-12))"
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------- daemon -------
|
||||
# start_daemon <name> : boots the sandbox's real engram binary on its isolated
|
||||
# port against its cloned data dir, with the SAME auto-remerge net the live soul
|
||||
# uses (so the sandbox faithfully reaches the live edge population on boot).
|
||||
start_daemon(){
|
||||
local name="$1" d; d="$(sdir "$name")"
|
||||
daemon_alive "$name" && { info "already running (pid $(daemon_pid "$name"))"; return 0; }
|
||||
local port bin data export key
|
||||
port="$(mget "$name" "['port']")"; bin="$d/bin/engram"; data="$d/data"
|
||||
key="sbx-$name"; export="$data/.scan-export.reseed-clean.json"
|
||||
[ -x "$bin" ] || die "sandbox binary missing: $bin"
|
||||
[ "$port" != "$LIVE_BIND_PORT" ] && [ "$port" != "$SOUL_PORT" ] || die "refusing forbidden port $port"
|
||||
[ -f "$data/neuron.egm" ] || die "sandbox has no cloned store: $data/neuron.egm"
|
||||
# HARD guard: never point a sandbox daemon at the live data dir.
|
||||
[ "$(cd "$data" && pwd -P)" != "$(cd "$LIVE_DATA_DIR" && pwd -P)" ] || die "refusing: sandbox data dir resolves to LIVE store"
|
||||
|
||||
log "boot engram on isolated :$port (data=$data)"
|
||||
(
|
||||
ENGRAM_DATA_DIR="$data" ENGRAM_BIND=":$port" ENGRAM_API_KEY="$key" \
|
||||
ENGRAM_STORE=1 ENGRAM_CHRONOCEPTION=1 ENGRAM_SELF_REIFY=1 ENGRAM_GC=1 \
|
||||
ENGRAM_POOL_FRAMES=16384 ENGRAM_WRITE_BARRIER=1 \
|
||||
exec "$bin"
|
||||
) >"$d/logs/daemon.log" 2>&1 &
|
||||
local pid=$!
|
||||
echo "$pid" > "$d/daemon.pid"
|
||||
# readiness poll
|
||||
local url="http://127.0.0.1:$port" i s
|
||||
for i in $(seq 1 30); do
|
||||
s="$(curl -s -m3 "$url/api/stats" 2>/dev/null)"
|
||||
[ -n "$s" ] && break; sleep 0.5
|
||||
done
|
||||
[ -n "$s" ] || { warn "daemon did not become ready (see $d/logs/daemon.log)"; return 1; }
|
||||
ok "ready pid=$pid boot-stats: $s"
|
||||
# auto-remerge net (idempotent): match live edge population if the export is present
|
||||
if [ -f "$export" ]; then
|
||||
local edges; edges="$(stat_field "$s" edge_count)"
|
||||
if [ -n "$edges" ] && [ "$edges" -lt "$REMERGE_THRESHOLD" ]; then
|
||||
log "auto-remerge: booted with $edges edges (< $REMERGE_THRESHOLD) — merging full edge export"
|
||||
local r; r="$(curl -s -m300 -X POST -H 'Content-Type: application/json' \
|
||||
-d "{\"_auth\":\"$key\",\"path\":\"$export\"}" "$url/api/load-merge" 2>/dev/null)"
|
||||
info "remerge resp: ${r:0:120}"
|
||||
ok "post-remerge stats: $(curl -s -m5 "$url/api/stats")"
|
||||
fi
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
# stop_daemon <name> : graceful TERM + settle-poll until the port is free.
|
||||
# (Sandbox daemons are plain supervised bg processes — not launchd — so teardown
|
||||
# is a signal + poll, never pkill of anything else.)
|
||||
stop_daemon(){
|
||||
local name="$1" pid port
|
||||
pid="$(daemon_pid "$name")"; port="$(mget "$name" "['port']")"
|
||||
[ -n "$pid" ] || { info "not running"; return 0; }
|
||||
log "stop daemon pid=$pid, settle-poll until :$port frees"
|
||||
kill "$pid" 2>/dev/null || true
|
||||
local i
|
||||
for i in $(seq 1 40); do
|
||||
kill -0 "$pid" 2>/dev/null || { port_free "$port" && { ok "stopped, port $port free"; : >"$(sdir "$name")/daemon.pid"; return 0; }; }
|
||||
printf '.' >&2; sleep 0.5
|
||||
done
|
||||
printf '\n' >&2
|
||||
kill -9 "$pid" 2>/dev/null || true; sleep 1
|
||||
: >"$(sdir "$name")/daemon.pid"
|
||||
port_free "$port" && ok "stopped (after SIGKILL), port $port free" || warn "port $port still busy"
|
||||
}
|
||||
|
||||
# ================================================================ create =======
|
||||
cmd_create(){
|
||||
local name="" port="" src="" branch="" repo="$EL_REPO" binpath=""
|
||||
# first positional arg is the name unless it's a flag; default to "<user>-dev"
|
||||
if [ $# -gt 0 ] && [ "${1#-}" = "$1" ]; then name="$1"; shift; else name="${USER:-dev}-dev"; fi
|
||||
while [ $# -gt 0 ]; do case "$1" in
|
||||
--port) port="$2"; shift 2;;
|
||||
--source) src="$2"; shift 2;;
|
||||
--branch) branch="$2"; shift 2;;
|
||||
--repo) repo="$2"; shift 2;;
|
||||
--binary) binpath="$2"; shift 2;;
|
||||
*) die "unknown flag: $1";;
|
||||
esac; done
|
||||
mexists "$name" && die "sandbox '$name' already exists (destroy it first)"
|
||||
need curl; need python3; need shasum
|
||||
[ -f "$LIVE_DATA_DIR/neuron.egm" ] || die "live store not found: $LIVE_DATA_DIR/neuron.egm"
|
||||
if [ -n "$port" ]; then
|
||||
{ [ "$port" = "$LIVE_BIND_PORT" ] || [ "$port" = "$SOUL_PORT" ]; } && die "refusing forbidden port $port (live)"
|
||||
port_free "$port" || die "port $port already in use"
|
||||
else port="$(alloc_port)"; fi
|
||||
|
||||
local d; d="$(sdir "$name")"
|
||||
mkdir -p "$d/data" "$d/bin" "$d/logs" "$d/build" "$d/baseline"
|
||||
log "sandbox '$name' at $d (isolated port $port)"
|
||||
|
||||
# ---- CONSISTENT snapshot of the live mind (file-copy: same set the rails backup
|
||||
# uses; WAL replay on sandbox boot reconciles the tail -> crash-consistent) ----
|
||||
log "snapshot live store -> clone (store + WAL + config)"
|
||||
local f
|
||||
for f in neuron.egm neuron.wal conf meta.json self_anchor .scan-export.reseed-clean.json; do
|
||||
if [ -e "$LIVE_DATA_DIR/$f" ]; then cp -p "$LIVE_DATA_DIR/$f" "$d/data/$f"; info "cloned $f ($(du -h "$d/data/$f" | awk '{print $1}'))"; fi
|
||||
done
|
||||
local egm_sha; egm_sha="$(sha "$d/data/neuron.egm")"
|
||||
|
||||
# ---- capture live baseline (READ only) ----
|
||||
local lstats; lstats="$(live_stats)"
|
||||
local base_nodes base_edges
|
||||
base_nodes="$(stat_field "$lstats" node_count)"; base_edges="$(stat_field "$lstats" edge_count)"
|
||||
info "live baseline stats: ${lstats:-<unavailable>}"
|
||||
|
||||
# ---- determine + place the runtime binary (versioned into the snapshot) ----
|
||||
local source_desc live_bin
|
||||
live_bin="$(_live_real_bin)"
|
||||
if [ -n "$binpath" ]; then
|
||||
[ -x "$binpath" ] || die "not an executable binary: $binpath"
|
||||
cp -p "$binpath" "$d/bin/engram"; source_desc="prebuilt:$binpath"
|
||||
elif [ -n "$src" ]; then
|
||||
_build_binary "$src" "$d/bin/engram" "$d/build"; source_desc="source:$src"
|
||||
elif [ -n "$branch" ]; then
|
||||
log "worktree: $repo @ $branch -> $d/build/worktree"
|
||||
git -C "$repo" worktree add --detach "$d/build/worktree" "$branch" >/dev/null 2>&1 \
|
||||
|| die "git worktree add failed ($repo @ $branch)"
|
||||
_build_binary "$d/build/worktree" "$d/bin/engram" "$d/build"; source_desc="branch:$branch@$repo"
|
||||
else
|
||||
[ -x "$live_bin" ] || die "cannot resolve live ENGRAM_REAL_BIN: $live_bin"
|
||||
cp -p "$live_bin" "$d/bin/engram"; source_desc="stock-prod:$live_bin"
|
||||
fi
|
||||
local bin_sha; bin_sha="$(sha "$d/bin/engram")"
|
||||
info "runtime: $source_desc (sha ${bin_sha:0:12})"
|
||||
|
||||
# ---- write manifest ----
|
||||
python3 - "$name" "$port" "$source_desc" "$bin_sha" "$egm_sha" "$base_nodes" "$base_edges" "$(sha "$live_bin" 2>/dev/null)" <<'PY' > "$(manifest "$name")"
|
||||
import json,sys,datetime
|
||||
name,port,src,binsha,egmsha,bn,be,livebinsha=sys.argv[1:9]
|
||||
json.dump({
|
||||
"name":name,"port":int(port),"created_at":datetime.datetime.now(datetime.timezone.utc).isoformat(),
|
||||
"source":src,"binary_sha256":binsha,"clone_egm_sha256":egmsha,
|
||||
"live_binary_sha256":livebinsha,
|
||||
"live_baseline":{"node_count":int(bn or 0),"edge_count":int(be or 0)},
|
||||
"keystones":["kn-efeb4a5b-5aff-4759-8a97-7233099be6ee","kn-5b606390-a52d-4ca2-8e0e-eba141d13440"]
|
||||
}, sys.stdout, indent=2)
|
||||
PY
|
||||
ok "manifest written"
|
||||
|
||||
# ---- boot + capture the sandbox's own settled baseline (reproducible target) ----
|
||||
start_daemon "$name" || die "daemon failed to start"
|
||||
local sstats; sstats="$(sbx_stats "$name")"
|
||||
local sbn sbe; sbn="$(stat_field "$sstats" node_count)"; sbe="$(stat_field "$sstats" edge_count)"
|
||||
_capture_retrieval "$name" "$d/baseline/retrieval.json"
|
||||
# fold sandbox baseline into manifest
|
||||
python3 - "$(manifest "$name")" "$sbn" "$sbe" <<'PY'
|
||||
import json,sys
|
||||
mf,bn,be=sys.argv[1],sys.argv[2],sys.argv[3]
|
||||
d=json.load(open(mf)); d["sbx_baseline"]={"node_count":int(bn or 0),"edge_count":int(be or 0)}
|
||||
json.dump(d,open(mf,'w'),indent=2)
|
||||
PY
|
||||
log "created."
|
||||
info "sandbox baseline (settled): nodes=$sbn edges=$sbe"
|
||||
info "next: nsbx validate $name | nsbx run $name api /api/stats"
|
||||
}
|
||||
|
||||
_live_real_bin(){
|
||||
python3 - "$LIVE_PLIST" <<'PY' 2>/dev/null
|
||||
import sys,plistlib
|
||||
try:
|
||||
d=plistlib.load(open(sys.argv[1],'rb'))
|
||||
print(d.get("EnvironmentVariables",{}).get("ENGRAM_REAL_BIN",""))
|
||||
except Exception: print("")
|
||||
PY
|
||||
}
|
||||
|
||||
_capture_retrieval(){ # <name> <outfile> : top-k ids for the fixed probe set
|
||||
local name="$1" out="$2" q res
|
||||
local port; port="$(mget "$name" "['port']")"; local key="sbx-$name"
|
||||
{
|
||||
echo "{"
|
||||
local first=1
|
||||
for q in "${PARITY_QUERIES[@]}"; do
|
||||
res="$(curl -s -m10 -X POST -H 'Content-Type: application/json' \
|
||||
-d "{\"_auth\":\"$key\",\"query\":\"$q\",\"limit\":5}" "http://127.0.0.1:$port/api/search" 2>/dev/null)"
|
||||
local ids; ids="$(printf '%s' "$res" | python3 -c 'import sys,json
|
||||
try:
|
||||
d=json.load(sys.stdin)
|
||||
rows=d if isinstance(d,list) else d.get("results",d.get("hits",[]))
|
||||
print(json.dumps([r.get("id") for r in rows][:5]))
|
||||
except Exception: print("[]")' 2>/dev/null)"
|
||||
[ $first -eq 1 ] || echo ","; first=0
|
||||
printf ' %s: %s' "$(python3 -c "import json,sys;print(json.dumps(sys.argv[1]))" "$q")" "${ids:-[]}"
|
||||
done
|
||||
echo ""; echo "}"
|
||||
} > "$out"
|
||||
}
|
||||
|
||||
# ================================================================ up ===========
|
||||
# Dead-simple one-command dev environment: `nsbx up` gives you (or Tim, or anyone)
|
||||
# a private, isolated copy of the live mind to build against. Creates it on first
|
||||
# run with sane defaults (stock prod binary, auto-allocated port), just starts it
|
||||
# thereafter. Prod on :$LIVE_BIND_PORT/:$SOUL_PORT is unreachable from here by design.
|
||||
cmd_up(){
|
||||
local name; if [ $# -gt 0 ] && [ "${1#-}" = "$1" ]; then name="$1"; shift; else name="${USER:-dev}-dev"; fi
|
||||
if mexists "$name"; then daemon_alive "$name" || start_daemon "$name"; else cmd_create "$name" "$@"; fi
|
||||
local port; port="$(mget "$name" "['port']")"
|
||||
echo >&2
|
||||
ok "your sandbox '$name' is ready at http://127.0.0.1:$port (a private copy of the mind — prod is untouchable)"
|
||||
info "experiment: nsbx run $name api /api/stats"
|
||||
info "prove it: nsbx validate $name"
|
||||
info "tear down: nsbx destroy $name"
|
||||
}
|
||||
|
||||
# ================================================================ build ========
|
||||
# Rebuild an existing sandbox's runtime from a source tree/branch and hot-restart
|
||||
# it on the SAME clone + port (the code-change dev loop, in place).
|
||||
cmd_build(){
|
||||
local name="$1"; shift || true
|
||||
mexists "$name" || die "no such sandbox: $name"
|
||||
local src="" branch="" repo="$EL_REPO"
|
||||
while [ $# -gt 0 ]; do case "$1" in
|
||||
--source) src="$2"; shift 2;; --branch) branch="$2"; shift 2;; --repo) repo="$2"; shift 2;;
|
||||
*) die "unknown flag: $1";; esac; done
|
||||
local d; d="$(sdir "$name")"
|
||||
stop_daemon "$name"
|
||||
if [ -n "$src" ]; then _build_binary "$src" "$d/bin/engram" "$d/build"
|
||||
elif [ -n "$branch" ]; then
|
||||
rm -rf "$d/build/worktree" 2>/dev/null; git -C "$repo" worktree prune 2>/dev/null
|
||||
git -C "$repo" worktree add --detach "$d/build/worktree" "$branch" >/dev/null 2>&1 || die "worktree add failed"
|
||||
_build_binary "$d/build/worktree" "$d/bin/engram" "$d/build"
|
||||
else die "usage: nsbx build <name> --source DIR | --branch REF [--repo R]"; fi
|
||||
# record new binary sha
|
||||
python3 - "$(manifest "$name")" "$(sha "$d/bin/engram")" "${src:-branch:$branch}" <<'PY'
|
||||
import json,sys; mf,s,src=sys.argv[1:4]
|
||||
d=json.load(open(mf)); d["binary_sha256"]=s; d["source"]="rebuilt:"+src
|
||||
json.dump(d,open(mf,'w'),indent=2)
|
||||
PY
|
||||
start_daemon "$name"
|
||||
ok "rebuilt + restarted on :$(mget "$name" "['port']")"
|
||||
}
|
||||
|
||||
# ================================================================ run ==========
|
||||
cmd_run(){
|
||||
local name="$1"; shift || true
|
||||
mexists "$name" || die "no such sandbox: $name"
|
||||
daemon_alive "$name" || start_daemon "$name"
|
||||
local d port; d="$(sdir "$name")"; port="$(mget "$name" "['port']")"
|
||||
# direct API form: nsbx run <name> api <path> [json]
|
||||
if [ "${1:-}" = "api" ]; then
|
||||
api "$name" "$2" "${3:-}"; echo; return 0
|
||||
fi
|
||||
[ "${1:-}" = "--" ] && shift # allow an explicit separator: nsbx run <name> -- <cmd...>
|
||||
[ $# -gt 0 ] || die "usage: nsbx run <name> <cmd...> | nsbx run <name> api <path> [json]"
|
||||
local ts log0; ts="$(now)"; log0="$d/logs/run-$ts.log"
|
||||
local s0 t0 t1 s1
|
||||
s0="$(sbx_stats "$name")"; t0="$(epoch)"
|
||||
log "run experiment against sandbox '$name' (:$port)"
|
||||
info "cmd: $*"
|
||||
( export SBX_NAME="$name" SBX_PORT="$port" SBX_URL="http://127.0.0.1:$port" \
|
||||
SBX_KEY="sbx-$name" SBX_DATA="$d/data" SBX_BIN="$d/bin/engram"
|
||||
"$@" ) 2>&1 | tee "$log0"
|
||||
local rc=${PIPESTATUS[0]}
|
||||
t1="$(epoch)"; s1="$(sbx_stats "$name")"
|
||||
{
|
||||
echo "--- nsbx run metrics ---"
|
||||
echo "exit_code: $rc"
|
||||
printf 'wall_secs: %.3f\n' "$(python3 -c "print($t1-$t0)")"
|
||||
echo "stats_before: $s0"
|
||||
echo "stats_after: $s1"
|
||||
} | tee -a "$log0" >&2
|
||||
return $rc
|
||||
}
|
||||
|
||||
# ================================================================ validate =====
|
||||
# The rails as first-class checks. Baseline = the sandbox's own settled state at
|
||||
# create (reproducible). zero-loss through sustained load AND reboot; reboot-prove;
|
||||
# RSS bound; retrieval parity; keystone integrity.
|
||||
cmd_validate(){
|
||||
local name="$1"; shift || true
|
||||
mexists "$name" || die "no such sandbox: $name"
|
||||
daemon_alive "$name" || start_daemon "$name"
|
||||
local d port key; d="$(sdir "$name")"; port="$(mget "$name" "['port']")"; key="sbx-$name"
|
||||
local url="http://127.0.0.1:$port"
|
||||
local bn be; bn="$(mget "$name" "['sbx_baseline']['node_count']")"; be="$(mget "$name" "['sbx_baseline']['edge_count']")"
|
||||
log "validate '$name' against baseline nodes=$bn edges=$be"
|
||||
local -a names=() results=() details=()
|
||||
|
||||
# 1) sustained load — no data loss under activity
|
||||
local s cur_n cur_e i
|
||||
log "check: sustained load (~15s: tick + reads) then zero-loss"
|
||||
for i in $(seq 1 15); do
|
||||
curl -s -m5 -X POST -H 'Content-Type: application/json' -d "{\"_auth\":\"$key\"}" "$url/api/tick" >/dev/null 2>&1
|
||||
curl -s -m5 "$url/api/stats" >/dev/null 2>&1
|
||||
done
|
||||
s="$(sbx_stats "$name")"; cur_n="$(stat_field "$s" node_count)"; cur_e="$(stat_field "$s" edge_count)"
|
||||
names+=("zero-loss-under-load"); if [ "${cur_n:-0}" -ge "${bn:-0}" ] && [ "${cur_e:-0}" -ge "${be:-0}" ]; then
|
||||
results+=("PASS"); else results+=("FAIL"); fi
|
||||
details+=("nodes $cur_n>=$bn, edges $cur_e>=$be")
|
||||
|
||||
# 2) reboot-prove — counts survive a real restart
|
||||
log "check: reboot-prove (stop -> start -> compare)"
|
||||
local pre_n pre_e; pre_n="$cur_n"; pre_e="$cur_e"
|
||||
stop_daemon "$name"; start_daemon "$name" >/dev/null
|
||||
s="$(sbx_stats "$name")"; cur_n="$(stat_field "$s" node_count)"; cur_e="$(stat_field "$s" edge_count)"
|
||||
names+=("reboot-prove"); if [ "${cur_n:-0}" -ge "${bn:-0}" ] && [ "${cur_e:-0}" -ge "${be:-0}" ]; then
|
||||
results+=("PASS"); else results+=("FAIL"); fi
|
||||
details+=("post-reboot nodes=$cur_n edges=$cur_e (pre $pre_n/$pre_e)")
|
||||
|
||||
# 3) RSS bound
|
||||
log "check: RSS bound (< ${RSS_BOUND_MB}MB)"
|
||||
local pid rss_kb rss_mb; pid="$(daemon_pid "$name")"
|
||||
rss_kb="$(ps -o rss= -p "$pid" 2>/dev/null | tr -d ' ')"; rss_mb=$(( ${rss_kb:-0} / 1024 ))
|
||||
names+=("rss-bound"); if [ "$rss_mb" -lt "$RSS_BOUND_MB" ] && [ "$rss_mb" -gt 0 ]; then results+=("PASS"); else results+=("FAIL"); fi
|
||||
details+=("RSS=${rss_mb}MB (bound ${RSS_BOUND_MB}MB)")
|
||||
|
||||
# 4) retrieval parity vs the create-time baseline
|
||||
log "check: retrieval parity vs baseline probe set"
|
||||
_capture_retrieval "$name" "$d/logs/retrieval-$( now ).json"
|
||||
local latest; latest="$(ls -t "$d/logs"/retrieval-*.json 2>/dev/null | head -1)"
|
||||
local parity; parity="$(python3 - "$d/baseline/retrieval.json" "$latest" <<'PY'
|
||||
import json,sys
|
||||
def load(p):
|
||||
try: return json.load(open(p))
|
||||
except Exception: return {}
|
||||
b,c=load(sys.argv[1]),load(sys.argv[2])
|
||||
tot=hit=0
|
||||
for q,ids in b.items():
|
||||
cb=set(ids or []); cc=set(c.get(q) or [])
|
||||
if not cb: continue
|
||||
tot+=len(cb); hit+=len(cb & cc)
|
||||
print(f"{hit}/{tot}" if tot else "0/0")
|
||||
PY
|
||||
)"
|
||||
local ph="${parity%/*}" pt="${parity#*/}"
|
||||
names+=("retrieval-parity"); if [ "${pt:-0}" -gt 0 ] && [ "${ph:-0}" -eq "${pt:-0}" ]; then results+=("PASS"); else results+=("FAIL"); fi
|
||||
details+=("top-k id overlap $parity vs baseline")
|
||||
|
||||
# 5) keystone integrity
|
||||
log "check: keystone integrity"
|
||||
local kfail=0 kid kres
|
||||
for kid in "${KEYSTONES[@]}"; do
|
||||
kres="$(curl -s -m5 "$url/api/node/$kid" 2>/dev/null)"
|
||||
printf '%s' "$kres" | grep -q "\"$kid\"" || kfail=1
|
||||
done
|
||||
names+=("keystone-integrity"); [ "$kfail" -eq 0 ] && results+=("PASS") || results+=("FAIL")
|
||||
details+=("kn-efeb4a5b + kn-5b606390 present")
|
||||
|
||||
# ---- report + stamp ----
|
||||
echo >&2
|
||||
printf '%s VALIDATION — %s%s\n' "$C_BLD" "$name" "$C_0" >&2
|
||||
local allpass=1 j
|
||||
for j in "${!names[@]}"; do
|
||||
local r="${results[$j]}" c="$C_GRN"; [ "$r" = FAIL ] && { c="$C_RED"; allpass=0; }
|
||||
printf ' %s%-6s%s %-22s %s%s%s\n' "$c" "$r" "$C_0" "${names[$j]}" "$C_DIM" "${details[$j]}" "$C_0" >&2
|
||||
done
|
||||
local status; [ "$allpass" -eq 1 ] && status="PASS" || status="FAIL"
|
||||
python3 - "$d/validate.json" "$status" "$(sha "$d/bin/engram")" "$(now)" "${names[*]}" "${results[*]}" <<'PY'
|
||||
import json,sys
|
||||
out,status,binsha,ts,ns,rs=sys.argv[1:7]
|
||||
checks=[{"name":n,"result":r} for n,r in zip(ns.split(),rs.split())]
|
||||
json.dump({"status":status,"binary_sha256":binsha,"ts":ts,"checks":checks},open(out,'w'),indent=2)
|
||||
PY
|
||||
printf ' %s==> %s%s\n' "$([ "$allpass" -eq 1 ] && echo "$C_GRN" || echo "$C_RED")" "$status" "$C_0" >&2
|
||||
[ "$allpass" -eq 1 ]
|
||||
}
|
||||
|
||||
# ================================================================ promote ======
|
||||
# The ONLY prod-touching op. Explicit, gated, per-use Will-approved. Rails ONLY:
|
||||
# snapshot-first -> additive binary swap -> launchctl bootout -> settle-poll ->
|
||||
# bootstrap -> verify -> auto-rollback on failure. NEVER pkill, NEVER kickstart -k.
|
||||
# Default is a DRY-RUN plan; requires --i-approve-prod-cutover to actually cut over.
|
||||
cmd_promote(){
|
||||
local name="$1"; shift || true
|
||||
mexists "$name" || die "no such sandbox: $name"
|
||||
local approve=0 do_data=0
|
||||
while [ $# -gt 0 ]; do case "$1" in
|
||||
--i-approve-prod-cutover) approve=1; shift;;
|
||||
--data) do_data=1; shift;;
|
||||
*) die "unknown flag: $1";; esac; done
|
||||
local d; d="$(sdir "$name")"
|
||||
# GATE 1: validation must have passed for the CURRENT binary
|
||||
[ -f "$d/validate.json" ] || die "GATE: no validation on record — run 'nsbx validate $name' first"
|
||||
local vstatus vsha bsha
|
||||
vstatus="$(python3 -c "import json;print(json.load(open('$d/validate.json'))['status'])")"
|
||||
vsha="$(python3 -c "import json;print(json.load(open('$d/validate.json'))['binary_sha256'])")"
|
||||
bsha="$(sha "$d/bin/engram")"
|
||||
[ "$vstatus" = PASS ] || die "GATE: last validation status is $vstatus (must be PASS)"
|
||||
[ "$vsha" = "$bsha" ] || die "GATE: validation is stale — binary changed since validate (re-run validate)"
|
||||
|
||||
local live_bin new_bin ts; ts="$(now)"
|
||||
live_bin="$(_live_real_bin)"
|
||||
new_bin="$HOME/.neuron/bin/engram.promote-$name-$ts" # additive: new file, old kept
|
||||
local bkp="$BACKUP_ROOT/promote-$name-$ts"
|
||||
|
||||
log "PROMOTE PLAN for '$name' -> live :$LIVE_BIND_PORT"
|
||||
info "current live ENGRAM_REAL_BIN : $live_bin"
|
||||
info "sandbox binary (validated) : $d/bin/engram (sha ${bsha:0:12})"
|
||||
info "will install as : $new_bin (additive; old binary retained)"
|
||||
info "snapshot-first backup dir : $bkp (egm+wal+plist+rollback.txt)"
|
||||
info "data promote : $([ $do_data -eq 1 ] && echo 'YES (--data: clone egm/wal -> live)' || echo 'no (binary only)')"
|
||||
info "rails : launchctl bootout -> settle-poll -> bootstrap"
|
||||
info "verify : /api/stats + edges>=baseline + keystones + retrieval; auto-rollback armed"
|
||||
|
||||
if [ "$approve" -ne 1 ]; then
|
||||
warn "DRY-RUN — not touching prod. Re-run with --i-approve-prod-cutover to execute (per-use Will-approved)."
|
||||
return 0
|
||||
fi
|
||||
|
||||
need launchctl
|
||||
local dom="gui/$(id -u)"
|
||||
# ---- snapshot-first ----
|
||||
log "snapshot-first backup -> $bkp"
|
||||
mkdir -p "$bkp"
|
||||
cp -p "$LIVE_DATA_DIR/neuron.egm" "$bkp/neuron.egm.bak"
|
||||
cp -p "$LIVE_DATA_DIR/neuron.wal" "$bkp/neuron.wal.bak" 2>/dev/null || true
|
||||
cp -p "$LIVE_PLIST" "$bkp/plist.bak"
|
||||
printf 'rollback REAL_BIN=%s\nNEWBIN=%s\ndata_promote=%s\n' "$live_bin" "$new_bin" "$do_data" > "$bkp/rollback.txt"
|
||||
ok "backup complete"
|
||||
|
||||
# ---- additive binary install + plist supersede ----
|
||||
cp -p "$d/bin/engram" "$new_bin"
|
||||
python3 - "$LIVE_PLIST" "$new_bin" <<'PY'
|
||||
import sys,plistlib
|
||||
p,new=sys.argv[1],sys.argv[2]
|
||||
d=plistlib.load(open(p,'rb')); d.setdefault("EnvironmentVariables",{})["ENGRAM_REAL_BIN"]=new
|
||||
plistlib.dump(d,open(p,'wb'))
|
||||
PY
|
||||
ok "installed $new_bin + updated plist ENGRAM_REAL_BIN"
|
||||
|
||||
# ---- optional data promote (after backup) ----
|
||||
if [ $do_data -eq 1 ]; then
|
||||
log "data promote: clone store -> live (backed up above)"
|
||||
cp -p "$d/data/neuron.egm" "$LIVE_DATA_DIR/neuron.egm"
|
||||
cp -p "$d/data/neuron.wal" "$LIVE_DATA_DIR/neuron.wal" 2>/dev/null || true
|
||||
fi
|
||||
|
||||
# ---- rails cutover: bootout -> settle-poll -> bootstrap ----
|
||||
log "rails: launchctl bootout $dom/$LIVE_LABEL"
|
||||
launchctl bootout "$dom/$LIVE_LABEL" 2>/dev/null || true
|
||||
local i
|
||||
for i in $(seq 1 60); do
|
||||
launchctl print "$dom/$LIVE_LABEL" >/dev/null 2>&1 || { ok "settle: job gone after ${i}x0.5s"; break; }
|
||||
printf ' settle: job still present (%d)\n' "$i" >&2; sleep 0.5
|
||||
done
|
||||
log "rails: launchctl bootstrap $dom <plist>"
|
||||
launchctl bootstrap "$dom" "$LIVE_PLIST" || warn "bootstrap returned nonzero"
|
||||
|
||||
# ---- verify ----
|
||||
log "verify prod health"
|
||||
local s="" ; for i in $(seq 1 60); do s="$(live_stats)"; [ -n "$s" ] && break; sleep 1; done
|
||||
local ok_verify=1 le; le="$(stat_field "$s" edge_count)"
|
||||
local base_e; base_e="$(mget "$name" "['live_baseline']['edge_count']")"
|
||||
[ -n "$s" ] || ok_verify=0
|
||||
[ "${le:-0}" -ge "${base_e:-0}" ] || ok_verify=0
|
||||
local kid; for kid in "${KEYSTONES[@]}"; do curl -s -m5 "$LIVE_URL/api/node/$kid" 2>/dev/null | grep -q "\"$kid\"" || ok_verify=0; done
|
||||
if [ "$ok_verify" -eq 1 ]; then
|
||||
ok "PROMOTED. live stats: $s (rollback: $bkp)"; return 0
|
||||
fi
|
||||
|
||||
# ---- auto-rollback ----
|
||||
warn "verify FAILED — auto-rollback"
|
||||
cp -p "$bkp/plist.bak" "$LIVE_PLIST"
|
||||
[ $do_data -eq 1 ] && { cp -p "$bkp/neuron.egm.bak" "$LIVE_DATA_DIR/neuron.egm"; cp -p "$bkp/neuron.wal.bak" "$LIVE_DATA_DIR/neuron.wal" 2>/dev/null || true; }
|
||||
launchctl bootout "$dom/$LIVE_LABEL" 2>/dev/null || true
|
||||
for i in $(seq 1 60); do launchctl print "$dom/$LIVE_LABEL" >/dev/null 2>&1 || break; sleep 0.5; done
|
||||
launchctl bootstrap "$dom" "$LIVE_PLIST" || true
|
||||
die "ROLLED BACK to $live_bin. See $bkp"
|
||||
}
|
||||
|
||||
# ================================================================ destroy ======
|
||||
cmd_destroy(){
|
||||
local name="$1"; shift || true
|
||||
mexists "$name" || die "no such sandbox: $name"
|
||||
local d; d="$(sdir "$name")"
|
||||
stop_daemon "$name"
|
||||
if [ -d "$d/build/worktree" ]; then
|
||||
log "removing git worktree"
|
||||
git -C "$EL_REPO" worktree remove --force "$d/build/worktree" 2>/dev/null || true
|
||||
git -C "$EL_REPO" worktree prune 2>/dev/null || true
|
||||
fi
|
||||
log "removing $d"
|
||||
rm -rf "$d"
|
||||
ok "destroyed '$name' (live untouched)"
|
||||
}
|
||||
|
||||
# ================================================================ list/status ==
|
||||
cmd_list(){
|
||||
[ -d "$SBX_ROOT" ] || { echo "no sandboxes"; return 0; }
|
||||
printf '%-16s %-6s %-8s %-9s %s\n' NAME PORT STATE PID SOURCE
|
||||
local m
|
||||
for m in "$SBX_ROOT"/*/manifest.json; do
|
||||
[ -f "$m" ] || continue
|
||||
local n p src pid state
|
||||
n="$(python3 -c "import json;print(json.load(open('$m'))['name'])")"
|
||||
p="$(python3 -c "import json;print(json.load(open('$m'))['port'])")"
|
||||
src="$(python3 -c "import json;print(json.load(open('$m'))['source'])")"
|
||||
pid="$(daemon_pid "$n")"; state="stopped"; daemon_alive "$n" && state="running"
|
||||
printf '%-16s %-6s %-8s %-9s %s\n' "$n" "$p" "$state" "${pid:-–}" "$src"
|
||||
done
|
||||
}
|
||||
cmd_status(){
|
||||
local name="$1"; mexists "$name" || die "no such sandbox: $name"
|
||||
python3 -m json.tool "$(manifest "$name")"
|
||||
daemon_alive "$name" && echo "state: running (pid $(daemon_pid "$name")) stats: $(sbx_stats "$name")" || echo "state: stopped"
|
||||
[ -f "$(sdir "$name")/validate.json" ] && { echo "--- last validation ---"; python3 -m json.tool "$(sdir "$name")/validate.json"; }
|
||||
}
|
||||
|
||||
# ================================================================ dev ==========
|
||||
# ONE-COMMAND isolated dev environment. Everything a newcomer (Tim, any agent)
|
||||
# needs to go from clone -> coding on an isolated running mind, in a single shot:
|
||||
# 1) a git BRANCH (dev/<name>, or --prefix)
|
||||
# 2) a git WORKTREE for it, at a visible path they can open + edit
|
||||
# 3) an ISOLATED engram bound to a NON-default port, on a clone of the live store
|
||||
# (stock prod binary by default — instant + safe; --build to compile the
|
||||
# worktree's own runtime instead). Prod :$LIVE_BIND_PORT/:$SOUL_PORT is untouchable.
|
||||
#
|
||||
# This is additive sugar over the proven primitives (git worktree + cmd_create).
|
||||
# It never binds a forbidden port and never touches ~/.neuron/engram (the live store)
|
||||
# except the same READ-only snapshot cmd_create already performs.
|
||||
#
|
||||
# nsbx dev <name> [--repo R] [--base REF] [--worktree DIR] [--port N]
|
||||
# [--prefix P] [--build] [--no-engram]
|
||||
cmd_dev(){
|
||||
local name="" repo="$EL_REPO" base="" wt="" port="" prefix="dev/" build=0 no_engram=0
|
||||
[ $# -gt 0 ] && [ "${1#-}" = "$1" ] && { name="$1"; shift; } || die "usage: nsbx dev <name> [flags]"
|
||||
while [ $# -gt 0 ]; do case "$1" in
|
||||
--repo) repo="$2"; shift 2;;
|
||||
--base) base="$2"; shift 2;;
|
||||
--worktree|--wt) wt="$2"; shift 2;;
|
||||
--port) port="$2"; shift 2;;
|
||||
--prefix) prefix="$2"; shift 2;;
|
||||
--build) build=1; shift;;
|
||||
--no-engram) no_engram=1; shift;;
|
||||
*) die "unknown flag: $1";;
|
||||
esac; done
|
||||
need git
|
||||
git -C "$repo" rev-parse --git-dir >/dev/null 2>&1 || die "not a git repo: $repo"
|
||||
|
||||
local branch="${prefix}${name}"
|
||||
local sbx="dev-${name}"
|
||||
# default worktree path: a PERSISTENT, git-managed dir — NEVER /tmp (which is
|
||||
# ablated on compaction). Default root = <repo-grandparent>/el-worktrees, i.e.
|
||||
# ~/Development/neuron-technologies/el-worktrees/<name>. Override with NSBX_DEV_WT_ROOT.
|
||||
local wt_root="${NSBX_DEV_WT_ROOT:-$(cd "$(dirname "$(dirname "$repo")")" && pwd -P)/el-worktrees}"
|
||||
[ -n "$wt" ] || wt="${wt_root}/${name}"
|
||||
case "$wt" in /tmp/*|/private/tmp/*|/var/tmp/*)
|
||||
die "refusing worktree under a temp dir ($wt) — temp dirs are ablated on compaction; set NSBX_DEV_WT_ROOT to a persistent path";;
|
||||
esac
|
||||
# default base: whatever the repo's working checkout is on now
|
||||
[ -n "$base" ] || base="$(git -C "$repo" rev-parse --abbrev-ref HEAD 2>/dev/null)"
|
||||
|
||||
# pre-flight (fail before creating anything)
|
||||
[ "$no_engram" -eq 1 ] || ! mexists "$sbx" || die "engram sandbox '$sbx' already exists (nsbx dev-down $name first)"
|
||||
[ -e "$wt" ] && die "worktree path already exists: $wt"
|
||||
if [ -n "$port" ]; then
|
||||
{ [ "$port" = "$LIVE_BIND_PORT" ] || [ "$port" = "$SOUL_PORT" ]; } && die "refusing forbidden port $port (live)"
|
||||
fi
|
||||
|
||||
log "dev env '$name' (branch=$branch worktree=$wt base=$base)"
|
||||
|
||||
# ---- 1+2) branch + worktree in one shot ----
|
||||
local gerr
|
||||
if git -C "$repo" show-ref --verify --quiet "refs/heads/$branch"; then
|
||||
info "branch $branch exists — checking it out into a new worktree"
|
||||
gerr="$(git -C "$repo" worktree add "$wt" "$branch" 2>&1)" \
|
||||
|| die "git worktree add failed for existing branch $branch:"$'\n'" $gerr"
|
||||
else
|
||||
gerr="$(git -C "$repo" worktree add -b "$branch" "$wt" "$base" 2>&1)" \
|
||||
|| die "git worktree add -b $branch (base $base) failed:"$'\n'" $gerr"$'\n'" (a bare 'dev' branch blocks 'dev/*' names — try --prefix, e.g. nsbx dev $name --prefix wt/)"
|
||||
fi
|
||||
ok "worktree ready: $wt (branch $branch)"
|
||||
|
||||
# ---- 3) isolated engram ----
|
||||
local eport="(none)"
|
||||
if [ "$no_engram" -eq 1 ]; then
|
||||
warn "--no-engram: skipped standing up an engram"
|
||||
else
|
||||
if [ "$build" -eq 1 ]; then
|
||||
log "isolated engram: building the worktree's own runtime"
|
||||
cmd_create "$sbx" ${port:+--port "$port"} --source "$wt" || die "engram create (--build) failed"
|
||||
else
|
||||
log "isolated engram: stock prod binary on a clone of the live store"
|
||||
cmd_create "$sbx" ${port:+--port "$port"} || die "engram create failed"
|
||||
fi
|
||||
eport="$(mget "$sbx" "['port']")"
|
||||
# record the dev linkage next to the sandbox so dev-down can clean up
|
||||
python3 - "$(sdir "$sbx")/dev.json" "$name" "$branch" "$wt" "$repo" "$eport" <<'PY'
|
||||
import json,sys
|
||||
p,name,branch,wt,repo,port=sys.argv[1:7]
|
||||
json.dump({"name":name,"branch":branch,"worktree":wt,"repo":repo,"port":int(port)},
|
||||
open(p,'w'),indent=2)
|
||||
PY
|
||||
# ---- pin the WHOLE worktree to the CLONE ----
|
||||
# Every var any El tooling in this worktree might read for an engram target now
|
||||
# points at the isolated clone. Sourcing .nsbx-env makes hitting live :$LIVE_BIND_PORT
|
||||
# or ~/.neuron/engram by accident structurally impossible from this shell.
|
||||
local edata ekey eurl ebin
|
||||
edata="$(sdir "$sbx")/data"; ekey="sbx-$sbx"; eurl="http://127.0.0.1:$eport"; ebin="$(sdir "$sbx")/bin/engram"
|
||||
cat > "$wt/.nsbx-env" <<ENV
|
||||
# nsbx dev env for '$name' — SOURCE this to pin THIS shell to the isolated clone.
|
||||
# The live mind (:$LIVE_BIND_PORT engram / :$SOUL_PORT soul / $LIVE_DATA_DIR) is deliberately
|
||||
# NOT referenced here. Regenerated by 'nsbx dev'. -> source .nsbx-env
|
||||
export NSBX_NAME="$sbx"
|
||||
export ENGRAM_URL="$eurl"
|
||||
export ENGRAM_BIND=":$eport"
|
||||
export ENGRAM_PORT="$eport"
|
||||
export ENGRAM_HOST="127.0.0.1"
|
||||
export ENGRAM_DATA_DIR="$edata"
|
||||
export ENGRAM_API_KEY="$ekey"
|
||||
export NEURON_ENGRAM_URL="$eurl"
|
||||
export NEURON_ENGRAM_KEY="$ekey"
|
||||
# nsbx run compatibility (same names 'nsbx run' exports)
|
||||
export SBX_NAME="$sbx" SBX_PORT="$eport" SBX_URL="$eurl" SBX_KEY="$ekey" SBX_DATA="$edata" SBX_BIN="$ebin"
|
||||
ENV
|
||||
# direnv users get it automatically on cd; everyone else runs 'source .nsbx-env'
|
||||
[ -e "$wt/.envrc" ] || printf 'source_env .nsbx-env 2>/dev/null || source .nsbx-env\n' > "$wt/.envrc"
|
||||
ok "wrote $wt/.nsbx-env (pins this worktree to the clone)"
|
||||
fi
|
||||
|
||||
# ---- summary ----
|
||||
echo >&2
|
||||
printf '%s DEV ENV READY — %s%s\n' "$C_BLD" "$name" "$C_0" >&2
|
||||
printf ' %-10s %s\n' "worktree" "$wt" >&2
|
||||
printf ' %-10s %s\n' "branch" "$branch" >&2
|
||||
if [ "$no_engram" -ne 1 ]; then
|
||||
printf ' %-10s %s\n' "engram" "http://127.0.0.1:$eport (isolated clone; prod :$LIVE_BIND_PORT untouchable)" >&2
|
||||
printf ' %-10s %s\n' "sandbox" "$sbx" >&2
|
||||
echo >&2
|
||||
printf '%s env exported into %s/.nsbx-env (source it -> pinned to the clone):%s\n' "$C_DIM" "$wt" "$C_0" >&2
|
||||
grep '^export' "$wt/.nsbx-env" | sed 's/^/ /' >&2
|
||||
fi
|
||||
echo >&2
|
||||
info "code in: cd $wt && source .nsbx-env # now every engram var points at the clone"
|
||||
[ "$no_engram" -ne 1 ] && info "poke it: nsbx run $sbx api /api/stats (or: make run NAME=$name)"
|
||||
[ "$no_engram" -ne 1 ] && info "edit->test: make build NAME=$name && make run NAME=$name # El change -> clone, seconds"
|
||||
info "tear down: nsbx dev-down $name (destroys engram + removes worktree; branch kept)"
|
||||
}
|
||||
|
||||
# nsbx dev-down <name> [--delete-branch] [--repo R]
|
||||
# Teardown counterpart: destroy the isolated engram, remove the git worktree,
|
||||
# and (optionally) delete the branch. Live prod is never touched.
|
||||
cmd_dev_down(){
|
||||
local name="" repo="$EL_REPO" del_branch=0
|
||||
[ $# -gt 0 ] && [ "${1#-}" = "$1" ] && { name="$1"; shift; } || die "usage: nsbx dev-down <name> [--delete-branch]"
|
||||
while [ $# -gt 0 ]; do case "$1" in
|
||||
--repo) repo="$2"; shift 2;;
|
||||
--delete-branch) del_branch=1; shift;;
|
||||
*) die "unknown flag: $1";;
|
||||
esac; done
|
||||
local sbx="dev-${name}" wt="" branch="dev/${name}"
|
||||
# recover worktree/branch/repo from the dev linkage if present
|
||||
if mexists "$sbx" && [ -f "$(sdir "$sbx")/dev.json" ]; then
|
||||
local dj; dj="$(sdir "$sbx")/dev.json"
|
||||
wt="$(python3 -c "import json;print(json.load(open('$dj'))['worktree'])" 2>/dev/null)"
|
||||
branch="$(python3 -c "import json;print(json.load(open('$dj'))['branch'])" 2>/dev/null)"
|
||||
repo="$(python3 -c "import json;print(json.load(open('$dj'))['repo'])" 2>/dev/null)"
|
||||
fi
|
||||
# 1) engram
|
||||
if mexists "$sbx"; then cmd_destroy "$sbx"; else info "no engram sandbox '$sbx'"; fi
|
||||
# 2) worktree
|
||||
if [ -n "$wt" ] && [ -d "$wt" ]; then
|
||||
log "removing git worktree $wt"
|
||||
git -C "$repo" worktree remove --force "$wt" 2>/dev/null || rm -rf "$wt"
|
||||
git -C "$repo" worktree prune 2>/dev/null || true
|
||||
ok "worktree removed"
|
||||
else info "no worktree to remove"; fi
|
||||
# 3) branch (opt-in)
|
||||
if [ "$del_branch" -eq 1 ]; then
|
||||
git -C "$repo" branch -D "$branch" 2>/dev/null && ok "deleted branch $branch" || warn "could not delete branch $branch"
|
||||
else info "branch $branch kept (use --delete-branch to remove)"; fi
|
||||
ok "dev-down '$name' complete (live untouched)"
|
||||
}
|
||||
|
||||
usage(){ cat >&2 <<EOF
|
||||
${C_BLD}nsbx${C_0} — Neuron Sandbox: experiments + code changes against the REAL engram
|
||||
runtime on an isolated snapshot of the live mind, with a gated promote-to-prod path.
|
||||
|
||||
nsbx dev <name> [--base REF] [--worktree DIR] ONE command onboarding: new branch (dev/<name>) + git
|
||||
[--port N] [--prefix P] [--build] worktree (persistent, never /tmp) + isolated engram on a
|
||||
[--no-engram] [--repo R] non-default port. clone -> coding on the mind in one shot.
|
||||
nsbx dev-down <name> [--delete-branch] [--repo R] teardown: destroy the engram + remove the worktree
|
||||
(branch kept unless --delete-branch). live untouched.
|
||||
nsbx up [name] [flags…] one command: your private, isolated copy of the mind
|
||||
(creates on first run, starts thereafter; prod untouchable)
|
||||
nsbx create [name] [--port N] [--source DIR | --branch REF [--repo R] | --binary PATH]
|
||||
clone live store+WAL+config, place/build the runtime, boot on an
|
||||
isolated port (never :$LIVE_BIND_PORT/:$SOUL_PORT). Default runtime = stock prod binary.
|
||||
nsbx build <name> --source DIR | --branch REF rebuild the runtime from a code change + hot-restart
|
||||
nsbx run <name> <cmd...> | api <path> [json] run an experiment; capture output + metrics
|
||||
nsbx validate <name> rails as checks: zero-loss(load+reboot), reboot-prove,
|
||||
RSS bound, retrieval parity, keystone integrity
|
||||
nsbx promote <name> [--data] [--i-approve-prod-cutover] GATED rails cutover to prod (DRY-RUN without approval)
|
||||
nsbx destroy <name> stop daemon, free port, remove clone (live untouched)
|
||||
nsbx list | nsbx status <name>
|
||||
|
||||
Env in 'run' cmds: \$SBX_URL \$SBX_PORT \$SBX_KEY \$SBX_DATA \$SBX_BIN \$SBX_NAME
|
||||
EOF
|
||||
}
|
||||
|
||||
main(){
|
||||
local cmd="${1:-}"; shift || true
|
||||
case "$cmd" in
|
||||
dev) cmd_dev "$@";;
|
||||
dev-down) cmd_dev_down "$@";;
|
||||
up) cmd_up "$@";;
|
||||
create) cmd_create "$@";;
|
||||
build) cmd_build "$@";;
|
||||
run) cmd_run "$@";;
|
||||
validate) cmd_validate "$@";;
|
||||
promote) cmd_promote "$@";;
|
||||
destroy) cmd_destroy "$@";;
|
||||
list|ls) cmd_list "$@";;
|
||||
status) cmd_status "$@";;
|
||||
""|-h|--help|help) usage;;
|
||||
*) die "unknown command: $cmd (try: nsbx help)";;
|
||||
esac
|
||||
}
|
||||
main "$@"
|
||||
Reference in New Issue
Block a user