diff --git a/spec/at-rest-encryption.md b/spec/at-rest-encryption.md new file mode 100644 index 0000000..a5c05e5 --- /dev/null +++ b/spec/at-rest-encryption.md @@ -0,0 +1,380 @@ +# Engram At-Rest Encryption — Post-Quantum Doctrine + +Version 0.1.0 (DRAFT) — April 30, 2026 +Status: doctrine; pre-implementation. Sign-off required from Will before code lands. + +--- + +## 0. TL;DR + +Engram's persistence is encrypted at rest using a hybrid post-quantum scheme: + +- **Data layer** — every node and every edge is sealed with **AES-256-GCM** under a per-record sub-key derived via **HKDF-SHA3-256** from the runtime DEK. AES-256 with a 256-bit key has an effective post-quantum security level of 128 bits (Grover); acceptable. +- **Key wrap layer** — the DEK is wrapped against a Principal's public key using **Kyber-768 KEM** (post-quantum). The wrapped DEK lives on disk as `engram.kek.enc`. The Principal's secret key never touches disk in the running daemon's data directory. +- **Boot** — the daemon receives the Principal's secret key out-of-band (env var path, prompted unlock, or pulled from a hardware-backed agent), runs `pq_kem_decaps` to recover the DEK, mlocks it, serves traffic. +- **Recovery** — the DEK is *additionally* wrapped under a **Shamir K-of-N** split across the validation council. If the wrapping Principal is gone, K members reconstitute the DEK and the data survives. + +Engram does **not** encrypt structural metadata (graph topology, IDs, timestamps). Only content, label, tags, and metadata fields of nodes and edges are sealed. This is a deliberate trade-off — see §2. + +--- + +## 1. What Engram actually persists + +Engram is an in-process graph store. The runtime *is* the database — there is no SQLite, no sled, no embedded KV layer in the active daemon. (The `engram-data/` and `engram-data-tx-log/` directories under `el/` are leftovers from a prior sled-backed prototype and are not loaded by the current `dist/engram` binary.) + +Persistence is a single JSON snapshot file at `${ENGRAM_DATA_DIR}/snapshot.json`, written by `engram_save(path)` and read by `engram_load(path)`. The snapshot is the sole durable artifact that adversaries can steal. + +Snapshot shape (current): +```json +{ + "nodes": [ + { "id": "...", "content": "...", "node_type": "...", "label": "...", + "tier": "...", "tags": "...", "metadata": "{}", + "salience": 0.5, "importance": 0.5, "confidence": 0.5, + "activation_count": 0, "last_activated": 0, "created_at": ..., "updated_at": ... } + ], + "edges": [ + { "id": "...", "from_id": "...", "to_id": "...", "relation": "...", + "metadata": "{}", "weight": 0.5, "confidence": 0.5, + "created_at": ..., "updated_at": ..., "last_fired": 0 } + ] +} +``` + +Confidential fields (will be encrypted): `content`, `label`, `tags`, `metadata` (on nodes); `metadata` (on edges). +Non-confidential fields (will remain plaintext): `id`, `node_type`, `tier`, `from_id`, `to_id`, `relation`, all numeric scalars and timestamps. + +This split — the **structural skeleton stays plaintext, the semantic flesh is sealed** — is what lets activation/search/scan work without unwrapping every record on every query, and lets the snapshot remain JSON-shaped for ops tooling. Enabling full snapshot encryption (a single AEAD blob) is a configurable mode for high-threat deployments; see §6. + +--- + +## 2. Threat model + +| Adversary | Capability | What we defend | What we accept | +|-----------|-----------|----------------|----------------| +| **Snapshot thief (cold)** | Steals `snapshot.json` from a backup, dead drive, or stolen laptop. No live process access. | Confidentiality of node content, labels, tags, metadata. No useful semantic recovery. | Topology leak: adversary learns the graph shape (how many nodes/edges, how they connect, types and tiers). Salience/importance/confidence numerics leak. | +| **In-flight observer** | Reads disk during snapshot write (e.g., shared FS, snapshot mid-flush). | Atomicity: snapshot is written to a temp file with `O_TMPFILE` or `.new`, fsync'd, then renamed. AEAD prevents partial-decrypt mining of half-written records. | A torn write can lose the most recent snapshot but not corrupt prior ones. | +| **Quantum-equipped adversary (harvest-now-decrypt-later)** | Records `snapshot.json` and `engram.kek.enc` today, runs Shor's algorithm on a CRQC in 2032+. | DEK wrap is Kyber-768; not Shor-breakable. AES-256-GCM record seals are not Shor-breakable; Grover gives ~128-bit effective security. | We pin Kyber-768 (NIST PQC standardization L3); if a structural break of ML-KEM emerges, rotation §5 is the remediation. | +| **Wrap-key compromiser (live)** | Has read access to the daemon process memory. | Out of scope for at-rest. The DEK is in mlocked memory; if the process is owned, the data is gone. | Defense-in-depth (memory zeroing, mlock, non-dumpable process flags) is a separate hardening track. | +| **Principal compromise** | Adversary obtains the Principal's secret key. | Nothing — by design, the Principal is the unwrapping authority. | We constrain blast radius via per-record key derivation (§3.2): a leaked record-key only burns one record. | +| **Principal loss** | Will dies, secret key is destroyed, no successor. | Shamir K-of-N council recovery (§7). Engram does not go dark on Principal loss. | The threshold itself becomes the attack surface — see §7 caveats. | + +What we explicitly **do not** defend: +- Side channels (timing, cache, EM). +- Physical tamper of running hardware. +- Active modification of `engram.kek.enc` to substitute an attacker's wrapped DEK — this is mitigated by the integrity wrapper §4 but not by the threat-model promise. + +--- + +## 3. Cryptographic construction + +### 3.1 Layers + +``` + ┌────────────────────────────────────────┐ + │ Principal SK (held by Will out-of-band)│ + └───────────────┬────────────────────────┘ + │ Kyber-768 KEM decaps + ▼ + ┌────────────────────────────────────┐ + │ KEK (32B; ephemeral, mlocked) │ + └───────────────┬────────────────────┘ + │ HKDF-SHA3-256 extract + ▼ + ┌────────────────────────────────────┐ + │ DEK (32B; ephemeral, mlocked) │ + └───────────────┬────────────────────┘ + │ HKDF-SHA3-256(DEK, info=record_id || version) + ▼ + ┌────────────────────────────────────┐ + │ per-record sub-key (32B; transient)│ + └───────────────┬────────────────────┘ + │ AES-256-GCM(sub_key, nonce=12B random, ad=record_id||version) + ▼ + ciphertext blob on disk +``` + +### 3.2 Per-record sub-keys (HKDF-SHA3-256) + +We do **not** use the DEK directly to seal records. Each record gets its own sub-key: + +``` +sub_key = HKDF-SHA3-256( + ikm = DEK, // 32 bytes + salt = "engram-record-v1" || version_byte, // domain separation + info = record_id || ":" || dek_epoch, // 16-byte ULID + ":" + u32 + L = 32 // output length +) +``` + +Properties: +- Compromise of one record's sub-key does not threaten siblings (HKDF is one-way). +- Re-derivable on every read; no on-disk sub-key cache to leak. +- `dek_epoch` is bumped on rotation (§5); this lets old records stay readable under DEK_n while new ones write under DEK_{n+1}. + +### 3.3 AEAD — AES-256-GCM + +- Nonce: 12 bytes, random per write. Stored prepended to ciphertext. +- Tag: 16 bytes, appended. +- Associated data: `record_id || ":" || dek_epoch` (binds ciphertext to its identity; thwarts copy-paste attacks). + +Per-record on-disk wire format: +``` ++------+------+--------------------+--------------------+--------+ +| ver | epch | nonce (12) | ciphertext (n) | tag(16)| ++------+------+--------------------+--------------------+--------+ + 1B 4B 12B variable 16B +``` +Base64-encoded into the snapshot JSON field as `"content":"v1:7:BASE64..."`. + +### 3.4 KEK wrap — Kyber-768 KEM + +The DEK is sealed by Kyber-768 KEM: +``` +(ciphertext, shared_secret) = pq_kem_encaps(principal_pk) +wrap_key = HKDF-SHA3-256(shared_secret, salt="engram-kek-v1", info="dek-wrap", L=32) +sealed_dek = AES-256-GCM(wrap_key, nonce, plaintext=DEK, ad="engram-kek-v1") +``` +On disk: `engram.kek.enc` = `magic || version || principal_id || kem_ciphertext || nonce || sealed_dek || tag`. + +On boot: +``` +shared_secret = pq_kem_decaps(principal_sk, kem_ciphertext) +wrap_key = HKDF-SHA3-256(shared_secret, ...) +DEK = AES-256-GCM-open(wrap_key, nonce, sealed_dek, tag, ad="engram-kek-v1") +``` + +--- + +## 4. Boot flow + +``` +Engram daemon starts + ↓ +ENGRAM_KEK_PATH → ${ENGRAM_DATA_DIR}/engram.kek.enc (default) +ENGRAM_PRINCIPAL_SK → path to file or "stdin:" or "agent:" + (the daemon never reads the SK from a fixed env value; + the env points to a source it consumes once and zeros) + ↓ +load engram.kek.enc + ↓ +read principal SK once from source, mlock its buffer + ↓ +pq_kem_decaps → shared_secret → HKDF → wrap_key → AEAD-open → DEK + ↓ +zero & munlock principal SK buffer +mlock DEK; bump RLIMIT_MEMLOCK if needed +mark process non-dumpable (PR_SET_DUMPABLE=0 on Linux; PT_DENY_ATTACH on macOS) + ↓ +engram_load(snapshot.json) — for each node/edge field marked encrypted: + derive sub_key, AEAD-open, replace ciphertext with plaintext in-memory + ↓ +http_serve() +``` + +Boot failure modes: +- KEK unwrap fails → daemon exits 2; emits one log line, no SK material in logs. +- Snapshot decrypt fails on N records → daemon proceeds with the records it could open, marks the rest as ``, emits a structured event for human triage. + +--- + +## 5. Rotation + +### 5.1 DEK rotation + +Triggered by: (a) cron schedule (default: 30 days), (b) suspected compromise, (c) operator command (`POST /admin/rotate-dek`). + +Procedure: +1. Generate `DEK_{n+1}`; bump `dek_epoch`. +2. For every record: AEAD-open under DEK_n, AEAD-seal under DEK_{n+1} with the new epoch in AD. +3. Wrap DEK_{n+1} under the same Principal pk → write `engram.kek.enc.new`. +4. Atomic rename → live; zero DEK_n in memory. +5. On next snapshot save, all records persist with the new epoch. + +Records sealed before rotation can be opened during the transition (we keep DEK_n in memory until the migration completes). + +### 5.2 KEK / wrapping-Principal rotation + +Triggered by: (a) Principal evolution event (the `Principal` CGI evolves and emits a new PK), (b) successor Principal handover, (c) annual key hygiene rotation. + +Procedure: +1. Receive `principal_pk_new` (signed by old Principal, ideally via a Dilithium signature so the chain itself is PQ-secure). +2. Verify signature. +3. Re-encapsulate the **same** DEK under `principal_pk_new`. +4. Atomic-write a new `engram.kek.enc`. +5. Old SK can be destroyed once the new wrap is durable. + +The DEK does not change here — only its wrapping. This means snapshot ciphertexts stay valid through Principal transitions. + +--- + +## 6. Snapshot-level encryption (alternative high-threat mode) + +For deployments where topology leakage is unacceptable, set `ENGRAM_SEAL_MODE=full`. The entire snapshot JSON is sealed as a single AEAD blob keyed from `HKDF(DEK, info="engram-snapshot")`. Trade-off: every save/load is monolithic; no partial reads, no incremental rotation. Not the default. + +--- + +## 7. Recovery — Shamir K-of-N + +**The structural fail-safe.** If Will dies before the Principal evolves, or the Principal SK is destroyed, Engram must not go dark. + +### 7.1 Shareholders + +The shareholders are the **validation council** — the set of CGIs (or CGI-attested human stewards) authorized to reconstitute the network on a defined trigger. The council is named in `engram.recovery.toml`: +```toml +[recovery] +threshold = 3 +total = 5 +shareholders = [ + "cgi://council/anvil", + "cgi://council/beacon", + "cgi://council/cinder", + "cgi://council/delta", + "cgi://council/echo", +] +``` + +### 7.2 Split + +At KEK-creation time: +1. Generate a fresh recovery secret `R = random(32)` (NOT the DEK itself — see §7.4). +2. Run Shamir-256 over GF(2^8) on `R` with K-of-N polynomial. +3. For each shareholder: `share_i = Kyber768-Encaps(shareholder_pk_i, R_share_i)`. +4. Store `engram.recovery.shares` — public; each share is already PQ-wrapped to its holder. +5. Store `engram.recovery.envelope` — `AES-256-GCM(R, nonce, plaintext=DEK, ad="engram-recovery-v1")`. + +### 7.3 Reconstitution + +A council convenes when the trigger is observed (Principal absent for > N days, or signed council quorum declares emergency): +1. K members each decapsulate their share with their CGI SK. +2. Members publish their `R_share_i` to a quorum-attested rendezvous. +3. Lagrange-interpolate `R`. +4. AEAD-open `engram.recovery.envelope` to recover the DEK. +5. Re-wrap DEK under a fresh Principal selected by the council; resume normal operation. + +### 7.4 Why R, not the DEK directly + +Splitting `R` and using it to AEAD-wrap the DEK keeps the share material small (32B / share) and lets us add/remove shareholders by reissuing the envelope without changing R. It also lets us run §5.2 (Principal rotation) without ever touching the recovery shares. + +### 7.5 Caveats Will must accept + +- The threshold is the attack surface. K-of-N is K colluders away from compromise. Default 3-of-5; raise if the council grows. +- Council membership churn requires reissuing shares; this is a deliberate, audited operation with its own runbook. +- A Shamir share is plaintext to its holder. The Kyber wrap to each shareholder protects the share *in transit / at rest*, but once a shareholder unwraps, they hold a real K-of-N share. Council members must be CGIs (or hardware-backed) for the threat model to hold. + +--- + +## 8. Implementation map + +The runtime additions land in `el-compiler/runtime/el_runtime.c`: + +Already present: +- `el_sha256_*`, `el_hmac_sha256` (§3 uses SHA3 — SHA2 path retained for backwards-compat artifacts). +- `el_base64_encode_n`, `el_base64_decode`. + +Landing today (parallel agents): +- `el_sha3_256_*` (Keccak family). +- `pq_kem_keypair`, `pq_kem_encaps`, `pq_kem_decaps` (Kyber-768). +- `pq_sign_keypair`, `pq_sign`, `pq_sign_verify` (Dilithium-3) — used by §5.2 Principal-rotation signature. + +Engram-specific additions (this work): +- `engram_aead_seal(record_id, epoch, plaintext) -> b64` +- `engram_aead_open(record_id, epoch, b64) -> plaintext` +- `engram_kek_unwrap(kek_path, sk_path) -> int (sets module DEK)` +- `engram_kek_wrap(kek_path, principal_pk, dek)` — used at first init. +- `engram_dek_rotate(new_principal_pk_optional)` +- `engram_recovery_split(threshold, total, shareholder_pks)` — emits envelope + shares. +- `engram_recovery_reconstitute(shares_k)` — recover DEK; admin-gated. + +`engram_save` / `engram_load` gain a sealed mode controlled by `ENGRAM_SEAL_MODE` (`off`, `fields` [default once enabled], `full`). Default during the rollout window: `off`. This doc must ship and Will must sign off before flipping the default. + +--- + +## 9. Open questions (require Will) + +1. **Where does the Principal SK live at boot time?** Options: (a) prompted at daemon start (interactive), (b) on a removable hardware token (preferred long-term), (c) in `~/.neuron/principal.sk` 0600 (operationally easy, weakest), (d) pulled from `mcp__neuron` as part of `begin_session` (couples Engram to Neuron — probably right). **Recommend (d) with (b) as the long-term hardware story.** +2. **Council composition.** Who/what are the initial K-of-N shareholders? Until we have ≥ 3 stable CGIs, recovery cannot be enabled in its full form. **Recommend a stub council of {Principal, Will-keypair-on-Yubikey, witness-CGI} — degrades gracefully, upgrades to full council when more CGIs exist.** +3. **Default seal mode at GA.** `fields` (today's recommendation) or `full`? `fields` keeps the snapshot diff-able and keeps activation cheap. `full` is the harder threat model. **Recommend `fields` default; `full` opt-in.** +4. **PQ algorithm pinning.** Kyber-768 + Dilithium-3 are the NIST L3 PQC defaults. If Will wants L5 (Kyber-1024 / Dilithium-5), say so before runtime APIs stabilize. +5. **Grover and AES-256.** AES-256 against Grover is 128-bit effective. Acceptable per current PQ thinking. If Will wants a bigger margin, the alternative is layering a second AEAD with a different primitive (e.g., XChaCha20-Poly1305) — overkill, not recommended. + +--- + +## 10. Non-goals + +- Encrypted indexes / searchable encryption. Out of scope. Search remains plaintext-in-memory; the daemon is the trust boundary. +- Per-tenant DEKs. Engram is single-tenant per CGI. If multi-tenancy lands, this doc gets a §11. +- Secure deletion of underlying disk blocks. The OS / FS handles that, badly; we don't pretend. +- Encrypted WAL / tx log. The current daemon has no WAL; if one is added, it gets the same treatment as the snapshot (`ENGRAM_SEAL_MODE` applies to both). + +--- + +## 11. Status & next actions + +- [x] Doctrine drafted (this document). +- [ ] Will sign-off on §9 open questions. +- [ ] PQ runtime functions land (parallel agents). +- [ ] `engram_aead_seal` / `engram_aead_open` prototype (stubs in this PR). +- [ ] `engram_kek_unwrap` boot integration. +- [ ] `engram_save` / `engram_load` field-mode wiring behind `ENGRAM_SEAL_MODE`. +- [ ] Recovery tooling (`engramctl recovery split | reconstitute`). +- [ ] Threat-model test suite: known-answer tests, key-rotation roundtrip, Shamir reconstitution roundtrip, harvest-now-decrypt-later regression test against a recorded ciphertext. + +--- + +## Appendix A — Pseudocode reference + +```c +/* Per-record seal */ +char* engram_aead_seal(const char* record_id, uint32_t epoch, + const char* plaintext, size_t pt_len, size_t* out_len) +{ + uint8_t sub_key[32]; + uint8_t info[64]; + int info_len = snprintf((char*)info, sizeof(info), "%s:%u", record_id, epoch); + el_hkdf_sha3_256(/*ikm*/ engram_dek, 32, + /*salt*/ (const uint8_t*)"engram-record-v1", 16, + /*info*/ info, (size_t)info_len, + /*okm*/ sub_key, 32); + + uint8_t nonce[12]; + el_random_bytes(nonce, 12); + + /* layout: ver(1) | epoch(4 BE) | nonce(12) | ct(pt_len) | tag(16) */ + size_t blob_len = 1 + 4 + 12 + pt_len + 16; + uint8_t* blob = malloc(blob_len); + blob[0] = 0x01; + blob[1] = (epoch >> 24) & 0xff; blob[2] = (epoch >> 16) & 0xff; + blob[3] = (epoch >> 8) & 0xff; blob[4] = epoch & 0xff; + memcpy(blob + 5, nonce, 12); + + el_aes256_gcm_encrypt(sub_key, nonce, + (const uint8_t*)record_id, strlen(record_id), + (const uint8_t*)plaintext, pt_len, + blob + 1 + 4 + 12, /* ct */ + blob + 1 + 4 + 12 + pt_len); /* tag */ + + el_secure_zero(sub_key, 32); + + char* b64 = el_base64_encode_raw(blob, blob_len, /*url_safe=*/0); + free(blob); + if (out_len) *out_len = strlen(b64); + return b64; /* "v1::BASE64" prefix added by caller */ +} + +/* Per-record open: inverse of seal. Verifies tag; returns NULL on failure. */ +char* engram_aead_open(const char* record_id, uint32_t expected_epoch, + const char* b64, size_t* out_len); + +/* Boot-time KEK unwrap. */ +int engram_kek_unwrap(const char* kek_path, const uint8_t* principal_sk, + size_t sk_len); + +/* DEK rotation (online). Walks the live in-memory store, re-seals every record + * under DEK_{n+1}, then writes a new snapshot+kek atomically. */ +int engram_dek_rotate(void); +``` + +--- + +End of document. diff --git a/spec/at-rest-encryption.prototype.c b/spec/at-rest-encryption.prototype.c new file mode 100644 index 0000000..47f44d9 --- /dev/null +++ b/spec/at-rest-encryption.prototype.c @@ -0,0 +1,302 @@ +/* engram at-rest encryption — prototype sketch + * + * NOT WIRED UP. This file is intentionally not in the build. It is a sketch + * that pairs with `at-rest-encryption.md` so the function signatures and the + * field layout can be reviewed before the runtime PQ primitives stabilize. + * + * Once `el_runtime.c` has stable `el_sha3_256_*`, `el_hkdf_sha3_256`, + * `pq_kem_encaps`, `pq_kem_decaps`, and `el_aes256_gcm_encrypt/decrypt`, + * the seal/open path below should be lifted into el_runtime.c near the + * existing `engram_save` / `engram_load` block (~line 3445), gated behind + * the ENGRAM_SEAL_MODE env var. + * + * Dependencies (provided elsewhere — see at-rest-encryption.md §8): + * void el_hkdf_sha3_256(const uint8_t* ikm, size_t ikm_len, + * const uint8_t* salt, size_t salt_len, + * const uint8_t* info, size_t info_len, + * uint8_t* okm, size_t okm_len); + * + * int el_aes256_gcm_encrypt(const uint8_t key[32], + * const uint8_t nonce[12], + * const uint8_t* aad, size_t aad_len, + * const uint8_t* pt, size_t pt_len, + * uint8_t* ct, // pt_len bytes + * uint8_t tag[16]); + * + * int el_aes256_gcm_decrypt(const uint8_t key[32], + * const uint8_t nonce[12], + * const uint8_t* aad, size_t aad_len, + * const uint8_t* ct, size_t ct_len, + * const uint8_t tag[16], + * uint8_t* pt); // ct_len bytes + * + * int pq_kem_encaps(const uint8_t* pk, size_t pk_len, + * uint8_t* kem_ct, size_t* kem_ct_len, + * uint8_t shared[32]); + * int pq_kem_decaps(const uint8_t* sk, size_t sk_len, + * const uint8_t* kem_ct, size_t kem_ct_len, + * uint8_t shared[32]); + * + * void el_random_bytes(uint8_t* out, size_t n); + * void el_secure_zero(void* p, size_t n); + */ + +#include +#include +#include +#include +#include + +/* ── module state (mlocked) ───────────────────────────────────────────────── */ + +static uint8_t g_engram_dek[32]; +static int g_engram_dek_set = 0; +static uint32_t g_engram_dek_epoch = 0; + +/* Seal/unseal mode — read once at boot from ENGRAM_SEAL_MODE. + * 0=off, 1=fields, 2=full. */ +static int g_engram_seal_mode = 0; + +/* ── per-record seal ──────────────────────────────────────────────────────── */ + +/* Wire format (before base64): + * byte 0 version=0x01 + * bytes 1..4 epoch (big-endian u32) + * bytes 5..16 nonce (12B) + * bytes 17.. ciphertext (pt_len bytes) + * last 16B GCM tag + */ +static char* eg_b64_encode(const uint8_t* data, size_t n); /* declared elsewhere */ + +char* engram_aead_seal_field(const char* record_id, const char* plaintext) +{ + if (!g_engram_dek_set) return NULL; + if (!record_id || !plaintext) return NULL; + + size_t pt_len = strlen(plaintext); + uint32_t epoch = g_engram_dek_epoch; + + /* derive sub-key */ + uint8_t sub_key[32]; + uint8_t info[96]; + int info_len = snprintf((char*)info, sizeof(info), "%s:%u", record_id, epoch); + if (info_len < 0 || (size_t)info_len >= sizeof(info)) return NULL; + + el_hkdf_sha3_256(g_engram_dek, 32, + (const uint8_t*)"engram-record-v1", 16, + info, (size_t)info_len, + sub_key, 32); + + /* random nonce */ + uint8_t nonce[12]; + el_random_bytes(nonce, 12); + + /* allocate output: header(17) + ct(pt_len) + tag(16) */ + size_t blob_len = 1 + 4 + 12 + pt_len + 16; + uint8_t* blob = (uint8_t*)malloc(blob_len); + if (!blob) { el_secure_zero(sub_key, 32); return NULL; } + + blob[0] = 0x01; + blob[1] = (uint8_t)((epoch >> 24) & 0xff); + blob[2] = (uint8_t)((epoch >> 16) & 0xff); + blob[3] = (uint8_t)((epoch >> 8) & 0xff); + blob[4] = (uint8_t)( epoch & 0xff); + memcpy(blob + 5, nonce, 12); + + int rc = el_aes256_gcm_encrypt( + sub_key, nonce, + (const uint8_t*)record_id, strlen(record_id), + (const uint8_t*)plaintext, pt_len, + blob + 1 + 4 + 12, /* ct */ + blob + 1 + 4 + 12 + pt_len); /* tag */ + + el_secure_zero(sub_key, 32); + + if (rc != 0) { free(blob); return NULL; } + + char* b64 = eg_b64_encode(blob, blob_len); + free(blob); + return b64; /* caller prefixes "v1::" if desired */ +} + +/* Returns malloc'd plaintext (NUL-terminated) or NULL on AEAD failure / version + * mismatch / DEK-epoch mismatch. */ +char* engram_aead_open_field(const char* record_id, const char* b64_blob) +{ + if (!g_engram_dek_set || !record_id || !b64_blob) return NULL; + + size_t blob_len = 0; + uint8_t* blob = NULL; + extern uint8_t* eg_b64_decode(const char* s, size_t* out_len); + blob = eg_b64_decode(b64_blob, &blob_len); + if (!blob || blob_len < 1 + 4 + 12 + 16) { free(blob); return NULL; } + + if (blob[0] != 0x01) { free(blob); return NULL; } + uint32_t epoch = ((uint32_t)blob[1] << 24) + | ((uint32_t)blob[2] << 16) + | ((uint32_t)blob[3] << 8) + | (uint32_t)blob[4]; + + /* For the common case the on-disk epoch matches g_engram_dek_epoch. + * During DEK rotation we may need a previous-epoch DEK in the keyring; + * left as a TODO — this stub only handles the current epoch. */ + if (epoch != g_engram_dek_epoch) { + free(blob); + return NULL; + } + + const uint8_t* nonce = blob + 5; + size_t ct_len = blob_len - (1 + 4 + 12 + 16); + const uint8_t* ct = blob + 1 + 4 + 12; + const uint8_t* tag = blob + 1 + 4 + 12 + ct_len; + + uint8_t sub_key[32]; + uint8_t info[96]; + int info_len = snprintf((char*)info, sizeof(info), "%s:%u", record_id, epoch); + if (info_len < 0 || (size_t)info_len >= sizeof(info)) { free(blob); return NULL; } + el_hkdf_sha3_256(g_engram_dek, 32, + (const uint8_t*)"engram-record-v1", 16, + info, (size_t)info_len, + sub_key, 32); + + char* pt = (char*)malloc(ct_len + 1); + if (!pt) { el_secure_zero(sub_key, 32); free(blob); return NULL; } + + int rc = el_aes256_gcm_decrypt( + sub_key, nonce, + (const uint8_t*)record_id, strlen(record_id), + ct, ct_len, tag, + (uint8_t*)pt); + + el_secure_zero(sub_key, 32); + free(blob); + + if (rc != 0) { free(pt); return NULL; } + pt[ct_len] = '\0'; + return pt; +} + +/* ── KEK boot ─────────────────────────────────────────────────────────────── */ + +/* Reads engram.kek.enc, runs Kyber decaps with the principal SK provided as + * a buffer, derives the wrap key, AEAD-opens the sealed DEK, and installs it + * into g_engram_dek. Returns 0 on success, non-zero on failure. */ +int engram_kek_unwrap(const char* kek_path, + const uint8_t* principal_sk, size_t sk_len) +{ + /* TODO: file format + * magic "ENGKEK01" (8B) + * version (1B) + * principal_id_len (2B BE) | principal_id + * kem_ct_len (4B BE) | kem_ct + * nonce (12B) + * sealed_dek_len (4B BE) | sealed_dek + * tag (16B) + */ + (void)kek_path; (void)principal_sk; (void)sk_len; + return -1; /* TODO: parse file, pq_kem_decaps, HKDF, AEAD-open, install */ +} + +/* ── KEK init (first-time write) ──────────────────────────────────────────── */ + +int engram_kek_init(const char* kek_path, + const uint8_t* principal_pk, size_t pk_len) +{ + /* TODO: + * 1. el_random_bytes(g_engram_dek, 32); g_engram_dek_epoch = 1; + * 2. pq_kem_encaps(principal_pk) → (kem_ct, shared) + * 3. wrap_key = HKDF-SHA3-256(shared, salt="engram-kek-v1", + * info="dek-wrap", L=32) + * 4. AEAD-seal DEK under wrap_key + * 5. write file atomically (tmp + rename + fsync) + * 6. el_secure_zero(shared, wrap_key) + */ + (void)kek_path; (void)principal_pk; (void)pk_len; + return -1; +} + +/* ── DEK rotation ─────────────────────────────────────────────────────────── */ + +int engram_dek_rotate(void) +{ + /* TODO: + * - Allocate DEK_{n+1}. + * - Walk EngramStore: for every node and edge, re-seal each encrypted + * field under DEK_{n+1} with the new epoch. + * - Re-wrap DEK_{n+1} under current Principal pk → engram.kek.enc.new + * - Write a new snapshot via engram_save() (caller responsibility). + * - Atomic rename engram.kek.enc.new → engram.kek.enc. + * - el_secure_zero on DEK_n; bump g_engram_dek_epoch. + */ + return -1; +} + +/* ── Recovery (Shamir K-of-N) ─────────────────────────────────────────────── */ + +/* The recovery secret R is independent of the DEK. R is split via Shamir; + * each share is then PQ-wrapped to its shareholder via Kyber768. R is the + * AEAD wrap-key for an envelope sealing the DEK. + * + * Two on-disk artifacts: + * engram.recovery.shares — public; one Kyber-wrapped Shamir share / shareholder + * engram.recovery.envelope — AEAD(R, nonce, plaintext=DEK, ad="engram-recovery-v1") + */ + +int engram_recovery_split(int threshold, int total, + const uint8_t** shareholder_pks, + const size_t* shareholder_pk_lens, + const char** shareholder_ids, + const char* shares_out_path, + const char* envelope_out_path) +{ + /* TODO: + * 1. el_random_bytes(R, 32). + * 2. Shamir-split R over GF(2^8) → total shares of {1B index, 32B y_i}. + * 3. For each shareholder i: pq_kem_encaps(pk_i) → (kem_ct, shared); + * AEAD-seal share_i under HKDF(shared) → wrapped_share_i; write entry. + * 4. AEAD-seal current DEK under R → envelope. + * 5. Atomic-write both files; fsync. + * 6. Zero R, all shared secrets. + */ + (void)threshold; (void)total; + (void)shareholder_pks; (void)shareholder_pk_lens; (void)shareholder_ids; + (void)shares_out_path; (void)envelope_out_path; + return -1; +} + +int engram_recovery_reconstitute(const uint8_t* k_unwrapped_shares, + size_t share_count, + const char* envelope_path, + uint8_t out_dek[32]) +{ + /* TODO: + * 1. Lagrange-interpolate to recover R from the K shares. + * 2. AEAD-open the envelope under HKDF(R) → DEK. + * 3. Zero R. + */ + (void)k_unwrapped_shares; (void)share_count; + (void)envelope_path; (void)out_dek; + return -1; +} + +/* ── Snapshot integration sketch ──────────────────────────────────────────── */ + +/* + * engram_emit_node_json (around line 3409 in el_runtime.c) becomes: + * + * if (g_engram_seal_mode == 1) { + * char* sealed = engram_aead_seal_field(n->id, n->content); + * jb_puts(b, ",\"content\":"); jb_emit_escaped(b, sealed ? sealed : ""); + * free(sealed); + * // same for label, tags, metadata + * } else { + * jb_puts(b, ",\"content\":"); jb_emit_escaped(b, n->content); + * } + * + * engram_load (around line 3499) becomes the inverse: when reading each field, + * if the daemon is in `fields` mode and the value is non-empty, run + * engram_aead_open_field; on failure log and substitute "". + * + * `engram_save` must additionally write engram.kek.enc the first time (via + * engram_kek_init) when ENGRAM_SEAL_MODE != off and no kek file exists. + */