engram: batch-cosine Adapter/Strategy/Factory over ggml, supersedes hand-rolled PR #114

Stop hand-rolling GPU kernels for batch cosine similarity — use ggml (the
MIT-licensed compute library underneath llama.cpp, installed standalone via
Homebrew) as the preferred backend, without ripping out PR #114's
carefully-verified hand-rolled Metal shader.

Structure: one stable public adapter (eg_cosine_batch.h, zero #ifdef at call
sites) backed by three selectable concrete Strategies behind an internal
vtable (eg_cosine_batch_strategy.h) chosen by a Factory (eg_cosine_batch.c):

  - eg_cosine_batch_strategy_ggml.c    — NEW. ggml + dynamically-loaded Metal
                                          backend plugin (ggml_backend_load_all_from_path
                                          + ggml_mul_mat for the batched dot
                                          product), gather/scatter around the
                                          -2.0 sentinel contract.
  - eg_cosine_batch_strategy_metal_hand.m — PR #114's original hand-rolled
                                          Metal shader bridge, preserved
                                          almost verbatim, now one strategy
                                          among several rather than the only
                                          option. eg_cosine_batch.metal kept
                                          byte-identical to the original.
  - eg_cosine_batch_strategy_cpu.c     — universal always-false fallback
                                          (direct descendant of PR #114's
                                          eg_metal_cosine_stub.c).

Selection: EL_COSINE_BATCH_STRATEGY=ggml|metal|cpu|auto (default: ggml first,
then hand-rolled Metal, then CPU — first available wins), plus back-compat
EL_METAL_COSINE=0 to disable every GPU-backed strategy. build_vindex_bench.sh
compiles all three strategies on Darwin, CPU-fallback-only elsewhere.

vindex_bench.c now reports BRUTE-GGML and BRUTE-METAL side by side against
the same CPU oracle, on the same dataset, in one run (real numbers vs. real
store snapshot in the PR body).

lang/runtime/eg_cosine_batch.h

@@ -0,0 +1,106 @@
+/* eg_cosine_batch.h — stable Adapter interface over batch-cosine-similarity
+ * BACKEND STRATEGIES. This header is the ONE thing call sites (el_runtime.c,
+ * vindex_bench.c, ...) talk to. Plain C11, safe to #include on every
+ * platform — the symbols declared here always exist and always link,
+ * regardless of what backend actually runs underneath. Zero #ifdef at call
+ * sites: which concrete strategy executes (ggml/Metal, hand-rolled Metal, or
+ * the always-false CPU fallback) is resolved once, lazily, inside
+ * eg_cosine_batch.c's factory — see eg_cosine_batch_strategy.h for that.
+ *
+ * This supersedes eg_metal_cosine.h (PR #114's single hand-rolled-Metal-only
+ * bridge). The contract is UNCHANGED — same shapes, same sentinel, same
+ * never-partial guarantee, same "caller must always be prepared to fall back
+ * to its own scalar per-node loop" rule — only the name changed, because the
+ * thing behind it is no longer "the Metal bridge," it is "whichever batch-
+ * cosine strategy the factory picked." eg_metal_cosine.h's original doc
+ * comments (byte-for-byte, this file is the direct descendant) are preserved
+ * below since they remain the precise spec any strategy must honor.
+ *
+ * On ANY failure at ANY step — no compute device, compile/init error, alloc
+ * failure, bad args — every function here returns false and writes nothing.
+ * Out-params are either fully populated or left completely untouched, never
+ * partial. Callers MUST always be prepared to fall back to their own scalar
+ * per-node CPU loop unconditionally. These functions must never crash, throw,
+ * or hang the calling process — several call sites run inside a long-lived
+ * daemon's request-handling hot path.
+ */
+#ifndef EG_COSINE_BATCH_H
+#define EG_COSINE_BATCH_H
+
+#include <stdint.h>
+#include <stdbool.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/* Batched cosine similarity: one query vector against `n` node vectors.
+ *
+ *   query      — qdim floats, the query embedding. Raw/unnormalized.
+ *   qdim       — query dimensionality (e.g. 768 for nomic-embed-text).
+ *   node_ptrs  — array of n pointers, node_ptrs[i] pointing at a (possibly
+ *                differently-owned, possibly NULL) float vector for node i.
+ *                NOT required to be contiguous — every strategy performs the
+ *                gather into a packed row-major matrix internally, exactly
+ *                mirroring how EngramNode.emb is one malloc per node.
+ *   node_dims  — array of n ints, node_dims[i] = that node's real emb_dim
+ *                (0 or mismatched vs qdim => that node scores -2.0, matching
+ *                eg_cosine's null/dim-mismatch/zero-norm sentinel exactly).
+ *   n          — number of nodes.
+ *   out_scores — caller-owned array of n doubles; out_scores[i] is filled
+ *                with the cosine similarity of node i against query, or
+ *                -2.0 for a null/dim-mismatched/zero-norm node — bit-for-bit
+ *                the same contract as eg_cosine(node_ptrs[i], query, qdim).
+ *
+ * Returns true iff a real backend strategy ran and out_scores was fully
+ * populated. Returns false (out_scores left untouched) on ANY failure or
+ * unavailability — no compute device, compile/init failure, allocation
+ * failure, n<=0, qdim<=0, null query/node_ptrs/node_dims/out_scores.
+ */
+bool eg_cosine_batch(const float* query, int32_t qdim,
+                      const float* const* node_ptrs,
+                      const int32_t* node_dims,
+                      int32_t n,
+                      double* out_scores);
+
+/* True iff a real (non-CPU-fallback) strategy is available right now (cheap
+ * after the first call — cached). Purely informational (e.g. a startup log
+ * line or /api/stats field); callers should still treat a false return from
+ * eg_cosine_batch()/eg_cosine_batch_multi() itself as the authoritative
+ * fallback signal, not this function. */
+bool eg_cosine_batch_available(void);
+
+/* Which concrete strategy is currently selected — "ggml", "metal-hand",
+ * or "cpu-fallback". Purely informational/diagnostic, same spirit as
+ * eg_cosine_batch_available(). Never NULL. */
+const char* eg_cosine_batch_strategy_name(void);
+
+/* Multi-query batched cosine: nq query vectors against the SAME n node
+ * vectors, in one call. A real strategy uploads/prepares the node population
+ * ONCE and reuses it for every query, instead of nq separate
+ * eg_cosine_batch() calls each paying the full gather+upload cost — PR #114
+ * measured this necessary: at N~=13.7k/dim=768, repeating the single-query
+ * call per query was slower than the CPU baseline; batching queries together
+ * is what makes a GPU-backed path a real win at this shape. Use this
+ * whenever multiple queries will run against an unchanged (or
+ * rarely-changing) node population; use eg_cosine_batch() for a genuinely
+ * one-off comparison.
+ *
+ *   queries    — nq*qdim floats, row-major (query i at queries+i*qdim).
+ *   out_scores — caller-owned nq*n doubles, row-major
+ *                (out_scores[i*n+j] = cosine(queries[i], node j)), same
+ *                -2.0 sentinel semantics as eg_cosine_batch().
+ *
+ * Returns true iff a real strategy ran and out_scores was fully populated
+ * (all nq*n entries); false (untouched) on any failure/unavailability. */
+bool eg_cosine_batch_multi(const float* queries, int32_t qdim, int32_t nq,
+                            const float* const* node_ptrs,
+                            const int32_t* node_dims,
+                            int32_t n,
+                            double* out_scores);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* EG_COSINE_BATCH_H */