engram: batch-cosine Adapter/Strategy/Factory over ggml, supersedes hand-rolled PR #114

Stop hand-rolling GPU kernels for batch cosine similarity — use ggml (the MIT-licensed compute library underneath llama.cpp, installed standalone via Homebrew) as the preferred backend, without ripping out PR #114's carefully-verified hand-rolled Metal shader. Structure: one stable public adapter (eg_cosine_batch.h, zero #ifdef at call sites) backed by three selectable concrete Strategies behind an internal vtable (eg_cosine_batch_strategy.h) chosen by a Factory (eg_cosine_batch.c): - eg_cosine_batch_strategy_ggml.c — NEW. ggml + dynamically-loaded Metal backend plugin (ggml_backend_load_all_from_path + ggml_mul_mat for the batched dot product), gather/scatter around the -2.0 sentinel contract. - eg_cosine_batch_strategy_metal_hand.m — PR #114's original hand-rolled Metal shader bridge, preserved almost verbatim, now one strategy among several rather than the only option. eg_cosine_batch.metal kept byte-identical to the original. - eg_cosine_batch_strategy_cpu.c — universal always-false fallback (direct descendant of PR #114's eg_metal_cosine_stub.c). Selection: EL_COSINE_BATCH_STRATEGY=ggml|metal|cpu|auto (default: ggml first, then hand-rolled Metal, then CPU — first available wins), plus back-compat EL_METAL_COSINE=0 to disable every GPU-backed strategy. build_vindex_bench.sh compiles all three strategies on Darwin, CPU-fallback-only elsewhere. vindex_bench.c now reports BRUTE-GGML and BRUTE-METAL side by side against the same CPU oracle, on the same dataset, in one run (real numbers vs. real store snapshot in the PR body).
2026-08-15 17:16:41 -05:00
parent 2555e363a6
commit b3f410fc91
9 changed files with 1372 additions and 5 deletions
@@ -0,0 +1,82 @@
+/* eg_cosine_batch_strategy.h — internal Strategy interface, NOT for call
+ * sites (they use eg_cosine_batch.h). Only eg_cosine_batch.c's factory and
+ * the concrete strategy implementation files include this.
+ *
+ * Each concrete strategy exposes exactly one getter returning a pointer to a
+ * static, immutable EgCosineBatchStrategy vtable. Which getters actually
+ * exist as linkable symbols is a BUILD-TIME concern (decided by
+ * build_vindex_bench.sh / the engram daemon's own build, via which .c/.m
+ * files get compiled per platform) gated by the EG_HAVE_STRATEGY_* macros
+ * below — the factory in eg_cosine_batch.c is the ONLY place that branches
+ * on those macros. Call sites never see them; that's the whole point of the
+ * Adapter in eg_cosine_batch.h.
+ *
+ * Three concrete strategies exist:
+ *   eg_cosine_batch_strategy_ggml()        — ggml + dynamically-loaded Metal
+ *                                             backend plugin. Darwin only in
+ *                                             this build; the default
+ *                                             preferred strategy wherever
+ *                                             available. EG_HAVE_STRATEGY_GGML.
+ *   eg_cosine_batch_strategy_metal_hand()  — the original hand-rolled Metal
+ *                                             compute shader from PR #114
+ *                                             (eg_cosine_batch.metal),
+ *                                             preserved verbatim as a
+ *                                             selectable fallback strategy,
+ *                                             not deleted. Darwin only.
+ *                                             EG_HAVE_STRATEGY_METAL_HAND.
+ *   eg_cosine_batch_strategy_cpu()         — universal always-false
+ *                                             fallback. Always compiled, on
+ *                                             every platform; this is what a
+ *                                             non-Darwin build links
+ *                                             exclusively (matching PR #114's
+ *                                             eg_metal_cosine_stub.c), and
+ *                                             what any platform falls back
+ *                                             to when no real strategy is
+ *                                             available at runtime.
+ */
+#ifndef EG_COSINE_BATCH_STRATEGY_H
+#define EG_COSINE_BATCH_STRATEGY_H
+
+#include <stdint.h>
+#include <stdbool.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+typedef struct EgCosineBatchStrategy {
+    /* Stable, short, lowercase-hyphenated identifier — what
+     * eg_cosine_batch_strategy_name() surfaces. Never NULL. */
+    const char* name;
+
+    /* Cheap after the first call (lazy init, cached internally). Must never
+     * throw/crash/hang — mirrors eg_cosine_batch_available()'s contract. */
+    bool (*available)(void);
+
+    /* Same shape/contract as eg_cosine_batch() in eg_cosine_batch.h. */
+    bool (*batch)(const float* query, int32_t qdim,
+                  const float* const* node_ptrs, const int32_t* node_dims,
+                  int32_t n, double* out_scores);
+
+    /* Same shape/contract as eg_cosine_batch_multi() in eg_cosine_batch.h. */
+    bool (*batch_multi)(const float* queries, int32_t qdim, int32_t nq,
+                         const float* const* node_ptrs, const int32_t* node_dims,
+                         int32_t n, double* out_scores);
+} EgCosineBatchStrategy;
+
+#ifdef EG_HAVE_STRATEGY_GGML
+const EgCosineBatchStrategy* eg_cosine_batch_strategy_ggml(void);
+#endif
+
+#ifdef EG_HAVE_STRATEGY_METAL_HAND
+const EgCosineBatchStrategy* eg_cosine_batch_strategy_metal_hand(void);
+#endif
+
+/* Always declared/linked, on every platform/build. */
+const EgCosineBatchStrategy* eg_cosine_batch_strategy_cpu(void);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* EG_COSINE_BATCH_STRATEGY_H */