bigmerge
b3f410fc91
El SDK CI - dev / build-and-test (pull_request) Failing after 4m29s
Stop hand-rolling GPU kernels for batch cosine similarity — use ggml (the MIT-licensed compute library underneath llama.cpp, installed standalone via Homebrew) as the preferred backend, without ripping out PR #114's carefully-verified hand-rolled Metal shader. Structure: one stable public adapter (eg_cosine_batch.h, zero #ifdef at call sites) backed by three selectable concrete Strategies behind an internal vtable (eg_cosine_batch_strategy.h) chosen by a Factory (eg_cosine_batch.c): - eg_cosine_batch_strategy_ggml.c — NEW. ggml + dynamically-loaded Metal backend plugin (ggml_backend_load_all_from_path + ggml_mul_mat for the batched dot product), gather/scatter around the -2.0 sentinel contract. - eg_cosine_batch_strategy_metal_hand.m — PR #114's original hand-rolled Metal shader bridge, preserved almost verbatim, now one strategy among several rather than the only option. eg_cosine_batch.metal kept byte-identical to the original. - eg_cosine_batch_strategy_cpu.c — universal always-false fallback (direct descendant of PR #114's eg_metal_cosine_stub.c). Selection: EL_COSINE_BATCH_STRATEGY=ggml|metal|cpu|auto (default: ggml first, then hand-rolled Metal, then CPU — first available wins), plus back-compat EL_METAL_COSINE=0 to disable every GPU-backed strategy. build_vindex_bench.sh compiles all three strategies on Darwin, CPU-fallback-only elsewhere. vindex_bench.c now reports BRUTE-GGML and BRUTE-METAL side by side against the same CPU oracle, on the same dataset, in one run (real numbers vs. real store snapshot in the PR body).
107 lines
5.5 KiB
C
107 lines
5.5 KiB
C
/* eg_cosine_batch.h — stable Adapter interface over batch-cosine-similarity
|
|
* BACKEND STRATEGIES. This header is the ONE thing call sites (el_runtime.c,
|
|
* vindex_bench.c, ...) talk to. Plain C11, safe to #include on every
|
|
* platform — the symbols declared here always exist and always link,
|
|
* regardless of what backend actually runs underneath. Zero #ifdef at call
|
|
* sites: which concrete strategy executes (ggml/Metal, hand-rolled Metal, or
|
|
* the always-false CPU fallback) is resolved once, lazily, inside
|
|
* eg_cosine_batch.c's factory — see eg_cosine_batch_strategy.h for that.
|
|
*
|
|
* This supersedes eg_metal_cosine.h (PR #114's single hand-rolled-Metal-only
|
|
* bridge). The contract is UNCHANGED — same shapes, same sentinel, same
|
|
* never-partial guarantee, same "caller must always be prepared to fall back
|
|
* to its own scalar per-node loop" rule — only the name changed, because the
|
|
* thing behind it is no longer "the Metal bridge," it is "whichever batch-
|
|
* cosine strategy the factory picked." eg_metal_cosine.h's original doc
|
|
* comments (byte-for-byte, this file is the direct descendant) are preserved
|
|
* below since they remain the precise spec any strategy must honor.
|
|
*
|
|
* On ANY failure at ANY step — no compute device, compile/init error, alloc
|
|
* failure, bad args — every function here returns false and writes nothing.
|
|
* Out-params are either fully populated or left completely untouched, never
|
|
* partial. Callers MUST always be prepared to fall back to their own scalar
|
|
* per-node CPU loop unconditionally. These functions must never crash, throw,
|
|
* or hang the calling process — several call sites run inside a long-lived
|
|
* daemon's request-handling hot path.
|
|
*/
|
|
#ifndef EG_COSINE_BATCH_H
|
|
#define EG_COSINE_BATCH_H
|
|
|
|
#include <stdint.h>
|
|
#include <stdbool.h>
|
|
|
|
#ifdef __cplusplus
|
|
extern "C" {
|
|
#endif
|
|
|
|
/* Batched cosine similarity: one query vector against `n` node vectors.
|
|
*
|
|
* query — qdim floats, the query embedding. Raw/unnormalized.
|
|
* qdim — query dimensionality (e.g. 768 for nomic-embed-text).
|
|
* node_ptrs — array of n pointers, node_ptrs[i] pointing at a (possibly
|
|
* differently-owned, possibly NULL) float vector for node i.
|
|
* NOT required to be contiguous — every strategy performs the
|
|
* gather into a packed row-major matrix internally, exactly
|
|
* mirroring how EngramNode.emb is one malloc per node.
|
|
* node_dims — array of n ints, node_dims[i] = that node's real emb_dim
|
|
* (0 or mismatched vs qdim => that node scores -2.0, matching
|
|
* eg_cosine's null/dim-mismatch/zero-norm sentinel exactly).
|
|
* n — number of nodes.
|
|
* out_scores — caller-owned array of n doubles; out_scores[i] is filled
|
|
* with the cosine similarity of node i against query, or
|
|
* -2.0 for a null/dim-mismatched/zero-norm node — bit-for-bit
|
|
* the same contract as eg_cosine(node_ptrs[i], query, qdim).
|
|
*
|
|
* Returns true iff a real backend strategy ran and out_scores was fully
|
|
* populated. Returns false (out_scores left untouched) on ANY failure or
|
|
* unavailability — no compute device, compile/init failure, allocation
|
|
* failure, n<=0, qdim<=0, null query/node_ptrs/node_dims/out_scores.
|
|
*/
|
|
bool eg_cosine_batch(const float* query, int32_t qdim,
|
|
const float* const* node_ptrs,
|
|
const int32_t* node_dims,
|
|
int32_t n,
|
|
double* out_scores);
|
|
|
|
/* True iff a real (non-CPU-fallback) strategy is available right now (cheap
|
|
* after the first call — cached). Purely informational (e.g. a startup log
|
|
* line or /api/stats field); callers should still treat a false return from
|
|
* eg_cosine_batch()/eg_cosine_batch_multi() itself as the authoritative
|
|
* fallback signal, not this function. */
|
|
bool eg_cosine_batch_available(void);
|
|
|
|
/* Which concrete strategy is currently selected — "ggml", "metal-hand",
|
|
* or "cpu-fallback". Purely informational/diagnostic, same spirit as
|
|
* eg_cosine_batch_available(). Never NULL. */
|
|
const char* eg_cosine_batch_strategy_name(void);
|
|
|
|
/* Multi-query batched cosine: nq query vectors against the SAME n node
|
|
* vectors, in one call. A real strategy uploads/prepares the node population
|
|
* ONCE and reuses it for every query, instead of nq separate
|
|
* eg_cosine_batch() calls each paying the full gather+upload cost — PR #114
|
|
* measured this necessary: at N~=13.7k/dim=768, repeating the single-query
|
|
* call per query was slower than the CPU baseline; batching queries together
|
|
* is what makes a GPU-backed path a real win at this shape. Use this
|
|
* whenever multiple queries will run against an unchanged (or
|
|
* rarely-changing) node population; use eg_cosine_batch() for a genuinely
|
|
* one-off comparison.
|
|
*
|
|
* queries — nq*qdim floats, row-major (query i at queries+i*qdim).
|
|
* out_scores — caller-owned nq*n doubles, row-major
|
|
* (out_scores[i*n+j] = cosine(queries[i], node j)), same
|
|
* -2.0 sentinel semantics as eg_cosine_batch().
|
|
*
|
|
* Returns true iff a real strategy ran and out_scores was fully populated
|
|
* (all nq*n entries); false (untouched) on any failure/unavailability. */
|
|
bool eg_cosine_batch_multi(const float* queries, int32_t qdim, int32_t nq,
|
|
const float* const* node_ptrs,
|
|
const int32_t* node_dims,
|
|
int32_t n,
|
|
double* out_scores);
|
|
|
|
#ifdef __cplusplus
|
|
}
|
|
#endif
|
|
|
|
#endif /* EG_COSINE_BATCH_H */
|