neuron-technologies/el-retired
Archived
3
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases 5 Wiki Activity

add BM25+ text ranking in EL, remove Ollama query-embedding dependency

Browse Source 
- Add list_set, math_exp, and float_add/sub/mul/div/gt/lt/eq/gte/lte builtins to
  el_runtime.c + el_runtime.h (float arithmetic builtins needed because EL operators
  +*/ operate on raw el_val_t bits, not IEEE 754 doubles)
- Remove engram_embed_query() and its forward declaration from el_runtime.c
- Remove Ollama cosine-similarity blend from activation scoring (reverts 9af2482):
  drops query_emb/query_edim variables, bias *= (1 + 0.3 * sim) block, and all
  free(query_emb) calls from the activation loop
- Implement BM25+ scoring in server.el (k1=1.2, b=0.75, delta=1.0):
  bm25_tokenize, bm25_count_term, bm25_score_doc, bm25_search_json
  V1 uses n_t=1 approximation (constant IDF per corpus size); acceptable as a
  first pass without an inverted index
- Wire /api/bm25/search POST/GET route in server.el dispatcher
- Zero Ollama calls in the activation/search path; embeddings on nodes are
  untouched (still written at node-creation time)
This commit is contained in:
Will Anderson
2026-05-14 11:52:43 -05:00
parent 1a8a16002e
commit 6121b33d25
5 changed files with 446 additions and 80 deletions
Download Patch File Download Diff File Expand all files Collapse all files
lang/releases/v1.0.0-20260501/el_runtime.h
+13
View File
@@ -430,10 +430,22 @@ el_val_t str_to_float(el_val_t s);
el_val_t math_sqrt(el_val_t f);
el_val_t math_log(el_val_t f);
el_val_t math_ln(el_val_t f);
el_val_t math_exp(el_val_t f);
el_val_t math_sin(el_val_t f);
el_val_t math_cos(el_val_t f);
el_val_t math_pi(void);
/* ── Float arithmetic builtins (correct IEEE 754 via bit-cast round-trip) ─── */
el_val_t float_add(el_val_t a, el_val_t b);
el_val_t float_sub(el_val_t a, el_val_t b);
el_val_t float_mul(el_val_t a, el_val_t b);
el_val_t float_div(el_val_t a, el_val_t b);
el_val_t float_gt(el_val_t a, el_val_t b);
el_val_t float_lt(el_val_t a, el_val_t b);
el_val_t float_eq(el_val_t a, el_val_t b);
el_val_t float_gte(el_val_t a, el_val_t b);
el_val_t float_lte(el_val_t a, el_val_t b);
/* ── String additions ────────────────────────────────────────────────────── */
el_val_t str_index_of(el_val_t s, el_val_t sub);
@@ -493,6 +505,7 @@ el_val_t str_join(el_val_t list, el_val_t sep); /* alias of list_joi
el_val_t list_push(el_val_t list, el_val_t elem);
el_val_t list_push_front(el_val_t list, el_val_t elem);
el_val_t list_set(el_val_t list, el_val_t index, el_val_t value);
el_val_t list_join(el_val_t list, el_val_t sep);
el_val_t list_range(el_val_t start, el_val_t end);