fix/engram-lexical-tokenized-search
El SDK Release / build-and-release (pull_request) Failing after 14m46s
engram search/activate/goal-bias matched the ENTIRE raw query string as a single case-insensitive substring (istr_contains(field, q)). Multi-word queries like "windows msi signing" only matched a node containing that exact contiguous run, so real multi-word queries returned ZERO on a graph saturated with the answer. This is Ctrl-F, not search — and search is the core of the engram being useful. Fix: split the query on whitespace into distinct tokens; a node matches if it contains ANY token in content/label/tags. Rank by distinct tokens matched (desc) then salience (desc). istr_contains is kept unchanged as the per-token primitive. Single-token queries are a strict special case (score 0 or 1) so the many single-word callers do not regress. Sites changed (all in el_runtime.c): - new helpers engram_tokenize_query / engram_node_match_score / engram_rank_cmp - engram_search (internal el_val_t path) - engram_search_json (HTTP /api/search path) - engram_activate seed loop (HTTP /api/activate path; seed activation scaled by token coverage so full-query matches seed more strongly) - engram_goal_bias overlap bonus upgraded to graded token coverage Proof (6591-node snapshot copy, rebuilt binary on :8799, POST JSON path): windows msi signing 0 -> 20 Will Anderson 0 -> 20 windows msi 0 -> 20 tokenized search fix 0 -> 20 Single-word parity preserved (VBD/volatility/elc capped at limit; unkey = all matching nodes). Top hits are relevant (e.g. "Will Anderson" surfaces the Project Design and VBD whitepapers). Note: GET ?q=a%20b still returns 0 because query_param (server.el) does not URL-decode — a separate EL-layer bug; the soul's POST-JSON path is fixed here.
Description
The Engram programming language — types as knowledge nodes, quantum-sealed prod target
147 MiB
Releases
5
El SDK (latest)
Latest
Languages
Emacs Lisp
74.6%
C
21.9%
HTML
1.5%
Objective-C
0.6%
Shell
0.6%
Other
0.8%