el/lang/tools/lsp/README.md

# El LSP — Language Server for El

Full Language Server Protocol implementation for the El programming language.

## Features

| Feature | Status |
|---------|--------|
| Syntax highlighting | Full TextMate grammar |
| Completions | Builtins (130+), user-defined fns, keywords, types |
| Hover | Signatures + descriptions for all builtins and user fns |
| Go-to-definition | Jump to `fn name(` in the open document |
| Diagnostics | Unclosed braces/parens/brackets, unterminated strings |
| Document sync | Full (re-sends entire document on every change) |

## Building

### Prerequisites

- The `elc` compiler binary at `dist/platform/elc` (built from the repo root)
- `cc` (clang or gcc), `libcurl`, `pthreads`

### Build

```bash
# From the el repo root:
./tools/lsp/build.sh

# Or with a custom elc path:
ELC=/path/to/elc ./tools/lsp/build.sh
```

Output: `tools/lsp/dist/el-lsp`

### Install system-wide

```bash
sudo cp tools/lsp/dist/el-lsp /usr/local/bin/el-lsp
```

## VSCode Extension

### Development install (recommended)

1. Build the binary first (see above).
2. Install the npm dependency:
   ```bash
   cd tools/lsp/vscode-extension
   npm install
   ```
3. Open `tools/lsp/vscode-extension/` in VSCode.
4. Press **F5** — this launches the Extension Development Host.
5. Open any `.el` file in the dev host window.

### Package as .vsix

```bash
npm install -g @vscode/vsce
cd tools/lsp/vscode-extension
vsce package
# Produces: el-language-1.0.0.vsix
```

Install the .vsix:
```bash
code --install-extension el-language-1.0.0.vsix
```

### Configuration

| Setting | Default | Description |
|---------|---------|-------------|
| `el.lspPath` | (bundled) | Path to `el-lsp` binary. Empty = use `../dist/el-lsp`. |
| `el.trace.server` | `off` | LSP message tracing. Set to `verbose` to see all messages. |

## Neovim / other editors

Any editor that supports LSP can use `el-lsp`. Example configuration for
Neovim with `nvim-lspconfig`:

```lua
local lspconfig = require('lspconfig')
local configs   = require('lspconfig.configs')

if not configs.el then
  configs.el = {
    default_config = {
      cmd        = { 'el-lsp' },
      filetypes  = { 'el' },
      root_dir   = lspconfig.util.root_pattern('.git', '*.el'),
      settings   = {},
    },
  }
end

lspconfig.el.setup({})
```

Add to `ftdetect/el.vim`:
```vim
au BufRead,BufNewFile *.el set filetype=el
```

## Wire protocol

El LSP communicates over stdin/stdout using standard LSP framing:

```
Content-Length: <N>\r\n
\r\n
<N bytes of UTF-8 JSON>
```

The `__read_n(n: Int) -> String` primitive in `el_runtime.c` reads exactly
`n` bytes from stdin — needed because `readline()` stops at `\n` and LSP
JSON bodies are not newline-terminated. `__print_raw(s: String)` writes with
`fwrite + fflush` to preserve embedded `\r\n` in headers.

## Smoke test

After building, verify the server responds to `initialize`:

```bash
python3 - << 'PY'
import subprocess, json

def frame(obj):
    body = json.dumps(obj).encode()
    return f"Content-Length: {len(body)}\r\n\r\n".encode() + body

def read_response(proc):
    hdr = b""
    while not hdr.endswith(b"\r\n\r\n"):
        hdr += proc.stdout.read(1)
    cl = int([l for l in hdr.decode().split("\r\n") if "Content-Length" in l][0].split(": ")[1])
    return json.loads(proc.stdout.read(cl))

proc = subprocess.Popen(["./dist/el-lsp"], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
proc.stdin.write(frame({"jsonrpc":"2.0","id":1,"method":"initialize","params":{"capabilities":{}}}))
proc.stdin.flush()
r = read_response(proc)
print("Server name:", r["result"]["serverInfo"]["name"])
print("Capabilities:", list(r["result"]["capabilities"].keys()))
proc.stdin.write(frame({"jsonrpc":"2.0","method":"exit","params":{}}))
proc.stdin.flush()
proc.wait()
print("OK")
PY
```

## Architecture

```
el-lsp.el
  lsp_read_message()      reads header bytes one-by-one, then __read_n(body_len)
  lsp_write_message()     __print_raw("Content-Length: N\r\n\r\n" + json)
  lsp_dispatch()          routes method string to handler
  lsp_builtin_catalog()   [String] of "name|signature|description" entries
  lsp_extract_fns()       scan source for "fn name(" patterns
  lsp_word_at()           expand identifier under cursor
  lsp_compute_diagnostics() scan for unclosed brackets + unterminated strings
```

## Files

```
tools/lsp/
  el-lsp.el                   LSP server source (El language)
  build.sh                    Build script
  README.md                   This file
  dist/
    el-lsp                    Compiled binary (after build)
    el-lsp.c                  Generated C (after build)
  vscode-extension/
    extension.js              Extension entry point
    package.json              Extension manifest
    language-configuration.json  Bracket matching, comment config
    syntaxes/
      el.tmGrammar.json       TextMate syntax grammar
    .vscode/
      launch.json             F5 debug configuration
      tasks.json              Pre-launch npm install task
```

## Runtime additions

Two new primitives added to `el-compiler/runtime/`:

### `__read_n(n: Int) -> String`
Reads exactly `n` bytes from stdin using `fread`. Returns `""` on EOF.
Required for reading JSON-RPC message bodies.

### `__print_raw(s: String) -> Void`
Writes a string to stdout using `fwrite + fflush`. Preserves embedded
`\r\n` bytes exactly. Required for LSP Content-Length headers.

Both are declared in `el_runtime.h` and registered in the `builtin_arity`
table in `el-compiler/src/codegen.el`.