diff --git a/Agents.md b/Agents.md new file mode 100644 index 0000000..48b4fbe --- /dev/null +++ b/Agents.md @@ -0,0 +1,281 @@ +# Agent System Reference + +Quick reference for AI agents working in Will's infrastructure. Read this before doing any infrastructure work. + +--- + +## Machines + +| Machine | Role | IP | SSH | +|---------|------|----|-----| +| Local Mac | Dev workstation | 192.168.68.55 | — | +| Legion | Ubuntu 24.04, k3s server | 192.168.68.77 | `ssh legion` | + +Legion runs all production services: k3s, Postgres, Redis, Ollama (GPU), Docker Registry, GitHub Actions runner, Vault, Gitea, AdGuard, Neuron. + +--- + +## Secrets — Always Use Vault + +**Vault is the source of truth for all credentials.** Never hardcode secrets. Never ask the user for a secret value. + +### CLI access (on local Mac) + +```bash +# Load all secrets into env +set -a; source ~/Secrets/credentials/infrastructure.env; set +a + +# Or with direnv (auto-loads in legion/ directory): +cd ~/Development/infrastructure/servers/servers/legion/ +# direnv auto-sources infrastructure.env + +# Friendly alias (after sourcing infrastructure.env): +secret civitai # → civitai API key +secret slack-bot-token # → Slack bot token +secret cf-key # → Cloudflare API key + +# Direct vault lookup: +vault kv get -field=api_key secret/ai # CivitAI +vault kv get -field=bot_token secret/slack # Slack +vault kv get -field=api_key secret/cloudflare # Cloudflare +``` + +Full path+field table is in `~/Secrets/credentials/infrastructure.env` — look at the `_v()` calls. + +### Vault connection + +```bash +VAULT_ADDR=https://vault.neuralplatform.ai +VAULT_TOKEN=$(cat ~/Secrets/tokens/vault-root-token) +``` + +If Vault is unreachable (CF tunnel down), port-forward: +```bash +ssh legion -L 8200:localhost:8200 # in background +export VAULT_ADDR=http://localhost:8200 +``` + +### How secrets flow to pods + +``` +Vault → TF_VAR_* (via infrastructure.env) → kubernetes_secret (Terraform) → pod env via secretKeyRef +``` + +Secrets in Terraform: +- `neuron-secrets` (neuron ns): GITHUB_WEBHOOK_SECRET, GITEA_WEBHOOK_SECRET, SLACK_BOT_TOKEN, SLACK_SIGNING_SECRET +- `cloudflared-secret` (neuron ns): TUNNEL_TOKEN +- `github-runner-secret` (ci ns): ACCESS_TOKEN +- `gitea-db` (git ns): password + +--- + +## Infrastructure Management + +### The split: Terraform vs Argo CD + +| Layer | Tool | What it owns | +|-------|------|-------------| +| Infrastructure | **Terraform** | Namespaces, PVCs, ConfigMaps, k8s Secrets, Ingresses, Helm releases | +| Applications | **Argo CD** | Deployments, Services | + +**Never modify k8s resources directly with kubectl.** Always go through Terraform or Argo CD (Git push). + +### Running Terraform + +```bash +cd ~/Development/infrastructure/servers/servers/legion/ +direnv exec . terraform plan # load secrets + plan +direnv exec . terraform apply # apply +``` + +State is stored in Cloudflare R2 bucket `legion-terraform-state`. + +### Argo CD + +Root app `legion-apps` watches `servers/legion/apps/` in the `will/infrastructure` Gitea repo. +Push to main → Argo CD syncs within ~30 seconds. + +App manifests: `~/Development/infrastructure/servers/servers/legion/apps/*.yaml` + +Argo CD UI: https://argocd.neuralplatform.ai + +### Repo layout + +``` +~/Development/ + infrastructure/ ← this repo (local only, no GitHub remote) + Agents.md ← you are here + servers/ ← nested repo (will/infrastructure on Gitea) + servers/legion/ + *.tf ← Terraform (infrastructure layer) + apps/*.yaml ← Argo CD manifests (app layer) + neural-platform/ + neuron/ ← github: harmonic-framework/neuron, gitea: neural-platform/neuron + harmonic-framework/ + harmonic-framework.com/ ← github: harmonic-framework/harmonic-framework.com + projects/ + personal/ + prism/ ← github: harmonic-framework/prism, gitea: neural-platform/prism + clients/ + ilih.life/ ← github: harmonic-framework/ilih.life, gitea: will/ilih.life +``` + +--- + +## Services & Domains + +### Internal (nook.family — LAN only, AdGuard DNS) + +| Service | URL | Notes | +|---------|-----|-------| +| AdGuard | https://dns.nook.family | DNS + ad blocking | +| Docker Registry | https://registry.nook.family | Push images here | + +### Public (neuralplatform.ai — via Cloudflare tunnel) + +| Service | URL | Notes | +|---------|-----|-------| +| Argo CD | https://argocd.neuralplatform.ai | GitOps UI | +| Vault | https://vault.neuralplatform.ai | Secrets | +| Gitea | https://git.neuralplatform.ai | Git server | +| Ollama | https://ollama.neuralplatform.ai | LLM API | +| Neuron MCP | https://neuron.neuralplatform.ai | MCP server (port 8001) | +| Axon webhooks | https://axon.neuralplatform.ai | Webhook hub (port 3847) | +| npm registry | https://npm.neuralplatform.ai | Verdaccio | +| PyPI registry | https://pypi.neuralplatform.ai | devpi | +| Registry UI | https://docker.neuralplatform.ai | Docker registry browser | + +### NodePort services (direct to Legion IP) + +| Service | Port | Notes | +|---------|------|-------| +| Gitea SSH | 30022 | `git@192.168.68.77:30022` | +| Ollama API | 31434 | `http://192.168.68.77:31434` | + +--- + +## Access Patterns + +### SSH + +```bash +ssh legion # Ubuntu 24.04, user: will +ssh legion "kubectl get pods -A" # run kubectl remotely +``` + +### kubectl (local) + +```bash +export KUBECONFIG=~/.kube/legion-config +kubectl get pods -A +kubectl logs -n neuron deployment/neuron +``` + +### Gitea API + +CF Access blocks direct calls from Mac. Always use Legion cluster IP via SSH: + +```bash +TOKEN=$(vault kv get -field=api_token secret/gitea) +ssh legion "curl -s -H 'Authorization: token $TOKEN' http://10.43.1.53:3000/api/v1/repos/search?limit=50" +``` + +--- + +## Key Paths + +| What | Where | +|------|-------| +| This repo | `~/Development/infrastructure/` | +| Infrastructure (Terraform + Argo CD) | `~/Development/infrastructure/servers/` | +| Neural Platform projects | `~/Development/neural-platform/` | +| Harmonic Framework projects | `~/Development/harmonic-framework/` | +| Personal projects | `~/Development/projects/personal/` | +| Client projects | `~/Development/projects/clients/` | +| Knowledge base | `~/Knowledge/` | +| Secrets & tokens | `~/Secrets/` | +| Infrastructure env | `~/Secrets/credentials/infrastructure.env` | +| kubeconfig | `~/.kube/legion-config` | +| Agent directives | `~/.claude/projects/-Users-will/memory/directives.md` | +| Agent memory index | `~/.claude/projects/-Users-will/memory/MEMORY.md` | + +--- + +## GitHub / Gitea + +**GitHub** (`github.com/harmonic-framework`) — open-source, CI runners, public-facing + +**Gitea** (`git.neuralplatform.ai`) — private/infra, three orgs: +- `will` — personal infra (`will/infrastructure`) +- `neural-platform` — platform projects (`neural-platform/neuron`, `neural-platform/prism`) +- `harmonic-framework` — org site/assets (`harmonic-framework/harmonic-framework.com`) + +Runner labels: `self-hosted,linux,x64,legion` + +--- + +## Namespaces (k8s) + +| Namespace | What | +|-----------|------| +| `dns` | AdGuard | +| `git` | Gitea | +| `neuron` | Neuron + cloudflared | +| `ollama` | Ollama | +| `ci` | GitHub runner | +| `packages` | Verdaccio + devpi | +| `registry` | Docker registry + UI | +| `platform` | Postgres, Redis | +| `monitoring` | Prometheus, Grafana, Loki, Tempo, Alloy | +| `vault` | HashiCorp Vault | +| `argocd` | Argo CD | +| `cert-manager` | cert-manager | + +--- + +## Common Operations + +### Deploy a config change to an app + +1. Edit `servers/servers/legion/apps/.yaml` +2. `git commit && git push` (from `servers/`) +3. Argo CD syncs in ~30s + +### Add a new secret to a pod + +1. Add `kubernetes_secret` resource to the relevant `.tf` file (value from `var.X`) +2. Add `variable "X"` to `variables.tf` (sensitive = true) +3. Add `TF_VAR_X=$(_v path field)` to `infrastructure.env` +4. Store the secret: `vault kv put secret/path field=value` +5. `direnv exec . terraform apply` +6. Reference `secretKeyRef: name: ` in the app manifest + +### Add a new Helm service (stays in Terraform) + +Add `helm_release` resource to appropriate `.tf` file, apply. + +### Rotate a secret + +1. Update value in Vault: `vault kv patch secret/path field=newvalue` +2. Re-run `direnv exec . terraform apply` (updates kubernetes_secret) +3. Restart the affected pod: `kubectl rollout restart deployment/ -n ` + +--- + +## Network + +- Subnet: `192.168.68.x` (TP-Link Deco, router mode) +- DHCP DNS: `192.168.68.77` (AdGuard) + `1.1.1.1` fallback +- AdGuard provides `*.nook.family` resolution → `192.168.68.77` +- Cloudflare tunnel ID: `54bc9b05-3953-47a2-9c3e-adecdcc53d51` +- systemd-resolved is **disabled** on Legion (conflicts with AdGuard port 53) + +--- + +## What NOT To Do + +- `kubectl apply` directly — use Terraform or push to Git +- `kubectl edit` — use Terraform or Git +- Hardcode any secret value — always use `var.X` → Vault +- Commit `terraform.tfstate` — state is in R2, never local +- Call Gitea API from Mac directly — CF Access blocks it, use SSH to Legion