docs: update AGENTS.md to reflect Terraform->ArgoCD migration and ESO secrets flow
- Terraform now owns infra only (namespaces, Cloudflare, R2), not k8s config - Argo CD manages all k8s resources via git - Secrets: Vault -> ExternalSecret -> ESO -> k8s Secret (no Terraform secrets) - Updated Common Operations for new GitOps flow - Added tea CLI note, corrected repo paths
This commit is contained in:
Will Anderson
@@ -58,14 +58,11 @@ export VAULT_ADDR=http://localhost:8200
|
||||
### How secrets flow to pods
|
||||
|
||||
```
|
||||
Vault → TF_VAR_* (via infrastructure.env) → kubernetes_secret (Terraform) → pod env via secretKeyRef
|
||||
Vault (source of truth) → ExternalSecret (in git) → ESO operator → k8s Secret → pod env
|
||||
```
|
||||
|
||||
Secrets in Terraform:
|
||||
- `neuron-secrets` (neuron ns): GITHUB_WEBHOOK_SECRET, GITEA_WEBHOOK_SECRET, SLACK_BOT_TOKEN, SLACK_SIGNING_SECRET
|
||||
- `cloudflared-secret` (neuron ns): TUNNEL_TOKEN
|
||||
- `github-runner-secret` (ci ns): ACCESS_TOKEN
|
||||
- `gitea-db` (git ns): password
|
||||
All k8s Secrets are managed by External Secrets Operator (ESO) pulling from Vault.
|
||||
No secrets are stored in Terraform. Do not add `kubernetes_secret` resources to Terraform.
|
||||
|
||||
---
|
||||
|
||||
@@ -75,17 +72,28 @@ Secrets in Terraform:
|
||||
|
||||
| Layer | Tool | What it owns |
|
||||
|-------|------|-------------|
|
||||
| Infrastructure | **Terraform** | Namespaces, PVCs, ConfigMaps, k8s Secrets, Ingresses, Helm releases |
|
||||
| Applications | **Argo CD** | Deployments, Services |
|
||||
| Infrastructure | **Terraform** | Namespaces, Cloudflare DNS/tunnels, R2 storage, Vault bootstrap |
|
||||
| Configuration | **Argo CD** | All k8s resources: Deployments, Services, ConfigMaps, Ingresses, PVCs, ExternalSecrets, Helm releases |
|
||||
| Secrets | **ESO → Vault** | All k8s Secrets — ExternalSecret manifests in git, values pulled from Vault at runtime |
|
||||
|
||||
**Never modify k8s resources directly with kubectl.** Always go through Terraform or Argo CD (Git push).
|
||||
|
||||
**Secrets flow:**
|
||||
```
|
||||
Vault (source of truth) → ExternalSecret (in git) → ESO operator → k8s Secret → pod env
|
||||
```
|
||||
|
||||
Adding a new secret:
|
||||
1. `vault kv put secret/<path> field=value`
|
||||
2. Add ExternalSecret manifest to `k8s/<service>/` referencing that path
|
||||
3. Push → Argo CD applies ExternalSecret → ESO syncs Secret within ~60s
|
||||
|
||||
### Running Terraform
|
||||
|
||||
```bash
|
||||
cd ~/Development/infrastructure/servers/servers/legion/
|
||||
direnv exec . terraform plan # load secrets + plan
|
||||
direnv exec . terraform apply # apply
|
||||
cd ~/Development/infrastructure/servers/legion/
|
||||
terraform plan # uses direnv to load credentials
|
||||
terraform apply
|
||||
```
|
||||
|
||||
State is stored in Cloudflare R2 bucket `legion-terraform-state`.
|
||||
@@ -95,20 +103,22 @@ State is stored in Cloudflare R2 bucket `legion-terraform-state`.
|
||||
Root app `legion-apps` watches `servers/legion/apps/` in the `will/infrastructure` Gitea repo.
|
||||
Push to main → Argo CD syncs within ~30 seconds.
|
||||
|
||||
App manifests: `~/Development/infrastructure/servers/servers/legion/apps/*.yaml`
|
||||
|
||||
Argo CD UI: https://argocd.neuralplatform.ai
|
||||
- App manifests: `~/Development/infrastructure/servers/legion/apps/*.yaml`
|
||||
- k8s config: `~/Development/infrastructure/servers/legion/k8s/<service>/*.yaml`
|
||||
- Argo CD UI: https://argocd.neuralplatform.ai
|
||||
- Gitea CLI: `tea pr ls`, `tea issue ls` (configured, default login: neuralplatform)
|
||||
|
||||
### Repo layout
|
||||
|
||||
```
|
||||
~/Development/
|
||||
infrastructure/ ← this repo (local only, no GitHub remote)
|
||||
Agents.md ← you are here
|
||||
AGENTS.md ← you are here
|
||||
servers/ ← nested repo (will/infrastructure on Gitea)
|
||||
servers/legion/
|
||||
*.tf ← Terraform (infrastructure layer)
|
||||
apps/*.yaml ← Argo CD manifests (app layer)
|
||||
legion/
|
||||
*.tf ← Terraform (infra layer: namespaces, Cloudflare, R2)
|
||||
apps/*.yaml ← Argo CD Application manifests (app definitions)
|
||||
k8s/<service>/*.yaml ← k8s config synced by Argo CD (PVCs, ExternalSecrets, etc.)
|
||||
neural-platform/
|
||||
neuron/ ← github: harmonic-framework/neuron, gitea: neural-platform/neuron
|
||||
harmonic-framework/
|
||||
@@ -237,28 +247,26 @@ Runner labels: `self-hosted,linux,x64,legion`
|
||||
|
||||
### Deploy a config change to an app
|
||||
|
||||
1. Edit `servers/servers/legion/apps/<app>.yaml`
|
||||
1. Edit `servers/legion/apps/<app>.yaml` or `servers/legion/k8s/<service>/<file>.yaml`
|
||||
2. `git commit && git push` (from `servers/`)
|
||||
3. Argo CD syncs in ~30s
|
||||
|
||||
### Add a new secret to a pod
|
||||
|
||||
1. Add `kubernetes_secret` resource to the relevant `.tf` file (value from `var.X`)
|
||||
2. Add `variable "X"` to `variables.tf` (sensitive = true)
|
||||
3. Add `TF_VAR_X=$(_v path field)` to `infrastructure.env`
|
||||
4. Store the secret: `vault kv put secret/path field=value`
|
||||
5. `direnv exec . terraform apply`
|
||||
6. Reference `secretKeyRef: name: <secret-name>` in the app manifest
|
||||
1. Store the secret: `vault kv put secret/path field=value`
|
||||
2. Add/update an ExternalSecret in `servers/legion/k8s/<service>/external-secrets.yaml`
|
||||
3. Reference `secretKeyRef: name: <secret-name>` in the app manifest
|
||||
4. Push → ESO syncs the k8s Secret within ~60s
|
||||
|
||||
### Add a new Helm service (stays in Terraform)
|
||||
### Add a new Helm release
|
||||
|
||||
Add `helm_release` resource to appropriate `.tf` file, apply.
|
||||
Add an Argo CD Application manifest to `servers/legion/apps/<name>.yaml` with `spec.source.chart` and `spec.source.helm.values`. Push to deploy.
|
||||
|
||||
### Rotate a secret
|
||||
|
||||
1. Update value in Vault: `vault kv patch secret/path field=newvalue`
|
||||
2. Re-run `direnv exec . terraform apply` (updates kubernetes_secret)
|
||||
3. Restart the affected pod: `kubectl rollout restart deployment/<name> -n <ns>`
|
||||
1. `vault kv patch secret/path field=newvalue`
|
||||
2. Force ESO refresh: `kubectl annotate externalsecret -n <ns> <name> force-sync=$(date +%s) --overwrite`
|
||||
3. Restart pod if needed: `kubectl rollout restart deployment/<name> -n <ns>`
|
||||
|
||||
---
|
||||
|
||||
@@ -275,8 +283,10 @@ Add `helm_release` resource to appropriate `.tf` file, apply.
|
||||
|
||||
## What NOT To Do
|
||||
|
||||
- `kubectl apply` directly — use Terraform or push to Git
|
||||
- `kubectl edit` — use Terraform or Git
|
||||
- Hardcode any secret value — always use `var.X` → Vault
|
||||
- `kubectl apply` directly — push to Git and let Argo CD sync
|
||||
- `kubectl edit` — edit the file in `servers/legion/`, push to Git
|
||||
- Hardcode any secret value — store in Vault, reference via ExternalSecret
|
||||
- Add `kubernetes_secret` to Terraform — secrets belong in ESO/Vault, not Terraform
|
||||
- Commit `terraform.tfstate` — state is in R2, never local
|
||||
- Call Gitea API from Mac directly — CF Access blocks it, use SSH to Legion
|
||||
- Call Gitea API from Mac directly — CF Access blocks it; use `tea` CLI or SSH to Legion
|
||||
- Use `var.X` pattern for secrets in Terraform — Terraform no longer owns k8s Secrets
|
||||
|
||||
Reference in New Issue
Block a user