tunnel/AGENTS.md

# AGENTS.md

Single-node, self-hosted OpenTunnel-compatible server in Go. The wire
protocol and API contract are **the reference's**, not ours to invent:

- Source of truth: `.reference/opentunnel/docs/protocol.md`
- API contract: `.reference/opentunnel/crates/opentunnel-server/tests/fixtures/worker-contract.json`
  (paths, status codes, bodies, JSON key order — installed clients depend on it)
- Conformance vectors: `.reference/opentunnel/spec/vectors/*.json`, run by
  `internal/server/*_test.go` (the reference checkout must exist; tests skip
  otherwise)

`.reference/` is untracked research input (see `.reference/opentunnel.md` for
what we take vs. skip). Never vendor it into the build.

## Layout

- `cmd/tunneld` — flags/env wiring, listeners, signal handling
- `cmd/tunnel` — the native Go client/CLI: JSON config, provisioning, routes,
  background service
- `internal/client` — client implementation: config/API, key/CSR/certificate
  storage, bridge WebSocket, TLS termination and local proxying
- `internal/server/limits.go`, `cooldown.go` — per-IP/per-tunnel rate limits
  and the ACME CA cool-down
- `internal/server/proto.go` — wire types (pinned JSON key order), IDs/tokens
  (same generation as the reference: 12-char base32 slugs, `rly_` tokens,
  hex SHA-256 token hashes), route validation, data frames
- `internal/server/clienthello.go` — TLS ClientHello parser (SNI + ALPN) that
  reads exact bytes only, so the connection can be replayed or forwarded
- `internal/server/localca.go` — the built-in insecure CA (the reference's
  `ISSUER=local` equivalent); signs tunnel CSRs after checking CN == hostname
  and SANs ⊆ {hostname, `*.hostname`}
- `internal/server/issuer.go` — the issuer interface plus the local-CA and
  ACME DNS-01 implementations (account persistence, order/challenge flow,
  propagation check, certificate for the client's CSR)
- `internal/server/dns.go`, `dns_porkbun.go`, `dns_challtestsrv.go` — the
  challenge DNS providers: Porkbun's API for production, pebble-challtestsrv
  for local ACME tests
- `internal/server/renew.go` — the renewal loop: 30-day window, 6-hour checks,
  retries, restart recovery, and local-CA → ACME migration
- `internal/server/store.go` — JSON-file store (0600, atomic rename); single
  writer, self-hosted scale
- `internal/server/api.go` — HTTP API per the contract; tunnel existence is
  checked before auth (404 before 401), body JSON syntax before everything
- `internal/server/bridge.go` — WebSocket bridge sessions: attach, route
  claiming (one owner per route), heartbeats, idle retirement, visitor pumps
- `internal/server/admin.go` — the admin dashboard handlers: `GET
  /api/admin/overview` (bearer `ADMIN_TOKEN`, 503 when unset, 401 when
  wrong) and the page/assets served from the embedded frontend
- `web/` — the dashboard frontend: Svelte 5 (runes) + TypeScript, Vite via
  bun, pure CSS, no component libraries. `bun run build` emits to
  `internal/dashboard/dist` (gitignored; the committed `.gitkeep` keeps
  `go:embed all:dist` valid, and the handler answers 503 until built).
  `internal/dashboard` embeds the output; the Dockerfile builds it in a bun
  stage before the Go build
- `internal/server/relay.go` — public TLS listener: parses the ClientHello,
  terminates the domain itself, passes tunnel hostnames to bridges untouched

## Commands

- `go build ./...` / `go vet ./...` / `gofmt -l .` — must stay clean
- `go test ./...`
- Frontend: `cd web && bun install && bun run check && bun run build` — the
  dashboard test skips when `internal/dashboard/dist` has no build (only the
  committed `.gitkeep`)
- Go client: `go build -o .local/tunnel ./cmd/tunnel`, then
  `.local/tunnel config set-api <url>` and `.local/tunnel route add <port> --name <name>`;
  `up`/`down` manage the background service, `status` shows local + server
  state. `import --profile <name> --api <url>` takes over an official CLI
  profile (same tunnel/key/cert).
- Dev loop: `go build -o .local/tunneld ./cmd/tunneld && .local/tunneld -domain localhost`,
  then `OPENTUNNEL_API=http://localhost:8080 opentunnel route add 4096 --name api`
  with the official CLI. `go build` drops a binary next to the source — it is
  gitignored; prefer `-o .local/`.

## Gotchas

- **Coolify deployment** (verified 2026-10-25): app `nokzgyrlhuuzduozr7yts4z3`
  in project Personal, `https://tunnel.tzgyn.dev`, deploy-key source on the
  shared `gitea` key, push to `main` auto-deploys via the Gitea webhook.
  Redeploy with `POST /api/v1/deploy?uuid=<app>` (the GET form is gone).
  Webhook details when debugging: Coolify's *manual* Gitea webhook verifies
  `X-Hub-Signature-256` (HMAC-SHA256 hex of the raw body with the app's
  `manual_webhook_secret_gitea`; Gitea sends this header natively) and
  matches apps by `repository.full_name` against the app's `git_repository`
  after canonicalization — the `:22222` port in the scp-style URL is ignored,
  so it matches fine. A curl replay needs `X-Gitea-Event: push` and the
  signature header, or it answers "No matching application or invalid
  signature."
  Coolify's audit log (MCP `list_audit_events`, `source=webhook`) is the
  debugging oracle: `webhook.deployment.queued` = authenticated and queued,
  `webhook.gitea.signature_failed` = the Gitea hook's secret does not match
  the app's current `manual_webhook_secret_gitea`.
  **Gitea's hook edit API does not update the stored secret** — PATCHing the
  hook returns 200 but deliveries keep failing signature checks; delete the
  hook and create a fresh one with the secret instead (current hook: id 6,
  branch_filter `main`). The app's secret is regenerated whenever the app is
  recreated, so recreate the hook after any app recreation.
  Coolify's webhook deploy pins the pushed commit (`git fetch origin <sha>`),
  and this Gitea only serves the *current tip* by SHA
  (`uploadpack.allowReachableSHA1InWant` is off; other SHAs fail with
  "not our ref"). Deploys fail if a later push lands before the fetch —
  re-trigger the deploy. Proper fix, inside the Gitea container:
  `git -C /data/git/repositories/TZGyn/tunnel.git config
  uploadpack.allowReachableSHA1InWant true`.
  This Coolify version stores `custom_labels` **base64-encoded** and does
  *not* merge its generated Traefik labels for API-created apps: the full
  label set must live in `custom_labels` (mirror a working app's generated
  format — http/https routers with `certresolver=letsencrypt` — plus the TCP
  passthrough lines). **On 2026-10-25 the TCP lines were found missing after a
  deploy** (custom_labels held only HTTP/Caddy labels), so Traefik terminated
  tunnel SNI with its default certificate. Restored with
  `PATCH /api/v1/applications/{uuid}` (`custom_labels` only, current HTTP
  labels preserved) and verified they survive a redeploy; check them after any
  app update. Deployment JSON from the API can break `jq` (invalid
  escapes in the logs field) — parse with python instead.
- **Deployment**: the multi-stage `Dockerfile` builds a static non-root
  alpine image; verify `CGO_ENABLED=0 go build -trimpath -ldflags="-s -w"
  ./cmd/tunneld` after changing build-affecting code (no Docker runtime on
  the dev machine). Tunnel visitor traffic must reach the SNI listener
  without TLS termination (raw TCP, or a Traefik **TCP passthrough** router);
  the API domain may sit behind a TLS-terminating proxy. On a
  Coolify-managed host (Traefik owns 443/80): API = app domain → container
  :8080 with `HTTP_MODE=serve`; tunnel traffic = custom Traefik TCP labels
  (`HostSNIRegexp` + `tls.passthrough=true`) → container :8443. See README
  "Deploy on Coolify".

- **ACME / Porkbun** (M2): with `PORKBUN_API_KEY` and
  `PORKBUN_SECRET_API_KEY` set, the issuer is Let's Encrypt production and
  every tunnel is reissued (the old local-CA cert keeps serving until the new
  one is ready). The key used here is restricted to `tzgyn.dev`, but Porkbun
  also requires **API access opt-in per domain**; without it every DNS call
  fails with `API_ACCESS_DISABLED`. Challenge records are
  `_acme-challenge.<id>.<domain>` (one name for hostname + wildcard, up to two
  TXT values) and are deleted when the attempt ends. The ACME account key and
  URL are stored in the data file, so the persistent volume matters. Failed
  renewals keep the old certificate serving (`state=failed`, still `ready`
  on the wire) and retry every 6 hours. Use
  `ACME_DIRECTORY_URL=https://acme-staging-v02.api.letsencrypt.org/directory`
  for tests; staging certs are untrusted. Local ACME E2E is Pebble +
  pebble-challtestsrv with `DNS_PROVIDER=challtestsrv` and
  `ACME_CA_BUNDLE=.../test/certs/pebble.minica.pem`. The official CLI caches
  its certificate and polls `GET /api/tunnel/{id}/certificate` every 12 hours
  (every minute within 30 days of expiry), applying a renewed certificate to
  new connections without disconnecting; after switching the issuer, restart
  the service (`opentunnel down` / `up`) to fetch it immediately. An issuance
  killed mid-flight can leave `_acme-challenge` TXT records behind; delete them
  with `dns/deleteByNameType` (production and test tunnels share the zone).
  The server sweeps leftover `_acme-challenge.*` records at startup, which is
  safe only for a single server instance sharing the zone. Five consecutive
  CA-side failures pause new orders for 1 minute (doubling to 1 hour); DNS and
  validation errors do not trigger it. Rate limits default to 60 creates/hour
  per client IP and 30 binds/hour per IP / 10 per tunnel (`0` disables); the
  API is behind Traefik, so `X-Forwarded-For` is trusted for this.
- **macOS system curl (LibreSSL) does not send SNI for `*.localhost` names**,
  so `curl https://api.<id>.localhost:8443/` gets closed (the relay requires
  SNI). Test with `openssl s_client -servername ...` or Homebrew curl.
- JSON key order of API bodies is contractual — always `json.Marshal` structs
  with declared field order, never maps.
- Bridge writes use separate prioritized **control** and ordered **data**
  queues per session. `open`, `reset`, pings and pongs cut ahead of queued
  visitor data; `end` travels in the data queue so it cannot overtake the final
  response bytes. Visitor data backpressures the socket when the client is
  slow. Do not move `end` back to the control queue (that truncates responses).
- The native Go client's config is `~/.config/tunnel/config.json` (0600; it
  contains the tunnel token), with `private-key.pem`, `csr.pem`,
  `certificate.pem` and `chain.pem` next to it. `--config` selects another
  tunnel. `end` in its own bridge client is likewise ordered with data.
- Both clients claim routes on the same tunnel: stop the official
  `opentunnel` service before starting `tunnel`, or the second attach gets
  `route_conflict`.
- The bridge WebSocket must use `coder/websocket` with compression disabled
  and the `opentunnel` subprotocol; read limits are 64 KiB before attach and
  4+32 KiB after.
- No third-party deps beyond `github.com/coder/websocket` and
  `golang.org/x/crypto` (ACME); keep it that way unless a milestone calls for it.
- **CLI release**: `v0.1.0` (tag on `ebae420`) is published on Gitea with
  darwin/linux arm64+amd64 tarballs and `checksums.txt`. The Homebrew formula
  is `tzgyn-tunnel` in `TZGyn/homebrew-tap` (it installs the `tunnel` binary;
  the name avoids the deprecated core `tunnel` formula). Rebuild with
  `-X main.version=<version>`, attach all four assets, and update the four
  URLs/SHAs in the formula on each release. `tunnel up` currently starts a
  detached process, not a launchd agent (the official CLI's agent was stopped
  when the live tunnel was switched to this client).
- Milestones: M1 core data path (done); M2 real ACME DNS-01 + renewals (done);
  M3 rate limits/admin/hardening (done); M4 Go client (done). Remaining optional
  work: export/import admin endpoints, token rotation, multi-region mesh,
  inbound PROXY protocol, per-connection wire flow control (the protocol has
  none; queues are bounded).