tunnel/AGENTS.md

AGENTS.md

Single-node, self-hosted OpenTunnel-compatible server in Go. The wire protocol and API contract are the reference's, not ours to invent:

• Source of truth: .reference/opentunnel/docs/protocol.md
• API contract: .reference/opentunnel/crates/opentunnel-server/tests/fixtures/worker-contract.json (paths, status codes, bodies, JSON key order — installed clients depend on it)
• Conformance vectors: .reference/opentunnel/spec/vectors/*.json, run by internal/server/*_test.go (the reference checkout must exist; tests skip otherwise)

.reference/ is untracked research input (see .reference/opentunnel.md for what we take vs. skip). Never vendor it into the build.

Layout

• cmd/tunneld — flags/env wiring, listeners, signal handling
• cmd/tunnel — the native Go client/CLI: JSON config, provisioning, routes, background service
• internal/client — client implementation: config/API, key/CSR/certificate storage, bridge WebSocket, TLS termination and local proxying
• internal/server/limits.go, cooldown.go — per-IP/per-tunnel rate limits and the ACME CA cool-down
• internal/server/proto.go — wire types (pinned JSON key order), IDs/tokens (same generation as the reference: 12-char base32 slugs, rly_ tokens, hex SHA-256 token hashes), route validation, data frames
• internal/server/clienthello.go — TLS ClientHello parser (SNI + ALPN) that reads exact bytes only, so the connection can be replayed or forwarded
• internal/server/localca.go — the built-in insecure CA (the reference's ISSUER=local equivalent); signs tunnel CSRs after checking CN == hostname and SANs ⊆ {hostname, *.hostname}
• internal/server/issuer.go — the issuer interface plus the local-CA and ACME DNS-01 implementations (account persistence, order/challenge flow, propagation check, certificate for the client's CSR)
• internal/server/dns.go, dns_porkbun.go, dns_challtestsrv.go — the challenge DNS providers: Porkbun's API for production, pebble-challtestsrv for local ACME tests
• internal/server/renew.go — the renewal loop: 30-day window, 6-hour checks, retries, restart recovery, and local-CA → ACME migration
• internal/server/store.go — JSON-file store (0600, atomic rename); single writer, self-hosted scale
• internal/server/api.go — HTTP API per the contract; tunnel existence is checked before auth (404 before 401), body JSON syntax before everything
• internal/server/bridge.go — WebSocket bridge sessions: attach, route claiming (one owner per route), heartbeats, idle retirement, visitor pumps
• internal/server/admin.go — the admin dashboard handlers: GET /api/admin/overview (bearer ADMIN_TOKEN, 503 when unset, 401 when wrong) and the page/assets served from the embedded frontend
• web/ — the dashboard frontend: Svelte 5 (runes) + TypeScript, Vite via bun, pure CSS, no component libraries. bun run build emits to internal/dashboard/dist (gitignored; the committed .gitkeep keeps go:embed all:dist valid, and the handler answers 503 until built). internal/dashboard embeds the output; the Dockerfile builds it in a bun stage before the Go build
• internal/server/relay.go — public TLS listener: parses the ClientHello, terminates the domain itself, passes tunnel hostnames to bridges untouched

Commands

• go build ./... / go vet ./... / gofmt -l . — must stay clean
• go test ./...
• Frontend: cd web && bun install && bun run check && bun run build — the dashboard test skips when internal/dashboard/dist has no build (only the committed .gitkeep)
• Go client: go build -o .local/tunnel ./cmd/tunnel, then .local/tunnel config set-api <url> and .local/tunnel route add <port> --name <name>; up/down manage the background service, status shows local + server state. import --profile <name> --api <url> takes over an official CLI profile (same tunnel/key/cert).
• Dev loop: go build -o .local/tunneld ./cmd/tunneld && .local/tunneld -domain localhost, then OPENTUNNEL_API=http://localhost:8080 opentunnel route add 4096 --name api with the official CLI. go build drops a binary next to the source — it is gitignored; prefer -o .local/.

Gotchas

• Coolify deployment (verified 2026-10-25): app nokzgyrlhuuzduozr7yts4z3 in project Personal, https://tunnel.tzgyn.dev, deploy-key source on the shared gitea key, push to main auto-deploys via the Gitea webhook. Redeploy with POST /api/v1/deploy?uuid=<app> (the GET form is gone). Webhook details when debugging: Coolify's manual Gitea webhook verifies X-Hub-Signature-256 (HMAC-SHA256 hex of the raw body with the app's manual_webhook_secret_gitea; Gitea sends this header natively) and matches apps by repository.full_name against the app's git_repository after canonicalization — the :22222 port in the scp-style URL is ignored, so it matches fine. A curl replay needs X-Gitea-Event: push and the signature header, or it answers "No matching application or invalid signature." Coolify's audit log (MCP list_audit_events, source=webhook) is the debugging oracle: webhook.deployment.queued = authenticated and queued, webhook.gitea.signature_failed = the Gitea hook's secret does not match the app's current manual_webhook_secret_gitea. Gitea's hook edit API does not update the stored secret — PATCHing the hook returns 200 but deliveries keep failing signature checks; delete the hook and create a fresh one with the secret instead (current hook: id 6, branch_filter main). The app's secret is regenerated whenever the app is recreated, so recreate the hook after any app recreation. Coolify's webhook deploy pins the pushed commit (git fetch origin <sha>), and this Gitea only serves the current tip by SHA (uploadpack.allowReachableSHA1InWant is off; other SHAs fail with "not our ref"). Deploys fail if a later push lands before the fetch — re-trigger the deploy. Proper fix, inside the Gitea container: git -C /data/git/repositories/TZGyn/tunnel.git config uploadpack.allowReachableSHA1InWant true. This Coolify version stores custom_labels base64-encoded and does not merge its generated Traefik labels for API-created apps: the full label set must live in custom_labels (mirror a working app's generated format — http/https routers with certresolver=letsencrypt — plus the TCP passthrough lines). On 2026-10-25 the TCP lines were found missing after a deploy (custom_labels held only HTTP/Caddy labels), so Traefik terminated tunnel SNI with its default certificate. Restored with PATCH /api/v1/applications/{uuid} (custom_labels only, current HTTP labels preserved) and verified they survive a redeploy; check them after any app update. Deployment JSON from the API can break jq (invalid escapes in the logs field) — parse with python instead.

• Deployment: the multi-stage Dockerfile builds a static non-root alpine image; verify CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" ./cmd/tunneld after changing build-affecting code (no Docker runtime on the dev machine). Tunnel visitor traffic must reach the SNI listener without TLS termination (raw TCP, or a Traefik TCP passthrough router); the API domain may sit behind a TLS-terminating proxy. On a Coolify-managed host (Traefik owns 443/80): API = app domain → container :8080 with HTTP_MODE=serve; tunnel traffic = custom Traefik TCP labels (HostSNIRegexp + tls.passthrough=true) → container :8443. See README "Deploy on Coolify".

• ACME / Porkbun (M2): with PORKBUN_API_KEY and PORKBUN_SECRET_API_KEY set, the issuer is Let's Encrypt production and every tunnel is reissued (the old local-CA cert keeps serving until the new one is ready). The key used here is restricted to tzgyn.dev, but Porkbun also requires API access opt-in per domain; without it every DNS call fails with API_ACCESS_DISABLED. Challenge records are _acme-challenge.<id>.<domain> (one name for hostname + wildcard, up to two TXT values) and are deleted when the attempt ends. The ACME account key and URL are stored in the data file, so the persistent volume matters. Failed renewals keep the old certificate serving (state=failed, still ready on the wire) and retry every 6 hours. Use ACME_DIRECTORY_URL=https://acme-staging-v02.api.letsencrypt.org/directory for tests; staging certs are untrusted. Local ACME E2E is Pebble + pebble-challtestsrv with DNS_PROVIDER=challtestsrv and ACME_CA_BUNDLE=.../test/certs/pebble.minica.pem. The official CLI caches its certificate and polls GET /api/tunnel/{id}/certificate every 12 hours (every minute within 30 days of expiry), applying a renewed certificate to new connections without disconnecting; after switching the issuer, restart the service (opentunnel down / up) to fetch it immediately. An issuance killed mid-flight can leave _acme-challenge TXT records behind; delete them with dns/deleteByNameType (production and test tunnels share the zone). The server sweeps leftover _acme-challenge.* records at startup, which is safe only for a single server instance sharing the zone. Five consecutive CA-side failures pause new orders for 1 minute (doubling to 1 hour); DNS and validation errors do not trigger it. Rate limits default to 60 creates/hour per client IP and 30 binds/hour per IP / 10 per tunnel (0 disables); the API is behind Traefik, so X-Forwarded-For is trusted for this.

• macOS system curl (LibreSSL) does not send SNI for *.localhost names, so curl https://api.<id>.localhost:8443/ gets closed (the relay requires SNI). Test with openssl s_client -servername ... or Homebrew curl.

• JSON key order of API bodies is contractual — always json.Marshal structs with declared field order, never maps.

• Bridge writes use separate prioritized control and ordered data queues per session. open, reset, pings and pongs cut ahead of queued visitor data; end travels in the data queue so it cannot overtake the final response bytes. Visitor data backpressures the socket when the client is slow. Do not move end back to the control queue (that truncates responses).

• The native Go client's config is ~/.config/tunnel/config.json (0600; it contains the tunnel token), with private-key.pem, csr.pem, certificate.pem and chain.pem next to it. --config selects another tunnel. end in its own bridge client is likewise ordered with data.

• Both clients claim routes on the same tunnel: stop the official opentunnel service before starting tunnel, or the second attach gets route_conflict.

• The bridge WebSocket must use coder/websocket with compression disabled and the opentunnel subprotocol; read limits are 64 KiB before attach and 4+32 KiB after.

• No third-party deps beyond github.com/coder/websocket and golang.org/x/crypto (ACME); keep it that way unless a milestone calls for it.

• CLI release: v0.1.0 (tag on ebae420) is published on Gitea with darwin/linux arm64+amd64 tarballs and checksums.txt. The Homebrew formula is tzgyn-tunnel in TZGyn/homebrew-tap (it installs the tunnel binary; the name avoids the deprecated core tunnel formula). Rebuild with -X main.version=<version>, attach all four assets, and update the four URLs/SHAs in the formula on each release. tunnel up currently starts a detached process, not a launchd agent (the official CLI's agent was stopped when the live tunnel was switched to this client).

• Milestones: M1 core data path (done); M2 real ACME DNS-01 + renewals (done); M3 rate limits/admin/hardening (done); M4 Go client (done). Remaining optional work: export/import admin endpoints, token rotation, multi-region mesh, inbound PROXY protocol, per-connection wire flow control (the protocol has none; queues are bounded).