307 lines
14 KiB
Markdown
307 lines
14 KiB
Markdown
# tunnel
|
||
|
||
A single-node, self-hosted [OpenTunnel](https://github.com/anomalyco/opentunnel)-compatible
|
||
server in Go: blind TLS tunnels where the server routes by SNI and never sees
|
||
plaintext or private keys. It speaks the exact wire protocol from
|
||
`docs/protocol.md`, so the official `opentunnel` CLI and SDK work against it
|
||
unchanged.
|
||
|
||
Status: **M5** — the data path, real ACME DNS-01 certificates with renewal,
|
||
rate limits/hardening, a native Go client, and a cross-platform desktop UI are
|
||
implemented. The official `opentunnel` CLI still works unchanged.
|
||
|
||
## Quick start (local development)
|
||
|
||
```bash
|
||
go build -o .local/tunneld ./cmd/tunneld
|
||
.local/tunneld -domain localhost
|
||
# tls: listening on 127.0.0.1:8443, http: listening on 127.0.0.1:8080 (mode serve)
|
||
```
|
||
|
||
In another terminal, with the official CLI (`npm i -g opentunnel` or
|
||
`brew install anomalyco/tap/opentunnel`):
|
||
|
||
```bash
|
||
python3 -m http.server 4096 & # something to expose
|
||
export OPENTUNNEL_API=http://localhost:8080
|
||
opentunnel route add 4096 --name api
|
||
# https://api.<id>.localhost
|
||
```
|
||
|
||
The tunnel's TLS is terminated by the CLI with a certificate from the server's
|
||
local CA; the CA certificate is written next to the data file (`ca.pem`). Test
|
||
the visitor side with SNI (macOS curl skips SNI on `*.localhost` names, so use
|
||
`openssl s_client` or Homebrew curl):
|
||
|
||
```bash
|
||
printf 'GET / HTTP/1.1\r\nHost: api.<id>.localhost\r\nConnection: close\r\n\r\n' \
|
||
| openssl s_client -connect 127.0.0.1:8443 -servername api.<id>.localhost \
|
||
-CAfile ~/.local/share/tunneld/ca.pem -quiet
|
||
```
|
||
|
||
## Go client
|
||
|
||
The repo includes a native Go client (`cmd/tunnel`) with a JSON config, so the
|
||
API URL is stored once instead of typed on every command:
|
||
|
||
```bash
|
||
go build -o .local/tunnel ./cmd/tunnel
|
||
|
||
.local/tunnel config set-api https://tunnel.tzgyn.dev
|
||
.local/tunnel route add 4096 --name api
|
||
.local/tunnel status
|
||
```
|
||
|
||
Or install the released binary (installs a `tunnel` command):
|
||
|
||
```bash
|
||
brew tap TZGyn/tap https://gitea.tzgyn.dev/TZGyn/homebrew-tap.git
|
||
brew install tzgyn-tunnel
|
||
```
|
||
|
||
The first `route add` creates a tunnel, generates the P-256 key/CSR, waits for
|
||
the certificate, starts the background service, and prints the public URL.
|
||
Routes reload without restarting the service.
|
||
|
||
| Command | |
|
||
| --- | --- |
|
||
| `tunnel config set-api <url>` | store the server URL in the JSON config |
|
||
| `tunnel config show` | show API, tunnel, certificate and routes (never the token) |
|
||
| `tunnel route add <port\|host:port> [--name NAME] [--proxy-protocol v1\|v2]` | add/update a route; random name if omitted |
|
||
| `tunnel route remove <name>` / `route list` | manage routes |
|
||
| `tunnel up` / `down` | start/stop the background service |
|
||
| `tunnel serve` | run in the foreground |
|
||
| `tunnel status` | local state, daemon, server state and certificate state |
|
||
| `tunnel logs` | tail the daemon log |
|
||
| `tunnel import --profile NAME --api URL` | take over an official CLI profile |
|
||
|
||
The config lives at `~/.config/tunnel/config.json` (0600; it also stores the
|
||
tunnel token); the private key, CSR, certificate and chain sit next to it. Use
|
||
`--config FILE` for a second independent tunnel.
|
||
|
||
### Taking over the official CLI profile
|
||
|
||
```bash
|
||
tunnel import --profile coolify --api https://tunnel.tzgyn.dev
|
||
opentunnel --profile coolify down # release the routes
|
||
tunnel up
|
||
```
|
||
|
||
`import` copies the tunnel ID, token, private key and certificate, and converts
|
||
the official TOML routes to JSON. Both clients cannot claim the same routes at
|
||
the same time — stop one before starting the other.
|
||
|
||
## Desktop app
|
||
|
||
A cross-platform desktop UI (Fyne, macOS/Linux/Windows) is built into the same
|
||
client service:
|
||
|
||
```bash
|
||
go build -o .local/tunnel-desktop ./cmd/tunnel-desktop
|
||
.local/tunnel-desktop # click Start
|
||
.local/tunnel-desktop --autostart
|
||
```
|
||
|
||
On macOS, build and ad-hoc sign a launchable bundle:
|
||
|
||
```bash
|
||
./scripts/build-desktop-app.sh 0.2.0
|
||
ditto Tunnel.app "$HOME/Applications/Tunnel.app"
|
||
open "$HOME/Applications/Tunnel.app" --args --autostart
|
||
```
|
||
|
||
It uses `~/.config/tunnel/config.json` like the CLI, shows tunnel status and
|
||
certificate expiry, routes with copy/open/remove buttons, live logs, settings
|
||
and profile import, and has a start/stop button that runs the service
|
||
in-process. The Settings dialog has **Connect on launch**; add the app to
|
||
macOS Login Items (or the Linux/Windows equivalent) to reconnect at login. If
|
||
the CLI service is already running, a banner offers to stop it first.
|
||
|
||
Fyne needs CGO: the macOS build is tested; Linux and Windows must be built on
|
||
those platforms (or with a native CGO cross toolchain), not with
|
||
`CGO_ENABLED=0`. Distribution of the macOS app still needs Developer ID signing
|
||
and notarization (the local script only ad-hoc signs).
|
||
|
||
## Configuration
|
||
|
||
Every flag also reads an environment variable (`tunneld -h`):
|
||
|
||
| Flag | Env | Default | |
|
||
| --- | --- | --- | --- |
|
||
| `-domain` | `OPENTUNNEL_DOMAIN` | `localhost` | API domain; tunnels are `<id>.<domain>` |
|
||
| `-tls-listen` | `TLS_LISTEN` | `127.0.0.1:8443` | public SNI router |
|
||
| `-http-listen` | `HTTP_LISTEN` | `127.0.0.1:8080` | `off` disables |
|
||
| `-http-mode` | `HTTP_MODE` | `serve` | `serve` = plain-HTTP app (dev); `redirect` = 301 to the domain |
|
||
| `-data` | `TUNNELD_DATA` | `$XDG_DATA_HOME/tunneld/data.json` | tunnel store (0600) |
|
||
| `-ca-file` | `LOCAL_CA_FILE` | `<data dir>/ca.pem` | where the local CA cert is written |
|
||
| `-admin-token` | `ADMIN_TOKEN` | *(empty)* | bearer token for the admin dashboard; empty disables the admin API |
|
||
| `-dns-provider` | `DNS_PROVIDER` | *(auto)* | `porkbun` or `challtestsrv`; defaults to Porkbun when its keys are set |
|
||
| `-porkbun-api-key` | `PORKBUN_API_KEY` | *(empty)* | Porkbun API key for DNS-01 records |
|
||
| `-porkbun-secret-api-key` | `PORKBUN_SECRET_API_KEY` | *(empty)* | Porkbun secret API key |
|
||
| `-dns-zone` | `DNS_ZONE` | derived from `-domain` | zone holding the challenge records (`tzgyn.dev` for `tunnel.tzgyn.dev`) |
|
||
| `-acme-directory` | `ACME_DIRECTORY_URL` | Let's Encrypt production | ACME directory URL (use staging for tests) |
|
||
| `-acme-email` | `ACME_EMAIL` | *(empty)* | contact email registered with the ACME account |
|
||
| `-acme-propagation-seconds` | `ACME_PROPAGATION_TIMEOUT_SECONDS` | `60` | how long to wait for challenge TXT records; `0` skips the check |
|
||
| `-acme-ca-bundle` | `ACME_CA_BUNDLE` | *(empty)* | extra roots for the ACME server itself (Pebble) |
|
||
| `-challtestsrv-url` | `CHALLTESTSRV_URL` | `http://127.0.0.1:8055` | pebble-challtestsrv management URL (dev only) |
|
||
| `-create-limit-per-hour` | `TUNNEL_CREATE_LIMIT_PER_HOUR` | `60` | tunnels one client address may create per hour; `0` disables |
|
||
| `-bind-limit-per-hour` | `CERT_BIND_LIMIT_PER_HOUR` | `30` | new certificate binds one client address may request per hour |
|
||
| `-bind-limit-per-tunnel-per-hour` | `CERT_BIND_LIMIT_PER_TUNNEL_PER_HOUR` | `10` | new certificate binds one tunnel may request per hour |
|
||
|
||
## Certificates
|
||
|
||
Certificates have two modes, selected at startup:
|
||
|
||
- **Local CA** (default): signs instantly and needs no network. Clients must
|
||
trust `<data dir>/ca.pem` (`--ca-cert` / `OPENTUNNEL_CA_CERT`), and visitors
|
||
see a certificate warning.
|
||
- **ACME DNS-01** (Porkbun): when `PORKBUN_API_KEY` and
|
||
`PORKBUN_SECRET_API_KEY` are set (or `DNS_PROVIDER=porkbun`), tunneld orders
|
||
a per-tunnel certificate from Let's Encrypt covering `<id>.<domain>` and,
|
||
when the CSR asks for it, `*.<id>.<domain>`. The client keeps its private
|
||
key: it sends a CSR and tunneld returns the certificate for that key.
|
||
Issuance is asynchronous — the bind answers `issuing`, and the client polls
|
||
until `ready` or `failed` (typically 20–60 seconds).
|
||
|
||
### Porkbun setup
|
||
|
||
1. Porkbun → **Account → API Access** → create an API key and secret key.
|
||
Restrict the key to the zone when Porkbun offers it.
|
||
2. **Domain Management → tzgyn.dev → Details** → enable **API access** for the
|
||
domain (or turn on **Opt In All Domains**). Without this, reads fail with
|
||
`API_ACCESS_DISABLED`.
|
||
3. Set on the server (for example in Coolify's environment variables):
|
||
`PORKBUN_API_KEY`, `PORKBUN_SECRET_API_KEY`.
|
||
4. Redeploy. Startup logs show `dns: Porkbun zone tzgyn.dev` and
|
||
`certificate issuer: ACME https://acme-v02.api.letsencrypt.org/directory`.
|
||
5. Set `DNS_ZONE` only if the derived zone is wrong. For
|
||
`-domain tunnel.tzgyn.dev`, the derived zone is `tzgyn.dev`.
|
||
|
||
DNS-01 uses `_acme-challenge.<id>.<domain>`; the hostname and its wildcard
|
||
share that name, so an attempt creates up to two TXT values there. Records are
|
||
deleted when the attempt ends. The ACME account key is persisted in the data
|
||
file, so restarts reuse the same account.
|
||
|
||
Certificates are renewed once they have less than 30 days left, checked every
|
||
6 hours; a failed renewal keeps the old certificate serving and is retried.
|
||
Switching a running server from the local CA to ACME re-issues every existing
|
||
tunnel automatically, keeping the old certificate in service until the new one
|
||
is ready.
|
||
|
||
At startup the server removes leftover `_acme-challenge.*` TXT records from the
|
||
zone (single-node assumption), and repeated CA-side failures trigger a
|
||
cool-down from one minute up to an hour instead of hammering Let's Encrypt.
|
||
New tunnel creation and CSR binds are rate-limited per client address and per
|
||
tunnel; the limits can be tuned with the settings above.
|
||
|
||
For testing against Let's Encrypt staging (untrusted certificates, separate
|
||
rate limits), set
|
||
`ACME_DIRECTORY_URL=https://acme-staging-v02.api.letsencrypt.org/directory`.
|
||
|
||
The official CLI checks the certificate every 12 hours (every minute within
|
||
30 days of expiry) and applies a renewed certificate to new connections; after
|
||
switching the server's issuer, restart the CLI service (`opentunnel down` /
|
||
`opentunnel up`) to fetch the new certificate immediately.
|
||
|
||
## Admin dashboard
|
||
|
||
With `ADMIN_TOKEN` set, `https://<domain>/` (and `/admin`) serves the admin
|
||
dashboard: server uptime and version, CA expiry, tunnels with
|
||
state/routes/cert expiry, and attached bridges with client version and live
|
||
connection counts. It is a Svelte 5 + TypeScript app (pure CSS, no component
|
||
libraries) in `web/`, built with bun and embedded into the binary; it polls
|
||
`GET /api/admin/overview` (`Authorization: Bearer <ADMIN_TOKEN>`) every 5
|
||
seconds and stores the token in `localStorage`. Without `ADMIN_TOKEN` the
|
||
endpoint answers 503 and the page is inert; with a wrong token, 401.
|
||
|
||
```bash
|
||
cd web && bun install
|
||
bun run build # writes internal/dashboard/dist (gitignored), embedded by go:embed
|
||
bun run check # svelte-check
|
||
```
|
||
|
||
The Dockerfile builds the frontend in a bun stage before the Go build.
|
||
|
||
## Deploy on Coolify (or any Docker host)
|
||
|
||
The repo has a multi-stage `Dockerfile` producing a static, non-root
|
||
`alpine` image. Two topologies work; both need DNS for `tunnel.example.com`
|
||
**and** `*.tunnel.example.com` pointing at the host.
|
||
|
||
**A. Coolify-managed host (Traefik already owns 443/80).** The API domain is
|
||
served through Traefik (Let's Encrypt); tunnel traffic is passed through
|
||
untouched by SNI, because visitor TLS is terminated by the clients:
|
||
|
||
- App from this repo, Dockerfile build pack, private repo → deploy key.
|
||
- Domain `https://tunnel.example.com`; `ports_exposes=8080` (Traefik routes
|
||
HTTPS → the plain-HTTP API port).
|
||
- Env: `OPENTUNNEL_DOMAIN=tunnel.example.com`, `HTTP_MODE=serve` (the API
|
||
sits behind Traefik; it must not redirect to HTTPS itself), and, for real
|
||
certificates, `PORKBUN_API_KEY` + `PORKBUN_SECRET_API_KEY` (see
|
||
[Certificates](#certificates)).
|
||
- Custom Traefik labels (Coolify stores them base64-encoded):
|
||
`traefik.tcp.routers.tunnel-data.entrypoints=https`,
|
||
`...rule=HostSNIRegexp(`^.+\.tunnel\.example\.com$`)`,
|
||
`...tls.passthrough=true`,
|
||
`traefik.tcp.services.tunnel-data.loadbalancer.server.port=8443`.
|
||
After any app update or recreation, confirm the TCP labels are still in
|
||
`custom_labels`: Coolify has dropped them when regenerating labels, leaving
|
||
Traefik to terminate tunnel SNI with its default certificate.
|
||
- Persistent volume on `/app/data`; healthcheck on `/health`.
|
||
|
||
**B. Plain Docker host.** tunneld owns the public ports itself: map host
|
||
`443` → container `8443` and `80` → container `8080` (`HTTP_MODE=redirect`
|
||
serves `/health` on 80 and 301s the rest), volume on `/app/data`. No
|
||
TLS-terminating proxy may sit in front of the tunnel ports.
|
||
|
||
Without ACME credentials, certificates come from the server's built-in local
|
||
CA: clients need `--ca-cert <data dir>/ca.pem` (or `OPENTUNNEL_CA_CERT`), and
|
||
visitors must trust that CA. With Porkbun configured, every tunnel gets a
|
||
publicly trusted Let's Encrypt certificate and no client or visitor trust
|
||
setup is needed.
|
||
|
||
## What works (M1–M4)
|
||
|
||
- The full HTTP API contract (paths, status codes, bodies, key order) as
|
||
recorded in the reference's `worker-contract.json`
|
||
- Provisioning: tunnel creation, CSR bind with the reference's hostname rules,
|
||
asynchronous issuance states (`issuing`/`ready`/`failed`), the local CA, and
|
||
ACME DNS-01 through Porkbun
|
||
- Per-tunnel Let's Encrypt certificates (`<id>.<domain>` + optional wildcard)
|
||
that keep the client's key, with automatic renewal 30 days before expiry
|
||
and retries on failure
|
||
- Rate limits, CA cool-down, stale challenge-record sweep, and admin metrics
|
||
- The bridge WebSocket: attach/attached, route claiming with conflicts,
|
||
heartbeats, idle retirement, `open`/`end`/`reset`, 32 KiB data frames,
|
||
prioritized control messages and ordered data
|
||
- SNI routing on the TLS listener: the domain is terminated (API), tunnel
|
||
hostnames are passed through still encrypted
|
||
- Admin dashboard at `/` (bearer `ADMIN_TOKEN`): tunnels, bridges, cert
|
||
state/expiry, issuer, CA expiry, metrics
|
||
- Persistence: JSON store with the CA, ACME account, server certificate, and
|
||
tunnels
|
||
- Native Go client (`cmd/tunnel`): JSON config, routes with live reload,
|
||
PROXY protocol, background service, and import of the official CLI profile
|
||
- Desktop UI (`cmd/tunnel-desktop`): Fyne app with status, routes, logs,
|
||
settings, and start/stop
|
||
- Graceful shutdown: `drain` + close to every bridge
|
||
|
||
## Not yet (later)
|
||
|
||
- Additional admin endpoints (export/import, token rotation)
|
||
- Multi-region mesh, inbound PROXY protocol on the TLS listener
|
||
- Per-connection flow control on the wire (the protocol has none; both sides
|
||
bound their queues)
|
||
- Replacing the official CLI on every platform; the Go client is macOS/Linux
|
||
(it uses POSIX process/session APIs for `up`/`down`)
|
||
|
||
## Development
|
||
|
||
```bash
|
||
go build ./...
|
||
go test ./... # includes the shared spec vectors from .reference/opentunnel
|
||
```
|
||
|
||
The reference implementation lives in `.reference/` (untracked); see
|
||
`.reference/opentunnel.md` for what we take from it and what we skip.
|