Add the native Go client and finish M3 hardening

The new cmd/tunnel client keeps the API URL and tunnel identity in
~/.config/tunnel/config.json (0600), provisions the key/CSR and
certificate, terminates TLS, proxies to local routes, reloads routes
without a restart, and runs as a background service. `tunnel import`
takes over an existing official CLI profile, preserving its tunnel ID,
token, key and certificate.

Server hardening: per-IP and per-tunnel rate limits for creates and
binds, a CA cool-down after repeated ACME failures, a startup sweep for
stale _acme-challenge records, admin metrics, a 60-second half-close
timeout, and separate prioritized control / ordered data bridge queues
(with `end` ordered after the final bytes). The dashboard shows the new
metrics.
This commit is contained in:
2026-10-26 00:32:31 +08:00
parent cba7d2366b
commit ebae420ab1
27 changed files with 3745 additions and 56 deletions
+70 -13
View File
@@ -6,10 +6,9 @@ plaintext or private keys. It speaks the exact wire protocol from
`docs/protocol.md`, so the official `opentunnel` CLI and SDK work against it
unchanged.
Status: **M2** — core data path works end-to-end with the official CLI, and
tunnels get publicly trusted certificates through ACME DNS-01 (Let's Encrypt
+ Porkbun) with automatic renewal. The built-in local CA remains the default
when no DNS provider is configured.
Status: **M4** — the data path, real ACME DNS-01 certificates with renewal,
rate limits/hardening, and a native Go client are implemented. The official
`opentunnel` CLI still works unchanged.
## Quick start (local development)
@@ -40,6 +39,51 @@ printf 'GET / HTTP/1.1\r\nHost: api.<id>.localhost\r\nConnection: close\r\n\r\n'
-CAfile ~/.local/share/tunneld/ca.pem -quiet
```
## Go client
The repo includes a native Go client (`cmd/tunnel`) with a JSON config, so the
API URL is stored once instead of typed on every command:
```bash
go build -o .local/tunnel ./cmd/tunnel
.local/tunnel config set-api https://tunnel.tzgyn.dev
.local/tunnel route add 4096 --name api
.local/tunnel status
```
The first `route add` creates a tunnel, generates the P-256 key/CSR, waits for
the certificate, starts the background service, and prints the public URL.
Routes reload without restarting the service.
| Command | |
| --- | --- |
| `tunnel config set-api <url>` | store the server URL in the JSON config |
| `tunnel config show` | show API, tunnel, certificate and routes (never the token) |
| `tunnel route add <port\|host:port> [--name NAME] [--proxy-protocol v1\|v2]` | add/update a route; random name if omitted |
| `tunnel route remove <name>` / `route list` | manage routes |
| `tunnel up` / `down` | start/stop the background service |
| `tunnel serve` | run in the foreground |
| `tunnel status` | local state, daemon, server state and certificate state |
| `tunnel logs` | tail the daemon log |
| `tunnel import --profile NAME --api URL` | take over an official CLI profile |
The config lives at `~/.config/tunnel/config.json` (0600; it also stores the
tunnel token); the private key, CSR, certificate and chain sit next to it. Use
`--config FILE` for a second independent tunnel.
### Taking over the official CLI profile
```bash
tunnel import --profile coolify --api https://tunnel.tzgyn.dev
opentunnel --profile coolify down # release the routes
tunnel up
```
`import` copies the tunnel ID, token, private key and certificate, and converts
the official TOML routes to JSON. Both clients cannot claim the same routes at
the same time — stop one before starting the other.
## Configuration
Every flag also reads an environment variable (`tunneld -h`):
@@ -62,6 +106,9 @@ Every flag also reads an environment variable (`tunneld -h`):
| `-acme-propagation-seconds` | `ACME_PROPAGATION_TIMEOUT_SECONDS` | `60` | how long to wait for challenge TXT records; `0` skips the check |
| `-acme-ca-bundle` | `ACME_CA_BUNDLE` | *(empty)* | extra roots for the ACME server itself (Pebble) |
| `-challtestsrv-url` | `CHALLTESTSRV_URL` | `http://127.0.0.1:8055` | pebble-challtestsrv management URL (dev only) |
| `-create-limit-per-hour` | `TUNNEL_CREATE_LIMIT_PER_HOUR` | `60` | tunnels one client address may create per hour; `0` disables |
| `-bind-limit-per-hour` | `CERT_BIND_LIMIT_PER_HOUR` | `30` | new certificate binds one client address may request per hour |
| `-bind-limit-per-tunnel-per-hour` | `CERT_BIND_LIMIT_PER_TUNNEL_PER_HOUR` | `10` | new certificate binds one tunnel may request per hour |
## Certificates
@@ -103,6 +150,12 @@ Switching a running server from the local CA to ACME re-issues every existing
tunnel automatically, keeping the old certificate in service until the new one
is ready.
At startup the server removes leftover `_acme-challenge.*` TXT records from the
zone (single-node assumption), and repeated CA-side failures trigger a
cool-down from one minute up to an hour instead of hammering Let's Encrypt.
New tunnel creation and CSR binds are rate-limited per client address and per
tunnel; the limits can be tuned with the settings above.
For testing against Let's Encrypt staging (untrusted certificates, separate
rate limits), set
`ACME_DIRECTORY_URL=https://acme-staging-v02.api.letsencrypt.org/directory`.
@@ -169,7 +222,7 @@ visitors must trust that CA. With Porkbun configured, every tunnel gets a
publicly trusted Let's Encrypt certificate and no client or visitor trust
setup is needed.
## What works (M1 + M2)
## What works (M1–M4)
- The full HTTP API contract (paths, status codes, bodies, key order) as
recorded in the reference's `worker-contract.json`
@@ -179,24 +232,28 @@ setup is needed.
- Per-tunnel Let's Encrypt certificates (`<id>.<domain>` + optional wildcard)
that keep the client's key, with automatic renewal 30 days before expiry
and retries on failure
- Rate limits, CA cool-down, stale challenge-record sweep, and admin metrics
- The bridge WebSocket: attach/attached, route claiming with conflicts,
heartbeats, idle retirement, `open`/`end`/`reset`, 32 KiB data frames
heartbeats, idle retirement, `open`/`end`/`reset`, 32 KiB data frames,
prioritized control messages and ordered data
- SNI routing on the TLS listener: the domain is terminated (API), tunnel
hostnames are passed through still encrypted
- Admin dashboard at `/` (bearer `ADMIN_TOKEN`): tunnels, bridges, cert
state/expiry, issuer, CA expiry
state/expiry, issuer, CA expiry, metrics
- Persistence: JSON store with the CA, ACME account, server certificate, and
tunnels
- Native Go client (`cmd/tunnel`): JSON config, routes with live reload,
PROXY protocol, background service, and import of the official CLI profile
- Graceful shutdown: `drain` + close to every bridge
## Not yet (later milestones)
## Not yet (later)
- Issuance rate limits, CA cool-down, stale-DNS-record sweeps
- Metrics, more admin endpoints
- Additional admin endpoints (export/import, token rotation)
- Multi-region mesh, inbound PROXY protocol on the TLS listener
- 60 s half-close timeout, per-connection flow control, control messages
cutting ahead of queued data (single ordered outbound queue today)
- Go client/CLI — for now use the official `opentunnel` CLI
- Per-connection flow control on the wire (the protocol has none; both sides
bound their queues)
- Replacing the official CLI on every platform; the Go client is macOS/Linux
(it uses POSIX process/session APIs for `up`/`down`)
## Development