Add the native Go client and finish M3 hardening
The new cmd/tunnel client keeps the API URL and tunnel identity in ~/.config/tunnel/config.json (0600), provisions the key/CSR and certificate, terminates TLS, proxies to local routes, reloads routes without a restart, and runs as a background service. `tunnel import` takes over an existing official CLI profile, preserving its tunnel ID, token, key and certificate. Server hardening: per-IP and per-tunnel rate limits for creates and binds, a CA cool-down after repeated ACME failures, a startup sweep for stale _acme-challenge records, admin metrics, a 60-second half-close timeout, and separate prioritized control / ordered data bridge queues (with `end` ordered after the final bytes). The dashboard shows the new metrics.
This commit is contained in:
@@ -6,10 +6,9 @@ plaintext or private keys. It speaks the exact wire protocol from
|
||||
`docs/protocol.md`, so the official `opentunnel` CLI and SDK work against it
|
||||
unchanged.
|
||||
|
||||
Status: **M2** — core data path works end-to-end with the official CLI, and
|
||||
tunnels get publicly trusted certificates through ACME DNS-01 (Let's Encrypt
|
||||
+ Porkbun) with automatic renewal. The built-in local CA remains the default
|
||||
when no DNS provider is configured.
|
||||
Status: **M4** — the data path, real ACME DNS-01 certificates with renewal,
|
||||
rate limits/hardening, and a native Go client are implemented. The official
|
||||
`opentunnel` CLI still works unchanged.
|
||||
|
||||
## Quick start (local development)
|
||||
|
||||
@@ -40,6 +39,51 @@ printf 'GET / HTTP/1.1\r\nHost: api.<id>.localhost\r\nConnection: close\r\n\r\n'
|
||||
-CAfile ~/.local/share/tunneld/ca.pem -quiet
|
||||
```
|
||||
|
||||
## Go client
|
||||
|
||||
The repo includes a native Go client (`cmd/tunnel`) with a JSON config, so the
|
||||
API URL is stored once instead of typed on every command:
|
||||
|
||||
```bash
|
||||
go build -o .local/tunnel ./cmd/tunnel
|
||||
|
||||
.local/tunnel config set-api https://tunnel.tzgyn.dev
|
||||
.local/tunnel route add 4096 --name api
|
||||
.local/tunnel status
|
||||
```
|
||||
|
||||
The first `route add` creates a tunnel, generates the P-256 key/CSR, waits for
|
||||
the certificate, starts the background service, and prints the public URL.
|
||||
Routes reload without restarting the service.
|
||||
|
||||
| Command | |
|
||||
| --- | --- |
|
||||
| `tunnel config set-api <url>` | store the server URL in the JSON config |
|
||||
| `tunnel config show` | show API, tunnel, certificate and routes (never the token) |
|
||||
| `tunnel route add <port\|host:port> [--name NAME] [--proxy-protocol v1\|v2]` | add/update a route; random name if omitted |
|
||||
| `tunnel route remove <name>` / `route list` | manage routes |
|
||||
| `tunnel up` / `down` | start/stop the background service |
|
||||
| `tunnel serve` | run in the foreground |
|
||||
| `tunnel status` | local state, daemon, server state and certificate state |
|
||||
| `tunnel logs` | tail the daemon log |
|
||||
| `tunnel import --profile NAME --api URL` | take over an official CLI profile |
|
||||
|
||||
The config lives at `~/.config/tunnel/config.json` (0600; it also stores the
|
||||
tunnel token); the private key, CSR, certificate and chain sit next to it. Use
|
||||
`--config FILE` for a second independent tunnel.
|
||||
|
||||
### Taking over the official CLI profile
|
||||
|
||||
```bash
|
||||
tunnel import --profile coolify --api https://tunnel.tzgyn.dev
|
||||
opentunnel --profile coolify down # release the routes
|
||||
tunnel up
|
||||
```
|
||||
|
||||
`import` copies the tunnel ID, token, private key and certificate, and converts
|
||||
the official TOML routes to JSON. Both clients cannot claim the same routes at
|
||||
the same time — stop one before starting the other.
|
||||
|
||||
## Configuration
|
||||
|
||||
Every flag also reads an environment variable (`tunneld -h`):
|
||||
@@ -62,6 +106,9 @@ Every flag also reads an environment variable (`tunneld -h`):
|
||||
| `-acme-propagation-seconds` | `ACME_PROPAGATION_TIMEOUT_SECONDS` | `60` | how long to wait for challenge TXT records; `0` skips the check |
|
||||
| `-acme-ca-bundle` | `ACME_CA_BUNDLE` | *(empty)* | extra roots for the ACME server itself (Pebble) |
|
||||
| `-challtestsrv-url` | `CHALLTESTSRV_URL` | `http://127.0.0.1:8055` | pebble-challtestsrv management URL (dev only) |
|
||||
| `-create-limit-per-hour` | `TUNNEL_CREATE_LIMIT_PER_HOUR` | `60` | tunnels one client address may create per hour; `0` disables |
|
||||
| `-bind-limit-per-hour` | `CERT_BIND_LIMIT_PER_HOUR` | `30` | new certificate binds one client address may request per hour |
|
||||
| `-bind-limit-per-tunnel-per-hour` | `CERT_BIND_LIMIT_PER_TUNNEL_PER_HOUR` | `10` | new certificate binds one tunnel may request per hour |
|
||||
|
||||
## Certificates
|
||||
|
||||
@@ -103,6 +150,12 @@ Switching a running server from the local CA to ACME re-issues every existing
|
||||
tunnel automatically, keeping the old certificate in service until the new one
|
||||
is ready.
|
||||
|
||||
At startup the server removes leftover `_acme-challenge.*` TXT records from the
|
||||
zone (single-node assumption), and repeated CA-side failures trigger a
|
||||
cool-down from one minute up to an hour instead of hammering Let's Encrypt.
|
||||
New tunnel creation and CSR binds are rate-limited per client address and per
|
||||
tunnel; the limits can be tuned with the settings above.
|
||||
|
||||
For testing against Let's Encrypt staging (untrusted certificates, separate
|
||||
rate limits), set
|
||||
`ACME_DIRECTORY_URL=https://acme-staging-v02.api.letsencrypt.org/directory`.
|
||||
@@ -169,7 +222,7 @@ visitors must trust that CA. With Porkbun configured, every tunnel gets a
|
||||
publicly trusted Let's Encrypt certificate and no client or visitor trust
|
||||
setup is needed.
|
||||
|
||||
## What works (M1 + M2)
|
||||
## What works (M1–M4)
|
||||
|
||||
- The full HTTP API contract (paths, status codes, bodies, key order) as
|
||||
recorded in the reference's `worker-contract.json`
|
||||
@@ -179,24 +232,28 @@ setup is needed.
|
||||
- Per-tunnel Let's Encrypt certificates (`<id>.<domain>` + optional wildcard)
|
||||
that keep the client's key, with automatic renewal 30 days before expiry
|
||||
and retries on failure
|
||||
- Rate limits, CA cool-down, stale challenge-record sweep, and admin metrics
|
||||
- The bridge WebSocket: attach/attached, route claiming with conflicts,
|
||||
heartbeats, idle retirement, `open`/`end`/`reset`, 32 KiB data frames
|
||||
heartbeats, idle retirement, `open`/`end`/`reset`, 32 KiB data frames,
|
||||
prioritized control messages and ordered data
|
||||
- SNI routing on the TLS listener: the domain is terminated (API), tunnel
|
||||
hostnames are passed through still encrypted
|
||||
- Admin dashboard at `/` (bearer `ADMIN_TOKEN`): tunnels, bridges, cert
|
||||
state/expiry, issuer, CA expiry
|
||||
state/expiry, issuer, CA expiry, metrics
|
||||
- Persistence: JSON store with the CA, ACME account, server certificate, and
|
||||
tunnels
|
||||
- Native Go client (`cmd/tunnel`): JSON config, routes with live reload,
|
||||
PROXY protocol, background service, and import of the official CLI profile
|
||||
- Graceful shutdown: `drain` + close to every bridge
|
||||
|
||||
## Not yet (later milestones)
|
||||
## Not yet (later)
|
||||
|
||||
- Issuance rate limits, CA cool-down, stale-DNS-record sweeps
|
||||
- Metrics, more admin endpoints
|
||||
- Additional admin endpoints (export/import, token rotation)
|
||||
- Multi-region mesh, inbound PROXY protocol on the TLS listener
|
||||
- 60 s half-close timeout, per-connection flow control, control messages
|
||||
cutting ahead of queued data (single ordered outbound queue today)
|
||||
- Go client/CLI — for now use the official `opentunnel` CLI
|
||||
- Per-connection flow control on the wire (the protocol has none; both sides
|
||||
bound their queues)
|
||||
- Replacing the official CLI on every platform; the Go client is macOS/Linux
|
||||
(it uses POSIX process/session APIs for `up`/`down`)
|
||||
|
||||
## Development
|
||||
|
||||
|
||||
Reference in New Issue
Block a user