Skip to content

fix(ts2021): document and verify control-protocol upgrade handling #33

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jaxxstorm merged 1 commit into from
Apr 13, 2026
Merged

fix(ts2021): document and verify control-protocol upgrade handling #33

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,6 @@

A lightweight, preconfigured proxy for the Tailscale control plane that enables Tailscale access in the event the Tailscale control plane is being blocked.


[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/proxyt?referralCode=ftkvtR)

## Overview

Tailscale connections between peers are incredibly resiliant. If you are authenticated to Tailscale, it will endeavour to use all mechanisms at its disposal to forge connections it needs between clients.
Expand All @@ -13,6 +10,12 @@ However, some networks block the Tailscale _control plane_ (ie, the tailscale.co

Proxyt allows you to host a proxy to the Tailscale control plane which can be used by clients. You can host Proxyt anywhere, register your own domain or even expose it via [Funnel](https://tailscale.com/kb/1223/funnel) giving you a reliable way of accessing the Tailscale control plane to authenticate clients.

## Deployment Note

ProxyT depends on the frontend preserving Tailscale's `/ts2021` control-protocol upgrade request intact. Direct deployments, Tailscale Funnel, and traditional reverse proxies such as Nginx, Apache, and Caddy are the most reliable options.

Managed CDN and edge-proxy platforms that only support standard WebSocket `GET` handshakes, or that normalize non-standard upgrade traffic, are not compatible. In practice this means Cloudflare proxy/tunnel/workers are not supported, and platforms such as CloudFront, Fastly free tier, and Railway-style managed HTTP edges may fail depending on how their edge handles `POST` upgrades.
Comment on lines +15 to +17
Copy link

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The README now refers to the project as "ProxyT" here, but nearby text still uses "Proxyt" to refer to the same thing. Please standardize the product name throughout this document (and reserve proxyt for the CLI command) to avoid confusing readers.

Copilot uses AI. Check for mistakes.

## 📖 Documentation

**Full documentation:** [proxyt.io](https://proxyt.io)
Expand Down
40 changes: 36 additions & 4 deletions cmd/serve.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -539,7 +539,12 @@ func handleTailscaleControlProtocol(w http.ResponseWriter, r *http.Request) {
logDebugRequest("TS2021_HANDLER", r)
}

logger.Info("Handling Tailscale control protocol upgrade request", log.String("remote_addr", r.RemoteAddr))
logger.Info("Handling Tailscale control protocol upgrade request",
log.String("remote_addr", r.RemoteAddr),
log.String("method", r.Method),
log.String("host", r.Host),
log.String("connection", r.Header.Get("Connection")),
log.String("upgrade", r.Header.Get("Upgrade")))

// Check if we can hijack the connection immediately
hijacker, ok := w.(http.Hijacker)
Expand All @@ -554,7 +559,12 @@ func handleTailscaleControlProtocol(w http.ResponseWriter, r *http.Request) {
ServerName: "controlplane.tailscale.com",
})
if err != nil {
logger.Error("Error connecting to backend", log.Error(err))
logger.Error("Error connecting to backend",
log.String("method", r.Method),
log.String("path", r.URL.Path),
log.String("connection", r.Header.Get("Connection")),
log.String("upgrade", r.Header.Get("Upgrade")),
log.Error(err))
http.Error(w, "Bad Gateway", http.StatusBadGateway)
return
}
Expand All @@ -563,7 +573,10 @@ func handleTailscaleControlProtocol(w http.ResponseWriter, r *http.Request) {
// Write the original request to the backend
err = r.Write(backendConn)
if err != nil {
logger.Error("Error writing request to backend", log.Error(err))
logger.Error("Error writing request to backend",
log.String("method", r.Method),
log.String("path", r.URL.Path),
log.Error(err))
http.Error(w, "Bad Gateway", http.StatusBadGateway)
return
}
Expand All @@ -572,11 +585,22 @@ func handleTailscaleControlProtocol(w http.ResponseWriter, r *http.Request) {
reader := bufio.NewReader(backendConn)
resp, err := http.ReadResponse(reader, r)
if err != nil {
logger.Error("Error reading response from backend", log.Error(err))
logger.Error("Error reading response from backend",
log.String("method", r.Method),
log.String("path", r.URL.Path),
log.Error(err))
http.Error(w, "Bad Gateway", http.StatusBadGateway)
return
}

logger.Info("Received control protocol response from upstream",
log.String("method", r.Method),
log.String("path", r.URL.Path),
log.Int("status_code", resp.StatusCode),
log.String("status", resp.Status),
log.String("connection", resp.Header.Get("Connection")),
log.String("upgrade", resp.Header.Get("Upgrade")))

// Copy response headers to client
for name, values := range resp.Header {
for _, value := range values {
Expand Down Expand Up @@ -622,6 +646,14 @@ func handleTailscaleControlProtocol(w http.ResponseWriter, r *http.Request) {
return
}

logger.Error("Control protocol upgrade did not switch protocols",
log.String("method", r.Method),
log.String("path", r.URL.Path),
log.Int("status_code", resp.StatusCode),
log.String("status", resp.Status),
log.String("connection", resp.Header.Get("Connection")),
log.String("upgrade", resp.Header.Get("Upgrade")))

// For non-upgrade responses, copy the body normally
if resp.Body != nil {
io.Copy(w, resp.Body)
Expand Down
27 changes: 25 additions & 2 deletions cmd/serve_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -268,13 +268,20 @@ func TestBuildMainHandlerRewritesResponses(t *testing.T) {
}
}

func TestTS2021HandlerUsesInjectedDialer(t *testing.T) {
func TestTS2021HandlerPreservesMethodAndUpgradeHeaders(t *testing.T) {
withProxyTestGlobals(t)

var receivedMethod string
var receivedConnection string
var receivedUpgrade string

backend := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/ts2021" {
t.Fatalf("backend path = %q, want /ts2021", r.URL.Path)
}
receivedMethod = r.Method
receivedConnection = r.Header.Get("Connection")
receivedUpgrade = r.Header.Get("Upgrade")
Comment on lines +282 to +284
Copy link

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These shared variables are written from the backend httptest handler goroutine and later read from the test goroutine without synchronization, which is a data race (will be reported under go test -race). Capture the received values via a channel, mutex-protected struct, or sync/atomic to make the test race-free.

Copilot uses AI. Check for mistakes.
w.Header().Set("X-Upstream", "local-fake")
_, _ = w.Write([]byte("controlplane ok"))
}))
Comment on lines 285 to 287
Copy link

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fake upstream responds with a normal 200 OK body here, so this test doesn't exercise the real /ts2021 upgrade flow (101 Switching Protocols + hijack/tunneling). If the intent is to regression-test upgrade handling, consider making the upstream perform an actual 101 upgrade (via hijack) and asserting the proxy forwards the 101 and tunnels data.

Copilot uses AI. Check for mistakes.
Expand All @@ -289,7 +296,14 @@ func TestTS2021HandlerUsesInjectedDialer(t *testing.T) {
proxy := httptest.NewServer(buildMainHandler(nil))
t.Cleanup(proxy.Close)

resp, err := proxy.Client().Get(proxy.URL + "/ts2021")
req, err := http.NewRequest(http.MethodPost, proxy.URL+"/ts2021", nil)
if err != nil {
t.Fatalf("new request: %v", err)
}
req.Header.Set("Connection", "upgrade")
req.Header.Set("Upgrade", "tailscale-control-protocol")

resp, err := proxy.Client().Do(req)
if err != nil {
t.Fatalf("ts2021 request: %v", err)
}
Expand All @@ -304,6 +318,15 @@ func TestTS2021HandlerUsesInjectedDialer(t *testing.T) {
if body := readBody(t, resp.Body); body != "controlplane ok" {
t.Fatalf("body = %q, want controlplane ok", body)
}
if receivedMethod != http.MethodPost {
t.Fatalf("backend method = %q, want POST", receivedMethod)
}
if receivedConnection != "upgrade" {
t.Fatalf("backend Connection = %q, want upgrade", receivedConnection)
}
if receivedUpgrade != "tailscale-control-protocol" {
t.Fatalf("backend Upgrade = %q, want tailscale-control-protocol", receivedUpgrade)
}
}

func TestTS2021HandlerReturnsBadGatewayOnDialFailure(t *testing.T) {
Expand Down
35 changes: 31 additions & 4 deletions docs/hosting.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,20 @@
# Hosting

Railway is by far the easiest way to deploy Proxyt. It's preconfigured, and provides a valid domain and certificate for you.

[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/proxyt?referralCode=ftkvtR)
ProxyT works best when the frontend either terminates TLS directly on ProxyT or preserves arbitrary HTTP/1.1 upgrade requests all the way through to `/ts2021`.
## Compatibility Matrix

The Tailscale control protocol uses a non-standard upgrade flow on `/ts2021`. Some CDNs and managed reverse proxies only support standard WebSocket `GET` handshakes or specific upgrade tokens, which breaks login flows before the request reaches ProxyT.

| Frontend | Status | Notes |
| --- | --- | --- |
| Direct public host | Supported | Best option if you control ports 80/443 |
| Nginx / Apache / Caddy | Supported | Must preserve HTTP/1.1 upgrade semantics |
| Tailscale Funnel | Supported | Known-good path for exposing ProxyT |
| L4 TCP/TLS passthrough load balancer | Supported | Avoids HTTP upgrade rewriting at the edge |
| Railway / other managed HTTP edge proxies | Fragile / provider-dependent | Works only if the platform forwards `POST` upgrade requests unchanged |
| Cloudflare proxy / tunnel / workers | Not supported | Cloudflare does not support the Tailscale control protocol upgrade flow |
| AWS CloudFront | Not supported | CloudFront commonly drops or normalizes the `/ts2021` upgrade request |
| Fastly free tier | Not supported | Free-tier feature limits commonly block the required upgrade flow |

# Deployment Scenarios

Expand Down Expand Up @@ -48,6 +60,8 @@ proxyt serve --domain proxy.example.com --http-only --port 3000

Then configure your proxy to forward `https://proxy.example.com` to `http://localhost:3000`.

This only works when the proxy preserves `/ts2021` as an HTTP/1.1 upgrade request. If the provider rewrites or rejects `POST` upgrades, mobile and browser login will fail.

## Behind Traditional Reverse Proxy (Nginx/Apache)

```bash
Expand All @@ -56,6 +70,11 @@ proxyt serve --domain proxy.example.com --http-only --port 8080 --bind 127.0.0.1

**Nginx configuration example:**
```nginx
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}

server {
listen 443 ssl;
server_name proxy.example.com;
Expand All @@ -73,11 +92,19 @@ server {
# Important for Tailscale protocol upgrades
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Connection $connection_upgrade;
proxy_buffering off;
}
}
```

Important requirements for any reverse proxy:

- Preserve the original HTTP method for `/ts2021`
- Forward `Connection` and `Upgrade` headers unchanged
- Allow non-standard upgrade tokens used by the Tailscale client
- Do not force HTTP/2 or transform the request before it reaches ProxyT

## Docker Deployment

### Standalone with Let's Encrypt
Expand Down
18 changes: 17 additions & 1 deletion docs/troubleshooting.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,18 +16,27 @@
- Enable debug mode (`--debug`) to see detailed request logs
- Check that Tailscale client can resolve your domain
- Verify proxy can reach `*.tailscale.com` domains
- Verify your frontend actually forwards `/ts2021` to ProxyT

**HTTP-Only Mode Issues**
- Ensure your reverse proxy is properly forwarding X-Forwarded headers
- Verify the reverse proxy supports HTTP/1.1 upgrades for `/ts2021` endpoints
- Verify the reverse proxy preserves the original request method for `/ts2021`
- Check that the bind address and port are correct
- Confirm your reverse proxy is handling TLS termination properly

**Protocol Upgrade Failures**
- Ensure reverse proxy supports WebSocket/HTTP upgrades
- Check that `Connection: upgrade` and `Upgrade` headers are forwarded
- Check whether your provider only supports standard WebSocket `GET` handshakes
- Check whether your provider only allows `Upgrade: websocket` and rejects other upgrade tokens
- Verify no intermediate proxies are stripping upgrade headers

**Known Unsupported Frontends**
- Cloudflare proxy, Cloudflare Tunnel, and Cloudflare Workers are not supported for `/ts2021`
- CloudFront and some managed edge platforms may also fail because they do not forward the control-protocol upgrade intact
- If ProxyT works behind Nginx, Caddy, or Funnel but fails behind a CDN, the CDN is the likely incompatibility point

## Debug Mode

Enable debug logging to troubleshoot issues:
Expand All @@ -36,4 +45,11 @@ Enable debug logging to troubleshoot issues:
proxyt serve --domain proxy.example.com --email admin@example.com --cert-dir /tmp/certs --debug
```

This provides detailed request/response logging including headers and routing decisions.
This provides detailed request/response logging including headers and routing decisions.

For `/ts2021`, debug logs are most useful when they show:

- The incoming request method
- The `Connection` and `Upgrade` headers from the client-facing side
- Whether ProxyT received a `101 Switching Protocols` response from upstream
- Any non-101 status returned before tunneling started
Loading