I use both every day. Ollama runs on my laptop and on the GPU workstation in the office, for development and for my own tools. vLLM serves a chat model from a single 96 GB GPU for a freelance project, first under Docker Compose and now in a small Kubernetes cluster. This post covers what each one is good at, how to install them, and the things that cost me time on the vLLM side.

vLLM vs Ollama at a glance

#

What Ollama is

#

Ollama is the easiest way I know to get a model answering on your own machine. It grew out of llama.cpp and runs quantised GGUF models, which is why an 8B model fits in about 5 GB and runs happily on a MacBook. You pull a model by name, it loads the first time you call it, and it drops out of memory after five minutes with nothing to do, so it shares a machine politely with everything else you have running.

That politeness is the trade-off. Ollama is built around one person at a time. It can answer a few requests in parallel, but every parallel slot costs memory, and it was never meant to batch dozens of users onto one GPU.

What vLLM is

#

vLLM is a serving engine, and it’s built for exactly that batching. Two ideas carry most of the weight. PagedAttention stores each request’s KV cache (the attention state a model builds up as it reads and writes tokens) in small fixed-size blocks, so GPU memory isn’t lost to padding and fragmentation. Continuous batching lets new requests join the batch between generation steps instead of waiting for the current batch to finish. Between them, one GPU running vLLM serves far more concurrent users than the same GPU working through requests one at a time.

The cost is that vLLM behaves as if it owns the card. It claims its share of GPU memory when it starts, fills whatever the weights don’t need with KV cache, and holds all of it whether anyone is calling or not. It also wants full Hugging Face weights or a GPU-friendly quantisation like FP8, not the GGUF files Ollama uses.

One API, two servers

#

Because both expose the OpenAI chat completions API, my code doesn’t know which one it’s talking to. The URL and model name come from the environment:

import os
from openai import OpenAI

# Laptop:     LLM_URL=http://localhost:11434/v1  LLM_MODEL=qwen3:8b
# GPU server: LLM_URL=http://localhost:8000/v1   LLM_MODEL=qwen3-8b
client = OpenAI(base_url=os.environ["LLM_URL"], api_key="not-needed")

reply = client.chat.completions.create(
    model=os.environ["LLM_MODEL"],
    messages=[{"role": "user", "content": "Say hello in Korean."}],
)
print(reply.choices[0].message.content)

On my laptop, which has no big GPU, the app points at Ollama. On the server it points at vLLM. Nothing else changes, and the app can be developed without the GPU box switched on at all. On the vLLM side, --served-model-name gives the model a short name, so clients don’t have to send the full Hugging Face path.

One caution, because it’s easy to forget: the same model on both doesn’t give the same answers. Ollama’s qwen3:8b is a 4-bit quantisation, while on vLLM you’d usually run 16-bit or FP8 weights. A prompt tuned against one can behave differently on the other, so anything you’re measuring, whether that’s answer quality, refusals or whether the JSON parses, has to be checked against the server production will actually use.

How to install Ollama

#

On a Mac or Windows, download the app from ollama.com. On Linux:

curl -fsSL https://ollama.com/install.sh | sh
ollama run qwen3:8b

That’s the whole job. The API is now at http://localhost:11434, and it only listens on localhost unless you set OLLAMA_HOST, which is what you want. Mine runs on the office workstation bound to 127.0.0.1, and I reach it from anywhere through a tunnel (how I set that up with Cloudflare WARP).

How to install vLLM

#

On Linux with an NVIDIA GPU and a recent driver:

python -m venv .venv && source .venv/bin/activate
pip install vllm
vllm serve Qwen/Qwen3-8B --served-model-name qwen3-8b --max-model-len 16384

The first run downloads the weights from Hugging Face. Once the log says Application startup complete, curl localhost:8000/v1/models lists the model, and the Python above works against it.

Running vLLM in Docker

#

For anything that has to stay up, I use the official image rather than a pip install, mostly so CUDA, PyTorch and vLLM come pinned together. A generic Compose service:

services:
  vllm:
    image: vllm/vllm-openai:v0.30.0   # pin a tag; the image is about 30 GB
    restart: unless-stopped
    ipc: host                          # PyTorch needs shared memory
    command: >
      --model Qwen/Qwen3-8B
      --served-model-name qwen3-8b
      --gpu-memory-utilization 0.85
      --max-model-len 16384
    volumes:
      - ${HOME}/.cache/huggingface:/root/.cache/huggingface
    ports:
      - "127.0.0.1:8000:8000"          # reach it over ssh -L, not the LAN
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    healthcheck:
      test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"]
      interval: 10s
      timeout: 5s
      retries: 90
      start_period: 120s

Pin the tag. vLLM moves quickly, flags and defaults change between releases, and you want to be the one who decides when that happens. ipc: host (or a generous shm_size) is in vLLM’s own Docker instructions. The long start_period is the first of the things below.

What bit me running vLLM

#

Startup takes minutes. Loading the weights, compiling and capturing CUDA graphs took 190 seconds on my server, and three to five minutes is normal for a large model. A health check written for a web app will kill it before it’s ready. In Compose that means a long start_period; in Kubernetes it means a startup probe. It also means every restart is a few minutes with no model, so restarts shouldn’t be casual.

It takes the GPU memory you give it, at startup. --gpu-memory-utilization is the share of the card vLLM may use, 0.92 by default in recent releases, and whatever the weights don’t need becomes KV cache. That’s fine when vLLM is alone on the card. Mine isn’t, because smaller models run beside it, so I set 0.70 on the 96 GB card. The model, a 35B mixture-of-experts in FP8, loaded at 47.7 GiB and left 15 GiB for KV cache, which is plenty at a 16k context. If vLLM refuses to start because the maximum sequence length won’t fit in the KV cache, lower --max-model-len before you go looking for a bigger GPU.

LoRA slots cost memory up front. vLLM can serve one base model with several LoRA adapters and switch between them per request, by model name, which Ollama can’t really do. But it reserves room for --max-loras adapters at --max-lora-rank when it starts, and on a mixture-of-experts model that reservation covers the experts too, so it adds up fast. Four adapters at rank 64 didn’t fit at 0.70. Two did.

The default sequence limit didn’t suit a hybrid model. The model I serve mixes Mamba-style layers in with attention, and each sequence needs its own cache block for those layers. At vLLM’s default --max-num-seqs on that card it failed to start; --max-num-seqs 256 fixed it. If a newer architecture won’t start, check this flag before anything more exotic.

A Kubernetes Service named vllm crashes vLLM. This one’s my favourite. Kubernetes injects environment variables for every Service in the namespace, named after the Service, so a Service called vllm hands every pod VLLM_PORT=tcp://<cluster-ip>:8000. vLLM reads VLLM_PORT as one of its own settings, couldn’t make sense of a URL there, and died. After loading the whole model, naturally, so every attempt cost three minutes. vLLM’s docs warn about this, which I found afterwards. The fix is enableServiceLinks: false on the pod, or a Service with a different name.

Here’s the shape of a Deployment that works, cut down to the parts that matter:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: vllm
spec:
  replicas: 1
  strategy:
    type: Recreate               # the old pod has to free the GPU first
  selector:
    matchLabels: {app: vllm}
  template:
    metadata:
      labels: {app: vllm}
    spec:
      enableServiceLinks: false  # no VLLM_PORT=tcp://... from the Service
      containers:
        - name: vllm
          image: vllm/vllm-openai:v0.30.0
          args: ["--model=Qwen/Qwen3-8B", "--served-model-name=qwen3-8b"]
          resources:
            limits:
              nvidia.com/gpu: "1"
              memory: 32Gi       # loading needs far more host RAM than serving
          volumeMounts:
            - {name: shm, mountPath: /dev/shm}
          startupProbe:          # don't judge it until the model is loaded
            httpGet: {path: /health, port: 8000}
            initialDelaySeconds: 120
            periodSeconds: 10
            failureThreshold: 90
          livenessProbe:         # a restart costs a full reload, so be patient
            httpGet: {path: /health, port: 8000}
            periodSeconds: 30
            timeoutSeconds: 5
            failureThreshold: 10
      volumes:
        - name: shm
          emptyDir: {medium: Memory, sizeLimit: 16Gi}

The probes are the part worth copying. The startup probe waits two minutes, then gives the model another fifteen to come up before Kubernetes calls it dead. The liveness probe only restarts it after about five minutes of failures, because /health can be slow on a busy engine and a restart costs a full reload anyway. Recreate matters on a single GPU: a rolling update starts the new pod first, and it sits there waiting for a GPU the old pod still holds. The memory limit looks generous because it is; mine settles at about 7 GiB of host RAM once the weights are on the GPU, but the load itself needs a lot more.

What to know about Ollama

#

Fewer surprises, mostly because it does less.

The first request after a quiet spell is slow, because the model was unloaded and has to load again. OLLAMA_KEEP_ALIVE sets how long a model stays in memory after its last request; raise it if that pause annoys you, or set it to -1 on a machine that does nothing else.

The context window depends on how much GPU memory you have. Under 24 GiB, Ollama defaults to 4k tokens, far less than most models support, and a prompt that doesn’t fit is trimmed quietly rather than refused. If answers seem to ignore the start of a long prompt, set OLLAMA_CONTEXT_LENGTH (or num_ctx per request) to what you actually need.

If you forward a remote Ollama to your laptop, don’t put it on 11434. That’s Ollama’s default port, and if a local Ollama ever starts, the two race for it and you can end up talking to the wrong one without noticing. I forward mine to 11435.

And it isn’t a multi-user server. It’s fine for an internal tool a handful of people use; once real concurrency turns up, that’s the point to move to vLLM.

How much GPU memory you need

#

Rough sizes for the weights alone:

That’s about 2 bytes per parameter at 16-bit, 1 at 8-bit, and a little over half a byte at Q4_K_M. On top of the weights goes the KV cache, which grows with context length and with how many requests are in flight. That second part is why the same model needs much more memory on vLLM serving fifty users than on Ollama serving you.

Which one to use

#

If you want a model on your own machine, install Ollama and stop there; you’ll have something answering in ten minutes. If you have a GPU server and users, or an app, that will call it all day, use vLLM, accept the slow start and the memory it takes, and get the probes right. Plenty of setups end up with both, mine included: Ollama where the code gets written, vLLM where the requests land.

There are other options. SGLang is the usual alternative to vLLM, but I haven’t run it in production, so I’ve left it out rather than guess. And if the model has to run on a phone, that’s llama.cpp again, which I wrote about in running an LLM on-device in React Native.

My working setup is three machines that are rarely in the same place. There’s a MacBook Air, which goes wherever I go. There’s a workstation in the office called devbox, with a real GPU, which runs Ollama, my fine-tuning tools, a small Kubernetes cluster with its own container registry, and whatever agent I’ve left grinding through a long job. And there’s my phone, which is where the apps I build actually get used, and which spends most of its day on mobile data rather than wifi.

For a long time those three didn’t really talk to each other. The laptop could reach devbox when it was on the office LAN and not otherwise. The phone could reach my laptop’s dev server when both were on the same wifi, and then only if I remembered to bind to 0.0.0.0 and typed in whatever DHCP had given the Mac that morning. Everything interesting on the workstation, the Ollama API, the cluster dashboards, the traces, the registry, was bound to localhost or to the cluster network, deliberately, because none of it belongs on the internet.

This post is how I joined them up with Cloudflare WARP and a pair of Cloudflare Tunnels, what it changed about how I build AI features and mobile apps, and the handful of things that broke without telling me.

What I actually wanted

#

Stated as requirements, it’s short:

1. The laptop should be a thin client. Anything that needs a GPU, a lot of RAM or hours of runtime should run on devbox, and the laptop should reach it the same way from the office, from home and from a café.
2. The phone should reach my dev servers and my shells from anywhere, on mobile data, without a VPN I have to remember to switch on and off.
3. The things I use to see how a system is behaving, dashboards, traces, docker info, model lists, should be reachable by me and nobody else, and should never get a public URL.
4. No inbound ports. Not on the office router, not on my home router, not anywhere.

The fourth one rules out the old answer, which is a port forward and a dynamic DNS name. It also rules out a lot of cleverness: both devbox and my laptop sit behind NAT I don’t control, and the laptop is behind a different NAT every few hours.

Before: a repo of SSH tunnels

#

The first version of this was a small repo of my own, and it’s still the base layer. It solves the “clean names for things I reach over SSH” problem on the laptop and nothing more. dnsmasq answers *.test with 127.0.0.1, Caddy routes by hostname to a local port with tls internal so the browser is happy, and a set of autossh launch agents hold the SSH forwards open and reconnect when they drop. The hostnames say where a thing lives:

So https://devbox-ollama.ssh.test/api/tags lists the models on the workstation, and https://api.local.test is the API of the app I’m working on, running on my own machine.

Subdomains of .test, not new top-level names like .ssh or .cloud, and not .local. RFC 6761 reserves .test for exactly this, and .local belongs to mDNS; take it over and Bonjour discovery stops working on your Mac. dnsmasq already wildcards the whole of *.test, so the extra label costs nothing: one /etc/resolver/test file pointing at 127.0.0.1 and no further DNS changes, ever.

HTTPS on names that don’t exist

#

Every one of those names is served over HTTPS with a certificate the browser trusts, which sounds impossible for a hostname that exists only on my laptop. Caddy does it with its own internal certificate authority. tls internal on a site tells Caddy to issue that site’s certificate from a local CA instead of asking Let’s Encrypt, and caddy trust, run once by the installer, puts the CA’s root into the macOS System keychain (Chrome and Safari read it) and into Firefox’s own store. From then on https://anything.test opens with no certificate warning, plain http:// redirects to it, and nothing leaves the machine.

It isn’t just cosmetic. A lot of the web platform now refuses to work on plain HTTP: secure cookies, service workers, the clipboard API, anything with SameSite=None. Testing a web build on plain http://localhost hides the bugs that show up on the real HTTPS domain. Testing on https://app.local.test surfaces them on my laptop instead.

A site block, which is the same for every service apart from the port:

devbox-ollama.ssh.test, devbox-ollama.test {
	tls internal
	import cors
	reverse_proxy 127.0.0.1:11435 {
		import proxyhdr
		transport http {
			keepalive off
			dial_timeout 5s
		}
	}
}

The two snippets carry the fiddly parts. cors reflects the caller’s Origin back rather than sending *, because * isn’t allowed once credentials are involved, and it answers OPTIONS preflights itself with a 204 so the upstream never has to. proxyhdr rewrites the Host header to the dial address. Fussy backends, router admin pages and Chrome’s remote-debugging endpoint among them, validate Host, and would otherwise reject devbox-ollama.ssh.test as a name they’ve never heard of. Ollama’s own origin check accepts loopback on any port, so rewriting Host to 127.0.0.1:11435 gets it through.

keepalive off is the one I’d have got wrong. The upstreams are SSH forwards that autossh tears down and rebuilds whenever the network changes. With keep-alive on, Caddy holds a pooled connection to a tunnel that has since died, and it returns baffling 404s until it’s reloaded. Off, every request dials fresh, which on loopback costs nothing measurable.

Each service also answers on its old flat name (devbox-ollama.test), so bookmarks from before the suffix scheme still work.

The menu bar

#

With a dozen forwards held open by launchd, the question I kept asking was “is it up?”, and the answer lived in three places: launchctl list, lsof and a curl. So a small menu-bar plugin puts it in one place, with a dot next to every hostname:

• 🟢 the service answered a request through the tunnel
• 🟡 the tunnel is open but the service behind it is silent
• 🔴 nothing is listening on the local port, so the tunnel itself is down

The yellow state is the useful one. A green/red check can’t tell “the SSH forward died” from “Ollama crashed on the far end”, and those have entirely different fixes. The plugin keeps no list of its own; it reads the same proxy and SSH config as everything else, so adding a service there adds it to the menu too.

The first version was greedy. It refreshed every ten seconds and spawned a few processes per service, about seventy a run, and macOS promptly listed it under “apps using significant energy”. Taking one snapshot per run and refreshing every few minutes brought it down to about 0.1% of a core. A status light that drains the battery is not helping, so it’s worth budgeting for from the start.

The whole setup had one hole, and it was the one that mattered. devbox has no public name. Its SSH forward dialled a 192.168.x address, so it only worked while the laptop was on the office LAN. Away from the office the menu bar went red and stayed red, and everything I’d moved onto the workstation to save the laptop was out of reach at exactly the times I wasn’t sitting next to it.

Two tunnels, pointing the right way

#

The fix has three parts, and the first two are Cloudflare Tunnels.

A Cloudflare Tunnel is cloudflared holding an outbound connection to Cloudflare’s edge. Cloudflare hands requests back down that connection, so a machine behind any amount of NAT gets reachable without anything listening on a router. It follows a laptop onto hotel wifi and it survives CGNAT.

I run two of them, one per machine, and the reason is worth a sentence. The obvious shortcut was a single tunnel on the laptop with one more ingress rule pointing at devbox’s SSH port on the office LAN. That needs nothing installed on devbox at all. It also routes through the laptop, and the laptop is the machine that leaves the office. It would have broken in exactly the case it existed to fix. devbox stays put, so devbox holds its own connection to the edge.

# config.yml on devbox (runs as a systemd service)
protocol: auto        # prefer QUIC, fall back to HTTP/2 over TCP 443
ha-connections: 4     # mains-powered, its job is to stay reachable

warp-routing:
  connectTimeout: 5s

ingress:
  - hostname: ssh-devbox.example.com
    service: tcp://127.0.0.1:22
  - service: http_status:404

tcp:// rather than ssh:// is an easy one to get wrong. ssh:// selects Cloudflare’s browser-rendered terminal, which wraps the stream for a web page; a normal ssh client fed through it gets the banner and then dies at key exchange. tcp:// is the plain byte pipe cloudflared access ssh expects, and as a bonus it carries scp, rsync, git over SSH and port forwards.

That last part is what closed the hole in the SSH layer. The SSH config now picks a route per connection:

Match originalhost devbox !exec "nc -z -G 1 <office-lan-ip> 22 >/dev/null 2>&1"
    ProxyCommand cloudflared access ssh --hostname ssh-devbox.example.com ...

On the office LAN the probe succeeds and SSH goes direct, one hop. Anywhere else it fails and SSH goes through Cloudflare, authenticated with an Access service token so there’s no browser login in the way. It’s Match originalhost, not Match host: host matches after HostName has been substituted, so by then the name is already an IP and the block never fires. The upshot is that devbox-ollama.ssh.test works in the office, at home and on a train, and the menu bar dot stays green.

The laptop’s tunnel goes the other direction. It gives dev servers on the Mac real public HTTPS hostnames: for each app I’m working on, its API, its WebSocket server and its web build. Every one of them sits behind Cloudflare Access, which I’ll come back to, because it’s the part that makes this safe rather than reckless.

The menu bar learned about both tunnels too, and it couldn’t do it by probing. An inbound tunnel has no local port to check, one of the two connectors doesn’t even run on the laptop, and the obvious test, curling ssh-devbox.example.com, gets an answer from Cloudflare’s edge whether or not a connector is attached, so it would read green on a dead tunnel. launchd is no better: cloudflared sits there “loaded, exit 0” with every edge connection dropped. So for tunnels the menu asks the Cloudflare API whether each one has a healthy connector, since that’s what decides whether an ssh lands. If the API can’t be reached, the section is left out rather than drawn red. The menu is read at a glance, and a false alarm there costs more than a missing line.

The third part: WARP makes it a network

#

Tunnels with hostnames are great for anything that speaks HTTPS, and for SSH from a machine that can run cloudflared. A phone can’t. And a tunnel hostname isn’t a TCP endpoint: Cloudflare’s edge only terminates HTTPS and WebSocket on it, so port 22 at ssh.example.com is closed and no amount of fiddling with Access will open it.

That’s the job for WARP. The Cloudflare One Agent (the enterprise WARP client, not the consumer “1.1.1.1” app, which can’t join an organisation) is enrolled on the laptop and the phone in my Zero Trust organisation. Each tunnel advertises private routes, and a device running WARP can then open a plain socket to those addresses and land on the right machine. It stops being “reach this service by name” and becomes “these machines are on a network with me”:

Those addresses are loopback aliases on each machine, created at boot, so they owe nothing to DHCP or whatever router I’m behind. 198.18.42.2 is devbox in the office, from home, and from a phone on 5G. The Mac’s real LAN address changes every time I move; this one never does.

The split tunnel is in include mode with only those routes in it (plus the Cloudflare ranges WARP needs for itself). That’s what makes the client liveable. The default is exclude mode, where WARP carries everything except a list of private ranges, so every byte the phone sends detours through Cloudflare. That has a real battery and latency cost, enough that I’d have ended up turning it on only when I needed it, which defeats the point. In include mode, traffic to my three machines goes through WARP and everything else goes out the normal path untouched. So the client just stays on. warp-cli status on the laptop says Connected all day and I don’t think about it.

For transport I have it on MASQUE, which is HTTP/3 on UDP 443 with an HTTP/2 fallback, rather than WireGuard on UDP 2408. Some carriers and guest networks block 2408; almost nobody blocks 443. My organisation’s policy also has post-quantum key agreement switched on for MASQUE, which I didn’t go looking for but won’t turn down.

Why 198.18, and why there’s an IPv6 address

#

The addresses look odd, and the first ones I picked were wrong in an instructive way.

My first attempt was 10.10.0.1, because that’s what you reach for. It fails in both split-tunnel modes. In exclude mode, WARP’s default list excludes every RFC 1918 range plus 100.64.0.0/10, so the client would have dropped traffic to it before it ever left the device. In include mode it would collide with any real 10.x network I happen to join; cafés, co-working spaces and plenty of offices use exactly that range. 198.18.0.0/15 is the RFC 2544 benchmarking range. It’s reserved, it’s never routed on the public internet, it isn’t in WARP’s exclude list, and no ordinary network uses it. The IPv6 addresses are ULA (fd00::/8) for the same reason.

Why there’s IPv6 at all is the more interesting story. Once the v4 address worked I tested it from the phone on wifi, it was fine, and I called it done. On 5G it didn’t work. The Agent said connected, the tunnel was healthy, Access was fine; the wifi test proved all of that. The problem was that mobile networks are mostly IPv6-only now. The phone had no native IPv4 at all and was reaching the v4 literal through the carrier’s 464XLAT translator, which sits outside the tunnel when the include list holds only a v4 prefix. The traffic went out to the public internet, where 198.18.0.0/15 doesn’t route, and was dropped without a sound. No error on the phone, nothing in the connector’s log.

The fix is to give every machine a second address the phone can reach natively over IPv6, and add it to the routes and to the include list. Now the fd12:3456:789a:: addresses are what my SSH client on the phone actually uses on mobile data.

Offloading: the laptop as a terminal

#

This is the part the whole setup exists for.

Models and fine-tuning. Ollama runs on devbox and binds to 127.0.0.1:11434. It’s invisible to the rest of the office LAN, let alone the internet. On the laptop it appears at https://devbox-ollama.ssh.test, or as a bare port for tools that want one:

OLLAMA_HOST=127.0.0.1:11435 ollama list

11435, not 11434, on purpose: 11434 is Ollama’s own default, and if the laptop ever runs a local Ollama too, the two would race for the port and I’d silently get devbox’s answers from the “local” one depending on which started first.

The practical effect is that when I’m building an AI feature, the model it calls is a workstation GPU rather than the laptop’s, from wherever I am. Evaluation runs, embedding a pile of documents, a fine-tune: they run on devbox, and the laptop’s fan stays off. When I’m prototyping an agent, it can point at a local model on the workstation and burn through tool-calling iterations without an API bill, and switch to a hosted model once the loop is right.

Long-running agents. An agent session that runs for an hour shouldn’t depend on my laptop lid staying open. Run it on devbox instead and the laptop is just a window onto it: close the lid, and later ssh devbox from wherever I am, or from the phone, and it’s still there. Before the tunnel this only worked in the office, which is exactly where I didn’t need it.

Builds and the cluster. The cluster on devbox has its own registry, so images can be built and pushed on the workstation, next to where they run, rather than uploaded over hotel wifi from a laptop. The cluster dashboard, the ingress dashboard and the registry UI are all *.ssh.test hostnames on the laptop.

The mobile loop. The phone is on the same private network as the laptop, and the laptop’s dev servers also have public hostnames behind Access. So I can take a native build of the app out of the building, on 5G, and it talks to the API running on my desk. Or it talks to the one on devbox, if that’s where I left it. The feedback loop on a real device over a real mobile network used to mean deploying to a shared staging server. Now it means saving a file.

There’s one trap here worth knowing before you debug it as an app bug. Access protects the WebSocket hostname too, and a native WebSocket client isn’t a browser. It carries no CF_Authorization cookie, so the native build’s connection to the WebSocket server is rejected at the edge with a 403 that looks nothing like an error from your own server. Browser tabs are fine because they already hold the cookie from logging in. For anything that isn’t a browser (the native app, a script, an agent calling a dev API) the answer is an Access service token, sent as two headers:

CF-Access-Client-Id:     <client_id>
CF-Access-Client-Secret: <client_secret>

The token grants nothing on its own. Access only honours it on apps whose policy names it, so its reach is exactly the list of dev hostnames and nothing wider. I file its policy under the non_identity decision rather than allow, which keeps it visible in the audit log as a bearer secret and not a person who logged in.

The stats the internet never sees

#

The other half of what I wanted was visibility. Most of what tells you how a system is behaving is, correctly, not public. On devbox that’s:

• the ingress proxy’s dashboard, and the tracing UI that holds its traces
• a dashboard for the Kubernetes cluster
• the container registry’s UI (read-only)
• Ollama’s model list and what’s loaded right now
• docker info and container stats

None of those have a public hostname, and none ever will. They’re bound to loopback or to cluster service IPs, reached over an SSH forward that itself rides the tunnel. A monitoring app on my phone reads docker info --format '{{json .}}' over SSH across WARP, so I can see from anywhere whether my containers are up or something has eaten all the memory.

That app is how I found a small bug worth mentioning. The tunnel sshd on the Mac was at LogLevel DEBUG2 from when I was debugging it. With -e, sshd logs to stderr, and for a session that stderr is the SSH channel’s stderr. Every command’s output came back with debug2: do_setup_env: set TMPDIR appended, and the phone app, which merges the two streams, got valid JSON followed by a log line, failed to parse it, and showed the whole thing as an error. LogLevel INFO fixed it. Put log levels back after debugging.

Just as deliberate is what I don’t put on this network. The old autossh tunnels also reach a few routers on other people’s networks that I help look after. Those get no public hostname and no WARP route. A router admin page on the internet is a liability even behind a login, and it isn’t my network to widen.

Access is not optional

#

A Cloudflare Tunnel hostname is on the public internet the moment its DNS record exists. There’s no firewall in front of it; that’s the whole point of not opening a port. The protection has to come from Cloudflare’s edge, and that’s what Access does. It refuses to forward a request at all until the visitor has proved who they are, so an unauthenticated scanner never reaches the laptop. That matters, because these are dev servers: a debug-mode traceback with settings in it, an open JavaScript bundler. Nothing behind these hostnames was built to face the internet.

The policy allows my own email addresses and nothing else, with a one-time PIN by email and single sign-on as the two ways to prove it. Sessions last eight hours, because the realistic threat is a laptop left open in a café with a day-long cookie, not someone brute-forcing a PIN.

SSH is authenticated twice, by two systems that don’t know about each other. Access decides whether a TCP stream may reach sshd at all; sshd still demands my key. Losing either doesn’t open the machine. The Mac’s tunnel sshd also trusts Cloudflare’s SSH CA, so Access can log me in with a short-lived certificate minted from my SSO session, while authorized_keys still works alongside it, so a broken Access config can’t lock me out of my own laptop.

The piece I’m happiest with is small. Exposure happens when three things line up: a DNS record, a running connector and an active zone. They become true at different times, in an order I don’t control; the zone flips to active on its own, some time after the nameservers change, quite possibly while the Mac is asleep. So the launch agent doesn’t run cloudflared directly. It runs a preflight that reads the hostnames out of the tunnel’s own config, asks the Access API which ones are covered, and refuses to start if any aren’t:

HOSTNAMES=$(grep -E '^\s*-\s*hostname:' "$CONFIG" | sed -E 's/.*hostname:\s*//' | sort -u)
PROTECTED=$(curl -fsS -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \
    "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/access/apps?per_page=100" \
    | python3 -c 'import sys,json; print("\n".join(a.get("domain","") for a in json.load(sys.stdin)["result"]))') \
    || { echo "could not reach the Access API. refusing (fail closed)."; exit 1; }

for h in $HOSTNAMES; do
    grep -qxF "$h" <<<"$PROTECTED" || { echo "NOT starting: $h has no Access app"; exit 1; }
done
exec cloudflared tunnel --config "$CONFIG" run

The launch agent has KeepAlive, so a refusal retries by itself every 30 seconds and the tunnel comes up on its own once the Access apps exist. Failing closed costs an outage. Failing open costs an exposure. I know which of those I’d rather explain.

Four things that bit me

#

Everything above works now. Getting there, three of the four had the same pattern: the thing that was broken produced no error anywhere, and the parts I could see all reported healthy.

The route and the include list are two different lists. A tunnel route makes Cloudflare willing to carry an address to the right connector. The WARP include list is what makes the client send it in the first place. devbox was unreachable from the phone because its routes existed but the include list still only had the Mac’s addresses. The phone dropped the packets locally, the connector never saw a flow, and every dashboard was green. Every new machine means adding its addresses to both.

IPv4 literals on IPv6-only mobile networks. Covered above: works on wifi, fails on 5G, no error. If a WARP route works on wifi and not on mobile data, and the Agent shows connected, it’s routing, not the tunnel.

macOS sshd behind cloudflared. macOS starts its sshd with launchd socket activation, one short-lived sshd -i per connection. Behind cloudflared, the banner reaches the client and then the stream is torn down before the client’s identification string gets back, so every login dies at kex_exchange_identification. I confirmed it from both ends and reproduced it with a plain TCP relay, which ruled out Cloudflare. The fix is a second, standalone sshd -D on 127.0.0.1:2222 and the WARP loopback addresses, running as my user, that exists only to be the tunnel’s origin. Remote Login in System Settings is untouched. devbox, being Linux with a normal long-lived sshd, never had the problem.

Config fields that stop existing. Older examples have warp-routing: enabled: true in the tunnel config. cloudflared 2026.8 rejects that field outright. WARP routing is on whenever the block exists and a route points at the tunnel. That one at least produced an error, which after the other three felt almost generous.

What it changed

#

The honest summary is that I stopped choosing where to work based on where the hardware is. The laptop is a good keyboard and a good screen with a long battery life, and it doesn’t need to be more than that. The workstation does the heavy lifting from wherever I am. The phone is a real test device on a real network rather than something tethered to office wifi. And the parts of the system I use to see what it’s doing are one click away for me and don’t exist for anyone else.

At this size it fits in Cloudflare’s free Zero Trust plan. What it cost me was time and a README I’m glad I wrote, because every one of the failures above would have been a fresh mystery the second time.

If you’re thinking about something similar, these are the questions I’d ask of your setup:

1. What runs on your laptop only because the laptop is the one machine you can always reach?
2. Can your phone reach your dev server on mobile data, not just on the office wifi? Test it on 5G, not wifi.
3. Which of your dashboards are public because that was the easy way to see them from outside? Each one is a candidate for a private route instead.
4. If a tunnel hostname came up without its Access app tomorrow, what would stop it? If the answer is “I’d remember”, write the preflight.
5. Are your private addresses in a range the networks you visit will never use?