<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:media="http://search.yahoo.com/mrss/">
  <channel>
    <title>Developer Experience · Jared Lynskey</title>
    <link>https://jared.lynskey.co.nz/en/tags/developer-experience/</link>
    <description>Short stories of value on software engineering and engineering leadership — performance reviews, hiring and scaling teams, CI/CD, DevOps monitoring and agentic AI. Written from Seoul.</description>
    <generator>Hugo</generator>
    <language>en</language>
    <copyright>© 2026 Jared Lynskey</copyright>
    <lastBuildDate>Sat, 07 Nov 2026 00:00:00 +1300</lastBuildDate>
    <atom:link href="https://jared.lynskey.co.nz/en/tags/developer-experience/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>How I Run Claude Code Day to Day: tmux, Permission Modes and My Phone</title>
      <link>https://jared.lynskey.co.nz/en/posts/2026/2026-10-22-claude-code-day-to-day/</link>
      <guid isPermaLink="true">https://jared.lynskey.co.nz/en/posts/2026/2026-10-22-claude-code-day-to-day/</guid>
      <pubDate>Thu, 22 Oct 2026 00:00:00 +1300</pubDate>
      <dc:creator>Jared Lynskey</dc:creator>
      <category>Claude Code</category>
      <category>AI Agents</category>
      <category>LLM</category>
      <category>Developer Experience</category>
      <category>DevOps</category>
      <category>tmux</category>
      <category>SSH</category>
      <category>Codex</category>
      <category>Automation</category>
      <description>Claude Code is Anthropic&#39;s coding agent: the Claude models working on your own machine, with your files and your shell. How I run it day to day, with numbers from my own logs: in a terminal inside tmux, five sessions, permission prompts mostly bypassed, reached from my phone over Remote Control, SSH and a chat bot, and why I can&#39;t tell you how it compares with Codex.</description>
      <content:encoded><![CDATA[<p>Claude Code is Anthropic&rsquo;s coding agent. It runs the same Claude models as the Claude app, but on your own machine, where it can read your files, run shell commands and change your code, and it comes with the paid Claude plans. I use it in a terminal rather than the VS Code extension or the desktop app: five sessions inside one tmux session on my laptop, nearly all started with permission prompts switched off, and reachable from my phone through the Claude app&rsquo;s Remote Control, over SSH, or through a chat bot.</p>
<p>What follows is that setup in detail, with numbers from my own machine rather than from memory: the settings files, the session transcripts Claude Code keeps under <code>~/.claude/projects</code> (mine go back to 28 August), and its prompt history, which goes back to February and holds 7,323 prompts.</p>

<h2 class="relative group">Claude and Claude Code are different things
    <div id="claude-and-claude-code-are-different-things" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#claude-and-claude-code-are-different-things" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>Claude is the family of models, and the chat app at claude.ai, on the desktop and on your phone, where you talk to them. Claude Code is an agent built on those models. You start it in a project folder and it works there in a loop, reading files, running commands, editing code and running the tests, until the job is done or it needs you. Ask Claude how to fix something and you get an answer. Ask Claude Code and you get the fix, and a diff to read.</p>
<p>I use both. Since 18 October a marketplace and community app I build has sent its translations straight to the Claude API from its Django backend, and the change that switched it over was made in a Claude Code session. In my transcripts, 94% of the replies came from Opus 5.5, and my settings have the effort level at high.</p>
<p>The native installer is one line and keeps itself up to date:</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">curl -fsSL https://claude.ai/install.sh <span class="p">|</span> bash</span></span></code></pre></div></div>
<p>It leaves a <code>claude</code> symlink in <code>~/.local/bin</code> pointing into <code>~/.local/share/claude/versions/</code>; mine moved from 2.1.305 to 2.1.306 on its own on 21 October. Claude Code needs a paid Claude plan (Pro, Max, Team or Enterprise) or an API account. Prices change often enough that I&rsquo;ll leave them to <a href="https://claude.com/pricing"  target="_blank" rel="noreferrer">Anthropic&rsquo;s pricing page</a>.</p>

<h2 class="relative group">One tmux session, seven windows
    <div id="one-tmux-session-seven-windows" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#one-tmux-session-seven-windows" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>As I write this, <code>tmux list-panes -a</code> shows one session on my laptop, <code>main</code>, with seven windows. Five are Claude Code, one per project: my marketplace app and my estimating app (both described in <a href="https://jared.lynskey.co.nz/en/posts/2026/2026-07-22-my-current-stack/" >Django on Fly.io</a>), this blog and two smaller things. My terminal profile opens every window with a login shell that attaches to <code>main</code>, so a new tab on the laptop and an SSH login from my phone land on the same sessions.</p>
<p>I tried another multiplexer from 17 September to 8 October and went back to tmux for a practical reason: the SSH app on my phone multiplexes with tmux, and its shortcut buttons send tmux&rsquo;s stock keys. Lines like these in <code>~/.tmux.conf</code> exist for the phone:</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="nb">set</span> -g prefix C-a                 <span class="c1"># Ctrl-a prefix</span>
</span></span><span class="line"><span class="cl"><span class="nb">set</span> -g prefix2 C-b                <span class="c1"># stock Ctrl-b too, for apps that send it</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Size each window to the client that used it last, not the smallest one,</span>
</span></span><span class="line"><span class="cl"><span class="c1"># so a phone attached to the same session doesn&#39;t shrink it.</span>
</span></span><span class="line"><span class="cl">setw -g window-size latest
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Copy to the clipboard of whichever terminal is attached (OSC 52),</span>
</span></span><span class="line"><span class="cl"><span class="c1"># so over SSH it lands on the phone&#39;s clipboard too.</span>
</span></span><span class="line"><span class="cl"><span class="nb">set</span> -g set-clipboard on</span></span></code></pre></div></div>
<p>Inside tmux, two things change. Claude Code moves a running command to the background with Ctrl+B, which is now my second tmux prefix, so <code>~/.claude/keybindings.json</code> switches it off and uses Meta+B instead:</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-json" data-lang="json"><span class="line"><span class="cl"><span class="p">{</span>
</span></span><span class="line"><span class="cl">  <span class="nt">&#34;bindings&#34;</span><span class="p">:</span> <span class="p">[</span>
</span></span><span class="line"><span class="cl">    <span class="p">{</span> <span class="nt">&#34;context&#34;</span><span class="p">:</span> <span class="s2">&#34;Global&#34;</span><span class="p">,</span> <span class="nt">&#34;bindings&#34;</span><span class="p">:</span> <span class="p">{</span> <span class="nt">&#34;ctrl+b&#34;</span><span class="p">:</span> <span class="kc">null</span> <span class="p">}</span> <span class="p">},</span>
</span></span><span class="line"><span class="cl">    <span class="p">{</span> <span class="nt">&#34;context&#34;</span><span class="p">:</span> <span class="s2">&#34;Task&#34;</span><span class="p">,</span> <span class="nt">&#34;bindings&#34;</span><span class="p">:</span> <span class="p">{</span> <span class="nt">&#34;ctrl+b&#34;</span><span class="p">:</span> <span class="kc">null</span><span class="p">,</span> <span class="nt">&#34;meta+b&#34;</span><span class="p">:</span> <span class="s2">&#34;task:background&#34;</span> <span class="p">}</span> <span class="p">}</span>
</span></span><span class="line"><span class="cl">  <span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span></span></span></code></pre></div></div>
<p>The other is new lines. Enter sends the prompt. The docs give <code>\</code> then Enter, and Ctrl+J, as the two that work in any terminal; Option+Enter works once the terminal sends Option as Meta; and Shift+Enter works natively in some terminals, but tmux only passes it through with <code>extended-keys</code> switched on. Mine isn&rsquo;t, so in my sessions Shift+Enter sends the prompt. I rarely need any of them: 84 of my 7,323 prompts contain a line break I typed, while 279 contain a paste, which Claude Code folds into a placeholder like <code>[Pasted text #1 +34 lines]</code> and sends in full.</p>

<h2 class="relative group">Terminal, VS Code extension or desktop app?
    <div id="terminal-vs-code-extension-or-desktop-app" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#terminal-vs-code-extension-or-desktop-app" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>The same agent comes in several wrappers, and they read the same settings files. Going by the entry point Claude Code records in each transcript, this is how my sessions split:</p>
<table>
	<thead>
			<tr>
					<th>Surface</th>
					<th>What it is</th>
					<th>My use, 28 August to 21 October</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td>CLI in a terminal</td>
					<td><code>claude</code> in a terminal, inside tmux</td>
					<td>128 sessions</td>
			</tr>
			<tr>
					<td>Desktop app, Code tab</td>
					<td>Claude Code inside the Claude app, with a diff view and panes</td>
					<td>2 sessions</td>
			</tr>
			<tr>
					<td>VS Code extension</td>
					<td>a chat panel in the editor</td>
					<td>none; it isn&rsquo;t installed now, and only its settings are left</td>
			</tr>
	</tbody>
</table>
<p>The Claude app and claude.ai/code aren&rsquo;t a fourth place to start a session: they steer a session already running on my laptop, or one in the cloud, and most of my sessions since October have had that link through Remote Control.</p>
<p>For me it&rsquo;s the terminal, and not out of loyalty. tmux, which my phone attaches to, needs one, and so do channels, which push outside events into a session and which the docs list as a CLI feature.</p>
<p>People also ask whether Claude Code needs git. The CLI doesn&rsquo;t: my home directory isn&rsquo;t a repository, and 208 of my prompts went to sessions started there. The desktop app&rsquo;s documentation says git is required for its session isolation, where each session gets its own worktree, and pulling a cloud session into the terminal needs a checkout of the same repository. I&rsquo;ve never seen the &ldquo;Git is required&rdquo; message myself. My last desktop session, on 20 October, ran in a folder that isn&rsquo;t a repository at all.</p>

<h2 class="relative group">Permission modes, and the one I actually use
    <div id="permission-modes-and-the-one-i-actually-use" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#permission-modes-and-the-one-i-actually-use" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>The permission mode decides how often Claude Code stops to ask, and Shift+Tab cycles through the modes. Manual (<code>default</code>) asks before most actions that edit files, run commands or reach the network. <code>acceptEdits</code> lets file edits and common filesystem commands through. <code>plan</code> reads and proposes without editing. <code>auto</code> has a second model, a classifier, block the risky actions instead of asking you. <code>bypassPermissions</code> asks about almost nothing. Since 2.1.283, a terminal session starts in auto mode unless <code>permissions.defaultMode</code> says otherwise.</p>
<p>So my honest answer to &ldquo;how do I run Claude Code without it asking for permission&rdquo; is that I start it like this:</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">claude --dangerously-skip-permissions</span></span></code></pre></div></div>
<p>62 of the 69 <code>claude</code> commands in my shell history carry that flag, and so do all five sessions running now. Of the 1,266 prompts I typed between 28 August and 21 October, 1,179 went to sessions in bypass mode, 82 to auto mode and five to Manual or accept-edits.</p>
<p>The docs are plain that bypass mode is meant for &ldquo;isolated environments like containers, VMs, or dev containers without internet access, where Claude Code cannot damage your host system.&rdquo; My laptop is none of those, and my settings record that I&rsquo;ve clicked through that warning (<code>&quot;skipDangerousModePermissionPrompt&quot;: true</code>). If you do the same, know that allow rules have no effect in bypass mode, so in my everyday sessions the 378 allow rules spread over 19 settings files do nothing at all. Deny and ask rules do still apply, and I have none of either.</p>
<p>The exception is the session that reads other people&rsquo;s words. Its launcher doesn&rsquo;t pass the flag, so it starts in auto mode. All 27 production events and 42 of the 43 chat messages that reached a session between 12 and 20 October arrived in auto mode, and none in bypass.</p>
<p>What actually keeps bypass mode from biting me is a few habits, written into Claude&rsquo;s memory after a day in September when two sessions in the same checkout overwrote each other&rsquo;s work and nearly deployed it (<a href="https://jared.lynskey.co.nz/en/posts/2026/2026-10-22-agentic-ai-security/#isolation-because-the-likeliest-threat-is-another-agent" >the agentic AI security post</a> has the whole story): commit a small fix the moment it works, stage files by name and never with <code>git add -A</code>, look at <code>git status</code> for changes you didn&rsquo;t make, and when <code>main</code> holds other sessions&rsquo; undeployed work, deploy from a worktree at the commit that&rsquo;s live.</p>

<h2 class="relative group">Reaching it from my phone
    <div id="reaching-it-from-my-phone" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#reaching-it-from-my-phone" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>There are three ways in, and each one reaches something different.</p>
<table>
	<thead>
			<tr>
					<th>Way in</th>
					<th>What I get on the phone</th>
					<th>What it runs on</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td>Remote Control, in the Claude app</td>
					<td>one session&rsquo;s conversation: type, answer its questions, attach a photo</td>
					<td><code>remoteControlAtStartup: true</code>, plus push notifications</td>
			</tr>
			<tr>
					<td>SSH into <code>main</code>, from a phone SSH app</td>
					<td>the terminal itself, every window</td>
					<td>a private network, an sshd running as my user, and a short alias to attach</td>
			</tr>
			<tr>
					<td>A chat bot</td>
					<td>a session that hears production errors and answers messages</td>
					<td>an official Claude Code channel plugin and an event channel I wrote</td>
			</tr>
	</tbody>
</table>
<p>Remote Control connects the Claude app on a phone, or claude.ai/code in a browser, to a session that keeps running on the laptop, files, tools and MCP servers included. I typed <code>/rc</code> 26 times between 13 September and 20 October. Now <code>remoteControlAtStartup</code> is on and new sessions connect by themselves: since the start of October, 67 of the 77 sessions I started had a Remote Control link, against 3 of 52 in the second half of September.</p>
<p>With <code>agentPushNotifEnabled</code> on, Claude can send a push notification when it decides one is needed, which the docs say is usually when a long task finishes or it needs a decision. It needs a claude.ai subscription rather than an API key, and because the session runs on the laptop, the laptop has to be awake. Since 15 October mine doesn&rsquo;t sleep with the lid shut (<code>sudo pmset -a disablesleep 1</code>).</p>
<p>It works the other way too. <code>/teleport</code>, or <code>claude --teleport</code>, pulls a cloud session that was started on the web or in the app down into the terminal, with its branch and its conversation. I&rsquo;ve done that twice, both times in July.</p>
<p>SSH is for when I want the terminal itself. The phone reaches the laptop over the private network from <a href="https://jared.lynskey.co.nz/en/posts/2026/2026-10-01-cloudflare-warp-dev-network/" >my private-network post</a>, and an alias attaches it to <code>main</code>, so I can swipe between the windows I left on the laptop. Getting the Mac to answer phone apps properly took four separate fixes, and the one that matters for Claude Code is that tmux must never start by itself: an app that opened a session to run <code>docker ps</code> got 72 KB of screen redraw back instead.</p>
<p>The chat bot is for the session I don&rsquo;t want to sit and watch. Since 12 October one session has run with a small channel I wrote, which checks the app&rsquo;s server once a minute for new production errors and project-board activity and hands them to the session, plus one of the official chat plugins, so the two of us on its allowlist can message it. The launch line looks like this, with the names made generic:</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="nb">exec</span> claude <span class="se">\
</span></span></span><span class="line"><span class="cl">  --channels plugin:&lt;chat-plugin&gt;@claude-plugins-official <span class="se">\
</span></span></span><span class="line"><span class="cl">  --dangerously-load-development-channels server:my-events <span class="s2">&#34;</span><span class="nv">$@</span><span class="s2">&#34;</span></span></span></code></pre></div></div>
<p>A supervisor script keeps it running inside tmux. After a restart it resumes the same conversation, so the session carries on rather than waking up blank, and it announces every restart in the chat. The session fixes things in the working tree and stops there, with one exception I added on 13 October: a bug that is actually hurting users in production it may fix, test, commit and deploy, and then tell us what it did.</p>

<h2 class="relative group">Skills, memory and /goal
    <div id="skills-memory-and-goal" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#skills-memory-and-goal" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>There&rsquo;s no <code>hooks</code> key in any of my settings files. The nearest thing is <code>/goal</code>, which I&rsquo;ve used again and again since 11 July. It takes a condition and sets up a Stop hook for the session that, in Claude Code&rsquo;s words, &ldquo;will block stopping until the condition holds&rdquo;. My goals read the way I&rsquo;d say them out loud. A real one from July, typos and all, told it the build was &ldquo;still broken, try a deeper fix and dont stop until&rdquo; it was out to testers. To check on a long run without derailing it there&rsquo;s <code>/btw</code>, a side question that stays out of the conversation. Mine are mostly some version of &ldquo;hows it going?&rdquo;.</p>
<p>Skills hold the things I&rsquo;d otherwise explain every time. One user-level skill reads a weekly keyword report for the sites I look after, this blog among them, and applies it to titles, descriptions and tags. The questions this post answers came from its report for this blog, dated 18 October, and the skill file is versioned in a repo alongside its code. The marketplace app&rsquo;s repo has its own skills: one starts the API in Docker and runs a smoke test, one works the project board, and one files the company&rsquo;s expenses. The two plugins I have installed are a Swift language-server plugin and the chat one.</p>
<p>Memory is where the lessons go. I have 160 memory files across 17 projects, 83 of them for the marketplace app, and many are about something that broke. The app&rsquo;s <code>CLAUDE.md</code> is now about six times as long as it was in June. This blog&rsquo;s <code>CLAUDE.md</code> is an empty file; how a post should sound lives in a memory file instead.</p>
<p>For bigger jobs Claude starts subagents, 120 times in my main sessions, 40 of them forks of the conversation. That&rsquo;s how this blog gets its Japanese, Korean, Chinese and Vietnamese versions. On 19 and 20 October a redesign of the app&rsquo;s community section went through the Workflow tool four times: an audit through seven lenses and a critic, which confirmed about 58 findings; fixes by five groups that each owned their own files and were each reviewed; a pass for the leftovers; and an independent check that every finding had been fixed. It shipped on the 20th.</p>

<h2 class="relative group">Driving a workstation over SSH
    <div id="driving-a-workstation-over-ssh" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#driving-a-workstation-over-ssh" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p><a href="https://jared.lynskey.co.nz/en/posts/2026/2026-10-01-cloudflare-warp-dev-network/" >My private-network post</a> is about putting the heavy work on devbox, the office workstation with the GPU, and reaching it from anywhere. Claude Code does most of that reaching. Between 24 September and 19 October, 21 sessions on the laptop ran 1,470 commands on devbox over plain <code>ssh</code>: <code>kubectl</code> into the cluster that runs there, <code>psql</code> inside a database pod, <code>tmux capture-pane</code> to see how a long job was getting on. Because <code>ssh devbox</code> falls back to a tunnel off the office network, the same session works from the office, from home or from a café. And since the laptop no longer sleeps, a session can keep driving devbox while I look in on it from the phone.</p>

<h2 class="relative group">Why I can&rsquo;t compare it with Codex
    <div id="why-i-cant-compare-it-with-codex" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#why-i-cant-compare-it-with-codex" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>I can&rsquo;t tell you from my own use whether Claude Code is better than Codex, and the table shows why:</p>
<table>
	<thead>
			<tr>
					<th></th>
					<th>Codex</th>
					<th>Claude Code</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td>When</td>
					<td>29 November to 4 December 2025</td>
					<td>since at least November 2025; prompt history from February 2026</td>
			</tr>
			<tr>
					<td>Where</td>
					<td>the VS Code extension</td>
					<td>the terminal (128 sessions), the desktop app (2)</td>
			</tr>
			<tr>
					<td>Model and effort</td>
					<td><code>gpt-5.1-codex</code>, medium</td>
					<td>Opus 5.5, high</td>
			</tr>
			<tr>
					<td>Approvals</td>
					<td><code>workspace-write</code> sandbox, network off, asks on request</td>
					<td>bypass in 93% of prompts, auto for the chat session</td>
			</tr>
			<tr>
					<td>How much</td>
					<td>4 threads</td>
					<td>7,323 prompts since February</td>
			</tr>
			<tr>
					<td>Project notes</td>
					<td>reads <code>AGENTS.md</code></td>
					<td>reads <code>CLAUDE.md</code>, plus its memory files</td>
			</tr>
	</tbody>
</table>
<p>Four threads in one week, on an image question, two changes to a website and a 3D scene, is a trial, not a comparison. One detail from them applies to any switch between agents. The 3D project kept its rules in a <code>claude.md</code>, and my first question to Codex there was whether it read Claude&rsquo;s files before a conversation. It said it didn&rsquo;t, and it only picked up the project&rsquo;s rules when I asked it to read that file. Whichever agent you move to, the instructions file stays behind unless you bring it. I haven&rsquo;t opened a Codex thread since. The extension loaded in VS Code on a handful of days between 24 September and 4 October without my starting one, and it&rsquo;s uninstalled now.</p>
<p>So the verdict I can stand behind is narrow. Claude Code is what my setup is built around, and I haven&rsquo;t gone back to see how Codex handles the same things today.</p>
<p>One last number says more about how I use it than anything above. In the main sessions on my laptop since late August, Claude Code ran 18,311 shell commands and read, wrote or edited a file 2,331 times. By count, most of its work is running the commands I&rsquo;d otherwise be typing, on the laptop and over SSH, from wherever I happen to be. When it does write the code, that code isn&rsquo;t welcome everywhere, which is a story of its own in <a href="https://jared.lynskey.co.nz/en/posts/2026/2026-10-18-pyhtml2text/" >the pyhtml2text post</a>.</p>
]]></content:encoded>
      <media:content url="https://jared.lynskey.co.nz/en/posts/2026/2026-10-22-claude-code-day-to-day/featured.jpg" medium="image" type="image/jpeg" />
    </item>
    <item>
      <title>Monitoring for Developers: Metrics, Logs, Traces, and CloudWatch vs Datadog</title>
      <link>https://jared.lynskey.co.nz/en/posts/2025/2025-10-13-devops-monitoring/</link>
      <guid isPermaLink="true">https://jared.lynskey.co.nz/en/posts/2025/2025-10-13-devops-monitoring/</guid>
      <pubDate>Sun, 12 Oct 2025 00:00:00 +0000</pubDate>
      <dc:creator>Jared Lynskey</dc:creator>
      <category>DevOps</category>
      <category>Monitoring</category>
      <category>Observability</category>
      <category>CloudWatch</category>
      <category>Datadog</category>
      <category>Developer Experience</category>
      <category>Site Reliability</category>
      <description>Monitoring for developers: how to link metrics, logs and traces, write alerts that say where to start looking, and when CloudWatch is enough and Datadog earns its price.</description>
      <content:encoded><![CDATA[<p>A monitoring system for developers has one job: when something breaks, tell someone fast, and show them what broke and where. That takes metrics, logs and traces linked to each other, alerts that point at a place to start, and dashboards people open without being asked. The tool comes second; on AWS, CloudWatch covers a small team, and Datadog earns its price once you&rsquo;re big.</p>
<p>There&rsquo;s a gap between &ldquo;works on my machine&rdquo; and &ldquo;works in production&rdquo;, and monitoring is supposed to bridge it. Most of the time it doesn&rsquo;t, because it gets built as an ops tool rather than something a developer reaches for when things break.</p>
<p>I&rsquo;ve just gone freelance doing DevOps and infrastructure work, and the monitoring setup is one of the first things I look at anywhere I land — it tells you more about a team than the README does. The failure mode is nearly always the same: the tools exist, but they don&rsquo;t connect to how developers actually work. Someone burns three hours on a bug that production logs would&rsquo;ve solved in five minutes. Failures arrive as support tickets instead of alerts. Or the opposite: 200 alerts a day, most of them noise, so everyone ignores all of them, and the first half hour of every incident is spent working out <em>what</em> broke rather than fixing it.</p>
<p>This is the general version. For the specifics, I&rsquo;ve since written up <a href="https://jared.lynskey.co.nz/en/posts/2026/2026-10-22-monitoring-tools-two-products/" >what I actually run today</a> on two small products.</p>

<h2 class="relative group">Metrics, logs, and traces — linked, or useless
    <div id="metrics-logs-and-traces--linked-or-useless" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#metrics-logs-and-traces--linked-or-useless" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>Metrics tell you something is wrong (error rate spiked, latency jumped). Logs tell you what happened (stack traces, request payloads, error messages). Traces tell you where (which service, which endpoint, which database call). None of that is news. What matters is whether they&rsquo;re linked: when an alert fires, you should be able to click from the metric to the relevant logs to the trace. If developers are manually correlating timestamps across three tools, the setup has failed, no matter how good each tool looks on paper.</p>
<p>Dashboards deserve the same test. CPU, memory and disk I/O matter for capacity planning, but they don&rsquo;t help anyone debug a 500. The dashboards people open voluntarily show business metrics next to technical ones (signups per hour beside error rate), recent deployments overlaid on the timeline, the top five errors in the last hour with links to logs, and latency percentiles — p50, p95, p99 — rather than averages. If nobody opens a dashboard unprompted, it isn&rsquo;t earning its place.</p>
<p>Alerts, same idea. Every alert should answer two questions: what&rsquo;s broken, and where do I start looking?</p>
<p>Bad alert: &ldquo;High CPU on web-server-3&rdquo;</p>
<p>Good alert: &ldquo;Error rate &gt; 5% on /api/payments since 14:32. Last deploy: 14:15 by @sarah. [View logs] [View trace]&rdquo;</p>
<p>A few things push alerts toward the second kind: alert on deviation from a baseline rather than an arbitrary threshold, route by ownership (payment errors go to the payments team, not the whole org), and group related errors into one Slack message instead of 50. Keep the loop tight, too — the time between &ldquo;something broke&rdquo; and &ldquo;a developer knows&rdquo; should be under a minute, which means real-time log streaming rather than five-minute batches, deploy markers on every dashboard, and tracing that survives service boundaries.</p>

<h2 class="relative group">Build it for the people who&rsquo;ll use it
    <div id="build-it-for-the-people-wholl-use-it" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#build-it-for-the-people-wholl-use-it" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>The biggest mistake ops teams make is building monitoring for themselves. The fix isn&rsquo;t sophisticated. Sit with a developer during a debugging session and watch where they get stuck. Ask what questions come up mid-incident — &ldquo;which service?&rdquo;, &ldquo;what changed?&rdquo;, &ldquo;what did the request look like?&rdquo;. Find out which metrics actually matter for the product; it&rsquo;s rarely CPU, and usually things like checkout completion rate or upload success rate.</p>
<p>Then remove friction. If instrumenting a new service takes a day, it won&rsquo;t happen. Provide libraries that auto-instrument the frameworks you already use (Django, Express, Spring), copy-paste templates for dashboards and alerts, and self-service metrics so nobody has to file a ticket. And treat the whole thing as code: config in version control, alert rules tested so you know they fire when they should, and multi-region considered early if that&rsquo;s where you&rsquo;re headed.</p>
<p>The same habits keep it alive afterwards. Instrumentation goes through code review like any other production code, and a custom metric gets a test. Every postmortem asks what monitoring would have caught this sooner, and the answer goes into the dashboards and alerts that week rather than into the write-up. And a few times a year I go through what&rsquo;s there and delete what nobody opens, because an alert nobody acts on is training people to ignore the next one.</p>

<h2 class="relative group">Small team on AWS? Start with CloudWatch
    <div id="small-team-on-aws-start-with-cloudwatch" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#small-team-on-aws-start-with-cloudwatch" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>If you&rsquo;re under 50 engineers and on AWS, CloudWatch is the right starting point, and I&rsquo;d resist buying anything fancier on day one. EC2, Lambda, RDS and ECS all report to it automatically, so you get visibility without writing instrumentation. It runs $10–50 a month for a small team with no platform fee, and metrics, logs, traces (via X-Ray) and alarms all live in one place. You can have meaningful alerts and dashboards up in an afternoon, not weeks.</p>
<p>CloudWatch Logs Insights is the underrated bit — you can query logs without standing up Elasticsearch:</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">fields @timestamp, @message
</span></span><span class="line"><span class="cl">| filter @message like /ERROR/
</span></span><span class="line"><span class="cl">| stats count() by bin(5m)</span></span></code></pre></div></div>
<p>Beyond that:</p>
<ul>
<li>Composite alarms cut noise — alert when error rate is high <em>and</em> response time is degraded, not on each condition separately.</li>
<li>Custom metrics are where the real value is. Push business metrics (signups, transactions, feature usage) through the CloudWatch SDK alongside the infrastructure ones.</li>
<li>CloudWatch Synthetics runs scheduled canaries through real user journeys, so you learn the checkout is broken before your users do.</li>
<li>X-Ray gives you distributed tracing with minimal setup — good enough for most microservice architectures.</li>
</ul>
<p>You&rsquo;ll know when you&rsquo;ve outgrown it: you go multi-cloud, you want proper anomaly detection, dashboard customisation starts to hurt, or the team passes 50 engineers and needs real collaboration features.</p>

<h2 class="relative group">Bigger teams: Datadog
    <div id="bigger-teams-datadog" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#bigger-teams-datadog" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>Datadog is expensive — $20–100K+ a year for a large org is normal — but at scale it earns its keep. One view across AWS, Azure, GCP, on-prem, containers, serverless, databases and frontend. Watchdog flags anomalies without hand-tuned thresholds, which matters because nobody can manually watch thousands of services. And you get the collaboration layer CloudWatch lacks: team dashboards, RBAC, shared investigation notebooks, PagerDuty/Opsgenie integration, multi-condition alerts, threshold forecasting, maintenance windows, and an APM that goes down to code-level profiling with auto-generated service maps.</p>
<p>Rolling it out, in roughly the order I&rsquo;d do it: instrument critical services first and tag everything by team and environment from day one. Agree naming conventions, dashboard templates, severity levels and SLO definitions early — retrofitting standards is miserable. Wire it into CI/CD deploy markers, incident management and Slack. Budget time for training, because Datadog is powerful but not self-explanatory. And watch the bill: filter noisy metrics, sample traces on high-volume services, and audit which features you&rsquo;re actually paying for.</p>
<p>Worth a look before committing: New Relic (similar, sometimes cheaper for high-volume tracing), Dynatrace (strong AIOps, popular in financial services), Splunk (best-in-class log analysis, especially if security already runs it), and Grafana Cloud (the natural home if you&rsquo;re already on Prometheus/Loki).</p>
<p>In practice a lot of teams land on a hybrid: CloudWatch for AWS-native services because it&rsquo;s automatic and cheap, Datadog for the application layer, with CloudWatch metrics ingested into Datadog for one unified view. It&rsquo;s unglamorous, and it works.</p>

<h2 class="relative group">CloudWatch vs Datadog at a glance
    <div id="cloudwatch-vs-datadog-at-a-glance" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#cloudwatch-vs-datadog-at-a-glance" aria-label="Anchor">#</a>
    </span>
    
</h2>
<table>
	<thead>
			<tr>
					<th></th>
					<th>CloudWatch</th>
					<th>Datadog</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td>Fits best</td>
					<td>Under ~50 engineers, all on AWS</td>
					<td>Bigger orgs, many services, more than one cloud</td>
			</tr>
			<tr>
					<td>Cost shape</td>
					<td>Pay as you go, no platform fee; roughly $10–50 a month for a small team</td>
					<td>$20–100K+ a year is normal for a large org, and the bill needs watching</td>
			</tr>
			<tr>
					<td>Setup effort</td>
					<td>EC2, Lambda, RDS and ECS report automatically; useful alerts in an afternoon</td>
					<td>Instrument critical services, agree tags and naming early, budget time for training</td>
			</tr>
			<tr>
					<td>Multi-cloud</td>
					<td>Built around AWS; going multi-cloud is a sign you&rsquo;ve outgrown it</td>
					<td>One view across AWS, Azure, GCP, on-prem, containers and serverless</td>
			</tr>
			<tr>
					<td>Anomaly detection</td>
					<td>Mostly alarms you tune yourself; wanting more is another sign</td>
					<td>Watchdog flags anomalies without hand-tuned thresholds</td>
			</tr>
			<tr>
					<td>Collaboration</td>
					<td>Thin; teams past ~50 engineers start to miss it</td>
					<td>Team dashboards, RBAC, shared notebooks, PagerDuty/Opsgenie</td>
			</tr>
			<tr>
					<td>Tracing</td>
					<td>X-Ray, minimal setup</td>
					<td>APM down to code-level profiling, auto-generated service maps</td>
			</tr>
	</tbody>
</table>

<h2 class="relative group">Starting from scratch: the build order
    <div id="starting-from-scratch-the-build-order" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#starting-from-scratch-the-build-order" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>The order I&rsquo;d do things in:</p>
<ol>
<li>Talk to developers about what hurts during incidents.</li>
<li>Define SLOs for the flows that matter. What does &ldquo;working&rdquo; mean for login, checkout, search?</li>
<li>Instrument that critical path first.</li>
<li>Add distributed tracing next; it has the best debugging ROI in a microservice setup.</li>
<li>Alert on those flows rather than on CPU, and send each alert to the team that owns it.</li>
<li>Write a runbook for every alert so 3am-you knows what to check.</li>
<li>Put a quarterly review in the calendar to delete stale alerts, fix drifted thresholds, and confirm the dashboards still match the architecture.</li>
</ol>
<p>And the traps I keep pulling teams out of: too many tools (three that work together beat six that don&rsquo;t), graphs with no baseline (is 500 rps normal or a 10x spike?), monitoring things users don&rsquo;t feel (disk I/O on a stateless container), and no redundancy for the monitoring itself — if your alerting dies during an incident, you&rsquo;re blind exactly when you can least afford it.</p>
<p>None of this needs the fanciest tooling. It needs developers to get the information they need, in the place they&rsquo;ll actually look, fast.</p>
]]></content:encoded>
      <media:content url="https://jared.lynskey.co.nz/en/posts/2025/2025-10-13-devops-monitoring/featured.jpeg" medium="image" type="image/jpeg" />
    </item>
  </channel>
</rss>
