<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:media="http://search.yahoo.com/mrss/">
  <channel>
    <title>Object Storage · Jared Lynskey</title>
    <link>https://jared.lynskey.co.nz/en/tags/object-storage/</link>
    <description>Short stories of value on software engineering and engineering leadership — performance reviews, hiring and scaling teams, CI/CD, DevOps monitoring and agentic AI. Written from Seoul.</description>
    <generator>Hugo</generator>
    <language>en</language>
    <copyright>© 2026 Jared Lynskey</copyright>
    <lastBuildDate>Sat, 07 Nov 2026 00:00:00 +1300</lastBuildDate>
    <atom:link href="https://jared.lynskey.co.nz/en/tags/object-storage/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>Upload Directly to S3 and Still Compress Images: Tigris Object Notifications with Django</title>
      <link>https://jared.lynskey.co.nz/en/posts/2026/tigris-object-notifications/</link>
      <guid isPermaLink="true">https://jared.lynskey.co.nz/en/posts/2026/tigris-object-notifications/</guid>
      <pubDate>Sat, 07 Nov 2026 00:00:00 +1300</pubDate>
      <dc:creator>Jared Lynskey</dc:creator>
      <category>Tigris</category>
      <category>S3</category>
      <category>S3 Event Notifications</category>
      <category>Presigned URLs</category>
      <category>Image Uploads</category>
      <category>Image Compression</category>
      <category>Object Storage</category>
      <category>Webhooks</category>
      <category>Django</category>
      <category>Fly.io</category>
      <category>Discord</category>
      <description>Direct-to-S3 uploads with presigned URLs are fast, but your backend never hears that the file arrived, so compression and thumbnails get awkward. Tigris object notifications fix that: the bucket POSTs to a Django webhook and the photo is processed while the user is still typing. The same signal turns log files into Discord alerts.</description>
      <content:encoded><![CDATA[<p>Every image upload I&rsquo;ve built has had the same tension in it. The person uploading wants the photo to go up fast. I want the server to compress it, make a thumbnail, pull a blurhash and a couple of colours out of it, drop the metadata that shouldn&rsquo;t be there, and store it the way the rest of the app expects. The easy way to get my list is to send the file through the API, and that is exactly what makes their list slow.</p>
<p>The fast way is a presigned URL. The API signs a URL, the client PUTs the file straight to the bucket, and the bytes never touch the API at all. It&rsquo;s the standard answer for good reason. But the moment you do it you&rsquo;ve cut your backend out of the loop. It handed out a URL and then heard nothing. Did the file arrive? Is it the size you expected? Is it even a JPEG? Something has to tell the backend, and the usual options are all a bit sad: trust the client to call back and say &ldquo;done&rdquo; (clients lie, crash and lose signal), poll the bucket on a schedule (slow and wasteful), or process lazily on the first read (the first viewer pays, and you still don&rsquo;t know when the file is safe to reference). This is what makes uploading directly to S3 look harder than it is. The upload is easy. The signalling is what&rsquo;s missing.</p>
<p>Object notifications are that signal. Tigris, the S3-compatible storage I use on Fly.io, will POST a small JSON body to your app every time an object is created or deleted in a bucket. On AWS the equivalent is S3 event notifications, which usually means wiring up a Lambda, an SQS queue or EventBridge before anything reaches your code (SNS can POST to an HTTPS endpoint, but then you&rsquo;re confirming subscriptions and verifying signatures first). Tigris keeps it to one HTTP request to a URL you own, which is about as simple as storage-to-backend signalling gets. The feature exists so the bucket can talk to your backend without you building a bridge for it.</p>
<p>This post is the whole flow: the setup, a Django endpoint that receives the events, a presigned upload that keeps every bit of server-side processing, and the gotchas (CORS, size limits, expiry, multiple files). At the end, the same signal doing back-office work, where log files landing in the bucket became Discord alerts and let me delete a cron job I&rsquo;d been nursing for months.</p>

<h2 class="relative group">The short answer
    <div id="the-short-answer" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#the-short-answer" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>In the Tigris dashboard, open the bucket, then Settings, then Object notifications. Give it an HTTPS URL on your app and pick basic auth (a username and password Tigris sends in the <code>Authorization</code> header; a bearer token is the other option). Add a filter so you only hear about the keys you care about. The filter is a SQL-like expression, so &ldquo;only the three prefixes this post handles&rdquo; is:</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sql" data-lang="sql"><span class="line"><span class="cl"><span class="k">WHERE</span><span class="w"> </span><span class="o">`</span><span class="k">key</span><span class="o">`</span><span class="w"> </span><span class="n">REGEXP</span><span class="w"> </span><span class="s2">&#34;^(uploads/raw|logs/api|media)/&#34;</span></span></span></code></pre></div></div>
<p><figure><span class="mc-img"><img
    class="my-0 rounded-md"
    loading="lazy"
    decoding="async"
    fetchpriority="auto"
    alt="Tigris bucket settings, Notifications tab: object notifications switched on, a webhook URL, basic authentication with a username and password, and a SQL-like key filter"
    width="1204"
    height="671"
    src="https://jared.lynskey.co.nz/en/posts/2026/tigris-object-notifications/tigris-object-notifications-settings_hu_7e9da7c6ca1125db.webp"
    srcset="https://jared.lynskey.co.nz/en/posts/2026/tigris-object-notifications/tigris-object-notifications-settings_hu_7e9da7c6ca1125db.webp 800w, https://jared.lynskey.co.nz/en/posts/2026/tigris-object-notifications/tigris-object-notifications-settings_hu_4ddc968851d1f052.webp 1280w"
    sizes="(min-width: 768px) 50vw, 65vw"
    data-zoom-src="https://jared.lynskey.co.nz/en/posts/2026/tigris-object-notifications/tigris-object-notifications-settings_hu_4ddc968851d1f052.webp"></span><figcaption>The whole setup: an HTTPS URL on your app, basic auth, and a filter on the key. The bucket name and URL here are swapped for examples.</figcaption></figure>
</p>
<p>Tigris then sends a batch like this on every change:</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-json" data-lang="json"><span class="line"><span class="cl"><span class="p">{</span>
</span></span><span class="line"><span class="cl">  <span class="nt">&#34;events&#34;</span><span class="p">:</span> <span class="p">[</span>
</span></span><span class="line"><span class="cl">    <span class="p">{</span>
</span></span><span class="line"><span class="cl">      <span class="nt">&#34;eventName&#34;</span><span class="p">:</span> <span class="s2">&#34;OBJECT_CREATED_PUT&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">      <span class="nt">&#34;eventTime&#34;</span><span class="p">:</span> <span class="s2">&#34;2026-11-07T09:12:44Z&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">      <span class="nt">&#34;bucket&#34;</span><span class="p">:</span> <span class="s2">&#34;example-bucket&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">      <span class="nt">&#34;object&#34;</span><span class="p">:</span> <span class="p">{</span> <span class="nt">&#34;key&#34;</span><span class="p">:</span> <span class="s2">&#34;uploads/raw/1f3c.jpg&#34;</span><span class="p">,</span> <span class="nt">&#34;size&#34;</span><span class="p">:</span> <span class="mi">48213</span><span class="p">,</span> <span class="nt">&#34;eTag&#34;</span><span class="p">:</span> <span class="s2">&#34;9b2c…&#34;</span> <span class="p">}</span>
</span></span><span class="line"><span class="cl">    <span class="p">}</span>
</span></span><span class="line"><span class="cl">  <span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span></span></span></code></pre></div></div>
<p>The event names you&rsquo;ll meet are <code>OBJECT_CREATED_PUT</code>, <code>OBJECT_CREATED_MULTIPART</code>, <code>OBJECT_CREATED_COPY</code> and <code>OBJECT_DELETED</code>, plus rename and storage-tier transitions. Your endpoint checks the auth, routes each event by key prefix, and returns a 2xx within ten seconds. Three things to plan for from the start: delivery is at least once, so you will see repeats; events are batched, so one request can carry several; and they can arrive out of order, so don&rsquo;t read &ldquo;latest&rdquo; off the order they came in.</p>

<h2 class="relative group">A Django webhook for the bucket&rsquo;s events
    <div id="a-django-webhook-for-the-buckets-events" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#a-django-webhook-for-the-buckets-events" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>The view is boring on purpose. It checks the auth, parses the body and hands each event to a router:</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">base64</span><span class="o">,</span> <span class="nn">hmac</span><span class="o">,</span> <span class="nn">json</span><span class="o">,</span> <span class="nn">logging</span>
</span></span><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">django.conf</span> <span class="kn">import</span> <span class="n">settings</span>
</span></span><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">django.http</span> <span class="kn">import</span> <span class="n">JsonResponse</span>
</span></span><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">django.views.decorators.csrf</span> <span class="kn">import</span> <span class="n">csrf_exempt</span>
</span></span><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">django.views.decorators.http</span> <span class="kn">import</span> <span class="n">require_POST</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">.bucket_events</span> <span class="kn">import</span> <span class="n">handle</span><span class="p">,</span> <span class="n">parse</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="vm">__name__</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">_authorised</span><span class="p">(</span><span class="n">request</span><span class="p">,</span> <span class="n">secret</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">bool</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">    <span class="n">header</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">headers</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&#34;Authorization&#34;</span><span class="p">,</span> <span class="s2">&#34;&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">if</span> <span class="ow">not</span> <span class="n">header</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">&#34;Basic &#34;</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">        <span class="k">return</span> <span class="kc">False</span>
</span></span><span class="line"><span class="cl">    <span class="k">try</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">        <span class="n">user</span><span class="p">,</span> <span class="n">_</span><span class="p">,</span> <span class="n">password</span> <span class="o">=</span> <span class="n">base64</span><span class="o">.</span><span class="n">b64decode</span><span class="p">(</span><span class="n">header</span><span class="p">[</span><span class="mi">6</span><span class="p">:])</span><span class="o">.</span><span class="n">decode</span><span class="p">()</span><span class="o">.</span><span class="n">partition</span><span class="p">(</span><span class="s2">&#34;:&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">except</span> <span class="p">(</span><span class="ne">ValueError</span><span class="p">,</span> <span class="ne">UnicodeDecodeError</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">        <span class="k">return</span> <span class="kc">False</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="n">hmac</span><span class="o">.</span><span class="n">compare_digest</span><span class="p">(</span><span class="n">user</span><span class="o">.</span><span class="n">encode</span><span class="p">(),</span> <span class="sa">b</span><span class="s2">&#34;tigris&#34;</span><span class="p">)</span> <span class="ow">and</span> <span class="n">hmac</span><span class="o">.</span><span class="n">compare_digest</span><span class="p">(</span>
</span></span><span class="line"><span class="cl">        <span class="n">password</span><span class="o">.</span><span class="n">encode</span><span class="p">(),</span> <span class="n">secret</span><span class="o">.</span><span class="n">encode</span><span class="p">()</span>
</span></span><span class="line"><span class="cl">    <span class="p">)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="nd">@csrf_exempt</span>
</span></span><span class="line"><span class="cl"><span class="nd">@require_POST</span>
</span></span><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">bucket_webhook</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">    <span class="n">secret</span> <span class="o">=</span> <span class="n">settings</span><span class="o">.</span><span class="n">BUCKET_WEBHOOK_SECRET</span>
</span></span><span class="line"><span class="cl">    <span class="k">if</span> <span class="ow">not</span> <span class="n">secret</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">        <span class="k">return</span> <span class="n">JsonResponse</span><span class="p">({</span><span class="s2">&#34;error&#34;</span><span class="p">:</span> <span class="s2">&#34;not configured&#34;</span><span class="p">},</span> <span class="n">status</span><span class="o">=</span><span class="mi">503</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">if</span> <span class="ow">not</span> <span class="n">_authorised</span><span class="p">(</span><span class="n">request</span><span class="p">,</span> <span class="n">secret</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">        <span class="k">return</span> <span class="n">JsonResponse</span><span class="p">({</span><span class="s2">&#34;error&#34;</span><span class="p">:</span> <span class="s2">&#34;unauthorised&#34;</span><span class="p">},</span> <span class="n">status</span><span class="o">=</span><span class="mi">401</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">try</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">        <span class="n">body</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">request</span><span class="o">.</span><span class="n">body</span> <span class="ow">or</span> <span class="sa">b</span><span class="s2">&#34;</span><span class="si">{}</span><span class="s2">&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">except</span> <span class="ne">ValueError</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">        <span class="k">return</span> <span class="n">JsonResponse</span><span class="p">({</span><span class="s2">&#34;error&#34;</span><span class="p">:</span> <span class="s2">&#34;invalid json&#34;</span><span class="p">},</span> <span class="n">status</span><span class="o">=</span><span class="mi">400</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">for</span> <span class="n">event</span> <span class="ow">in</span> <span class="n">parse</span><span class="p">(</span><span class="n">body</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">        <span class="k">try</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">            <span class="n">handle</span><span class="p">(</span><span class="n">event</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">        <span class="k">except</span> <span class="ne">Exception</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">            <span class="n">logger</span><span class="o">.</span><span class="n">exception</span><span class="p">(</span><span class="s2">&#34;bucket event failed: </span><span class="si">%s</span><span class="s2">&#34;</span><span class="p">,</span> <span class="n">event</span><span class="o">.</span><span class="n">key</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="n">JsonResponse</span><span class="p">({</span><span class="s2">&#34;ok&#34;</span><span class="p">:</span> <span class="kc">True</span><span class="p">})</span></span></span></code></pre></div></div>
<p><code>compare_digest</code> is there so the check takes the same time whatever you send it (on bytes, because on strings it raises for non-ASCII input, and a crafted header would get a 500 instead of a 401). If the secret isn&rsquo;t set, say on a fresh staging box, the endpoint refuses everything rather than accepting everything. And a handler that throws is logged and still answered 200: Tigris retries anything that isn&rsquo;t a 2xx, a retry would hit the same bug, and by then the event has been claimed (two sections down) so the retry would do nothing anyway. The log line is what gets it fixed. That assumes the failure is deterministic. A transient storage timeout consumes the event for good: the photos don&rsquo;t mind, because the save can process them itself, but a log file&rsquo;s alert would be lost. If that matters to you, release the claim in the <code>except</code> and answer 500 so Tigris retries, and accept that a real bug then retries until it&rsquo;s fixed.</p>

<h2 class="relative group">Routing by prefix
    <div id="routing-by-prefix" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#routing-by-prefix" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>Everything in the bucket is already organised by prefix, so that&rsquo;s the natural way to route. Each use is one function and one line in a table:</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="n">HANDLERS</span> <span class="o">=</span> <span class="p">[</span>
</span></span><span class="line"><span class="cl">    <span class="p">(</span><span class="s2">&#34;uploads/raw/&#34;</span><span class="p">,</span>   <span class="n">on_raw_photo</span><span class="p">),</span>
</span></span><span class="line"><span class="cl">    <span class="p">(</span><span class="s2">&#34;logs/api/&#34;</span><span class="p">,</span>      <span class="n">on_api_log_file</span><span class="p">),</span>
</span></span><span class="line"><span class="cl">    <span class="p">(</span><span class="s2">&#34;media/&#34;</span><span class="p">,</span>         <span class="n">on_media_change</span><span class="p">),</span>
</span></span><span class="line"><span class="cl"><span class="p">]</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">handle</span><span class="p">(</span><span class="n">event</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">    <span class="k">for</span> <span class="n">prefix</span><span class="p">,</span> <span class="n">fn</span> <span class="ow">in</span> <span class="n">HANDLERS</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">        <span class="k">if</span> <span class="n">event</span><span class="o">.</span><span class="n">key</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="n">prefix</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">            <span class="k">if</span> <span class="n">claim</span><span class="p">(</span><span class="n">event</span><span class="p">):</span>  <span class="c1"># see below</span>
</span></span><span class="line"><span class="cl">                <span class="n">fn</span><span class="p">(</span><span class="n">event</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">            <span class="k">return</span></span></span></code></pre></div></div>
<p>Adding a fourth thing later is a new function and a new row. I like that more than I expected to.</p>

<h2 class="relative group">At least once means &ldquo;maybe twice&rdquo;
    <div id="at-least-once-means-maybe-twice" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#at-least-once-means-maybe-twice" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>Tigris retries if your endpoint is slow or returns an error, and like most webhook senders it makes no promise that you&rsquo;ll see an event only once. So before a handler runs, the event is claimed by <code>(key, eTag, eventName)</code>. The eTag matters: if a file is overwritten, it&rsquo;s a new version with a new eTag and should be handled again. A repeat of the same version shouldn&rsquo;t.</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">django.core.cache</span> <span class="kn">import</span> <span class="n">cache</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">claim</span><span class="p">(</span><span class="n">event</span><span class="p">,</span> <span class="n">ttl</span><span class="o">=</span><span class="mi">7</span> <span class="o">*</span> <span class="mi">24</span> <span class="o">*</span> <span class="mi">3600</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">bool</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">    <span class="n">k</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">&#34;bucket-event:</span><span class="si">{</span><span class="n">event</span><span class="o">.</span><span class="n">name</span><span class="si">}</span><span class="s2">:</span><span class="si">{</span><span class="n">event</span><span class="o">.</span><span class="n">key</span><span class="si">}</span><span class="s2">:</span><span class="si">{</span><span class="n">event</span><span class="o">.</span><span class="n">etag</span><span class="si">}</span><span class="s2">&#34;</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="n">cache</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">k</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="n">ttl</span><span class="p">)</span>  <span class="c1"># False if someone already claimed it</span></span></span></code></pre></div></div>
<p><code>cache.add</code> only sets the key if it isn&rsquo;t there already, which in Redis is atomic, so two machines receiving the same retry can&rsquo;t both win. I use a database-backed lease for this in the real thing because I wanted it to survive a Redis flush, but the idea is identical.</p>

<h2 class="relative group">Direct-to-S3 image uploads, with the pipeline intact
    <div id="direct-to-s3-image-uploads-with-the-pipeline-intact" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#direct-to-s3-image-uploads-with-the-pipeline-intact" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>This is the part that pays for the plumbing.</p>
<p>Previously, picking photos for a listing meant the app held them, and when you hit Post it sent them all to the API as base64. The API resized them, made thumbnails, worked out colours and a blurhash, and only then replied. Ten photos on a slow connection was a long spinner. And a request carrying ten base64 photos is a big, slow body for an API server to hold in memory, which was quietly why I kept photos small.</p>
<p>Now it&rsquo;s four steps, and the backend is told about every one of them.</p>

<h3 class="relative group">1. The client shrinks the photo and asks for a slot
    <div id="1-the-client-shrinks-the-photo-and-asks-for-a-slot" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#1-the-client-shrinks-the-photo-and-asks-for-a-slot" aria-label="Anchor">#</a>
    </span>
    
</h3>
<p>The app still resizes on the device, to the longest edge the server keeps (1080px) as a JPEG at 0.7 quality. That resize is for the user&rsquo;s bandwidth, nothing else. Then it asks the API for an upload slot:</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">uuid</span>
</span></span><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">datetime</span> <span class="kn">import</span> <span class="n">timedelta</span>
</span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">boto3</span>
</span></span><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">django.utils</span> <span class="kn">import</span> <span class="n">timezone</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="n">s3</span> <span class="o">=</span> <span class="n">boto3</span><span class="o">.</span><span class="n">client</span><span class="p">(</span><span class="s2">&#34;s3&#34;</span><span class="p">,</span> <span class="n">endpoint_url</span><span class="o">=</span><span class="s2">&#34;https://fly.storage.tigris.dev&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="n">BUCKET</span> <span class="o">=</span> <span class="s2">&#34;example-bucket&#34;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">create_upload_slot</span><span class="p">(</span><span class="n">user</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">    <span class="n">public_id</span> <span class="o">=</span> <span class="n">uuid</span><span class="o">.</span><span class="n">uuid4</span><span class="p">()</span>
</span></span><span class="line"><span class="cl">    <span class="n">key</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">&#34;uploads/raw/</span><span class="si">{</span><span class="n">public_id</span><span class="si">}</span><span class="s2">.jpg&#34;</span>
</span></span><span class="line"><span class="cl">    <span class="n">upload</span> <span class="o">=</span> <span class="n">PhotoUpload</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">create</span><span class="p">(</span>
</span></span><span class="line"><span class="cl">        <span class="n">user</span><span class="o">=</span><span class="n">user</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">        <span class="n">public_id</span><span class="o">=</span><span class="n">public_id</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">        <span class="n">raw_key</span><span class="o">=</span><span class="n">key</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">        <span class="n">expires_at</span><span class="o">=</span><span class="n">timezone</span><span class="o">.</span><span class="n">now</span><span class="p">()</span> <span class="o">+</span> <span class="n">timedelta</span><span class="p">(</span><span class="n">days</span><span class="o">=</span><span class="mi">2</span><span class="p">),</span>
</span></span><span class="line"><span class="cl">    <span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="n">url</span> <span class="o">=</span> <span class="n">s3</span><span class="o">.</span><span class="n">generate_presigned_url</span><span class="p">(</span>
</span></span><span class="line"><span class="cl">        <span class="s2">&#34;put_object&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">        <span class="n">Params</span><span class="o">=</span><span class="p">{</span><span class="s2">&#34;Bucket&#34;</span><span class="p">:</span> <span class="n">BUCKET</span><span class="p">,</span> <span class="s2">&#34;Key&#34;</span><span class="p">:</span> <span class="n">key</span><span class="p">,</span> <span class="s2">&#34;ContentType&#34;</span><span class="p">:</span> <span class="s2">&#34;image/jpeg&#34;</span><span class="p">},</span>
</span></span><span class="line"><span class="cl">        <span class="n">ExpiresIn</span><span class="o">=</span><span class="mi">15</span> <span class="o">*</span> <span class="mi">60</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">        <span class="s2">&#34;upload_id&#34;</span><span class="p">:</span> <span class="nb">str</span><span class="p">(</span><span class="n">upload</span><span class="o">.</span><span class="n">public_id</span><span class="p">),</span>
</span></span><span class="line"><span class="cl">        <span class="s2">&#34;url&#34;</span><span class="p">:</span> <span class="n">url</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">        <span class="s2">&#34;method&#34;</span><span class="p">:</span> <span class="s2">&#34;PUT&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">        <span class="s2">&#34;headers&#34;</span><span class="p">:</span> <span class="p">{</span><span class="s2">&#34;Content-Type&#34;</span><span class="p">:</span> <span class="s2">&#34;image/jpeg&#34;</span><span class="p">},</span>
</span></span><span class="line"><span class="cl">        <span class="s2">&#34;expires_in&#34;</span><span class="p">:</span> <span class="mi">15</span> <span class="o">*</span> <span class="mi">60</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="p">}</span></span></span></code></pre></div></div>
<p>The slot is a row before it&rsquo;s a file. That&rsquo;s what lets the server know who asked for it, when it expires, and later whether it was ever used.</p>

<h3 class="relative group">2. The client PUTs straight to the bucket
    <div id="2-the-client-puts-straight-to-the-bucket" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#2-the-client-puts-straight-to-the-bucket" aria-label="Anchor">#</a>
    </span>
    
</h3>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-ts" data-lang="ts"><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">slot</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">api</span><span class="p">.</span><span class="nx">post</span><span class="p">(</span><span class="s2">&#34;/photo-uploads/&#34;</span><span class="p">);</span>
</span></span><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">res</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">fetch</span><span class="p">(</span><span class="nx">slot</span><span class="p">.</span><span class="nx">url</span><span class="p">,</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">  <span class="nx">method</span>: <span class="kt">slot.method</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">  <span class="nx">headers</span>: <span class="kt">slot.headers</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">  <span class="nx">body</span>: <span class="kt">blob</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"><span class="p">});</span>
</span></span><span class="line"><span class="cl"><span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">res</span><span class="p">.</span><span class="nx">ok</span><span class="p">)</span> <span class="k">throw</span> <span class="k">new</span> <span class="nb">Error</span><span class="p">(</span><span class="s2">&#34;upload refused&#34;</span><span class="p">);</span>
</span></span><span class="line"><span class="cl"><span class="c1">// keep slot.upload_id for the save
</span></span></span></code></pre></div></div>
<p>The API never sees the bytes. This is the same from a browser, from React Native or from a native app; the only difference is that a browser needs a CORS rule on the bucket, which is in the gotchas below.</p>

<h3 class="relative group">3. Tigris tells the backend, and the pipeline runs
    <div id="3-tigris-tells-the-backend-and-the-pipeline-runs" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#3-tigris-tells-the-backend-and-the-pipeline-runs" aria-label="Anchor">#</a>
    </span>
    
</h3>
<p>Within a few seconds Tigris posts an <code>OBJECT_CREATED_PUT</code> for <code>uploads/raw/…</code>, and the handler runs the same image pipeline the base64 path always used, while the person is still typing their title:</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">io</span> <span class="kn">import</span> <span class="n">BytesIO</span>
</span></span><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">PIL</span> <span class="kn">import</span> <span class="n">Image</span><span class="p">,</span> <span class="n">ImageOps</span>
</span></span><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">django.core.files.base</span> <span class="kn">import</span> <span class="n">ContentFile</span>
</span></span><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">django.utils</span> <span class="kn">import</span> <span class="n">timezone</span>
</span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">blurhash</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="n">MAX_BYTES</span> <span class="o">=</span> <span class="mi">15</span> <span class="o">*</span> <span class="mi">1024</span> <span class="o">*</span> <span class="mi">1024</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">on_raw_photo</span><span class="p">(</span><span class="n">event</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">    <span class="k">if</span> <span class="ow">not</span> <span class="n">event</span><span class="o">.</span><span class="n">name</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">&#34;OBJECT_CREATED&#34;</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">        <span class="k">return</span>
</span></span><span class="line"><span class="cl">    <span class="k">if</span> <span class="n">event</span><span class="o">.</span><span class="n">size</span> <span class="o">&gt;</span> <span class="n">MAX_BYTES</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">        <span class="n">s3</span><span class="o">.</span><span class="n">delete_object</span><span class="p">(</span><span class="n">Bucket</span><span class="o">=</span><span class="n">event</span><span class="o">.</span><span class="n">bucket</span><span class="p">,</span> <span class="n">Key</span><span class="o">=</span><span class="n">event</span><span class="o">.</span><span class="n">key</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">        <span class="k">return</span>
</span></span><span class="line"><span class="cl">    <span class="n">upload</span> <span class="o">=</span> <span class="n">PhotoUpload</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">filter</span><span class="p">(</span><span class="n">raw_key</span><span class="o">=</span><span class="n">event</span><span class="o">.</span><span class="n">key</span><span class="p">,</span> <span class="n">processed_at</span><span class="o">=</span><span class="kc">None</span><span class="p">)</span><span class="o">.</span><span class="n">first</span><span class="p">()</span>
</span></span><span class="line"><span class="cl">    <span class="k">if</span> <span class="n">upload</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">        <span class="k">return</span>
</span></span><span class="line"><span class="cl">    <span class="n">data</span> <span class="o">=</span> <span class="n">s3</span><span class="o">.</span><span class="n">get_object</span><span class="p">(</span><span class="n">Bucket</span><span class="o">=</span><span class="n">event</span><span class="o">.</span><span class="n">bucket</span><span class="p">,</span> <span class="n">Key</span><span class="o">=</span><span class="n">event</span><span class="o">.</span><span class="n">key</span><span class="p">)[</span><span class="s2">&#34;Body&#34;</span><span class="p">]</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
</span></span><span class="line"><span class="cl">    <span class="n">process</span><span class="p">(</span><span class="n">upload</span><span class="p">,</span> <span class="n">data</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">to_webp</span><span class="p">(</span><span class="n">im</span><span class="p">,</span> <span class="n">longest</span><span class="p">,</span> <span class="n">quality</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">    <span class="n">im</span> <span class="o">=</span> <span class="n">im</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
</span></span><span class="line"><span class="cl">    <span class="n">im</span><span class="o">.</span><span class="n">thumbnail</span><span class="p">((</span><span class="n">longest</span><span class="p">,</span> <span class="n">longest</span><span class="p">))</span>
</span></span><span class="line"><span class="cl">    <span class="n">buf</span> <span class="o">=</span> <span class="n">BytesIO</span><span class="p">()</span>
</span></span><span class="line"><span class="cl">    <span class="n">im</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">buf</span><span class="p">,</span> <span class="s2">&#34;WEBP&#34;</span><span class="p">,</span> <span class="n">quality</span><span class="o">=</span><span class="n">quality</span><span class="p">,</span> <span class="n">icc_profile</span><span class="o">=</span><span class="n">im</span><span class="o">.</span><span class="n">info</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&#34;icc_profile&#34;</span><span class="p">))</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="n">ContentFile</span><span class="p">(</span><span class="n">buf</span><span class="o">.</span><span class="n">getvalue</span><span class="p">())</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">process</span><span class="p">(</span><span class="n">upload</span><span class="p">,</span> <span class="n">data</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">    <span class="k">with</span> <span class="n">Image</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="n">BytesIO</span><span class="p">(</span><span class="n">data</span><span class="p">))</span> <span class="k">as</span> <span class="n">im</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">        <span class="n">im</span> <span class="o">=</span> <span class="n">ImageOps</span><span class="o">.</span><span class="n">exif_transpose</span><span class="p">(</span><span class="n">im</span><span class="p">)</span><span class="o">.</span><span class="n">convert</span><span class="p">(</span><span class="s2">&#34;RGB&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">        <span class="n">im</span><span class="o">.</span><span class="n">thumbnail</span><span class="p">((</span><span class="mi">1080</span><span class="p">,</span> <span class="mi">1080</span><span class="p">))</span>
</span></span><span class="line"><span class="cl">        <span class="n">upload</span><span class="o">.</span><span class="n">width</span><span class="p">,</span> <span class="n">upload</span><span class="o">.</span><span class="n">height</span> <span class="o">=</span> <span class="n">im</span><span class="o">.</span><span class="n">size</span>
</span></span><span class="line"><span class="cl">        <span class="n">upload</span><span class="o">.</span><span class="n">dominant_color</span> <span class="o">=</span> <span class="s2">&#34;#</span><span class="si">%02x%02x%02x</span><span class="s2">&#34;</span> <span class="o">%</span> <span class="n">im</span><span class="o">.</span><span class="n">resize</span><span class="p">((</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">))</span><span class="o">.</span><span class="n">getpixel</span><span class="p">((</span><span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">))</span>
</span></span><span class="line"><span class="cl">        <span class="n">upload</span><span class="o">.</span><span class="n">image</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="sa">f</span><span class="s2">&#34;</span><span class="si">{</span><span class="n">upload</span><span class="o">.</span><span class="n">public_id</span><span class="si">}</span><span class="s2">.webp&#34;</span><span class="p">,</span> <span class="n">to_webp</span><span class="p">(</span><span class="n">im</span><span class="p">,</span> <span class="mi">1080</span><span class="p">,</span> <span class="mi">80</span><span class="p">),</span> <span class="n">save</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">        <span class="n">upload</span><span class="o">.</span><span class="n">thumbnail</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="sa">f</span><span class="s2">&#34;</span><span class="si">{</span><span class="n">upload</span><span class="o">.</span><span class="n">public_id</span><span class="si">}</span><span class="s2">_t.webp&#34;</span><span class="p">,</span> <span class="n">to_webp</span><span class="p">(</span><span class="n">im</span><span class="p">,</span> <span class="mi">320</span><span class="p">,</span> <span class="mi">70</span><span class="p">),</span> <span class="n">save</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">        <span class="n">upload</span><span class="o">.</span><span class="n">blurhash</span> <span class="o">=</span> <span class="n">blurhash</span><span class="o">.</span><span class="n">encode</span><span class="p">(</span><span class="n">im</span><span class="o">.</span><span class="n">resize</span><span class="p">((</span><span class="mi">32</span><span class="p">,</span> <span class="mi">32</span><span class="p">)),</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="n">upload</span><span class="o">.</span><span class="n">processed_at</span> <span class="o">=</span> <span class="n">timezone</span><span class="o">.</span><span class="n">now</span><span class="p">()</span>
</span></span><span class="line"><span class="cl">    <span class="n">upload</span><span class="o">.</span><span class="n">save</span><span class="p">()</span></span></span></code></pre></div></div>
<p>The first line of <code>on_raw_photo</code> matters more than it looks. The router sends every event under <code>uploads/raw/</code> here, including the deletes from the orphan sweep and from the size check itself, and <code>get_object</code> on a key that&rsquo;s gone would only raise.</p>
<p>So the client&rsquo;s shrink was only ever for the pipe. The server still decides the format (WebP), the sizes, the thumbnail, the placeholder, and the dimensions it stores. Re-encoding through Pillow also drops the EXIF block, which is where phone cameras put the GPS position, so the privacy strip comes free. The ICC profile would go the same way, which shifts wide-gamut phone photos a touch, so <code>to_webp</code> passes it through. If you&rsquo;d rather keep pixel-perfect originals, don&rsquo;t shrink on the client; the flow is identical, the PUTs are just bigger, and bigger PUTs are precisely the thing this design makes cheap. One catch: originals from an iPhone are HEIC, which Pillow only opens with <code>pillow-heif</code> installed.</p>

<h3 class="relative group">4. The save attaches the photos and never waits on the webhook
    <div id="4-the-save-attaches-the-photos-and-never-waits-on-the-webhook" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#4-the-save-attaches-the-photos-and-never-waits-on-the-webhook" aria-label="Anchor">#</a>
    </span>
    
</h3>
<p>When they hit Post, the request carries <code>upload_ids</code> instead of image data. The server looks each one up, with three outcomes:</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">take</span><span class="p">(</span><span class="n">user</span><span class="p">,</span> <span class="n">upload_ids</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">    <span class="n">uploads</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">PhotoUpload</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">filter</span><span class="p">(</span>
</span></span><span class="line"><span class="cl">        <span class="n">user</span><span class="o">=</span><span class="n">user</span><span class="p">,</span> <span class="n">public_id__in</span><span class="o">=</span><span class="n">upload_ids</span><span class="p">,</span> <span class="n">expires_at__gt</span><span class="o">=</span><span class="n">timezone</span><span class="o">.</span><span class="n">now</span><span class="p">()</span>
</span></span><span class="line"><span class="cl">    <span class="p">))</span>
</span></span><span class="line"><span class="cl">    <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">uploads</span><span class="p">)</span> <span class="o">!=</span> <span class="nb">len</span><span class="p">(</span><span class="nb">set</span><span class="p">(</span><span class="n">upload_ids</span><span class="p">)):</span>     <span class="c1"># expired, or not this user&#39;s</span>
</span></span><span class="line"><span class="cl">        <span class="k">raise</span> <span class="n">UploadNotReadyError</span><span class="p">(</span><span class="n">upload_ids</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">for</span> <span class="n">upload</span> <span class="ow">in</span> <span class="n">uploads</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">        <span class="k">if</span> <span class="n">upload</span><span class="o">.</span><span class="n">processed_at</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>          <span class="c1"># the webhook hasn&#39;t got here yet</span>
</span></span><span class="line"><span class="cl">            <span class="n">data</span> <span class="o">=</span> <span class="n">read_raw</span><span class="p">(</span><span class="n">upload</span><span class="p">)</span>              <span class="c1"># None if the PUT never landed</span>
</span></span><span class="line"><span class="cl">            <span class="k">if</span> <span class="n">data</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">                <span class="k">raise</span> <span class="n">UploadNotReadyError</span><span class="p">(</span><span class="n">upload</span><span class="o">.</span><span class="n">public_id</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">            <span class="n">process</span><span class="p">(</span><span class="n">upload</span><span class="p">,</span> <span class="n">data</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="n">uploads</span></span></span></code></pre></div></div>
<p>If the notification already ran, the processed files are attached as they are. If the file is in the bucket but the notification hasn&rsquo;t arrived, the save processes it inline, right there, so a post never waits on the webhook. The notification is a head start, not a dependency. And if the file isn&rsquo;t there at all, or an id has expired or belongs to someone else, the save returns a validation error the app retries after a moment; the app waits for its own PUTs before sending, so in practice that&rsquo;s rare.</p>
<p>A few details that made it hold up. The slot belongs to a user, so the server only accepts upload ids the same person created; otherwise anyone who saw an id could attach your photo to their listing. Everything falls back: if the slot request fails, the PUT is refused, or the API is an older version, the app sends base64 exactly as before. It only takes the upload path when every new photo has a slot, so a post&rsquo;s pictures are all uploads or all base64, and their order stays one list. And orphans expire: people pick photos and then give up, so an upload that&rsquo;s never attached expires after two days and the existing orphaned-media sweep cleans its files up.</p>

<h2 class="relative group">Presigned URL upload gotchas: CORS, size limits, expiry, multiple files
    <div id="presigned-url-upload-gotchas-cors-size-limits-expiry-multiple-files" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#presigned-url-upload-gotchas-cors-size-limits-expiry-multiple-files" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>Most of what goes wrong with presigned uploads goes wrong before the notification ever fires. These are the ones I&rsquo;d want on one page.</p>

<h3 class="relative group">CORS, for browsers only
    <div id="cors-for-browsers-only" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#cors-for-browsers-only" aria-label="Anchor">#</a>
    </span>
    
</h3>
<p>A native app doesn&rsquo;t care. A browser sends a preflight <code>OPTIONS</code> to the bucket before the PUT, and the upload fails with a CORS error unless the bucket&rsquo;s CORS rules allow <code>PUT</code> from your origin with the <code>Content-Type</code> header. Tigris takes the standard S3 CORS configuration, so the rule looks the same as on AWS. It&rsquo;s the first thing to check when a presigned PUT works from <code>curl</code> and not from the page.</p>
<p><figure><span class="mc-img"><img
    class="my-0 rounded-md"
    loading="lazy"
    decoding="async"
    fetchpriority="auto"
    alt="Tigris bucket settings, CORS and headers tab: one rule allowing PUT from two origins with the Content-Type header"
    width="1204"
    height="671"
    src="https://jared.lynskey.co.nz/en/posts/2026/tigris-object-notifications/tigris-bucket-cors_hu_7f61261034a8a56.webp"
    srcset="https://jared.lynskey.co.nz/en/posts/2026/tigris-object-notifications/tigris-bucket-cors_hu_7f61261034a8a56.webp 800w, https://jared.lynskey.co.nz/en/posts/2026/tigris-object-notifications/tigris-bucket-cors_hu_3aa47360232f6475.webp 1280w"
    sizes="(min-width: 768px) 50vw, 65vw"
    data-zoom-src="https://jared.lynskey.co.nz/en/posts/2026/tigris-object-notifications/tigris-bucket-cors_hu_3aa47360232f6475.webp"></span><figcaption>One CORS rule is all a browser upload needs: your origins, PUT, and the Content-Type header the signature expects.</figcaption></figure>
</p>

<h3 class="relative group">The content type is part of the signature
    <div id="the-content-type-is-part-of-the-signature" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#the-content-type-is-part-of-the-signature" aria-label="Anchor">#</a>
    </span>
    
</h3>
<p>Signing with <code>ContentType</code> means the client must send exactly that header, or the bucket answers 403 <code>SignatureDoesNotMatch</code>. That&rsquo;s why the slot returns the headers along with the URL. Sign it on purpose, but know what it buys you: the header is enforced, the content isn&rsquo;t, so a PDF labelled <code>image/jpeg</code> gets through. The pipeline is the real check; Pillow refuses to open what isn&rsquo;t an image, and the row is never marked processed.</p>

<h3 class="relative group">Size limits
    <div id="size-limits" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#size-limits" aria-label="Anchor">#</a>
    </span>
    
</h3>
<p>A presigned PUT URL carries no size limit. If you need a hard stop before the bytes land, a presigned POST with a <code>content-length-range</code> policy condition does that. I didn&rsquo;t bother, because the notification carries the object&rsquo;s <code>size</code>: the handler deletes anything over the limit before reading it, and the save-time path reads at most the limit plus one byte so it can tell. The app sends a few hundred kilobytes; anything near fifteen megabytes isn&rsquo;t ours.</p>

<h3 class="relative group">Expiry
    <div id="expiry" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#expiry" aria-label="Anchor">#</a>
    </span>
    
</h3>
<p>Two clocks. The presigned URL is good for fifteen minutes, which is plenty for one PUT and short enough that a leaked URL is useless soon. The slot row is good for two days, which covers someone picking photos, putting the phone down and finishing the listing tomorrow. The client treats a pending upload as stale after a day and starts a fresh one rather than hoping.</p>

<h3 class="relative group">Multiple files, and bigger ones
    <div id="multiple-files-and-bigger-ones" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#multiple-files-and-bigger-ones" aria-label="Anchor">#</a>
    </span>
    
</h3>
<p>One slot per file. Ten photos is ten slots and ten parallel PUTs, each with its own progress (an <code>XMLHttpRequest</code> gives you <code>upload.onprogress</code>; <code>fetch</code> doesn&rsquo;t), which you can&rsquo;t get from one giant base64 POST. Past a few megabytes, use multipart (parts are 5 MiB minimum, so it&rsquo;s for files well beyond that): create the multipart upload on the server, presign each <code>upload_part</code>, and complete it when the parts are in. Tigris sends <code>OBJECT_CREATED_MULTIPART</code> when the upload is completed, so the same handler fires and nothing else changes.</p>

<h3 class="relative group">Out of order
    <div id="out-of-order" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#out-of-order" aria-label="Anchor">#</a>
    </span>
    
</h3>
<p>Events can arrive out of order, because the object may have been written in more than one region. For uploads it doesn&rsquo;t matter: one key, one event. For a key that gets overwritten, trust the eTag and the object&rsquo;s <code>Last-Modified</code>, not the order the webhooks reached you.</p>

<h2 class="relative group">The same signal for logs: alerts to Discord
    <div id="the-same-signal-for-logs-alerts-to-discord" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#the-same-signal-for-logs-alerts-to-discord" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>Once the bucket can talk to the backend, anything that lands in it can be reacted to the same way, and the second thing I pointed it at was logs.</p>
<p>The app&rsquo;s logs are shipped to the bucket as small gzipped files, one every five minutes or so per app. For most of the year a GitHub Action on a 15-minute cron read the last 20 minutes of that archive with DuckDB, found the slow-request lines, and posted a digest to Discord. On paper that&rsquo;s fine. In practice GitHub&rsquo;s scheduled workflows don&rsquo;t run on time when the platform is busy, so &ldquo;every 15 minutes&rdquo; was more like &ldquo;every few hours&rdquo;, and when it finally ran it had a bigger window to chew through and occasionally ran out of memory. The overlap I&rsquo;d added so nothing slipped between runs meant the same slow request sometimes showed up twice. It was a lot of machinery for &ldquo;tell me when a request is slow&rdquo;.</p>
<p>Now each log file is read once, as it lands. They&rsquo;re a few kilobytes gzipped. The handler pulls the slow-request lines out, and if there are enough of them, or one really slow one, it posts a digest to a Discord channel through a webhook. Same for a burst of 5xx responses in one file. A Discord webhook from Python is one call:</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">requests</span>
</span></span><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">django.conf</span> <span class="kn">import</span> <span class="n">settings</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">discord_alert</span><span class="p">(</span><span class="n">text</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">    <span class="n">requests</span><span class="o">.</span><span class="n">post</span><span class="p">(</span>
</span></span><span class="line"><span class="cl">        <span class="n">settings</span><span class="o">.</span><span class="n">DISCORD_WEBHOOK_URL</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">        <span class="n">json</span><span class="o">=</span><span class="p">{</span><span class="s2">&#34;content&#34;</span><span class="p">:</span> <span class="n">text</span><span class="p">[:</span><span class="mi">1900</span><span class="p">]},</span>  <span class="c1"># Discord caps a message at 2000 chars</span>
</span></span><span class="line"><span class="cl">        <span class="n">timeout</span><span class="o">=</span><span class="mi">5</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="p">)</span></span></span></code></pre></div></div>
<p>No cron, no overlap, no duplicates, and the alert arrives within a few minutes of the request instead of whenever GitHub got round to it. The Action is gone.</p>
<p>While each file is open anyway, I also count what&rsquo;s in it (errors, warnings, health checks) into an hourly tally table. The daily summary used to scan a whole day of the archive. Now it reads a few hundred rows, and only falls back to the archive if the tally has a gap.</p>

<h2 class="relative group">A guard on deletes
    <div id="a-guard-on-deletes" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#a-guard-on-deletes" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>This one I didn&rsquo;t plan. It fell out of having <code>OBJECT_DELETED</code> events. When a media file is deleted, the handler checks whether any row still points at it. If one does, that&rsquo;s a bug somewhere (or a person in the bucket console with a heavy finger), so it clears the column (the app shows a placeholder instead of a broken image) and reports it.</p>
<p>It also counts deletions per hour. Normal housekeeping removes a handful. If it ever sees a couple of hundred in an hour, I want to hear about that while it&rsquo;s happening, not when users start reporting blank photos.</p>

<h2 class="relative group">Things worth knowing
    <div id="things-worth-knowing" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#things-worth-knowing" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>Respond fast. Tigris gives each delivery ten seconds. Reading a 5 KB log file or processing one phone photo fits; anything that might take longer goes to a queue and the view returns 200, or Tigris times out and retries, and you&rsquo;re back to deduplicating.</p>
<p>Filter at the bucket. My log shipper writes a file every few minutes per app, forever. If you only care about one prefix, say so in the notification&rsquo;s filter so your app isn&rsquo;t woken for everything else.</p>
<p>Test with a fake body. The payload is small and stable, so the tests post JSON at the view with the right <code>Authorization</code> header and assert what the handler did. No real storage involved.</p>
<p>Don&rsquo;t make it load-bearing on day one. Every use here has a fallback: the save can process a photo itself, the daily summary can still scan the archive, the app can still send base64. Notifications made each of them faster and cheaper, but none of them falls over if a webhook goes missing.</p>
<p>The best bit of the week was deleting that workflow file. Forty lines of YAML, a Python script, two repository secrets and a recurring &ldquo;why didn&rsquo;t the alert fire&rdquo; are gone, and the replacement is one handler that reads one small file when it arrives. The uploads were the point, though. The photos go up as fast as the network allows, the server still gets the last word on every one of them, and nobody had to build a bridge between the bucket and the backend, because the bucket now just says what happened.</p>
]]></content:encoded>
      <media:content url="https://jared.lynskey.co.nz/en/posts/2026/tigris-object-notifications/featured.jpg" medium="image" type="image/jpeg" />
    </item>
    <item>
      <title>Deploy Frontend and Backend Separately, Without Redeploying the API</title>
      <link>https://jared.lynskey.co.nz/en/posts/2026/2026-09-14-web-releases-without-redeploying/</link>
      <guid isPermaLink="true">https://jared.lynskey.co.nz/en/posts/2026/2026-09-14-web-releases-without-redeploying/</guid>
      <pubDate>Mon, 14 Sep 2026 00:00:00 +1200</pubDate>
      <dc:creator>Jared Lynskey</dc:creator>
      <category>Fly.io</category>
      <category>Django</category>
      <category>DevOps</category>
      <category>Cloudflare</category>
      <category>Deployment</category>
      <category>Object Storage</category>
      <description>How I deploy the frontend and backend separately: upload the build to object storage under a versioned prefix, flip a pointer Django reads, and purge Cloudflare via the API.</description>
      <content:encoded><![CDATA[<p>To deploy the frontend and backend separately, I stopped copying the web build into the API&rsquo;s Docker image. A script now builds the front end, uploads it to object storage under a versioned prefix and flips one pointer object; Django reads that pointer on every request and serves whichever version it names. One call to Cloudflare&rsquo;s purge cache API then clears the cached HTML. A CSS fix takes about a minute and restarts nothing.</p>
<p>Until recently, every change to the web side of my marketplace app went out the same way as a change to the Django API: build the Docker image, push it, and roll it across every machine on Fly.io. The web front end (the mobile app&rsquo;s web export, plus the marketing pages a static site generator builds) was copied into the image next to the API, so there was no other way to get it out. A CSS fix took about seven minutes and restarted eleven machines. One day in August I did that six times.</p>
<p>The rest of this post walks through that deploy script step by step, including how it waits until every machine is serving the new version, and then the Django side that reads the pointer.</p>

<h2 class="relative group">Why the old way was a problem
    <div id="why-the-old-way-was-a-problem" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#why-the-old-way-was-a-problem" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>The restart was the biggest cost. Eleven machines rolling means eleven cold boots, and on a shared CPU Django takes about thirteen seconds to come up, seven of which is just importing Django. I had a warm-up step at the end of the deploy script that hit every machine so a real visitor wouldn&rsquo;t be the one to pay for that, which meant every web release ended with waiting on the slowest machine. I&rsquo;ve written before about what those cold boots cost on Fly.io, in <a href="https://jared.lynskey.co.nz/en/posts/2026/2026-09-09-scale-to-zero-504s/" >Scale to Zero, and Every Restart Is a 30-Second Outage</a>. The short version is that every restart is a window where a visitor can get a 504, so a deploy path that restarts nothing closes that window for web releases entirely.</p>
<p>The second problem was that the static site&rsquo;s output was split. The HTML went into the image, but the RSS feeds and the search index went straight to the bucket. They were built from the same source, just not at the same moment, so they could disagree with each other and nobody would notice until they did.</p>
<p>The third one is more about habit. When a release costs seven minutes and a restart, you stop doing small ones. I&rsquo;d let three or four fixes pile up rather than ship them one at a time, and that&rsquo;s exactly how a one-line change ends up going out next to something broken.</p>

<h2 class="relative group">The layout in the bucket
    <div id="the-layout-in-the-bucket" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#the-layout-in-the-bucket" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>Everything lives in an S3-compatible bucket under a <code>web/</code> prefix:</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">web/current                              ← a text file containing one version string
</span></span><span class="line"><span class="cl">web/a1b2c3d-20260914021500/manifest.json ← every file in this version
</span></span><span class="line"><span class="cl">web/a1b2c3d-20260914021500/app/...       ← the app&#39;s web export
</span></span><span class="line"><span class="cl">web/a1b2c3d-20260914021500/site/...      ← the static site&#39;s output
</span></span><span class="line"><span class="cl">web/9f8e7d6-20260912093012/...           ← the previous version, still there</span></span></code></pre></div></div>
<p>A version is the short git SHA plus a UTC timestamp, so you can read off what commit it is and when it went up. <code>web/current</code> is the only thing that changes when a release goes live.</p>

<h2 class="relative group">What the deploy script does
    <div id="what-the-deploy-script-does" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#what-the-deploy-script-does" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>The whole thing is one script, run from the repo root. In order:</p>
<p><strong>1. Refuse a dirty tree.</strong> The first thing it does is <code>git status --porcelain</code>, and if anything comes back it exits. Whatever is on disk is what gets uploaded, and the old deploy script for the marketing site once shipped uncommitted copy that way. There&rsquo;s an override flag for when you know what you&rsquo;re doing, but I haven&rsquo;t used it. It then computes the version string from <code>git rev-parse --short HEAD</code> and the current time.</p>
<p><strong>2. Build both trees.</strong> For the app it runs the web export with the production API and WebSocket URLs baked in, followed by a Brotli precompression pass. The precompression is worth explaining: Cloudflare will Brotli-compress on the fly, but at a low quality level, and on this bundle it measured worse than gzip (2.04MB against 1.66MB). Compressing once at build time with quality 11 gets it to 1.28MB, so the script writes a <code>.br</code> twin next to every JS, CSS and JSON file over 1KB. For the marketing site it runs the static site generator with minification on.</p>
<p><strong>3. Upload.</strong> It walks both build directories and uploads every file to <code>web/&lt;version&gt;/app/...</code> and <code>web/&lt;version&gt;/site/...</code> with sixteen threads. Each object gets a <code>Content-Type</code> guessed from its filename (with the <code>.br</code> stripped first), and the Brotli twins get <code>Content-Encoding: br</code> as well. Only when every file is up does it write <code>manifest.json</code>, a JSON object with two lists of relative paths. The order matters: the Django side treats a version without a manifest as not existing, so a half-uploaded version can never be made current. Right now that&rsquo;s a few hundred files and the upload takes about six seconds.</p>
<p><strong>4. Flip the pointer.</strong> One <code>put_object</code> writes the version string to <code>web/current</code>. That&rsquo;s the release.</p>
<p><strong>5. Wait for the machines.</strong> Each API machine caches the pointer for five seconds, so for a moment after the flip some of them are still on the old version. The script polls a version endpoint on the API every half second and counts consecutive answers that carry the new version. Fly load-balances across all the machines, so a run of sixteen matching answers in a row (more than the machine count) is how it knows every one of them has re-read the pointer. One stale answer resets the count. It gives up after 90 seconds and tells you to check by hand; the first real flip took a bit over 30 seconds, which is why the limit isn&rsquo;t lower.</p>
<p><strong>6. Purge Cloudflare.</strong> One API call, covered below. If it fails, the script prints a warning and carries on.</p>
<p><strong>7. Post to the team chat.</strong> The same webhook the runtime error alerts use gets a one-liner with the version and who ran it, so a burst of errors can be read against what just changed. Rollbacks post in amber.</p>
<p>Start to finish, a release is a bit over a minute, most of which is the web export.</p>

<h2 class="relative group">The Django side
    <div id="the-django-side" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#the-django-side" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>Django was already the thing serving the front end. Every non-API URL is answered with the prerendered page from the web export for that route, and the marketing, privacy and support pages come from the static site&rsquo;s output. Those views used to open files on disk. Now they read through one small layer that knows about the bucket.</p>
<p>Per request it resolves three things: the current version (from <code>web/current</code>, cached for five seconds), the manifest for that version (cached for as long as that version is current), and the file itself (a small in-memory LRU of file bodies, so a page&rsquo;s HTML and its main bundle stay in memory). The manifest is what makes 404s cheap: if a path isn&rsquo;t listed, the answer is no without asking the bucket. If the client accepts Brotli, it reads the <code>.br</code> twin instead and sets the encoding header. Assets go out with a one-year immutable cache header, since their filenames are content-hashed. Anything derived from the file list, such as the table that maps routes to prerendered pages, is cached per version too, so a new deploy rebuilds it on the first request and not before.</p>
<p>Then there&rsquo;s the endpoint the deploy script polls:</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">web_version</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">    <span class="c1"># bundle: the layer that reads web/current and the manifest</span>
</span></span><span class="line"><span class="cl">    <span class="n">response</span> <span class="o">=</span> <span class="n">JsonResponse</span><span class="p">({</span><span class="s2">&#34;source&#34;</span><span class="p">:</span> <span class="n">bundle</span><span class="o">.</span><span class="n">source</span><span class="p">(),</span> <span class="s2">&#34;version&#34;</span><span class="p">:</span> <span class="n">bundle</span><span class="o">.</span><span class="n">version</span><span class="p">()})</span>
</span></span><span class="line"><span class="cl">    <span class="n">response</span><span class="p">[</span><span class="s2">&#34;Cache-Control&#34;</span><span class="p">]</span> <span class="o">=</span> <span class="s2">&#34;no-store&#34;</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="n">response</span></span></span></code></pre></div></div>
<p>It answers with <code>bucket:&lt;version&gt;</code> in production or <code>filesystem</code> when it&rsquo;s serving from the image. Each Fly machine answers for itself, so hitting it a dozen times tells you whether the fleet has converged. It&rsquo;s also the quickest way to check what a machine thinks is live when something looks off.</p>

<h2 class="relative group">Purging the Cloudflare cache through the API
    <div id="purging-the-cloudflare-cache-through-the-api" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#purging-the-cloudflare-cache-through-the-api" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>HTML on the site is cached at Cloudflare&rsquo;s edge for five minutes. Without a purge, a deploy would be live on the machines but most visitors would keep getting the old page for up to five minutes, which defeats the purpose.</p>
<p>The purge is a single API call:</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="n">urllib</span><span class="o">.</span><span class="n">request</span><span class="o">.</span><span class="n">Request</span><span class="p">(</span>
</span></span><span class="line"><span class="cl">    <span class="sa">f</span><span class="s2">&#34;https://api.cloudflare.com/client/v4/zones/</span><span class="si">{</span><span class="n">zone</span><span class="si">}</span><span class="s2">/purge_cache&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="n">data</span><span class="o">=</span><span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">({</span><span class="s2">&#34;hosts&#34;</span><span class="p">:</span> <span class="p">[</span><span class="s2">&#34;example.com&#34;</span><span class="p">,</span> <span class="s2">&#34;www.example.com&#34;</span><span class="p">]})</span><span class="o">.</span><span class="n">encode</span><span class="p">(),</span>
</span></span><span class="line"><span class="cl">    <span class="n">headers</span><span class="o">=</span><span class="p">{</span><span class="s2">&#34;Authorization&#34;</span><span class="p">:</span> <span class="sa">f</span><span class="s2">&#34;Bearer </span><span class="si">{</span><span class="n">token</span><span class="si">}</span><span class="s2">&#34;</span><span class="p">,</span> <span class="s2">&#34;Content-Type&#34;</span><span class="p">:</span> <span class="s2">&#34;application/json&#34;</span><span class="p">},</span>
</span></span><span class="line"><span class="cl"><span class="p">)</span></span></span></code></pre></div></div>
<p>I purge by host rather than by URL so I don&rsquo;t have to list every page. The assets don&rsquo;t need purging at all: the filenames are content-hashed, so a new build points at new filenames and the old ones just stop being referenced. Only the HTML that references them needs to be refreshed.</p>
<p>The script waits for the machines <em>before</em> purging, on purpose. If it purged first, Cloudflare could refill its cache from a machine that was still on the old version, and that page would then sit at the edge for another five minutes.</p>
<p>If the purge fails, the script prints a warning and finishes anyway. The cache expires on its own within five minutes, so the worst case is a slow release rather than a broken one. I&rsquo;d rather that than have the deploy abort at the last step.</p>

<h2 class="relative group">Checking a build before it goes live
    <div id="checking-a-build-before-it-goes-live" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#checking-a-build-before-it-goes-live" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>A no-flip mode does everything up to step 3 and stops. The version is in the bucket but nothing serves it. To look at it, I set an environment variable on one machine that pins it to that build regardless of what <code>web/current</code> says. When it looks right, the rollback command makes it current. It&rsquo;s called rollback but it&rsquo;s really &ldquo;point <code>current</code> at this version&rdquo;, and going forward and going back are the same operation.</p>

<h2 class="relative group">Rolling back
    <div id="rolling-back" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#rolling-back" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>Old versions stay in the bucket, so rolling back is the same command with an earlier version, and a list command shows what&rsquo;s there, newest first. It does the same wait and purge as a forward deploy. I haven&rsquo;t needed it yet. Nothing prunes old versions at the moment; they&rsquo;re small and I&rsquo;d rather have them than not.</p>

<h2 class="relative group">What stayed the same
    <div id="what-stayed-the-same" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#what-stayed-the-same" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>The API still deploys the old way, through its own deploy script, and it should: it&rsquo;s stateful and it&rsquo;s where the migrations run. The Dockerfile still copies both front-end builds into the image too. That&rsquo;s deliberate. When there&rsquo;s no bucket configured (local development, the test suite) or the bucket can&rsquo;t be read, the serving layer falls back to those directories on disk, so a storage outage means visitors get the front end that shipped with the last image rather than an error page. One environment variable forces that behaviour in production if the bucket ever needs to be taken out of the loop.</p>
<p>The front end changes far more often than the API does, and each release now takes about a minute instead of seven. I&rsquo;ve stopped batching without trying to.</p>
]]></content:encoded>
      <media:content url="https://jared.lynskey.co.nz/en/posts/2026/2026-09-14-web-releases-without-redeploying/featured.jpg" medium="image" type="image/jpeg" />
    </item>
  </channel>
</rss>
