<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:media="http://search.yahoo.com/mrss/">
  <channel>
    <title>Presigned URLs · 仁才德</title>
    <link>https://jared.lynskey.co.nz/zh-cn/tags/presigned-urls/</link>
    <description>关于软件工程与研发团队领导力的短文——绩效评估、招聘与团队扩张、CI/CD、DevOps 监控、智能体 AI。写于首尔。</description>
    <generator>Hugo</generator>
    <language>zh-CN</language>
    <copyright>© 2026 仁才德</copyright>
    <lastBuildDate>Fri, 13 Nov 2026 00:00:00 +1300</lastBuildDate>
    <atom:link href="https://jared.lynskey.co.nz/zh-cn/tags/presigned-urls/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>AWS Emulator 搭建教程：免费的 LocalStack 替代</title>
      <link>https://jared.lynskey.co.nz/zh-cn/posts/2026/aws-emulator-setup/</link>
      <guid isPermaLink="true">https://jared.lynskey.co.nz/zh-cn/posts/2026/aws-emulator-setup/</guid>
      <pubDate>Fri, 13 Nov 2026 00:00:00 +1300</pubDate>
      <dc:creator>Jared Lynskey</dc:creator>
      <category>AWS Emulator</category>
      <category>LocalStack</category>
      <category>Moto</category>
      <category>Floci</category>
      <category>S3</category>
      <category>Testing</category>
      <category>Docker</category>
      <category>GitHub Actions</category>
      <category>Python</category>
      <category>Presigned URLs</category>
      <category>OTA Updates</category>
      <description>用 Docker、Linux 和 GitHub Actions 搭起 AWS 官方免费模拟器，让 boto3 指向它，再看它在我的 S3 本地测试里抓到哪些 LocalStack 4.12 漏掉的问题。</description>
      <content:encoded><![CDATA[<p>AWS 在 2026 年 11 月 11 日发布了 <a href="https://github.com/aws/aws-emulator"  target="_blank" rel="noreferrer">AWS Emulator</a>。它是 AWS API 的本地副本，跑在你自己的机器上：免费，不需要 AWS 账号，代码属于公有领域（CC0），覆盖 200 多个服务。距 LocalStack 开始要求当前镜像必须有账号和 auth token 才能运行，刚好过去七个月，所以很多人都把它看作自己一直在等的那个免费 LocalStack 替代。</p>
<p>这篇文章大部分是搭建步骤：Mac 或 Windows 上的 Docker、Linux 安装包、让 AWS CLI 和 boto3 指向它的环境变量、Docker Compose、pytest 和 GitHub Actions。文中每一条命令我都在 1.0.342 版本上亲手跑过。搭建之后，我把自己两个项目的 S3 测试指向它，和 LocalStack 4.12、Moto、Floci 放在一起比，看它哪些做对了、哪些做错了。这两个项目，一个是我的交易平台应用里的上传流程，另一个是自托管的 Codemagic Patch 服务器用来存放 OTA bundle 的 bucket。</p>

<h2 class="relative group">一分钟看懂 AWS Emulator
    <div id="一分钟看懂-aws-emulator" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e4%b8%80%e5%88%86%e9%92%9f%e7%9c%8b%e6%87%82-aws-emulator" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>AWS 没有发布容器镜像。你要用一个九行的 Dockerfile 自己构建（它会安装官方的 Linux 安装包），在 9297 端口跑起来，再设几个环境变量。之后你的 SDK 连的就是模拟器而不是 AWS，代码一行都不用改：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">docker build -t aws-emulator .           <span class="c1"># the Dockerfile is below</span>
</span></span><span class="line"><span class="cl">mkdir -p ~/.aws/emulator
</span></span><span class="line"><span class="cl">docker run --rm -d --name aws-emulator -p 127.0.0.1:9297:9297 <span class="se">\
</span></span></span><span class="line"><span class="cl">  -e <span class="nv">HOME</span><span class="o">=</span><span class="s2">&#34;</span><span class="nv">$HOME</span><span class="s2">&#34;</span> -v <span class="s2">&#34;</span><span class="nv">$HOME</span><span class="s2">/.aws/emulator:</span><span class="nv">$HOME</span><span class="s2">/.aws/emulator&#34;</span> <span class="se">\
</span></span></span><span class="line"><span class="cl">  aws-emulator -p <span class="m">9297</span> --telemetry off
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="nb">export</span> <span class="nv">AWS_ENDPOINT_URL</span><span class="o">=</span>https://127.0.0.1:9297
</span></span><span class="line"><span class="cl"><span class="nb">export</span> <span class="nv">AWS_SHARED_CREDENTIALS_FILE</span><span class="o">=</span>~/.aws/emulator/9297/emulator_credentials
</span></span><span class="line"><span class="cl"><span class="nb">export</span> <span class="nv">AWS_CONFIG_FILE</span><span class="o">=</span>~/.aws/emulator/9297/config
</span></span><span class="line"><span class="cl"><span class="nb">export</span> <span class="nv">AWS_CA_BUNDLE</span><span class="o">=</span>~/.aws/emulator/9297/ca.pem
</span></span><span class="line"><span class="cl">aws sts get-caller-identity              <span class="c1"># &#34;Account&#34;: &#34;000000000000&#34;</span></span></span></code></pre></div></div>
<figure><video
    class="my-0 rounded-md"
    autoplay loop muted playsinline preload="metadata"
    poster="https://jared.lynskey.co.nz/en/posts/2026/aws-emulator-setup/aws-emulator-setup.jpg" style="background:url('data:image/webp;base64,UklGRloAAABXRUJQVlA4IE4AAACQBACdASogABIAPw1yskwmIyIhKAqoxiGJ6QDG9BPdK4I2hDwC1QaVpF521UAA/ui2Gzha/NBVWRJqzq2oTgehY/P2cqAeQAizoM8AAAA=') center/cover no-repeat"
    width="1280"
    height="720"
    aria-label="终端：Dockerfile、启动 AWS Emulator 容器、健康检查返回 ok、aws sts get-caller-identity 返回账号 000000000000、一次 S3 上传，以及一个收到 HTTP 501 的 CORS 预检">
    <source src="https://jared.lynskey.co.nz/en/posts/2026/aws-emulator-setup/aws-emulator-setup.mp4" type="video/mp4">
  </video><figcaption>在我的 Mac 上录下的整个搭建过程：启动、健康检查、模拟器的账号、一次 S3 上传，还有它目前做不到的那件事（CORS 预检返回 501，下文细说）。</figcaption>
</figure>

<p>它大约半秒就能就绪，空闲时占大约 175 MiB 内存。所有数据都在内存里，所以容器一停就全没了。</p>

<h2 class="relative group">需要注册账号吗？（LocalStack 与 AWS Emulator 对比）
    <div id="需要注册账号吗localstack-与-aws-emulator-对比" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e9%9c%80%e8%a6%81%e6%b3%a8%e5%86%8c%e8%b4%a6%e5%8f%b7%e5%90%97localstack-%e4%b8%8e-aws-emulator-%e5%af%b9%e6%af%94" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>不需要。AWS Emulator、Moto 和 Floci 都不用注册任何东西就能跑。AWS Emulator 甚至不要你的 AWS 凭证：它每次启动都会自己生成一把 access key，并且只接受用这把 key 签名的请求。</p>
<p>变了的是 LocalStack。从 2026 年 3 月起，它的当前镜像需要一个 <code>LOCALSTACK_AUTH_TOKEN</code>，来自免费账号（免费套餐面向个人爱好和非商业用途）或付费账号。没有 token，容器在启动时就会停掉。两种我都试了：标签为 <code>latest</code> 的镜像（2026.10.7，11 月 11 日构建）以退出码 55 退出；而 4.12，也就是最后一个社区版、LocalStack 自己也建议锁定的版本，不带 token 仍然能启动。</p>
<figure><video
    class="my-0 rounded-md"
    autoplay loop muted playsinline preload="metadata"
    poster="https://jared.lynskey.co.nz/en/posts/2026/aws-emulator-setup/localstack-needs-account.jpg" style="background:url('data:image/webp;base64,UklGRlYAAABXRUJQVlA4IEoAAABQBACdASogABIAPw10pUymIyGhKA1QxiGJ6QDJTPgAtfKr5WoM7/AUVx4AAP7ozNYDExNyKOZLAQjfzKlQvzdAcPxaBgAGLrjAAA==') center/cover no-repeat"
    width="1280"
    height="720"
    aria-label="终端：localstack/localstack:latest 报出 &#39;License activation failed&#39;，以退出码 55 退出；接着 localstack/localstack:4.12 启动，健康检查显示版本 4.12.0，S3 可用">
    <source src="https://jared.lynskey.co.nz/en/posts/2026/aws-emulator-setup/localstack-needs-account.mp4" type="video/mp4">
  </video><figcaption>不带 token 的 LocalStack 当前镜像，然后是锁定的 4.12。锁定版本现在还能用，但 4.12 不会再有修复了。</figcaption>
</figure>

<p>AWS Emulator 确实想要的一样东西是遥测，而且默认开启。AWS 说遥测里绝不包含你的请求、代码或凭证，但如果你希望它什么都不发，有三个开关：命令行加 <code>--telemetry off</code>，环境变量设 <code>EMULATOR_TELEMETRY=off</code>，或者在 <code>~/.aws/emulator/config.toml</code> 里写上：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-toml" data-lang="toml"><span class="line"><span class="cl"><span class="p">[</span><span class="nx">telemetry</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="nx">enabled</span> <span class="p">=</span> <span class="kc">false</span></span></span></code></pre></div></div>

<h2 class="relative group">在 macOS 或 Windows 上用 Docker 搭建 AWS Emulator
    <div id="在-macos-或-windows-上用-docker-搭建-aws-emulator" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e5%9c%a8-macos-%e6%88%96-windows-%e4%b8%8a%e7%94%a8-docker-%e6%90%ad%e5%bb%ba-aws-emulator" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>在 Mac 或 Windows 上，走 Docker 这条路。在一个空文件夹里把下面的内容存成 <code>Dockerfile</code>。这是 AWS 自己的版本，出自<a href="https://docs.aws.amazon.com/emulator/latest/userguide/getting-started-docker.html"  target="_blank" rel="noreferrer">他们的 Docker 指南</a>：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-dockerfile" data-lang="dockerfile"><span class="line"><span class="cl"><span class="k">FROM</span><span class="w"> </span><span class="s">public.ecr.aws/amazonlinux/amazonlinux:2023-minimal</span><span class="err">
</span></span></span><span class="line"><span class="cl"><span class="k">ADD</span> https://downloads.aws.com/aws-emulator/packages/aws-emulator/LTS/latest/aws-emulator-x86_64.rpm /tmp/aws-emulator.rpm<span class="err">
</span></span></span><span class="line"><span class="cl"><span class="k">RUN</span> rpm -i /tmp/aws-emulator.rpm <span class="o">&amp;&amp;</span> rm /tmp/aws-emulator.rpm <span class="se">\
</span></span></span><span class="line"><span class="cl">    <span class="o">&amp;&amp;</span> mkdir -p /home/emulator <span class="o">&amp;&amp;</span> chown <span class="m">1000</span> /home/emulator<span class="err">
</span></span></span><span class="line"><span class="cl"><span class="k">USER</span><span class="w"> </span><span class="s">1000</span><span class="err">
</span></span></span><span class="line"><span class="cl"><span class="k">ENV</span> <span class="nv">HOME</span><span class="o">=</span>/home/emulator<span class="err">
</span></span></span><span class="line"><span class="cl"><span class="k">EXPOSE</span><span class="w"> </span><span class="s">9297</span><span class="err">
</span></span></span><span class="line"><span class="cl"><span class="k">ENTRYPOINT</span> <span class="p">[</span><span class="s2">&#34;aws-emulator&#34;</span><span class="p">,</span> <span class="s2">&#34;-b&#34;</span><span class="p">,</span> <span class="s2">&#34;0.0.0.0&#34;</span><span class="p">]</span></span></span></code></pre></div></div>
<p>在 Apple 芯片或其他 ARM 机器上，把 <code>x86_64.rpm</code> 改成 <code>aarch64.rpm</code>。然后构建并启动：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">docker build -t aws-emulator .
</span></span><span class="line"><span class="cl">mkdir -p ~/.aws/emulator
</span></span><span class="line"><span class="cl">docker run --rm -d --name aws-emulator --memory<span class="o">=</span>2g --memory-swap<span class="o">=</span>2g <span class="se">\
</span></span></span><span class="line"><span class="cl">  -p 127.0.0.1:9297:9297 <span class="se">\
</span></span></span><span class="line"><span class="cl">  -e <span class="nv">HOME</span><span class="o">=</span><span class="s2">&#34;</span><span class="nv">$HOME</span><span class="s2">&#34;</span> -v <span class="s2">&#34;</span><span class="nv">$HOME</span><span class="s2">/.aws/emulator:</span><span class="nv">$HOME</span><span class="s2">/.aws/emulator&#34;</span> <span class="se">\
</span></span></span><span class="line"><span class="cl">  aws-emulator -p <span class="m">9297</span> --telemetry off</span></span></code></pre></div></div>
<p>在我的 Mac 上构建花了 88 秒，镜像 1.67 GB。这行 <code>docker run</code> 里有三处比看上去重要：</p>
<ul>
<li><strong>用 <code>-p 127.0.0.1:9297:9297</code>，不要用 <code>-p 9297:9297</code>。</strong> 短的写法会把模拟器暴露给整个网络。</li>
<li><strong>卷挂载和 <code>-e HOME</code>。</strong> 启动时，模拟器会往 <code>~/.aws/emulator/9297/</code> 写四个文件：它的凭证、一个 AWS <code>config</code> 文件、它的证书（<code>ca.pem</code>），以及一个 Java 信任库。把这个文件夹以相同的路径挂进容器，宿主机上的 SDK 才读得到它们。</li>
<li><strong>端口号出现了三次。</strong> 要改就全部一起改。模拟器会把端口写进这些文件，两个共用同一个文件夹的容器会互相覆盖对方的文件。</li>
</ul>

<h2 class="relative group">在 Linux 上安装 AWS Emulator
    <div id="在-linux-上安装-aws-emulator" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e5%9c%a8-linux-%e4%b8%8a%e5%ae%89%e8%a3%85-aws-emulator" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>在 Linux 上不需要 Docker，有现成的 Debian 包和 RPM 包：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="c1"># Debian / Ubuntu (use aws-emulator_arm64.deb on ARM)</span>
</span></span><span class="line"><span class="cl">curl -fsSLo aws-emulator.deb https://downloads.aws.com/aws-emulator/packages/aws-emulator/LTS/latest/aws-emulator_amd64.deb
</span></span><span class="line"><span class="cl">sudo apt-get install -y ./aws-emulator.deb
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Amazon Linux, Fedora, RHEL (aarch64.rpm on ARM)</span>
</span></span><span class="line"><span class="cl">sudo dnf install -y https://downloads.aws.com/aws-emulator/packages/aws-emulator/LTS/latest/aws-emulator-x86_64.rpm
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">aws-emulator --telemetry off</span></span></code></pre></div></div>
<p>安装包 150 MB。它在前台运行，监听 <code>127.0.0.1:9297</code>，同样把那四个文件写到 <code>~/.aws/emulator/9297/</code>。按 Ctrl+C 就能干净地停掉它，文件也会一并删除。跑测试时加上 <code>--idle-timeout 900</code>，15 分钟没有请求它就会自己关掉，免得一个被忘掉的模拟器在那儿挂上一整周。</p>

<h2 class="relative group">让 AWS CLI 和 boto3 指向模拟器
    <div id="让-aws-cli-和-boto3-指向模拟器" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e8%ae%a9-aws-cli-%e5%92%8c-boto3-%e6%8c%87%e5%90%91%e6%a8%a1%e6%8b%9f%e5%99%a8" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>需要的环境变量就是这些。任何读取标准 AWS 环境变量的 SDK 都会用上它们，所以被测代码里既不用写 endpoint，也不用写 key：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="nv">E</span><span class="o">=</span>~/.aws/emulator/9297
</span></span><span class="line"><span class="cl"><span class="nb">export</span> <span class="nv">AWS_ENDPOINT_URL</span><span class="o">=</span>https://127.0.0.1:9297
</span></span><span class="line"><span class="cl"><span class="nb">export</span> <span class="nv">AWS_SHARED_CREDENTIALS_FILE</span><span class="o">=</span><span class="nv">$E</span>/emulator_credentials
</span></span><span class="line"><span class="cl"><span class="nb">export</span> <span class="nv">AWS_CONFIG_FILE</span><span class="o">=</span><span class="nv">$E</span>/config
</span></span><span class="line"><span class="cl"><span class="nb">export</span> <span class="nv">AWS_CA_BUNDLE</span><span class="o">=</span><span class="nv">$E</span>/ca.pem
</span></span><span class="line"><span class="cl"><span class="nb">export</span> <span class="nv">AWS_REGION</span><span class="o">=</span>us-east-1
</span></span><span class="line"><span class="cl"><span class="nb">export</span> <span class="nv">AWS_EC2_METADATA_DISABLED</span><span class="o">=</span><span class="nb">true</span>
</span></span><span class="line"><span class="cl"><span class="nb">export</span> <span class="nv">AWS_DISABLE_HOST_PREFIX_INJECTION</span><span class="o">=</span><span class="nb">true</span>
</span></span><span class="line"><span class="cl"><span class="nb">unset</span> AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN AWS_PROFILE
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">curl -s --cacert <span class="nv">$AWS_CA_BUNDLE</span> <span class="nv">$AWS_ENDPOINT_URL</span>/__admin/health   <span class="c1"># ok</span>
</span></span><span class="line"><span class="cl">aws sts get-caller-identity                                          <span class="c1"># 000000000000</span></span></span></code></pre></div></div>
<p>之后，最普通的 boto3 代码直接就能用：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">boto3</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="n">s3</span> <span class="o">=</span> <span class="n">boto3</span><span class="o">.</span><span class="n">client</span><span class="p">(</span><span class="s2">&#34;s3&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="n">s3</span><span class="o">.</span><span class="n">create_bucket</span><span class="p">(</span><span class="n">Bucket</span><span class="o">=</span><span class="s2">&#34;uploads&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="nb">print</span><span class="p">(</span><span class="n">s3</span><span class="o">.</span><span class="n">list_buckets</span><span class="p">()[</span><span class="s2">&#34;Buckets&#34;</span><span class="p">])</span></span></span></code></pre></div></div>
<p><code>unset</code> 那一行一定要留着。环境变量的优先级高于凭证文件，所以如果你真实的 AWS key 还 export 着，SDK 就会用它们签名，模拟器拒绝请求，然后你白白花十分钟琢磨为什么。<code>aws sts get-caller-identity</code> 是个快速的自检：模拟器的账号永远是 <code>000000000000</code>。如果看到的是你自己的账号，说明你连的是 AWS。</p>
<p>其他几个我踩过、或者差点踩到的坑：</p>
<ul>
<li><strong>endpoint 里写 <code>127.0.0.1</code>，别写 <code>localhost</code>。</strong> 用 IP 地址时，SDK 会把 bucket 名放在路径里。用 <code>localhost</code> 时，JavaScript、Java 和 Go 的 SDK 会把它挪到主机名里（<code>uploads.localhost</code>），而模拟器不接受这种写法。</li>
<li><strong>只支持 HTTPS，而且每次启动都换一张新证书。</strong> boto3、AWS CLI 和 Go 会读 <code>AWS_CA_BUNDLE</code>。Node 需要把 <code>NODE_EXTRA_CA_CERTS</code> 指向同一个 <code>ca.pem</code>，Java 则通过 <code>JAVA_TOOL_OPTIONS</code> 使用 <code>truststore.p12</code>。<code>AWS_CA_BUNDLE</code> 会替换掉这个进程信任的证书，所以如果测试里还要调用真实的 HTTPS API，就需要一个合并后的证书包。</li>
<li><strong>没有重置功能。</strong> 没有哪个管理路由能把它清空，所以要给每个测试用一个独立的 bucket 名，或者在两组测试之间重启模拟器。</li>
<li><strong>key 每次启动都会变。</strong> 别把它们抄进 <code>.env</code> 文件，直接指向凭证文件。</li>
<li><strong>如果你的应用自己用显式的 endpoint 创建 client</strong>（比如 django-storages 的 <code>AWS_S3_ENDPOINT_URL</code>），也要用 <code>AWS_ENDPOINT_URL</code> 来设置它，否则这一个 client 还是会连到真实的服务。</li>
</ul>

<h2 class="relative group">用 Docker Compose 运行 AWS Emulator
    <div id="用-docker-compose-运行-aws-emulator" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e7%94%a8-docker-compose-%e8%bf%90%e8%a1%8c-aws-emulator" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>如果项目里已经有 <code>compose.yaml</code>，可以加上下面这个服务。它用上面的 Dockerfile 构建镜像，把模拟器的文件放在项目文件夹里而不是你的主目录，还带一个健康检查，所以 <code>docker compose up --wait</code> 会等它就绪后才返回：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">services</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">aws</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">build</span><span class="p">:</span><span class="w"> </span><span class="l">./aws-emulator           </span><span class="w"> </span><span class="c"># the folder with the Dockerfile</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">command</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">&#34;-p&#34;</span><span class="p">,</span><span class="w"> </span><span class="s2">&#34;9297&#34;</span><span class="p">,</span><span class="w"> </span><span class="s2">&#34;--telemetry&#34;</span><span class="p">,</span><span class="w"> </span><span class="s2">&#34;off&#34;</span><span class="p">]</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">ports</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">&#34;127.0.0.1:9297:9297&#34;</span><span class="p">]</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">environment</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">HOME</span><span class="p">:</span><span class="w"> </span><span class="l">${PWD}/.aws-emulator-home</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">volumes</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="l">./.aws-emulator-home/.aws/emulator:${PWD}/.aws-emulator-home/.aws/emulator</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">mem_limit</span><span class="p">:</span><span class="w"> </span><span class="l">2g</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">healthcheck</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">test</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">&#34;CMD&#34;</span><span class="p">,</span><span class="w"> </span><span class="s2">&#34;curl&#34;</span><span class="p">,</span><span class="w"> </span><span class="s2">&#34;-sf&#34;</span><span class="p">,</span><span class="w"> </span><span class="s2">&#34;--cacert&#34;</span><span class="p">,</span><span class="w"> </span><span class="s2">&#34;${PWD}/.aws-emulator-home/.aws/emulator/9297/ca.pem&#34;</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">             </span><span class="s2">&#34;https://127.0.0.1:9297/__admin/health&#34;</span><span class="p">]</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">interval</span><span class="p">:</span><span class="w"> </span><span class="l">1s</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">retries</span><span class="p">:</span><span class="w"> </span><span class="m">30</span></span></span></code></pre></div></div>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">mkdir -p .aws-emulator-home/.aws/emulator
</span></span><span class="line"><span class="cl">docker compose up -d --wait aws       <span class="c1"># healthy in about 5 s with the image already built</span>
</span></span><span class="line"><span class="cl"><span class="nb">export</span> <span class="nv">E</span><span class="o">=</span><span class="nv">$PWD</span>/.aws-emulator-home/.aws/emulator/9297   <span class="c1"># then the variables from above</span></span></span></code></pre></div></div>
<p>记得把 <code>.aws-emulator-home/</code> 加进 <code>.gitignore</code>，里面放着这次运行的 key。</p>

<h2 class="relative group">在 pytest 里使用 AWS Emulator
    <div id="在-pytest-里使用-aws-emulator" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e5%9c%a8-pytest-%e9%87%8c%e4%bd%bf%e7%94%a8-aws-emulator" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>测试本身没有任何专门针对模拟器的东西。我额外加的是一道保护，因为我担心的失败不是测试变红，而是测试变绿了，却是对着一个真实的 bucket 跑的：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="c1"># conftest.py</span>
</span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">os</span>
</span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">uuid</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">boto3</span>
</span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">pytest</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="nd"><span>@</span>pytest.fixture</span><span class="p">(</span><span class="n">scope</span><span class="o">=</span><span class="s2">&#34;session&#34;</span><span class="p">,</span> <span class="n">autouse</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">only_against_the_emulator</span><span class="p">():</span>
</span></span><span class="line"><span class="cl">    <span class="n">url</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">environ</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&#34;AWS_ENDPOINT_URL&#34;</span><span class="p">,</span> <span class="s2">&#34;&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">if</span> <span class="ow">not</span> <span class="n">url</span><span class="o">.</span><span class="n">startswith</span><span class="p">((</span><span class="s2">&#34;https://127.0.0.1&#34;</span><span class="p">,</span> <span class="s2">&#34;http://127.0.0.1&#34;</span><span class="p">)):</span>
</span></span><span class="line"><span class="cl">        <span class="n">pytest</span><span class="o">.</span><span class="n">exit</span><span class="p">(</span><span class="sa">f</span><span class="s2">&#34;AWS_ENDPOINT_URL is </span><span class="si">{</span><span class="n">url</span><span class="si">!r}</span><span class="s2">; refusing to run against real AWS&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="nd"><span>@</span>pytest.fixture</span>
</span></span><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">s3</span><span class="p">():</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="n">boto3</span><span class="o">.</span><span class="n">client</span><span class="p">(</span><span class="s2">&#34;s3&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="nd"><span>@</span>pytest.fixture</span>
</span></span><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">bucket</span><span class="p">(</span><span class="n">s3</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">    <span class="n">name</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">&#34;test-</span><span class="si">{</span><span class="n">uuid</span><span class="o">.</span><span class="n">uuid4</span><span class="p">()</span><span class="o">.</span><span class="n">hex</span><span class="p">[:</span><span class="mi">12</span><span class="p">]</span><span class="si">}</span><span class="s2">&#34;</span>   <span class="c1"># no reset route, so never reuse a name</span>
</span></span><span class="line"><span class="cl">    <span class="n">s3</span><span class="o">.</span><span class="n">create_bucket</span><span class="p">(</span><span class="n">Bucket</span><span class="o">=</span><span class="n">name</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="n">name</span></span></span></code></pre></div></div>
<p>之后测试写起来就和平常一样：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">os</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">requests</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">test_presigned_upload_rejects_the_wrong_content_type</span><span class="p">(</span><span class="n">s3</span><span class="p">,</span> <span class="n">bucket</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">    <span class="n">url</span> <span class="o">=</span> <span class="n">s3</span><span class="o">.</span><span class="n">generate_presigned_url</span><span class="p">(</span>
</span></span><span class="line"><span class="cl">        <span class="s2">&#34;put_object&#34;</span><span class="p">,</span> <span class="n">ExpiresIn</span><span class="o">=</span><span class="mi">300</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">        <span class="n">Params</span><span class="o">=</span><span class="p">{</span><span class="s2">&#34;Bucket&#34;</span><span class="p">:</span> <span class="n">bucket</span><span class="p">,</span> <span class="s2">&#34;Key&#34;</span><span class="p">:</span> <span class="s2">&#34;media/a.jpg&#34;</span><span class="p">,</span> <span class="s2">&#34;ContentType&#34;</span><span class="p">:</span> <span class="s2">&#34;image/jpeg&#34;</span><span class="p">},</span>
</span></span><span class="line"><span class="cl">    <span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="n">r</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="sa">b</span><span class="s2">&#34;&lt;html&gt;&#34;</span><span class="p">,</span> <span class="n">headers</span><span class="o">=</span><span class="p">{</span><span class="s2">&#34;Content-Type&#34;</span><span class="p">:</span> <span class="s2">&#34;text/html&#34;</span><span class="p">},</span>
</span></span><span class="line"><span class="cl">                     <span class="n">verify</span><span class="o">=</span><span class="n">os</span><span class="o">.</span><span class="n">environ</span><span class="p">[</span><span class="s2">&#34;AWS_CA_BUNDLE&#34;</span><span class="p">])</span>
</span></span><span class="line"><span class="cl">    <span class="k">assert</span> <span class="n">r</span><span class="o">.</span><span class="n">status_code</span> <span class="o">==</span> <span class="mi">403</span></span></span></code></pre></div></div>
<p>这个测试来自我那个交易平台应用的上传流程，它也很好地说明了为什么选哪个模拟器很重要：我试过的五个里，有三个对它返回 200。下文会细说。如果你每次改动只跑受影响的测试，<a href="https://jared.lynskey.co.nz/zh-cn/posts/2026/2026-10-26-pytest-testmon/" >pytest-testmon</a> 不用任何修改就能配合这套用法。</p>

<h2 class="relative group">在 GitHub Actions 里运行 AWS Emulator
    <div id="在-github-actions-里运行-aws-emulator" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e5%9c%a8-github-actions-%e9%87%8c%e8%bf%90%e8%a1%8c-aws-emulator" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>在 GitHub 的 runner 上，跳过 Docker，直接装 Debian 包。某一步里启动的后台进程，会在这个 job 余下的步骤里一直运行：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">jobs</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">test</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">runs-on</span><span class="p">:</span><span class="w"> </span><span class="l">ubuntu-latest</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">steps</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="nt">uses</span><span class="p">:</span><span class="w"> </span><span class="l">actions/checkout@v7</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="nt">uses</span><span class="p">:</span><span class="w"> </span><span class="l">astral-sh/setup-uv@v10</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Start AWS Emulator</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">run</span><span class="p">:</span><span class="w"> </span><span class="p">|</span><span class="sd">
</span></span></span><span class="line"><span class="cl"><span class="sd">          curl -fsSLo aws-emulator.deb https://downloads.aws.com/aws-emulator/packages/aws-emulator/LTS/latest/aws-emulator_amd64.deb
</span></span></span><span class="line"><span class="cl"><span class="sd">          sudo apt-get install -y ./aws-emulator.deb
</span></span></span><span class="line"><span class="cl"><span class="sd">          aws-emulator --telemetry off --idle-timeout 900 &gt; emulator.log 2&gt;&amp;1 &amp;
</span></span></span><span class="line"><span class="cl"><span class="sd">          E=$HOME/.aws/emulator/9297
</span></span></span><span class="line"><span class="cl"><span class="sd">          for i in $(seq 30); do
</span></span></span><span class="line"><span class="cl"><span class="sd">            curl -sf --cacert $E/ca.pem https://127.0.0.1:9297/__admin/health &amp;&amp; break
</span></span></span><span class="line"><span class="cl"><span class="sd">            sleep 1
</span></span></span><span class="line"><span class="cl"><span class="sd">          done
</span></span></span><span class="line"><span class="cl"><span class="sd">          {
</span></span></span><span class="line"><span class="cl"><span class="sd">            echo &#34;AWS_ENDPOINT_URL=https://127.0.0.1:9297&#34;
</span></span></span><span class="line"><span class="cl"><span class="sd">            echo &#34;AWS_SHARED_CREDENTIALS_FILE=$E/emulator_credentials&#34;
</span></span></span><span class="line"><span class="cl"><span class="sd">            echo &#34;AWS_CONFIG_FILE=$E/config&#34;
</span></span></span><span class="line"><span class="cl"><span class="sd">            echo &#34;AWS_CA_BUNDLE=$E/ca.pem&#34;
</span></span></span><span class="line"><span class="cl"><span class="sd">            echo &#34;AWS_REGION=us-east-1&#34;
</span></span></span><span class="line"><span class="cl"><span class="sd">            echo &#34;AWS_EC2_METADATA_DISABLED=true&#34;
</span></span></span><span class="line"><span class="cl"><span class="sd">            echo &#34;AWS_DISABLE_HOST_PREFIX_INJECTION=true&#34;
</span></span></span><span class="line"><span class="cl"><span class="sd">          } &gt;&gt; &#34;$GITHUB_ENV&#34;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="nt">run</span><span class="p">:</span><span class="w"> </span><span class="l">uv run pytest</span></span></span></code></pre></div></div>
<p>我在一个全新的 <code>ubuntu:24.04</code> 容器里跑了同样的步骤（用的是 arm64 包，因为我的 Mac 是 ARM）。安装花了 29 秒，几乎全花在下载那 150 MB 上，健康检查在 4 秒内就有了响应。这个 job 里干脆别放真实的 AWS 凭证。要确保里面没有任何东西连到 AWS，这是最省事的办法。</p>
<p><code>LTS/latest</code> 指向的是最新发布的版本，下文会看到，它一天能变好几次。如果你希望 CI 可复现，AWS 有一页讲<a href="https://docs.aws.amazon.com/emulator/latest/userguide/versioning.html"  target="_blank" rel="noreferrer">如何安装指定版本</a>。至于流水线的其余部分我是怎么安排的，可以看<a href="https://jared.lynskey.co.nz/zh-cn/posts/2026/2026-10-22-docker-compose-ci-github-actions/" >我现在在用的 GitHub Actions 流水线</a>。</p>

<h2 class="relative group">从 LocalStack 迁移过来
    <div id="从-localstack-迁移过来" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e4%bb%8e-localstack-%e8%bf%81%e7%a7%bb%e8%bf%87%e6%9d%a5" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>如果你的测试已经在用 LocalStack，迁移的大部分工作都是改配置：</p>
<table>
	<thead>
			<tr>
					<th></th>
					<th>LocalStack</th>
					<th>AWS Emulator</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td>Endpoint</td>
					<td><code>http://localhost:4566</code></td>
					<td><code>https://127.0.0.1:9297</code> 加上 <code>AWS_CA_BUNDLE</code></td>
			</tr>
			<tr>
					<td>凭证</td>
					<td>随便填（<code>test</code> / <code>test</code>）</td>
					<td>每次运行时它写出的文件，或者 <code>--auth off</code></td>
			</tr>
			<tr>
					<td>账号 ID</td>
					<td><code>000000000000</code></td>
					<td><code>000000000000</code></td>
			</tr>
			<tr>
					<td>初始化脚本</td>
					<td><code>/etc/localstack/init/ready.d</code> 里的 init hook</td>
					<td>没有：在健康检查之后，在 fixture 里创建资源</td>
			</tr>
			<tr>
					<td>镜像</td>
					<td><code>localstack/localstack</code>（4.12 之后需要 token）</td>
					<td>用 AWS 的安装包自己构建</td>
			</tr>
	</tbody>
</table>
<p>如果你大量使用 init hook，这部分会是真正要花工夫的地方。我没怎么用，因为每个测试都会自己建 bucket。另外，重启之后什么都不会留下：模拟器把所有东西都放在内存里。</p>

<h2 class="relative group">AWS Emulator 在我的 S3 本地测试里抓到了什么、漏掉了什么
    <div id="aws-emulator-在我的-s3-本地测试里抓到了什么漏掉了什么" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#aws-emulator-%e5%9c%a8%e6%88%91%e7%9a%84-s3-%e6%9c%ac%e5%9c%b0%e6%b5%8b%e8%af%95%e9%87%8c%e6%8a%93%e5%88%b0%e4%ba%86%e4%bb%80%e4%b9%88%e6%bc%8f%e6%8e%89%e4%ba%86%e4%bb%80%e4%b9%88" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>搭建是容易的部分。我真正想知道的是：在我的代码依赖 S3 行为的那些地方，它表现得像不像 S3。我写了 12 项检查，每一项都按 AWS 文档断言真实 S3 的行为：</p>
<ul>
<li><strong>六项来自交易平台应用的上传流程。</strong> 应用交给浏览器一个预签名 URL，浏览器把照片直接 PUT 进 bucket（所以 bucket 需要配 CORS），再由一个 <code>ObjectCreated</code> 事件告诉后端文件到了。这就是<a href="https://jared.lynskey.co.nz/zh-cn/posts/2026/tigris-object-notifications/" >我那篇讲对象通知的文章</a>里的流程，只是用 SQS 代替了 webhook。</li>
<li><strong>六项来自一个 Codemagic Patch 的 bucket。</strong> Patch 是给 React Native 应用用的 OTA 更新服务器（<a href="https://jared.lynskey.co.nz/zh-cn/posts/2026/2026-09-13-moving-to-codemagic-patch/" >我是怎么迁过去的</a>）。它的 bucket 是公开的，好让手机下载 bundle；只有 <code>_internal/</code> 前缀例外，上传的东西在那里等待发布，由一条 bucket 策略让这个前缀保持私有。</li>
</ul>
<p>我把这些检查分别跑在了 AWS Emulator 1.0.342（默认配置一次，<code>--auth off</code> 一次）、LocalStack 4.12（默认配置，以及设置 <code>S3_SKIP_SIGNATURE_VALIDATION=0</code> 之后）、server 模式下的 Moto 5.2.6、Floci 1.5.5，以及 Silo 上。Silo 是 Patch 自托管套件所用的那个 MinIO 分支。</p>
<figure><video
    class="my-0 rounded-md"
    autoplay loop muted playsinline preload="metadata"
    poster="https://jared.lynskey.co.nz/en/posts/2026/aws-emulator-setup/s3-emulator-test-run.jpg" style="background:url('data:image/webp;base64,UklGRk4AAABXRUJQVlA4IEIAAAAQBACdASogABIAPw10rkymIyIhKAqoxiGJ6QDM0PgAtf/5Tl8J29EIQAD+6MzcUMCFo8DULSYQXf8FTN9rmZMAAAA=') center/cover no-repeat"
    width="1280"
    height="720"
    aria-label="终端：一个脚本对七种模拟器配置跑同样的 12 项 S3 检查，并打印出通过/失败矩阵；每种配置最终都是 12 项中通过 6 到 8 项">
    <source src="https://jared.lynskey.co.nz/en/posts/2026/aws-emulator-setup/s3-emulator-test-run.mp4" type="video/mp4">
  </video><figcaption>真实的运行结果。没有一个 12 项全过。</figcaption>
</figure>

<p>没有一种配置 12 项全过，成绩都在 6 到 8 项之间。总分不如各自漏掉了哪几项重要，所以下面按对我伤害的大小，从最严重的说起。</p>
<p><strong>AWS Emulator 在权限上最严格。</strong> 拿我的 app 为一张 JPEG 签出的预签名 URL 来说：在默认配置的 LocalStack 4.12、Moto 和 Floci 上，我能用同一个 URL 上传一个 HTML 文件，还拿到 200。放在一个对外提供公开文件的 bucket 上，这就是一个随时会炸的存储型 XSS。AWS Emulator 和 S3 一样回了 403，URL 过期后也照样拒绝。（LocalStack 4.12 设置了 <code>S3_SKIP_SIGNATURE_VALIDATION=0</code> 之后两样都能拦住；Floci 能拦住过期，但拦不住被调包的文件类型。）也只有它知道新建的 S3 bucket 默认开着 Block Public Access，在你关掉之前公开的 bucket 策略会被拒绝；也只有它把显式的 <code>Deny</code> 用在了服务器自己签名的请求上，而不只是针对陌生人。恰恰是这类错误，会通过所有测试，然后在第一次真正部署时翻车。</p>
<p><strong>浏览器那一侧的事情，它暂时还测不了。</strong> CORS 预检（浏览器在跨源 PUT 之前发出的 <code>OPTIONS</code> 请求）会收到 <code>501 Not Implemented</code>。在默认开启认证的情况下，它还会拒绝所有未签名的请求，所以即使策略允许，从公开 bucket 匿名下载也会得到 403。这两个缺口都已经在仓库里登记了。这个项目把已知缺口以文件的形式放在 <code>issues/things-that-do-not-work/</code> 下，11 月 13 日那天有 6,840 个，另有 13,409 个已经挪到了 <code>things-that-now-work/</code>。</p>
<p><figure><span class="mc-img"><img
    class="my-0 rounded-md"
    loading="lazy"
    decoding="async"
    fetchpriority="auto"
    alt="GitHub 上 things-that-do-not-work 文件夹里 AWS Emulator 针对 S3 的 issue 文件，标题为 &ldquo;7 net-new causes — CORS preflight answers 501…&rdquo;"
    width="1400"
    height="804"
    src="https://jared.lynskey.co.nz/en/posts/2026/aws-emulator-setup/aws-emulator-cors-preflight-issue_hu_33033b091b284c0d.webp"
    srcset="https://jared.lynskey.co.nz/en/posts/2026/aws-emulator-setup/aws-emulator-cors-preflight-issue_hu_33033b091b284c0d.webp 800w, https://jared.lynskey.co.nz/en/posts/2026/aws-emulator-setup/aws-emulator-cors-preflight-issue_hu_f0294a8a0a022038.webp 1280w"
    sizes="(min-width: 768px) 50vw, 65vw"
    data-zoom-src="https://jared.lynskey.co.nz/en/posts/2026/aws-emulator-setup/aws-emulator-cors-preflight-issue_hu_925176ba1af6cb05.webp"></span><figcaption>CORS 这个缺口，是项目自己负责找 issue 的 agent 在 11 月 9 日登记的。</figcaption></figure>
</p>
<p><strong><code>--auth off</code> 是拿这一半换那一半。</strong> 关掉认证后，匿名读取能用了，bucket 策略也能正确评估，所以 Patch 的六项检查全部通过，只有这一种配置做到了。但预签名 URL 也不再校验了，于是上传那一组只剩两项通过。在不同端口上跑两个模拟器，一个开认证、一个不开，就能把两半都覆盖到，只是下一个看 CI 文件的人，又多了一件得跟他解释的事。</p>
<p><strong>它变得很快。</strong> v1.0.290 到 v1.0.342 是三天之内发出来的，每个版本都带着一长串修复（AWS 说它是 coding agent 构建的，从这个节奏就看得出来），其中 11 月 11 日的一个修复就是 S3 的 &ldquo;presigned GET refused with 403&rdquo;。我表里的任何一项，到下周都可能变样，变好变坏都有可能。在 CI 里锁定一个版本，升级时有意识地去升。</p>
<p><figure><span class="mc-img"><img
    class="my-0 rounded-md"
    loading="lazy"
    decoding="async"
    fetchpriority="auto"
    alt="GitHub 上 aws/aws-emulator 的 releases 页面：v1.0.342 标着 Latest，3 小时前发布，侧边栏里还列着另外九个版本"
    width="1600"
    height="1125"
    src="https://jared.lynskey.co.nz/en/posts/2026/aws-emulator-setup/aws-emulator-releases_hu_e39a0eec6b783a7a.webp"
    srcset="https://jared.lynskey.co.nz/en/posts/2026/aws-emulator-setup/aws-emulator-releases_hu_e39a0eec6b783a7a.webp 800w, https://jared.lynskey.co.nz/en/posts/2026/aws-emulator-setup/aws-emulator-releases_hu_df197ee02b7c435c.webp 1280w"
    sizes="(min-width: 768px) 50vw, 65vw"
    data-zoom-src="https://jared.lynskey.co.nz/en/posts/2026/aws-emulator-setup/aws-emulator-releases_hu_59bd3abd9b9aaf92.webp"></span><figcaption>三天十个版本。锁定一个吧。</figcaption></figure>
</p>

<h2 class="relative group">Patch 的 bucket：一个只在真实 S3 上才起作用的策略条件
    <div id="patch-的-bucket一个只在真实-s3-上才起作用的策略条件" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#patch-%e7%9a%84-bucket%e4%b8%80%e4%b8%aa%e5%8f%aa%e5%9c%a8%e7%9c%9f%e5%ae%9e-s3-%e4%b8%8a%e6%89%8d%e8%b5%b7%e4%bd%9c%e7%94%a8%e7%9a%84%e7%ad%96%e7%95%a5%e6%9d%a1%e4%bb%b6" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>这部分是我没想到的。Patch 的自托管套件把所有东西都放在一个 bucket 里，它给自带存储设置的策略也是最直观的那种：谁都能读，<code>_internal/</code> 下面除外：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-json" data-lang="json"><span class="line"><span class="cl"><span class="p">{</span>
</span></span><span class="line"><span class="cl">  <span class="nt">&#34;Version&#34;</span><span class="p">:</span> <span class="s2">&#34;2012-10-17&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">  <span class="nt">&#34;Statement&#34;</span><span class="p">:</span> <span class="p">[</span>
</span></span><span class="line"><span class="cl">    <span class="p">{</span> <span class="nt">&#34;Effect&#34;</span><span class="p">:</span> <span class="s2">&#34;Allow&#34;</span><span class="p">,</span> <span class="nt">&#34;Principal&#34;</span><span class="p">:</span> <span class="s2">&#34;*&#34;</span><span class="p">,</span> <span class="nt">&#34;Action&#34;</span><span class="p">:</span> <span class="s2">&#34;s3:GetObject&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">      <span class="nt">&#34;Resource&#34;</span><span class="p">:</span> <span class="s2">&#34;arn:aws:s3:::example-ota/*&#34;</span> <span class="p">},</span>
</span></span><span class="line"><span class="cl">    <span class="p">{</span> <span class="nt">&#34;Effect&#34;</span><span class="p">:</span> <span class="s2">&#34;Deny&#34;</span><span class="p">,</span> <span class="nt">&#34;Principal&#34;</span><span class="p">:</span> <span class="s2">&#34;*&#34;</span><span class="p">,</span> <span class="nt">&#34;Action&#34;</span><span class="p">:</span> <span class="s2">&#34;s3:GetObject&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">      <span class="nt">&#34;Resource&#34;</span><span class="p">:</span> <span class="s2">&#34;arn:aws:s3:::example-ota/_internal/*&#34;</span> <span class="p">}</span>
</span></span><span class="line"><span class="cl">  <span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span></span></span></code></pre></div></div>
<p>在 Silo 上用管理员凭证签名时，它的行为和字面意思一致：陌生人读不了 <code>_internal/</code>，服务器自己把文件从 <code>_internal/</code> 复制到公开前缀（也就是&quot;发布&quot;这一步）也能顺利完成。但在真实的 S3 上，带 <code>&quot;Principal&quot;: &quot;*&quot;</code> 的显式 <code>Deny</code> 对所有人生效，包括你自己的凭证，所以同一条策略会挡住服务器自己的发布步骤。AWS Emulator 用 403 拒绝了这次复制，按 AWS 的策略评估规则，S3 正是这么做的。</p>
<p>Patch 的维护者显然知道这一点，因为他们<a href="https://github.com/codemagic-ci-cd/codemagic-patch/blob/main/deploy/selfhost/aws-s3-bucket-policy.example.json"  target="_blank" rel="noreferrer">给 AWS S3 的示例策略</a>在 <code>Deny</code> 上多加了一个条件：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-json" data-lang="json"><span class="line"><span class="cl"><span class="s2">&#34;Condition&#34;</span><span class="err">:</span> <span class="p">{</span> <span class="nt">&#34;StringNotEquals&#34;</span><span class="p">:</span> <span class="p">{</span> <span class="nt">&#34;aws:PrincipalAccount&#34;</span><span class="p">:</span> <span class="s2">&#34;YOUR_AWS_ACCOUNT_ID&#34;</span> <span class="p">}</span> <span class="p">}</span></span></span></code></pre></div></div>
<p>加上这一行之后，AWS Emulator（开着 <code>--auth off</code>，这样匿名那一半也能测）和 Moto 都允许 worker 发布，同时照样拒绝对 <code>_internal/</code> 的匿名读取。但只有 AWS Emulator 抓到了没加这一行的版本。Moto、LocalStack 4.12 和 Floci 不管有没有这一行都放行了复制，所以换成它们中的任何一个，我都会把有问题的策略发出去。要说为什么应该拿手头最严格的模拟器来测策略，这是我找到的最有力的理由。把 MinIO 环境里的策略搬到 S3 上又没加这个条件，在第一次真正发布之前，什么都不会报错。</p>
<p>（我自己的 Patch 服务器把 bundle 存在 Tigris 上，而 Tigris 根本没有 bucket 策略，所以我把 <code>_internal/</code> 放在一个单独的私有 bucket 里，Patch 配置里的 <code>S3_INTERNAL_BUCKET</code> 就是干这个用的。另外，我也在给 Codemagic 写关于 Patch 的付费客座文章。这篇不是，也没有人为它付钱。）</p>

<h2 class="relative group">现在我会用哪个
    <div id="现在我会用哪个" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e7%8e%b0%e5%9c%a8%e6%88%91%e4%bc%9a%e7%94%a8%e5%93%aa%e4%b8%aa" aria-label="锚点">#</a>
    </span>
    
</h2>
<table>
	<thead>
			<tr>
					<th></th>
					<th>就绪时间</th>
					<th>空闲内存</th>
					<th>镜像</th>
					<th>账号</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td>AWS Emulator 1.0.342</td>
					<td>0.4–0.5 s</td>
					<td>176 MiB</td>
					<td>1.67 GB（需自己构建）</td>
					<td>不需要</td>
			</tr>
			<tr>
					<td>LocalStack 4.12</td>
					<td>0.9–2.2 s</td>
					<td>90 MiB</td>
					<td>1.89 GB</td>
					<td>4.12 不需要，之后的版本要 token</td>
			</tr>
			<tr>
					<td>Moto 5.2.6（server 模式）</td>
					<td>0.7–0.8 s</td>
					<td>45 MiB</td>
					<td>560 MB</td>
					<td>不需要</td>
			</tr>
			<tr>
					<td>Floci 1.5.5</td>
					<td>0.3–0.4 s</td>
					<td>11 MiB</td>
					<td>292 MB</td>
					<td>不需要</td>
			</tr>
	</tbody>
</table>
<p>在我的 Mac 上用 Docker 测的，每个都冷启动了三次。没有哪个慢到会成为问题。</p>
<p>后端测试我会从 AWS Emulator 开始，因为它能抓到的错误（权限、签名、Block Public Access）恰恰是不然只会在生产环境里冒出来的那些；而且在这份名单里，只有它同时做到了免费、持续更新、出自做 S3 的那拨人之手。在 CORS 预检实现之前，我会留一个小 job，在 Floci 上跑面向浏览器的那几项检查（预检，以及从公开前缀匿名下载）。开了严格签名的 LocalStack 4.12 也能通过这些，但它已经冻结，不会再有修复。如果你的测试是 Python 写的，又不想跑容器，Moto 的进程内 mock 仍然是最轻量的选择，只是别指望它来检查预签名 URL。</p>
<p>这 12 项检查都很短，一个下午就能针对你自己的 bucket 写出来。我建议在信任任何模拟器之前都先这么做，包括这一个。表里的每个工具没通过的检查都不一样，而那些正是你的测试会悄悄放过去的缺口。</p>
]]></content:encoded>
      <media:content url="https://jared.lynskey.co.nz/en/posts/2026/aws-emulator-setup/featured.jpg" medium="image" type="image/jpeg" />
    </item>
    <item>
      <title>直传 S3 也不放弃图片压缩：Tigris 对象通知与 Django</title>
      <link>https://jared.lynskey.co.nz/zh-cn/posts/2026/tigris-object-notifications/</link>
      <guid isPermaLink="true">https://jared.lynskey.co.nz/zh-cn/posts/2026/tigris-object-notifications/</guid>
      <pubDate>Sat, 07 Nov 2026 00:00:00 +1300</pubDate>
      <dc:creator>Jared Lynskey</dc:creator>
      <category>Tigris</category>
      <category>S3</category>
      <category>S3 Event Notifications</category>
      <category>Presigned URLs</category>
      <category>Image Uploads</category>
      <category>Image Compression</category>
      <category>Object Storage</category>
      <category>Webhooks</category>
      <category>Django</category>
      <category>Fly.io</category>
      <category>Discord</category>
      <description>用预签名 URL 直传 S3 很快，但后端根本不知道文件什么时候到了，图片压缩和缩略图就变得别扭。Tigris 对象通知解决了这个问题：bucket 向 Django 的 webhook 发一个 POST，用户还在打字，照片就已经处理好了。同一个信号还能把日志文件变成 Discord 告警。</description>
      <content:encoded><![CDATA[<p>我做过的每一个图片上传功能，骨子里都有同一个矛盾。上传的人希望照片赶紧传上去。我希望服务器把图片压缩好、生成缩略图、算出 blurhash 和几个主色、去掉不该留在里面的元数据，再按应用其他部分期望的方式存起来。要把我这份清单做全，最省事的办法是让文件经过 API，而上传的人那份清单，恰恰就是被这一步拖慢的。</p>
<p>快的办法是预签名 URL。API 签出一个 URL，客户端把文件直接 PUT 进 bucket，字节完全不经过 API。它能成为标准答案是有道理的。可你一这么做，后端就被踢出了整个流程：它发出一个 URL，然后就什么都听不到了。文件到了吗？大小是预期的吗？它到底是不是 JPEG？总得有什么东西告诉后端，而常见的几个选项都有点让人叹气：相信客户端会回调一声&quot;传好了&quot;（客户端会撒谎、会崩溃、会断网），按计划轮询 bucket（又慢又浪费），或者等第一次读取时再懒处理（第一个看的人买单，而且你依然不知道文件什么时候才能放心引用）。这就是为什么直接上传到 S3 看起来比实际更难。上传本身很容易，缺的是信号。</p>
<p>对象通知就是这个信号。我在 Fly.io 上用的 S3 兼容存储 Tigris，每当 bucket 里有对象被创建或删除，都会向你的应用 POST 一小段 JSON。在 AWS 上对应的东西叫 S3 事件通知，但通常要先接上 Lambda、SQS 队列或者 EventBridge，事件才能到你的代码（SNS 倒是能直接向 HTTPS 端点 POST，可那样你得先确认订阅、再验证签名）。Tigris 把它压缩成一个发往你自己 URL 的 HTTP 请求，存储到后端的信号传递，大概没法比这更简单了。这个功能存在的意义，就是让 bucket 能直接跟你的后端说话，不用你在中间搭桥。</p>
<p>这篇文章讲完整个流程：怎么配置，接收事件的 Django 端点，服务端处理一步不少的预签名上传，以及几个坑（CORS、大小限制、过期、多文件）。最后再看同一个信号怎么干后台的活：日志文件一落进 bucket 就变成 Discord 告警，让我删掉了一个伺候了好几个月的 cron 任务。</p>

<h2 class="relative group">简短的答案
    <div id="简短的答案" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e7%ae%80%e7%9f%ad%e7%9a%84%e7%ad%94%e6%a1%88" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>在 Tigris 控制台里打开 bucket，进 Settings，再进 Object notifications。填一个你应用上的 HTTPS URL，认证选 basic auth（一组用户名和密码，Tigris 会放在 <code>Authorization</code> 头里发过来；另一个选项是 bearer token）。再加一个过滤条件，只接收你关心的那些 key。过滤条件是一个类 SQL 的表达式，&ldquo;只要这篇文章处理的那三个前缀&quot;写出来是这样：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sql" data-lang="sql"><span class="line"><span class="cl"><span class="k">WHERE</span><span class="w"> </span><span class="o">`</span><span class="k">key</span><span class="o">`</span><span class="w"> </span><span class="n">REGEXP</span><span class="w"> </span><span class="s2">&#34;^(uploads/raw|logs/api|media)/&#34;</span></span></span></code></pre></div></div>
<p><figure><span class="mc-img"><img
    class="my-0 rounded-md"
    loading="lazy"
    decoding="async"
    fetchpriority="auto"
    alt="Tigris bucket 设置的 Notifications 标签页：对象通知已打开，一个 webhook URL，用户名加密码的 basic auth，以及一个类 SQL 的 key 过滤条件"
    width="1204"
    height="671"
    src="https://jared.lynskey.co.nz/en/posts/2026/tigris-object-notifications/tigris-object-notifications-settings_hu_7e9da7c6ca1125db.webp"
    srcset="https://jared.lynskey.co.nz/en/posts/2026/tigris-object-notifications/tigris-object-notifications-settings_hu_7e9da7c6ca1125db.webp 800w, https://jared.lynskey.co.nz/en/posts/2026/tigris-object-notifications/tigris-object-notifications-settings_hu_4ddc968851d1f052.webp 1280w"
    sizes="(min-width: 768px) 50vw, 65vw"
    data-zoom-src="https://jared.lynskey.co.nz/en/posts/2026/tigris-object-notifications/tigris-object-notifications-settings_hu_4ddc968851d1f052.webp"></span><figcaption>整套配置就这些：你应用上的一个 HTTPS URL、basic auth，以及对 key 的过滤条件。这里的 bucket 名和 URL 已换成示例。</figcaption></figure>
</p>
<p>之后每有变化，Tigris 就会发来这样一批事件：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-json" data-lang="json"><span class="line"><span class="cl"><span class="p">{</span>
</span></span><span class="line"><span class="cl">  <span class="nt">&#34;events&#34;</span><span class="p">:</span> <span class="p">[</span>
</span></span><span class="line"><span class="cl">    <span class="p">{</span>
</span></span><span class="line"><span class="cl">      <span class="nt">&#34;eventName&#34;</span><span class="p">:</span> <span class="s2">&#34;OBJECT_CREATED_PUT&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">      <span class="nt">&#34;eventTime&#34;</span><span class="p">:</span> <span class="s2">&#34;2026-11-07T09:12:44Z&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">      <span class="nt">&#34;bucket&#34;</span><span class="p">:</span> <span class="s2">&#34;example-bucket&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">      <span class="nt">&#34;object&#34;</span><span class="p">:</span> <span class="p">{</span> <span class="nt">&#34;key&#34;</span><span class="p">:</span> <span class="s2">&#34;uploads/raw/1f3c.jpg&#34;</span><span class="p">,</span> <span class="nt">&#34;size&#34;</span><span class="p">:</span> <span class="mi">48213</span><span class="p">,</span> <span class="nt">&#34;eTag&#34;</span><span class="p">:</span> <span class="s2">&#34;9b2c…&#34;</span> <span class="p">}</span>
</span></span><span class="line"><span class="cl">    <span class="p">}</span>
</span></span><span class="line"><span class="cl">  <span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span></span></span></code></pre></div></div>
<p>你会遇到的事件名有 <code>OBJECT_CREATED_PUT</code>、<code>OBJECT_CREATED_MULTIPART</code>、<code>OBJECT_CREATED_COPY</code> 和 <code>OBJECT_DELETED</code>，另外还有重命名和存储层级变更。你的端点负责校验 auth，按 key 前缀把每个事件分发出去，然后在十秒内返回 2xx。有三件事最好一开始就考虑进去：投递是 at least once 的，所以你会收到重复；事件是成批发送的，所以一个请求里可能带着好几个；它们还可能乱序到达，所以别拿到达顺序去判断哪个是&quot;最新&rdquo;。</p>

<h2 class="relative group">接收 bucket 事件的 Django webhook
    <div id="接收-bucket-事件的-django-webhook" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e6%8e%a5%e6%94%b6-bucket-%e4%ba%8b%e4%bb%b6%e7%9a%84-django-webhook" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>这个 view 是故意写得很无聊的。它校验 auth、解析请求体，然后把每个事件交给路由：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">base64</span><span class="o">,</span> <span class="nn">hmac</span><span class="o">,</span> <span class="nn">json</span><span class="o">,</span> <span class="nn">logging</span>
</span></span><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">django.conf</span> <span class="kn">import</span> <span class="n">settings</span>
</span></span><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">django.http</span> <span class="kn">import</span> <span class="n">JsonResponse</span>
</span></span><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">django.views.decorators.csrf</span> <span class="kn">import</span> <span class="n">csrf_exempt</span>
</span></span><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">django.views.decorators.http</span> <span class="kn">import</span> <span class="n">require_POST</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">.bucket_events</span> <span class="kn">import</span> <span class="n">handle</span><span class="p">,</span> <span class="n">parse</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="vm">__name__</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">_authorised</span><span class="p">(</span><span class="n">request</span><span class="p">,</span> <span class="n">secret</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">bool</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">    <span class="n">header</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">headers</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&#34;Authorization&#34;</span><span class="p">,</span> <span class="s2">&#34;&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">if</span> <span class="ow">not</span> <span class="n">header</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">&#34;Basic &#34;</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">        <span class="k">return</span> <span class="kc">False</span>
</span></span><span class="line"><span class="cl">    <span class="k">try</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">        <span class="n">user</span><span class="p">,</span> <span class="n">_</span><span class="p">,</span> <span class="n">password</span> <span class="o">=</span> <span class="n">base64</span><span class="o">.</span><span class="n">b64decode</span><span class="p">(</span><span class="n">header</span><span class="p">[</span><span class="mi">6</span><span class="p">:])</span><span class="o">.</span><span class="n">decode</span><span class="p">()</span><span class="o">.</span><span class="n">partition</span><span class="p">(</span><span class="s2">&#34;:&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">except</span> <span class="p">(</span><span class="ne">ValueError</span><span class="p">,</span> <span class="ne">UnicodeDecodeError</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">        <span class="k">return</span> <span class="kc">False</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="n">hmac</span><span class="o">.</span><span class="n">compare_digest</span><span class="p">(</span><span class="n">user</span><span class="o">.</span><span class="n">encode</span><span class="p">(),</span> <span class="sa">b</span><span class="s2">&#34;tigris&#34;</span><span class="p">)</span> <span class="ow">and</span> <span class="n">hmac</span><span class="o">.</span><span class="n">compare_digest</span><span class="p">(</span>
</span></span><span class="line"><span class="cl">        <span class="n">password</span><span class="o">.</span><span class="n">encode</span><span class="p">(),</span> <span class="n">secret</span><span class="o">.</span><span class="n">encode</span><span class="p">()</span>
</span></span><span class="line"><span class="cl">    <span class="p">)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="nd">@csrf_exempt</span>
</span></span><span class="line"><span class="cl"><span class="nd">@require_POST</span>
</span></span><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">bucket_webhook</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">    <span class="n">secret</span> <span class="o">=</span> <span class="n">settings</span><span class="o">.</span><span class="n">BUCKET_WEBHOOK_SECRET</span>
</span></span><span class="line"><span class="cl">    <span class="k">if</span> <span class="ow">not</span> <span class="n">secret</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">        <span class="k">return</span> <span class="n">JsonResponse</span><span class="p">({</span><span class="s2">&#34;error&#34;</span><span class="p">:</span> <span class="s2">&#34;not configured&#34;</span><span class="p">},</span> <span class="n">status</span><span class="o">=</span><span class="mi">503</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">if</span> <span class="ow">not</span> <span class="n">_authorised</span><span class="p">(</span><span class="n">request</span><span class="p">,</span> <span class="n">secret</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">        <span class="k">return</span> <span class="n">JsonResponse</span><span class="p">({</span><span class="s2">&#34;error&#34;</span><span class="p">:</span> <span class="s2">&#34;unauthorised&#34;</span><span class="p">},</span> <span class="n">status</span><span class="o">=</span><span class="mi">401</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">try</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">        <span class="n">body</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">request</span><span class="o">.</span><span class="n">body</span> <span class="ow">or</span> <span class="sa">b</span><span class="s2">&#34;</span><span class="si">{}</span><span class="s2">&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">except</span> <span class="ne">ValueError</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">        <span class="k">return</span> <span class="n">JsonResponse</span><span class="p">({</span><span class="s2">&#34;error&#34;</span><span class="p">:</span> <span class="s2">&#34;invalid json&#34;</span><span class="p">},</span> <span class="n">status</span><span class="o">=</span><span class="mi">400</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">for</span> <span class="n">event</span> <span class="ow">in</span> <span class="n">parse</span><span class="p">(</span><span class="n">body</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">        <span class="k">try</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">            <span class="n">handle</span><span class="p">(</span><span class="n">event</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">        <span class="k">except</span> <span class="ne">Exception</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">            <span class="n">logger</span><span class="o">.</span><span class="n">exception</span><span class="p">(</span><span class="s2">&#34;bucket event failed: </span><span class="si">%s</span><span class="s2">&#34;</span><span class="p">,</span> <span class="n">event</span><span class="o">.</span><span class="n">key</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="n">JsonResponse</span><span class="p">({</span><span class="s2">&#34;ok&#34;</span><span class="p">:</span> <span class="kc">True</span><span class="p">})</span></span></span></code></pre></div></div>
<p><code>compare_digest</code> 的作用，是让校验不管收到什么都花同样的时间（要传 bytes，因为传字符串时它遇到非 ASCII 输入会直接抛异常，一个精心构造的头拿到的就是 500 而不是 401）。如果 secret 没有配置，比如在一台刚开出来的 staging 机器上，端点会拒绝所有请求，而不是放行所有请求。handler 抛了异常，会记一条日志，但照样回 200：Tigris 对一切非 2xx 的响应都会重试，重试撞上的还是同一个 bug，而且到那时事件已经被认领了（往下两节会讲），重试其实什么也干不了。真正让问题得到修复的，是那条日志。这里有个前提：失败得是确定性的。存储那边一次短暂的超时，会把这个事件永远消耗掉：照片无所谓，因为保存时可以自己处理；但日志文件的告警就丢了。如果你在意这一点，就在 <code>except</code> 里释放认领，然后回 500 让 Tigris 重试，并接受真正的 bug 会一直重试到修好为止。</p>

<h2 class="relative group">按前缀路由
    <div id="按前缀路由" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e6%8c%89%e5%89%8d%e7%bc%80%e8%b7%af%e7%94%b1" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>bucket 里的东西本来就是按前缀组织的，所以按前缀路由最顺手。每种用途就是一个函数，加上表里的一行：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="n">HANDLERS</span> <span class="o">=</span> <span class="p">[</span>
</span></span><span class="line"><span class="cl">    <span class="p">(</span><span class="s2">&#34;uploads/raw/&#34;</span><span class="p">,</span>   <span class="n">on_raw_photo</span><span class="p">),</span>
</span></span><span class="line"><span class="cl">    <span class="p">(</span><span class="s2">&#34;logs/api/&#34;</span><span class="p">,</span>      <span class="n">on_api_log_file</span><span class="p">),</span>
</span></span><span class="line"><span class="cl">    <span class="p">(</span><span class="s2">&#34;media/&#34;</span><span class="p">,</span>         <span class="n">on_media_change</span><span class="p">),</span>
</span></span><span class="line"><span class="cl"><span class="p">]</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">handle</span><span class="p">(</span><span class="n">event</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">    <span class="k">for</span> <span class="n">prefix</span><span class="p">,</span> <span class="n">fn</span> <span class="ow">in</span> <span class="n">HANDLERS</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">        <span class="k">if</span> <span class="n">event</span><span class="o">.</span><span class="n">key</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="n">prefix</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">            <span class="k">if</span> <span class="n">claim</span><span class="p">(</span><span class="n">event</span><span class="p">):</span>  <span class="c1"># 见下文</span>
</span></span><span class="line"><span class="cl">                <span class="n">fn</span><span class="p">(</span><span class="n">event</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">            <span class="k">return</span></span></span></code></pre></div></div>
<p>以后要加第四种用途，就是多一个函数、多一行。我比自己预想的还要喜欢这种写法。</p>

<h2 class="relative group">At least once 的意思是&quot;可能两次&quot;
    <div id="at-least-once-的意思是可能两次" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#at-least-once-%e7%9a%84%e6%84%8f%e6%80%9d%e6%98%af%e5%8f%af%e8%83%bd%e4%b8%a4%e6%ac%a1" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>端点响应慢或者返回错误时，Tigris 会重试，而且和大多数 webhook 发送方一样，它不保证每个事件你只会看到一次。所以在 handler 运行之前，先按 <code>(key, eTag, eventName)</code> 认领这个事件。eTag 很关键：文件被覆盖后就是一个新版本，eTag 也是新的，理应再处理一次；同一个版本的重复投递则不该再处理。</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">django.core.cache</span> <span class="kn">import</span> <span class="n">cache</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">claim</span><span class="p">(</span><span class="n">event</span><span class="p">,</span> <span class="n">ttl</span><span class="o">=</span><span class="mi">7</span> <span class="o">*</span> <span class="mi">24</span> <span class="o">*</span> <span class="mi">3600</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">bool</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">    <span class="n">k</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">&#34;bucket-event:</span><span class="si">{</span><span class="n">event</span><span class="o">.</span><span class="n">name</span><span class="si">}</span><span class="s2">:</span><span class="si">{</span><span class="n">event</span><span class="o">.</span><span class="n">key</span><span class="si">}</span><span class="s2">:</span><span class="si">{</span><span class="n">event</span><span class="o">.</span><span class="n">etag</span><span class="si">}</span><span class="s2">&#34;</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="n">cache</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">k</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="n">ttl</span><span class="p">)</span>  <span class="c1"># 已经被别人认领就返回 False</span></span></span></code></pre></div></div>
<p><code>cache.add</code> 只在 key 不存在时才写入，这在 Redis 里是原子操作，所以两台机器收到同一次重试，不可能都抢到。实际项目里我用的是基于数据库的租约，因为我希望它在 Redis 被清空后还能留着，但思路完全一样。</p>

<h2 class="relative group">图片直传 S3，处理流程一样不少
    <div id="图片直传-s3处理流程一样不少" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e5%9b%be%e7%89%87%e7%9b%b4%e4%bc%a0-s3%e5%a4%84%e7%90%86%e6%b5%81%e7%a8%8b%e4%b8%80%e6%a0%b7%e4%b8%8d%e5%b0%91" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>前面那些管道活，值就值在这一部分。</p>
<p>以前，给一条帖子选照片时，应用会先把照片攥在手里，等你点了&quot;发布&quot;，再把它们全部以 base64 发给 API。API 接着缩放尺寸、生成缩略图、算出主色和 blurhash，全部做完才回复。十张照片再碰上慢网速，那个转圈就转得很久。而且一个带着十张 base64 照片的请求，对 API 服务器来说是一个又大又慢、还得整个攥在内存里的请求体，这也是我一直悄悄把照片压得很小的真正原因。</p>
<p>现在变成了四步，而且每一步后端都知道。</p>

<h3 class="relative group">1. 客户端先缩图，再申请一个槽位
    <div id="1-客户端先缩图再申请一个槽位" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#1-%e5%ae%a2%e6%88%b7%e7%ab%af%e5%85%88%e7%bc%a9%e5%9b%be%e5%86%8d%e7%94%b3%e8%af%b7%e4%b8%80%e4%b8%aa%e6%a7%bd%e4%bd%8d" aria-label="锚点">#</a>
    </span>
    
</h3>
<p>应用仍然会在设备上缩图，缩到服务器保留的最长边（1080px），存成质量 0.7 的 JPEG。这一步缩放只是为了省用户的流量，没有别的目的。然后它向 API 申请一个上传槽位：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">uuid</span>
</span></span><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">datetime</span> <span class="kn">import</span> <span class="n">timedelta</span>
</span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">boto3</span>
</span></span><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">django.utils</span> <span class="kn">import</span> <span class="n">timezone</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="n">s3</span> <span class="o">=</span> <span class="n">boto3</span><span class="o">.</span><span class="n">client</span><span class="p">(</span><span class="s2">&#34;s3&#34;</span><span class="p">,</span> <span class="n">endpoint_url</span><span class="o">=</span><span class="s2">&#34;https://fly.storage.tigris.dev&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="n">BUCKET</span> <span class="o">=</span> <span class="s2">&#34;example-bucket&#34;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">create_upload_slot</span><span class="p">(</span><span class="n">user</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">    <span class="n">public_id</span> <span class="o">=</span> <span class="n">uuid</span><span class="o">.</span><span class="n">uuid4</span><span class="p">()</span>
</span></span><span class="line"><span class="cl">    <span class="n">key</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">&#34;uploads/raw/</span><span class="si">{</span><span class="n">public_id</span><span class="si">}</span><span class="s2">.jpg&#34;</span>
</span></span><span class="line"><span class="cl">    <span class="n">upload</span> <span class="o">=</span> <span class="n">PhotoUpload</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">create</span><span class="p">(</span>
</span></span><span class="line"><span class="cl">        <span class="n">user</span><span class="o">=</span><span class="n">user</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">        <span class="n">public_id</span><span class="o">=</span><span class="n">public_id</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">        <span class="n">raw_key</span><span class="o">=</span><span class="n">key</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">        <span class="n">expires_at</span><span class="o">=</span><span class="n">timezone</span><span class="o">.</span><span class="n">now</span><span class="p">()</span> <span class="o">+</span> <span class="n">timedelta</span><span class="p">(</span><span class="n">days</span><span class="o">=</span><span class="mi">2</span><span class="p">),</span>
</span></span><span class="line"><span class="cl">    <span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="n">url</span> <span class="o">=</span> <span class="n">s3</span><span class="o">.</span><span class="n">generate_presigned_url</span><span class="p">(</span>
</span></span><span class="line"><span class="cl">        <span class="s2">&#34;put_object&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">        <span class="n">Params</span><span class="o">=</span><span class="p">{</span><span class="s2">&#34;Bucket&#34;</span><span class="p">:</span> <span class="n">BUCKET</span><span class="p">,</span> <span class="s2">&#34;Key&#34;</span><span class="p">:</span> <span class="n">key</span><span class="p">,</span> <span class="s2">&#34;ContentType&#34;</span><span class="p">:</span> <span class="s2">&#34;image/jpeg&#34;</span><span class="p">},</span>
</span></span><span class="line"><span class="cl">        <span class="n">ExpiresIn</span><span class="o">=</span><span class="mi">15</span> <span class="o">*</span> <span class="mi">60</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">        <span class="s2">&#34;upload_id&#34;</span><span class="p">:</span> <span class="nb">str</span><span class="p">(</span><span class="n">upload</span><span class="o">.</span><span class="n">public_id</span><span class="p">),</span>
</span></span><span class="line"><span class="cl">        <span class="s2">&#34;url&#34;</span><span class="p">:</span> <span class="n">url</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">        <span class="s2">&#34;method&#34;</span><span class="p">:</span> <span class="s2">&#34;PUT&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">        <span class="s2">&#34;headers&#34;</span><span class="p">:</span> <span class="p">{</span><span class="s2">&#34;Content-Type&#34;</span><span class="p">:</span> <span class="s2">&#34;image/jpeg&#34;</span><span class="p">},</span>
</span></span><span class="line"><span class="cl">        <span class="s2">&#34;expires_in&#34;</span><span class="p">:</span> <span class="mi">15</span> <span class="o">*</span> <span class="mi">60</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="p">}</span></span></span></code></pre></div></div>
<p>槽位先是数据库里的一行，然后才是一个文件。正因为这样，服务器才知道是谁申请的、什么时候过期，以及后来它到底有没有被用过。</p>

<h3 class="relative group">2. 客户端直接 PUT 到 bucket
    <div id="2-客户端直接-put-到-bucket" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#2-%e5%ae%a2%e6%88%b7%e7%ab%af%e7%9b%b4%e6%8e%a5-put-%e5%88%b0-bucket" aria-label="锚点">#</a>
    </span>
    
</h3>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-ts" data-lang="ts"><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">slot</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">api</span><span class="p">.</span><span class="nx">post</span><span class="p">(</span><span class="s2">&#34;/photo-uploads/&#34;</span><span class="p">);</span>
</span></span><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">res</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">fetch</span><span class="p">(</span><span class="nx">slot</span><span class="p">.</span><span class="nx">url</span><span class="p">,</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">  <span class="nx">method</span>: <span class="kt">slot.method</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">  <span class="nx">headers</span>: <span class="kt">slot.headers</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">  <span class="nx">body</span>: <span class="kt">blob</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"><span class="p">});</span>
</span></span><span class="line"><span class="cl"><span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">res</span><span class="p">.</span><span class="nx">ok</span><span class="p">)</span> <span class="k">throw</span> <span class="k">new</span> <span class="nb">Error</span><span class="p">(</span><span class="s2">&#34;upload refused&#34;</span><span class="p">);</span>
</span></span><span class="line"><span class="cl"><span class="c1">// 把 slot.upload_id 留着，保存时要用
</span></span></span></code></pre></div></div>
<p>API 从头到尾见不到这些字节。浏览器、React Native 或原生应用都是同一套做法；唯一的区别是浏览器需要在 bucket 上配一条 CORS 规则，这个放在下面的坑里讲。</p>

<h3 class="relative group">3. Tigris 通知后端，处理流程跑起来
    <div id="3-tigris-通知后端处理流程跑起来" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#3-tigris-%e9%80%9a%e7%9f%a5%e5%90%8e%e7%ab%af%e5%a4%84%e7%90%86%e6%b5%81%e7%a8%8b%e8%b7%91%e8%b5%b7%e6%9d%a5" aria-label="锚点">#</a>
    </span>
    
</h3>
<p>几秒之内，Tigris 就会针对 <code>uploads/raw/…</code> 发来一个 <code>OBJECT_CREATED_PUT</code>，handler 跑的是 base64 路径一直在用的那套图片处理流程，而这时用户还在输入标题：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">io</span> <span class="kn">import</span> <span class="n">BytesIO</span>
</span></span><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">PIL</span> <span class="kn">import</span> <span class="n">Image</span><span class="p">,</span> <span class="n">ImageOps</span>
</span></span><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">django.core.files.base</span> <span class="kn">import</span> <span class="n">ContentFile</span>
</span></span><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">django.utils</span> <span class="kn">import</span> <span class="n">timezone</span>
</span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">blurhash</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="n">MAX_BYTES</span> <span class="o">=</span> <span class="mi">15</span> <span class="o">*</span> <span class="mi">1024</span> <span class="o">*</span> <span class="mi">1024</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">on_raw_photo</span><span class="p">(</span><span class="n">event</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">    <span class="k">if</span> <span class="ow">not</span> <span class="n">event</span><span class="o">.</span><span class="n">name</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">&#34;OBJECT_CREATED&#34;</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">        <span class="k">return</span>
</span></span><span class="line"><span class="cl">    <span class="k">if</span> <span class="n">event</span><span class="o">.</span><span class="n">size</span> <span class="o">&gt;</span> <span class="n">MAX_BYTES</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">        <span class="n">s3</span><span class="o">.</span><span class="n">delete_object</span><span class="p">(</span><span class="n">Bucket</span><span class="o">=</span><span class="n">event</span><span class="o">.</span><span class="n">bucket</span><span class="p">,</span> <span class="n">Key</span><span class="o">=</span><span class="n">event</span><span class="o">.</span><span class="n">key</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">        <span class="k">return</span>
</span></span><span class="line"><span class="cl">    <span class="n">upload</span> <span class="o">=</span> <span class="n">PhotoUpload</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">filter</span><span class="p">(</span><span class="n">raw_key</span><span class="o">=</span><span class="n">event</span><span class="o">.</span><span class="n">key</span><span class="p">,</span> <span class="n">processed_at</span><span class="o">=</span><span class="kc">None</span><span class="p">)</span><span class="o">.</span><span class="n">first</span><span class="p">()</span>
</span></span><span class="line"><span class="cl">    <span class="k">if</span> <span class="n">upload</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">        <span class="k">return</span>
</span></span><span class="line"><span class="cl">    <span class="n">data</span> <span class="o">=</span> <span class="n">s3</span><span class="o">.</span><span class="n">get_object</span><span class="p">(</span><span class="n">Bucket</span><span class="o">=</span><span class="n">event</span><span class="o">.</span><span class="n">bucket</span><span class="p">,</span> <span class="n">Key</span><span class="o">=</span><span class="n">event</span><span class="o">.</span><span class="n">key</span><span class="p">)[</span><span class="s2">&#34;Body&#34;</span><span class="p">]</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
</span></span><span class="line"><span class="cl">    <span class="n">process</span><span class="p">(</span><span class="n">upload</span><span class="p">,</span> <span class="n">data</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">to_webp</span><span class="p">(</span><span class="n">im</span><span class="p">,</span> <span class="n">longest</span><span class="p">,</span> <span class="n">quality</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">    <span class="n">im</span> <span class="o">=</span> <span class="n">im</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
</span></span><span class="line"><span class="cl">    <span class="n">im</span><span class="o">.</span><span class="n">thumbnail</span><span class="p">((</span><span class="n">longest</span><span class="p">,</span> <span class="n">longest</span><span class="p">))</span>
</span></span><span class="line"><span class="cl">    <span class="n">buf</span> <span class="o">=</span> <span class="n">BytesIO</span><span class="p">()</span>
</span></span><span class="line"><span class="cl">    <span class="n">im</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">buf</span><span class="p">,</span> <span class="s2">&#34;WEBP&#34;</span><span class="p">,</span> <span class="n">quality</span><span class="o">=</span><span class="n">quality</span><span class="p">,</span> <span class="n">icc_profile</span><span class="o">=</span><span class="n">im</span><span class="o">.</span><span class="n">info</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&#34;icc_profile&#34;</span><span class="p">))</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="n">ContentFile</span><span class="p">(</span><span class="n">buf</span><span class="o">.</span><span class="n">getvalue</span><span class="p">())</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">process</span><span class="p">(</span><span class="n">upload</span><span class="p">,</span> <span class="n">data</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">    <span class="k">with</span> <span class="n">Image</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="n">BytesIO</span><span class="p">(</span><span class="n">data</span><span class="p">))</span> <span class="k">as</span> <span class="n">im</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">        <span class="n">im</span> <span class="o">=</span> <span class="n">ImageOps</span><span class="o">.</span><span class="n">exif_transpose</span><span class="p">(</span><span class="n">im</span><span class="p">)</span><span class="o">.</span><span class="n">convert</span><span class="p">(</span><span class="s2">&#34;RGB&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">        <span class="n">im</span><span class="o">.</span><span class="n">thumbnail</span><span class="p">((</span><span class="mi">1080</span><span class="p">,</span> <span class="mi">1080</span><span class="p">))</span>
</span></span><span class="line"><span class="cl">        <span class="n">upload</span><span class="o">.</span><span class="n">width</span><span class="p">,</span> <span class="n">upload</span><span class="o">.</span><span class="n">height</span> <span class="o">=</span> <span class="n">im</span><span class="o">.</span><span class="n">size</span>
</span></span><span class="line"><span class="cl">        <span class="n">upload</span><span class="o">.</span><span class="n">dominant_color</span> <span class="o">=</span> <span class="s2">&#34;#</span><span class="si">%02x%02x%02x</span><span class="s2">&#34;</span> <span class="o">%</span> <span class="n">im</span><span class="o">.</span><span class="n">resize</span><span class="p">((</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">))</span><span class="o">.</span><span class="n">getpixel</span><span class="p">((</span><span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">))</span>
</span></span><span class="line"><span class="cl">        <span class="n">upload</span><span class="o">.</span><span class="n">image</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="sa">f</span><span class="s2">&#34;</span><span class="si">{</span><span class="n">upload</span><span class="o">.</span><span class="n">public_id</span><span class="si">}</span><span class="s2">.webp&#34;</span><span class="p">,</span> <span class="n">to_webp</span><span class="p">(</span><span class="n">im</span><span class="p">,</span> <span class="mi">1080</span><span class="p">,</span> <span class="mi">80</span><span class="p">),</span> <span class="n">save</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">        <span class="n">upload</span><span class="o">.</span><span class="n">thumbnail</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="sa">f</span><span class="s2">&#34;</span><span class="si">{</span><span class="n">upload</span><span class="o">.</span><span class="n">public_id</span><span class="si">}</span><span class="s2">_t.webp&#34;</span><span class="p">,</span> <span class="n">to_webp</span><span class="p">(</span><span class="n">im</span><span class="p">,</span> <span class="mi">320</span><span class="p">,</span> <span class="mi">70</span><span class="p">),</span> <span class="n">save</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">        <span class="n">upload</span><span class="o">.</span><span class="n">blurhash</span> <span class="o">=</span> <span class="n">blurhash</span><span class="o">.</span><span class="n">encode</span><span class="p">(</span><span class="n">im</span><span class="o">.</span><span class="n">resize</span><span class="p">((</span><span class="mi">32</span><span class="p">,</span> <span class="mi">32</span><span class="p">)),</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="n">upload</span><span class="o">.</span><span class="n">processed_at</span> <span class="o">=</span> <span class="n">timezone</span><span class="o">.</span><span class="n">now</span><span class="p">()</span>
</span></span><span class="line"><span class="cl">    <span class="n">upload</span><span class="o">.</span><span class="n">save</span><span class="p">()</span></span></span></code></pre></div></div>
<p><code>on_raw_photo</code> 的第一行比看上去重要。路由会把 <code>uploads/raw/</code> 下的每一个事件都送到这里，包括孤儿清理和上面那个大小检查自己触发的删除，而对一个已经不在的 key 调 <code>get_object</code>，只会抛异常。</p>
<p>所以客户端那一步缩放，从头到尾只是为了传输。格式（WebP）、尺寸、缩略图、占位图，以及存下来的宽高，依然由服务器说了算。经 Pillow 重新编码还会顺手丢掉 EXIF 块，手机相机就是把 GPS 位置放在那里的，所以隐私清理是白送的。ICC 配置文件本来也会一起丢掉，那会让广色域的手机照片颜色稍微偏一点，所以 <code>to_webp</code> 把它原样带了过去。如果你更想保留像素级的原图，那就别在客户端缩；流程完全一样，只是 PUT 更大，而更大的 PUT 恰恰是这套设计让它变便宜的那件事。有一个坑：iPhone 的原图是 HEIC，Pillow 只有装了 <code>pillow-heif</code> 才打得开。</p>

<h3 class="relative group">4. 保存时挂上照片，从不等 webhook
    <div id="4-保存时挂上照片从不等-webhook" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#4-%e4%bf%9d%e5%ad%98%e6%97%b6%e6%8c%82%e4%b8%8a%e7%85%a7%e7%89%87%e4%bb%8e%e4%b8%8d%e7%ad%89-webhook" aria-label="锚点">#</a>
    </span>
    
</h3>
<p>用户点&quot;发布&quot;时，请求里带的是 <code>upload_ids</code>，而不是图片数据。服务器逐个查找，有三种结果：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">take</span><span class="p">(</span><span class="n">user</span><span class="p">,</span> <span class="n">upload_ids</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">    <span class="n">uploads</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">PhotoUpload</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">filter</span><span class="p">(</span>
</span></span><span class="line"><span class="cl">        <span class="n">user</span><span class="o">=</span><span class="n">user</span><span class="p">,</span> <span class="n">public_id__in</span><span class="o">=</span><span class="n">upload_ids</span><span class="p">,</span> <span class="n">expires_at__gt</span><span class="o">=</span><span class="n">timezone</span><span class="o">.</span><span class="n">now</span><span class="p">()</span>
</span></span><span class="line"><span class="cl">    <span class="p">))</span>
</span></span><span class="line"><span class="cl">    <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">uploads</span><span class="p">)</span> <span class="o">!=</span> <span class="nb">len</span><span class="p">(</span><span class="nb">set</span><span class="p">(</span><span class="n">upload_ids</span><span class="p">)):</span>     <span class="c1"># 已过期，或不属于这个用户</span>
</span></span><span class="line"><span class="cl">        <span class="k">raise</span> <span class="n">UploadNotReadyError</span><span class="p">(</span><span class="n">upload_ids</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">for</span> <span class="n">upload</span> <span class="ow">in</span> <span class="n">uploads</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">        <span class="k">if</span> <span class="n">upload</span><span class="o">.</span><span class="n">processed_at</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>          <span class="c1"># webhook 还没到</span>
</span></span><span class="line"><span class="cl">            <span class="n">data</span> <span class="o">=</span> <span class="n">read_raw</span><span class="p">(</span><span class="n">upload</span><span class="p">)</span>              <span class="c1"># PUT 没落地就是 None</span>
</span></span><span class="line"><span class="cl">            <span class="k">if</span> <span class="n">data</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">                <span class="k">raise</span> <span class="n">UploadNotReadyError</span><span class="p">(</span><span class="n">upload</span><span class="o">.</span><span class="n">public_id</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">            <span class="n">process</span><span class="p">(</span><span class="n">upload</span><span class="p">,</span> <span class="n">data</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="n">uploads</span></span></span></code></pre></div></div>
<p>如果通知已经跑过了，处理好的文件就原样挂上去。如果文件在 bucket 里，通知却还没到，保存就当场把它处理掉，所以发帖从来不用等 webhook。通知是抢跑，不是依赖。如果文件根本不在，或者某个 id 已经过期、属于别人，保存会返回一个校验错误，应用稍等一下再重试；应用在发送前会先等自己的 PUT 完成，所以实际上这种情况很少见。</p>
<p>让这套东西站得住的几个细节。槽位属于某个用户，所以服务器只接受同一个人创建的上传 id，不然任何看到 id 的人都能把你的照片挂到他自己的帖子上。处处都有退路：申请槽位失败、PUT 被拒，或者 API 是旧版本，应用就照老办法发 base64。只有当每张新照片都有槽位时它才走上传这条路，所以一条帖子的图片要么全是上传、要么全是 base64，顺序始终是同一份列表。孤儿会过期：有些人选了照片又放弃了，所以一直没被挂上的上传两天后过期，现有的孤立媒体清理任务会把它的文件清掉。</p>

<h2 class="relative group">预签名 URL 上传的坑：CORS、大小限制、过期、多文件
    <div id="预签名-url-上传的坑cors大小限制过期多文件" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e9%a2%84%e7%ad%be%e5%90%8d-url-%e4%b8%8a%e4%bc%a0%e7%9a%84%e5%9d%91cors%e5%a4%a7%e5%b0%8f%e9%99%90%e5%88%b6%e8%bf%87%e6%9c%9f%e5%a4%9a%e6%96%87%e4%bb%b6" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>预签名上传出的问题，大多出在通知触发之前。下面这些，是我希望能在一页里看全的。</p>

<h3 class="relative group">CORS，只有浏览器需要
    <div id="cors只有浏览器需要" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#cors%e5%8f%aa%e6%9c%89%e6%b5%8f%e8%a7%88%e5%99%a8%e9%9c%80%e8%a6%81" aria-label="锚点">#</a>
    </span>
    
</h3>
<p>原生应用不在乎这个。浏览器在 PUT 之前会先向 bucket 发一个预检 <code>OPTIONS</code>，除非 bucket 的 CORS 规则允许来自你的源的 <code>PUT</code> 并带上 <code>Content-Type</code> 头，否则上传会以 CORS 错误告终。Tigris 用的是标准的 S3 CORS 配置，所以规则写法和 AWS 上一样。预签名 PUT 用 <code>curl</code> 能通、在页面上不行时，第一个要查的就是它。</p>
<p><figure><span class="mc-img"><img
    class="my-0 rounded-md"
    loading="lazy"
    decoding="async"
    fetchpriority="auto"
    alt="Tigris bucket 设置的 CORS and headers 标签页：一条规则，允许来自两个源的 PUT 并带上 Content-Type 头"
    width="1204"
    height="671"
    src="https://jared.lynskey.co.nz/en/posts/2026/tigris-object-notifications/tigris-bucket-cors_hu_7f61261034a8a56.webp"
    srcset="https://jared.lynskey.co.nz/en/posts/2026/tigris-object-notifications/tigris-bucket-cors_hu_7f61261034a8a56.webp 800w, https://jared.lynskey.co.nz/en/posts/2026/tigris-object-notifications/tigris-bucket-cors_hu_3aa47360232f6475.webp 1280w"
    sizes="(min-width: 768px) 50vw, 65vw"
    data-zoom-src="https://jared.lynskey.co.nz/en/posts/2026/tigris-object-notifications/tigris-bucket-cors_hu_3aa47360232f6475.webp"></span><figcaption>浏览器上传需要的 CORS 规则只有一条：你的源、PUT，以及签名所期望的 Content-Type 头。</figcaption></figure>
</p>

<h3 class="relative group">Content-Type 是签名的一部分
    <div id="content-type-是签名的一部分" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#content-type-%e6%98%af%e7%ad%be%e5%90%8d%e7%9a%84%e4%b8%80%e9%83%a8%e5%88%86" aria-label="锚点">#</a>
    </span>
    
</h3>
<p>签名时带上 <code>ContentType</code>，意味着客户端必须原样发送这个头，否则 bucket 会回 403 <code>SignatureDoesNotMatch</code>。这就是槽位把 headers 和 URL 一起返回的原因。要有意识地签上它，但也要清楚它能换来什么：被强制的是头，不是内容，所以一个标着 <code>image/jpeg</code> 的 PDF 照样能进来。真正的检查在处理流程里；Pillow 拒绝打开不是图片的东西，那一行也就永远不会被标记为已处理。</p>

<h3 class="relative group">大小限制
    <div id="大小限制" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e5%a4%a7%e5%b0%8f%e9%99%90%e5%88%b6" aria-label="锚点">#</a>
    </span>
    
</h3>
<p>预签名的 PUT URL 不带任何大小限制。如果你需要在字节落地之前就硬性拦住，用预签名 POST 配一个 <code>content-length-range</code> 策略条件可以做到。我没费这个劲，因为通知里带着对象的 <code>size</code>：handler 在读取之前先把超限的删掉，保存时那条路径最多只读上限加一个字节，够它判断就行。应用发的也就几百 KB；接近 15 MB 的东西，不会是我们的。</p>

<h3 class="relative group">过期
    <div id="过期" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e8%bf%87%e6%9c%9f" aria-label="锚点">#</a>
    </span>
    
</h3>
<p>这里有两个时钟。预签名 URL 的有效期是十五分钟，做一次 PUT 绰绰有余，又短到泄露了也很快就没用。槽位那一行的有效期是两天，足够覆盖有人选好照片、把手机一放、第二天再把帖子写完的情况。客户端把挂了一天的待上传视为过期，直接重新申请一个，而不是抱着侥幸。</p>

<h3 class="relative group">多个文件，以及更大的文件
    <div id="多个文件以及更大的文件" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e5%a4%9a%e4%b8%aa%e6%96%87%e4%bb%b6%e4%bb%a5%e5%8f%8a%e6%9b%b4%e5%a4%a7%e7%9a%84%e6%96%87%e4%bb%b6" aria-label="锚点">#</a>
    </span>
    
</h3>
<p>一个文件一个槽位。十张照片就是十个槽位、十个并行的 PUT，每一个都有自己的进度（<code>XMLHttpRequest</code> 有 <code>upload.onprogress</code>，<code>fetch</code> 没有），这是一个巨大的 base64 POST 给不了的。超过几 MB 就用分片上传（每个分片最小 5 MiB，所以它是给远大于这个数的文件准备的）：在服务器上创建 multipart upload，给每个 <code>upload_part</code> 预签名，分片都到齐后再完成它。上传完成时 Tigris 会发 <code>OBJECT_CREATED_MULTIPART</code>，同一个 handler 照样触发，其他什么都不用改。</p>

<h3 class="relative group">乱序
    <div id="乱序" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e4%b9%b1%e5%ba%8f" aria-label="锚点">#</a>
    </span>
    
</h3>
<p>事件可能乱序到达，因为对象可能写入了不止一个区域。对上传来说无所谓：一个 key，一个事件。对会被覆盖的 key，相信 eTag 和对象的 <code>Last-Modified</code>，别相信 webhook 到你这里的先后顺序。</p>

<h2 class="relative group">同一个信号用在日志上：Discord 告警
    <div id="同一个信号用在日志上discord-告警" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e5%90%8c%e4%b8%80%e4%b8%aa%e4%bf%a1%e5%8f%b7%e7%94%a8%e5%9c%a8%e6%97%a5%e5%bf%97%e4%b8%8adiscord-%e5%91%8a%e8%ad%a6" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>bucket 一旦能跟后端说话，任何落进去的东西都能用同样的方式响应，我把它指向的第二样东西就是日志。</p>
<p>应用的日志以 gzip 压缩的小文件形式送进 bucket，每个应用大约五分钟一个。今年大部分时间里，一个按 15 分钟 cron 跑的 GitHub Actions workflow 用 DuckDB 读归档里最近 20 分钟的数据，找出慢请求的行，再往 Discord 发一份汇总。纸面上没毛病。实际上，平台一忙，GitHub 的定时 workflow 就不会准点跑，于是&quot;每 15 分钟&quot;更像是&quot;每隔几个小时&quot;，等它终于跑起来，要啃的时间窗口变大了，偶尔还会内存不足。我为了不让数据在两次运行之间漏掉而加的重叠窗口，又让同一个慢请求有时出现两次。就为了&quot;请求慢了告诉我一声&quot;，搭了这么一大套机器。</p>
<p>现在每个日志文件在落地时读一次，只读一次。它们 gzip 之后也就几 KB。handler 把慢请求的行挑出来，如果数量够多，或者有一个特别慢的，就通过 webhook 往一个 Discord 频道发一份汇总。一个文件里突然冒出一堆 5xx 也同样处理。从 Python 调 Discord webhook 就是一次请求：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">requests</span>
</span></span><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">django.conf</span> <span class="kn">import</span> <span class="n">settings</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">discord_alert</span><span class="p">(</span><span class="n">text</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">    <span class="n">requests</span><span class="o">.</span><span class="n">post</span><span class="p">(</span>
</span></span><span class="line"><span class="cl">        <span class="n">settings</span><span class="o">.</span><span class="n">DISCORD_WEBHOOK_URL</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">        <span class="n">json</span><span class="o">=</span><span class="p">{</span><span class="s2">&#34;content&#34;</span><span class="p">:</span> <span class="n">text</span><span class="p">[:</span><span class="mi">1900</span><span class="p">]},</span>  <span class="c1"># Discord 单条消息上限 2000 字符</span>
</span></span><span class="line"><span class="cl">        <span class="n">timeout</span><span class="o">=</span><span class="mi">5</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="p">)</span></span></span></code></pre></div></div>
<p>没有 cron，没有重叠窗口，没有重复告警，而且告警会在请求发生后几分钟内到达，不用再等 GitHub 什么时候想起来。那个 Action 已经删了。</p>
<p>反正文件已经打开了，我顺便把里面的内容（错误、警告、健康检查）按小时计数，写进一张统计表。每日汇总以前要扫一整天的归档，现在只读几百行，只有统计出现缺口时才回头去扫归档。</p>

<h2 class="relative group">给删除加一道防线
    <div id="给删除加一道防线" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e7%bb%99%e5%88%a0%e9%99%a4%e5%8a%a0%e4%b8%80%e9%81%93%e9%98%b2%e7%ba%bf" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>这个是计划外的，有了 <code>OBJECT_DELETED</code> 事件之后自然就冒出来了。媒体文件被删除时，handler 会检查还有没有数据行指向它。如果有，那就是某处有 bug（或者有人在 bucket 控制台里手滑了），于是它清掉那一列，让应用显示占位图而不是一张裂开的图片，再把这事报告出来。</p>
<p>它还会按小时统计删除数量。正常的清理一小时也就删几个。要是哪天一小时里删了两三百个，我希望在事情发生的当下就知道，而不是等用户开始反馈照片变成空白。</p>

<h2 class="relative group">值得知道的几件事
    <div id="值得知道的几件事" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e5%80%bc%e5%be%97%e7%9f%a5%e9%81%93%e7%9a%84%e5%87%a0%e4%bb%b6%e4%ba%8b" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>响应要快。Tigris 给每次投递十秒。读一个 5 KB 的日志文件、处理一张手机照片都来得及；可能更久的活就丢进队列，view 直接返回 200，否则 Tigris 会超时重试，你又得回头去重。</p>
<p>在 bucket 那一层过滤。我的日志传送程序每个应用每隔几分钟就写一个文件，永不停歇。如果你只关心某一个前缀，就写进通知的过滤条件里，别让其他事情也把你的应用叫醒。</p>
<p>用假的请求体测试。payload 小而稳定，所以测试里直接带上正确的 <code>Authorization</code> 头，往 view 上 POST 一段 JSON，再断言 handler 做了什么。完全不用碰真实的存储。</p>
<p>别第一天就让它承重。这里的每一种用途都有后备方案：保存可以自己处理照片，每日汇总仍然可以扫描归档，应用仍然可以发 base64。通知让它们每一个都更快、更省，但哪个 webhook 丢了，也没有谁会因此倒下。</p>
<p>这周最爽的一刻，是删掉那个 workflow 文件。四十行 YAML、一个 Python 脚本、两个仓库 secret，外加隔三差五的一句&quot;告警怎么没响&quot;，全都没了，取而代之的是一个 handler：文件到了，读一个小文件，完事。不过重点还是上传。照片以网络允许的最快速度传上去，服务器对每一张仍然握有最终决定权，而且没有人需要在 bucket 和后端之间搭一座桥，因为 bucket 现在会自己说发生了什么。</p>
]]></content:encoded>
      <media:content url="https://jared.lynskey.co.nz/en/posts/2026/tigris-object-notifications/featured.jpg" medium="image" type="image/jpeg" />
    </item>
  </channel>
</rss>
