<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:media="http://search.yahoo.com/mrss/">
  <channel>
    <title>Privacy · 仁才德</title>
    <link>https://jared.lynskey.co.nz/zh-cn/tags/privacy/</link>
    <description>关于软件工程与研发团队领导力的短文——绩效评估、招聘与团队扩张、CI/CD、DevOps 监控、智能体 AI。写于首尔。</description>
    <generator>Hugo</generator>
    <language>zh-CN</language>
    <copyright>© 2026 仁才德</copyright>
    <lastBuildDate>Sat, 07 Nov 2026 00:00:00 +1300</lastBuildDate>
    <atom:link href="https://jared.lynskey.co.nz/zh-cn/tags/privacy/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>不自建 Sentry，也能收 React Native 崩溃报告：保留 SDK，接自己的服务器</title>
      <link>https://jared.lynskey.co.nz/zh-cn/posts/2026/2026-10-09-sentry-sdk-own-server/</link>
      <guid isPermaLink="true">https://jared.lynskey.co.nz/zh-cn/posts/2026/2026-10-09-sentry-sdk-own-server/</guid>
      <pubDate>Fri, 09 Oct 2026 00:00:00 +1300</pubDate>
      <dc:creator>Jared Lynskey</dc:creator>
      <category>React Native</category>
      <category>Expo</category>
      <category>Sentry</category>
      <category>Django</category>
      <category>Crash Reporting</category>
      <category>Self-Hosted</category>
      <category>iOS</category>
      <category>Android</category>
      <category>Privacy</category>
      <description>不用自建 Sentry，也不用付 sentry.io 的钱：保留 Sentry 的开源 SDK，把 DSN 指向自己 Django API 里的接口，自己接收 React Native 应用的崩溃报告，并做分组、报警和脱敏。</description>
      <content:encoded><![CDATA[<p>如果你想自己接收 React Native 应用的崩溃报告，其实不必自建 Sentry 服务端，也不必给 sentry.io 付费。Sentry 的 SDK 是开源的，DSN 可以指向任何主机。我把它指向自己 Django API 里的一个接口，在那里接收、分组、报警和保存，除了本来就在跑的 API，没有额外成本。目前还没替代掉的是符号化，所以堆栈还对应不到源码的具体行。</p>
<p>直到这周之前，我知道 我们的应用崩了的唯一途径，是有人来告诉我。大多数人不会说。他们重新打开应用，如果又崩了，就不再打开了。API 那边是有覆盖的（所有 ERROR 日志都会发到团队聊天的一个频道），但手机上的一次崩溃，在我能看到的地方不留任何痕迹。</p>
<p>其实应用里已经有一个上报机制。应用每次把错误信息展示给用户时，都会把这条信息 POST 到 API 上的一个小接口，服务端以 ERROR 级别记录，所以同样会进聊天频道。加这个是因为有一次删除操作其实 <em>成功了</em> ，可那一行还留在列表里，再点一下就弹出 &ldquo;No Item matches the given query&rdquo;。服务端只记了两条普通的 404，什么也没说。它覆盖的是应用捕获并展示出来的错误。崩溃不一样：本该发送报告的那段 JavaScript，恰恰就是刚刚死掉的那个。</p>
<p>最直接的办法是用 Sentry。我很喜欢 Sentry 的 SDK，React Native 上的崩溃上报之所以算是已解决的问题，靠的就是它。我不想要的是：又多一家供应商持有用户数据，隐私政策里又多一行，又多一张账单，而以我们的规模，这东西一天顶多几条报告。所以我留下 SDK，换掉服务：应用的 DSN 指向我们自己的 API，由那边 Django 应用里的一小块代码负责接收、分组、报警和保存。</p>
<p>这篇文章讲它是怎么工作的，以及我怎么确认一次真实的原生崩溃确实能送到。</p>

<h2 class="relative group">DSN 的小把戏
    <div id="dsn-的小把戏" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#dsn-%e7%9a%84%e5%b0%8f%e6%8a%8a%e6%88%8f" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>Sentry 的 DSN 长这样：<code>https://&lt;public-key&gt;@&lt;host&gt;/&lt;project-id&gt;</code>。SDK 会把它变成一个发往 <code>https://&lt;host&gt;/api/&lt;project-id&gt;/envelope/</code> 的 POST 请求，key 放在 <code>X-Sentry-Auth</code> 头里。这里面没有任何东西是 sentry.io 专属的。所以打包进应用的 DSN 是：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">https://&lt;CRASH_REPORT_KEY&gt;<span>@</span>api.example.com/1</span></span></code></pre></div></div>
<p>Django 里加一条路由，正好匹配 SDK 会调用的地址：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="n">path</span><span class="p">(</span><span class="s2">&#34;api/&lt;int:project_id&gt;/envelope/&#34;</span><span class="p">,</span> <span class="n">crash_envelope</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="s2">&#34;crash_envelope&#34;</span><span class="p">),</span></span></span></code></pre></div></div>
<p>这个 key 存在于每一个安装包里，所以它不是秘密。它的作用是：万一这个接口开始招来垃圾请求，我可以换掉它；另外服务端没配置 key 时会拒绝一切请求，而不是来者不拒：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">crash_envelope</span><span class="p">(</span><span class="n">request</span><span class="p">,</span> <span class="n">project_id</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">    <span class="n">expected</span> <span class="o">=</span> <span class="n">settings</span><span class="o">.</span><span class="n">CRASH_REPORT_KEY</span>
</span></span><span class="line"><span class="cl">    <span class="k">if</span> <span class="ow">not</span> <span class="n">expected</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">        <span class="k">return</span> <span class="n">Response</span><span class="p">({</span><span class="s2">&#34;detail&#34;</span><span class="p">:</span> <span class="s2">&#34;Crash reporting is not configured&#34;</span><span class="p">},</span> <span class="n">status</span><span class="o">=</span><span class="mi">503</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">if</span> <span class="ow">not</span> <span class="n">hmac</span><span class="o">.</span><span class="n">compare_digest</span><span class="p">(</span><span class="n">_key</span><span class="p">(</span><span class="n">request</span><span class="p">)</span><span class="o">.</span><span class="n">encode</span><span class="p">(),</span> <span class="n">expected</span><span class="o">.</span><span class="n">encode</span><span class="p">()):</span>
</span></span><span class="line"><span class="cl">        <span class="k">return</span> <span class="n">Response</span><span class="p">({</span><span class="s2">&#34;detail&#34;</span><span class="p">:</span> <span class="s2">&#34;Unknown key&#34;</span><span class="p">},</span> <span class="n">status</span><span class="o">=</span><span class="mi">401</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="o">...</span></span></span></code></pre></div></div>
<p><code>_key</code> 从 <code>X-Sentry-Auth</code> 头里读出 <code>sentry_key=</code>，读不到就退回到查询字符串，有些 SDK 是放在那里的。</p>

<h2 class="relative group">SDK 发的是什么：envelope
    <div id="sdk-发的是什么envelope" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#sdk-%e5%8f%91%e7%9a%84%e6%98%af%e4%bb%80%e4%b9%88envelope" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>Sentry 的传输格式叫 <a href="https://develop.sentry.dev/sdk/foundations/envelopes/"  target="_blank" rel="noreferrer">envelope</a>：先是一行 JSON 头，后面跟任意多个 item，每个 item 是一行 JSON 头加一段 payload。如果 item 头里有 <code>length</code>，payload 就恰好是这么多字节；否则一直读到下一个换行符。</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">{&#34;event_id&#34;:&#34;9ec79c33...&#34;,&#34;dsn&#34;:&#34;https://...<span>@</span>api.example.com/1&#34;}
</span></span><span class="line"><span class="cl">{&#34;type&#34;:&#34;event&#34;,&#34;length&#34;:4123}
</span></span><span class="line"><span class="cl">{ ...the event JSON, 4123 bytes... }
</span></span><span class="line"><span class="cl">{&#34;type&#34;:&#34;client_report&#34;}
</span></span><span class="line"><span class="cl">{&#34;timestamp&#34;:1791400000.5,&#34;discarded_events&#34;:[]}</span></span></code></pre></div></div>
<p>有两个细节让我栽了跟头：</p>
<ul>
<li><strong>原生 SDK 会 gzip 请求体，JavaScript 的不会。</strong> <code>@sentry/react-native</code> 捕获的 JS 错误是明文发来的，而 sentry-cocoa 或 sentry-android 发来的原生崩溃带着 <code>Content-Encoding: gzip</code>。解码器能处理 gzip、zlib 包装的 deflate 和裸 deflate，解压到 1MB 就停，因为一个崩溃事件只有几 KB，再大就不是崩溃事件了。</li>
<li><strong>带长度前缀的 payload 里可以有换行符。</strong> 如果按 <code>\n</code> 切分，原生 SDK 发来的每个事件都会被切坏。有 <code>length</code> 的时候必须按它来读。</li>
</ul>
<p>解析器很短。它只保留 <code>event</code> 类型的 item；session、transaction、附件和 client report 照单全收，然后丢弃。</p>

<h2 class="relative group">用 SDK 听得懂的方式回应
    <div id="用-sdk-听得懂的方式回应" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e7%94%a8-sdk-%e5%90%ac%e5%be%97%e6%87%82%e7%9a%84%e6%96%b9%e5%bc%8f%e5%9b%9e%e5%ba%94" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>这一点比我预想的重要。SDK 在磁盘上维护一个队列，凡是不确定送达的都会重试，所以服务端怎么回应，决定了它会不会一直重试下去：</p>
<ul>
<li>存好了，<em>或者是故意丢弃的</em>，都回 <strong>200</strong>。如果我对 session item 回一个 4xx，SDK 就会留着它，永远重试。</li>
<li>被限流时回 <strong>429 加 <code>Retry-After</code></strong>。这个接口按 IP 限流（每小时 120 次），因为陷入崩溃循环的手机每次启动都会发一条报告，而 SDK 收到 429 会像你期望的那样退避。</li>
<li>envelope 格式不对回 <strong>400</strong>，记成 warning 而不是 error：一份坏掉的报告只说明某个客户端坏了，下一份正常的报告会告诉我崩在哪。</li>
</ul>
<p>SDK 如果不确定第一次是否送达，还会重发同一个事件，所以 <code>event_id</code> 上有唯一约束，重复的就悄悄跳过。</p>

<h2 class="relative group">没有 source map 时怎么分组
    <div id="没有-source-map-时怎么分组" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e6%b2%a1%e6%9c%89-source-map-%e6%97%b6%e6%80%8e%e4%b9%88%e5%88%86%e7%bb%84" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>自己写接收端，就得做一个 Sentry 替你做了的选择。Sentry 按堆栈给崩溃分组。但 release 版的 Hermes bundle 是压缩过的，函数名都是单个字母，行列号每次构建都会变。按堆栈帧分组，同一个 bug 每次发版都会开一个新 issue。</p>
<p>没有 source map（文末再说），JavaScript 错误里最稳定的就是它的类型和信息。所以指纹由异常类型加信息组成，并把每次出现都不一样的部分遮掉：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="n">_MASKS</span> <span class="o">=</span> <span class="p">[</span>
</span></span><span class="line"><span class="cl">    <span class="p">(</span><span class="n">re</span><span class="o">.</span><span class="n">compile</span><span class="p">(</span><span class="sa">r</span><span class="s2">&#34;0x[0-9a-fA-F]+&#34;</span><span class="p">),</span> <span class="s2">&#34;&lt;addr&gt;&#34;</span><span class="p">),</span>
</span></span><span class="line"><span class="cl">    <span class="p">(</span><span class="n">re</span><span class="o">.</span><span class="n">compile</span><span class="p">(</span><span class="sa">r</span><span class="s2">&#34;\b[0-9a-fA-F]</span><span class="si">{8}</span><span class="s2">-[0-9a-fA-F]</span><span class="si">{4}</span><span class="s2">-[0-9a-fA-F]</span><span class="si">{4}</span><span class="s2">-[0-9a-fA-F]</span><span class="si">{4}</span><span class="s2">-[0-9a-fA-F]</span><span class="si">{12}</span><span class="s2">\b&#34;</span><span class="p">),</span> <span class="s2">&#34;&lt;uuid&gt;&#34;</span><span class="p">),</span>
</span></span><span class="line"><span class="cl">    <span class="p">(</span><span class="n">re</span><span class="o">.</span><span class="n">compile</span><span class="p">(</span><span class="sa">r</span><span class="s2">&#34;\b[0-9a-fA-F]{16,}\b&#34;</span><span class="p">),</span> <span class="s2">&#34;&lt;hex&gt;&#34;</span><span class="p">),</span>
</span></span><span class="line"><span class="cl">    <span class="p">(</span><span class="n">re</span><span class="o">.</span><span class="n">compile</span><span class="p">(</span><span class="sa">r</span><span class="s2">&#34;\d+&#34;</span><span class="p">),</span> <span class="s2">&#34;&lt;n&gt;&#34;</span><span class="p">),</span>
</span></span><span class="line"><span class="cl"><span class="p">]</span></span></span></code></pre></div></div>
<p><code>Cannot read property 'title' of undefined (reading index 3)</code> 和 <code>(reading index 17)</code> 会归为同一个 issue。如果 SDK 自己带了 <code>fingerprint</code> 且其中没有 <code>{{ default }}</code>，就以它为准，这样默认分组不对时我可以从客户端强制指定。没有异常的事件退而使用遮罩后的日志信息，两者都没有就用最上面的应用内堆栈帧。原生信号崩溃的信息大概是 <code>Signal 6, Code 0</code>，所以这类崩溃按信号分组，粒度很粗，但在有符号化之前，这样是诚实的。</p>

<h2 class="relative group">报警，但不刷屏
    <div id="报警但不刷屏" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e6%8a%a5%e8%ad%a6%e4%bd%86%e4%b8%8d%e5%88%b7%e5%b1%8f" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>每个指纹对应一个 issue，每份报告作为一个事件存在它下面。issue 只在以下几种情况下以 ERROR 级别记录（聊天报警监听的就是这个级别）：</p>
<ul>
<li><strong>新出现</strong>的 issue</li>
<li><strong>已解决</strong>的 issue <strong>又出现</strong>，此时会重新打开它</li>
<li>第 <strong>10、100、1,000、10,000</strong> 次出现</li>
</ul>
<p>普通的重复只会让计数器加一。标记为 <em>忽略</em> 的 issue 照样计数，但从不报警。再加上聊天 handler 自己的限流，坏版本上的崩溃循环在频道里只是几条消息，而不是一面墙。</p>
<p>存储也一样有上限：每个 issue 保留 <strong>最近 50 条</strong> 事件，新的进来就删掉旧的。这让第一版图表出了问题，因为 50 条事件说不出上周某件事发生了多少次。所以每个 issue 还另外记着 90 天的每日计数，和计数器加一放在同一个事务里更新。图表从这些计数取数；事件只用来阅读。</p>
<p>如果你也自己做一个，有两个小的竞态值得处理。一个全新的崩溃同时打到几台手机上时，两份报告可能都想创建这个 issue，所以给指纹加唯一约束，让输的那个重新读取已有的那一行。另外，90 天的窗口要从服务器的&quot;今天&quot;算起，而不是从最新一份报告算起，因为时钟设错的手机可能把报告的日期标成任何一天，包括明年。</p>
<p>payload 上限是 128KB。超出的事件先丢 breadcrumbs 和调试元数据；还超，就只保留异常、信息、版本和平台，并做个标记，让我知道它被截过。</p>

<h2 class="relative group">脱敏
    <div id="脱敏" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e8%84%b1%e6%95%8f" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>SDK 开着 <code>sendDefaultPii: false</code>，所以理论上不会发送 IP 地址、cookie 或请求体。服务端并不依赖这一点：存储之前，<code>user</code> 里只留 <code>id</code> 和 <code>username</code>，<code>request</code> 整个删掉，以防以后某个 SDK 版本或设置开始发送这些东西。</p>
<p>用真机测试又发现了两样需要去掉的东西：</p>
<ul>
<li><strong>breadcrumbs 记录的是完整 URL，包括查询字符串。</strong> 应用的搜索请求会把搜索词放在查询字符串里，所以搜索之后紧跟着的一次崩溃，会把用户搜了什么存下来。现在服务端会把每个 breadcrumb 的 URL 在 <code>?</code> 处截断。</li>
<li><strong>console 类 breadcrumb 带着传给 <code>console.log</code> 的原始参数</strong>，可能是应用恰好打印出来的任何对象。消息文本保留（它有用），参数丢掉。</li>
</ul>
<p>这两样都是看了一份真实报告才发现的，这也是下一节存在的主要理由。</p>

<h2 class="relative group">客户端
    <div id="客户端" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e5%ae%a2%e6%88%b7%e7%ab%af" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>应用里所有相关代码都走一个小模块，原因和我把 OTA 更新 SDK 包一层一样：OTA bundle 也会跑在原生模块还不存在时构建的安装包上。在没有 <code>RNSentry</code> 的安装包上，顶层写 <code>import * as Sentry from &quot;@sentry/react-native&quot;</code> 会在导入时就抛异常，结果就是白屏。所以 SDK 是延迟 require 的，而且要先确认原生模块真的存在：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-ts" data-lang="ts"><span class="line"><span class="cl"><span class="kr">import</span> <span class="p">{</span> <span class="nx">NativeModules</span><span class="p">,</span> <span class="nx">Platform</span><span class="p">,</span> <span class="nx">TurboModuleRegistry</span> <span class="p">}</span> <span class="kr">from</span> <span class="s2">&#34;react-native&#34;</span><span class="p">;</span>
</span></span><span class="line"><span class="cl"><span class="kr">import</span> <span class="nx">Constants</span> <span class="kr">from</span> <span class="s2">&#34;expo-constants&#34;</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">DSN</span> <span class="o">=</span> <span class="nx">process</span><span class="p">.</span><span class="nx">env</span><span class="p">.</span><span class="nx">EXPO_PUBLIC_CRASH_DSN</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="kd">function</span> <span class="nx">nativeModulePresent</span><span class="p">()</span><span class="o">:</span> <span class="kr">boolean</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">  <span class="k">try</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="nb">Boolean</span><span class="p">(</span><span class="nx">TurboModuleRegistry</span><span class="p">.</span><span class="kr">get</span><span class="p">(</span><span class="s2">&#34;RNSentry&#34;</span><span class="p">)</span> <span class="o">??</span> <span class="nx">NativeModules</span><span class="p">.</span><span class="nx">RNSentry</span><span class="p">);</span>
</span></span><span class="line"><span class="cl">  <span class="p">}</span> <span class="k">catch</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="kc">false</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">  <span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="kr">export</span> <span class="kd">function</span> <span class="nx">initCrashReporting</span><span class="p">()</span><span class="o">:</span> <span class="kr">boolean</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">  <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">DSN</span> <span class="o">||</span> <span class="nx">Platform</span><span class="p">.</span><span class="nx">OS</span> <span class="o">===</span> <span class="s2">&#34;web&#34;</span> <span class="o">||</span> <span class="o">!</span><span class="nx">nativeModulePresent</span><span class="p">())</span> <span class="k">return</span> <span class="kc">false</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">  <span class="kr">const</span> <span class="nx">Sentry</span> <span class="o">=</span> <span class="kr">require</span><span class="p">(</span><span class="s2">&#34;@sentry/react-native&#34;</span><span class="p">);</span>
</span></span><span class="line"><span class="cl">  <span class="nx">Sentry</span><span class="p">.</span><span class="nx">init</span><span class="p">({</span>
</span></span><span class="line"><span class="cl">    <span class="nx">dsn</span>: <span class="kt">DSN</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="nx">release</span>: <span class="kt">Constants.expoConfig?.version</span> <span class="o">??</span> <span class="s2">&#34;unknown&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="nx">sendDefaultPii</span>: <span class="kt">false</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="nx">tracesSampleRate</span>: <span class="kt">0</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="nx">enableAutoSessionTracking</span>: <span class="kt">false</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="c1">// ...turn off the other tracing options the same way
</span></span></span><span class="line"><span class="cl">    <span class="nx">maxBreadcrumbs</span>: <span class="kt">50</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="nx">beforeSend</span><span class="p">(</span><span class="nx">event</span><span class="p">)</span> <span class="p">{</span> <span class="cm">/* tag the OTA bundle */</span> <span class="k">return</span> <span class="nx">event</span><span class="p">;</span> <span class="p">},</span>
</span></span><span class="line"><span class="cl">  <span class="p">});</span>
</span></span><span class="line"><span class="cl">  <span class="k">return</span> <span class="kc">true</span><span class="p">;</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span></span></span></code></pre></div></div>
<p>只上报错误。服务端不存 trace、session、profile 或 replay，所以这些一个都没开。没有 DSN（开发环境、E2E 构建）时 SDK 根本不会启动。构建 profile 会设置 DSN 和环境（<code>staging</code> 或 <code>production</code>），两类构建上报到同一个接口，又能区分开。登录用户用账号 ID 标识，绝不用名字或邮箱。</p>
<p>错误边界需要显式调用。Expo Router 的根 <code>ErrorBoundary</code> 会捕获渲染错误并显示自己的界面，所以这个错误根本到不了 SDK 挂钩的全局处理器。解决办法是一个小包装组件：在 <code>useEffect</code> 里调用 <code>captureException</code>，然后照旧渲染路由自带的 boundary，根 boundary 和每个路由的 boundary 都用它。放在 effect 里而不是渲染过程中，是为了只上报一次，而不是每次重新渲染都报。根组件还用 <code>Sentry.wrap</code> 包了一层，如果这一步抛异常，就退回到原始组件。</p>
<p>Web 端有一个 <code>.web.ts</code> 版本的模块，所有函数都是空操作。Metro 会选平台专属的文件，所以 SDK 根本不会进入 web bundle。Web 应用也没有原生模块可以运行它。</p>

<h2 class="relative group">证明原生崩溃能送到
    <div id="证明原生崩溃能送到" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e8%af%81%e6%98%8e%e5%8e%9f%e7%94%9f%e5%b4%a9%e6%ba%83%e8%83%bd%e9%80%81%e5%88%b0" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>单元测试 POST 的是手写的 envelope，只能证明解析器能处理我 <em>以为</em> SDK 会发的东西。发布两天后，生产环境一份崩溃报告都没有。当时只有三个人更新了，所以零是说得通的，但从服务端看，&ldquo;没人崩溃&quot;和&quot;崩溃送不到&quot;长得一模一样。我想亲眼看到一份真的。</p>
<p>于是我构建了一个 Release 版的模拟器包，把 DSN 指向本地 8000 端口上的 Django，用一个临时 key：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="nv">EXPO_PUBLIC_CRASH_DSN</span><span class="o">=</span>http://localcrashtestkey0000@localhost:8000/1 <span class="se">\
</span></span></span><span class="line"><span class="cl"><span class="nv">EXPO_PUBLIC_CRASH_ENVIRONMENT</span><span class="o">=</span>local-verify <span class="se">\
</span></span></span><span class="line"><span class="cl">xcodebuild -workspace ios/MyApp.xcworkspace -scheme MyApp -configuration Release <span class="se">\
</span></span></span><span class="line"><span class="cl">  -destination <span class="s1">&#39;platform=iOS Simulator,id=&lt;udid&gt;&#39;</span> -derivedDataPath /tmp/iosbuild</span></span></code></pre></div></div>
<p>（用 Release 而不是 Debug：Debug 构建会链接 dev launcher，行为不一样，而用户跑的是 Release。）然后安装、启动，等几秒钟，再像真实故障那样把它杀掉：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">xcrun simctl install booted /tmp/iosbuild/Build/Products/Release-iphonesimulator/MyApp.app
</span></span><span class="line"><span class="cl">xcrun simctl launch booted com.example.myapp      <span class="c1"># prints the pid</span>
</span></span><span class="line"><span class="cl"><span class="nb">kill</span> -ABRT &lt;pid&gt;</span></span></code></pre></div></div>
<p>模拟器里的应用就是普通的 macOS 进程，所以信号到达 SentryCrash 处理器的方式和真机上完全一样。处理器把报告写到磁盘；进程临死时什么都不发送。<strong>下次启动</strong>时 SDK 找到它并发出去。我重新启动应用，一秒后模拟器的网络日志里就出现了一条到 <code>:8000</code> 的连接。</p>
<p>然后 Django 日志里什么都没有，这让我愣了一会儿：在我设的日志级别下，开发服务器只记录非 2xx 的响应。数据库里是有的：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">SIGABRT: Signal 6, Code 0 | cocoa 1.4.0 iOS 27.1 iPhone19,4 handled=False
</span></span><span class="line"><span class="cl">received 09:19:53, crashed 09:19:38
</span></span><span class="line"><span class="cl">contexts: app, device, os, ota_updates, trace · 24 breadcrumbs</span></span></code></pre></div></div>
<p>从崩溃到入库十五秒，大部分时间花在我手动重启上。而看真实的 payload，教会了我测试教不了的东西：</p>
<ul>
<li><strong>用户 ID 不是账号 ID。</strong> 测试包并没有真正登录，所以 sentry-cocoa 退而使用它自己的按安装生成的 ID。这一点值得知道，也值得写进隐私说明：登录时报告带的是账号 ID，未登录时是一个随机的安装 ID。</li>
<li><strong>没有时区。</strong> 我原以为 culture context 会有。device context 里有 <code>locale</code>、内存、屏幕尺寸、低电量模式和内核版本，就是没有时区。</li>
<li><strong>原生崩溃不经过 <code>beforeSend</code>。</strong> 我在 JS 的 <code>beforeSend</code> 里加了一个 <code>ota</code> 标签，标明正在运行的 OTA bundle，但原生崩溃是下次启动时由原生 SDK 组装的，根本不经过 JS 钩子。要给原生崩溃打标签，得在 bundle 确定后用 <code>setTag</code> 设到 scope 上，这样它才会同步到原生层。</li>
<li><strong>breadcrumbs 暴露了应用在做什么。</strong> console 警告、view controller 生命周期，还有 HTTP 请求，其中包括分析 SDK 自己发往它的数据收集接口的请求，这也提醒了我应用还在跟谁通信。而且这些 URL 带着查询字符串，前面说的搜索词问题就是这么发现的。</li>
</ul>

<h2 class="relative group">怎么看这些报告
    <div id="怎么看这些报告" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e6%80%8e%e4%b9%88%e7%9c%8b%e8%bf%99%e4%ba%9b%e6%8a%a5%e5%91%8a" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>报告在 Django 后台里看，页面大致仿照 Sentry 的 issue 页：顶部是计数和 Resolve / Ignore / Reopen 按钮，然后是每日柱状图、按版本、系统、设备和 OTA bundle 的分布，再往下一次展示一个事件，包括异常、堆栈帧（应用帧高亮，库的帧折叠起来）、breadcrumbs 和设备信息。不需要任何新依赖。</p>
<p>做这个页面时顺手修了一个问题：原生崩溃的&quot;culprit&quot;帧原本是二进制文件在设备上的路径，里面带着每次安装都不同的 UUID，结果同一个崩溃在每台手机上看起来都不一样。现在只显示地址和二进制名，比如 <code>0x1000a1b2c (MyApp)</code>。</p>

<h2 class="relative group">隐私政策，整份重写
    <div id="隐私政策整份重写" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e9%9a%90%e7%a7%81%e6%94%bf%e7%ad%96%e6%95%b4%e4%bb%bd%e9%87%8d%e5%86%99" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>崩溃报告差不多就是个人信息：一个账号 ID、一个手机型号，以及某人在应用崩溃前一分钟做了什么。所以隐私政策需要加一段。要写好，就得准确说明一份报告里有什么，于是我照着刚看过的 payload 列了出来：错误本身、应用版本和 OTA 更新、手机型号、系统和语言、可用内存和屏幕尺寸、最近几十步操作（打开的页面、去掉 <code>?</code> 之后部分的网址、应用自己的日志），以及账号 ID 或随机的安装 ID。政策里也写明了报告直接发到我们自己的服务器，不会发给 Sentry 或任何其他崩溃上报公司。保存期限在表格里单独占一行：每个问题最近 50 份报告，外加 90 天的每日计数。</p>
<p>一旦开始拿代码核对其中一段，就很难不去核对其余部分。原来的政策经不起核对。删除账号页面写着：关闭账号时，推送通知的 token 和 passkey 会立即删除，服务器日志会在 90 天内清除。这些都不是真的。关闭账号会解除登录方式的绑定，并把资料匿名化，但推送 token 和 passkey 会一直留着，直到有人手动删除；日志归档目前也没有过期时间。所以整份政策根据代码实际做了什么的清单重写了一遍，那几行现在写的是&quot;暂未自动删除，请发邮件给我们&rdquo;。听起来没那么让人安心，但它是准确的。</p>
<p>现在代码仓库里有一条规则：政策描述的是代码做了什么，而不是我们打算做什么；任何在新地方收集、发送或保存个人数据的改动，都要在同一个 commit 里更新它。这包括应用里的摘要页面、删除账号页面，以及只存在于商店后台的应用商店隐私表单。正是崩溃上报这个改动，让我把这条规则写了下来。</p>

<h2 class="relative group">还缺什么
    <div id="还缺什么" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e8%bf%98%e7%bc%ba%e4%bb%80%e4%b9%88" aria-label="锚点">#</a>
    </span>
    
</h2>
<p><strong>符号化。</strong> 这是真正的缺口，也是 sentry.io 能做而我还没替代的那件事。JavaScript 的堆栈是压缩过的，原生代码的堆栈没有符号化：</p>
<ul>
<li><strong>JavaScript：</strong> 堆栈帧指向 Hermes bundle 内部。要解决，就得为每个商店构建和每个 OTA bundle 保留合成后的 Hermes source map，按版本和 bundle 索引，在页面渲染时套用到 JS 帧上。这是下一步要做的，因为我们的崩溃大部分会是 JavaScript 的。</li>
<li><strong>iOS 原生：</strong> 指令地址需要该次构建的 dSYM。</li>
<li><strong>Android：</strong> 开启压缩时 Java 帧会被 R8 混淆（需要 <code>mapping.txt</code>），C++ 帧需要 NDK 符号。</li>
</ul>
<p>前面还发现了一个小一点的缺口：原生崩溃目前还不带 OTA 标签，因为它只在 <code>beforeSend</code> 里设置。在运行中的 bundle 确认之后把它设到 scope 上，只要几行代码，我还没写。</p>
<p>目前，一个 issue 能告诉我 <em>什么</em> 崩了、在哪个版本、哪个 OTA bundle、哪台设备上，以及用户崩溃前在做什么。它没法告诉我崩在源码的 <em>哪里</em> 。对一个一周发几次版的小应用来说，到目前为止，异常类型、信息和 breadcrumbs 已经足够找到问题了。</p>
<p>接收端、后台页面和测试加起来代码量不大，在原本就在运行的 API 之外不额外花一分钱。难的部分由 SDK 完成：在手机上捕获崩溃，并在下次启动时把报告送出来。</p>
]]></content:encoded>
      <media:content url="https://jared.lynskey.co.nz/en/posts/2026/2026-10-09-sentry-sdk-own-server/featured.jpg" medium="image" type="image/jpeg" />
    </item>
    <item>
      <title>React Native 端侧大模型：用 llama.rn 和 llama.cpp 在手机上离线运行 Qwen</title>
      <link>https://jared.lynskey.co.nz/zh-cn/posts/2026/2026-01-30-on-device-ai/</link>
      <guid isPermaLink="true">https://jared.lynskey.co.nz/zh-cn/posts/2026/2026-01-30-on-device-ai/</guid>
      <pubDate>Fri, 30 Jan 2026 00:00:00 +0000</pubDate>
      <dc:creator>Jared Lynskey</dc:creator>
      <category>React Native</category>
      <category>AI</category>
      <category>LLM</category>
      <category>llama.cpp</category>
      <category>Qwen</category>
      <category>On-Device AI</category>
      <category>Privacy</category>
      <category>Mobile Development</category>
      <description>我怎么在 React Native 和 Expo 应用里把一个小型 LLM 完全放在手机上离线运行：llama.rn、Q4_K_M 量化的 Qwen GGUF 模型、流式输出、小模型也能应付的工具调用，以及 iPhone 和 Galaxy S23 上的实测数据。</description>
      <content:encoded><![CDATA[<p>我做的一个报价应用里有个 AI 助手，完全跑在手机上。数据一个字节都不出设备。用户用自然语言就能创建工作单、搜索客户、管理项目，开了飞行模式照样能用。</p>
<p>这篇文章讲讲我是怎么用 <strong>llama.rn</strong>（llama.cpp 的 React Native 绑定）和 <strong>Qwen 1.7B</strong> 把它做出来的。这个尺寸的模型，能干的活比我预想的多得多。</p>
<p>简单来说：装上 llama.rn，应用第一次打开时下载一个 4 位量化的 GGUF 模型（1 GB 左右），用 <code>initLlama</code> 加载，然后流式生成就行。在 iPhone 14 Pro 上，完全不联网也能跑到每秒 15 个 token 左右。这篇剩下的内容，讲的是让它真正派上用场花了哪些功夫：选模型，让它稳定地吐出工具调用，还有把内存控制住。</p>
<blockquote><p><strong>📝 更新：</strong> 现在的实现已经换成 <a href="https://huggingface.co/unsloth/Qwen3.5-2B-GGUF"  target="_blank" rel="noreferrer">Qwen3.5-2B-GGUF</a>，替代了最初的 Qwen3-1.7B。另外我也不再用 <code>/no_think</code> 消息这种土办法，改用 <code>chat_template_kwargs</code> 参数正式关闭思考模式：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="n">chat_response</span> <span class="o">=</span> <span class="n">client</span><span class="o">.</span><span class="n">chat</span><span class="o">.</span><span class="n">completions</span><span class="o">.</span><span class="n">create</span><span class="p">(</span>
</span></span><span class="line"><span class="cl">    <span class="n">model</span><span class="o">=</span><span class="s2">&#34;Qwen/Qwen3.5-27B&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="n">messages</span><span class="o">=</span><span class="n">messages</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="n">max_tokens</span><span class="o">=</span><span class="mi">32768</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="n">temperature</span><span class="o">=</span><span class="mf">0.7</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="n">top_p</span><span class="o">=</span><span class="mf">0.8</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="n">presence_penalty</span><span class="o">=</span><span class="mf">1.5</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="n">extra_body</span><span class="o">=</span><span class="p">{</span>
</span></span><span class="line"><span class="cl">        <span class="s2">&#34;top_k&#34;</span><span class="p">:</span> <span class="mi">20</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">        <span class="s2">&#34;chat_template_kwargs&#34;</span><span class="p">:</span> <span class="p">{</span><span class="s2">&#34;enable_thinking&#34;</span><span class="p">:</span> <span class="kc">False</span><span class="p">},</span>
</span></span><span class="line"><span class="cl">    <span class="p">},</span>
</span></span><span class="line"><span class="cl"><span class="p">)</span></span></span></code></pre></div></div>
<p>比在提示词里塞开关干净多了。</p>
</blockquote><figure><div class="film-pair">
  <div class="mc-film film-dark">
  <video
    class="my-0 rounded-md"
    controls playsinline preload="none"
    poster="https://jared.lynskey.co.nz/en/posts/2026/2026-01-30-on-device-ai/on-device-ai-film-poster.jpg" style="background:url('data:image/webp;base64,UklGRnAAAABXRUJQVlA4IGQAAACwBACdASogABIAPw1uqUemIyIYBABjEMT0gAFRZzp3IyiSND5CxppNutGcCYkAAP7ixmKvS+0H/fAThx4cbifiH8PzygNYbl6sJuQFubAi+gzkxN3eXPGHfroT5K82AG+x8AAA') center/cover no-repeat"
    width="1280"
    height="720"
    aria-label="讲述这篇文章的一段短片">
    <source src="https://jared.lynskey.co.nz/en/posts/2026/2026-01-30-on-device-ai/on-device-ai-film.webm" type="video/webm">
    <track kind="subtitles" src="https://jared.lynskey.co.nz/en/posts/2026/2026-01-30-on-device-ai/on-device-ai-film_en.vtt" srclang="en" label="English">
    <track kind="subtitles" src="https://jared.lynskey.co.nz/en/posts/2026/2026-01-30-on-device-ai/on-device-ai-film_ja.vtt" srclang="ja" label="日本語">
    <track kind="subtitles" src="https://jared.lynskey.co.nz/en/posts/2026/2026-01-30-on-device-ai/on-device-ai-film_ko.vtt" srclang="ko" label="한국어">
    <track kind="subtitles" src="https://jared.lynskey.co.nz/en/posts/2026/2026-01-30-on-device-ai/on-device-ai-film_vi.vtt" srclang="vi" label="Tiếng Việt">
    <track kind="subtitles" src="https://jared.lynskey.co.nz/en/posts/2026/2026-01-30-on-device-ai/on-device-ai-film_zh-cn.vtt" srclang="zh-cn" label="简体中文" default>
  </video>
  </div>
  <div class="mc-film film-light">
  <video
    class="my-0 rounded-md"
    controls playsinline preload="none"
    poster="https://jared.lynskey.co.nz/en/posts/2026/2026-01-30-on-device-ai/on-device-ai-film-light-poster.jpg" style="background:url('data:image/webp;base64,UklGRoIAAABXRUJQVlA4IHYAAACwBACdASogABIAPw1uqUemIyIYBABjEMT0gAFRZY2mogaLUpH79C93CfzUrxKAAP7vul6EGA0ytD4npxsGq682/6nTvmGx0/XY0S/eONnq1STOt/i7tLHFRM4DTibTbGueLxPoaxVrlM6FoYdfCU8gCNqwAAAA') center/cover no-repeat"
    width="1280"
    height="720"
    aria-label="讲述这篇文章的一段短片">
    <source src="https://jared.lynskey.co.nz/en/posts/2026/2026-01-30-on-device-ai/on-device-ai-film-light.webm" type="video/webm">
    <track kind="subtitles" src="https://jared.lynskey.co.nz/en/posts/2026/2026-01-30-on-device-ai/on-device-ai-film_en.vtt" srclang="en" label="English">
    <track kind="subtitles" src="https://jared.lynskey.co.nz/en/posts/2026/2026-01-30-on-device-ai/on-device-ai-film_ja.vtt" srclang="ja" label="日本語">
    <track kind="subtitles" src="https://jared.lynskey.co.nz/en/posts/2026/2026-01-30-on-device-ai/on-device-ai-film_ko.vtt" srclang="ko" label="한국어">
    <track kind="subtitles" src="https://jared.lynskey.co.nz/en/posts/2026/2026-01-30-on-device-ai/on-device-ai-film_vi.vtt" srclang="vi" label="Tiếng Việt">
    <track kind="subtitles" src="https://jared.lynskey.co.nz/en/posts/2026/2026-01-30-on-device-ai/on-device-ai-film_zh-cn.vtt" srclang="zh-cn" label="简体中文" default>
  </video>
  </div>
  <script>window.mcFilm && window.mcFilm()</script></div><figcaption>这篇文章的九分钟视频版。旁白是英文，配有中文字幕。</figcaption>
</figure>


<h2 class="relative group">我为什么不想用云端 AI
    <div id="我为什么不想用云端-ai" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e6%88%91%e4%b8%ba%e4%bb%80%e4%b9%88%e4%b8%8d%e6%83%b3%e7%94%a8%e4%ba%91%e7%ab%af-ai" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>移动应用里的 AI 功能，套路基本一样：用户输入消息，应用发给云端的模型服务商，响应回来，账单往上涨。</p>
<p>但这是一个施工商家拿来做生意的应用。照这个套路走，客户的姓名、地址、电话就交给了第三方，项目细节、报价、备注躺在别人的服务器上，还得操心 GDPR 和数据驻留这些我根本不想碰的问题。API 账单也随着用户数一路上涨。</p>
<p>另一条路是把模型直接放到用户手机上跑。我选了这条。</p>

<h2 class="relative group">为什么现在做得成了
    <div id="为什么现在做得成了" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e4%b8%ba%e4%bb%80%e4%b9%88%e7%8e%b0%e5%9c%a8%e5%81%9a%e5%be%97%e6%88%90%e4%ba%86" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>放在一年前我不会碰这个，但有三件事几乎同时起了变化。一是量化模型变小了：Q4_K_M 量化后的 Qwen 1.7B 只有 1.1 GB，比大多数游戏还小，下载一次就常驻应用存储。二是手机 GPU 够快了：llama.cpp 在 iOS 上走 Metal，在搭载较新高通 GPU 的 Android 手机上走 OpenCL，iPhone 14 Pro 能跑到每秒 15 个 token 左右，实时流式输出绰绰有余。三是小模型真的能用了：Qwen 1.7B 能遵循结构化指令、解析 JSON、串起多步推理。业务逻辑要的正是这些能力，又没人指望它写诗。</p>

<h2 class="relative group">选模型
    <div id="选模型" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e9%80%89%e6%a8%a1%e5%9e%8b" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>定下 Qwen3-1.7B 之前，我试了四个小模型。</p>
<p><strong>TinyLlama 1.1B</strong>（637 MB）最快，但结构化输出始终不稳，动不动就编造客户 ID、漏掉必填字段。<strong>Phi-3-mini</strong>（1.8 GB）推理不错，可惜话太多，20 个字能说清的事非要写 200 字的小作文。<strong>Gemma-2B</strong>（1.2 GB）做分类又快又准，但函数调用不行，我的工具系统需要的 <code>&lt;action&gt;</code> 标签它输出不稳定。<strong>Qwen3-1.7B</strong>（1.1 GB）正好卡在甜点位：结构化输出可靠，指令遵循精确，还支持用 <code>&lt;think&gt;</code> 标签做思维链。</p>
<p>Q4_K_M 是 llama.cpp 的 k-quant 量化格式之一：权重按块存成 4 位数值，每一块带着自己的缩放系数，M 这一档还会把少数几个敏感的张量保留在 6 位。这样文件比 16 位权重小约 70%，质量只损失一点点。</p>

<h2 class="relative group">配置 llama.rn
    <div id="配置-llamarn" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e9%85%8d%e7%bd%ae-llamarn" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>llama.rn 把 llama.cpp 封装给 React Native 用，安装是整件事里最简单的部分：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">npm install llama.rn
</span></span><span class="line"><span class="cl"><span class="nb">cd</span> ios <span class="o">&amp;&amp;</span> pod install</span></span></code></pre></div></div>
<p>用 Expo 的话，llama.rn 是原生模块，在 Expo Go 里跑不起来。要加上它的 config plugin，改用 development build：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-js" data-lang="js"><span class="line"><span class="cl"><span class="c1">// app.config.js
</span></span></span><span class="line"><span class="cl"><span class="kr">export</span> <span class="k">default</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">  <span class="nx">expo</span><span class="o">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="nx">plugins</span><span class="o">:</span> <span class="p">[</span><span class="s2">&#34;llama.rn&#34;</span><span class="p">],</span>
</span></span><span class="line"><span class="cl">  <span class="p">},</span>
</span></span><span class="line"><span class="cl"><span class="p">};</span></span></span></code></pre></div></div>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">npx expo install llama.rn
</span></span><span class="line"><span class="cl">npx expo run:ios</span></span></code></pre></div></div>
<p>打生产包时，这个插件还会顺手加上 llama.rn 为大模型推荐的两项 iOS entitlement（Extended Virtual Addressing 和 Increased Memory Limit，也就是扩展虚拟寻址和提高内存上限）。</p>
<p>模型本体在首次使用时下载：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-typescript" data-lang="typescript"><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">MODEL_URL</span> <span class="o">=</span> <span class="s2">&#34;https://huggingface.co/unsloth/Qwen3-1.7B-GGUF/resolve/main/Qwen3-1.7B-Q4_K_M.gguf&#34;</span><span class="p">;</span>
</span></span><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">MODEL_PATH</span> <span class="o">=</span> <span class="nx">FileSystem</span><span class="p">.</span><span class="nx">documentDirectory</span> <span class="o">+</span> <span class="s2">&#34;llama-models/Qwen3-1.7B-Q4_K_M.gguf&#34;</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">downloadModel</span> <span class="o">=</span> <span class="kr">async</span> <span class="p">()</span> <span class="o">=&gt;</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">  <span class="kr">const</span> <span class="nx">downloadResumable</span> <span class="o">=</span> <span class="nx">FileSystem</span><span class="p">.</span><span class="nx">createDownloadResumable</span><span class="p">(</span>
</span></span><span class="line"><span class="cl">    <span class="nx">MODEL_URL</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="nx">MODEL_PATH</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="p">{},</span>
</span></span><span class="line"><span class="cl">    <span class="p">(</span><span class="nx">progress</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">      <span class="kr">const</span> <span class="nx">pct</span> <span class="o">=</span> <span class="nx">progress</span><span class="p">.</span><span class="nx">totalBytesWritten</span> <span class="o">/</span> <span class="nx">progress</span><span class="p">.</span><span class="nx">totalBytesExpectedToWrite</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">      <span class="nx">setDownloadProgress</span><span class="p">(</span><span class="nx">pct</span><span class="p">);</span>
</span></span><span class="line"><span class="cl">    <span class="p">}</span>
</span></span><span class="line"><span class="cl">  <span class="p">);</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">  <span class="k">await</span> <span class="nx">downloadResumable</span><span class="p">.</span><span class="nx">downloadAsync</span><span class="p">();</span>
</span></span><span class="line"><span class="cl"><span class="p">};</span></span></span></code></pre></div></div>
<p>WiFi 下两三分钟，LTE 大概五到八分钟。下载完成后，开着 GPU 加速加载：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-typescript" data-lang="typescript"><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">ctx</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">initLlama</span><span class="p">({</span>
</span></span><span class="line"><span class="cl">  <span class="nx">model</span>: <span class="kt">MODEL_PATH</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">  <span class="nx">n_ctx</span>: <span class="kt">8192</span><span class="p">,</span>      <span class="c1">// 8K context window
</span></span></span><span class="line"><span class="cl">  <span class="nx">n_gpu_layers</span>: <span class="kt">99</span><span class="p">,</span> <span class="c1">// Use GPU for all layers
</span></span></span><span class="line"><span class="cl"><span class="p">});</span></span></span></code></pre></div></div>
<p><code>n_gpu_layers: 99</code> 千万别省。把计算从 CPU 挪到 Metal 或 OpenCL 上，速度能提 5 倍左右。</p>

<h2 class="relative group">流式推理
    <div id="流式推理" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e6%b5%81%e5%bc%8f%e6%8e%a8%e7%90%86" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>没人愿意盯着加载动画看十秒钟，所以响应按 token 逐个流出：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-typescript" data-lang="typescript"><span class="line"><span class="cl"><span class="kd">let</span> <span class="nx">fullResponse</span> <span class="o">=</span> <span class="s2">&#34;&#34;</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">await</span> <span class="nx">llamaContext</span><span class="p">.</span><span class="nx">completion</span><span class="p">(</span>
</span></span><span class="line"><span class="cl">  <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="nx">messages</span><span class="o">:</span> <span class="p">[</span>
</span></span><span class="line"><span class="cl">      <span class="p">{</span> <span class="nx">role</span><span class="o">:</span> <span class="s2">&#34;system&#34;</span><span class="p">,</span> <span class="nx">content</span>: <span class="kt">systemPrompt</span> <span class="p">},</span>
</span></span><span class="line"><span class="cl">      <span class="p">{</span> <span class="nx">role</span><span class="o">:</span> <span class="s2">&#34;user&#34;</span><span class="p">,</span> <span class="nx">content</span><span class="o">:</span> <span class="s2">&#34;Create a job for John Smith&#34;</span> <span class="p">}</span>
</span></span><span class="line"><span class="cl">    <span class="p">],</span>
</span></span><span class="line"><span class="cl">    <span class="nx">n_predict</span>: <span class="kt">512</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="nx">temperature</span>: <span class="kt">0.7</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="nx">top_p</span>: <span class="kt">0.8</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">  <span class="p">},</span>
</span></span><span class="line"><span class="cl">  <span class="p">(</span><span class="nx">data</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="c1">// Called for each token
</span></span></span><span class="line"><span class="cl">    <span class="nx">fullResponse</span> <span class="o">+=</span> <span class="nx">data</span><span class="p">.</span><span class="nx">token</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">    <span class="nx">setStreamingText</span><span class="p">(</span><span class="nx">fullResponse</span><span class="p">);</span>
</span></span><span class="line"><span class="cl">  <span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="p">);</span></span></span></code></pre></div></div>
<p>在近几年的手机上，首个 token 约 200ms（这是提示词处理的时间），之后每个 token 50-80ms。跟走网络的 API 一比，体感就是即时的。</p>

<h2 class="relative group">小模型也玩得转的函数调用
    <div id="小模型也玩得转的函数调用" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e5%b0%8f%e6%a8%a1%e5%9e%8b%e4%b9%9f%e7%8e%a9%e5%be%97%e8%bd%ac%e7%9a%84%e5%87%bd%e6%95%b0%e8%b0%83%e7%94%a8" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>大型云端模型那套函数调用靠 JSON schema，小模型在这里会栽跟头：JSON 格式坏掉、必填字段丢失。所以我用了个更简单的 XML 协议：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-typescript" data-lang="typescript"><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">systemPrompt</span> <span class="o">=</span> <span class="sb">`You are an AI assistant. When you need to take an action, output:
</span></span></span><span class="line"><span class="cl"><span class="sb">&lt;action&gt;{&#34;type&#34;:&#34;search_customers&#34;,&#34;query&#34;:&#34;Smith&#34;}&lt;/action&gt;
</span></span></span><span class="line"><span class="cl"><span class="sb">
</span></span></span><span class="line"><span class="cl"><span class="sb">Available actions:
</span></span></span><span class="line"><span class="cl"><span class="sb">- search_customers: {&#34;type&#34;:&#34;search_customers&#34;,&#34;query&#34;:&#34;John&#34;}
</span></span></span><span class="line"><span class="cl"><span class="sb">- create_job: {&#34;type&#34;:&#34;create_job&#34;,&#34;customer_id&#34;:5,&#34;alias&#34;:&#34;Living Room&#34;}
</span></span></span><span class="line"><span class="cl"><span class="sb">- update_job: {&#34;type&#34;:&#34;update_job&#34;,&#34;job_id&#34;:&#34;J-0042&#34;,&#34;status&#34;:&#34;quoting&#34;}
</span></span></span><span class="line"><span class="cl"><span class="sb">
</span></span></span><span class="line"><span class="cl"><span class="sb">Rules:
</span></span></span><span class="line"><span class="cl"><span class="sb">1. ONE action tag per response, at the very end
</span></span></span><span class="line"><span class="cl"><span class="sb">2. When searching/creating → end with &lt;action&gt;
</span></span></span><span class="line"><span class="cl"><span class="sb">3. When just chatting → no action tag
</span></span></span><span class="line"><span class="cl"><span class="sb">`</span><span class="p">;</span></span></span></code></pre></div></div>
<p>选 XML 标签的理由都很朴素：<code>&lt;action&gt;</code> 和 <code>&lt;/action&gt;</code> 的起止毫不含糊，解析只要一条正则，提示词里的示例本身就是格式说明，而且就算模型在标签前后多说几句废话也不影响。</p>
<p>解析代码简单到不好意思展示：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-typescript" data-lang="typescript"><span class="line"><span class="cl"><span class="kd">function</span> <span class="nx">parseAction</span><span class="p">(</span><span class="nx">text</span>: <span class="kt">string</span><span class="p">)</span><span class="o">:</span> <span class="nx">Action</span> <span class="o">|</span> <span class="kc">null</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">  <span class="kr">const</span> <span class="nx">match</span> <span class="o">=</span> <span class="nx">text</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="sr">/&lt;action&gt;([\s\S]*?)&lt;\/action&gt;/</span><span class="p">);</span>
</span></span><span class="line"><span class="cl">  <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">match</span><span class="p">)</span> <span class="k">return</span> <span class="kc">null</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">  <span class="k">try</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="nx">JSON</span><span class="p">.</span><span class="nx">parse</span><span class="p">(</span><span class="nx">match</span><span class="p">[</span><span class="mi">1</span><span class="p">].</span><span class="nx">trim</span><span class="p">());</span>
</span></span><span class="line"><span class="cl">  <span class="p">}</span> <span class="k">catch</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="kc">null</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">  <span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span></span></span></code></pre></div></div>
<p>解析出 action 就执行，再把结果塞回对话里：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-typescript" data-lang="typescript"><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">action</span> <span class="o">=</span> <span class="nx">parseAction</span><span class="p">(</span><span class="nx">modelResponse</span><span class="p">);</span>
</span></span><span class="line"><span class="cl"><span class="k">if</span> <span class="p">(</span><span class="nx">action</span><span class="p">)</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">  <span class="kr">const</span> <span class="nx">result</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">executeAction</span><span class="p">(</span><span class="nx">action</span><span class="p">);</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">  <span class="c1">// Inject result as a system message
</span></span></span><span class="line"><span class="cl">  <span class="nx">conversationHistory</span><span class="p">.</span><span class="nx">push</span><span class="p">({</span>
</span></span><span class="line"><span class="cl">    <span class="nx">role</span><span class="o">:</span> <span class="s2">&#34;user&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="nx">content</span><span class="o">:</span> <span class="sb">`[TOOL_RESULT] </span><span class="si">${</span><span class="nx">result</span><span class="p">.</span><span class="nx">message</span><span class="si">}</span><span class="err">\</span><span class="sb">n</span><span class="err">\</span><span class="sb">n→ NEXT: Tell the user what happened.`</span>
</span></span><span class="line"><span class="cl">  <span class="p">});</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">  <span class="c1">// Continue generation
</span></span></span><span class="line"><span class="cl">  <span class="k">await</span> <span class="nx">generateNextResponse</span><span class="p">();</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span></span></span></code></pre></div></div>
<p>就这么点东西，多步工作流已经能顺畅跑起来了：</p>
<p><strong>用户</strong>：&ldquo;Create a job for Smith&rdquo;</p>
<ol>
<li><strong>模型</strong>：<code>&lt;action&gt;{&quot;type&quot;:&quot;search_customers&quot;,&quot;query&quot;:&quot;Smith&quot;}&lt;/action&gt;</code></li>
<li><strong>系统</strong>：<code>[TOOL_RESULT] Found: Jane Smith (id:42), Bob Smith (id:89)</code></li>
<li><strong>模型</strong>：&ldquo;找到两位 Smith：Jane 和 Bob。您要哪一位？&rdquo;</li>
<li><strong>用户</strong>：&ldquo;Jane&rdquo;</li>
<li><strong>模型</strong>：<code>&lt;action&gt;{&quot;type&quot;:&quot;create_job&quot;,&quot;customer_id&quot;:42}&lt;/action&gt;</code></li>
<li><strong>系统</strong>：<code>[TOOL_RESULT] Job J-0073 created</code></li>
<li><strong>模型</strong>：&ldquo;完成！已为 Jane Smith 创建工作单 J-0073。&rdquo;</li>
</ol>

<h2 class="relative group">用 &lt;think&gt; 标签做思维链
    <div id="用-think-标签做思维链" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e7%94%a8-think-%e6%a0%87%e7%ad%be%e5%81%9a%e6%80%9d%e7%bb%b4%e9%93%be" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>让小模型把推理&quot;说出来&quot;，稳定性会肉眼可见地提高。Qwen 自带思考模式：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-typescript" data-lang="typescript"><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">systemPrompt</span> <span class="o">=</span> <span class="sb">`Before each response, wrap your reasoning in &lt;think&gt; tags:
</span></span></span><span class="line"><span class="cl"><span class="sb">
</span></span></span><span class="line"><span class="cl"><span class="sb">&lt;think&gt;
</span></span></span><span class="line"><span class="cl"><span class="sb">INTENT: what does the user want?
</span></span></span><span class="line"><span class="cl"><span class="sb">HAVE: what data do I already have?
</span></span></span><span class="line"><span class="cl"><span class="sb">NEED: what&#39;s still missing?
</span></span></span><span class="line"><span class="cl"><span class="sb">DECISION: call tool | ask user | just respond
</span></span></span><span class="line"><span class="cl"><span class="sb">&lt;/think&gt;
</span></span></span><span class="line"><span class="cl"><span class="sb">
</span></span></span><span class="line"><span class="cl"><span class="sb">Then output your actual response.`</span><span class="p">;</span></span></span></code></pre></div></div>
<p>模型内部大概是这样运转的：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">&lt;think&gt;
</span></span><span class="line"><span class="cl">INTENT: create a job
</span></span><span class="line"><span class="cl">HAVE: customer query &#34;Smith&#34;
</span></span><span class="line"><span class="cl">NEED: exact customer_id
</span></span><span class="line"><span class="cl">DECISION: search first
</span></span><span class="line"><span class="cl">&lt;/think&gt;
</span></span><span class="line"><span class="cl">&lt;action&gt;{&#34;type&#34;:&#34;search_customers&#34;,&#34;query&#34;:&#34;Smith&#34;}&lt;/action&gt;</span></span></code></pre></div></div>
<p><code>&lt;think&gt;</code> 标签在进 UI 之前会被剥掉，开发阶段我会留着它们调试。效果非常明显：模型会先把逻辑捋顺，再决定动手。</p>

<h2 class="relative group">把长对话塞进 8K 上下文
    <div id="把长对话塞进-8k-上下文" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e6%8a%8a%e9%95%bf%e5%af%b9%e8%af%9d%e5%a1%9e%e8%bf%9b-8k-%e4%b8%8a%e4%b8%8b%e6%96%87" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>工具结果一多，8K 上下文窗口比想象中满得快。我这边大约 30 轮就顶到上限了。</p>
<p>我的办法是对话压缩，让模型自己总结自己：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-typescript" data-lang="typescript"><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">compactPrompt</span> <span class="o">=</span> <span class="sb">`Summarize this conversation in under 200 words.
</span></span></span><span class="line"><span class="cl"><span class="sb">Include: user&#39;s goal, customers/jobs created (with IDs), and any unfinished tasks.
</span></span></span><span class="line"><span class="cl"><span class="sb">
</span></span></span><span class="line"><span class="cl"><span class="si">${</span><span class="nx">conversationHistory</span><span class="p">.</span><span class="nx">map</span><span class="p">(</span><span class="nx">m</span> <span class="o">=&gt;</span> <span class="sb">`</span><span class="si">${</span><span class="nx">m</span><span class="p">.</span><span class="nx">role</span><span class="si">}</span><span class="sb">: </span><span class="si">${</span><span class="nx">m</span><span class="p">.</span><span class="nx">content</span><span class="si">}</span><span class="sb">`</span><span class="p">).</span><span class="nx">join</span><span class="p">(</span><span class="s1">&#39;\n\n&#39;</span><span class="p">)</span><span class="si">}</span><span class="sb">`</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">summary</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">chat</span><span class="p">([{</span> <span class="nx">role</span><span class="o">:</span> <span class="s2">&#34;user&#34;</span><span class="p">,</span> <span class="nx">content</span>: <span class="kt">compactPrompt</span> <span class="p">}]);</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1">// Replace history with summary
</span></span></span><span class="line"><span class="cl"><span class="nx">conversationHistory</span> <span class="o">=</span> <span class="p">[</span>
</span></span><span class="line"><span class="cl">  <span class="p">{</span> <span class="nx">role</span><span class="o">:</span> <span class="s2">&#34;user&#34;</span><span class="p">,</span> <span class="nx">content</span><span class="o">:</span> <span class="sb">`[CONTEXT SUMMARY]</span><span class="err">\</span><span class="sb">n</span><span class="si">${</span><span class="nx">summary</span><span class="si">}</span><span class="err">\</span><span class="sb">n[END SUMMARY]`</span> <span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="p">];</span></span></span></code></pre></div></div>
<p>50 条消息的对话能压到 150 个 token 左右，模型接着聊，也不会忘记之前建了哪些工作单、客户 ID 是多少。</p>

<h2 class="relative group">会话保存
    <div id="会话保存" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e4%bc%9a%e8%af%9d%e4%bf%9d%e5%ad%98" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>聊天会话也会存到应用的后端。值得借鉴的一点是：每条助手消息存两份，一份清理过、用于显示，一份保留 <code>&lt;think&gt;</code> 和 <code>&lt;action&gt;</code> 标签的原文。这样聊天记录在界面上回放时干干净净，恢复会话时又能拿回包含 action 在内的完整上下文。用户可以像用一般的聊天应用那样，在历史下拉菜单里切换对话。</p>

<h2 class="relative group">生产环境的性能数据
    <div id="生产环境的性能数据" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e7%94%9f%e4%ba%a7%e7%8e%af%e5%a2%83%e7%9a%84%e6%80%a7%e8%83%bd%e6%95%b0%e6%8d%ae" aria-label="锚点">#</a>
    </span>
    
</h2>
<p><strong>iPhone 14 Pro</strong>（A16 Bionic，6GB RAM）：</p>
<ul>
<li>模型加载：约 3 秒</li>
<li>首个 token：180-220ms</li>
<li>流式输出：14-16 tokens/秒</li>
<li>内存：约 1.8 GB</li>
</ul>
<p><strong>Samsung Galaxy S23</strong>（Snapdragon 8 Gen 2，8GB RAM）：</p>
<ul>
<li>模型加载：约 4 秒</li>
<li>首个 token：250-300ms</li>
<li>流式输出：10-12 tokens/秒</li>
<li>内存：约 2.1 GB</li>
</ul>
<p><strong>iPhone 11</strong>（A13，4GB RAM）：</p>
<ul>
<li>模型加载：约 6 秒</li>
<li>首个 token：400-500ms</li>
<li>流式输出：6-8 tokens/秒</li>
<li>内存：约 2.2 GB（内存吃紧时偶尔崩溃）</li>
</ul>
<p>我的经验值：想用得顺畅，RAM 至少 6GB。4GB 也能跑，但应用退到后台时多半得把模型卸载掉。</p>

<h2 class="relative group">内存管理
    <div id="内存管理" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e5%86%85%e5%ad%98%e7%ae%a1%e7%90%86" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>应用占着太多内存，iOS 会毫不客气地把它杀掉，所以一离开前台我就释放模型：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-typescript" data-lang="typescript"><span class="line"><span class="cl"><span class="nx">useEffect</span><span class="p">(()</span> <span class="o">=&gt;</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">  <span class="kr">const</span> <span class="nx">subscription</span> <span class="o">=</span> <span class="nx">AppState</span><span class="p">.</span><span class="nx">addEventListener</span><span class="p">(</span><span class="s2">&#34;change&#34;</span><span class="p">,</span> <span class="p">(</span><span class="nx">nextAppState</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="k">if</span> <span class="p">(</span><span class="nx">nextAppState</span> <span class="o">===</span> <span class="s2">&#34;background&#34;</span><span class="p">)</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">      <span class="nx">llamaContext</span><span class="o">?</span><span class="p">.</span><span class="nx">release</span><span class="p">();</span> <span class="c1">// Free ~2 GB
</span></span></span><span class="line"><span class="cl">    <span class="p">}</span> <span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="nx">nextAppState</span> <span class="o">===</span> <span class="s2">&#34;active&#34;</span><span class="p">)</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">      <span class="nx">loadModel</span><span class="p">();</span> <span class="c1">// Reload when foregrounded
</span></span></span><span class="line"><span class="cl">    <span class="p">}</span>
</span></span><span class="line"><span class="cl">  <span class="p">});</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">  <span class="k">return</span> <span class="p">()</span> <span class="o">=&gt;</span> <span class="nx">subscription</span><span class="p">.</span><span class="nx">remove</span><span class="p">();</span>
</span></span><span class="line"><span class="cl"><span class="p">},</span> <span class="p">[]);</span></span></span></code></pre></div></div>

<h2 class="relative group">电量消耗
    <div id="电量消耗" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e7%94%b5%e9%87%8f%e6%b6%88%e8%80%97" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>端侧推理在电量上不是免费的。实测数据：</p>
<table>
	<thead>
			<tr>
					<th>使用模式</th>
					<th>额外电池消耗</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td>轻度（每天 5-10 次查询）</td>
					<td>每天 &lt;1%</td>
			</tr>
			<tr>
					<td>中度（每天 20-30 次查询）</td>
					<td>每天约 3-4%</td>
			</tr>
			<tr>
					<td>重度（每天 50+ 次查询）</td>
					<td>每天约 6-8%</td>
			</tr>
	</tbody>
</table>
<p>GPU 推理比 CPU 费电，但快得多。用户明显更喜欢秒回，所以我保留了 GPU。</p>

<h2 class="relative group">上线两个月，约 200 名测试用户
    <div id="上线两个月约-200-名测试用户" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e4%b8%8a%e7%ba%bf%e4%b8%a4%e4%b8%aa%e6%9c%88%e7%ba%a6-200-%e5%90%8d%e6%b5%8b%e8%af%95%e7%94%a8%e6%88%b7" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>查询的分布有点出乎意料：</p>
<ol>
<li>&ldquo;Create a job for [customer name]&rdquo; - 占查询的 68%</li>
<li>&ldquo;Find jobs for [customer]&rdquo; - 15%</li>
<li>&ldquo;Update job [ID] to [status]&rdquo; - 9%</li>
<li>关于应用本身的问题 - 8%</li>
</ol>
<p>91% 的查询一次就成功。失败的原因分三类：用户把客户名字拼错导致搜不到（4%），超长对话撑爆上下文（3%），模型自己编客户 ID（2%）。</p>
<p>最后这个编 ID 的毛病以前严重得多。后来我把工具结果从叙述句改成 <code>customer_id=42</code> 这样的显式字段，问题基本消失了。小模型就是需要把饭喂到这个程度。</p>

<h2 class="relative group">成本账
    <div id="成本账" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e6%88%90%e6%9c%ac%e8%b4%a6" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>端侧方案：基础设施 $0/月，除了首次那 1.1 GB 下载，每个用户的成本是 $0，用户再多也是这个数。</p>
<p>云端方案粗算一下：平均一次对话约 2,000 个 token，200 个用户每月 30 次对话就是 6,000 次，也就是每月 1200 万 token。便宜的云端模型大约 $18/月，顶级模型则要 $180 到 $360/月。这个规模不至于伤筋动骨，但端侧方案第一天就回了本，哪怕涨到一万用户，还是 $0。</p>

<h2 class="relative group">得心里有数的几个限制
    <div id="得心里有数的几个限制" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e5%be%97%e5%bf%83%e9%87%8c%e6%9c%89%e6%95%b0%e7%9a%84%e5%87%a0%e4%b8%aa%e9%99%90%e5%88%b6" aria-label="锚点">#</a>
    </span>
    
</h2>
<p><strong>模型能力。</strong> Qwen 1.7B 擅长结构化任务，但多跳的复杂推理、事实性知识（它不是搜索引擎）、创意写作、语言里的微妙之处，它都不行。围绕小模型擅长的事来设计功能，就不会失望。</p>
<p><strong>设备要求。</strong> 底线是 3 GB RAM 加 2 GB 可用存储；实际上你需要 6 GB RAM、64 位处理器和 GPU 支持。iPhone 8、Galaxy S9 这类老设备很吃力，记得做特性检测和体面的降级。</p>
<p><strong>模型更新。</strong> 换模型意味着用户要重新下载 1.1 GB。我把版本号写进了存储路径：</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-typescript" data-lang="typescript"><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">MODEL_PATH</span> <span class="o">=</span> <span class="sb">`</span><span class="si">${</span><span class="nx">FileSystem</span><span class="p">.</span><span class="nx">documentDirectory</span><span class="si">}</span><span class="sb">llama-models/v2/Qwen3-1.7B-Q4_K_M.gguf`</span><span class="p">;</span></span></span></code></pre></div></div>
<p>发新模型时把路径版本号加一就行，旧模型在应用卸载时自动清掉。</p>

<h2 class="relative group">iOS 和 Android 的差异
    <div id="ios-和-android-的差异" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#ios-%e5%92%8c-android-%e7%9a%84%e5%b7%ae%e5%bc%82" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>走 Metal 的 iOS 比 Android 快 20% 左右，内存管理也更好，模型默认会备份到 iCloud（不想占用户备份空间的话，用 <code>NSURLIsExcludedFromBackupKey</code> 排除掉）。Android 机型间的差异则大得多：llama.rn 的 GPU 路径是高通 Adreno 700 系列 GPU 上的 OpenCL，没有这类 GPU 的手机就只能跑在 CPU 上；存储也不会自动备份。两个平台都得测。好安卓机和差安卓机之间的差距，比两个平台之间的差距还大。</p>

<h2 class="relative group">调试技巧
    <div id="调试技巧" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e8%b0%83%e8%af%95%e6%8a%80%e5%b7%a7" aria-label="锚点">#</a>
    </span>
    
</h2>
<p><strong>1. 打开详细日志</strong></p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-typescript" data-lang="typescript"><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">ctx</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">initLlama</span><span class="p">({</span>
</span></span><span class="line"><span class="cl">  <span class="nx">model</span>: <span class="kt">MODEL_PATH</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">  <span class="nx">n_ctx</span>: <span class="kt">8192</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">  <span class="nx">n_gpu_layers</span>: <span class="kt">99</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">  <span class="nx">verbose</span>: <span class="kt">true</span><span class="p">,</span> <span class="c1">// Logs every token + timings
</span></span></span><span class="line"><span class="cl"><span class="p">});</span></span></span></code></pre></div></div>
<p><strong>2. 跟踪 token 数量</strong></p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-typescript" data-lang="typescript"><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">tokens</span> <span class="o">=</span> <span class="nx">fullResponse</span><span class="p">.</span><span class="nx">split</span><span class="p">(</span><span class="sr">/\s+/</span><span class="p">).</span><span class="nx">length</span> <span class="o">*</span> <span class="mf">1.3</span><span class="p">;</span> <span class="c1">// Rough estimate
</span></span></span><span class="line"><span class="cl"><span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="sb">`Generated </span><span class="si">${</span><span class="nx">tokens</span><span class="si">}</span><span class="sb"> tokens in </span><span class="si">${</span><span class="nx">duration</span><span class="si">}</span><span class="sb">ms`</span><span class="p">);</span></span></span></code></pre></div></div>
<p><strong>3. 监控内存</strong></p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-typescript" data-lang="typescript"><span class="line"><span class="cl"><span class="kr">import</span> <span class="p">{</span> <span class="nx">MemoryInfo</span> <span class="p">}</span> <span class="kr">from</span> <span class="s1">&#39;react-native-device-info&#39;</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">memoryUsage</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">MemoryInfo</span><span class="p">.</span><span class="nx">getUsedMemory</span><span class="p">();</span>
</span></span><span class="line"><span class="cl"><span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="sb">`Memory: </span><span class="si">${</span><span class="p">(</span><span class="nx">memoryUsage</span> <span class="o">/</span> <span class="mi">1024</span> <span class="o">/</span> <span class="mi">1024</span><span class="p">).</span><span class="nx">toFixed</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span><span class="si">}</span><span class="sb"> MB`</span><span class="p">);</span></span></span></code></pre></div></div>
<p><strong>4. 离线测试</strong></p>
<p>开飞行模式，认真把应用用一遍。模型应该从缓存加载，推理应该完整跑完，唯一失败的应该只有后端持久化，而且要败得体面。</p>

<h2 class="relative group">接下来想做的
    <div id="接下来想做的" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e6%8e%a5%e4%b8%8b%e6%9d%a5%e6%83%b3%e5%81%9a%e7%9a%84" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>llama.cpp 已经支持视觉模型，&ldquo;这是现场的照片，加到工作单里&quot;这种用法跟这个应用简直是天作之合。我还想拿过去的工作单描述和客户对话微调一个 LoRA 适配器，估计领域内查询的准确率能涨 20-30%。再远一点：配上端侧语音识别模型做完全离线的语音助手，以及在原始数据不出手机的前提下，用联邦学习让模型在所有用户间共同进步。</p>

<h2 class="relative group">值不值得做？
    <div id="值不值得做" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e5%80%bc%e4%b8%8d%e5%80%bc%e5%be%97%e5%81%9a" aria-label="锚点">#</a>
    </span>
    
</h2>
<p>如果你在做处理敏感数据的业务应用，走离线优先的工作流，主要任务是分类、录入、搜索这类结构化工作，尤其是担心规模上去之后的 API 账单，那答案是值得，今天就能动手。</p>
<p>如果你要的是开放式聊天，用大型云端模型。知识密集型任务（小模型真的没什么见识）、必须支持低端设备、领域知识变化快过你更新模型的速度，这几种情况也一样。</p>
<p>整个实现花了我 12 个小时左右。每月运行成本 $0，飞机上照常工作，客户数据从头到尾没离开过客户的手机。挑不出毛病。</p>
<hr>
<p><em>有问题或想聊聊？我是 <a href="https://twitter.com/jaredlynskey"  target="_blank" rel="noreferrer">@jaredlynskey</a>。llama.rn 库在 <a href="https://github.com/mybigday/llama.rn"  target="_blank" rel="noreferrer">github.com/mybigday/llama.rn</a>。</em></p>
]]></content:encoded>
      <media:content url="https://jared.lynskey.co.nz/en/posts/2026/2026-01-30-on-device-ai/featured.jpg" medium="image" type="image/jpeg" />
    </item>
  </channel>
</rss>
