Dan Luu 写系统问题时，习惯先测量、再对照、最后才下结论。把「There's no reason for software to be slow anymore」放到智能体、评测和线上系统里，真正要问的是：默认做法会不会系统性失败。下面用中文整理成可执行的工程笔记，去掉原站导航和无关链接。

Optimizing for a class of workload

The other day, I saw a viral tweet saying that people talking about how LLMs are causing slow, bloated, code are going to eat crow once they re-write everything in super-optimized assembly. We're not quite at the point where we want to write everything in assembly , but some variant of what Nolan Lawson said about testing, you can choose how many bugs you want now , which I less eloquently noted here , is becoming more true for performance.

In response to a comment in my last post that the cost of formerly specialized performance work has dropped by many orders of magnitude and performance work that used to require a person or team that had a rare set of skills can be done by anyone who can type a few sentences 1 , which means that you can do all sorts of optimizations that used to be too expensive to be worthwhile for all but the largest scale or most lucrative projects, Marc Brooker respond

Build an index?

Completely agree with your closing point. Dynamic custom software, fitted to a particular workload rather than a class of workloads, seems like a very likely outcome. (Which comes with all kinds of fun risks and opportunities of its own). Kind of reminds me of FFTW . And a ton of weird old demoscene techniques which were all about being super fast and small on a very particular problem (and often very particular hardware). For example, I remember a demo th

There’s been a meme circulating about how AI doesn’t help because “code was never the hard part.” 一个稳妥判断是 that’s true in some domains, but in others, writing the code absolutely was the hard part. JIT compilers are a great example of that. For many pieces of software, a JIT compiler would help a lot with speeding up the code. The rarity of JIT compilers makes me believe that implementing a JIT compiler historically was too difficult for it to be worthwhile

Optimizations are cheap

Let's try this out with FRE , the regex engine we built in the last post. Recall that it was created by having an agent loop for a month on improving regex engine performance with access to the rebar regex benchmark suite . This resulted in FRE being heavily overfit to rebar until we warned our agent that we had a holdout benchmark, which caused the agent to generalize the optimizations enough that performance was ok-ish on our holdout. There's no particul

In the same way we could build a regex engine in a few minutes of human time, we can also just try this experiment in a few minutes of human time. I typed a few sentences and an agent went and did the work to allow this to happen (which would be a decent chunk of code surgery for a human) and it ran the benchmark on actual ripgrep queries that come from my codex history. For longer queries, we see a 2x-4x performance improvement here for a few very simple

Workload-specific optimization

This is arguably a silly thing to do, since if we're repeatedly searching for text on a computer, the obvious thing to do to speed that up isn't to write a native code compiler for regex matching, it's to create an index. But the point here is just that this kind of technical work, which used to take a fair amount of time and expertise, can just be done trivially now. And if we wanted to build a text index, it just so happens that I worked on BitFunnel, th

If I were working at an AI lab and had access to things like SOTA models running on Cerebras chips or other accelerators that greatly increase tok/s and therefore load/demand for search, I might actually survey the existing indexers to see if they're fast enough or if I'd want to build something custom myself. While the open source version of BitFunnel "only" contains a bytecode interpreter and one JIT, the Bing version contains multiple JIT compilers. A p

Appendix: There's no reason for software to be slow anymore

The drastic reduction in the cost of optimizations has been true going back to November 2025 and maybe even somewhat before then with public models (and I'm sure before that still with what folks at AI labs had access to). For an example from the GPT-5.1 or 5.2 days, with no knowledge of game AIs, I tried building an Azul AI. This ended up being the strongest AI in the world for the game by a pretty large margin. From reading the thesis that describes the

There's a bunch of standard stuff it makes sense to do to debug and verify a multithreading algorithm for something like this, like implementing replay from debug logs that can reproduce bugs despite the algorithm being nondetermistic. Doing that alone would've probably been days to a week of work had I done it by hand, but it's exactly the kind of thing an agent can trivially do in a loop (just have it try to replay logs and insert logging for non-determi

Appendix: How is codex running ripgrep?

This also applies to a lot of other tricky optimizations. From having written CPU microcode, done CPU verification, worked on optimizing a search engine index, etc., I have a lot of experience looking at optimizations and thinking "hmm, this would increase performance by 2%, but it's going to take N person-days to verify that this tricky optimization works" and making a call to go ahead or not based on whether or not it's worth the time to get the optimiza

Going back to the game AI case, at least for the AI I tried, it seems like you gain about 100 Elo for every doubling in speed (more than in chess, I suspect because draws are very rare). Just adding multithreading alone is enough to wipe the floor with an otherwise comparable AI on a large machine. If you stack in 10-20 more optimizations that seem too annoying for most people to do by hand, the difference in strength is tremendous and it's not really reas

值得单独记下的观察

• The performance probably would've been better if I had an agent just modify a ripgrep fork directly, but I was curious if this could also solve the FRE overfitting problem with respect to my queries. [return]

落地时建议先做的 5 件事

1. 用自己的真实负载测，而不是只用公开榜或厂商数字。
2. 把评测设计成能抓到失败模式：平均分好看但尾部崩溃，仍然算失败。
3. 智能体默认不会好好用测试；要写进流程，而不是写在口头规范里。
4. 性能和正确性都要有基线，改模型或改语言前后必须能对比。
5. 结论写成可回滚的决策：哪一版配置、哪一版评测集、谁签字。

和智能体产品怎么接

龙虾PRO做 OpenClaw 落地时，同样吃「先测量再扩面」这条纪律：技能、数字员工和网关都要有可复现评测，而不是只看一次演示通过。

本文侧重全链路风控方法论。落地时请用自身业务单据做回放验证，不要把示例阈值直接当生产策略。 相关：风控体检 · 方案资源