Dan Luu 写系统问题时,习惯先测量、再对照、最后才下结论。把「Files are hard」放到智能体、评测和线上系统里,真正要问的是:默认做法会不会系统性失败。下面用中文整理成可执行的工程笔记,去掉原站导航和无关链接。
Crash Consistency
I haven't used a desktop email client in years. None of them could handle the volume of email I get without at least occasionally corrupting my mailbox. Pine, Eudora, and outlook have all corrupted my inbox, forcing me to restore from backup. How is it that desktop mail clients are less reliable than gmail, even though my gmail account not only handles more email than I ever had on desktop clients, but also allows simultaneous access from multiple location
Well, what sort of failures can occur? Crash consistency (maintaining consistent state even if there's a crash) is probably the easiest property to consider, since we can assume that everything, from the filesystem to the disk, works correctly; let's consider that first.
Filesystem semantics
Pillai et al. had a paper and presentation at OSDI '14 on exactly how hard it is to save data without corruption or data loss.
Let's look at a simple example of what it takes to save data in a way that's robust against a crash. Say we have a file that contains the text a foo and we want to update the file to contain a bar . The pwrite function looks like it's designed for this exact thing. It takes a file descriptor, what we want to write, a length, and an offset. So we might try
Filesystem correctness
pwrite([file], “bar”, 3, 2) // write 3 bytes at offset 2 What happens? If nothing goes wrong, the file will contain a bar , but if there's a crash during the write, we could get a boo , a far , or any other combination. Note that you may want to consider this an example over sectors or blocks and not chars/bytes.
If we want atomicity (so we either end up with a foo or a bar but nothing in between) one standard technique is to make a copy of the data we're about to change in an undo log file, modify the “real” file, and then delete the log file. If a crash happens, we can recover from the log. We might write something like
Error recovery
creat(/dir/log); write(/dir/log, “2,3,foo”, 7); pwrite(/dir/orig, “bar”, 3, 2); unlink(/dir/log); This should allow recovery from a crash without data corruption via the undo log, at least if we're using ext3 and we made sure to mount our drive with data=journal . But we're out of luck if, like most people, we're using the default 1 — with the default data=ordered , the write and pwrite syscalls can be reordered, causing the write to orig to happen before
creat(/dir/log); write(/dir/log, “2, 3, foo”); fsync(/dir/log); // don't allow write to be reordered past pwrite pwrite(/dir/orig, 2, “bar”); fsync(/dir/orig); unlink(/dir/log); That should force things to occur in the correct order, at least if we're using ext3 with data=journal or data=ordered . If we're using data=writeback , a crash during the the write or fsync to log can leave log in a state where the filesize has been adjusted for the write of “bar”
Error frequency
We can fix that by adding a checksum to the log file when creating it. If the contents of log don't contain a valid checksum, then we'll know that we ran into the situation described above.
creat(/dir/log); write(/dir/log, “2, 3, [checksum], foo”); // add checksum to log file fsync(/dir/log); pwrite(/dir/orig, 2, “bar”); fsync(/dir/orig); unlink(/dir/log); That's safe, at least on current configurations of ext3. But it's legal for a filesystem to end up in a state where the log is never created unless we issue an fsync to the parent directory.
Conclusion
creat(/dir/log); write(/dir/log, “2, 3, [checksum], foo”); fsync(/dir/log); fsync(/dir); // fsync parent directory of log file pwrite(/dir/orig, 2, “bar”); fsync(/dir/orig); unlink(/dir/log); That should prevent corruption on any Linux filesystem, but if we want to make sure that the file actually contains “bar”, we need another fsync at the end.
creat(/dir/log); write(/dir/log, “2, 3, [checksum], foo”); fsync(/dir/log); fsync(/dir); pwrite(/dir/orig, 2, “bar”); fsync(/dir/orig); unlink(/dir/log); fsync(/dir); That results in consistent behavior and guarantees that our operation actually modifies the file after it's completed, as long as we assume that fsync actually flushes to disk. OS X and some versions of ext3 have an fsync that doesn't really flush to disk. OS X requires fcntl(F_FULLFSYNC) to
值得单独记下的观察
- rename isn't atomic on crash. POSIX says that rename is atomic, but this only applies to normal operation, not to crashes.
- even if the techinque worked, the performance is very poor
- how do you handle hardlinks?
- metadata can be lost; this can sometimes be preserved, under some filesystems, with ioctls, but now you have filesystem specific code just for the non-crash case
落地时建议先做的 5 件事
- 用自己的真实负载测,而不是只用公开榜或厂商数字。
- 把评测设计成能抓到失败模式:平均分好看但尾部崩溃,仍然算失败。
- 智能体默认不会好好用测试;要写进流程,而不是写在口头规范里。
- 性能和正确性都要有基线,改模型或改语言前后必须能对比。
- 结论写成可回滚的决策:哪一版配置、哪一版评测集、谁签字。
和智能体产品怎么接
龙虾PRO做 OpenClaw 落地时,同样吃「先测量再扩面」这条纪律:技能、数字员工和网关都要有可复现评测,而不是只看一次演示通过。
本文侧重全链路风控方法论。落地时请用自身业务单据做回放验证,不要把示例阈值直接当生产策略。 相关:风控体检 · 方案资源
常见问题 FAQ
什么是AI智能系统?
「AI智能系统」可概括为:I haven't used a desktop email client in years. None of them could handle the volume of email I get without at least occasionally corrupting my mailbox. Pine, Eudora, and outlook h 本文从定义、方法与实践要点展开说明。
为什么要关注AI智能系统?
关注AI智能系统,是因为它直接影响效率、风险与可复制性。文中指出:I haven't used a desktop email client in years. None of them could handle the volume of email I get without at least occasionally corrupting my mailbox. Pine, Eudora, and outlook have all corrupted my inbox, forcing me to restore from backup…
如何落地AI智能系统?有哪些关键步骤?
建议按以下路径推进AI智能系统:1) rename isn't atomic on crash. POSIX says that rename is atomic, but this o…;2) even if the techinque worked, the performance is very poor;3) how do you handle hardlinks?;4) 用自己的真实负载测,而不是只用公开榜或厂商数字。;5) 把评测设计成能抓到失败模式:平均分好看但尾部崩溃,仍然算失败。。细节见正文对应章节。
AI智能系统适合哪些人或团队?
AI智能系统更适合:产品/技术负责人、运营与增长团队、需要落地智能体或自动化的中小团队、关注「AI智能系统」方向的读者。若你只需要单次聊天式问答,可先读概念;若要上生产,请重点看步骤、权限与风控相关段落。
关于「Crash Consistency」,本文给出了什么结论?
在「Crash Consistency」部分,要点是:ve all corrupted my inbox, forcing me to restore from backup. How is it that desktop mail clients are less reliable than gmail, even though my gmail account not only handles more email than I ever had on desktop clients,
关于「Filesystem semantics」,本文给出了什么结论?
在「Filesystem semantics」部分,要点是:e data in a way that's robust against a crash. Say we have a file that contains the text a foo and we want to update the file to contain a bar . The pwrite function looks like it's designed for this exact thing. It takes