Dan Luu 写系统问题时，习惯先测量、再对照、最后才下结论。把「95%-ile isn't that good」放到智能体、评测和线上系统里，真正要问的是：默认做法会不会系统性失败。下面用中文整理成可执行的工程笔记，去掉原站导航和无关链接。

Overwatch

Reaching 95%-ile isn't very impressive because it's not that hard to do. 一个稳妥判断是 this is one of my most ridiculable ideas. It doesn't help that, when stated nakedly, that sounds elitist. But 一个稳妥判断是 it's just the opposite: most people can become (relatively) good at most things.

Note that when I say 95%-ile, I mean 95%-ile among people who participate, not all people (for many activities, just doing it at all makes you 99%-ile or above across all people). I'm also not referring to 95%-ile among people who practice regularly. The " one weird trick " is that, for a lot of activities, being something like 10%-ile among people who practice can make you something like 90%-ile or 99%-ile among people who participate.

Real life

This post is going to refer to specifics since the discussions I've seen about this are all in the abstract , which turns them into Rorschach tests. For example, Scott Adams has a widely cited post claiming that it's better to be a generalist than a specialist because, to become "extraordinary", you have to either be "the best" at one thing or 75%-ile at two things. If that were strictly true, it would surely be better to be a generalist, but that's of cou

Personally, in every activity I've participated in where it's possible to get a rough percentile ranking, people who are 95%-ile constantly make mistakes that seem like they should be easy to observe and correct. "Real world" activities typically can't be reduced to a percentile rating, but achieving what appears to be a similar level of proficiency seems similarly easy.

Back to this blog's regularly scheduled topic: programming

We'll start by looking at Overwatch (a video game) in detail because it's an activity I'm familiar with where it's easy to get ranking information and observe what's happening, and then we'll look at some "real world" examples where we can observe the same phenomena, although we won't be able to get ranking information for real world examples 1 .

At 90%-ile and 95%-ile ranks in Overwatch, the vast majority of players will pretty much constantly make basic game losing mistakes. These are simple mistakes like standing next to the objective instead of on top of the objective while the match timer runs out, turning a probable victory into a certain defeat. See the attached footnote if you want enough detail about specific mistakes that you can decide for yourself if a mistake is "basic" or not 2 .

Some meta-techniques for improving

In Overwatch, you may see a lot of (1), people who don’t seem to care about winning, at lower ranks, but by the time you get to 30%-ile, it's common to see people indicate their desire to win in various ways, such as yelling at players who are perceived as uncaring about victory or unskilled, complaining about people who they perceive to make mistakes that prevented their team from winning, etc. 3 . Other than the occasional troll, it's not unreasonable to

(2), not having put in time enough to fix their mistakes will, at some point, apply to all players who are improving, but if you look at the median time played at 50%-ile, people who are stably ranked there have put in hundreds of hours (and the median time played at higher ranks is higher). Given how simple the mistakes we're discussing are, not having put in enough time cannot be the case for most players.

Appendix: other most ridiculable ideas

A common complaint among low-ranked Overwatch players in Overwatch forums is that they're just not talented and can never get better. Most people probably don't have the talent to play in a professional league regardless of their practice regimen, but when you can get to 95%-ile by fixing mistakes like "not realizing that you should stand on the objective", you don't really need a lot of talent to get to 95%-ile.

While (4), people not understanding how to spot and fix their mistakes, isn't the only other possible explanation 4 , I believe it's the most likely explanation for most players. Most players who express frustration that they're stuck at a rank up to maybe 95%-ile or 99%-ile don't seem to realize that they could drastically improve by observing their own gameplay or having someone else look at their gameplay.

Appendix: commentary on improvement

One thing that's curious about this is that Overwatch makes it easy to spot basic mistakes (compared to most other activities). After you're killed, the game shows you how you died from the viewpoint of the player who killed you, allowing you to see what led to your death. Overwatch also records the entire game and lets you watch a replay of the game, allowing you to figure out what happened and why the game was won or lost. In many other games, you'd have

If you read Overwatch forums, you'll see a regular stream of posts that are basically "I'm SOOOOOO FRUSTRATED! I've played this game for 1200 hours and I'm still ranked 10%-ile, [some Overwatch specific stuff that will vary from player to player]". Another user will inevitably respond with something like "we can't tell what's wrong from your text, please post a video of your gameplay". In the cases where the original poster responds with a recording of the

值得单独记下的观察

• People don't want to win or don't care about winning
• People understand their mistakes but haven't put in enough time to fix them
• People don't understand how to spot their mistakes and fix them
• Improve slowly when getting feedback would make improving quickly easy
• learning the basics of the game
• reading a beginner book on cardplay
• practicing applying the material
• Get feedback and practice Ideally from an expert coach but, if not, this can be from a layperson or even yourself (if you have some way of recording/tracing what you're doing)

落地时建议先做的 5 件事

1. 用自己的真实负载测，而不是只用公开榜或厂商数字。
2. 把评测设计成能抓到失败模式：平均分好看但尾部崩溃，仍然算失败。
3. 智能体默认不会好好用测试；要写进流程，而不是写在口头规范里。
4. 性能和正确性都要有基线，改模型或改语言前后必须能对比。
5. 结论写成可回滚的决策：哪一版配置、哪一版评测集、谁签字。

和智能体产品怎么接

龙虾PRO做 OpenClaw 落地时，同样吃「先测量再扩面」这条纪律：技能、数字员工和网关都要有可复现评测，而不是只看一次演示通过。

本文侧重全链路风控方法论。落地时请用自身业务单据做回放验证，不要把示例阈值直接当生产策略。 相关：风控体检 · 方案资源