Dan Luu 写系统问题时，习惯先测量、再对照、最后才下结论。把「Misidentifying talent」放到智能体、评测和线上系统里，真正要问的是：默认做法会不会系统性失败。下面用中文整理成可执行的工程笔记，去掉原站导航和无关链接。

Appendix: other factors

Out of context, you might think they were scouting actors or models, but these are baseball players ("A" is Lloyd Moseby, "B" is Jim Abbott, and "C" is Derek Jeter), ones that were quite good (Lloyd Moseby was arguably only a very good player for perhaps four years, but that makes him extraordinary compared to most players who are scouted). If you read other baseball scouting reports, you'll see a lot of comments about how someone has a "good face", who th

Basically everyone wants to hire talented folks. But even in baseball, where returns to hiring talent are obvious and high and which is the most easily quantified major U.S. sport, people made fairly obvious blunders for a century due to relying on incorrectly honed gut feelings that relied heavily on unconscious as well as conscious biases. Later, we'll look at what baseball hiring means for other fields, but first, let's look at how players who didn't re

Appendix: related discussions

On the flip side, scouts would also pan players who would later turn out to be great based on their physical appearance, such as these scouts who were concerned about Albert Pujols's weight:

Pujols ended up becoming one of the best baseball players of all time ( currently ranked 32nd by WAR ). His weight wasn't a problem, but if you read scouting reports on other great players who were heavy or short, they were frequently underrated. Of course, baseball scouting reports didn't only look at people's appearances, but scouts were generally highly biased by what they thought an athlete should look like.

值得单独记下的观察

• Recruit A: … will be a real specimen with chance to have a Dave Parker body. Facially looks like Leon Wagner. Good body flexibility. Very large hands.
• Recruit B: Outstanding physical specimen – big athletic frame with broad shoulders and long, solid arms and leg. Good bounce in his step and above avg body control. Good strong face.
• Recruit C: Hi butt, longish arms & legs, leanish torso, young colt
• [different scout]: Wiry loose good agility with good face
• [another scout]: Athletic looking body, loose, rangy, slightly bow legged.
• Scout 3: 6'1"-6'2" 180 solid upper and lower half. Room to pack another 15 without hurting
• Scout 1: Heavy, bulky body. Extra (weight) on lower half. Future (weight) problem. Aggressive hitter with mistake HR power. Tends to be a hacker.
• Scout 2: Good bat (speed) with very strong hands. Competes well and battles at the plate. Contact seems fair. Swing gets a little long at times. Will over pull. He did not hit the ball hard to CF or RF. Weight will becom

落地时建议先做的 5 件事

1. 用自己的真实负载测，而不是只用公开榜或厂商数字。
2. 把评测设计成能抓到失败模式：平均分好看但尾部崩溃，仍然算失败。
3. 智能体默认不会好好用测试；要写进流程，而不是写在口头规范里。
4. 性能和正确性都要有基线，改模型或改语言前后必须能对比。
5. 结论写成可回滚的决策：哪一版配置、哪一版评测集、谁签字。

和智能体产品怎么接

龙虾PRO做 OpenClaw 落地时，同样吃「先测量再扩面」这条纪律：技能、数字员工和网关都要有可复现评测，而不是只看一次演示通过。

本文侧重全链路风控方法论。落地时请用自身业务单据做回放验证，不要把示例阈值直接当生产策略。 相关：风控体检 · 方案资源