Tim Dettmers 做量化、多卡和智能体时，习惯先把硬件账和算法账摊开。把「A Full Hardware Guide to Deep Learning」写成能落地的笔记，核心不是复述论文摘要，而是：显存、精度、通信和评测会在哪一层先爆。下面用中文重排，去掉原站广告和分享条。

GPU

Deep Learning is very computationally intensive, so you will need a fast CPU with many cores, right? Or is it maybe wasteful to buy a fast CPU? One of the worst things you can do when building a deep learning system is to waste money on hardware that is unnecessary. Here I will guide you step by step through the hardware you will need for a cheap high-performance system.

Over the years, I build a total of 7 different deep learning workstations and despite careful research and reasoning, I made my fair share of mistake in selecting hardware parts. In this guide, I want to share my experience that I gained over the years so that you do not make the same mistakes that I did before.

RAM

The blog post is ordered by mistake severity. This means the mistakes where people usually waste the most money come first.

This blog post assumes that you will use a GPU for deep learning. If you are building or upgrading your system for deep learning, it is not sensible to leave out the GPU. The GPU is just the heart of deep learning applications – the improvement in processing speed is just too huge to ignore.

Needed RAM Clock Rate

I talked at length about GPU choice in my GPU recommendations blog post , and the choice of your GPU is probably the most critical choice for your deep learning system. There are three main mistakes that you can make when choosing a GPU: (1) bad cost/performance, (2) not enough memory, (3) poor cooling.

For good cost/performance, I generally recommend an RTX 2070 or an RTX 2080 Ti. If you use these cards you should use 16-bit models. Otherwise, GTX 1070, GTX 1080, GTX 1070 Ti, and GTX 1080 Ti from eBay are fair choices and you can use these GPUs with 32-bit (but not 16-bit).

RAM Size

Be careful about the memory requirements when you pick your GPU. RTX cards, which can run in 16-bits, can train models which are twice as big with the same memory compared to GTX cards. As such RTX cards have a memory advantage and picking RTX cards and learn how to use 16-bit models effectively will carry you a long way. In general, the requirements for memory are roughly the following:

Another problem to watch out for, especially if you buy multiple RTX cards is cooling. If you want to stick GPUs into PCIe slots which are next to each other you should make sure that you get GPUs with a blower-style fan. Otherwise you might run into temperature issues and your GPUs will be slower (about 30%) and die faster.

CPU

The main mistakes with RAM is to buy RAM with a too high clock rate. The second mistake is to buy not enough RAM to have a smooth prototyping experience.

RAM clock rates are marketing stints where RAM companies lure you into buying “faster” RAM which actually yields little to no performance gains. This is best explained by “ Does RAM speed REALLY matter? ” video on RAM von Linus Tech Tips.

CPU and PCI-Express

Furthermore, it is important to know that RAM speed is pretty much irrelevant for fast CPU RAM->GPU RAM transfers. This is so because (1) if you used pinned memory , your mini-batches will be transferred to the GPU without involvement from the CPU, and (2) if you do not use pinned memory the performance gains of fast vs slow RAMs is about 0-3% — spend your money elsewhere!

RAM size does not affect deep learning performance. However, it might hinder you from executing your GPU code comfortably (without swapping to disk). You should have enough RAM to comfortable work with your GPU. This means you should have at least the amount of RAM that matches your biggest GPU. For example, if you have a Titan RTX with 24 GB of memory you should have at least 24 GB of RAM. However, if you have more GPUs you do not necessarily need more RA

PCIe Lanes and Multi-GPU Parallelism

The problem with this “match largest GPU memory in RAM” strategy is that you might still fall short of RAM if you are processing large datasets. The best strategy here is to match your GPU and if you feel that you do not have enough RAM just buy some more.

A different strategy is influenced by psychology: Psychology tells us that concentration is a resource that is depleted over time. RAM is one of the few hardware pieces that allows you to conserve your concentration resource for more difficult programming problems. Rather than spending lots of time on circumnavigating RAM bottlenecks, you can invest your concentration on more pressing matters if you have more RAM. With a lot of RAM you can avoid those bott

值得单独记下的要点

• Research that is hunting state-of-the-art scores: >=11 GB
• Research that is hunting for interesting architectures: >=8 GB
• Any other research: 8 GB
• Startups: 8 GB (but check the specific application area for model sizes)
• Companies: 8 GB for prototyping, >=11 GB for training
• Forward and backward pass: 216 milliseconds (ms)
• 16 PCIe lanes CPU->GPU transfer: About 2 ms (1.1 ms theoretical)
• 8 PCIe lanes CPU->GPU transfer: About 5 ms (2.3 ms)

落地时建议先做的 5 件事

1. 用自己的模型和序列长度算显存账，不要只看官网 TFLOPS。
2. 量化前后都要有同一套评测：平均分和长尾失败都记。
3. 多卡训练先分清数据并行和模型并行，通信量写进预算。
4. 智能体工作流要有可复现实验，而不是一次演示通过。
5. 硬件选型写清接口、功耗和两年后还能不能买到。

和智能体产品怎么接

龙虾PRO做 OpenClaw 落地时，量化和多卡经验会直接变成推理成本和技能能否本地跑。数字员工和网关应共用同一套 GPU/精度预算，而不是每条链路各买各的卡。

本文侧重全链路风控方法论。落地时请用自身业务单据做回放验证，不要把示例阈值直接当生产策略。 相关：风控体检 · 方案资源