I gave a non-technical talk today to the local chapter of the Pi Mu Epsi…

后 Transformer 时代的判断不该是「哪个架构赢」，而是系统边界问题。本文用 Jamba、Griffin、NVIDIA 的受控对照实验说…

I often talk to startups that claim that their compensation package has …

Becoming an RL diehard in the past year and thinking about RL for most o…

The parity problem is a notorious problem in sieve theory: this theory w…

线性注意力把 softmax attention 改写成可累积的矩阵状态，RWKV 让训练走 Transformer 的路、推理走 RNN 的路…

I can’t think of a single large software company that doesn’t regularly …

I recently got a Tesla, and using full self-driving has been a wake up c…

For the last year or so, I’ve maintained two advice pages on my web site…

Transformer 用 KV Cache 保存全部历史，SSM 用固定状态压缩历史。本文讲清 S4 为何长程却难训、Mamba 的 sele…