FlashAttention 优化的是 full attention 这张完全图的 IO 路径，稀疏 attention 直接改写连接图本身。本…

The other day, I saw a viral tweet saying that people talking about how …

[This post is authored by Alexandre Borovik. An extended version of this…

FlashAttention 的关键不是近似注意力，也不是把公式改掉，而是重新安排标准 attention 在 GPU 内存层级里的计算路径。本…

Most real-world problems are big enough that you can’t just head for the…

Atle Selberg, who made immense and fundamental contributions to analytic…

RoPE 把位置从”加在输入上的向量”焊进了 Q·Kᵀ 本身——本文推导旋转后点积为何只依赖相对位置，核对高频通道在长距离上的绕圈失真、RoPE…

One of the most common mistakes I see people make when looking at data i…

[This post is authored by Gil Kalai, who has kindly “guest blogged” this…

Encoder-only、Encoder-Decoder、Decoder-only 的差异不是谁更聪明，而是谁能看见谁的信息流不变量。本文从可见…