This is one of those “N technical things every programmer must read” lis…

I’ve just uploaded to the ArXiV my paper “Norm convergence of multiple e…

自回归推理里，历史 token 的 K/V 因果掩码下不变、可以缓存，Q 不行——本文给出可自证的归纳证明，推导显存公式与长上下文并发上限，拆解…

An acquaintance of mine, let’s call him Mike, is looking for work after …

I’ve added some new pages to this blog. Firstly, I’ve moved my old page …

语言模型每一步只输出一组 logits，真正决定输出哪个 token 的是解码策略。本文从 softmax 与 temperature 的几何直…

A common topic of discussion among my close friends is where the bottlen…

Last week, as mentioned previously, I attended a very inspiring interdis…

DiT 把扩散模型的去噪网络从 U-Net 换成 Transformer，靠的是把图像压成 latent patch token、用 adaLN…

I love reading postmortems. They’re educational, but unlike most educati…