从人类阅读时的眼动出发，把「注意力」拆成视觉生理、翻译对齐、加权平均三件事。讲清楚为什么权重必须满足非负与和为一、为什么 softmax 不是审…

The book starts with a story about a time Margaret Hamilton brought her …

This week I am visiting the University of Washington in Seattle, giving …

RNN 三难（长程依赖、梯度稳定、训练并行）的系统分析；attention 如何作为补丁逐步把 RNN 推向极限；Vaswani 2017 抛弃…

Last week, Quartz published an article titled “There is no gender gap in…

This is my second Milliman lecture, in which I talk about recent applica…

在 Transformer 出现之前，序列建模属于 RNN 的世界。本文从 Vanilla RNN 讲起，经过 BPTT、梯度消失爆炸、LSTM…

I’ve been reading a lot of predictions from people who are looking to un…

This is my final Milliman lecture, in which I talk about the sum-product…

embedding 是把离散的词变成稠密向量的桥梁。从 one-hot 的痛苦出发，经过 Firth 的分布假设、word2vec、GloVe、…