作者对word2vec的相关代码和算法做了相关调研，本文就是作者关于word2vec调研的总结，也是作为自己以后备用
目录
一、什么是 word2vec？ ..........................................................................................................2
二、快速入门...........................................................................................................................3
三、作者八卦...........................................................................................................................4
四、背景知识...........................................................................................................................5
4.1 词向量........................................................................................................................5
4.2 统计语言模型............................................................................................................5
4.3 NNLM ..........................................................................................................................7
4.4 其他 NNLM ................................................................................................................9
4.5 Log-Linear 模型...........................................................................................................9
4.6 Log-Bilinear 模型 ......................................................................................................10
4.6 层次化 Log-Bilinear 模型.........................................................................................10
五、模型.................................................................................................................................11
5.1 CBOW ........................................................................................................................11
5.2 Skip-Gram..................................................................................................................13
5.3 为什么要使用 Hierarchical Softmax 或 Negative Sampling....................................16
六、 Tricks................................................................................................................................17
6.1 指数运算..................................................................................................................17
6.2 按 word 分布随机抽样 ...........................................................................................18
6.3 哈希编码..................................................................................................................20
6.4 随机数......................................................................................................................20
6.5 回车符......................................................................................................................20
6.6 高频词亚采样..........................................................................................................21
七、分布式实现.....................................................................................................................21
八、总结.................................................................................................................................22
参考代码.................................................................................................................................22
参考文献.................................................................................................................................23
0
105
1.87MB
2016-11-28
9
