《语音与语言处理》第三版(草稿)-《SpeechandLanguageProcessing,3rdeditiondraft》

5星 · 超过95%的资源需积分: 19 186 浏览量 2019-07-02 13:58:56 上传评论 2 收藏 18.33MB PDF 举报

资源推荐

资源详情

资源评论

Speech and Language Processing

An Introduction to Natural Language Processing,

Computational Linguistics, and Speech Recognition

Third Edition draft

Daniel Jurafsky

Stanford University

James H. Martin

University of Colorado at Boulder

2018

Draft of September 23, 2018. Comments and typos welcome!

Summary of Contents

1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2 Regular Expressions, Text Normalization, Edit Distance. . . . . . . . . 10

3 N-gram Language Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

4 Naive Bayes and Sentiment Classiﬁcation . . . . . . . . . . . . . . . . . . . . . . . 63

5 Logistic Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

6 Vector Semantics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

7 Neural Networks and Neural Language Models . . . . . . . . . . . . . . . . . 131

8 Part-of-Speech Tagging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

9 Sequence Processing with Recurrent Networks. . . . . . . . . . . . . . . . . . 177

10 Formal Grammars of English . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

11 Syntactic Parsing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223

12 Statistical Parsing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237

13 Dependency Parsing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270

14 The Representation of Sentence Meaning . . . . . . . . . . . . . . . . . . . . . . . 295

15 Computational Semantics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325

16 Semantic Parsing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326

17 Information Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327

18 Semantic Role Labeling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356

19 Lexicons for Sentiment, Affect, and Connotation. . . . . . . . . . . . . . . . 378

20 Coreference Resolution and Entity Linking . . . . . . . . . . . . . . . . . . . . . 399

21 Discourse Coherence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .400

22 Machine Translation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401

23 Question Answering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402

24 Dialog Systems and Chatbots. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .422

25 Advanced Dialog Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446

26 Speech Recognition and Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461

Appendix 463

A Hidden Markov Models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464

B Spelling Correction and the Noisy Channel . . . . . . . . . . . . . . . . . . . . . 480

C WordNet: Word Relations, Senses, and Disambiguation . . . . . . . . . 493

Bibliography. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .517

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 543

Subject Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 551

Contents

1 Introduction 9

2 Regular Expressions, Text Normalization, Edit Distance 10

2.1 Regular Expressions . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.2 Words . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.3 Corpora . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2.4 Text Normalization . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.5 Minimum Edit Distance . . . . . . . . . . . . . . . . . . . . . . . 30

2.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

Bibliographical and Historical Notes . . . . . . . . . . . . . . . . . . . . 34

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3 N-gram Language Models 37

3.1 N-Grams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

3.2 Evaluating Language Models . . . . . . . . . . . . . . . . . . . . 43

3.3 Generalization and Zeros . . . . . . . . . . . . . . . . . . . . . . 45

3.4 Smoothing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

3.5 Kneser-Ney Smoothing . . . . . . . . . . . . . . . . . . . . . . . 53

3.6 The Web and Stupid Backoff . . . . . . . . . . . . . . . . . . . . 55

3.7 Advanced: Perplexity’s Relation to Entropy . . . . . . . . . . . . 56

3.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

Bibliographical and Historical Notes . . . . . . . . . . . . . . . . . . . . 60

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

4 Naive Bayes and Sentiment Classiﬁcation 63

4.1 Naive Bayes Classiﬁers . . . . . . . . . . . . . . . . . . . . . . . 65

4.2 Training the Naive Bayes Classiﬁer . . . . . . . . . . . . . . . . . 67

4.3 Worked example . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

4.4 Optimizing for Sentiment Analysis . . . . . . . . . . . . . . . . . 70

4.5 Naive Bayes for other text classiﬁcation tasks . . . . . . . . . . . 71

4.6 Naive Bayes as a Language Model . . . . . . . . . . . . . . . . . 72

4.7 Evaluation: Precision, Recall, F-measure . . . . . . . . . . . . . . 73

4.8 Test sets and Cross-validation . . . . . . . . . . . . . . . . . . . . 76

4.9 Statistical Signiﬁcance Testing . . . . . . . . . . . . . . . . . . . 77

4.10 Advanced: Feature Selection . . . . . . . . . . . . . . . . . . . . 79

4.11 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

Bibliographical and Historical Notes . . . . . . . . . . . . . . . . . . . . 80

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

5 Logistic Regression 82

5.1 Classiﬁcation: the sigmoid . . . . . . . . . . . . . . . . . . . . . 83

5.2 Learning in Logistic Regression . . . . . . . . . . . . . . . . . . . 87

5.3 The cross-entropy loss function . . . . . . . . . . . . . . . . . . . 88

5.4 Gradient Descent . . . . . . . . . . . . . . . . . . . . . . . . . . 89

5.5 Regularization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

5.6 Multinomial logistic regression . . . . . . . . . . . . . . . . . . . 95

5.7 Interpreting models . . . . . . . . . . . . . . . . . . . . . . . . . 97

5.8 Advanced: Deriving the Gradient Equation . . . . . . . . . . . . . 98

5.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

4 CONTENTS

Bibliographical and Historical Notes . . . . . . . . . . . . . . . . . . . . 99

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

6 Vector Semantics 101

6.1 Lexical Semantics . . . . . . . . . . . . . . . . . . . . . . . . . . 102

6.2 Vector Semantics . . . . . . . . . . . . . . . . . . . . . . . . . . 106

6.3 Words and Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . 108

6.4 Cosine for measuring similarity . . . . . . . . . . . . . . . . . . . 111

6.5 TF-IDF: Weighing terms in the vector . . . . . . . . . . . . . . . 112

6.6 Applications of the tf-idf vector model . . . . . . . . . . . . . . . 115

6.7 Optional: Pointwise Mutual Information (PMI) . . . . . . . . . . . 116

6.8 Word2vec . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

6.9 Visualizing Embeddings . . . . . . . . . . . . . . . . . . . . . . . 123

6.10 Semantic properties of embeddings . . . . . . . . . . . . . . . . . 123

6.11 Bias and Embeddings . . . . . . . . . . . . . . . . . . . . . . . . 125

6.12 Evaluating Vector Models . . . . . . . . . . . . . . . . . . . . . . 126

6.13 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

Bibliographical and Historical Notes . . . . . . . . . . . . . . . . . . . . 127

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

7 Neural Networks and Neural Language Models 131

7.1 Units . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

7.2 The XOR problem . . . . . . . . . . . . . . . . . . . . . . . . . . 134

7.3 Feed-Forward Neural Networks . . . . . . . . . . . . . . . . . . . 137

7.4 Training Neural Nets . . . . . . . . . . . . . . . . . . . . . . . . 140

7.5 Neural Language Models . . . . . . . . . . . . . . . . . . . . . . 145

7.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149

Bibliographical and Historical Notes . . . . . . . . . . . . . . . . . . . . 149

8 Part-of-Speech Tagging 151

8.1 (Mostly) English Word Classes . . . . . . . . . . . . . . . . . . . 151

8.2 The Penn Treebank Part-of-Speech Tagset . . . . . . . . . . . . . 154

8.3 Part-of-Speech Tagging . . . . . . . . . . . . . . . . . . . . . . . 156

8.4 HMM Part-of-Speech Tagging . . . . . . . . . . . . . . . . . . . 157

8.5 Maximum Entropy Markov Models . . . . . . . . . . . . . . . . . 167

8.6 Bidirectionality . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

8.7 Part-of-Speech Tagging for Other Languages . . . . . . . . . . . . 172

8.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173

Bibliographical and Historical Notes . . . . . . . . . . . . . . . . . . . . 174

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

9 Sequence Processing with Recurrent Networks 177

9.1 Simple Recurrent Networks . . . . . . . . . . . . . . . . . . . . . 177

9.2 Applications of RNNs . . . . . . . . . . . . . . . . . . . . . . . . 184

9.3 Deep Networks: Stacked and Bidirectional RNNs . . . . . . . . . 186

9.4 Managing Context in RNNs: LSTMs and GRUs . . . . . . . . . . 188

9.5 Words, Characters and Byte-Pairs . . . . . . . . . . . . . . . . . . 192

9.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

10 Formal Grammars of English 194

10.1 Constituency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

10.2 Context-Free Grammars . . . . . . . . . . . . . . . . . . . . . . . 195

剩余558页未读，继续阅读

评论收藏

内容反馈

_pinnacle_

2019-12-26

不错，要是正式版就更好了

解惑者小双子

粉丝: 0
资源: 98

《语音与语言处理》第三版(草稿)-《Speech and Language Processing, 3rd edition dr...

最新资源

《语音与语言处理》第三版(草稿)-《Speech and Language Processing, 3rd edition dr...

Speech and Language Processing draft en 3rd edition , 自然语言处理综论 第三版 9/23, 2018

SPEECH and LANGUAGE PROCESSING

Speech and Language Processing

斯坦福自然语言经典教材 Speech and Language Processing 第三版发布重大更新

Speech and Language Processing（语音和语言处理）

2017年Dan Jurafsky and James H. Martin经典力作Speech and Language Processing (第三版)

语音和语言处理英文版第3版带书签

Speech and Language Processing, 2nd - 2008 PDF 非草稿

Speech and Language Processing 2nd edition.pdf

自然语言处理综述第三版

语音与自然语言处理（Speech and Language Processing）

Speech.and.Language.Processing.pdf

Speech_and_ Language_ Processing_ed3book_jan122022.pdf

speech and language processing 2nd

Speech and Language Processing, 2nd Edition.pdf

Speech and Language Processing 斯坦福大学教程pdf

speech and language processing 2nd edition

Speech and Language Processing 2nd 文字版非扫描非草稿

Speech and Language Processing 2nd edition

Speech and Language Processing_2nd_draft

Speech and Language Processing 2nd

增强学习导论代码Reinforcement Learning - An Introduction(Second edition, Draft)

CSS The Definitive Guide 3rd Edition

jurafsky and martin speech and language processing 2nd edition pdf download

ChatGPT教程（终极版）最全整理

博客中Kmeans以及FCM算法数据（免积分）

最新资源

Speech and Language Processing draft en 3rd edition , 自然语言处理综论第三版 9/23, 2018