Deep Learning for Natural Language Processing

所需积分/C币:9 2018-11-25 18:11:45 7.21MB PDF
26
收藏 收藏
举报

We are awash with text, from books, papers, blogs, tweets, news, and increasingly text from spoken utterances. Every day, I get questions asking how to develop machine learning models for text data. Working with text is hard as it requires drawing upon knowledge from diverse domains such as linguistics, machine learning, statistical natural language processing, and these days, deep learning. I have done my best to write blog posts to answer frequently asked questions on the topic and decided to pull together my best knowledge on the matter into this book. I designed this book to teach you step-by-step how to bring modern deep learning methods to your natural language processing projects. I chose the programming language, programming libraries, and tutorial topics to give you the skills you need. Python is the go-to language for applied machine learning and deep learning, both in terms of demand from employers and employees. This is not least because it could be a renaissance for machine learning tools. I have focused on showing you how to use the best of breed Python tools for natural language processing such as Gensim and NLTK, and even a little scikit-learn. Key to getting results is speed of development, and for this reason, we use the Keras deep learning library as you can define, train, and use complex deep learning models with just a few lines of Python code. There are three key areas that you must know when working with text: 1. How to clean text. This includes loading, analyzing, filtering and cleaning tasks required prior to modeling. 2. How to represent text. This includes the classical bag-of-words model and the modern and powerful distributed representation in word embeddings. 3. How to generate text. This includes the range of most interesting problems, such as image captioning and translation. These key topics provide the backbone for the book and the tutorials you will work through. I believe that after completing this book, you will have the skills t
Contents Copyright ontents Preface I Introductions Welcome Who is this book for? About your outcomes How to read this book about the book structure About Python Code examples About, Further Reading About getting help II Foundations 1 Natural Language processing 1.1 NaturaI Language 1.2 Challenge of Natural Language 1.3 From Linguistics to Natural Language Processin 1.4 Natural Language Processing 1.5 Further reading 6 1.6 Summary 2 Deep Learning 2. 1 Deep learning is Large Neural Networks 2.2 Deep Learning is Hierarchical Feature Learning 2.3 Deep Learning as Scalable Learning Across Domains 12 2.4 Further Reading 13 2.5 Summary 14 CONTENTS 3 Promise of Deep Learning for Natural Language 16 3.1 Promise of Deep Learning 16 3.2 Promise of Drop-in Replacement Models 3. 3 Promise of New NLP models 3.4 Promise of feature learnin 1 8 3.5 Promise of Continued Improvement .18 3.6 Promise of End-to-End Models 19 3.7 Further reading 3. 8 Summary 4 How to Develop Deep Learning Models With Keras 21 4.1 Keras Model Life-Cycle 21 4.2 Keras functional models 26 4.3 Standard Network models 27 4.4 Further Reading .32 4.5 Summary III Data Preparation 34 5 How to Clean Text Manually and with NLTK 35 5.1 Tutorial Overvieτ 5.2 Metamorphosis by Franz Kafka 5.3 Text Cleaning Is Task specific 5.4 Manual Tokenization 5.5 Tokenization and Cleaning with NLTK 5.6 Additional Text Cleaning Considerations 46 5. 7 Further reading 46 5.8 Summary 6 How to Prepare Text Data with scikit-learn 48 6.1 The Bag-of-Words Model .48 6.2 Word Counts with Count Vectorizer 49 6. 3 Word Frequencies with Tfidf Vectorizer 6.4 Lashing with HashingVectorizer 6.5 Further readin 52 6.6 Summary 53 7 How to Prepare Text Data With Keras 54 7.1 Tutorial O 54 7. 2 Split Words with text-to -word_sequence 7.3 Encoding with one hot 7.4 Hash Encoding with hashing-trick ..56 7.5 Tokenizer APi 57 7.6 Further Reading 7. 7 Summary 59 CONTENTS I Bag-of-Words 61 8 The Bag-of-Words Model 62 8.1 Tutorial overview 62 8.2 The Problem with Text 8. 3 What is a Bag-of-Words? 8.4 Example of the Bag-of-Words Model 8.5 Managing Vocabulary 8.6 Scoring words ..66 8.7 Limitations of Bag-of-Words .67 8. 8 Further readin 67 8.9 Summary .68 9 How to Prepare Movie Review Data for Sentiment Analysis 69 9.1 Tutorial Overview 9.2 Movie Review Dataset 9.3 Load Text Data 71 9.4 Clean Text Data 9.5 Develop Vocabulary .76 9.6 Save Prepared Data 9.7 Further reading 9.8 Summary 84 10 Project: Develop a Neural Bag- of- Words Model for Sentiment Analysis 85 10.1 Tutorial overview 10.2 Movie Review Dataset 8 10.3 Data Preparation 10.4 Bag-of-Words Representation 6628 10.5 Sentiment Analysis Models 10.6 Comparing Word Scoring Methods ,,103 10.7 Predicting Sentiment for New reviews 10.8 Extensions 10.9 Further Reading 112 10.10Summary 113 V Word Embeddings 114 11 The Word Embedding model 115 11.1 Overview 115 11.2 What Are Word Embeddings? 115 11.3 Word Embedding Algorithms 116 11.4 USing Word Embeddings .119 11.5 Further Reading .120 11.6 Summary l21 CONTENTS 12 How to Develop Word Embeddings with Gensim 122 12.1 Tutorial overview 122 12.2 Word Embeddings .123 12.3 Gensim Python Library 123 12.4 Develop Word2 Vec Embedding 123 12.5 Visualize Word Embedding 126 12.6 Load Google's Word2 Vec Embedding 128 12.7 Load Stanfords Glo Ve embedding ..129 12.8 Further Reading l31 12.9 Summary ,,,.132 13 How to Learn and Load Word Embeddings in Keras 133 13.1 Tutorial Overview 133 13.2 Word embedding 134 13.3 Keras Embedding layer 134 13.4 Example of Learning an Embedding 135 13.5 Example of Using Pre-Trained Glo Ve Embedding .138 13.6 Tips for Cleaning Text for Word embedding .142 13.7 Further Reading ,142 13.8 Summary ,143 VI Text Classification 144 14 Neural models for Document classification 145 14.1 Overview ..145 14.2 Word Embeddings CNn- Text Classification 146 14.3 Use a Single Layer Cnn Architecture 14.4 Dial in CNN Hyperparameters 148 14.5 Consider Character-Level cNas ..,150 14.6 Consider Deeper CNNs for Classification .151 14.7 Further Reading 152 14. 8 Summary 152 15 Project: Develop an Embedding CNN Model for Sentiment Analysis 153 15.1 Tutorial Overview 153 15.2 Movie Review Dataset 153 15.3 Data Preparation 154 15.4 Train CNn With Embedding layer 15.5 Evaluate model .167 15.6 Extensions l71 15.7 Further Reading 15.8 Summary 173 CONTENTS 16 Project: Develop an n-gram CNn Model for Sentiment Analysis 174 16.1 Tutorial Overview 174 16.2 Movie review dataset .174 16.3 Data Preparation 16.4 Develop Multichannel Model 179 16.5 Evaluate model .185 16.6 Extensions l86 16.7 Further reading 16. 8 Summary 187 ⅵ II Language Modeling 189 17 Neural Language Modeling 190 17.1 Overview .190 17.2 Problem of Modeling Language 17.3 Statistical Language Modeling l91 17.4 Neural language models 192 17.5 Further Reading .194 17.6 Summary· 196 18 How to Develop a Character-Based Neural Language Model 197 18.1 Tutorial Overview 197 18.2 Sing a Song of Sixpence ..198 18.3 Data Preparation ..198 18.4 Train Language Model 18.5 Generate Text 20 186 Further Reading:.·· 18.7 Summary 209 19 How to Develop a Word-Based Neural Language Model 211 19.1 Tutorial Overview .211 19.2 Framing language modeling ,212 19.3 Jack and Jill Nursery Rhyme 19.4 Model 1: One-Word-In, One-Word-Out Sequences 212 19.5 Model 2: Line-by-Line Sequel 218 19.6 Model 3: Two-Words-In, One-Word-Out Sequence 222 19. 7 Further reading 224 19.8 Summary 225 20 Project: Develop a Neural Language Model for Text Generation 226 20.1 Tutorial overview 226 20.2 The Republic by Plato 227 20.3 Data Preparation .227 20.4 Train Language Model 233 20.5 USe Language model 239 20.6 Extensions .243 CONTENTS 20.7 Further Reading 244 20.8 Summary 244 ⅵ III Image Captioning 245 21 Neural Image Caption Generation 246 21.1 Overview 246 21.2 Describing an Image with Text 246 21.3 Neural Captioning model ,247 21. 4 Encoder-Decoder Architecture 249 21. 5 Further reading 250 21.6 Summary ..251 22 Neural Network Models for Caption Generation 252 22.1 Image Caption generation 22.2 Inject Model 253 22.3 Merge Model 254 22.4 More on the Merge Model 255 22.5 Further Reading 256 22.6 Summary 256 23 How to Load and Use a Pre-Trained Object Recognition Model 257 23.1 Tutorial Overview .257 23.2 ImageNet 258 23.3 The Oxford VGG Models 259 23.4 Load the vgg model in Keras 23.5 Develop a Simple Photo Classifier 263 23.6 Further Reading 266 23.7 Summary 266 24 How to evaluate generated Text with the bleu score 268 24.1 Tutorial Overview 24.2 Bilingual Evaluation Understudy Score 268 24.3 Calculate bleu Scores 270 24.4 Cumulative and Individual bleu Scores 271 24.5 Worked Examples 24.6 Further Reading 275 24.7 Summary 276 25 How to Prepare a Photo Caption Dataset For Modeling 277 25.1 Tutorial Overview .277 25.2 Download the Flickr8K Dataset 25.3 How to Load Photographs 25.4 Pre-Calculate Photo Features 280 25.5 How to Load Descriptions 25.6 Prepare Description Text 84 CONTENTS VI 25.7 Whole Description Sequence Model 287 25.8 Word-By-Word Model 290 25.9 Progressive Loading .293 25. 10Further Reading 25.sUMmary 26 Project: Develop a Neural Image Caption Generation Model 299 26.1 Tutorial Overview 299 26.2 Photo and Caption Dataset ..300 26.3 Prepare Photo Data .301 26.4 Prepare Text Data 26.5 Develop Deep Learning Model 303 .307 26.6 Evaluate model 3l8 26.7 Generate New Captions 324 26.8 Extensions 329 26.9 Further readin 329 26.10Summary .331 IX Machine translation 332 27 Neural machine ranslation 333 27.1 What is Machine Translation? 27.2 What is Statistical Machine Translation? ..334 27.3 What is Neural machine Translation? ..335 27.4 Further Reading .,,,..337 27.5 Summary 338 28 What are Encoder-Decoder Models for Neural machine Translation 339 28.1 Encoder-Decoder Architecture for NMT 339 28.2 Sutskever nmt Model 340 28.3 Cho NMr Model .342 28.4 Further reading ..345 28.5 Summary 346 29 How to Configure Encoder-Decoder Models for Machine Translation 347 29.1 Encoder-Decoder Model for Neural machine translation 347 29.2 Baseline model 348 29.3 Word Embedding Size 349 29. 4 RNN Cell Type ..349 29.5 Encoder-Decoder Depth 350 29.6 Direction of Encoder Input ,,,,,,,,.350 29.7 Attention Mechanism .351 29. 8 Inference .351 29.9 Final model 352 29. 10Further Reading 352 29.suMmary .353 CONTENTS 1 30 Project: Develop a Neural Machine Translation Model 354 30.1 Tutorial Overview 354 30.2 German to English Translation Dataset .354 30.3 Preparing the Text Data 355 30.4 Train Neural Translation Model 359 30.5 Evaluate Neural Translation model .36 30.6 Extensions 370 30.7 Further reading ..371 30.8 Summary X Appendix 373 A Getting Help 374 A 1 Official Keras Destinations A 2 Where to Get Help with Keras ..374 A. 3 Where to Get Hlelp with Natural language ...375 A 4 Ilow to Ask Questions A.5 Contact the authe B How to Setup a Workstation for Deep Learning 376 B.1 Overview 376 B 2 Download anaconda 376 B 3 Install anaconda B 4 Start and Update Anaconda 380 B5 Install Deep Learning Libr B 6 Further readil 384 B 7 Summary 384 How to use Deep learning in the Cloud 385 C 1 Overview 385 C 2 Setup Your Aws Account .386 C 3 Launch Your server Instance ..387 C4 Login, Configure and Run 391 C5 Build and run Models on aws 392 C 6 Close your ec2 instance 393 C 7 Tips and Tricks for Using Keras on AWS C8 Further Reading 395 C9 Summary 395 XI Conclusions 396 How Far You have Come 397

...展开详情
试读 127P Deep Learning for Natural Language Processing
立即下载
限时抽奖 低至0.43元/次
身份认证后 购VIP低至7折
一个资源只可评论一次,评论内容不能少于5个字
您会向同学/朋友/同事推荐我们的CSDN下载吗?
谢谢参与!您的真实评价是我们改进的动力~
上传资源赚钱or赚积分
最新推荐
Deep Learning for Natural Language Processing 9积分/C币 立即下载
1/127
Deep Learning for Natural Language Processing第1页
Deep Learning for Natural Language Processing第2页
Deep Learning for Natural Language Processing第3页
Deep Learning for Natural Language Processing第4页
Deep Learning for Natural Language Processing第5页
Deep Learning for Natural Language Processing第6页
Deep Learning for Natural Language Processing第7页
Deep Learning for Natural Language Processing第8页
Deep Learning for Natural Language Processing第9页
Deep Learning for Natural Language Processing第10页
Deep Learning for Natural Language Processing第11页
Deep Learning for Natural Language Processing第12页
Deep Learning for Natural Language Processing第13页
Deep Learning for Natural Language Processing第14页
Deep Learning for Natural Language Processing第15页
Deep Learning for Natural Language Processing第16页
Deep Learning for Natural Language Processing第17页
Deep Learning for Natural Language Processing第18页
Deep Learning for Natural Language Processing第19页
Deep Learning for Natural Language Processing第20页

试读结束, 可继续阅读

9积分/C币 立即下载