中英文实体识别数据集，中英文机器翻译数据集,中文分词数据集.zip

## NLP-dataset (General) * [Huggingface, datasets](https://huggingface.co/datasets) * [Awesome-Chinese-NLP, Chinese](https://github.com/crownpku/Awesome-Chinese-NLP) * [CLUEDatasetSearch, Chinese](https://github.com/CLUEbenchmark/CLUEDatasetSearch) * [funNLP, Chinese](https://github.com/fighting41love/funNLP) * [ChineseNLPCorpus1, Chinese](https://github.com/InsaneLife/ChineseNLPCorpus) * [ChineseNLPCorpus2, Chinese](https://github.com/SophonPlus/ChineseNlpCorpus) * [CLUE, Chinese](https://www.cluebenchmarks.com/introduce.html) * [Chinese NLP data by ShannonAI, Chinese](https://github.com/ShannonAI/glyce/blob/master/docs/dataset_download.md) * [nlp-datasets, Multilingual](https://github.com/niderhoff/nlp-datasets) * [awesome-nlp, Multilingual](https://github.com/keon/awesome-nlp#datasets) ## Word Segmentation (Chinese) * [SIGHAN2005](http://sighan.cs.uchicago.edu/bakeoff2005/) * [multi-criteria-cws](https://github.com/hankcs/multi-criteria-cws) * [Chinese NLP data by ShannonAI, Chinese](https://github.com/ShannonAI/glyce/blob/master/docs/dataset_download.md) ## NER dataset (English) * [various NER dataset](https://github.com/juand-r/entity-recognition-datasets) * [CoNLL-2003, Offical](https://www.clips.uantwerpen.be/conll2003/ner/), [CoNLL-2003, other link](https://github.com/synalp/NER/tree/master/corpus/CoNLL-2003) * [WNUT-2016, Twitter](https://github.com/aritter/twitter_nlp/tree/master/data/annotated/wnut16) * [OntoNotes-5.0, broadcase news, braodcase conversation, weblogs, magzine genre](https://github.com/yuchenlin/OntoNotes-5.0-NER-BIO) * [Wikigold](https://github.com/juand-r/entity-recognition-datasets/tree/master/data/wikigold) * [Twitter](https://github.com/aritter/twitter_nlp/blob/master/data/annotated/ner.txt) * [kaggle](https://www.kaggle.com/abhinavwalia95/entity-annotated-corpus/data) * [MUC6](https://catalog.ldc.upenn.edu/LDC2003T13) * [MUC7](https://catalog.ldc.upenn.edu/LDC2001T02) ## NER dataset (Chinese) - [MSRA, OntoNotes 4.0, Resume, Weibo](https://drive.google.com/file/d/1mDKkc2-8e4wXAuAnGiZMHI59UgVbl1q4/view) - [CLUENER](https://storage.googleapis.com/cluebenchmark/tasks/cluener_public.zip) - [RenMinRiBao](https://github.com/quincyliang/nlp-dataset/tree/master/ner-data/renMinRiBao) - [MSRA](https://github.com/quincyliang/nlp-dataset/tree/master/ner-data/MSRA) - [Boson](https://github.com/quincyliang/nlp-dataset/tree/master/ner-data/boson) - [Weibo](https://github.com/quincyliang/nlp-dataset/tree/master/ner-data/weibo) - [Others](https://github.com/OYE93/Chinese-NLP-Corpus/tree/master/NER) ## Machine Translation (Chinese-English) - [WMT 2020](http://statmt.org/wmt20/translation-task.html) - [AI challenger](https://challenger.ai/) (英中翻译规模最大的口语领域英中双语对照数据集) - [UM-Corpus: A Large English-Chinese Parallel Corpus](http://nlp2ct.cis.umac.mo/um-corpus/) - [OpenSubtitles2016](http://opus.nlpl.eu/OpenSubtitles2016.php) - [MultiUN](http://opus.nlpl.eu/MultiUN.php)