令人惊叹的预训练中文NLP模型，高质量的中文预训练模型&大型模型&多模态模型&大型语言模型集合.zip

# Awesome Pretrained Chinese NLP Models[](https://awesome.re)  <div align="center"> <a href="https://arxiv.org/pdf/2303.18223.pdf">论文: A Survey of Large Language Models</a> </div> 在自然语言处理领域中，预训练语言模型（Pretrained Language Models）已成为非常重要的基础技术，本仓库主要收集目前网上公开的一些高质量中文预训练模型、中文多模态模型、中文大语言模型等内容(感谢分享资源的大佬)，并将持续更新...... > 国内下载HuggingFace仓库模型推荐使用HuggingFace镜像地址: https://hf-mirror.com/ # Expand Table of Contents + [更新日志](#更新) + [通用基础大模型](#Base-LLM) + [垂直基础大模型](#Domain-Base-LLM) + [通用对话大模型](#ChatLLM) + [垂直对话大模型](#Domain-ChatLLM) + [多模态对话大模型](#MultiModal-ChatLLM) + [大模型评估基准](#大模型评估基准) + [在线体验大模型](#在线体验大模型) + [开源模型库平台](#开源模型库平台) + [开源数据集库](#开源数据集库) + [开源中文指令数据集](#中文指令数据集) + [Embedding](#Embedding) + [Other-Awesome](#other-awesome) + <details><summary>NLU系列</summary> - [BERT](#BERT) - [RoBERTa](#RoBERTa) - [ALBERT](#ALBERT) - [NEZHA](#NEZHA) - [XLNET](#XLNET) - [MacBERT](#MacBERT) - [WoBERT](#WoBERT) - [ELECTRA](#ELECTRA) - [ZEN](#ZEN) - [ERNIE](#ERNIE) - [ERNIE3](#ERNIE3) - [RoFormer](#RoFormer) - [StructBERT](#StructBERT) - [Lattice-BERT](#Lattice-BERT) - [Mengzi-BERT](#Mengzi-BERT) - [ChineseBERT](#ChineseBERT) - [TaCL](#TaCL) - [MC-BERT](#MC-BERT) - [二郎神](#二郎神) - [PERT](#PERT) - [MobileBERT](#MobileBERT) - [GAU-α](#GAU-α) - [DeBERTa](#DeBERTa) - [GlyphBERT](#GlyphBERT) - [CKBERT](#CKBERT) - [LERT](#LERT) - [RoCBert](#RoCBert) - [m3e](#M3E) - [LEALLA](#LEALLA) </details> + <details><summary>NLG系列</summary> - [GPT](#GPT) - [GPT-3](#GPT-3) - [NEZHA-GEN](#NEZHA-GEN) - [CPM-Generate](#CPM-Generate) - [T5](#T5) - [T5-PEGASUS](#T5-PEGASUS) - [Mengzi-T5](#Mengzi-T5) - [盘古α](#PanGu-Alpha) - [EVA](#EVA) - [BART](#BART) - [闻仲](#闻仲) - [余元](#余元) - [RWKV](#RWKV) - [Bloom](#Bloom) - [PromptCLUE](#PromptCLUE) - [ChatYuan](#ChatYuan) - [SkyText](#SkyText) - [ProphetNet](#ProphetNet) </details> + <details><summary>NLU-NLG系列</summary> - [UniLM](#UniLM) - [Simbert](#Simbert) - [RoFormer-sim](#RoFormer-sim) - [CPM-2](#CPM-2) - [CPT](#CPT) - [周文王](#周文王) - [GLM](#GLM) - [PLUG](#PLUG) - [OPD](#OPD) </details> + <details><summary>Multi-Modal</summary> - [WenLan](#WenLan) - [CogView](#CogView) - [紫东太初](#紫东太初) - [Mengzi-oscar](#Mengzi-oscar) - [R2D2](#R2D2) - [Chinese-CLIP](#Chinese-CLIP) - [TaiYi-CLIP](#TaiYi-CLIP) - [AltCLIP](#AltCLIP) - [AltDiffusion](#AltDiffusion) - [Taiyi-Stable-Diffusion](#Taiyi-Stable-Diffusion) - [wukong](#wukong) - [OFA](#OFA) - [QA-CLIP](#QA-CLIP) </details> + <details><summary>Table</summary> - [SDCUP](#SDCUP) </details> ` 备注` >ND: Non-Causal Decoder or Prefix LM >CD: Causal Decoder >ED: Encoder-Decoder ## Base-LLM > 大规模基础模型：表格中只罗列出参数量`大于7B`以上模型。 | 模型 | 大小 | 时间 | 语言 | 领域 | 下载 | 项目地址 | 机构/个人 | 架构 | 文献 | 备注 | | :--------: | :------: | :-------: | :----: | :----: | :-----------: | :------: | :---------------: | :--: | :--------------: | ----- | | XVERSE-MoE | 255B/A36B | 2024-09 | 中英 | 通用 | [🤗HF](https://huggingface.co/xverse/XVERSE-MoE-A36B) | [XVERSE-MoE-A36B](https://github.com/xverse-ai/XVERSE-MoE-A36B) | [xverse-ai](https://github.com/xverse-ai) | MoE | | | Qwen-2.5 | 0.5/1.5/3/7/14/32/72B | 2024-09 | 中英 | 通用 | [🤗HF](https://huggingface.co/collections/Qwen/qwen25-66e81a666513e518adb90d9e) | [Qwen2.5](https://github.com/QwenLM/Qwen2.5) | [QwenLM](https://github.com/QwenLM) | CD | [Blog](https://qwenlm.github.io/blog/qwen2.5/) | | | Tele-FLM | 52B/102B/1TB | 2024-07 | 多语 | 通用 | [[🤗HF\]](https://huggingface.co/CofeAI) | / | [CofeAI](https://huggingface.co/CofeAI) | CD | [Tele-FLM Technical Report](https://arxiv.org/pdf/2404.16645) | | meta-llama-3.1 | 8/70/405B | 2024-07 | 多语 | 通用 | [[🤗HF\]](https://huggingface.co/meta-llama) | [llama3](https://github.com/meta-llama/llama3) | [meta-llama](https://github.com/meta-llama) | CD | | | | internlm2.5-Base | 7B | 2024-07 | 中英 | 通用 | [[🤗HF\]](https://huggingface.co/internlm) | [InternLM](https://github.com/InternLM/InternLM)[](https://camo.githubusercontent.com/f330929a514fa88e296d3f4aa78863614ccc13d6d1903e4d7b23fd85b69cddba/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f73746172732f496e7465726e4c4d2f496e7465726e4c4d2e7376673f7374796c653d736f6369616c266c6162656c3d53746172) | [InternLM](https://github.com/InternLM) | CD | [📜Technical Report](https://arxiv.org/abs/2403.17297) | | | MAP-NEO-Base | 2/7B | 2024-06 | 中英 | 通用 | [🤗HF](https://huggingface.co/collections/m-a-p/neo-models-66395a5c9662bb58d5d70f04) | [MAP-NEO](https://github.com/multimodal-art-projection/MAP-NEO) | [multimodal-art-projection](https://github.com/multimodal-art-projection) | CD | [Paper](https://arxiv.org/abs/2405.19327) | | | Nemotron-4-Base | 340B | 2024-06 | 多语 | 通用 | [🤗HF](https://huggingface.co/nvidia) | / | [NVIDIA](https://github.com/NVIDIA) | CD | [technical report](https://research.nvidia.com/publication/2024-06_nemotron-4-340b). | | | Index-Base | 1.9B | 2024-06 | 中英 | 通用 | [🤗HF](https://huggingface.co/IndexTeam/Index-1.9B-Chat) | [Index-1.9B](https://github.com/bilibili/Index-1.9B) | [bilibili](https://github.com/bilibili) | CD | [Report](https://github.com/bilibili/Index-1.9B/blob/main/Index-1.9B%20%E6%8A%80%E6%9C%AF%E6%8A%A5%E5%91%8A.pdf) | | | Qwen2-Base | 0.5/2/5/7/72B | 2024-06 | 多语 | 通用 | [🤗HF](https://huggingface.co/Qwen) | [Qwen2](https://github.com/QwenLM/Qwen2) | [QwenLM](https://github.com/QwenLM) | CD | [Blog](https://qwenlm.github.io/) | | | GLM-4-Base | 9B | 2024-06 | 多语 | 通用 | [🤗HF](https://huggingface.co/THUDM) | [GLM-4](https://github.com/THUDM/GLM-4) | [THUDM](https://github.com/THUDM) | / | | | | Yi-1.5-Base | 6/9/34B | 2024-05 | 中英 | 通用 | [🤗HF](https://huggingface.co/01-ai) | [Yi-1.5](https://github.com/01-ai/Yi-1.5) | [01-ai](https://github.com/01-ai) | CD | [Paper](https://arxiv.org/abs/2403.04652) | | | DeepSeek-V2-Base | A21B/236B | 2024-05 | 中英 | 通用 | [🤗HF](https://huggingface.co/deepseek-ai/DeepSeek-V2) | [DeepSeek-V2](https://github.com/deepseek-ai/DeepSeek-V2) | [deepseek-ai](https://github.com/deepseek-ai) | MOE | [Paper](https://github.com/deepseek-ai/DeepSeek-V2/blob/main/deepseek-v2-tech-report.pdf) | | | Llama-3-Base | 8/70B | 2024-04 | 多语 | 通用 | [🤗HF](https://hf-mirror.com/meta-llama) | **[llama3](https://github.com/meta-llama/llama3)** | [Meta Llama](https://github.com/meta-llama) | CD | | | | Zhinao-Base | 7B | 2024-04 | 中英 | 通用 | [🤗HF](https://huggingface.co/qihoo360) [ 🤖](https://www.modelscope.cn/models/qihoo360/36