<div align="center">
<h1>ð£ï¸ Large Language Model Course</h1>
<p align="center">
ð¦ <a href="https://twitter.com/maximelabonne">Follow me on X</a> â¢
ð¤ <a href="https://huggingface.co/mlabonne">Hugging Face</a> â¢
ð» <a href="https://mlabonne.github.io/blog">Blog</a> â¢
ð <a href="https://github.com/PacktPublishing/Hands-On-Graph-Neural-Networks-Using-Python">Hands-on GNN</a> â¢
ð£ï¸ <a href="https://chat.openai.com/g/g-yviLuLqvI-llm-course">Interactive GPT</a>
</p>
</div>
<br/>
The LLM course is divided into three parts:
1. 𧩠**LLM Fundamentals** covers essential knowledge about mathematics, Python, and neural networks.
2. ð§âð¬ **The LLM Scientist** focuses on building the best possible LLMs using the latest techniques.
3. ð· **The LLM Engineer** focuses on creating LLM-based applications and deploying them.
## ð Notebooks
A list of notebooks and articles related to large language models.
### Tools
| Notebook | Description | Notebook |
|----------|-------------|----------|
| ð§ [LLM AutoEval](https://github.com/mlabonne/llm-autoeval) | Automatically evaluate your LLMs using RunPod | <a href="https://colab.research.google.com/drive/1Igs3WZuXAIv9X0vwqiE90QlEPys8e8Oa?usp=sharing"><img src="img/colab.svg" alt="Open In Colab"></a> |
| 𥱠LazyMergekit | Easily merge models using mergekit in one click. | <a href="https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing"><img src="img/colab.svg" alt="Open In Colab"></a> |
| â¡ AutoGGUF | Quantize LLMs in GGUF format in one click. | <a href="https://colab.research.google.com/drive/1P646NEg33BZy4BfLDNpTz0V0lwIU3CHu?usp=sharing"><img src="img/colab.svg" alt="Open In Colab"></a> |
| ð³ Model Family Tree | Visualize the family tree of merged models. | <a href="https://colab.research.google.com/drive/1s2eQlolcI1VGgDhqWIANfkfKvcKrMyNr?usp=sharing"><img src="img/colab.svg" alt="Open In Colab"></a> |
### Fine-tuning
| Notebook | Description | Article | Notebook |
|---------------------------------------|-------------------------------------------------------------------------|---------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------|
| Fine-tune Llama 2 in Google Colab | Step-by-step guide to fine-tune your first Llama 2 model. | [Article](https://mlabonne.github.io/blog/posts/Fine_Tune_Your_Own_Llama_2_Model_in_a_Colab_Notebook.html) | <a href="https://colab.research.google.com/drive/1PEQyJO1-f6j0S_XJ8DV50NkpzasXkrzd?usp=sharing"><img src="img/colab.svg" alt="Open In Colab"></a> |
| Fine-tune LLMs with Axolotl | End-to-end guide to the state-of-the-art tool for fine-tuning. | [Article](https://mlabonne.github.io/blog/posts/A_Beginners_Guide_to_LLM_Finetuning.html) | <a href="https://colab.research.google.com/drive/1Xu0BrCB7IShwSWKVcfAfhehwjDrDMH5m?usp=sharing"><img src="img/colab.svg" alt="Open In Colab"></a> |
| Fine-tune Mistral-7b with DPO | Boost the performance of supervised fine-tuned models with DPO. | [Article](https://medium.com/towards-data-science/fine-tune-a-mistral-7b-model-with-direct-preference-optimization-708042745aac) | <a href="https://colab.research.google.com/drive/15iFBr1xWgztXvhrj5I9fBv20c7CFOPBE?usp=sharing"><img src="img/colab.svg" alt="Open In Colab"></a> |
### Quantization
| Notebook | Description | Article | Notebook |
|---------------------------------------|-------------------------------------------------------------------------|---------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------|
| 1. Introduction to Quantization | Large language model optimization using 8-bit quantization. | [Article](https://mlabonne.github.io/blog/posts/Introduction_to_Weight_Quantization.html) | <a href="https://colab.research.google.com/drive/1DPr4mUQ92Cc-xf4GgAaB6dFcFnWIvqYi?usp=sharing"><img src="img/colab.svg" alt="Open In Colab"></a> |
| 2. 4-bit Quantization using GPTQ | Quantize your own open-source LLMs to run them on consumer hardware. | [Article](https://mlabonne.github.io/blog/4bit_quantization/) | <a href="https://colab.research.google.com/drive/1lSvVDaRgqQp_mWK_jC9gydz6_-y6Aq4A?usp=sharing"><img src="img/colab.svg" alt="Open In Colab"></a> |
| 3. Quantization with GGUF and llama.cpp | Quantize Llama 2 models with llama.cpp and upload GGUF versions to the HF Hub. | [Article](https://mlabonne.github.io/blog/posts/Quantize_Llama_2_models_using_ggml.html) | <a href="https://colab.research.google.com/drive/1pL8k7m04mgE5jo2NrjGi8atB0j_37aDD?usp=sharing"><img src="img/colab.svg" alt="Open In Colab"></a> |
| 4. ExLlamaV2: The Fastest Library to Run LLMs | Quantize and run EXL2 models and upload them to the HF Hub. | [Article](https://mlabonne.github.io/blog/posts/ExLlamaV2_The_Fastest_Library_to_Run%C2%A0LLMs.html) | <a href="https://colab.research.google.com/drive/1yrq4XBlxiA0fALtMoT2dwiACVc77PHou?usp=sharing"><img src="img/colab.svg" alt="Open In Colab"></a> |
### Other
| Notebook | Description | Article | Notebook |
|---------------------------------------|-------------------------------------------------------------------------|---------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------|
| Decoding Strategies in Large Language Models | A guide to text generation from beam search to nucleus sampling | [Article](https://mlabonne.github.io/blog/posts/2022-06-07-Decoding_strategies.html) | <a href="https://colab.research.google.com/drive/19CJlOS5lI29g-B3dziNn93Enez1yiHk2?usp=sharing"><img src="img/colab.svg" alt="Open In Colab"></a> |
| Visualizing GPT-2's Loss Landscape | 3D plot of the loss landscape based on weight perturbations. | [Tweet](https://twitter.com/maximelabonne/status/1667618081844219904) | <a href="https://colab.research.google.com/drive/1Fu1jikJzFxnSPzR_V2JJyDVWWJNXssaL?usp=sharing"><img src="img/colab.svg" alt="Open In Colab"></a> |
| Improve ChatGPT with Knowledge Graphs | Augment ChatGPT's answers with knowledge graphs. | [Article](https://mlabonne.github.io/blog/posts/Article_Improve_ChatGPT_with_Knowledge_Graphs.html) | <a href="https://colab.research.google.com/drive/1mwhOSw9Y9bgEaIFKT4CLi0n18pXRM4cj?usp=sharing"><img src="img/colab.svg" alt="Open In Colab"></a> |
| Merge LLMs with mergekit | Create your own models easily, no GPU required! | [Article](https://towardsdatascience.com/merge-large-language-models-with-mergekit-2118fb392b54) | <a href="https://colab.research.google.com/drive/1_JS7JKJAQozD48-LhYdegcuuZ2ddgXfr?usp=sharing"><img src="img/colab.svg" alt="Open In Colab"></a> |
## 𧩠LLM Fundamentals
![](img/roadmap_fundamentals.png)
### 1. Mathematics for Machine Learning
Before mastering machine learning, it is important to understand the fundamental mathematical concepts that power these algorithms.
- **Linear Algebra**: This is crucial for understanding many algorithms, especially those used in deep learning. Key concepts include vectors, matrices, determinants, eigenvalues and eigenvectors, vector spaces, and linear transformations.
- **Calculus**: Many machine learning algorithms involve the optimization of continuous functions, which requires an understanding of derivatives, integrals, limits, and series. Multivariable calculus and the concept of gradients are also important.
- **Probability and Statistics**: These are crucial for understanding how models learn from data and make predictions. Key concepts include probability theory, random variables, probability distributions, expectations, varia
没有合适的资源?快使用搜索试试~ 我知道了~
温馨提示
大模型技术文章精选合集 项目标签:[机器学习] [深度学习] [大规模语言模型] [LLM] [计算机科学] 简介: 随着人工智能技术的飞速发展,大规模语言模型(LLM)成为了近年来备受瞩目的技术焦点。为了满足广大研发人员对大模型技术的深入学习和研究需求,我们精心策划并整理了“大模型技术文章精选合集”。这一合集汇集了图神经网络、大型语言模型和凸优化等多个技术领域的优质文章,旨在为广大研发人员提供一个系统、全面、深入的技术学习平台。 推荐理由: 内容丰富:合集涵盖了图神经网络的原理、大型语言模型的应用以及凸优化在相关领域中的重要性,内容全面且深入,满足不同领域研发人员的学习需求。 技术前沿:所选文章均来自业界知名专家和技术大咖,内容前沿、权威,确保读者能够接触到最新、最先进的技术动态。 实用性强:合集不仅提供了深入的技术讨论,还结合实际案例和应用场景,帮助读者更好地理解和应用大模型技术。 学习便捷:合集采用电子文档形式,方便读者随时随地查阅和学习,提高学习效率。 如何学习: 浏览合集:首先,建议读者浏览整个合集,了解各个文章的主题和内容,以便选择自己感兴趣的技术领域进行深入学
资源推荐
资源详情
资源评论
收起资源包目录
llm-course-main.zip (15个子文件)
llm-course-main
4_bit_LLM_Quantization_with_GPTQ.ipynb 9KB
Introduction_to_Weight_Quantization.ipynb 587KB
Quantize_Llama_2_models_using_GGUF_and_llama_cpp.ipynb 173KB
Quantize_models_with_ExLlamaV2.ipynb 513KB
Fine_tune_a_Mistral_7b_model_with_DPO.ipynb 27KB
LICENSE 11KB
Fine_tune_Llama_2_in_Google_Colab.ipynb 118KB
img
roadmap_fundamentals.png 150KB
colab.svg 2KB
roadmap_engineer.png 246KB
roadmap_scientist.png 304KB
Mergekit.ipynb 62KB
Fine_tune_LLMs_with_Axolotl.ipynb 77KB
Decoding_Strategies_in_Large_Language Models.ipynb 3.02MB
README.md 47KB
共 15 条
- 1
资源评论
全栈海哥
- 粉丝: 1424
- 资源: 99
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功