一文彻底搞懂BP算法：原理推导+数据演示+项目实战.zip

# 上篇 <div class="output_wrapper" id="output_wrapper_id" style="font-size: 16px; color: rgb(62, 62, 62); line-height: 1.6; word-spacing: 0px; letter-spacing: 0px; font-family: 'Helvetica Neue', Helvetica, 'Hiragino Sans GB', 'Microsoft YaHei', Arial, sans-serif;"><p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.5em 0px;">反向传播算法（Backpropagation Algorithm，简称BP算法）是深度学习的重要思想基础，对于初学者来说也是必须要掌握的基础知识！本文希望以一个清晰的脉络和详细的说明，来让读者彻底明白BP算法的原理和计算过程。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.5em 0px;">全文分为上下两篇，上篇主要介绍BP算法的原理（即公式的推导），介绍完原理之后，我们会将一些具体的数据带入一个简单的三层神经网络中，去完整的体验一遍BP算法的计算过程；下篇是一个项目实战，我们将带着读者一起亲手实现一个BP神经网络（不使用任何第三方的深度学习框架）来解决一个具体的问题。</p> <h1 id="h1bp" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.5em 0px; font-weight: bold; font-size: 1.6em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">1. BP算法的推导</span></h1> <figure style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;"><img src="https://upload-images.jianshu.io/upload_images/13056713-8fa1e0378e60af5b?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240" alt="图1 一个简单的三层神经网络" title="图1 一个简单的三层神经网络" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; display: block; margin: 0px auto; max-width: 100%;"><figcaption style="line-height: inherit; margin: 0px; padding: 0px; margin-top: 10px; text-align: center; color: rgb(153, 153, 153); font-size: 0.7em;">图1 一个简单的三层神经网络</figcaption></figure> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.5em 0px;">图 1 所示是一个简单的三层（两个隐藏层，一个输出层）神经网络结构，假设我们使用这个神经网络来解决二分类问题，我们给这个网络一个输入样本<span class="katex" style="font: 1.21em/1.2 KaTeX_Main, 'Times New Roman', serif; text-indent: 0px; text-rendering: auto; font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 8px 3px;"><img src="http://qiniu.aihubs.net/FsetpUHcGTLKUvxXzVdZV5wxuKXX" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 0px auto; max-width: 100%; display: inline-block; vertical-align: middle;"></span>，通过前向运算得到输出<span class="katex" style="font: 1.21em/1.2 KaTeX_Main, 'Times New Roman', serif; text-indent: 0px; text-rendering: auto; font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 8px 3px;"><img src="http://qiniu.aihubs.net/FhwEK6wSnzkiU3wFH_ax7WKP3h6o" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 0px auto; max-width: 100%; display: inline-block; vertical-align: middle;"></span>。输出值<span class="katex" style="font: 1.21em/1.2 KaTeX_Main, 'Times New Roman', serif; text-indent: 0px; text-rendering: auto; font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 8px 3px;"><img src="http://qiniu.aihubs.net/FhwEK6wSnzkiU3wFH_ax7WKP3h6o" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 0px auto; max-width: 100%; display: inline-block; vertical-align: middle;"></span>的值域为<span class="katex" style="font: 1.21em/1.2 KaTeX_Main, 'Times New Roman', serif; text-indent: 0px; text-rendering: auto; font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 8px 3px;"><img src="http://qiniu.aihubs.net/FufSaWB_0zRV0gb725WoXFXACs63" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 0px auto; max-width: 100%; display: inline-block; vertical-align: middle;"></span>，例如<span class="katex" style="font: 1.21em/1.2 KaTeX_Main, 'Times New Roman', serif; text-indent: 0px; text-rendering: auto; font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 8px 3px;"><img src="http://qiniu.aihubs.net/FhwEK6wSnzkiU3wFH_ax7WKP3h6o" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 0px auto; max-width: 100%; display: inline-block; vertical-align: middle;"></span>的值越接近0，代表该样本是"0"类的可能性越大，反之是"1"类的可能性大。</p> <h2 id="h11" style="line-height: inherit; margin: 1.5em 0px; font-weight: bold; font-size: 1.4em; margin-bottom: 2em; margin-right: 5px; padding: 8px 15px; letter-spacing: 2px; background-image: linear-gradient(to right bottom, rgb(0, 188, 212), rgb(63, 81, 181)); background-color: rgb(63, 81, 181); color: rgb(255, 255, 255); border-left: 10px solid rgb(51, 51, 51); border-radius: 5px; text-shadow: rgb(102, 102, 102) 1px 1px 1px; box-shadow: rgb(102, 102, 102) 1px 1px 2px;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">1.1 前向传播的计算</span></h2> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.5em 0px;">为了便于理解后续的内容，我们需要先搞清楚前向传播的计算过程，以图1所示的内容为例：</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.5em 0px;">输入的样本为：</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.5em 0px;"><span class="katex-display" style="display: block; text-align: center; color: inherit; line-height: inherit; margin: 0px; padding: 0px; font-size: 1.22em;"><span class="katex" style="font: 1.21em/1.2 KaTeX_Main, 'Times New Roman', serif; text-indent: 0px; text-rendering: auto; font-size: inherit; color: inherit; line-height: inherit; margin: 0px; white-space: nowrap; display: inline-block; text-align: center; padding: 3px;"><img src="http://qiniu.aihubs.net/Fg_MKVdqcjv21Bwse4NxTOG9uIMu" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 0px auto; max-width: 100%; display: inline-block; vertical-align: middle;"></span></span></p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.5em 0px;">第一层网络的参数为：</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.5em 0px;"><span class="katex-display" style="display: block; text-align: center; color: inherit; line-height: inherit; margin: 0px; padding: 0px; font-size: 1.22em;"><span class="katex" style="font: 1.21em/1.2 KaTeX_Main, 'Times New Roman', serif; text-indent: 0px; text-rendering: auto; font-size: inherit; color: inherit; line-height: inherit; margin: 0px; white-space: nowrap; display: inline-block; text-align: center; padding: 3px;"><img src="http://qiniu.aihubs.net/FiGENpVoTsbf_5B2-nCCXR9TvfcE" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 0px auto; max-width: 100%; display: inline-block; vertical-align: middle;"></span></span></p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.5em 0px;">第二层网络的参数为：</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.5em 0px;"><span class="katex-display" style="display: block; text-align: center; color: inherit; line-height: inherit; margin: 0px; padding: 0px; font-size: 1.22em;"><span class="katex" style="font: 1.21em/1.2 KaTeX_Main, 'Times New Roman', serif; text-indent: 0px; text-rendering: auto; font-size: inherit; c