# classfication_demo
用logistic回归，SVM，神经网络实现分类算法
## Logistic分类
采用随机梯度下降方法实现。
```python
def train(self, num_iteration=150):
"""随机梯度上升算法
Args:
data (numpy.ndarray): 训练数据集
labels (numpy.ndarray): 训练标签
num_iteration (int): 迭代次数
"""
for j in xrange(num_iteration):
data_index = range(self.data_num)
for i in xrange(self.data_num):
# 学习速率
alpha = 0.01
rand_index = int(random.uniform(0, len(data_index)))
error = self.label[rand_index] - sigmoid(sum(self.data[rand_index] * self.weights + self.b))
self.weights += alpha * error * self.data[rand_index]
self.b += alpha * error
del(data_index[rand_index])
```
效果图：
## 神经网络
实现一个只有两层的神经网络
### BGD实现
批量梯度下降实现代码：
```python
def batch_gradient_descent(self, num_passes=20000):
"""批量梯度下降训练模型"""
for i in xrange(0, num_passes):
# Forward propagation
z1 = self.data.dot(self.W1) + self.b1
a1 = np.tanh(z1)
z2 = a1.dot(self.W2) + self.b2
exp_scores = np.exp(z2)
probs = exp_scores / np.sum(exp_scores, axis=1, keepdims=True)
# Backpropagation
delta3 = probs
delta3[range(self.num_examples), self.label] -= 1
dW2 = (a1.T).dot(delta3)
db2 = np.sum(delta3, axis=0, keepdims=True)
delta2 = delta3.dot(self.W2.T) * (1 - np.power(a1, 2))
dW1 = np.dot(self.data.T, delta2)
db1 = np.sum(delta2, axis=0)
# Add regularization terms (b1 and b2 don't have regularization terms)
dW2 += self.reg_lambda * self.W2
dW1 += self.reg_lambda * self.W1
# Gradient descent parameter update
self.W1 += -self.epsilon * dW1
self.b1 += -self.epsilon * db1
self.W2 += -self.epsilon * dW2
self.b2 += -self.epsilon * db2
```
隐藏层只有三个神经元时的效果图：
### SGD实现
随机梯度下降实现：
```python
def stochastic_gradient_descent(self, num_passes=1000):
"""随机梯度下降训练模型"""
for i in xrange(0, num_passes):
data_index = range(self.num_examples)
for j in xrange(self.num_examples):
rand_index = int(np.random.uniform(0, len(data_index)))
x = np.mat(self.data[rand_index])
y = self.label[rand_index]
# Forward propagation
z1 = x.dot(self.W1) + self.b1
a1 = np.tanh(z1)
z2 = a1.dot(self.W2) + self.b2
exp_scores = np.exp(z2)
probs = exp_scores / np.sum(exp_scores, axis=1, keepdims=True)
# Backpropagation
delta3 = probs
if y:
delta3[0, 0] -= 1
else:
delta3[0, 1] -= 1
dW2 = (a1.T).dot(delta3)
db2 = np.sum(delta3, axis=0, keepdims=True)
va = delta3.dot(self.W2.T)
vb = 1 - np.power(a1, 2)
delta2 = np.mat(np.array(va) * np.array(vb))
dW1 = x.T.dot(delta2)
db1 = np.sum(delta2, axis=0)
# Add regularization terms (b1 and b2 don't have regularization terms)
dW2 += self.reg_lambda * self.W2
dW1 += self.reg_lambda * self.W1
# Gradient descent parameter update
self.W1 += -self.epsilon * dW1
self.b1 += -self.epsilon * db1
self.W2 += -self.epsilon * dW2
self.b2 += -self.epsilon * db2
del(data_index[rand_index])
```
隐藏层只有三个神经元时的效果图：
## SVM
暂时还没写完
用logistic回归，SVM，神经网络实现分类算法.zip （10个子文件） 
project_code_20703 doc 
logistic.png 57KB nn_batch.png 56KB nn_sto.png 58KB classfy 
svm.py 56B logistic.py 2KB common.py 1KB nerual.py 4KB
requirements.txt 30B .gitignore 789B README.md 4KB资源评论
