Linear Regression
MATH1900: Machine Learning
Location: http://people.sc.fsu.edu/∼jburkardt/classes/ml 2019/linear regression/linear regression.pdf
Find coefficients for an approximating formula to data.
Linear Regression Problem
Given n sets of data, define a formula that minimizes the least squares approximation error .
1 Simple linear regression problem
In the previous class, we looked at a problem in which we wanted to fit n = 23 Ford Escort mileage (x) and
price (y) data values with a simple linear formula:
y = b + m ∗ x
We first normalized our data so that 0 ≤ x, y ≤ 1. Then we used gradient descent to estimate the values of
b and m:
1 import numpy as np
2 from g r a d i e n t de s c e n t 2 import g r a d i e n t d e s c e n t 2
3 bm0 = np . a r ray ( [ 0 . 5 , 0 . 0 ] )
4 r = 0. 0 1
5 d x t o l = 0 .0 0 1
6 d f t o l = 0 .0 0 1
7 itmax = 1000
8 bm, i t = g r a d i e n t d e s c e n t 2 ( f o r d f , fo r d d f , bm0 , r , dx t o l , df t o l , itmax )
Listing 1: Calling gradient descent2 for the Ford data.
This calculation came up with values of b and m for the formula:
y = 1.04232 − 0.796815 ∗ x
1