machinelearningweek9作业答案资源-CSDN文库

共44个文件

m：28个

mat：8个

pdf：2个

异常检测

推荐系统

需积分: 10 13 浏览量 2015-04-27 09:41:10 上传评论收藏 797KB RAR 举报

资源推荐

资源详情

资源评论

收起资源包目录

mlclass-ex8-008.rar （44个子文件）

mlclass-ex8-008

__MACOSX

._mlclass-ex8-008 222B

mlclass-ex8-008

._ex8.pdf 222B

._.DS_Store 222B

._mlclass-ex8 222B

mlclass-ex8

._ex8data2.mat 222B

._movie_ids.txt 222B

._checkCostFunction.m 222B

._fmincg.m 222B

._submit.m 273B

._normalizeRatings.m 222B

._ex8data1.mat 222B

._loadMovieList.m 222B

._ex8_movieParams.mat 222B

._cofiCostFunc.m 222B

._selectThreshold.m 222B

._multivariateGaussian.m 222B

._computeNumericalGradient.m 222B

._visualizeFit.m 222B

._ex8_movies.mat 222B

._estimateGaussian.m 222B

._ex8_cofi.m 222B

._ex8.m 222B

._submitWeb.m 222B

mlclass-ex8-008

.DS_Store 6KB

mlclass-ex8

loadMovieList.m 651B

computeNumericalGradient.m 1KB

fmincg.m 9KB

estimateGaussian.m 1KB

cofiCostFunc.m 3KB

checkCostFunction.m 2KB

ex8_movies.mat 218KB

multivariateGaussian.m 829B

ex8data2.mat 91KB

visualizeFit.m 582B

ex8_movieParams.mat 196KB

submit.m 17KB

ex8_cofi.m 7KB

normalizeRatings.m 479B

movie_ids.txt 47KB

submitWeb.m 827B

ex8data1.mat 9KB

ex8.m 4KB

selectThreshold.m 1KB

ex8.pdf 246KB

Programming Exercise 8:

Anomaly Detection and Recommender

Systems

Machine Learning

Introduction

In this exercise, you will implement the anomaly detection algorithm and

apply it to detect failing servers on a network. In the second part, you will

use collaborative ﬁltering to build a recommender system for movies. Before

starting on the programming exercise, we strongly recommend watching the

video lectures and completing the review questions for the associated topics.

To get started with the exercise, you will need to download the starter

code and unzip its contents to the directory where you wish to complete the

exercise. If needed, use the cd command in Octave to change to this directory

before starting this exercise.

Files included in this exercise

ex8.m - Octave/Matlab script for ﬁrst part of exercise

ex8 cofi.m - Octave/Matlab script for second part of exercise

ex8data1.mat - First example Dataset for anomaly detection

ex8data2.mat - Second example Dataset for anomaly detection

ex8 movies.mat - Movie Review Dataset

ex8 movieParams.mat - Parameters provided for debugging

multivariateGaussian.m - Computes the probability density function

for a Gaussian distribution

visualizeFit.m - 2D plot of a Gaussian distribution and a dataset

checkCostFunction.m - Gradient checking for collaborative ﬁltering

computeNumericalGradient.m - Numerically compute gradients

fmincg.m - Function minimization routine (similar to fminunc)

loadMovieList.m - Loads the list of movies into a cell-array

movie ids.txt - List of movies

normalizeRatings.m - Mean normalization for collaborative ﬁltering

[?] estimateGaussian.m - Estimate the parameters of a Gaussian dis-

tribution with a diagonal covariance matrix

[?] selectThreshold.m - Find a threshold for anomaly detection

[?] cofiCostFunc.m - Implement the cost function for collaborative ﬁl-

tering

? indicates ﬁles you will need to complete

Throughout the ﬁrst part of the exercise (anomaly detection) you will be

using the script ex8.m. For the second part of collaborative ﬁltering, you

will use ex8 cofi.m. These scripts set up the dataset for the problems and

make calls to functions that you will write. You are only required to modify

functions in other ﬁles, by following the instructions in this assignment.

Where to get help

We also strongly encourage using the online Q&A Forum to discuss exer-

cises with other students. However, do not look at any source code written

by others or share your source code with others.

If you run into network errors using the submit script, you can also use

an online form for submitting your solutions. To use this alternative submis-

sion interface, run the submitWeb script to generate a submission ﬁle (e.g.,

submit ex8 part2.txt). You can then submit this ﬁle through the web

submission form in the programming exercises page (go to the programming

exercises page, then select the exercise you are submitting for). If you are

having no problems submitting through the standard submission system us-

ing the submit script, you do not need to use this alternative submission

interface.

1 Anomaly detection

In this exercise, you will implement an anomaly detection algorithm to detect

anomalous behavior in server computers. The features measure the through-

put (mb/s) and latency (ms) of response of each server. While your servers

were operating, you collected m = 307 examples of how they were behaving,

and thus have an unlabeled dataset {x

(1)

, . . . , x

(m)

}. You suspect that the

vast majority of these examples are “normal” (non-anomalous) examples of

the servers operating normally, but there might also be some examples of

servers acting anomalously within this dataset.

You will use a Gaussian model to detect anomalous examples in your

dataset. You will ﬁrst start on a 2D dataset that will allow you to visualize

what the algorithm is doing. On that dataset you will ﬁt a Gaussian dis-

tribution and then ﬁnd values that have very low probability and hence can

be considered anomalies. After that, you will apply the anomaly detection

algorithm to a larger dataset with many dimensions. You will be using ex8.m

for this part of the exercise.

The ﬁrst part of ex8.m will visualize the dataset as shown in Figure 1.

0 5 10 15 20 25 30

Latency (ms)

Throughput (mb/s)

Figure 1: The ﬁrst dataset.

1.1 Gaussian distribution

To perform anomaly detection, you will ﬁrst need to ﬁt a model to the data’s

distribution.

Given a training set {x

(1)

, ..., x

(m)

} (where x

(i)

∈ R

), you want to esti-

mate the Gaussian distribution for each of the features x

. For each feature

i = 1 . . . n, you need to ﬁnd parameters µ

and σ

that ﬁt the data in the

i-th dimension {x

(1)

, ..., x

(m)

} (the i-th dimension of each example).

The Gaussian distribution is given by

p(x; µ, σ

) =

√

2πσ

−

(x−µ)

2σ

where µ is the mean and σ

controls the variance.

1.2 Estimating parameters for a Gaussian

You can estimate the parameters, (µ

, σ

), of the i-th feature by using the

following equations. To estimate the mean, you will use:

j=1

(j)

, (1)

and for the variance you will use:

j=1

(j)

− µ

)

. (2)

Your task is to complete the code in estimateGaussian.m. This function

takes as input the data matrix X and should output an n-dimension vector

mu that holds the mean of all the n features and another n-dimension vector

sigma2 that holds the variances of all the features. You can implement this

using a for-loop over every feature and every training example (though a vec-

torized implementation might be more eﬃcient; feel free to use a vectorized

implementation if you prefer). Note that in Octave, the var function will

(by default) use

m−1

, instead of

, when computing σ

Once you have completed the code in estimateGaussian.m, the next

part of ex8.m will visualize the contours of the ﬁtted Gaussian distribution.

You should get a plot similar to Figure 2. From your plot, you can see that

most of the examples are in the region with the highest probability, while

the anomalous examples are in the regions with lower probabilities.

You should now submit your estimate Gaussian parameters function.

1.3 Selecting the threshold, ε

Now that you have estimated the Gaussian parameters, you can investigate

which examples have a very high probability given this distribution and which

examples have a very low probability. The low probability examples are

0 5 10 15 20 25 30

Latency (ms)

Throughput (mb/s)

Figure 2: The Gaussian distribution contours of the distribution ﬁt to the

dataset.

more likely to be the anomalies in our dataset. One way to determine which

examples are anomalies is to select a threshold based on a cross validation

set. In this part of the exercise, you will implement an algorithm to select

the threshold ε using the F

score on a cross validation set.

You should now complete the code in selectThreshold.m. For this, we

will use a cross validation set {(x

(1)

, y

(1)

), . . . , (x

)

, y

)

)}, where the la-

bel y = 1 corresponds to an anomalous example, and y = 0 corresponds

to a normal example. For each cross validation example, we will com-

pute p(x

(i)

). The vector of all of these probabilities p(x

(1)

), . . . , p(x

cv)

) is

passed to selectThreshold.m in the vector pval. The corresponding labels

(1)

, . . . , y

cv)

is passed to the same function in the vector yval.

The function selectThreshold.m should return two values; the ﬁrst is

the selected threshold ε. If an example x has a low probability p(x) < ε,

then it is considered to be an anomaly. The function should also return the

score, which tells you how well you’re doing on ﬁnding the ground truth

anomalies given a certain threshold. For many diﬀerent values of ε, you will

compute the resulting F

score by computing how many examples the current

threshold classiﬁes correctly and incorrectly.

The F

score is computed using precision (prec) and recall (rec):

2 · prec · rec

prec + rec

, (3)

评论收藏

内容反馈

BYR_jiandong

粉丝: 540
资源: 6

machine learning week9 作业答案

最新资源

machine learning week9 作业答案

Coursera Machine Learning 第九周week9ex8AnomalyDetectionRecommedMachines编程全套满分题目+注释

（全）2020吴恩达机器学习MachineLearning第九周编程作业ex8

coursera机器学习课后习题答案全套

国外机器学习和模式识别经典教材PRML 习题解答

coursera machine learning编程作业答案

Coursera machine learning-答案

Coursera machine learning答案

作业数据---machine-learning-master.zip_machine learning_smartcab age

MIT关于machine learning 的所有习题和答案

machine learning week7 作业解答

吴恩达机器学习2022 supervised machine learning week1 C2 作业

吴恩达机器学习2022 supervised machine learning week1 C1 作业

Coursera Machine Learning week3

吴恩达 machine learning 第五次作业代码

machine-learning-ex1.rar_machine learning_机器学习答案

吴恩达 machine learning 第二次作业代码

吴恩达 machine learning 第四次作业代码

吴恩达 machine learning 第三次作业代码

吴恩达机器学习2022 supervised machine learning week1 C3

Andrew Ng机器学习课程WEEK2作业

斯坦福公开课machine learning机器学习第五周编程作业ex4

Introduction to Machine Learning with Applications in Information Security-2018

Basics of Linear Algebra for Machine Learning

coursera上吴恩达的《machine learning》的所有大作业Code

machine learning第一周编程题答案

machine learning 课程笔记及编程作业

Ng machine learning ex1 作业代码

coursera上的《machine learning》的所有大作业

machine learning Regularization (Week 3) exercise

最新资源