ImbalancedExtremeLearningMachineBasedonProbabilityDensityEstimation资源-CSDN文库

研究论文

113 浏览量 2021-02-11 15:29:17 上传评论收藏 644KB PDF 举报

资源推荐

资源详情

资源评论

Imbalanced Extreme Learning Machine Based

on Probability Density Estimation

Jü Yang, Hualong Yu

(&)

, Xibei Yang, and Xin Zuo

School of Computer Science and Engineering,

Jiangsu University of Science and Technology,

Zhenjiang 212003, Jiangsu, People’s Republic of China

yangju_justcs@126.com, yuhualong@just.edu.cn,

yangxibei1980@sina.com, 632343650@qq.com

Abstract. Extreme learning machine (ELM) is a fast algorithm to train

single-hidden layer feedforward neural networks (SLFNs). Like the traditional

classiﬁcation algorithms, such as decision tree, Naïve Bayes classiﬁer and

support vector machine, ELM also tends to provide biased classiﬁcation results

when the classiﬁcation tasks are imbalanced. In this article, we ﬁrst analyze the

relationship between ELM and Naïve Bayes classiﬁer, and then take the deci-

sion outputs of all training instances in ELM as probability density represen-

tation by kernel probability density estimation method. Finally, the optimal

classiﬁcation hyperplane can be determined by ﬁnding the intersection point of

two probability density distribution curves. Experimental results on thirty-two

imbalanced data sets indicate that the proposed algorithm can address class

imbalance problem effectively, as well outperform some existing class imbal-

ance learning algorithms in the context of ELM.

Keywords: Extreme learning machine

 Class imbalance learning  Probability

density estimation

 Naïve Bayes classiﬁer

1 Introduction

Extreme learning machine proposed by Huang et al.,[1] has become a popular research

topic in machine learning in recent years [2]. It is proved that single-hidden layer

feedforward neural networks (SLFNs) with arbitrary hidden parameters and continuous

activation funct ion can universally approximate to any continuous functions [1]. Some

recent research [3–6], however, indicated that the performance of ELM could be

destroyed by class imbalance distribution, which is similar with some traditional

classiﬁers, such as support vector machine, Naïve Bayes classiﬁer and decision tree. In

class imbalance scenario, the accuracy of the minority class always tends to be

underestimated, causing meaningless classiﬁcation results [7]. Therefore, it is necessary

to adopt some strategies to make the classiﬁcation model provide impartial classiﬁ-

cation results.

In the context of ELM, some researchers have presented several class imbalance

learning algorithms. Weighted extreme learning machine (WELM) appoints different

penalty parameter s for the training errors belonging to the instances in different cate-

gories, decreasing the possibility of misclassifying the minority class samples [3]. The

A. Bikakis and X. Zheng (Eds.): MIWAI 2015, LNAI 9426, pp. 160–167, 2015.

DOI: 10.1007/978-3-319-26181-2_15

penalty parameters, however, can be only allocated empirically. A similar algorithm

called Fuzzy ELM (FELM) was proposed in [4], which changes the distributions of

penalty parameters by inserting a fuzzy matrix. As two well-known data-layer class

imbalance learning algorithms, random oversampling (ROS) and synthetic minority

oversampling technology (SMOTE) have also be integrated into ELM to deal with

practical class imbalance applications [5, 6].

In this article, we try to present a novel algorithm to deal with class imbalance

problem in the context of ELM. First, we analyze the relationship between ELM and

Naïve Bayes classiﬁer, and indicate that the decision output in ELM approximately

equals to the posterior probability in Naïve Bayes classiﬁer. Then, on the decision

output space, we estimate the probability density distributions for two different classes,

respectively. Finally, the optimal position of the classiﬁcation hyperplane can be

determined by ﬁnding the intersection point of two probability density distribution

curves. We compare the proposed algorithm with several popular class imbalance

learning algorithms, and the experimental results indicate its superiority.

2 Theories and Methods

2.1 Extreme Learning Machine

Considering a supervised learning problem where we have a training set with

N training instances and m classes, ðx

; t

Þ2R

 R

. Here, x

is an n × 1 input vector

and t

is the corresponding m × 1 target vector. ELM aims to learn a decision rule or an

approximation function based on the training data. In other words, ELM is used to

create an approximately accurate mapping relationship between x

and t

Unlike the traditional back-propagation (BP) algorithm [8], ELM provides the

hidden parameters randomly to training SLFNs. Suppose there are L hidden layer

nodes, then for an instance x, the corresponding hidden layer output can be presented

by a row vector h xðÞ¼½h

xðÞ; ...; h

xðÞ, thus the mathematical model of ELM is:

Hb ¼ T ð1Þ

where H ¼ hx

ðÞ; ...; hx

ðÞ½

is the hidden layer output matrix for the whole training

set, β is the output weight matrix and T is the target vector. Here, only the output

weight matrix β is unknown. Then we can adopt least square method to acquire the

solution of β that can be described as follows:

b ¼

T ¼ H

þ HH

1

T; when N  L

b ¼

T ¼ð

þ H

HÞ

1

T; when N [ L

ð2Þ

Here,

is the Moore-Penrose “generalized” inverse of the hidden layer output matrix

H, which can guarantee the solution is least norm least square solution of Eq. (1). C is

the penalty parameter to mediate the balance relationship between the training errors

and the generalization ability.

Imbalanced ELM Basedon Probability Density Estimation 161

剩余7页未读，继续阅读

评论收藏

内容反馈

weixin_38628612

粉丝: 8
资源: 942

Imbalanced Extreme Learning Machine Based on Probability Density...

最新资源

Imbalanced Extreme Learning Machine Based on Probability Density...

extreme learning machine

extreme-learning-machine

Machine learning

Machine learning A Probabilistic Perspective

Statistical background subtraction based on imbalanced learning

Imbalanced-Learning:现实世界数据集的学习失衡。 在加拿大的公共道路上对汽车碰撞数据集建模

数据不均衡问题经典文献《Learning from Imbalanced Data》

MATLAB.rar_probability density

A study on effectiveness of extreme learning machine

极限学习机extreme learning machine

机器学习对不平衡数据学习处理方案

本项目是一个由我个人开发的python机器学习项目，基于极限学习机（ELM）

处理不均衡数据 (深度学习)! Dealing with imbalanced data (deep learning)

imbalanced-learning-papers:关于学习失衡的论文

python imbalanced-learn库实用例子（examples）代码

Extreme learning machine for classification over uncertain data

A nonlinear PLS modeling method based on extreme learningmachine

极限学习机（Extreme Learning Machine, ELM）

Classification of imbalanced bioinformatics data by using boundary movement-based ELM

imbalanced-learning:该讲习班是在明斯特大学商学院进行的“ R语言机器学习”研究生课程的一部分（202021年冬季）

RSG A Simple but Effective Module for Learning Imbalanced.pdf

MOTLD代码复现(运行主程序即可)

Medical Big Data and Internet of Medical Things

polikar2012.pdf

最新资源

Imbalanced-Learning:现实世界数据集的学习失衡。在加拿大的公共道路上对汽车碰撞数据集建模