LocalCoupledExtremeLearningMachineBasedonParticleSwarmOptimization资源-CSDN文库

28 浏览量 2021-02-08 06:08:52 上传评论收藏 880KB PDF 举报

本文介绍了一种新的基于粒子群优化算法（Particle Swarm Optimization, PSO）的局部耦合极端学习机（Local Coupled Extreme Learning Machine, LC-ELM）的智能优化策略。在此方法中，输入层与隐藏层之间的权重和偏置值，以及局部耦合参数中的地址和半径大小，都是基于粒子群优化算法来确定和优化的。通过与其他几种网络结构相同或紧凑配置的算法（如极端学习机ELM和基于粒子优化的ELM，PSO-ELM）在回归和分类基准问题上的仿真结果比较，提出的LC-PSO-ELM算法展现出了改善的泛化性能和鲁棒性。文章的标题“Local Coupled Extreme Learning Machine Based on Particle Swarm Optimization”涵盖了以下知识点： 1. 局部耦合极端学习机（LC-ELM）：这是一种改进的单隐藏层前馈神经网络（Single-hidden layer feed-forward neural networks, SLFNs），其关键特性是在模型中引入了局部耦合结构。通过耦合结构的局部化，可以更有效地捕捉数据的局部特征，提高学习性能。 2. 粒子群优化（PSO）：PSO是一种基于群体智能的优化技术，通过模拟鸟群的社会行为来解决优化问题。粒子群优化算法中，每个粒子代表问题空间中的一个潜在解，通过迭代搜索过程来优化问题。粒子的位置更新是根据个体经验（自身最佳位置）和群体经验（群体中最佳位置）来进行的。 3. 极端学习机（ELM）：ELM是一种用于SLFNs的快速学习算法，由Huang等人于2006年提出。ELM算法的核心在于随机选择输入层到隐藏层的权重和偏置，然后根据普通最小二乘法确定隐藏层到输出层的权重。ELM算法因为训练速度快和良好的泛化能力而受到关注。文章的描述部分提到了LC-ELM相较于传统学习方法具有明显的优势，在于其学习速度，而在改善泛化性能和鲁棒性方面，结合了PSO算法的LC-PSO-ELM则进一步提高了网络的表现。这里涉及的知识点包括了对传统SLFNs学习方法的局限性，以及针对这些局限性所提出的快速学习算法ELM的优缺点。文章的标签指出了其作为研究论文的性质，这通常意味着文章将深入探讨该主题的理论和实验研究结果。此外，研究论文通常会包含严谨的方法论、详实的实验设计、数据收集和分析过程以及对实验结果的讨论和结论。综上，文章所涉及的知识点集中在使用PSO算法来优化LC-ELM中的参数，旨在提高神经网络模型的训练效率和预测准确性。由于文档内容的扫描识别技术原因，文档中可能含有少量文字识别错误或遗漏，但总体上已经尽力确保文章内容的通顺性。

资源推荐

资源详情

资源评论

algorithms

Article

Local Coupled Extreme Learning Machine Based on

Particle Swarm Optimization

Hongli Guo

, Bin Li

* , Wei Li

, Fengjuan Qiao

, Xuewen Rong

and Yibin Li

School of Mathematics and Statistics, Qilu University of Technology (Shandong Academy of Sciences),

Jinan 250353, China; guohongli0127@163.com (H.G.); 15106900205@163.com (W.L.); qiao_fj@126.com (F.Q.)

School of Control Science and Engineering, Shandong University, Jinan 250061, China;

scrobot@163.com (X.R.); liyb@sdu.edu.cn (Y.L.)

* Correspondence: ribbenlee@126.com

Received: 20 August 2018; Accepted: 29 October 2018; Published: 1 November 2018



 

Abstract:

We developed a new method of intelligent optimum strategy for a local coupled extreme

learning machine (LC-ELM). In this method, both the weights and biases between the input layer and

the hidden layer, as well as the addresses and radiuses in the local coupled parameters, are determined

and optimized based on the particle swarm optimization (PSO) algorithm. Compared with extreme

learning machine (ELM), LC-ELM and extreme learning machine based on particle optimization

(PSO-ELM) that have the same network size or compact network conﬁguration, simulation results in

terms of regression and classiﬁcation benchmark problems show that the proposed algorithm, which

is called LC-PSO-ELM, has improved generalization performance and robustness.

Keywords: extreme learning machine; LC-ELM; particle swarm optimization; LC-PSO-ELM

1. Introduction

The mathematical model of single-hidden layer feed-forward neural networks (SLFNs) has been

widely used in many domains because of its ability to approximate strongly nonlinear input-output

mappings. However, traditional learning methods are usually much slower than required while few

faster learning algorithms for SLFNs are generated [

]. In 2006, a novel learning algorithm for SLFNs

called extreme learning machine (ELM) [

] was presented by Huang et al. for decreasing the training

time of SLFNs.

Different from the existing learning algorithms of SLFNs, the weights and biases between the

input layer and hidden layer of the ELM were chosen randomly, then the weights between the

hidden layer and the output layer were determined based on the ordinary least squares. The ELM

learning algorithm has fast learning speed and good generalization performance with little human

intervention, which makes the algorithm applicable to many areas, such as stock prediction [

], image

classiﬁcation [4], fault diagnosis [5], etc.

In the ELM, the number of hidden neurons is required to be greater than or equal to the number

of the training samples so as to guarantee the convergence of the algorithm. Therefore, there will be

quite a lot of input-hidden weights when the number of input neurons is large [

], which may reduce

the generalization performance of SLFNs. The original ELM model has been equipped with various

extensions to make it more suitable and efﬁcient for speciﬁc applications [

]. For example, based

on the structure of the local coupled feed-forward neural network (LCFNN) [

] and the learning

mechanism of the ELM algorithm, the local coupled extreme learning machine (LC-ELM) learning

algorithm was proposed by Qu in 2014 [

]. The algorithm could decrease the researching complexity

of the weights between the input layer and the hidden layer by means of assigning the addresses to

Algorithms 2018, 11, 174; doi:10.3390/a11110174 www.mdpi.com/journal/algorithms

Algorithms 2018, 11, 174 2 of 16

the hidden neurons [

]. The advantage of the LC-ELM on image watermarking was examined by

Mehta et al. [11].

In the LC-ELM learning algorithm, the addresses and radiuses were generally preset empirically

or randomly. And, thus, those parameters might not be optimal for the LC-ELM, and the algorithm

may yield an inappropriate underlying model. In 2015, Qu et al. presented an evolutionary local

coupled extreme learning machine (ELC-ELM). In the ELC-ELM, the differential evolutionary (DE)

algorithm was used to optimize the addresses and the radiuses of the fuzzy membership functions in

hidden neurons for improving the generalization performance [

]. However, it should be noted that

the hidden biases and input weights in the ELC-ELM were also set randomly.

The DE algorithm has good global converge property by means of utilizing the differential

information of the population. However, the instability performance of DE can also be caused because

of the above reason and the algorithm may be trapped in local optima [

]. Moreover, three

parameters of the DE algorithm should be controlled manually [

]. In 1995, the particle swarm

optimization (PSO) algorithm was presented by Eberhart et al. [

] and has been used in many

optimization ﬁelds as it can converge to the global minima quickly. Compared with other stochastic

optimization techniques, the advantages of the PSO algorithm are that it is easy to be implemented in

practice and few parameters need to be adjusted [

]. The PSO algorithm and its improved variants,

such as APSO (Adaptive PSO) and PSOGSA (The hybrid PSO and gravitational search algorithm),

were used to select the optimal parameters between the input layer and the hidden layer (input weights

and biases) of the ELM [19,20].

Therefore, in order to overcome the limitation of the DE, a new method combining the LC-ELM

with an improved PSO called LC-PSO-ELM is proposed in this paper. In the proposed algorithm, the

improved PSO algorithm is used to optimize the address and window radius of the local coupled

parameters. In addition, the input weights and hidden layer biases of the ELM are also optimized to

further improve the generalization performance of the LC-ELM, and the MP generalized inverse is

used to calculate the weights between the hidden layer and the output layer analytically. In order to

prove the superiority of the proposed algorithm, we compared the computer simulation results from

our developed algorithm to those from the ELM, LC-ELM and PSO-ELM algorithms, respectively. The

comparison results demonstrated that the newly developed algorithm exhibits improved generalization

performance with the highest accuracy.

The rest of this paper is organized as follows. The local coupled extreme learning machine

(LC-ELM) and the improved particle swarm optimization algorithm are given in Section 2. The

local coupled extreme learning machine based on the PSO algorithm is introduced in Section 3.

Section 4 includes different simulation results and analysis of the proposed algorithm in regression

and classiﬁcation benchmark problems. Finally, the conclusions are summarized in Section 5.

2. Theoretical Background

2.1. Local Coupled Extreme Learning Machine

The ELM learning algorithm is a simple, fast and efﬁcient method. For further improving the

generalization performance of the ELM, the LC-ELM learning algorithm was proposed by Qu [

]

in which the efﬁciency of LC-ELM in terms of classiﬁcation and regression benchmark problems

was investigated.

In the LC-ELM, due to the utilization of the fuzzy membership function

F(·)

and the similarity

relation

S(x

)

, the complexity of the weight searching space was reduced and the generalization

performance was correspondingly improved in terms of the simple neural networks structure. The

mathematical formulation of the LC-ELM is presented as follows:

For

arbitrary distinct examples

(

, t

)

, where

= [x

· · · x

] ∈ R

is the input and

= [t

· · · t

] ∈ R

is the expected output,

i =

. . .

. The output of the hidden layer

neurons

g(w

·x

+ b

)

for the ELM is modiﬁed with the help of fuzzy membership function as

Algorithms 2018, 11, 174 3 of 16

g(w

·x

+ b

)F(S(x

))

. Therefore, the network output of the LC-ELM with

hidden neurons

are mathematically modeled by

f (x

) =

∑

i=1

g(w

·x

+ b

)F(S(x

, d

)), i = 1, . . . , N, j = 1, . . . , M (1)

where

g(·)

denotes the activation function of the ELM, which can not only be sigmoid functions,

however also other functions such as sin, cos, cubic, etc.

denotes the weight vector connecting the

ith hidden neuron and the output neurons,

is the weight vector connecting the ith hidden neuron

and the input neurons.

is the bias of the ith hidden neuron.

is the address of the ith hidden node.

In the LC-ELM learning algorithm, the similarity relation

S(x

)

is the distance between the input

and the ith hidden node with address

. Various forms of fuzzy membership functions

F(·)

, such as

Gaussian function, sigmoid function and reverse sigmoid function [

], are utilized. In addition,

the underlying radius parameter

is kept in

F(·)

for adjusting the width of the activation area, which

is also an optimized parameter, to the same as the address parameter

. Combining the structure

of the LCFNN with the learning mechanism of the ELM, the LC-ELM also is a three step learning

algorithm and the parameters (input weights

and biases

between the input layer and hidden layer,

the address

of the hidden neurons) of the networks are assigned randomly, which is the same as the

ELM [10].

The standard LC-ELM learning algorithm can approximate these

examples with zero error

means

∑

j=1



− t



0, where

is the actual output of the LC-ELM. i.e., the corresponding relation

is deﬁned by

f (x

) = t

, j = 1, . . . , M (2)

The above M equations can be written compactly as a linear system:

Hβ = T, j = 1, . . . M (3)

where H is the output matrix of the hidden layer and can be expressed as

H =



= g(w

+ b

)F(S(x, d

))



i = 1, . . . , N, j = 1, .., M. (4)

in the above Equation (4),

= g(w

+ b

)

denotes the output of the ith hidden neuron with respect

β = [β

, . . . , β

]

N×q

is the matrix of the output weights and

denotes the weight vector

connecting the ith hidden node and the output layer.

T = [t

, . . . , t

]

M×q

is the matrix of the target of

the LC-ELM.

The smallest norm least squares solution of Equation (3) is

β = H

T (5)

where H

is the Moore-Penrose generalized inverse of the hidden layer output matrix H [23].

Based on the above discussion, the LC-ELM algorithm can be summarized in Algorithm 1.

Algorithm 1. The algorithm ﬂow of LC-LEM

(1) Input weights w, hidden bias b and the node address d are allocated randomly.

(2) The output matrix of the hidden layer H is computed using Equation (4).

(3) Calculate the output weights

between the hidden layer and the output layer based on Equation (5):

β = H

剩余15页未读，继续阅读

评论收藏

内容反馈

weixin_38725950

粉丝: 3
资源: 901

Local Coupled Extreme Learning Machine Based on Particle Swarm O...

最新资源

Local Coupled Extreme Learning Machine Based on Particle Swarm O...

Dynamic analysis on Magneto-rheological Fluid Damper Coupled with Stay Cable Based on Experimental Results

Flat broadband slow light with low dispersion in coupled resonator optical waveguide based on one-dimensional photonic crystals

Practical Machine Learning and Image Processing

High-Quality Rectifier Based on Coupled-Inductor Sepic Topol.zip

Dual Waveguide Coupled Microring Resonator Sensor Based on Intensity Detection

full wave coupled resonator filter optimization

Novel Microwave Sensors Based on Split Ring.pdf

Pan-sharpening based on multilevel coupled deep network

Machine.Learning.for.Adaptive.Many-Core.Machines.331

Calibration transfer and drift compensation of e-noses via coupled task learning

Keyframe-Based Visual-Inertial Odometry Using Nonlinear Optimization.pdf

论文研究-Wavelength-routed four-port optical router based on polymeric parallel-coupled microring routing element.pdf

VCPA-based hybrid strategy.zip

Denoising based new Noise model

An numerical method for fast nonlinear model predictive control

基于CORDIC的反正弦和反余弦计算的FPGA实现

BA无标度网络中的SIR模型

使用3DCNN和卷积LSTM进行手势识别学习时空特征

基于三次贝塞尔曲线的类汽车曲率连续路径平滑

基于机器学习的设备剩余寿命预测方法综述

基于维纳过程的退化模型，具有递归过滤算法，可用于估计剩余使用寿命

基于FPGA的奇异值和特征值分解的快速实现。

最新资源