A nonlinear PLS modeling method based on extreme l earning
machine
Chunxia Wang
1
,JingHu
2
, and Chenglin Wen
1
1. Institute of Systems Science and Control Engineering, School of Automation, Hangzhou Dianzi University, Hangzhou 310018
E-mail: chxiawang@163.com; wencl@hdu.edu.cn
2. Department of Control Science and Control Engineering, Zhejiang University, Hangzhou 310027
E-mail: jhu@iipc.zju.edu.cn
Abstract: This paper proposes a nonlinear PLS modeling method. The extreme learning machine (ELM) is embedded in the
process of linear PLS modeling. Thus the linear PLS modeling is transformed into the nonlinear frame which can deal with
nonlinear data. The multi-input-multi-output (MIMO) nonlinear modeling task is decomposed into two parts: the external linear
modeling and inner univariate nonlinear modeling problems. The linear PLS method is used to establish the external model,
while the extreme learning machine is used to capture the inner nonlinear model. Compared to the standard PLS method, the
method in this paper has the potential of modeling any continuous nonlinear relationship and has better robust properties. And it
is less time-consuming than other neural networks PLS (NNPLS) methods. Because extreme learning machine can capture the
inner nonlinearity of data, the proposed method has better prediction performance than linear PLS regression method. Simulation
veri¿es the better prediction performance and the validity of the proposed method.
Key Words: Linear PLS model; nonlinear PLS; extreme learning machine; NNPLS
1 Introduction
Statistical analysis methods can solve practical problems
by establishing models based on historical data or experi-
mental data. PLS (partial least squares) is one of the typical
methods [1]. As early as in 1966, Wold proposed the PLS al-
gorithm and used it to regression analysis. It has been wide-
ly used in social economic, stoichiometry, chemical process
and so on, and has become a principal component extrac-
tion or quality monitoring method [2]. The PLS method can
remove the correlation between input data, and extract the
corresponding principal component scores which are orthog-
onal with each other. However, PLS regression decomposes
process variables X and quality variables Y into bilinear for-
m. It can only extract linear information between two data
sets. There will be inaccurate modeling problems when the
data sets have nonlinear relationship. To capture nonlinear
characteristics of data, many researchers improved the tradi-
tional PLS method with varying degrees such as kernel PLS
[3] and a quadratic PLS modeling method [4].
Neural networks can also apply to regression applications
[5]. In theory, as an universal approximator, it can approxi-
mate continuous objective function with arbitrary precision.
There are many neural networks methods such as SVM and
BP neural networks. In recent years, the extreme learning
machine (ELM) has been continuously developing. Com-
pared with above two methods whose input parameters re-
quire to be adjusted iteratively, it’s hidden nodes p arameters
can be randomly generated through a p robability distribution
function. Its training process is less time-consuming and
it has good generalization performance. Although in some
cases, the prediction precision of direct application of neu-
ral networks is higher than the linear regression method, its
prediction performance will be limited with correlated in-
puts or limited observations. Especially in the case of limit-
ed data, the number of input weights need to be determined
This work is supported by National Natural Science Foundation (NNS-
F) of China under Grant 61304109, 61473159, 61174112.
are more than observations, and the direct neural networks
methods lead to over-¿tting. Then the NNPLS method is
proposed. The standard PLS method is extended to non-
linear framework [6]. The external linear model is estab-
lished by PLS, while neural networks is used to obtain in-
ner nonlinear relationship. The NNPLS method decompos-
es multi-input-multi-output modeling problem into several
single-input-single-output (SISO) problems. Compared to
the direct application of n eural networks methods, it has less
parameters to be determined, and the process is simpler.
According to the advantages and disadvantages of ELM
and PLS methods and motivated by the NNPLS approach,
this paper apply ELM to the process of PLS modeling. By
using its universal approximate capacity, we apply the ELM
method to the process of establishing internal model to im-
prove prediction accuracy of traditional PLS method.
The rest of the paper is organized as follows. Section II
introduces the model of linear PLS model. In section III,
we describe the ELM theory. Section IV is the introduction
of the new method proposed. Section V is the simulation.
Finally, we conclude this paper in section VI.
2LinearPLSmodel
The PLS regression algorithm divides the model into inner
model and outer model. The outer model converts raw data
to the latent variable space and obtains mutually orthogonal
score vectors, while the inner model establishes a linear rela-
tionship between the scores. Then one can use it to describe
the quality variables which have strong correlation with the
process variables. Fin ally, one can monitor the quality vari-
ables and obtain quality prediction value simultaneously.
The modeling data is standardized in advance. Process
variables are X ∈ R
n×m
. Quality variables are Y ∈ R
n×s
.
The sample number is n. The dimensions of input and output
variables are m and s respectively. The linear PLS method
decomposes X and Y into a bilinear form [2].
Proceedings of the 34th Chinese Control Conference
Jul
y
28-30, 2015, Han
g
zhou, China
3507
资源评论
