【免费】强化学习深度学习神经网络论文

神经网络

深度学习

需积分: 0 84 浏览量 2023-03-23 21:32:58 上传评论收藏 2.32MB PDF 举报

资源推荐

资源详情

资源评论

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 29, NO. 4, APRIL 2018 981

A Collaborative Neurodynamic Approach to

Multiple-Objective Distributed Optimization

Shaofu Yang, Qingshan Liu, Senior Member, IEEE, and Jun Wang, Fellow, IEEE

Abstract—This paper is concerned with multiple-objective

distributed optimization. Based on objective weighting and

decision space decomposition, a collaborative neurodynamic

approach to multiobjective distributed optimization is presented.

In the approach, a system of collaborative neural networks is

developed to search for Pareto optimal solutions, where each

neural network is associated with one objective function and

given constraints. Sufﬁcient conditions are derived for ascer-

taining the convergence to a Pareto optimal solution of the

collaborative neurodynamic system. In addition, it is proved

that each connected subsystem can generate a Pareto optimal

solution when the communication topology is disconnected.

Then, a switching-topology-based method is proposed to compute

multiple Pareto optimal solutions for discretized approximation

of Pareto front. Finally, simulation results are discussed to

substantiate the performance of the collaborative neurodynamic

approach. A portfolio selection application is also given.

Index Terms—Collaborative neurodynamic approach, distrib-

uted optimization, multiobjective optimization, neural networks,

Pareto optimal solutions.

I. INTRODUCTION

ANY problems in science and engineering, including

multicriteria decision making, machine learning, model

predictive control, smart building design, can be formulated as

multiple-objective optimization problems (see [1]–[6], to name

a few), which inherently contain several conﬂicting objectives

to be optimized simultaneously. Usually, no single solution

can be found, such that every objective function attains its

optimum. Therefore, a concept of optimality known as Pareto

optimality is utilized in multiobjective framework, which

describes compromise between the competing objectives. The

set of all Pareto optimal values is called Pareto front.

Different from single-objective optimization, in multiobjec-

tive optimization, it is required to compute a set of Pareto

optimal solutions distributed in the Pareto front [7]. Various

methods have been proposed for obtaining Pareto optimal

solutions. One class of methods is the scalarization approach.

Manuscript received July 11, 2016; revised October 4, 2016; accepted

December 31, 2016. Date of publication February 1, 2017; date of current

version March 15, 2018. This work was supported in part by the Research

Grants Council of the Hong Kong Special Administrative Region of China,

under Grant 14207614, and in part by the National Natural Science Foundation

of China under Grant 61673330 and Grant 61473333.

S. Yang is with the School of Computer Science and Engineering, Southeast

University, Nanjing 210018, China, and was also with the Department of

Computer Science, City University of Hong Kong, Kowloon, Hong Kong

(e-mail: sfyang@seu.edu.cn).

Q. Liu is with the School of Automation, Huazhong University of Science

and Technology, Wuhan 430074, China (e-mail: qsliu@hust.edu.cn).

J. Wang is with the Department of Computer Science, City University of

Hong Kong, Kowloon, Hong Kong (e-mail: jwang.cs@cityu.edu.hk).

Color versions of one or more of the ﬁgures in this paper are available

online at http://ieeexplore.ieee.org.

Digital Object Identiﬁer 10.1109/TNNLS.2017.2652478

A Pareto optimal solution can be obtained by solving a para-

meterized single-objective optimization problem. By varying

the parameters, multiple Pareto optimal solutions are then

obtained. Many techniques have been developed to aggre-

gate objective functions, such as weighted-sum method [8],

goal programming [9], and normal boundary intersection

method [10]. Another class of approaches is population-

based method. Among which, evolutionary multiobjective

optimization approaches [11], [12] have been widely inves-

tigated. Numerous algorithms have been proposed, such as

NSGA-II [13], SPEA2 [14], MOEA/D [15], to name a few.

In addition, particle swarm optimization has also been applied

for multiobjective optimization [16].

Note that the aforementioned methods are mainly discussed

from a centralized manner. In the scalarization approach, all

objective functions need to be combined in advance. In the

population-based method, each individual has to know the full

knowledge of all objective functions to evaluate its quality,

which may not be fulﬁlled in some multiobjective optimization

problems, such as multiparty negotiations, where decision

makers know their own value functions only due to privacy

protection [17]. Another example is the multiobjective opti-

mization problems modeling business activities within a large

international corporation, in which decisions under multiple

objectives are made locally in each country so that the corpora-

tion performs at its best [18]. For these problems, a distributed

approach is necessary. In addition, with an increase in problem

size, a distributed approach is also demanded due to the limited

computing ability of a single computer.

In recent years, many efforts have been devoted to dis-

tributed optimization with a single-objective function based

on multiagent systems, in which each agent knows partial

information of an objective function, and all agents coop-

eratively seek for an optimal solution. Many multiagent

systems have been developed [19]–[28]. In contrast, a few

works on multiobjective distributed optimization are avail-

able. In [18] and [29], multiobjective distributed optimization

based on iterative augmented Lagrangian coordination tech-

niques and diffusion strategies are proposed. More recently,

a subgradient-based algorithm is proposed in [30] for multi-

objective distributed optimization and the relationship between

the selection of weight vector and the approximation error of

the Pareto front is discussed. The aforementioned subgradient-

based approaches require a diminishing step size to obtain

an exact solution, which may limit the performance of algo-

rithms [20]. In [31], neural network models are developed

for multiobjective optimization with equality constraints only

based on a decomposition-coordination principle.

See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

Authorized licensed use limited to: Southeast University. Downloaded on March 10,2022 at 22:18:57 UTC from IEEE Xplore. Restrictions apply.

982 IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 29, NO. 4, APRIL 2018

Over past three decades, neurodynamic optimization based

on neural networks, which have the inherent nature of parallel

computing and the potential of electronic implementation, has

received great development. A neurodynamic approach can

solve optimization problems in running times at the orders

of magnitude much faster than the most popular optimiza-

tion algorithms executed on digital computers. In mid-1980s,

Hopﬁeld and Tank [32] and Tank and Hopﬁeld [33] spear-

headed a neural network model for solving the traveling

salesmen problem. From then on, neurodynamic optimization

approach has been extensively investigated. Various neural

network models have been developed for different single-

objective optimization problems (see [34]–[46], and references

therein). The key idea is to establish neural network models

based on optimality condition by using the Lagrange multi-

plier method, projection method, duality theory, and so on.

More recently, a neural network model with two-time-scale is

proposed for multiobjective optimization [47]. However, it is

still a centralized manner. In addition, by using multiple neural

network models, collaborative neurodynamic approaches have

been developed in [48] and [49] for global optimization with

a single objective function.

Motivated by the above discussions, this paper aims to

develop a collaborative neurodynamic approach for multiob-

jective distributed optimization. By using the weighted sum

method [8], a single-objective optimization problem can be

formulated, in which the scalar objective is summation of

weighted original objective functions. Based on decomposition

principle, a system consisting of multiple neural networks is

developed to cooperatively seek Pareto optimal solutions of a

multiobjective optimization problem. Each neural network is

required to know a single objective function only and even

partial information of constraints. The cooperative scheme

relies on the transmission of partial state information among

them. Each neural network optimizes its local objective func-

tion while seeks for state agreement with others, under the

guidance of a consensus-based coupling scheme. Sufﬁcient

conditions are presented to guarantee the convergence of

the collaborative neurodynamic system to a Pareto optimal

solution.

In order to characterize a Pareto front, a switching-topology-

based method is proposed for computing a set of Pareto

optimal solutions. In this method, a switching process of the

communication topology in the developed system is designed.

Except for the last step, the communication topology is not

connected during the process. We show that each connected

component is also able to converge to a Pareto optimal

solution. Therefore, when the system consists of multiple

connected components, such method has a merit of generating

multiple Pareto optimal solutions in parallel.

In summary, the distinguished features of the approach

are twofold. First, a collaborative neurodynamic system is

developed for computing Pareto optimal solutions to a mul-

tiobjective distributed optimization problem by integrating

existing neurodynamic model for single-objective optimization

with a decomposition principle. In contrast to the existing

algorithms in [18], [29], and [30] for multiobjective distrib-

uted optimization, the approach herein can deal with general

constraints. Second, switching topology is used for generating

multiple Pareto optimal solutions to characterize Pareto fronts.

The remaining part of this paper is organized as

follows. In Section II, preliminary concepts on multiobjective

optimization, projection operator, and graph theory are intro-

duced. In Section III, the collaborative neurodynamic system is

formulated, and then, the stability properties of the system are

analyzed. In Section IV, a switching-topology-based method

is proposed to generate multiple Pareto optimal solutions.

In Section V, numerical examples are presented to illustrate

the proposed approach. In Section VI, an application of the

collaborative neurodynamic approach in portfolio selection

problem is shown. Finally, the conclusion remark is given

in Section VII.

Notation: Throughout this paper, vectors are column vec-

tors and written with bold face. · denotes 2-norm.

diag{A

, A

, ··· , A

} denotes a (block) diagonal matrix

consisting of diagonal entries (blocks) A

, A

, ··· , A

col{x

, x

, ··· , x

} denotes a column vector formed by

stacking vectors x

, x

, ··· , x

on top of each other. I

denoted for an identity matrix with dimension n. 1

denotes a

vector of k dimensions with all entries being 1. I

denotes the

set {1, 2, ··· , k}. R

denotes the half space {x ≥ 0 | x ∈ R

k refers to the largest integer no larger than k.

II. P

RELIMINARIES

A. Multiobjective Optimization

A multiobjective optimization problem has a number of

objective functions to be minimized or maximized [7].

In mathematical notation, it can be posed as

min f (x) = ( f

(x), f

(x), ··· , f

(x))

s.t. x ∈ X (1)

where x ∈ R

is the vector of decision variables, and X =

{x ∈ R

: Ax = b, g(x) ≤ 0, x ∈ } is called feasible

region. f (x) ∈ R

is a vector of objective functions or criteria

with f

(x) : R

→ R (i ∈ I

). k is the number of objective

functions and satisﬁes k ≥ 2. g(x) : R

→ R

represents the

inequality constraint. A ∈ R

p×n

has a full row rank.  ⊂ R

a closed convex set. The feasible objective region F is deﬁned

as the set { f (x) : x ∈ X }. Problem (1) is said to be convex

if all the objective functions and the feasible region X are

convex.

Due to the contradiction of the objective functions, it is often

not possible to ﬁnd a single solution what would be optimal for

all the objectives simultaneously. Consequently, the solution of

problem (1) is described by the concept of Pareto optimality,

which refers to the solution that none of the objective functions

can be improved without detriment to at least one of the other

objective functions. A formal deﬁnition of Pareto optimality

is given in the following.

Deﬁnition 1 [7]: A decision vector x

∗

∈ X is called

Pareto optimal if there does not exist x ∈ X such that

(x) ≤ f

∗

) for all i ∈ I

,and f

(x)< f

∗

) for one

j ∈ I

. A decision vector x

∗

∈ X is called weak Pareto

optimal if there does not exist x ∈ X such that f

(x)< f

∗

)

for all i ∈ I

. An objective vector z

∗

∈ F is called (weak)

Authorized licensed use limited to: Southeast University. Downloaded on March 10,2022 at 22:18:57 UTC from IEEE Xplore. Restrictions apply.

YANG et al.: COLLABORATIVE NEURODYNAMIC APPROACH TO MULTIPLE-OBJECTIVE DISTRIBUTED OPTIMIZATION 983

Pareto optimal if its corresponding decision vector is (weak)

Pareto optimal.

To obtain Pareto optimal solutions, one widely used

approach is the weighted sum method, which transforms

the multiple objectives into a single one by multiplying

each objective with a weight. Given a weighting vector

w = (w

, ··· ,w

)

, problem (1) is converted into

min



i=1

(x)

s.t. x ∈ X (2)

where w

is the weight of the ith objective function. Generally,

it is required that w ∈ S

,whereS

={w ∈ R

: w ≥ 0,

w

= 1} is the unit simplex in R

. The solution set of

problem (2) with a given w is denoted as X

∗

(w).

Denote the set of Pareto optima and weak Pareto optima

of problem (1) by P and P

, respectively. By Deﬁnition 1,

P ⊂ P

. In addition, let P



w∈S

∗

(w) and P

int(S

)



w∈int(S

)

∗

(w),whereint(S

) represents the inner part of

set S

. Then, the relationship among P

, P

int(S

)

, P,andP

is stated by following lemma.

Lemma 1 [7], [50]: For problem (1), P

⊂ P

and

int(S

)

⊂ P. If problem (1) is convex, then P = P

int(S

)

⊂

= P

. If all objective functions in problem (1) are strictly

convex on ,thenP = P

int(S

)

= P

The following assumptions hold throughout of this paper.

Assumption 1: All f

and g

are convex and twice contin-

uously differentiable on .

Assumption 2: The minimizer of each objective function f

exists.

Under Assumptions 1 and 2, any weak Pareto optimal

solution of a convex multiobjective optimization problem can

be obtained by solving problem (2) with a weighting vector

w basedonLemma1.

B. Projection Operator

Let P



(x) = arg min

y∈

x − y,wherex ∈ R

and

 ⊂ R

is a nonempty closed convex set. Then, P



(x) is

called the projection of x on . The properties of projection

operator are given in the following.

Lemma 2 [51]: Let  ⊂ R

be a nonempty closed convex

set. Then, for any x ∈ R

, there is as follows.

1) P



(x) − P



( y)≤x − y,foranyy ∈ R

2) (P



(x) − x)

( y − P



(x)) ≥ 0, for all y ∈ .

3) (P



(x) − x)

( y − x) ≥P



(x) − x

,forall y ∈ .

Note that it is not easy to compute the projection of a point

onto a general convex set. However, for some special convex

sets, the projection operators are well deﬁned. If  ={x ∈

: l

≤ x

≤ h

, i ∈ I

}, P



(x) is also a vector with its

component deﬁned as



(x)

⎧

⎨

⎩

, x

< l

, l

≤ x

≤ h

, h

< x

In addition, if  = R

,thenP



(x) = (x)

,where(·)

deﬁned componentwise as (x)

= max{x, 0}.If = R

,then



(x) = x for any x ∈ R

C. Graph Theory

AgraphG = (V, E) consists of a vertex set V ={v

··· ,v

} and an edge set E ⊂ V × V. The adjacent matrix

A =[a

]

n×n

associated with G is deﬁned as a

> 0if

) ∈ E,otherwise,a

= 0. It is assumed that a

= 0,

i.e., no self-loop in G. The degree of node v

is deﬁned as



j=1

. Then, the Laplace matrix of G is deﬁned as

L = D − A,whereD = diag{d

, d

, ··· , d

}. G is undirected

if A is symmetric. We assume that G is undirected throughout

of this paper. For an undirected graph G, L is symmetric and

positive semideﬁnite. G is connected if and only if L has

a simple zero eigenvalue with L1

= 0

, and all the other

eigenvalues are positive [52].

III. C

OLLABORATIVE NEURODYNAMIC SYSTEM

This section concentrates on solving the multiobjective

optimization problem cooperatively by using multiple neural

networks. To establish a model in the framework of collabora-

tion, the key step is to decompose the optimization problem.

By using state decomposition method, problem (2) can be

equivalently converted to

min



i=1

)

s.t. L ˜x = 0, x

∈ X (3)

where ˜x = col{x

, x

, ··· , x

}∈R

, L = L ⊗ I

,andL is

the Laplacian matrix of a connected graph G with k vertexes.

It can be seen that each objective function in (3) has its own

state variable, which is independent of the others. Note that

L ˜x = 0 is equivalent to x

= x

=···= x

. Therefore, if ˜x

∗

is an optimal solution of (3), then ˜x

∗

can be written as 1

⊗x

∗

where x

∗

∈ R

is an optimal solution of (2). The converse

is also true. In the following, the collaborative neurodynamic

system is derived based on problem (3).

For notational convenience, denote

f ( ˜x) = ( f

), f

), ··· , f

))

˜g( ˜x) = col{g(x

), g(x

), ··· , g(x

)}

F( ˜x) = diag{∇ f

), ∇ f

), ··· , ∇ f

)}

G( ˜x) = diag{∇ g(x

), ∇ g(x

), ··· , ∇ g(x

)}

A = I

⊗ A, b = 1

⊗ b.

Under Assumption 1, ˜x

∗

is an optimal solution of (3) if and

only if there exist ˜y

∗

∈ R

, ˜z

∗

∈ R

and ˜η

∗

∈ R

such

that ( ˜x

∗

, ˜y

∗

, ˜z

∗

, ˜η

∗

) satisfying the following equations:

˜x = P



( ˜x − F( ˜x)w − G( ˜x) ˜y − A

˜z + L ˜η) (4a)

˜y = ( ˜y +˜g( ˜x))

, A˜x = b, L ˜x = 0. (4b)

where 

is deﬁned as

  

 ×  ×···×. Note that the

equations in (4) are equivalent to Karush–Kuhn–Tucker (KKT)

conditions of problem (3) [36]. According to (4), a collabora-

tive neurodynamic system consisting of k neural networks for

solving problem (1) is then formulated. Note that ˜y can be

written as ˜y = col{ y

, y

, ··· , y

} with y

∈ R

. Similarly,

we write that ˜z = col{ z

, z

, ··· , z

} with z

∈ R

and

Authorized licensed use limited to: Southeast University. Downloaded on March 10,2022 at 22:18:57 UTC from IEEE Xplore. Restrictions apply.

剩余11页未读，继续阅读

评论收藏

内容反馈

Mx_wangDD

粉丝: 5
资源: 2

强化学习 深度学习 神经网络 论文

神经网络：强化学习高频面试题整理

论文研究 - 使用深度量子神经网络进行强化学习

硕士论文：自动驾驶中的深度强化学习：用于使汽车学习在TORCS中驾驶的A3C算法； Python 3.5，Tensorflow，张量板，numpy，gym-torcs，ubuntu，乳胶

master-thesis:关于强化学习（RL），深度RL和产品交付优化的硕士学位论文

计算机围棋强化学习中的神经网络

基于深度强化学习的交通信号灯控制论文

论文研究-深度学习应用技术研究.pdf

人工智能-项目实践-强化学习-基于深度强化学习的资源调度研究.zip

深度强化学习极简入门与Pytorch实战

深度学习理论与架构最新进展综述论文

使用深度强化学习的虚拟自动驾驶汽车-研究论文

基于matlab的表情识别代码-Deep-learning-papers:深度学习论文

论文研究-深度学习在图像识别中的应用研究综述.pdf

AI Paper阅读记录与收藏：机器学习/深度学习/自然语言处理/计算机视觉/智能语音/推荐系统/知识图谱

图深度学习(GraphDL)，下一个人工智能算法热点

图神经网络(GNN)文献大列表-python

reading-list:我已阅读并发现有趣的深度学习论文博客列表

GA-NN-Car:一个 MATLAB 简单的交互式强化学习环境，用于带有接近传感器的基于进化神经网络的汽车-matlab开发

相关实用应用程序（Windows可用）

免费可用的ChatGPT网页版.zip

ChatGPT使用总结：150个ChatGPT提示词模板（完整版）

chromedriver-win64.zip

全国计算机二级WPSoffice精选350道选择题题库（含答案）.pdf

哈尔滨工业大学-ChatGPT调研报告-2023.3.6-94页.pdf

2023泛娱乐社交出海手册-ZEGO即构科技

4个亲测好用的ChatGPT4渠道

HAI-2024斯坦福AI指数报告（中文译版）.pdf

学术海报模板+论文科研+研究生

北森能力测评题库.zip

最新资源

强化学习深度学习神经网络论文