SimulationandverificationofZhangneuralnetworkforonlinetime-varyingmatrixinversion资源-CSDN文库

需积分: 9 17 浏览量 2010-08-03 16:58:40 上传评论收藏 1.36MB PDF 举报

### 张神经网络在线时变矩阵求逆的模拟与验证 #### 概述本文讨论了张神经网络（Zhang Neural Network, ZNN）在在线时变矩阵求逆中的应用及其实现方法。该研究由中山大学信息科学与技术学院的Yunong Zhang、Chenfu Yi和Weimu Ma共同完成。 #### 关键知识点解析 ##### 1. 张神经网络（ZNN）张神经网络是一种特殊的递归神经网络，不同于基于梯度的神经网络（Gradient-Based Neural Networks, GNN），它特别设计用于实时求解时变矩阵的逆问题。ZNN的设计基于矩阵值误差函数而非标量值的规范能量函数，并且它的动态特性是隐式的而不是显式的。 **特点：** - **矩阵值误差函数**：ZNN采用了一种新的优化目标，即矩阵值误差函数，这种函数更适用于矩阵运算的场景。 - **隐式动态**：其动态更新规则是非显式的，这使得它可以更好地处理在线学习和实时计算任务。 ##### 2. 实时时变矩阵求逆实时时变矩阵求逆是指在数据流不断变化的情况下，实时地对矩阵进行求逆操作。这类问题常见于控制理论、信号处理、电磁系统、机器人控制等领域。 **应用场景：** - **优化**：如线性规划、非线性规划等优化问题中需要求解矩阵的逆。 - **信号处理**：滤波器设计、谱估计等。 - **机器人控制**：路径规划、动力学控制等。 ##### 3. 四项重要模拟技术为了模拟ZNN模型，研究人员采用了四项重要的模拟技术： - **克罗内克积**：通过克罗内克积将矩阵微分方程转换为向量微分方程，进而简化为标准的一阶常微分方程（Ordinary Differential Equation, ODE）。 - **MATLAB ode45**：利用MATLAB中的ode45函数来模拟转换后的初始值隐式ODE系统，其中包含质量矩阵属性。 - **符号数学工具箱**：利用符号数学工具箱中的diff函数来获得矩阵导数。 - **实施误差与激活函数**：分析了不同类型的实施误差以及多种激活函数的效果，进一步验证了ZNN模型的优势。 **关键技术：** - **克罗内克积**：这是一种高效的矩阵操作技术，可以将高维的矩阵运算转换为低维的向量运算。 - **ode45**：这是一个广泛使用的数值积分函数，可以高效准确地解决复杂的微分方程问题。 - **符号数学**：利用符号数学工具箱能够精确地处理复杂的数学表达式，包括矩阵导数的计算。 ##### 4. 实验验证文章还提供了三个具体的计算机仿真示例，这些示例证明了ZNN模型在线时变矩阵求逆方面的理论结果和有效性。 **实验案例：** - **案例一**：针对特定类型的数据集，验证ZNN模型在处理时变矩阵求逆问题时的准确性。 - **案例二**：探讨不同激活函数对于模型性能的影响。 - **案例三**：评估各种实施误差如何影响模型的稳定性和准确性。 #### 结论张神经网络作为一种创新的递归神经网络模型，在处理在线时变矩阵求逆问题方面展现出了显著的优势。通过对模型的详细设计、关键模拟技术的应用以及丰富的实验验证，本研究不仅证实了ZNN的有效性，也为未来的相关研究提供了一个坚实的基础。

资源推荐

资源详情

资源评论

Simulation and veriﬁcation of Zhang neural network for online

time-varying matrix inversion

Yunong Zhang

, Chenfu Yi, Weimu Ma

School of Information Science and Technology, Sun Yat-Sen University, Guangzhou 510275, China

article info

Article history:

Received 18 January 2009

Received in revised form 3 June 2009

Accepted 3 July 2009

Available online xxxx

Keywords:

Simulation and veriﬁcation

Recurrent neural network

Time-varying matrix inversion

Kronecker product

Symbolic math

abstract

Differing from gradient-based neural networks (GNN), a special kind of recurrent neural

network has recently been proposed by Zhang et al. for real-time inversion of time-varying

matrices. The design of such a recurrent neural network is based on a matrix-valued error

function instead of a scalar-valued norm-based energy-function. In addition, it is depicted

in an implicit dynamics instead of an explicit dynamics. This paper investigates the simu-

lation and veriﬁcation of such a Zhang neural network (ZNN). Four important simulation

techniques are employed to simulate this system: (1) Kronecker product of matrices is

introduced to transform a matrix-differential-equation (MDE) to a vector differential equa-

tion (VDE) [i.e., ﬁnally, there is a standard ordinary-differential-equation (ODE) formula-

tion]. (2) MATLAB routine ‘‘ode45” with a mass-matrix property is introduced to

simulate the transformed initial-value implicit ODE system. (3) Matrix derivatives are

obtained using the routine ‘‘diff” and symbolic math toolbox. (4) Various implementation

errors and different types of activation functions are investigated, further demonstrating

the advantages of the ZNN model. Three illustrative computer-simulation examples sub-

stantiate the theoretical results and efﬁcacy of the ZNN model for online time-varying

matrix inversion.

1. Introduction

Online matrix-inversion problems are widely encountered in science and engineering ﬁelds, e.g., in optimization [1,2],

signal-processing [3,4], electromagnetic systems [5], robot control [6,7], statistics [4,8], and physics [9].

There are two general types of solutions to the matrix-inversion problem. One type of solution is the numerical algo-

rithms performed on digital computers (i.e., on today’s computers). Usually, the minimal arithmetic operations for a ma-

trix-inversion algorithm are proportional to the cube of the matrix dimension n, i.e., Oðn

Þ operations. Consequently, such

serial-processing numerical algorithms may not be efﬁcient enough for large-scale online or real-time applications. To rem-

edy this computational problem, some Oðn

Þ-operation algorithms were proposed (e.g., in [4,8]); however, they may still not

be fast enough [4]. Many parallel processing methods, the second type of matrix inversion, have been proposed, analyzed,

and implemented on speciﬁc architectures [3,6,7,10–20].

The dynamic system approach has been an important parallel processing method for solving matrix-inversion problems

[3,6,10–15,21]. Recently, due to in-depth research in neural networks, numerous dynamic and analog solvers based on recur-

rent neural networks (RNN) have been developed and investigated [3,6,10–15,21–28]. Thus, the neural-dynamic approach is

now regarded as a powerful alternative for online computation in view of its parallel distributed nature and the convenience

of circuits implementation [6,7,16,20,29].

doi:10.1016/j.simpat.2009.07.001

* Corresponding author. Tel./fax: +86 20 84113597.

E-mail address: ynzhang@ieee.org (Y. Zhang).

Simulation Modelling Practice and Theory xxx (2009) xxx–xxx

Contents lists available at ScienceDirect

Simulation Modelling Practice and Theory

journal homepage: www.elsevier.com/locate/simpat

ARTICLE IN PRESS

Please cite this article in press as: Y. Zhang et al., Simulation and veriﬁcation of Zhang neural network for online time-varying matrix inver-

sion, Simulat. Modell. Pract. Theory (2009), doi:10.1016/j.simpat.20 09.07.001

In March 2001 [10–13], Zhang et al. proposed a special kind of RNN for the real-time inversion of time-varying matrices,

differing from gradient-based neural networks (GNN) for constant matrix inversion [6,14–16]. To solve for a time-varying

matrix inverse A

1

ðtÞ2R

nn

, the design of the neural network is based on the deﬁning equation, AðtÞXðtÞI ¼ 0, over time

t 2½0; þ1Þ, where AðtÞ2R

nn

is a smoothly time-varying nonsingular matrix, and I denotes an appropriately-dimensioned

identity matrix. As proposed and investigated in [10–13,21–25], the following recurrent neural network [simply termed,

Zhang neural network (ZNN), for ease of presentation] can be used to solve for the time-varying matrix inverse A

1

ðtÞ in real

time t:

AðtÞ

XðtÞ¼

Aðt ÞXðtÞ

FðAðtÞXðtÞIÞ ð1Þ

where

 XðtÞ2R

nn

, starting from initial condition Xð0Þ2R

nn

, is the activation state matrix corresponding to the theoretical

inverse A

1

ðtÞ;

 the matrix-valued parameter

C 2 R

nn

could simply be

I with

> 0 2 R;



AðtÞ2R

nn

denotes the derivative of matrix AðtÞ with respect to time t [or simply termed, the time derivative of matrix

AðtÞ]; and,

 FðÞ : R

nn

! R

nn

denotes an activation-function array of the neural network, of which a simple example is the linear one,

i.e., FðAðtÞXðtÞIÞ¼AðtÞXðtÞI.

As compared to gradient-based recurrent neural networks, the difference and novelty of the ZNN model lie in the follow-

ing facts.

(1) The design of ZNN model (1) is based on the elimination of every entry e

ðtÞ of the matrix-valued error function

EðtÞ¼AðtÞXðtÞI, where i; j 2f1; 2; ...; ng. In contrast, the design of the GNN model

XðtÞ¼

FðAXðtÞIÞ ð2Þ

is based on the elimination of the scalar-valued norm-based energy function EðtÞ¼kAXðtÞIk

(note that, in such a

GNN design, matrix A could only be constant) [6,14–16].

(2) ZNN model (1) is depicted in an implicit dynamics, i.e., Aðt Þ

XðtÞ¼, which coincides well with systems in nature

and in practice (e.g., with analogue electronic circuits and mechanical systems, owing to Kirchhoff’s and Newton’s

laws, respectively [10–13,21–25]). Evidently, with simple operations, the implicit systems can be transformed into

explicit systems [13,30], if necessary. In contrast, GNN model (2) is depicted in an explicit dynamics, i.e.,

XðtÞ¼,

which is usually associated with conventional Hopﬁeld-type and/or gradient-based neural networks [6,14–16].

Comparing the implicit and explicit dynamic systems, it can be seen that the former could have a greater ability

to preserve more physical parameters, e.g., even in the coefﬁcient matrix on the left hand side of the system [i.e.,

AðtÞ in (1)].

(3) ZNN model (1) could systematically and methodologically exploit the time-derivative information (or say, trend) of

coefﬁcient matrix AðtÞ during its real-time inverting process. This appears to be an important reason why the neural

state XðtÞ of ZNN model (1) could globally converge to the exact inverse of a time-varying matrix [12]. On the other

hand, GNN model (2) has not exploited this important information of

AðtÞ , and thus may not be effective enough on

the time-varying matrix inversion.

(4) By making good use of the time-derivative information of matrix AðtÞ, ZNN model (1) belongs to a predictive approach,

which is more effective on the system convergence to a time-varying theoretical inverse [10–13]. In contrast, GNN

model (2) belongs to a tracking approach, which adapts to the change of matrix AðtÞ in a posterior passive manner.

(5) As shown in the following sections, the neural state XðtÞ computed by ZNN model (1) can globally exponentially con-

verge

the theoretical time-varying inverse A

1

ðtÞ. In contrast, GNN model (2) can only generate the approximation

results of such a theoretical inverse A

1

ðtÞ with much larger steady-state errors, thus is less effective on time-varying

matrix inversion.

To the best of the authors’ knowledge, to date, few studies have been published dealing with the simulation of recurrent

neural networks. In this paper, the simulation and veriﬁcation of ZNN model (1) using MATLAB [31] is investigated. To sim-

ulate such a special implicit dynamic system, four important simulation techniques (as briefed below) are employed.

 Kronecker product of matrices is introduced to transform the matrix differential Eq. (1) to a vector differential equation

(VDE). That is, ﬁnally, there is a standard ordinary differential equation (ODE) formulation.

 MATLAB routine ‘‘ode45” with a mass-matrix property is introduced to simulate the transformed initial-value ODE sys-

tem, where matrix AðtÞ on the left hand side of ZNN model (1) is termed a mass matrix.

 Matrix derivatives [e.g.,

AðtÞ] are obtained using MATLAB routine ‘‘diff” and symbolic math toolbox [31].

 In addition to various implementation errors, different types of activation-function arrays FðÞ are coded and simulated in

order to illustrate the characteristics of ZNN model (1) working in different situations.

2 Y. Zhang et al. / Simulation Modelling Practice and Theory xxx (2009) xxx–xxx

ARTICLE IN PRESS

Please cite this article in press as: Y. Zhang et al., Simulation and veriﬁcation of Zhang neural network for online time-varying matrix inver-

sion, Simulat. Modell. Pract. Theory (2009), doi:10.1016/j.simpat.2009.07.001

Three illustrative computer-simulation examples further substantiate the theoretical results and efﬁcacy of ZNN model

(1) for online time-varying matrix inversion. The rest of this paper is organized as follows. In Section 2, the theoretical results

from ZNN model (1) are presented, especially focusing on the network convergence and robustness. In Section 3, four MAT-

LAB coding and simulation techniques are provided for ZNN model (1) which solves for A

1

ðtÞ in general situations. In Section

4, three illustrative examples (i.e., a sinusoidal-matrix inversion, a Toeplitz matrix inversion, and a constant matrix inver-

sion) are presented to show the network convergence and robustness. Concluding remarks are given in Section 5.

2. Theoretical results

Before presenting the theoretical results, a description of the design parameters appearing in ZNN model (1) is presented.

Similar to the conventional neural-network approaches, the matrix parameter

C ¼

I in Eq. (1), like a set of inductance

parameters or reciprocals of capacitance parameters in electronic circuits, is set as large as the hardware permits [29] or se-

lected appropriately for experimental and/or simulative purposes. Moreover, it follows from ZNN model (1) that different

design parameter values and activation-function arrays FðÞ lead to different ZNN performance. In general, any monotoni-

cally-increasing odd function can be chosen as the activation function of the neural network. Since March 2001 [21], four

types of activation functions (i.e., linear activation function, power activation function, sigmoid activation function and

power-sigmoid function) have been introduced and used for the proposed ZNN models (for more details, see [10–13,21–25]).

From the analysis results of [10–13,21–25], the following general observations on global convergence and exponential

convergence are summarized and presented for ZNN model (1).

Lemma 1. Given a smoothly time-varying nonsingular matrix AðtÞ2R

nn

, if a monotonically-increasing odd activation-function

array FðÞ is used, then the state matrix XðtÞ of ZNN model (1), starting from any initial state Xð0Þ, will always converge to the

time-varying theoretical inverse A

1

ðtÞ. In addition, more detailed properties of ZNN model (1) can be referred to in [10–13,21–

25].

The analog implementation or simulation of recurrent neural networks is usually assumed to be under ideal conditions.

However, there are always some realization errors involved. For example, the matrix-implementation error, differentiation

error, and the model-implementation error appear frequently in hardware simulation and realization. For the appearance of

these errors in ZNN model (1), the following lemma is used:

Lemma 2 ([10–12,21,23]). Consider the perturbed ZNN model

X ¼

AX 

Fð

AX  IÞ with imprecise matrix-implementation

A ¼ A þ

.Ifk

ðtÞk 6

for any time instant t 2½0; 1Þ, 90 6

< þ1, then the steady-state computational error

lim

t!1

kXðtÞA

1

ðtÞk is uniformly upper bounded by a positive scalar, provided that the invertibility of matrix

A still holds

true.

For the differentiation error and the model-implementation error, the following dynamics is considered, as compared to

the original dynamic Eq. (1).

Aðt Þ

XðtÞ¼ð

Aðt Þþ

ðtÞÞXðtÞ

FðAðtÞXðtÞIÞþ

ðtÞð3Þ

where

ðtÞ2R

nn

and

ðtÞ2R

nn

, respectively denote the differentiation error and the model-implementation error,

which result from truncating/roundoff errors in digital realization and/or high-order residual errors of circuit components

in analog realization. Generally speaking, if k

ðtÞk 6

and k

ðtÞk 6

for any t 2½0; 1Þ, then the computational error

kXðtÞA

1

ðtÞk is uniformly upper bounded by some scalar, and its steady-state error could be expressed as in Theorem 2

of [11]. More detailed robustness properties of ZNN model (3) can be seen in [10–12,21,23].

The robustness study could give us more insights on how to handle the problem of imperfect implementation of the pre-

sented ZNN model (1) and/or other related issues on neural network realization. In summary, we need to:

 implement matrix A as accurately as possible,

 use a large value of design parameter

if the hardware permits, and

 use the activation-function nonlinearity (e.g., using power-sigmoid activation functions) to remedy hardware-implemen-

tation errors.

3. Simulation techniques

While Section 2 presents the main theoretical results of the presented ZNN model (1), the following MATLAB simulation

techniques [31] are investigated in this section to show the characteristics of such a ZNN model.

3.1. Coding of activation function

To simulate the general matrix-inversion ZNN model (1), the activation functions are to be designed and coded ﬁrstly in

MATLAB. Inside the body of a user-deﬁned function, the routine ‘‘nargin” returns the number of input arguments, which are

Y. Zhang et al. / Simulation Modelling Practice and Theory xxx (2009) xxx–xxx

ARTICLE IN PRESS

Please cite this article in press as: Y. Zhang et al., Simulation and veriﬁcation of Zhang neural network for online time-varying matrix inver-

sion, Simulat. Modell. Pract. Theory (2009), doi:10.1016/j.simpat.20 09.07.001

剩余14页未读，继续阅读

评论收藏

内容反馈

Ethane

粉丝: 0
资源: 4

Simulation and verification of Zhang neural network for online t...

最新资源

Simulation and verification of Zhang neural network for online t...

Prediction of Market Demand Based on AdaBoost_BP Neural Network

论文研究-BP Neural Network principle and MATLAB Simulation.pdf

C++ for Neural-Networks-and fuzzy logic

Information-Theoretic Aspects of Neural Networks

A High-order Internal Model Based Iterative LearningControl Scheme for Discrete Linear Time-varying Systems

The Neural Simulation Language A System for Brain Modeling

Consensusabilization for continuous-time high-order multiagent systems with time-varying delays

Simulation of packet-switched network

[源代码]simulation and software radio for mobile communications.rar

09067568.pdf

Simulation and Software Radio for Mobile Communications

Distributed Optimal Consensus Control for Multiagent Systems With Input Delay

Computer.Network.Simulation.Using.NS2.epub

Design and Simulation of Multi Channel UART for Serial Communication

simulation and software radio for communications 源码

Computer Simulation of Liquids 免费下载

论文研究-Kinematics and Control of an omni-directional mobile microrobot for a micro-factory.pdf

Intelligent Adaptive Control: Industrial Applications

Neural Network Applications in

Simulation and Evaluation for a Network on Chip Architecture Using Ns-2.pdf

System Dynamics_ Modeling, Simulation, and Control of Mechatronic Systems

Aircraft control and simulation

Network Modeling and Simulation A Practical Perspective

Numerical simulation of coal seam joints and stiffness effects on coal bumps

最新资源