图卷积神经网络的应用研究_卷积神经网络的应用资源-CSDN文库

需积分: 50 183 浏览量 2019-04-01 14:41:04 上传评论收藏 2.67MB PDF 举报

图卷积神经网络（Graph Convolutional Networks，GCNs）是近年来兴起的一种神经网络架构，主要用于处理图结构数据，即节点与节点之间通过边相连形成的数据结构。在深度学习领域，GCNs作为一种重要的图神经网络（Graph Neural Networks，GNNs），在多个领域展现出了显著的应用前景和效果。 1. 图结构数据特点与应用领域：图是由节点（对象）和边（关系）组成的数据结构。图数据结构在机器学习分析领域越来越受到关注，这是因为图的表达能力非常强大，可以表示各种系统中的大量对象及其相互关系。在社会科学领域，如社交网络可以表示为图结构；在自然科学领域，物理系统、蛋白质-蛋白质相互作用网络等可以通过图来建模；此外，知识图谱和其他研究领域也广泛使用图结构。图数据结构具有非欧几里得的特性，这为传统的神经网络处理带来了挑战。 2. 图神经网络模型：GNNs是通过节点间的消息传递来捕捉图依赖性的连接主义模型。它们与标准的神经网络不同，能够保持一个状态，这个状态能够代表任意深度邻居的信息。图神经网络的设计目标是能够在每个节点上聚合其邻居节点的信息，以学习到整个图的表示。 3. 训练难题与突破：最初版本的GNN存在训练困难的问题，尤其是在固定点不易训练。然而，最近在网络架构、优化技术以及并行计算方面取得的进展使得图神经网络的训练变得可行。这为GNN在各种任务上的应用提供了坚实的基础。 4. 应用实例与系统性能：基于图卷积网络（GCN）和门控图神经网络（GGNN）的系统近年来在许多提到的任务上展示了开创性的性能。例如，在建模物理系统、学习分子指纹、预测蛋白质界面以及分类疾病等领域，GNNs已经证明它们能够从图输入中学习模型。在其他领域，如从非结构化的文本和图像数据中学习时，利用提取的结构进行推理（例如，句子的依赖树和图像的场景图）也是一个重要的研究课题，这同样需要图推理模型。 5. 研究现状与未来方向：本文提供了一个关于现有图神经网络模型的详细回顾，系统地分类了其应用，并提出了未来研究的四个开放问题。由于图神经网络的多样性和潜力，它们在深度学习领域的研究和应用正逐渐成为一个活跃的前沿领域。总结而言，图卷积神经网络之所以能够受到广泛关注，是因为它突破了传统机器学习方法在图结构数据上的局限，能够有效处理复杂的、非欧几里得的数据结构。它们在结构化推理、图形识别、知识图谱构建、社交网络分析、推荐系统等众多领域都有潜在应用价值。随着图神经网络技术的不断发展和完善，未来的应用前景将更加广阔。

资源推荐

资源详情

资源评论

Graph Neural Networks:

A Review of Methods and Applications

Jie Zhou

∗

, Ganqu Cui

∗

, Zhengyan Zhang

∗

, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li,

Maosong Sun

Abstract—Lots of learning tasks require dealing with graph data which contains rich relation information among elements. Modeling

physics system, learning molecular ﬁngerprints, predicting protein interface, and classifying diseases require that a model learns from

graph inputs. In other domains such as learning from non-structural data like texts and images, reasoning on extracted structures, like

the dependency tree of sentences and the scene graph of images, is an important research topic which also needs graph reasoning

models. Graph neural networks (GNNs) are connectionist models that capture the dependence of graphs via message passing

between the nodes of graphs. Unlike standard neural networks, graph neural networks retain a state that can represent information

from its neighborhood with arbitrary depth. Although the primitive GNNs have been found difﬁcult to train for a ﬁxed point, recent

advances in network architectures, optimization techniques, and parallel computation have enabled successful learning with them. In

recent years, systems based on graph convolutional network (GCN) and gated graph neural network (GGNN) have demonstrated

ground-breaking performance on many tasks mentioned above. In this survey, we provide a detailed review over existing graph neural

network models, systematically categorize the applications, and propose four open problems for future research.

Index Terms—Deep Learning, Graph Neural Network

1 INTRODUCTION

Graphs are a kind of data structure which models a set

of objects (nodes) and their relationships (edges). Recently,

researches of analyzing graphs with machine learning have

been receiving more and more attention because of the

great expressive power of graphs, i.e. graphs can be used

as denotation of a large number of systems across various

areas including social science (social networks) [1], [2], nat-

ural science (physical systems [3], [4] and protein-protein

interaction networks [5]), knowledge graphs [6] and many

other research areas [7]. As a unique non-Euclidean data

structure for machine learning, graph analysis focuses on

node classiﬁcation, link prediction, and clustering. Graph

neural networks (GNNs) are deep learning based methods

that operate on graph domain. Due to its convincing per-

formance and high interpretability, GNN has been a widely

applied graph analysis method recently. In the following

paragraphs, we will illustrate the fundamental motivations

of graph neural networks.

The ﬁrst motivation of GNNs roots in convolutional

neural networks (CNNs) [8]. CNNs have the ability to

extract multi-scale localized spatial features and compose

* indicates equal contribution.

• Jie Zhou, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu (corresponding

author) and Maosong Sun are with the Department of Computer Science

and Technology, Tsinghua University, Beijing 100084, China.

E-mail: {zhoujie18, zhangzhengyan14, cheng-ya14}@mails.tsing-

hua.edu.cn, {sms, liuzy}@tsinghua.edu.cn

• Ganqu Cui is with the Department of Physics, Tsinghua University,

Beijing 100084, China.

Email: cgq15@mails.tsinghua.edu.cn

• Lifeng Wang, Changcheng Li are with the Tencent Incorporation, Shen-

zhen, China.

Email:{fandywang, harrychli}@tencent.com

them to construct highly expressive representations, which

led to breakthroughs in almost all machine learning areas

and started the new era of deep learning [9]. However,

CNNs can only operate on regular Euclidean data like

images (2D grid) and text (1D sequence) while these data

structures can be regarded as instances of graphs. As we

are going deeper into CNNs and graphs, we found the keys

of CNNs: local connection, shared weights and the use of

multi-layer [9]. These are also of great importance in solving

problems of graph domain, because 1) graphs are the most

typical locally connected structure. 2) shared weights reduce

the computational cost compared with traditional spectral

graph theory [10]. 3) multi-layer structure is the key to deal

with hierarchical patterns, which captures the features of

various sizes. Therefore, it is straightforward to think of

ﬁnding the generalization of CNNs to graphs. However, as

shown in 1, it is hard to deﬁne localized convolutional ﬁlters

and pooling operators, which hinders the transformation of

CNN from Euclidean domain to non-Euclidean domain.

Fig. 1. Left: image in Euclidean space. Right: graph in non-Euclidean

space

The other motivation comes from graph embedding, which

learns to represent graph nodes, edges or subgraphs in low-

arXiv:1812.08434v3 [cs.LG] 7 Mar 2019

dimensional vectors. In the ﬁeld of graph analysis, tradi-

tional machine learning approaches usually rely on hand

engineered features and are limited by its inﬂexibility and

high cost. Following the idea of representation learning and

the success of word embedding [11], DeepWalk [12], which

is regarded as the ﬁrst graph embedding method based

on representation learning, applies SkipGram model [11]

on the generated random walks. Similar approaches such

as node2vec [13], LINE [14] and TADW [15] also achieved

breakthroughs. However, these methods suffer two severe

drawbacks [16]. First, no parameters are shared between

nodes in the encoder, which leads to computationally in-

efﬁciency, since it means the number of parameters grows

linearly with the number of nodes. Second, the direct em-

bedding methods lack the ability of generalization, which

means they cannot deal with dynamic graphs or generalize

to new graphs.

Based on CNNs and graph embedding, graph neural

networks (GNNs) are proposed to collectively aggregate in-

formation from graph structure. Thus they can model input

and/or output consisting of elements and their dependency.

Further, graph neural network can simultaneously model

the diffusion process on the graph with the RNN kernel.

In the following part, we explain the fundamental rea-

sons why graph neural networks are worth investigating.

Firstly, the standard neural networks like CNNs and RNNs

cannot handle the graph input properly in that they stack

the feature of nodes by a speciﬁc order. However, there

isn’t a natural order of nodes in the graph. To present a

graph completely, we should traverse all the possible orders

as the input of the model like CNNs and RNNs, which is

very redundant when computing. To solve this problem,

GNNs propagate on each node respectively, ignoring the

input order of nodes. In other words, the output of GNNs

is invariant for the input order of nodes. Secondly, an

edge in a graph represents the information of dependency

between two nodes. In the standard neural networks, the

dependency information is just regarded as the feature of

nodes. However, GNNs can do propagation guided by

the graph structure instead of using it as part of features.

Generally, GNNs update the hidden state of nodes by a

weighted sum of the states of their neighborhood. Thirdly,

reasoning is a very important research topic for high-level

artiﬁcial intelligence and the reasoning process in human

brain is almost based on the graph which is extracted from

daily experience. The standard neural networks have shown

the ability to generate synthetic images and documents by

learning the distribution of data while they still cannot learn

the reasoning graph from large experimental data. However,

GNNs explore to generate the graph from non-structural

data like scene pictures and story documents, which can be

a powerful neural model for further high-level AI. Recently,

it has been proved that an untrained GNN with a simple

architecture also perform well [17].

There exist several comprehensive reviews on graph

neural networks. [18] gives a formal deﬁnition of early

graph neural network approaches. And [19] demonstrates

the approximation properties and computational capabil-

ities of graph neural networks. [20] proposed a uniﬁed

framework, MoNet, to generalize CNN architectures to

non-Euclidean domains (graphs and manifolds) and the

framework could generalize several spectral methods on

graphs [2], [21] as well as some models on manifolds [22],

[23]. [24] provides a thorough review of geometric deep

learning, which presents its problems, difﬁculties, solutions,

applications and future directions. [20] and [24] focus on

generalizing convolutions to graphs or manifolds, how-

ever in this paper we only focus on problems deﬁned on

graphs and we also investigate other mechanisms used in

graph neural networks such as gate mechanism, attention

mechanism and skip connection. [25] proposed the message

passing neural network (MPNN) which could generalize

several graph neural network and graph convolutional net-

work approaches. It presents the deﬁnition of the message

passing neural network and demonstrates its application on

quantum chemistry. [26] proposed the non-local neural net-

work (NLNN) which uniﬁes several “self-attention”-style

methods. However, the model is not explicitly deﬁned on

graphs in the original paper. Focusing on speciﬁc applica-

tion domains, [25] and [26] only give examples of how to

generalize other models using their framework and they

do not provide a review over other graph neural network

models. [27] proposed the graph network (GN) framework.

The framework has a strong capability to generalize other

models and its relational inductive biases promote combi-

natorial generalization, which is thought to be a top priority

for AI. However, [27] is part position paper, part review

and part uniﬁcation and it only gives a rough classiﬁcation

of the applications. In this paper, we provide a thorough

review of different graph neural network models as well as

a systematic taxonomy of the applications.

To summarize, this paper presents an extensive survey

of graph neural networks with the following contributions.

• We provide a detailed review over existing graph

neural network models. We introduce the original

model, its variants and several general frameworks.

We examine various models in this area and provide

a uniﬁed representation to present different propaga-

tion steps in different models. One can easily make a

distinction between different models using our repre-

sentation by recognizing corresponding aggregators

and updaters.

• We systematically categorize the applications and

divide the applications into structural scenarios, non-

structural scenarios and other scenarios. We present

several major applications and their corresponding

methods in different scenarios.

• We propose four open problems for future research.

Graph neural networks suffer from over-smoothing

and scaling problems. There are still no effective

methods for dealing with dynamic graphs as well

as modeling non-structural sensory data. We provide

a thorough analysis of each problem and propose

future research directions.

The rest of this survey is organized as follows. In Sec. 2,

we introduce various models in the graph neural network

family. We ﬁrst introduce the original framework and its

limitations. And then we present its variants that try to

release the limitations. And ﬁnally, we introduce several

general frameworks proposed recently. In Sec. 3, we will in-

troduce several major applications of graph neural networks

applied to structural scenarios, non-structural scenarios and

other scenarios. In Sec. 4, we propose four open problems

of graph neural networks as well as several future research

directions. And ﬁnally, we conclude the survey in Sec. 5.

2 MODELS

Graph neural networks are useful tools on non-Euclidean

structures and there are various methods proposed in the

literature trying to improve the model’s capability.

In Sec 2.1, we describe the original graph neural net-

works proposed in [18]. We also list the limitations of

the original GNN in representation capability and train-

ing efﬁciency. In Sec 2.2 we introduce several variants of

graph neural networks aiming to release the limitations.

These variants operate on graphs with different types, uti-

lize different propagation functions and advanced training

methods. In Sec 2.3 we present three general frameworks

which could generalize and extend several lines of work.

In detail, the message passing neural network (MPNN) [25]

uniﬁes various graph neural network and graph convolu-

tional network approaches; the non-local neural network

(NLNN) [26] uniﬁes several “self-attention”-style methods.

And the graph network(GN) [27] could generalize almost

every graph neural network variants mentioned in this

paper.

Before going further into different sections, we give

the notations that will be used throughout the paper. The

detailed descriptions of the notations could be found in

Table 1.

2.1 Graph Neural Networks

The concept of graph neural network (GNN) was ﬁrst

proposed in [18], which extended existing neural networks

for processing the data represented in graph domains. In a

graph, each node is naturally deﬁned by its features and

the related nodes. The target of GNN is to learn a state

embedding h

∈ R

which contains the information of

neighborhood for each node. The state embedding h

is an

s-dimension vector of node v and can be used to produce

an output o

such as the node label. Let f be a parametric

function, called local transition function, that is shared among

all nodes and updates the node state according to the input

neighborhood. And let g be the local output function that

describes how the output is produced. Then, h

and o

are

deﬁned as follows:

= f(x

, x

co[v]

, h

ne[v]

, x

ne[v]

) (1)

= g(h

, x

) (2)

where x

, x

co[v]

, h

ne[v]

, x

ne[v]

are the features of v, the fea-

tures of its edges, the states, and the features of the nodes in

the neighborhood of v, respectively.

Let H, O, X, and X

be the vectors constructed by

stacking all the states, all the outputs, all the features, and

TABLE 1

Notations used in this paper.

Notations Descriptions

m-dimensional Euclidean space

a, a, A Scalar, vector, matrix

Matrix transpose

Identity matrix of dimension N

? x Convolution of g

and x

N Number of nodes in the graph

Number of nodes in the graph

Number of edges in the graph

Neighborhood set of node v

Vector a of node v at time step t

Hidden state of node v

Hidden state of node v at time step t

Features of edge from node v to w

Features of edge with label k

Output of node v

, U

, ...

Matrices for computing i, o, ...

, b

, ... Vectors for computing i, o, ...

σ The logistic sigmoid function

ρ An alternative non-linear function

tanh The hyperbolic tangent function

LeakyReLU The LeakyReLU function

 Element-wise multiplication operation

k Vector concatenation

all the node features, respectively. Then we have a compact

form as:

H = F (H, X) (3)

O = G(H, X

) (4)

where F , the global transition function, and G, the global

output function are stacked versions of f and g for all nodes

in a graph, respectively. The value of H is the ﬁxed point of

Eq. 3 and is uniquely deﬁned with the assumption that F is

a contraction map.

With the suggestion of Banach’s ﬁxed point theorem [28],

GNN uses the following classic iterative scheme for comput-

ing the state:

t+1

= F (H

, X) (5)

where H

denotes the t-th iteration of H. The dynamical

system Eq. 5 converges exponentially fast to the solution of

Eq. 3 for any initial value H(0). Note that the computations

described in f and g can be interpreted as the feedforward

neural networks.

When we have the framework of GNN, the next question

is how to learn the parameters of f and g. With the target

information (t

for a speciﬁc node) for the supervision, the

loss can be written as follow:

loss =

i=1

− o

) (6)

where p is the number of supervised nodes. The learning

algorithm is based on a gradient-descent strategy and is

composed of the following steps.

剩余19页未读，继续阅读

评论收藏

内容反馈

Yngly

粉丝: 0
资源: 2

图卷积神经网络的应用研究

卷积神经网络在图像分类中的应用研究_吴正文

卷积神经网络知识图谱

卷积神经网络的训练流程图

CNN卷积神经网络应用于人脸识别（带详细流程+代码实现）

卷积神经网络图像分类应用研究

卷积神经网络研究综述_周飞燕.pdf

深度卷积神经网络在计算机视觉中的应用研究综述_卢宏涛.pdf

如何用卷积神经网络快速打造图片识别应用

卷积神经网络CNN框架的实现与应用

图卷积神经网络及其应用（Graph neural networks） - ICLR 2019.zip

卷积神经网络案例-基于tensorflow的人脸识别

卷积神经网络研究综述.pdf

卷积神经网络相关文章学习资料22个合集.zip

卷积神经网络预测实例

BN层在卷积神经网络 应用实例（python）

Python-图卷积神经网络相关资源

2018-arXiv-Graph Neural Networks A Review of Methods and Applications.pdf

卷积神经网络课程报告

论文研究-基于深度卷积神经网络的快速图像分类算法.pdf

最新资源

BN层在卷积神经网络应用实例（python）