【免费】图神经网络——分类、进展和趋势(2020)1资源-CSDN文库

需积分: 0 87 浏览量 2022-08-03 22:01:47 上传评论收藏 1.25MB PDF 举报

资源详情

资源评论

资源推荐

Graph Neural Networks: Taxonomy, Advances and Trends

YU ZHOU

∗

, College of Data Science/Shanxi Spatial Information Network Engineering Technology Research

Center, Taiyuan University of Technology, China

HAIXIA ZHENG

†

, College of Data Science/Shanxi Spatial Information Network Engineering Technology

Research Center, Taiyuan University of Technology, China

XIN HUANG, College of Data Science, Taiyuan University of Technology, China

DENGAO LI, College of Data Science/Shanxi Spatial Information Network Engineering Technology Research

Center, Taiyuan University of Technology, China

JUMIN ZHAO, College of Information and Computer/Shanxi Intelligent Perception Engineering Research

Center, Taiyuan University of Technology, China

Graph neural networks provide a powerful toolkit for embedding real-world graphs into low-dimensional

spaces according to specic tasks. Up to now, there have been several surveys on this topic. However, they

usually lay emphasis on dierent angles so that the readers can not see a panorama of the graph neural

networks. This survey aims to overcome this limitation, and provide a systematic and comprehensive review

on the graph neural networks. First of all, we provide a novel taxonomy for the graph neural networks, and

then refer to up to 250 relevant literatures to show the panorama of the graph neural networks. All of them

are classied into the corresponding categories. In order to drive the graph neural networks into a new stage,

we summarize four future research directions so as to overcome the facing challenges. It is expected that more

and more scholars can understand and exploit the graph neural networks, and use them in their research

community.

CCS Concepts: • Computing methodologies → Neural networks; Learning latent representations.

Additional Key Words and Phrases: Graph Convolutional Neural Network, Graph Recurrent Neural Network,

Graph Pooling Operator, Graph Attention Mechanism, Graph Neural Network

ACM Reference Format:

Yu Zhou, Haixia Zheng, Xin Huang, Dengao Li, and Jumin Zhao. 2020. Graph Neural Networks: Taxonomy,

Advances and Trends. ACM Trans. Intell. Syst. Technol. 00, 0, Article 000 ( 2020), 42 pages. https://doi.org/0000

∗

Corresponding Author.

†

Corresponding Author.

Authors’ addresses: Yu Zhou, zhouyu@tyut.edu.cn, College of Data Science/Shanxi Spatial Information Network Engineering

Technology Research Center, Taiyuan University of Technology, No. 79 Yingze West Street, WanBaiLin District, Taiyuan,

Shanxi, China, 030024; Haixia Zheng, zhenghaixia@tyut.edu.cn, College of Data Science/Shanxi Spatial Information Network

Engineering Technology Research Center, Taiyuan University of Technology, No. 79 Yingze West Street, WanBaiLin

District, Taiyuan, Shanxi, China, 030024; Xin Huang, huangxin@tyut.edu.cn, College of Data Science, Taiyuan University of

Technology, No. 79 Yingze West Street, WanBaiLin District, Taiyuan, Shanxi, China, 030024; Dengao Li, lidengao@tyut.

edu.cn, College of Data Science/Shanxi Spatial Information Network Engineering Technology Research Center, Taiyuan

University of Technology, No. 79 Yingze West Street, WanBaiLin District, Taiyuan, Shanxi, China, 030024; Jumin Zhao,

zhaojumin@tyut.edu.cn, College of Information and Computer/Shanxi Intelligent Perception Engineering Research Center,

Taiyuan University of Technology, No. 79 Yingze West Street, WanBaiLin District, Taiyuan, Shanxi, China, 030024.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee

provided that copies are not made or distributed for prot or commercial advantage and that copies bear this notice and

the full citation on the rst page. Copyrights for components of this work owned by others than ACM must be honored.

Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires

prior specic permission and/or a fee. Request permissions from permissions@acm.org.

2157-6904/2020/0-ART000 $15.00

https://doi.org/0000

ACM Trans. Intell. Syst. Technol., Vol. 00, No. 0, Article 000. Publication date: 2020.

arXiv:2012.08752v2 [cs.LG] 18 Jan 2021

000:2 Yu Zhou, et al.

1 INTRODUCTION

Graph, as a complex data structure, consists of nodes (or vertices) and edges (or links). It can be used

to model lots of complex systems in real world, e.g. social networks, protein-protein interaction

networks, brain networks, road networks, physical interaction networks and knowledge graph etc.

Thus, Analyzing the complex networks becomes an intriguing research frontier. With the rapid

development of deep learning techniques, many scholars employ the deep learning architectures

to tackle the graphs. Graph Neural Networks (GNNs) emerge under these circumstances. Up to

now, the GNNs have evolved into a prevalent and powerful computational framework for tackling

irregular data such as graphs and manifolds.

The GNNs can learn task-specic node/edge/graph representations via hierarchical iterative

operators so that the traditional machine learning methods can be employed to perform graph-

related learning tasks, e.g. node classication, graph classication, link prediction and clustering

etc. Although the GNNs has attained substantial success over the graph-related learning tasks, they

still face great challenges. Firstly, the structural complexity of graphs incurs expensive computa-

tional cost on large graphs. Secondly, perturbing the graph structure and/or initial features incurs

sharp performance decay. Thirdly, the Wesfeiler-Leman (WL) graph isomorphism test impedes

the performance improvement of the GNNs. At last, the blackbox work mechanism of the GNNs

hinders safely deploying them to real-world applications.

In this paper, we generalize the conventional deep architectures to the non-Euclidean domains,

and summarize the architectures, extensions and applications, benchmarks and evaluation pitfalls

and future research directions of the graph neural networks. Up to now, there have been several

surveys on the GNNs. However, they usually discuss the GNN models from dierent angles and with

dierent emphasises. To the best of our knowledge, the rst survey on the GNNs was conducted

by Michael M. Bronstein et al[

124

]. Peng Cui et al[

249

] reviewed dierent kinds of deep learning

models applied to graphs from three aspects: semi-supervised learning methods including graph

convolutional neural networks, unsupervised learning methods including graph auto-encoders, and

recent advancements including graph recurrent neural networks and graph reinforcement learning.

This survey laid emphasis on semi-supervised learning models, i.e. the spatial and spectral graph

convolutional neural networks, yet comparatively less emphasis on the other two aspects. Due to the

space limit, this survey only listed a few of key applications of the GNNs, but ignored the diversity of

the applications. Maosong Sun et al[

] provided a detailed review of the spectral and spatial graph

convolutional neural networks from three aspects: graph types, propagation step and training

method, and divided its applications into three scenarios: structural scenarios, non-structural

scenarios and other scenarios. However, this article did not involve the other GNN architectures

such as graph auto-encoders, graph recurrent neural networks and graph generative networks.

Philip S. Yu et al[

250

] conducted a comprehensive survey on the graph neural networks, and

investigated available datasets, open-source implementations and practical applications. However,

they only listed a few of core literatures on each research topic. Davide Bacciu et al[

] gives

a gentle introduction to the eld of deep learning for graph data. The goal of this article is to

introduce the main concepts and building blocks to construct neural networks for graph data, and

therefore it falls short of an exposition of recent works on graph neural networks.

It is noted that all of the aforementioned surveys do not concern capability and interpretability of

GNNs, combinations of the probabilistic inference and GNNs, and adversarial attacks on graphs. In

this article, we provide a panorama of GNNs for readers from 4 perspectives: architectures, exten-

sions and applications, benchmarks and evaluations pitfalls, future research directions, as shown in

Fig. 1. For the architectures of GNNs, we investigate the studies on graph convolutional neural net-

works (GCNNs), graph pooling operators, graph attention mechanisms and graph recurrent neural

ACM Trans. Intell. Syst. Technol., Vol. 00, No. 0, Article 000. Publication date: 2020.

Graph Neural Networks: Taxonomy, Advances and Trends 000:3

Graph Neural Networks

Architectures

Benchmarks & Evaluation Pitfalls

Extensions & Applications

Future Research Directions

Graph Convolutional Neural Networks

Graph Recurrent Neural Networks

Capabilities and Interpretability

Deep Graph Representation Learning

Deep Graph Generative Models

Combinations of the PI and GNNs

Adversarial Attacks for the GNNS

Graph Neural Architecture Search

Graph Reinforcement Learning

Applications

Robust GNNs

GNNs Going Beyond WL Test

Interpretable GNNs

Benchmarks

Evaluation Pitfalls

Spectral Graph Convolution Operators

Spatial Graph Convolution Operators

Graph Wavelet Neural Networks

GCNNs on Special Graphs

Highly Scalable GNNs

Graph Poling Operators

Graph Attention Mechanisms

Computer Vision

Natural Language Processing

Complex Network

Combinatorial Optimization

Knowledge Graph

Chemistry and Biology

Physical System

Programming Source Code

Recommender System

Intelligent Traffic

Graph LSTM

GRNNs for Dynamic Graphs

GRNNs based on Vanilla RNNs

Global Graph Pooling Operators

Hierarchical Graph Pooling Operators

Softmax-based Graph Attention

Similarity-based Graph Attention

Spectral Graph Attention

Attention-Guided Walk

Capability

Interpretability

Network Embedding based on GNNs

Conditional Random Field Layer Preserving Similarities between Nodes

Conditional GCNNs for Semi-supervised Node Classification

GCNN-based Gaussian Process Inference

Other GCNN-based Probabilistic Inference

Adversarial Attacks on Graphs

Defense against the Adversarial Attacks

Fig. 1. The architecture of this paper.

networks (GRNNs). The extensions and applications demonstrate some notable research topics on

the GNNs through integrating the above architectures. Specically, this perspective includes the

capabilities and interpretability, deep graph representation learning, deep graph generative models,

combinations of the Probabilistic Inference (PI) and the GNNs, adversarial attacks for GNNs, Graph

Neural Architecture Search and graph reinforcement learning and applications. In summary, our

article provides a complete taxonomy for GNNs, and comprehensively review the current advances

and trends of the GNNs. These are our main dierences from the aforementioned surveys.

Contributions. Our main contributions boils down to the following three-fold aspects.

(1)

We propose a novel taxonomy for the GNNs, which has three levels. The rst includes

architectures, benchmarks and evaluation pitfalls, and applications. The architectures are

classied into 9 categories, the benchmarks and evaluation pitfalls into 2 categories, and the

ACM Trans. Intell. Syst. Technol., Vol. 00, No. 0, Article 000. Publication date: 2020.

000:4 Yu Zhou, et al.

applications into 10 categories. Furthermore, the graph convolutional neural networks, as a

classic GNN architecture, are again classied into 6 categories.

(2)

We provide a comprehensive review of the GNNs. All of the literatures fall into the corre-

sponding categories. It is expected that the readers not only understand the panorama of the

GNNs, but also comprehend the basic principles and various computation modules of the

GNNs through reading this survey.

(3)

We summarize four future research directions for the GNNs according to the current facing

challenges, most of which are not mentioned the other surveys. It is expected that the research

on the GNNs can progress into a new stage by overcoming these challenges.

Roadmap.

The remainder of this paper is organized as follows. First of all, we provide some

basic notations and denitions that will be often used in the following sections. Then, we start

reviewing the GNNs from 4 aspects: architectures in section 3, extensions and applications in

section 4, benchmarks and evaluation pitfalls in section 5 and future research directions in section

6. Finally, we conclude our paper.

2 PRELIMINARIES

In this section, we introduce relevant notations so as to conveniently describe the graph neural

network models. A simple graph can be denoted by

𝐺 = (𝑉, 𝐸)

where

𝑉

and

𝐸

respectively denote

the set of

𝑁

nodes (or vertices) and

𝑀

edges. Without loss of generality, let

𝑉 =

{

𝑣

, ··· , 𝑣

𝑁

}

and

𝐸 =

{

𝑒

, ··· , 𝑒

𝑀

}

. Each edge

𝑒

𝑗

∈ 𝐸

can be denoted by

𝑒

𝑗



𝑣

𝑠

𝑗

, 𝑣

𝑟

𝑗



where

𝑣

𝑠

𝑗

, 𝑣

𝑟

𝑗

∈ 𝑉

. Let

𝐴

𝐺

denote the adjacency matrix of

𝐺

where

𝐴

𝐺

(𝑠, 𝑟 ) =

1 i there is an edge between

𝑣

𝑠

and

𝑣

𝑟

. If

𝐺

is edge-weighted,

𝐴

𝐺

(𝑠, 𝑟 )

equals the weight value of the edge

(

𝑣

𝑠

, 𝑣

𝑟

)

. If

𝐺

is directed,



𝑣

𝑠

𝑗

, 𝑣

𝑟

𝑗



≠



𝑣

𝑟

𝑗

, 𝑣

𝑠

𝑗



and therefore

𝐴

𝐺

is asymmetric. A directed edge

𝑒

𝑗



𝑣

𝑠

𝑗

, 𝑣

𝑟

𝑗



is also called

an arch, i.e.

𝑒

𝑗



𝑣

𝑠

𝑗

, 𝑣

𝑠

𝑗



. Otherwise



𝑣

𝑠

𝑗

, 𝑣

𝑟

𝑗





𝑣

𝑟

𝑗

, 𝑣

𝑠

𝑗



and

𝐴

𝐺

is symmetric. For a node

𝑣

𝑠

∈ 𝑉

, let

𝑁

𝐺

(𝑣

𝑠

)

denote the set of neighbors of

𝑣

𝑠

, and

𝑑

𝐺

(𝑣

𝑠

)

denote the degree of

𝑣

𝑠

. If

𝐺

directed, let

𝑁

𝐺

(𝑣

𝑠

)

and

𝑁

−

𝐺

(𝑣

𝑠

)

respectively denote the incoming and outgoing neighbors of

𝑣

𝑠

and

𝑑

𝐺

(𝑣

𝑠

)

and

𝑑

−

𝐺

(𝑣

𝑠

)

respectively denote the incoming and outgoing degree of

𝑣

𝑠

. Given a vector

𝑎 = (𝑎

, ··· , 𝑎

𝑁

) ∈ R

𝑁

diag(𝑎)

(or

diag(𝑎

, ··· , 𝑎

𝑁

)

) denotes a diagonal matrix consisting of the

elements 𝑎

𝑛

, 𝑛 = 1, ··· , 𝑁 .

A vector

𝑥 ∈ R

𝑁

is called a 1-dimensional graph signal on

𝐺

. Similarly,

𝑋 ∈ R

𝑁 ×𝑑

is called a

𝑑-dimensiaonl

graph signal on

𝐺

. In fact,

𝑋

is also called a feature matrix of nodes on

𝐺

. Without

loss of generality, let

𝑋 [𝑗, 𝑘]

denote the

(𝑗, 𝑘)-th

entry of the matrix

𝑋 ∈ R

𝑁 ×𝑑

𝑋 [𝑗,

] ∈ R

𝑑

denote

the feature vector of the node

𝑣

𝑗

and

𝑋 [

, 𝑗]

denote the 1

-dimensional

graph signal on

𝐺

. Let

𝑁

denote a

𝑁 ×𝑁

identity matrix. For undirected graphs,

𝐿

𝐺

= 𝐷

𝐺

−𝐴

𝐺

is called the Laplacian matrix

𝐺

, where

𝐷

𝐺

[𝑟, 𝑟] =

𝑁

𝑐=1

𝐴

𝐺

[𝑟, 𝑐]

. For a 1-dimensional graph signal

𝑥

, its smoothness

𝑠 (𝑥)

dened as

𝑠 (𝑥) = 𝑥

𝑇

𝐿

𝐺

𝑥 =

𝑁



𝑟,𝑐=1

𝐴

𝐺

(𝑟, 𝑐)

(

𝑥 [𝑟] −𝑥 [𝑐]

)

. (1)

The normalization of

𝐿

𝐺

is dened by

𝐿

𝐺

= I

𝑁

− 𝐷

−

𝐺

𝐴

𝐺

𝐷

−

𝐺

𝐿

𝐺

is a real symmetric semi-

positive denite matrix. So, it has

𝑁

ordered real non-negative eigenvalues

{

𝜆

𝑛

: 𝑛 = 1, ··· , 𝑁

}

and corresponding orthonormal eigenvectors

{

𝑢

𝑛

: 𝑛 = 1, ··· , 𝑁

}

, namely

𝐿

𝐺

= 𝑈 Λ𝑈

𝑇

where

Λ = diag(𝜆

, ··· , 𝜆

𝑁

)

and

𝑈 =

(

𝑢

, ··· ,𝑢

𝑁

)

denotes a orthonormomal matrix. Without loss of

generality, 0

= 𝜆

≤ 𝜆

≤ ··· ≤ 𝜆

𝑁

= 𝜆

max

. The eigenvectors

𝑢

𝑛

, 𝑛 =

, ··· , 𝑁

are also called the

graph Fourier bases of

𝐺

. Obviously, the graph Fourier basis are also the 1-dimensional graph signal

ACM Trans. Intell. Syst. Technol., Vol. 00, No. 0, Article 000. Publication date: 2020.

剩余41页未读，继续阅读

评论收藏

内容反馈

FloritaScarlett

粉丝: 23
资源: 308

图神经网络——分类、进展和趋势(2020)1

评论0

最新资源

图神经网络——分类、进展和趋势(2020)1

评论0

2020年全球网络演习进展及趋势分析.docx

神经网络在CT肺结节的分类应用进展.pdf

2020年二季度电力设备新能源行业投资策略：关注疫情进展，新能源大趋势不改.pdf

北京智源人工智能研究院-2020年AI进展及2021年技术趋势报告.pdf

2020年人工智能十大技术进展及2021年十大技术趋势.pdf

图神经网络——图注意力网络（GAT）原始论文与源码

光子神经网络——重新定义AI芯片.pdf

应用于图像语义分割的神经网络——从SegNet到U-Net.pdf

RBF神经网络--分类和回归

2020年AI进展及2021年技术趋势报告-北京智源人工智能研究院-202101.pdf

2020年AI进展及2021年技术趋势行业报告

2020年AI进展及2021年技术趋势报告.pdf

思维进化算法优化BP神经网络——非线性函数拟合.zip

AlexNet卷积神经网络-猫狗大战（分类）.zip

ANN神经网络入门——分类问题（MATLAB）

BP神经网络——语音特征信号分类

LVQ神经网络的分类——乳腺肿瘤诊断

神经网络图像分类代码,图神经网络图像分类,matlab

ST-GDN——图神经网络预测交通流量代码

自己动手写神经网络——随书代码

图神经网络在视频分类中的应用，SFFAI-第27期图神经网络专题.zip

MATLAB 神经网络案例：并行运算与神经网络——基于CPUGPU的并行神经网络运算.zip

离散Hopfield神经网络的分类——高校科研能力评价(matlab实现).zip

BP神经网络——上证指数预测

离散Hopfield神经网络的分类——高校科研能力评价

BurpLoaderKeygen.jar.zip

最新版ISO/IEC 27001:2022、ISO 27002:2022中英文合集

最新资源