Heterogeneous Graph Information Bottleneck
Liang Yang
1,2,3
, Fan Wu
1
, Zichen Zheng
1
, Bingxin Niu
1,3
, Junhua Gu
1,3
,
Chuan Wang
2
, Xiaochun Cao
2
and Yuanfang Guo
4∗
1
School of Artificial Intelligence, Hebei University of Technology, Tianjin, China
2
State Key Laboratory of Information Security, Institute of Information Engineering, CAS, Beijing, China
3
Hebei Province Key Laboratory of Big Data Calculation, Hebei University of Technology, China
4
Beijing Advanced Innovation Center for Big Data and Brain Computing, School of Computer Science
and Engineering, Beihang University, Beijing, China
yangliang@vip.qq.com, andyguo@buaa.edu.cn
Abstract
Most attempts on extending Graph Neural Net-
works (GNNs) to Heterogeneous Information Net-
works (HINs) implicitly take the direct assump-
tion that the multiple homogeneous attributed net-
works induced by different meta-paths are com-
plementary. The doubts about the hypothesis of
complementary motivate an alternative assumption
of consensus. That is, the aggregated node at-
tributes shared by multiple homogeneous attributed
networks are essential for node representations,
while the specific ones in each homogeneous at-
tributed network should be discarded. In this pa-
per, a novel Heterogeneous Graph Information Bot-
tleneck (HGIB) is proposed to implement the con-
sensus hypothesis in an unsupervised manner. To
this end, information bottleneck (IB) is extended
to unsupervised representation learning by leverag-
ing self-supervision strategy. Specifically, HGIB
simultaneously maximizes the mutual information
between one homogeneous network and the repre-
sentation learned from another homogeneous net-
work, while minimizes the mutual information be-
tween the specific information contained in one ho-
mogeneous network and the representation learned
from this homogeneous network. Model analysis
reveals that the two extreme cases of HGIB corre-
spond to the supervised heterogeneous GNN and
the infomax on homogeneous graph, respectively.
Extensive experiments on real datasets demonstrate
that the consensus-based unsupervised HGIB sig-
nificantly outperforms most semi-supervised SOTA
methods based on complementary assumption.
1 Introduction
Heterogeneous Information Networks (HINs) possess the ad-
vantage of modeling rich relations in real work compared to
homogeneous networks, which have been well studied by the
researchers from mathematics, physics and computer science
[
Shi et al., 2017; Wang et al., 2020
]
. Thus, by effectively
∗
Corresponding author.
exploiting these multiple relations via meta-paths, HINs pro-
vide more clues for accurate network analysis, e.g. network
embedding
[
Dong et al., 2017
]
, and have been successfully
applied to recommendation system
[
Shi et al., 2019
]
, natural
language processing
[
Hu et al., 2019
]
and knowledge graph.
Graph neural networks (GNNs)
[
Wu et al., 2021
]
, espe-
cially graph convolutional neural networks (GCNNs)
[
Kipf
and Welling, 2017; Bruna et al., 2014
]
, have became a pow-
erful tool for homogeneous attributed network embedding.
And, their success can be attributed to the Laplacian smooth-
ing
[
Li et al., 2018
]
from spatial perspective or the low-pass
filtering
[
Wu et al., 2019
]
from spectral perspective.
Recent attempts extend GNNs to heterogeneous informa-
tion networks
[
Wang et al., 2019; Fu et al., 2020; Yun
et al., 2019; Hu et al., 2020
]
. Most of them follow the
pipeline of transforming a heterogeneous attributed network
with multiple relations into multiple attributed homogeneous
networks via meta-paths and combining the embedding re-
sults of multiple homogeneous attributed networks obtained
from GNNs. And, the supervision information is utilized
to learn how to map from node feature to label and how to
combine the multiple embedding results
[
Wang et al., 2019;
Yun et al., 2019
]
. These semi-supervised methods implic-
itly take the direct assumption that the multiple homoge-
neous attributed networks induced by different meta-paths are
complementary. That is, the information contained in each
homogeneous attributed network is insufficient to represent
nodes, thus, multiple homogeneous attributed networks are
necessary to complete the information.
Here, the direct assumption of complementarity is investi-
gated. The doubts about this hypothesis stem from both the
characteristic of the homogeneous attributed networks and
the nature of the adopted GNNs. First, the homogeneous at-
tributed networks induced by meta-paths are not independent.
In fact, they share the common node attributes (feature) and
possess different network topologies. Second, the essence
of GNNs, which are applied to each homogeneous attributed
network, is the attributes smoothing according to the topol-
ogy, i.e., discarding noises. Based on these two characteris-
tics, the same attributes are smoothed according to the dif-
ferent topologies of multiple homogeneous networks. Thus,
the smoothed node attributes in each homogeneous attributed
network may not be significantly different.
