没有合适的资源?快使用搜索试试~ 我知道了~
第一代gcn的应用1
需积分: 0 0 下载量 198 浏览量
2022-08-03
14:52:56
上传
评论
收藏 4.57MB PDF 举报
温馨提示
试读
10页
第一代gcn的应用1
资源详情
资源评论
资源推荐
Deep Convolutional Networks on Graph-Structured
Data
Mikael Henaff
Courant Institute of Mathematical Sciences
New York University
mbh305@nyu.edu
Joan Bruna
University of California, Berkeley
joan.bruna@berkeley.edu
Yann LeCun
Courant Institute of Mathematical Sciences
New York University
yann@cs.nyu.edu
Abstract
Deep Learning’s recent successes have mostly relied on Convolutional Networks,
which exploit fundamental statistical properties of images, sounds and video data:
the local stationarity and multi-scale compositional structure, that allows express-
ing long range interactions in terms of shorter, localized interactions. However,
there exist other important examples, such as text documents or bioinformatic
data, that may lack some or all of these strong statistical regularities.
In this paper we consider the general question of how to construct deep architec-
tures with small learning complexity on general non-Euclidean domains, which
are typically unknown and need to be estimated from the data. In particular, we
develop an extension of Spectral Networks which incorporates a Graph Estima-
tion procedure, that we test on large-scale classification problems, matching or
improving over Dropout Networks with far less parameters to estimate.
1 Introduction
In recent times, Deep Learning models have proven extremely successful on a wide variety of tasks,
from computer vision and acoustic modeling to natural language processing [9]. At the core of their
success lies an important assumption on the statistical properties of the data, namely the stationarity
and the compositionality through local statistics, which are present in natural images, video, and
speech. These properties are exploited efficiently by ConvNets [8, 7], which are designed to extract
local features that are shared across the signal domain. Thanks to this, they are able to greatly
reduce the number of parameters in the network with respect to generic deep architectures, without
sacrificing the capacity to extract informative statistics from the data. Similarly, Recurrent Neural
Nets (RNNs) trained on temporal data implicitly assume a stationary distribution.
One can think of such data examples as being signals defined on a low-dimensional grid. In this
case stationarity is well defined via the natural translation operator on the grid, locality is defined
via the metric of the grid, and compositionality is obtained from downsampling, or equivalently
thanks to the multi-resolution property of the grid. However, there exist many examples of data that
lack the underlying low-dimensional grid structure. For example, text documents represented as
bags of words can be thought of as signals defined on a graph whose nodes are vocabulary terms and
whose weights represent some similarity measure between terms, such as co-occurence statistics. In
medicine, a patient’s gene expression data can be viewed as a signal defined on the graph imposed
by the regulatory network. In fact, computer vision and audio, which are the main focus of research
efforts in deep learning, only represent a special case of data defined on an extremely simple low-
dimensional graph. Complex graphs arising in other domains might be of higher dimension, and
the statistical properties of data defined on such graphs might not satisfy the stationarity, locality
1
arXiv:1506.05163v1 [cs.LG] 16 Jun 2015
又可乐
- 粉丝: 63
- 资源: 309
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功
评论0