X. Zhou et al.
experiments results on several popular multi-view datasets show its feasibility and effective-
ness.
Keywords Canonical correlation analysis · Semi-paired learning · Two-view learning ·
Neighborhood relationship · Neighborhood correlation
1 Introduction
High-dimensional co-occurring data associated with an object frequently and abundantly
emerge in the real world. For example, an Internet web page as an object can be repre-
sented as (co-occurring) page text and links to the page, and a human can be represented as
co-occurring visual and audio contents. A lot of works have been done for analyzing this
kind of data [1–7]. Among these works, canonical correlation analysis (CCA) is one of the
most widely adopted methods [8–12].
CCA is a classical but useful multivariate statistical analysis method [13]. It aims to find
maximally correlated projections between two sets of variables, which can be considered
as two views (views x and y) or representations of the same set of objects. However CCA
requires that such two views be fully-paired, i.e., each sample in view x should have a corre-
spondence in view y, and vice versa. Conversely, we are often faced such a scenario where
most samples in view x have no correspondences in view y, and vice versa, thus forming
the semi-paired scenario called here. For such a scenario, CCA is generally prone to overfit-
ting and thus performs poorly, since its definition itself makes it only suitful for the paired
scenario, so its applications are limited in the real world. Actually, abundant unpaired sam-
ples (i.e. x-andy-only samples) often contain much useful information which will benefit
the learning task, just as the unlabeled samples benefit semi-supervised leaning [14,15]by
exploiting the intrinsic data structure under clustering assumption or manifold assumption.
Recently, several works have concerned such new scenario [16–18]. Blaschko et al. [16]pro-
posed a semi-supervised Laplacian regularization of kernel CCA (SemiLRKCCA), which
utilizes intrinsic geometry structure of each view to regularize kernel CCA (KCCA) [19].
As a result, SemiLRKCCA can find a set of meaningful directions which not only make the
two view’s paired samples highly correlated but also capture each view’s manifold struc-
ture. SemiCCA [17] utilizes global structure of each view’s whole training samples (paired
and unpaired samples together) to regularize CCA in order to bridge CCA and principal
component analysis (PCA) [20,21] seamlessly. Both SemiLRKCCA and SemiCCA can take
sufficient advantage of unpaired samples in addition to paired samples, and consequently
achieve better results than CCA just based on the paired samples. It is necessary to mention
that the actual meaning of “semi-” in SemiLRKCCA and SemiCCA is “semi-paired” rather
than “semi-supervised” in popular semi-supervised learning literature [14,15]. Compared
with SemiLRKCCA and SemiCCA, more recent work termed as semi-paired and semi-
supervised generalized correlation analysis (S
2
GCA) [18] make further research for dealing
with semi-paired and semi-supervised scenario. S
2
GCA utilizes within-view structural infor-
mation and within-view discriminant information jointly, to preserve the individual view’s
structure of unlabeled data and separate labeled data in different classes from each other
simultaneously. Without semi-supervised information, S
2
GCA is similar to SemiLRKCCA
and SemiCCA.
In SemiLRKCCA, SemiCCA and S
2
GCA, unpaired samples are just used in the way of
single-view leaning to capture individual views’ structure information for regularizing KCCA
or CCA. Consequently, CCA and its variants (SemiLRKCCA, SemiCCA and S
2
GCA) only
123
