【免费】AdaptiveFeatureProjectionWithDistributionAlignmentforDeep资源-CSDN文库

需积分: 0 23 浏览量 2023-11-30 21:34:04 上传评论 1 收藏 33.34MB PDF 举报

资源推荐

资源详情

资源评论

1354 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 32, 2023

Adaptive Feature Projection With Distribution

Alignment for Deep Incomplete

Multi-View Clustering

Jie Xu , Chao Li, Liang Peng , Yazhou Ren , Member, IEEE, Xiaoshuang Shi ,

Heng Tao Shen , Fellow, IEEE, and Xiaofeng Zhu , Senior Member, IEEE

Abstract— Incomplete multi-view clustering (IMVC) analysis,

where some views of multi-view data usually have missing data,

has attracted increasing attention. However, existing IMVC meth-

ods still have two issues: 1) they pay much attention to imputing

or recovering the missing data, without considering the fact that

the imputed values might be inaccurate due to the unknown

label information, 2) the common features of multiple views are

always learned from the complete data, while ignoring the feature

distribution discrepancy between the complete and incomplete

data. To address these issues, we propose an imputation-free

deep IMVC method and consider distribution alignment in

feature learning. Concretely, the proposed method learns the

features for each view by autoencoders and utilizes an adaptive

feature projection to avoid the imputation for missing data.

All available data are projected into a common feature space,

where the common cluster information is explored by maximizing

mutual information and the distribution alignment is achieved

by minimizing mean discrepancy. Additionally, we design a new

mean discrepancy loss for incomplete multi-view learning and

make it applicable in mini-batch optimization. Extensive experi-

ments demonstrate that our method achieves the comparable or

superior performance compared with state-of-the-art methods.

Index Terms— Incomplete multi-view clustering, adaptive fea-

ture projection, distribution alignment, deep feature learning.

Manuscript received 25 April 2022; revised 7 November 2022 and 4 January

2023; accepted 27 January 2023. Date of publication 13 February 2023;

date of current version 22 February 2023. This work was supported in part

by the National Key Research and Development Program of China under

Grant 2022YFA1004100 and in part by the Medico-Engineering Cooperation

Funds from the University of Electronic Science and Technology of China

under Grant ZYGX2022YGRH009 and Grant ZYGX2022YGRH014. The

associate editor coordinating the review of this manuscript and approv-

ing it for publication was Dr. Claudio R. Jung. (Corresponding authors:

Xiaofeng Zhu; Heng Tao Shen.)

Jie Xu, Chao Li, Liang Peng, Xiaoshuang Shi, and Heng Tao Shen are with

the School of Computer Science and Engineering, University of Electronic

Science and Technology of China, Chengdu 611731, China (e-mail:

jiexuwork@outlook.com; lichao.cfm@gmail.com; larrypengliang@

gmail.com; xsshi2013@gmail.com; shenhengtao@hotmail.com).

Yazhou Ren is with the School of Computer Science and Engineering,

University of Electronic Science and Technology of China, Chengdu 611731,

China, and also with the Shenzhen Institute for Advanced Study, University

of Electronic Science and Technology of China, Shenzhen 518000, China

(e-mail: yazhou.ren@uestc.edu.cn).

Xiaofeng Zhu is with the School of Computer Science and Engineering,

University of Electronic Science and Technology of China, Chengdu 611731,

China, also with the Shenzhen Institute for Advanced Study, University of

Electronic Science and Technology of China, Shenzhen 518000, China, and

also with the Guangxi Academy of Sciences, Nanning 530007, China (e-mail:

seanzhuxf@gmail.com).

This article has supplementary downloadable material available at

https://doi.org/10.1109/TIP.2023.3243521, provided by the authors.

Digital Object Identiﬁer 10.1109/TIP.2023.3243521

I. INTRODUCTION

ULTI-VIEW data widely exist in real-world applica-

tions [1], [2], [3], [4], [5], [6], [7], where each sample

consists of multiple views/modalities extracted by different

sensors or different preprocessing methods [8], [9], [10],

[11], [12], [13], [14]. As an important unsupervised learning

approach, multi-view clustering analysis aims to explore the

common cluster patterns among multi-view data and has been

widely investigated in machine learning and image processing

communities [15], [16], [17], [18], [19], [20], [21].

In real-world scenarios, however, the data collection is

easily incomplete for multiple views, where the data are

partially missing in some views. For example, video data

can simultaneously contain multiple views such as images,

captions, and sounds, but some of videos lack captions or

sounds. Unfortunately, existing multi-view clustering methods

are inapplicable to the incomplete multi-view data, thereby

raising the currently attention-getting incomplete multi-view

clustering (IMVC) problem [22]. The goal of IMVC is to

discover the common cluster patterns hidden in incomplete

multi-view data, and previous IMVC methods can be roughly

divided into two groups, ie, traditional and deep methods.

Traditional IMVC methods can be further classiﬁed into

four categories: kernel based IMVC methods [23], [24], non-

negative matrix factorization based IMVC methods [22], [25],

[26], tensor based IMVC methods [27], [28], [29], and graph

based IMVC methods [30], [31], [32]. They usually tend to

impute/recover/infer values for the missing data of incomplete

multi-view data sets, and then explore cluster information.

However, traditional IMVC methods have limited capability

to learn feature representations, and meanwhile, some of them

consume high computation costs (e.g., in inverse operations of

matrices) [33].

In recent years, deep IMVC methods (e.g., [34], [35]) have

achieved impressive progresses by combining clustering with

the representation learning of deep models, e.g., deep autoen-

coders. Generally, deep methods leverage the generalization

capability of deep models to achieve the imputation for miss-

ing data. For instance, Lin et al. [36] reconstruct the missing

data with additional prediction networks in a contrastive learn-

ing framework. Some works combine discriminator networks

with autoencoders and employ adversarial training to generate

possible values for the missing data [33], [37].

See https://www.ieee.org/publications/rights/index.html for more information.

Authorized licensed use limited to: Tsinghua University. Downloaded on November 25,2023 at 01:00:29 UTC from IEEE Xplore. Restrictions apply.

XU et al.: ADAPTIVE FEATURE PROJECTION WITH DISTRIBUTION ALIGNMENT FOR DEEP IMVC 1355

Fig. 1. The framework of APADC. (a) APADC learns the view-speciﬁc features [Z

; Z

] by optimizing the reconstruction loss L

R EC

of individual

autoencoder and avoids the imputation for missing data via the adaptive feature projection F . (b) Based on an indicator matrix A and a weight matrix W,

F obtains the common features Z for clustering by mapping {[Z

; Z

]}

v=1

into a common feature space, where (c) the mutual information loss L

M M I

is optimized to explore the common cluster information contained in {Z

}

v=1

, meanwhile, the mean discrepancy loss L

M M D

is designed to align the

distributions between Z and {[Z

; Z

]}

v=1

However, previous IMVC methods still have two issues as

follows. (1) Most of the existing IMVC methods, including

the traditional and deep methods, incorporate the imputation

process for missing data to complete the incomplete multi-

view data [35]. The imputation of missing data is time-

consuming, moreover, its precision depends on the estimation

of the distribution of available data. As incomplete multi-view

data usually have biased data distribution, the imputation of

missing data might produce noise even incorrect information,

thereby limiting the effectiveness of the imputation models.

(2) Many IMVC methods handle the incomplete multi-view

data in a two-stage process, i.e., ﬁrst exploring the consistency

among multiple views from the complete part of data, and then

extending the learned consistency to the incomplete part of

data. For example, the methods [33], [36] learn cluster infor-

mation of multiple views on the complete multi-view data, and

then produce imputation values to ﬁll the incomplete multi-

view data. However, feature learning performed on partial data

might cause the distribution discrepancy between the features

of complete data and that of incomplete data, resulting in the

degradation of model generalization capability.

To address the aforementioned issues, we propose APADC:

Adaptive feature Projection with distribution Alignment for

Deep incomplete multi-view Clustering. The proposed method

is an imputation-free framework shown in Fig. 1(a), which

consists of multiple deep autoencoders (to learn the view-

speciﬁc features for individual views) and an adaptive fea-

ture projection (to obtain the common features shared by

all views). Speciﬁcally, to avoid the imputation of missing

data, we present the adaptive feature projection as shown in

Fig. 1(b), which produces the common features by projecting

the features of all views’ available data into a common feature

space. In particular, an indicator matrix and a weight matrix

are embedded in the adaptive feature projection, to make it

adaptive to varying amounts of missing data and provide dif-

ferent weights to the views according to their feature qualities.

For the feature learning in Fig. 1(c), we maximize the mutual

information between the common features and the view-

speciﬁc features of complete multi-view data to explore their

common cluster information. Simultaneously, we minimize the

mean discrepancy to align the feature distributions between

the complete and incomplete data. In this way, the common

features obey the same distribution, which is conducive to

discovering the common cluster patterns across multiple views.

Compared to previous IMVC methods, the main contribu-

tions of our method are summarized as follows.

• We propose an imputation-free deep IMVC method via an

adaptive feature projection, which avoids the imputation

process for missing data so as to prevent noise generation

when handling incomplete multi-view data.

• Our method explores the common cluster information

hidden in multi-view data as well as aligns the distribu-

tions between the complete and incomplete data, which

is usually neglected in existing IMVC methods.

• We design a novel mean discrepancy loss for incomplete

multi-view learning and make it applicable in mini-batch

optimization. Extensive experiments on ﬁve popular data

sets indicate that our method obtains comparable or supe-

rior performance over recent state-of-the-art methods.

II. RELATED WORK

A. Deep Multi-View Clustering

Inspired by the success of deep learning, people proposed a

number of deep models recently for multi-view clustering and

obtained impressive achievement [38], [39]. One of the most

popular network architectures in deep multi-view clustering is

autoencoder, which learns features or representations by opti-

mizing the reconstruction loss between the input and the out-

put [40], [41]. Deep autoencoders usually are combined with

existing clustering objectives to perform multi-view clustering.

For instance, Abavisani et al. [38] proposed a multi-view

Authorized licensed use limited to: Tsinghua University. Downloaded on November 25,2023 at 01:00:29 UTC from IEEE Xplore. Restrictions apply.

1356 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 32, 2023

subspace clustering method based on the latent representations

learned by autoencoders. Li et al. [42] incorporated the

adversarial training [43] with autoencoders in a uniﬁed multi-

view clustering framework. Fan et al. [44] introduced graph

autoencoders with graph constraints for multi-view clustering.

Yin et al. [45] proposed variational autoencoder based multi-

view clustering, where the shared features were learned with a

mixture of Gaussian distributions. Xu et al. [46] pointed out

to learn disentangled multi-view representations for clustering

with common and peculiar variables in variational autoen-

coders. Different from using the reconstruction objectives of

autoencoders, some works (e.g., [47], [48], [49]) proposed to

penalize the representation space with regularization constrains

for deep multi-view clustering. For example, Zhou et al. [47]

utilized encoder networks to extract informative features and

leveraged Gaussian kernel matrices to avoid the feature degen-

eration.

However, the real-word multi-view data always contain

missing data in some views, resulting in the inapplicability

of existing multi-view clustering methods. Therefore, deep

incomplete multi-view clustering is an important topic which

has attracted researchers’ attention in recent years.

B. Incomplete Multi-View Clustering

Incomplete multi-view clustering (IMVC) is also called par-

tial multi-view clustering in the literature. Traditional IMVC

methods utilize classic machine learning techniques such as

non-negative matrix factorization, kernel trick, graph learning,

and tensor techniques. Li et al. [22] proposed the non-negative

matrix factorization based method to handle incomplete multi-

view data. Hu et al. [26] incorporated weighted and regular-

ized matrix factorization in an online IMVC framework. The

recent matrix factorization based method [50] employed the

cosine similarity to preserve the manifold structures. Matrix

factorization based IMVC usually recovers the non-negative

matrix for the missing data with the available data. Similarly,

kernel based IMVC usually imputes the kernel matrix of

incomplete multi-view data by utilizing that of complete

multi-view data. For example, Guo et al. [23] presented a

kernel similarity based method with an anchor strategy for

partial multi-view clustering. Liu et al. [24] proposed multi-

ple kernel IMVC method, which completed each incomplete

base matrix of incomplete views with the learned consensus

matrix. The graph based IMVC is able to leverage graph

structure information to improve the recognition ability for

cluster patterns. For instance, the literature [31] established

graph regularization to achieve the consistency between the

available data and the imputed values for missing data. The

recent graph based method [51] considered the instance-to-

anchor and instance-to-instance similarities for spectral clus-

tering. Fang et al. [52] leveraged the biological evolution

theory to handle the unbalanced incompleteness in IMVC.

Recent works of tensor based IMVC (e.g., [27], [28], [29])

usually introduce low-rank tensor constraints to characterize

the high-order correlation and the inner structure among

multiple views. In recent years, deep learning based IMVC

has been attracting increasing attention. One of the natural

motivations is that the generative adversarial net (GAN [43])

can be applied to generate data for incomplete multi-view

data [33], [37]. Besides, Wen et al. [35] proposed a cognitive

deep incomplete multi-view clustering network, where the

nearest neighbor graph was constructed and the missing data

was ﬁlled by average values. Wei et al. [34] utilized shared

subspace representations to reconstruct the missing data via

a decoder of individual view. Recent work [36] stacked dual

prediction networks on autoencoders to perform data recovery

for incomplete data. Xu et al. [53] proposed to mine the non-

linear cluster complementarity among the incomplete multi-

view data. Tang et al. [54] proposed to dynamically impute

missing views with the learned semantic neighbors.

Most of the traditional and deep IMVC methods han-

dle incomplete multi-view data with imputation/recovery/

inference strategies. However, the inaccurate imputation values

for missing data will negatively affect the performance. This

issue is likely to occur when the number of missing data

is large. Additionally, previous IMVC methods usually learn

the common representations from the complete multi-view

data and generalize them to incomplete multi-view data. This

process might cause the distribution discrepancy between the

complete data and incomplete data. In this paper, we propose

an imputation-free deep IMVC method by considering dis-

tribution alignment in feature learning to address the above

issues.

III. METHOD

Notations: In this paper, {X

∈ R

N ×D

}

v=1

represents a

multi-view data set with V views, where D

is the dimen-

sionality of samples in the v-th view and N is the number

of samples. Moreover, we employ an indicator matrix A ∈

{0, 1}

N ×V

, where a

∈ A, a

= 0 denotes that the data of the

i-th sample in the v-th view is missing, and a

= 1 represents

that data is not missing. Denoting the complete data of all

views as {X

}

v=1

and the incomplete data of individual view

as X

, respectively, for each x

, if there exists

v=1

= V ,

then x

∈ X

; otherwise, x

∈ X

. Therefore, [X

; X

] ∈

×D

and the missing data result in N

≤ N . Table I lists

the deﬁned notations and descriptions.

A. Motivation and Framework

Deep autoencoder has been widely applied in IMVC meth-

ods due to its ability of learning clustering-friendly fea-

tures [33], [35], [36]. Concretely, these methods optimize the

reconstruction loss L

R EC

of all views by

R EC

v=1

R EC

v=1



− D

))



, (1)

where E

and D

denote the encoder and decoder net-

works of the v-th view, respectively. The encoder network

converts the raw data [X

; X

] into the view-speciﬁc features

; Z

] ∈ R

×L

to learn underlying characteristics, i.e.,

; Z

] = E

([X

; X

]). (2)

Authorized licensed use limited to: Tsinghua University. Downloaded on November 25,2023 at 01:00:29 UTC from IEEE Xplore. Restrictions apply.

剩余12页未读，继续阅读

评论收藏

内容反馈

麻辣小凉皮

粉丝: 100
资源: 4

Adaptive Feature Projection With Distribution Alignment for Deep

Rebuild_Strong-Weak-Distribution-Alignment-for-Adaptive-Object-Detection:这是纸的个人重建

Deep Spatial-Temporal Feature Fusion From Adaptive Dynamic Functional Connectivi

Adaptive Filtering Primer with Matlab

Robust fragments-based tracking with adaptive feature selection

deep learning(Adaptive Computation and Machine Learning series)

Adaptive fringe-pattern projection

Robust and Adaptive Control With Aerospace Applications

Improvements of some self-adaptive projection methods for co-coercive variational inequalities

Adaptive Deconvolutional Networks for Mid and High Level Feature Learning

Deep Learning (Adaptive Computation and Machine Learning series) 最新中文版

Adaptive power allocation with quality-of-service guarantee

An_Adaptive_Distribution-Free_Algorithm_for_the_Ne.pdf

Adaptive Support-Weight Approach for Correspondence Search.pdf

Adaptive Filtering Primer with MATLAB

Algorithm and Communication Performance of an Adaptive Protection System for Distribution Line

Engineering Adaptive Software Systems

A Self-Adaptive Deep Learning-Based System for Anomaly Detection

Adaptive Multi-Task Transfer 文献总结翻译

Hotchips2022 AMD 400G Adaptive SmartNIC SoC - Final for Distribution.pdf

YOLOv8-deepsort 实现智能车辆目标检测+车辆跟踪+车辆计数

YOLOv8网络结构图，自制visio文件，yolov8.vsds，需要的自取，在原有的基础上直接改就行了

yolov8(2023年8月版本),已经下好yolov8s.pt和yolov8n.pt

Transformer模型实现长期预测并可视化结果（附代码+数据集+原理介绍）

请移步至：https://blog.csdn.net/m0-48932261/article/details/139472005

社交平台上经济类话题的文章热度信息，数据是真实的，但不是真实日期

行人跌倒数据集（VOC格式）

Unet眼底血管图像分割数据集+代码+模型+系统界面+教学视频.zip

YOLOV5 + 双目相机实现三维测距（新版本）

全新的SOTA模型YOLOv9

基于YOLOv8-Pose的姿态识别项目，带数据集可直接跑通的源码

最新资源