tation which introduces modality weights for adaptive fusion of different source data.27
The performance of these trackers, however, might be limited by the following two28
aspects. Firstly, the joint sparse constraints on RGB and thermal information make29
modal consistency too strong to achieve effective fusion. For example, the l
2,1
-norm30
used in [11, 12] encourages column-wise sparse of the joint modal matrix, i.e., all ele-31
ments of one column are zeros or non-zeros, while RGB and thermal data are hetero-32
geneous. Secondly, Bayesian filtering algorithm needs sample a large set of candidates33
for effective tracking, which makes optimization to sparse representation models has a34
high computational complexity and is thus time consuming.35
In this paper, we propose a novel and flexible algorithm for effective RGB-T track-36
ing, which achieves state-of-the-art results in terms of both accuracy and efficiency.37
The proposed approach builds on the correlation filter due to its high computational38
efficiency and robust tracking performance [7, 16, 17, 18, 19, 20, 21]. We employ cor-39
relation filter for each modality, and propose a flexible way to achieve collaborative40
fusion of multiple modalities. In particular, we observe that different modality features41
should have similar correlation filter to make them have consistent localization of the42
target object. To this end, the low-rank constraint is utilized to learn filters jointly to43
take the interdependency between RGB and thermal modalities. Benefiting from the44
proposed fusion method, the learned filters could incorporate useful information from45
different source data and thus obtain robust tracking results, as shown in Fig. 2. For46
optimization to the proposed model, we present an efficient ADMM (Alternating Di-47
rection Method of Multipliers) algorithm [22], which makes our tracker fast enough48
for satisfying real-time applications (e.g., 227 frames per second in the experiments).49
The contributions of the proposed approach are as follows. 1) We propose to use50
low-rank constraint to perform cross-modal correlation filters collaboration which can51
take the interdependency between RGB and thermal modalities in the learning process.52
It is generic, and could incorporate more modalities (e.g., near infrared spectrum). 2)53
We design en efficient solver based on the ADMM algorithm to the proposed cross-54
modal correlation filter model, in which the time consuming step can be converted into55
Fourier domain to improve efficiency significantly. 3) Extensive experiments on large-56
scale benchmark datasets are conducted, and our tracker achieves the state-of-the-art57
3
