Context_Encoders_Feature_Learning_by_Inpainting.pdf

基于上下文的编码器：图像修复的特征学习 We present an unsupervised visual feature learning algo- rithm driven by context-based pixel prediction. By analogy with auto-encoders, we propose Context Encoders – a con- volutional neural network trained to generate the contents of an arbitrary image region conditioned on its surround- ings. In order to succeed at this task, context encoders need to both understand the content of the entire image, as well as produce a plausible hypothesis for the missing part(s). When training context encoders, we have experi- mented with both a standard pixel-wise reconstruction loss, as well as a reconstruction plus an adversarial loss. The latter produces much sharper results because it can better handle multiple modes in the output. We found that a con- text encoder learns a representation that captures not just appearance but also the semantics of visual structures. We quantitatively demonstrate the effectiveness of our learned features for CNN pre-training on classification, detection, and segmentation tasks. Furthermore, context encoders can be used for semantic inpainting tasks, either stand-alone or as initialization for non-parametric methods.