2D/3D Pose Estimation and Action Recognition using Multitask Dee...

Action recognition and human pose estimation are closely related but both problems are generally handled as distinct tasks in the literature. In this work, we pro- pose a multitask framework for jointly 2D and 3D pose estimation from still images and human action recogni- tion from video sequences. We show that a single archi- tecture can be used to solve the two problems in an effi- cient way and still achieves state-of-the-art results. Ad- ditionally, we demonstrate that optimization from end-to- end leads to significantly higher accuracy than separated learning. The proposed architecture can be trained with data from different categories simultaneously in a seam- lessly way. The reported results on four datasets (MPII, Human3.6M, Penn Action and NTU) demonstrate the effec- tiveness of our method on the targeted tasks.