Action recognition and human pose estimation are
closely related but both problems are generally handled
as distinct tasks in the literature. In this work, we pro-
pose a multitask framework for jointly 2D and 3D pose
estimation from still images and human action recogni-
tion from video sequences. We show that a single archi-
tecture can be used to solve the two problems in an effi-
cient way and still achieves state-of-the-art results. Ad-
ditionally, we demonstrate that optimization from end-to-
end leads to significantly higher accuracy than separated
learning. The proposed architecture can be trained with
data from different categories simultaneously in a seam-
lessly way. The reported results on four datasets (MPII,
Human3.6M, Penn Action and NTU) demonstrate the effec-
tiveness of our method on the targeted tasks.
