Joint domain adaptation aims to learn a high-quality classifier for an unlabeled dataset with the help of auxiliary data. Most methods reduce domain shifts through some carefully designed distance measures. Adversarial learning, which is rarely used for joint domain adaptation, can learn more transferable features while avoiding explicit distance measures. However, it usually suffers from a gradient vanishing problem during the training process. In order to solve the above problems, we propose a novel adversarial joint domain adaptation method, namely Asymmetric Feature mapping based on Least Squares Distance (AFLSD), which consists of asymmetric marginal distribution alignment and conditional distribution alignment. The asymmetric feature mapping, which can get closer features with more flexible parameters, is optimized by the least squares distance to reduce the gradient vanishing problem. The results of classification and other comparative experiments show that AFLSD is superior to the most advanced domain adaptation methods.