Face recognition systems have grown rapidly in a variety of applications recently, such as surveillance, access control, mobile payments, and forensic investigations. However, face recognition systems are highly likely to be deceived because imposters attempt to gain unauthorized access to the system by spoofing biometric data. In this paper, we proposed a multimodal presentation attack detection (PAD) method based on a challenge-response scenario. When the user speaks a word as required, the mouth movement is detected and the recognized speech is referenced to determine if the user is a real person or not. Two weighted score level fusion rules are adopted in the machine learning algorithm for training and testing. Experimental results show the proposed method is very effective in resisting photo-attack and video-attack targeting to face recognition systems.