To study the role of zero fly test in naval gun weapon system test, the simulation model of the direct-hit CIWS was built on the base of zero fly error data collected in tracking the real target. The naval gun weapon system's fire effectiveness was solved by simulation in approach to real conditions, the corresponding simulation solution of hit probability and kill probability was also given. Since the simulation result was close to the live fire result, showing that this simulation method can not only be used to predict weapon system's accuracy, but also to verify the confidence level of firing test result.