There are serious soil sickness in ginseng cultivation. Crop rotation is an effective agricultural management to improve soil sustainability and reduce soil sickness. To explore an appropriate ginseng rotation system, Medicago sativa (alfalfa) and Perilla frutescens (perilla) were planted on ginseng cultivation soil for 1 year to evaluate the improvement effect of both. Through chemical analysis and high-throughput sequencing technology, we found that after alfalfa and perilla cultivation for one-year, various nutrients and enzyme activities in ginseng cultivation soil were significantly improved. In addition, perilla significantly increased the diversity and richness of soil fungal communities. Cultivation of alfalfa and perilla significantly changed the composition of soil bacterial and fungal communities and significantly reduced the abundance of the potentially pathogenic fungi Ilyonectria. Further pot experiments also showed that the improved soil could significantly increase root activity of ginseng plant after two plants were planted. It should be noted that, unlike alfalfa, perilla decreased soil electrical conductivity, increased soil organic matter, soil urease, and may significantly improve the diversity and richness of soil fungal community. Moreover, in the pot experiment, the root fresh weight of ginseng cultured in perilla treated soil increased significantly. This study highlights that perilla may have better soil improvement effect than alfalfa and it has the potential to be used in the soil improvement of ginseng cultivation.