COVID-19 has led to unprecedented challenges and requires local and global efforts for its mitigation. Poor and marginalized populations are more vulnerable to the health, social and economic effects of the pandemic. The objective of this study was to know about the knowledge, attitude and practices towards COVID-19 among poor and marginalized communities in central India and the factors associated with them so that effective risk communication messages can be designed and community engagement needs and strategies can be identified. A cross-sectional survey was conducted using an Interactive Voice Response System as part of the NISHTHA-Swasthya Vani intervention, which is a platform for dissemination of key messages related to COVID-19, social welfare schemes, national health programs and other important information. A total of 1673 respondents participated in the survey. The mean knowledge, attitude and practice scores of the respondents was 4.06 (SD = 1.67) out of 8, 2.46 (SD = 1.18) out of 4 and 3.65 (SD = 0.73) out of 4 respectively. More than 50% respondents exhibited stigma towards recovered COVID-19 patients(n = 347) and towards health workers(n = 384) catering to COVID-19 patients. The factors associated with higher KAP scores were education, occupation, age and primary source of information on COVID-19. There was a positive correlation between knowledge and attitude (co-efficient: 0.32) and a negative correlation between knowledge and stigma (co-efficient: -0.28). The knowledge, and attitude scores related to COVID-19 were low among the poor and marginalized communities, while the prevalence of stigma was high. Therefore, there is a need for effective risk communication for these communities through alternate channels.