Games have been designed to foster children's computational thinking (CT) skills, which is widely recognized as a key ability that should be mastered in this computing-driven era. Because learning coding skills is cognitively difficult and uninvolving for novices and it is recommended to learn CT concepts first, many educational games have been developed to teach CT in a non-coding environment. However, despite the breadth of such games and the demand for it, little is known about how to design such games or why such games are designed in a particular way. To address this gap, this paper presents a design framework that describes the building blocks for developing (non-coding) gameplay elements in games teaching CT. The framework focuses specifically on computational problem-solving because this is where games can specifically aid with. We illustrate the design framework through LittleWorld, a mobile game that teaches CT concepts through insects.