With the vigorous development of modern science technologies and the rising cost of human resources, many modern factories and stores have introduced various robotic arms to replace humans to accomplish tedious work. Particularly in an unmanned store, the time used for ordering, making, and delivering food can be reduced as well. Accordingly, this work develops and integrates three subtasks which are the Application (APP) software for ordering, Raspberry Pi for computation, and robotic arm for delivering the ordered items to fulfill the unmanned shop. The APP is implemented by JAVA to interact with a customer via a smart phone. The order message is then sent to the Raspberry Pi via Bluetooth. The camera on the robotic arm is to take the picture of donuts, which is transmitted to the Raspberry Pi via USB. The Gaussian mixture model is adopted to obtain the adaptive thresholds to segment the foreground and background of the captured picture. The top donut(s) in the foreground are extracted by the GraphCut scheme with the automatic line marking, and determined by the shape characteristics. The extracted top donut(s) are then recognized by the modified MobileNetV2 of the Convolutional Neural Network (CNN) to identify their flavors. Once the identified donut matches the flavor selected by a customer, the distance between the robotic arm and the central point of the selected donut is derived. Additionally, the corresponding angles of six axes of the robotic arm are calculated as well. Based on the calculated angles, the robotic arm starts the movement, grabs the donut, and put it on the tray. The experimental results reveal that our robotic arm can effectively reach the target at the average accuracy rate of 95% for identifying nine donut flavors by using the proposed CNN. Through the integration of these three subtasks, we have successfully created the unmanned shop service to facilitate the development of new business models.
