Homeless people and people who have only recently escaped homelessness are at risk of being malnourished due to several factors. Personalized cooking recipe recommendation systems are one technology-based approach to provide information to such populations that can assist them with making more informed choices about their food. Current recipe recommendation systems do not provide any guidance on how to cook for a person who has limited experience with cooking. This work describes a system that acts as a smart cooking assistant. The main idea is for the system to observe the user perform the steps of cooking based on a recipe, and then provide automatic reminders on when to move to the next step. The system consists of both hardware and machine learning-based software components. The hardware consists of a camera, infrared thermal camera, and temperature sensor. These are integrated around a Raspberry Pi mini-computer. This suite of sensors is mounted over a stovetop and constantly monitors the cooking area, specifically the area where a cooking pot is over the stovetop. Convolutional Neural Network-based image processing algorithms are used to analyze the sequence of images from the cameras to identify the stage of cooking that the user is performing.