As the elderly population of Taiwan continues to rise, appearing the shortage of care measures and human resource. Emotional sounds can reflect physical and mental states of children, which are very important in child care. In this paper, we use emotional voice recognition technology to assist caregivers to understand the demands of the elderly and children, so that they can give appropriate care in time. Empirical mode decomposition (EMD) is applied to improve the detection and recognition of emotional sounds. Moreover, ensemble deep learning is further applied to alleviate the problem of overfitting. The experiments show that the classification accuracy of the proposed approach is 91.6%, which is much higher than that without EMD. We hope this technology will bring the benefit to the care of elderly and children.