Effectively fusing the technologies of sense intelligence, emotion intelligence and communication intelligence, we design a new model of smart home. The system can implement unsupervised and continuous smart monitor for empty nest elder in terms of non obtrusive methods in home environment. It is a multi-agent system based on middleware, and implements the mechanism that both low level sensors and high level perceptual components are pluggable to the middleware architecture. Being convenient for knowledge representation and reasoning during the course of context-aware, the approach of semantic layering and semantic-abstracting step by step is utilized to design a set of new multimedia ontologies for bridging perceivable media world and high-level conceptual world. Hierarchical hidden Markov model is used to identify abnormal behaviors of occupants.