Giving robots the ability to perceive our world as we do is still an unsolved problem. Without being able to perceive (and recognize) its ambience, robots are unable to coexist with humans in a domestic environment. Even though recognition by low-level image processing is still widely studied and necessary, we believe that robots should also understand our world semantically. In this paper, we design semantic and ontology model to organize knowledge representation. Relationships in our model are semantic network links and any entity which can be identified using recognition methods. Therefore, semantic understanding in this research refers to recognized entities that can be linked together using semantic network. We evaluate semantic coherence using dynamic Bayesian networks complying with our designed ontology database. Using our semantic relationship method, probability of the SURF object recognition can be increased up to 64.4% compared to the ordinary SURF object recognition without semantic relationship.