With the development of social network, current research on social network is analyzed by graph theory. According to the topology, studies can determine the social network, and analyze many practical problems, such as social ties, group cooperation, team influence, hot spread node of overlapping community and so on. Now, available research for community mining establishes algorithms widely by analyzing the section topology of neighbor vertexes. However, little consider topology formability or symmetries. For the research, we propose an algorithm, in which the complex graphs are considered as some isolated vertexes through the model of granular graph. This paper proposes an approach to mine outliers based on graph adjacency matrix and combinatorial mathematics, then analyses the symmetries of outliers for overall structure.