This paper presents a gender-specific stress model to analyze the psychological stress factors on intentional facial expressions. We have focused on the relationship between facial expression intensity and Stress Response Scale (SRS-18). In this paper, we extract three facial expressions (i.e., happiness, anger, and sadness) from the basic six facial expressions defined by Ekman, and then represent a graphical model of the relationship between these three facial expressions and the psychological stress factors (“i.e., depression and anxiety”, “displeasure and anger”, and “lassitude”). In the experiment, we created an original facial expression dataset consisting of three facial expressions and a psychological stress dataset by SRS-18 obtained from 10 subjects during 7–20 weeks at one week interval. As the results of probabilistic reasoning based on the observed values of each facial expression, such trends are obtained as follows. BNs shows trends of different stress factor between men and women in relations of expression levels and psychological stress. Stress models appeared on happiness faces of “lassitude” factor in men, the anger faces of “displeasure and anger” were affected with stress factors in women.