Physical stress from workload for speaker exerts limitations on his speech production in physiological system causing speech variability, and thereby reduces speech system performance. The speech under stress presents a marked difference from the speech under neural condition. The distribution for the stress samples in the feature space shows the discontinuity because of the existence of different stress levels and different physiological characteristics for each speaker under the stress condition. In this paper, we use a Gaussian Mixture Models (GMM) framework with the physical features derived from the speech production model for neutral/stress speech classification. Cluster analysis is performed within stress class, and several cluster areas can be found for the classification, which is following Gaussian distribution. Experimental results show that GMM outperforms other classifiers for differentiating neutral speech from stress.