Coronary artery disease (CAD) is the leading cause of death for both men and women worldwide. In cardiac perfusion imaging, images are acquired in two separate phases (rest and stress). For stress scanning, the patient is asked to do some physical activity in order to increase the heart rate to its peak. The patient’s heart rate and blood pressure are monitored during this phase of the test. The exercise stress phase test has several drawbacks such as extra radiation exposure, sudden heart attack, chest pain during the exercise increased anxiety, or even irregular heartbeat Moreover it is a time-consuming and costly process. In this study, a clinical cardiac PET dataset includes 20 gated cardiac Nitrogen-13 ammonia ( 13 N-NH3) rest and stress images were gathered. After data pre-processing a residual neural network was trained two times on 80% of the data and tested on the other 20%. In the first strategy, a model was trained to predict the rest images from stress images and in the second strategy, the inverse of the previous strategy was performed. The quantitative analysis was performed by calculation of structural similarity index metrics (SSIM), root means squared error (RMSE), and peak signal-to-noise ratio (PSNR). The visual inception shows the model was able to extract the pattern of rest and stress images approximately. The SSIM and PSNR for rest-to-stress and stress-to-rest models were 0.79, 0.85 and 22.69 dB, 22.65 dB, respectively. We developed two separated deep learning models for synthesizing the stress phase from the rest phase and vice versa. Our study can decrease the possible risks of exercise needed for stress imaging, decrease the patient’s absorbed dose, reduce the scanning time, and improve the patient’s experience during scanning.