Carotid ultrasound measurement of total plaque area (TPA) provides a method for quantifying carotid plaque burden and monitoring changes in carotid atherosclerosis in response to medical treatment. Plaque boundary segmentation is required to generate the TPA measurement; however, training of observers and manual delineation are time consuming. Thus, our objective was to develop an automated plaque segmentation method to generate TPA from longitudinal carotid ultrasound images. In this study, a deep learning-based method, modified U-Net, was used to train the segmentation model and generate TPA measurement. A total of 510 plaques from 144 patients were used in our study, where the Monte Carlo cross-validation was used by randomly splitting the data set into 2/3 and 1/3 for training and testing. Two observers were trained to manually delineate the 510 plaques separately, which were used as the ground-truth references. Two U-Net models (M1 and M2) were trained using the two different ground-truth data sets from the two observers to evaluate the accuracy, variability and sensitivity on the ground-truth data sets used for training our method. The results of the algorithm segmentations of the two models yielded strong agreement with the two manual segmentations with the Pearson correlation coefficient r = 0.989 (p0.0001) and r = 0.987 (p0.0001). Comparison of the U-Net and manual segmentations resulted in mean TPA differences of 0.05 ± 7.13 mm