Summary: Background: Alzheimer's disease (AD) is a heterogeneously progressive neurodegeneration disorder with varied rates of deterioration, either between subjects or within different stages of a certain subject. Estimating the course of AD at early stages has treatment implications. We aimed to analyze disease progression to identify distinct patterns in AD trajectory. Methods: We proposed a deep learning model to identify underlying patterns in the trajectory from cognitively normal (CN) to a state of mild cognitive impairment (MCI) to AD dementia, by jointly predicting time-to-conversion and clustering out distinct subgroups characterized by comprehensive features as well as varied progression rates. We designed and validated our model on the ADNI dataset (1370 participants). Prediction of time-to-conversion in AD trajectory was used to validate the expression of the identified patterns. Causality between patterns and time-to-conversion was further inferred using Mendelian randomization (MR) analysis. External validation was performed on the AIBL dataset (233 participants). Findings: The proposed model clustered out patterns characterized by significantly different biomarkers and varied progression rates. The discovered patterns also showed a strong prediction ability, as indicated by hazard ratio (CN→MCI, HR = 3.51, p