Background: The evolution and predictors of cognitive impairment (CI) in multiple sclerosis (MS) are poorly understood. We aimed to define the temporal dynamics of cognition throughout the disease course and identify clinical and neuroimaging measures that predict CI. (2) Methods: This paper features a longitudinal study with 212 patients who underwent several cognitive examinations at different time points. Dynamics of cognition were assessed using mixed-effects linear spline models. Machine learning techniques were used to identify which baseline demographic, clinical, and neuroimaging measures best predicted CI. (3) Results: In the first 5 years of MS, we detected an increase in the z-scores of global cognition, verbal memory, and information processing speed, which was followed by a decline in global cognition and memory (p < 0.05) between years 5 and 15. From 15 to 30 years of disease onset, cognitive decline continued, affecting global cognition and verbal memory. The baseline measures that best predicted CI were education, disease severity, lesion burden, and hippocampus and anterior cingulate cortex volume. (4) Conclusions: In MS, cognition deteriorates 5 years after disease onset, declining steadily over the next 25 years and more markedly affecting verbal memory. Education, disease severity, lesion burden, and volume of limbic structures predict future CI and may be helpful when identifying at-risk patients.