Knowledge of the characteristics of natural climate variability is vital when assessing the range of plausible future climate trajectories in the next decades to centuries. The reliable detection of climate fluctuations on multidecadal to centennial timescales depends on proxy reconstructions and model simulations, as the instrumental record extends back only a few decades in most parts of the world. Systematic comparisons between model-simulated and proxy-based inferences of natural variability, however, often seem contradictory. Locally, simulated temperature variability is consistently smaller on multidecadal and longer timescales than is indicated by proxy-based reconstructions, implying that climate models or proxy interpretations might have deficiencies. In contrast, at global scales, studies found agreement between simulated and proxy reconstructed temperature variations. Here we review the evidence regarding the scale of natural temperature variability during recent millennia. We identify systematic reconstruction deficiencies that may contribute to differing local and global model–proxy agreement but conclude that they are probably insufficient to resolve such discrepancies. Instead, we argue that regional climate variations persisted for longer timescales than climate models simulating past climate states are able to reproduce. This would imply an underestimation of the regional variability on multidecadal and longer timescales and would bias climate projections and attribution studies. Thus, efforts are needed to improve the simulation of natural variability in climate models accompanied by further refining proxy-based inferences of variability.