Dietary change is often invoked as a major selective force in recent human evolution, with increased copy number of the salivary amylase gene (AMY1) widely cited as an adaptation to starch-rich agricultural diets. However, most evidence for this model comes from limited geographical sampling and analyses that do not fully account for shared ancestry. Here we combine newly generated droplet digital PCR estimates from 390 individuals representing 30 Sub-Saharan African populations with published copy number data from up to 1,307 individuals worldwide and re-evaluate AMY1 evolution using ancestry-aware and phylogenetically informed models. Across Africa, AMY1 copy number shows no consistent association with agriculture once population structure is accounted for. At a global scale, differences between agriculturalists and non-agriculturalists are substantially smaller than previously reported and are largely explained by shared ancestry rather than diet. Phylogenetic analyses further reveal baseline differences in AMY1 copy number between Sub-Saharan and non-Sub-Saharan populations, pointing to deep demographic processes shaping present-day variation. These results challenge the long-standing "agriculture hypothesis" and identify demographic history, rather than subsistence strategy, as the primary driver of AMY1 CN evolution worldwide.