RNA viruses exist as populations of closely related genomes, characterized by a high diversity of low-frequency variants. As viral genomes from one population disperse to establish new sites of replication, the fate of these low-frequency variants depends to a large extent on the size of the founding population. Focusing on foot-and-mouth disease virus (FMDV) we conjecture that variants are more likely to be transmitted through wide bottlenecks, but more likely to approach fixation in new populations following narrow bottlenecks; therefore, the longer-term rate of accumulation of 'nearly neutral' variants at high frequencies is likely to be inversely related to the bottleneck size. We examine this conjecture in vivo by estimating bottleneck sizes relating 'parent' and 'daughter' populations observed at different scales ranging from within host to between host (within the same herd, and in different herds) using a previously established method. Within hosts, we find bottleneck sizes to range from 5 to 20 viral genomes between populations transmitted from the pharynx to the serum, and from 4 to 54 between serum and lesion populations. Between hosts, we find bottleneck sizes to range from 2 to 39, suggesting inter-host bottlenecks are of a similar size to intra-host bottlenecks. We establish a statistically significant negative relationship between the probability of genomic consensus level change and bottleneck size, and present a simple sampling model that captures this empirical relationship. We also present a novel in vitro experiment to investigate the impact of bottleneck size on the frequency of mutations within FMDV populations, demonstrate that variant frequency in a population increases more rapidly during small population passages, and provide evidence for positive selection during the passage of large populations.