Litter size is an important reproductive trait as it makes a major contribution to fitness. Generally, traits closely related to fitness show low heritability perhaps because of the corrosive effects of directional natural selection on the additive genetic variance. Nonetheless, low heritability does not imply, necessarily, a complete absence of genetic variation because genetic interactions (epistasis and dominance) contribute to variation in traits displaying strong heterosis in crosses, such as litter size. In our study, we investigated the genetic architecture of litter size in 166 females from an F2 intercross of the SM/J and LG/J inbred mouse strains. Litter size had a low heritability (h2 = 12%) and a low repeatability (r = 33%). Using interval‐mapping methods, we located two quantitative trait loci (QTL) affecting litter size at locations D7Mit21 + 0 cM and D12Mit6 + 8 cM, on chromosomes 7 and 12 respectively. These QTL accounted for 12.6% of the variance in litter size. In a two‐way genome‐wide epistasis scan we found eight QTL interacting epistatically involving chromosomes 2, 4, 5, 11, 14, 15 and 18. Taken together, the QTL and their interactions explain nearly 49% (39.5% adjusted multiple r2) of the phenotypic variation for litter size in this cross, an increase of 36% over the direct effects of the QTL. This indicates the importance of epistasis as a component of the genetic architecture of litter size and fitness in our intercross population.