The rapid evolution of male genitalia is a pre-eminent evolutionary pattern, but testing functional hypotheses have been traditionally hampered by two obstacles. First, like many biological structures, many male genitalia lack obvious landmarks, so comparing them in three dimensions is not straightforward. Second, we lack a basic understanding of the genetic basis of genital variation, so techniques of genetic manipulation remain unavailable. Here we develop a novel morphometric approach to characterize size and shape variation from three dimensional micro-CT scans taken from 342 bacula (a bone in the penis of many mammals), representing 75 distinct strains of the “BxD” recombinant inbred panel of mice. We identify a quantitative trait locus (QTL) that explained 32% of the variance in baculum size, and two QTL that together explained 46% of the variance in shape. From the QTL, we identified candidate genes using bioinformatic investigations and new data on RNA expression from the baculum. Several promising candidate genes affecting baculum size and shape are discussed. Our study opens the way to experimental manipulation and powerful functional characterization of this bizarre structure.