Studies comparing phenotypic variation with neutral genetic variation in modern humans have shown that genetic drift is a main factor of evolutionary diversification among populations. The genetic population history of our closest living relatives, the chimpanzees and bonobos, is now equally well documented, but phenotypic variation among these taxa remains relatively unexplored, and phenotype-genotype correlations are not yet documented. Also, while the adult phenotype is typically used as a reference, it remains to be investigated how phenotype-genotye correlations change during development. Here we address these questions by analyzing phenotypic evolutionary and developmental diversification in the species and subspecies of the genus Pan. Our analyses focus on the morphology of the femoral diaphysis, which represents a functionally constrained element of the locomotor system. Results show that during infancy phenotypic distances between taxa are largely congruent with non-coding (neutral) genotypic distances. Later during ontogeny, however, phenotypic distances deviate from genotypic distances, mainly as an effect of heterochronic shifts between taxon-specific developmental programs. Early phenotypic differences between Pan taxa are thus likely brought about by genetic drift while late differences reflect taxon-specific adaptations.