Efficiencies of organic photovoltaic (OPV) devices have been steadily climbing but there is still a prominent gap in understanding the relationship between fabrication and performance. Sidechain substitution is one processing parameter that can change OPV device efficiency considerably, primarily due to variations in morphology. In this work, we explain the morphological link between sidechain selection and device performance in one polymer to aid in the development of design rules more broadly. We study the morphology of an OPV active layer using a PBDTTPD backbone polymer with four different sidechain configurations, which are shown to change device efficiency by up to 4 times. The optimal device has the smallest domain sizes, the highest degree of crystallinity, and the most face-on character. This is achieved with two branched 2-ethylhexyl (2EH) sidechains placed symmetrically on the BDT unit and a linear C8 sidechain on the TPD unit. Substituting either sidechain (C14 on BDT and/or 2EH on TPD) makes the orientation less face-on, while the TPD sidechain primarily affects domain size. For all sidechains, addition of fullerene improves polymer crystallization compared to the neat film, but the degree of mixing between polymer and fullerene varies with sidechain. Interestingly, the optimal device has a negligible amount of mixed phase. The domain sizes present in the optimal system are remarkably unchanged with changing fullerene ratio between 10 and 90%, hinting that the polymer preferentially self-assembles into 5-10 nm crystallites regardless of concentration. The formation of this crystallite may be the key factor inhibiting mixed phase.