Stokes shift, an energy difference between the excitonic absorption and emission, is a property of colloidal quantum dots (CQDs) typically ascribed to splitting between dark and bright excitons. In some materials, e.g., PbS, CuInS2, CdHgTe, a Stokes shift of up to 200 meV is observed, substantially larger than the estimates of dark-bright state splitting or vibronic relaxations. The shift origin remains highly debated, as contradictory signatures of both surface and bulk character were reported for the Stokes-shifted electronic state. Here we show that the energy transfer among CQDs in a polydispersed ensemble in solution suffices to explain the excess Stokes shift. This energy transfer is primarily due to CQD aggregation, and can be substantially eliminated by extreme dilution, higher-viscosity solvent, or better-dispersed colloids. Our findings highlight that ensemble polydispersity remains the primary source of the Stokes shift in CQDs in solution, propagating into the Stokes shift in films and the open-circuit voltage deficit in CQD solar cells. Improved synthetic control can bring notable advancements in CQD photovoltaics, and the Stokes shift continues to provide a sensitive and significant metric to monitor ensemble size distribution.