This study investigates the efficiency of low-power ultrasound in the range of 3.5–30.0 W to improve permeate flux and alleviate membrane fouling in an air–gap membrane distillation (AGMD) system. Natural groundwater and reverse osmosis (RO) reject water were fed into the AGMD system on which fouling experiments were conducted with hydrophobic polyvinylidene fluoride (PVDF) membrane. After 35 h of AGMD system operation with groundwater and RO reject water, fouling caused the permeate flux to decrease by 30% and 40% respectively. Concentration polarization, intermediate pore blocking, and cake filtration appear to be the main reasons for flux decline with both feedwater types. Ultrasound application for a short period of 15 min resulted in flux improvement by as high as 400% and 250% for RO reject and groundwater, respectively. Modelling of the heat and mass transfers showed that the flux increase was mainly due to membrane permeability improvements under ultrasonic vibration. Fouling visualisation using Scanning Electron Microscopy revealed that ultrasound effectively removed membrane fouling without compromising the membrane's structure. Importantly, permeate flux improvements with targeted low-power ultrasound appears to be proportionally higher than those of high-power ultrasound applied to the whole system, on a flux improvement per ultrasound W/m2 basis.