Understanding the impact of metal contacts on the recombination within a passivated silicon wafer is crucial for the optimization of various photovoltaic devices such as passivating-contact-based solar cells. To investigate the effect of the metal work function, a selection of metals is applied to aluminum-oxide-passivated n-type crystalline silicon wafers. The saturation current density of the metalized contact (J0m) is determined using the quasi-steady-state photoluminescence method and used as a figure of merit to quantify the effect. We find that J0m increases with the metal work function and that this effect is modulated with the passivation layer thickness. It is more pronounced for thinner passivation layers, which can be attributed to a significant change in the populations of electrons and holes near the silicon surface induced by the metal. Meanwhile thicker layers prevent the charge transfer between the silicon and metal more efficiently leading to insignificant changes in J0m. Based on these findings, we suggest a suitable metal work function range to optimize contact recombination in silicon-based solar cells.