Marine prokaryotes play a key role in cycling of organic matter and nutrients in the ocean. Using a unique dataset (>14,500 samples), we applied a space-for-time substitution analysis to assess the temperature dependence of prokaryotic heterotrophic production (PHP) in epi- (0-200 m), meso- (201-1000 m) and bathypelagic waters (1001-4000 m) of the global ocean. Here, we show that the temperature dependence of PHP is fundamentally different between these major oceanic depth layers, with an estimated ecosystem-level activation energy (E) of 36 ± 7 kJ mol for the epipelagic, 72 ± 15 kJ mol for the mesopelagic and 274 ± 65 kJ mol for the bathypelagic realm. We suggest that the increasing temperature dependence with depth is related to the parallel vertical gradient in the proportion of recalcitrant organic compounds. These Ea predict an increased PHP of about 5, 12, and 55% in the epi-, meso-, and bathypelagic ocean, respectively, in response to a water temperature increase by 1°C. Hence, there is indication that a major thus far underestimated feedback mechanism exists between future bathypelagic ocean warming and heterotrophic prokaryotic activity.