We investigate the impact of players with heterogeneous update rules on the long-term behavior of a population under stochastic learning dynamics. We show that under certain conditions, the presence of even a single heterogeneous player with a different decision making strategy can significantly alter the long-term behavior of the entire population. To quantify the impact of a heterogeneous player, we define a new notion of robustness of stochastic learning dynamics to player heterogeneity. Based on our proposed notion, an action profile that is stochastically stable under the standard setup is robust to player heterogeneity if it can still explain the long-run behavior of all the players other than the heterogeneous players. We consider two types of heterogeneous players: A confused player who randomly updates his actions and a stubborn player who never updates his action. For each of these types, we present a qualitative description of scenarios in which an action profile that is stochastically stable under the standard setup is not robust to the presence of a heterogeneous player of a particular type.