Covariance tapering is a popular approach for reducing the computational cost of spatial prediction and parameter estimation for Gaussian process models. However, tapering can have poor performance when the process is sampled at spatially irregular locations or when non-stationary covariance models are used. This work introduces an adaptive tapering method in order to improve the performance of tapering in these problematic cases. This is achieved by introducing a computationally convenient class of compactly supported non-stationary covariance functions, combined with a new method for choosing spatially varying taper ranges. Numerical experiments are used to show that the performance of both kriging prediction and parameter estimation can be improved by allowing for spatially varying taper ranges. However, although adaptive tapering outperforms regular tapering, simply dividing the data into blocks and ignoring the dependence between the blocks is often a better method for parameter estimation.