Total Variation (TV) is an effective and popular prior model in the field of regularization-based image processing. This paper focuses on TV for image restoration in the presence of impulse noise. This type of noise frequently arises in data acquisition and transmission due to many reasons, e.g. a faulty sensor or analog-to-digital converter errors. Removing this noise is an important task in image restoration. State-of-the-art methods such as Adaptive Outlier Pursuit(AOP) , which is based on TV with l(02)-norm data fidelity, only give sub-optimal performance. In this paper, we propose a new method, called l(0)TV-PADMM, which solves the TV-based restoration problem with to-norm data fidelity. To effectively deal with the resulting non-convex non-smooth optimization problem, we first reformulate it as an equivalent MPEC (Mathematical Program with Equilibrium Constraints), and then solve it using a proximal Alternating Direction Method of Multipliers (PADMM). Our l(0)TV-PADMM method finds a desirable solution to the original to-norm optimization problem and is proven to be convergent under mild conditions. We apply l(0)TV-PADMM to the problems of image denoising and deblurring in the presence of impulse noise. Our extensive experiments demonstrate that l(0)TV-PADMM outperforms state-of-the-art image restoration methods.