GPUs are gradually becoming mainstream in supercomputing as their capabilities to significantly accelerate a large spectrum of scientific applications have been identified and proven. Moreover, with the introduction of directive based programming models such as OpenACC, these devices are becoming more accessible and practical to use by a larger scientific community. However, performance optimization of OpenACC applications usually requires an indepth knowledge of the hardware and software specifications. We suggest a prediction-based performance tuning mechanism to quickly tune OpenACC parameters to dynamically adapt to the execution environment on a given system. This approach is applied to a finite difference kernel to tune the OpenACC gang and vector clauses for mapping the computations into the underlying accelerator architecture. Our experiments show a good performance improvement against the default compiler parameters and a faster tuning by an order of magnitude compared to the brute force search tuning.