Full waveform inversion (FWI) aims fully benefit from all the data characteristics to estimate the parameters describing the assumed physics of the subsurface. However, current efforts to utilize full waveform inversion as a tool beyond acoustic imaging applications, for example in reservoir analysis, faces inherent challenges related to the limited resolution and the potential trade-off between the elastic model parameters. Adding rock physics constraints does help to mitigate these issues, but current approaches to add such constraints are based on including them as a priori knowledge mostly valid around the well or as a boundary condition for the whole area. Since certain rock formations inside the Earth admit consistent elastic properties and relative values of elastic and anisotropic parameters (facies), utilizing such localized facies information in FWI can improve the resolution of inverted parameters. We propose a novel confidence map based approach to utilize the facies-based constraints in both isotropic and anisotropic elastic FWI. We invert for such a confidence map using Bayesian theory, in which the confidence map is updated at each iteration of the inversion using both the inverted models and a prior information. The numerical examples show that the proposed method can reduce the trade-offs and also can improve the resolution of the inverted elastic and anisotropic properties.