Reliable quantification of the global mean surface temperature (GMST) response to radiative forcing is essential for assessing the risk of dangerous anthropogenic climate change. We present the statistical foundations for an observation-based approach using a stochastic linear response model that is consistent with the long-range temporal dependence observed in global temperature variability. We have incorporated the model in a latent Gaussian modeling framework, which allows for the use of integrated nested Laplace approximations (INLAs) to perform full Bayesian analysis. As examples of applications, we estimate the GMST response to forcing from historical data and compute temperature trajectories under the Representative Concentration Pathways (RCPs) for future greenhouse gas forcing. For historic runs in the Model Intercomparison Project Phase 5 (CMIP5) ensemble, we estimate response functions and demonstrate that one can infer the transient climate response (TCR) from the instrumental temperature record. We illustrate the effect of long-range dependence by comparing the results with those obtained from one-box and two-box energy balance models. The software developed to perform the given analyses is publicly available as the R package INLA.climate.