Since the emergence of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), large-scale social contact surveys are now longitudinally measuring the fundamental changes in human interactions in the face of the pandemic and non-pharmaceutical interventions. Here, we present a model-based Bayesian approach that can reconstruct contact patterns at 1-year resolution even when the age of the contacts is reported coarsely by 5 or 10-year age bands. This innovation is rooted in population-level consistency constraints in how contacts between groups must add up, which prompts us to call the approach presented here the Bayesian rate consistency model. The model can also quantify time trends and adjust for reporting fatigue emerging in longitudinal surveys through the use of computationally efficient Hilbert Space Gaussian process priors. We illustrate estimation accuracy on simulated data as well as social contact data from Europe and Africa for which the exact age of contacts is reported, and then apply the model to social contact data with coarse information on the age of contacts that were collected in Germany during the COVID-19 pandemic from April to June 2020 across five longitudinal survey waves. We estimate the fine age structure in social contacts during the early stages of the pandemic and demonstrate that social contact intensities rebounded in an age-structured, non-homogeneous manner. The Bayesian rate consistency model provides a model-based, non-parametric, computationally tractable approach for estimating the fine structure and longitudinal trends in social contacts and is applicable to contemporary survey data with coarsely reported age of contacts as long as the exact age of survey participants is reported.