Primary Study Quality in Psychological Meta-Analyses: An Empirical Assessment of Recent Practice

Abstract

As meta-analytic research has come to occupy a sizeable contingent of published work in the psychological sciences, clarity in the reporting of such work is crucial to its interpretability and reproducibility. This is especially true regarding the assessment of primary study quality, as notions of study quality can vary across research domains. The present study examines the general state of reporting practices related to primary study quality in a sample of 382 published psychological meta-analyses, as well as the reporting decisions and motivations of the authors that published them. Our findings suggest adherence to reporting standards has remained poor for assessments of primary study quality and that the discipline remains inconsistent in its reporting practices generally. We discuss several potential reasons for the poor adherence to reporting standards in our sample, including whether quality assessments are being conducted in the first place, whether standards are well-known within the discipline, and the potential conflation of assessing primary study quality with other facets of conducting a meta-analysis. The implications of suboptimal reporting practices related to primary study quality are discussed.