TY - JOUR
T1 - Consensus on the definition and assessment of external validity of randomized controlled trials
T2 - A Delphi study
AU - Jung, Andres
AU - Braun, Tobias
AU - Armijo-Olivo, Susan
AU - Challoumas, Dimitris
AU - Luedtke, Kerstin
N1 - Publisher Copyright:
© 2023 The Authors. Research Synthesis Methods published by John Wiley & Sons Ltd.
PY - 2023/12/25
Y1 - 2023/12/25
N2 - External validity is an important parameter that needs to be considered for decision making in health research, but no widely accepted measurement tool for the assessment of external validity of randomized controlled trials (RCTs) exists. One of the most limiting factors for creating such a tool is probably the substantial heterogeneity and lack of consensus in this field. The objective of this study was to reach consensus on a definition of external validity and on criteria to assess the external validity of RCTs included in systematic reviews. A three-round online Delphi study was conducted. The development of the Delphi survey was based on findings from a previous systematic review. Potential panelists were identified through a comprehensive web search. Consensus was reached when at least 67% of the panelists agreed to a proposal. Eighty-four panelists from different countries and various disciplines participated in at least one round of this study. Consensus was reached on the definition of external validity (“External validity is the extent to which results of trials provide an acceptable basis for generalization to other circumstances such as variations in populations, settings, interventions, outcomes, or other relevant contextual factors”), and on 14 criteria to assess the external validity of RCTs in systematic reviews. The results of this Delphi study provide a consensus-based reference standard for future tool development. Future research should focus on adapting, pilot testing, and validating these criteria to develop measurement tools for the assessment of external validity.
AB - External validity is an important parameter that needs to be considered for decision making in health research, but no widely accepted measurement tool for the assessment of external validity of randomized controlled trials (RCTs) exists. One of the most limiting factors for creating such a tool is probably the substantial heterogeneity and lack of consensus in this field. The objective of this study was to reach consensus on a definition of external validity and on criteria to assess the external validity of RCTs included in systematic reviews. A three-round online Delphi study was conducted. The development of the Delphi survey was based on findings from a previous systematic review. Potential panelists were identified through a comprehensive web search. Consensus was reached when at least 67% of the panelists agreed to a proposal. Eighty-four panelists from different countries and various disciplines participated in at least one round of this study. Consensus was reached on the definition of external validity (“External validity is the extent to which results of trials provide an acceptable basis for generalization to other circumstances such as variations in populations, settings, interventions, outcomes, or other relevant contextual factors”), and on 14 criteria to assess the external validity of RCTs in systematic reviews. The results of this Delphi study provide a consensus-based reference standard for future tool development. Future research should focus on adapting, pilot testing, and validating these criteria to develop measurement tools for the assessment of external validity.
UR - http://www.scopus.com/inward/record.url?scp=85180893977&partnerID=8YFLogxK
U2 - 10.1002/jrsm.1688
DO - 10.1002/jrsm.1688
M3 - Journal articles
AN - SCOPUS:85180893977
SN - 1759-2887
JO - Research Synthesis Methods
JF - Research Synthesis Methods
ER -
