Jornal de Biometria e Bioestatística

Statistical Evaluation of the Validity of Real-World Data and Real-World Evidence

Abstract

Yuankang Zhao* and Shein-Chung Chow

Real-world data (RWD) often consist of positive or negative studies and the data may be structured or unstructured. In this case, the validity of realworld evidence (RWE) that derived from RWD is a concern for providing substantial evidence regarding the safety and efficacy of the test treatment under investigation. The validity of RWD/RWE is essential, especially when it is intended to support regulatory submission. In practice, studies with positive results are more likely accepted in RWD, which may cause substantial selection bias. In this article, a quantitative form of selection bias is defined and studied. Based on the form of bias, three reproducibility probability-based approaches are proposed to estimate the true proportion of positive studies in the structural and unstructured data. The reproducibility probability-based approach provides effective bias adjustment when the proportion of significant studies in RWD is different as designed power based on the result of simulation study. The Estimated Power approach and Bayesian approach provide robust and effective bias adjustment in most cases and the Confidence Bound approach provide huge and effective adjustment only when bias is larger than 10%. The proposed adjustment method in conjunction with other treatment effect specification method is useful in estimating the treatment effect based on RWD.