Understanding RWE
Randomized Controlled Trials (RCTs) are the gold standard of drug assessment, as they assess the efficacy and safety results for a new therapeutic option. However, the patient populations that are treated in everyday practice may be somewhat different from those studied in RCTs due to the strict inclusion and exclusion criteria of RCTs. Thus, RWE helps provide important additional information about the effectiveness and safety profile of new therapies in wider patient populations. This section provides a background on what RWE is and why it can be important to consider in clinical decision making.
Why RWE matters
What is real-world evidence and why is it important to clinical decision making?
RWE is clinical evidence regarding the use and potential benefits/risks of a medical product, derived from analysis of RWD.1–4 RWD relates to patient health, therapeutic effectiveness, safety and delivery of healthcare routinely collected from a variety of sources and study designs.1–4
RWE is generated outside of RCTs, in standard clinical practice via observational (non‑interventional) studies1,3 and can include the following data sources:1–4
- Local/national clinical registries
- Electronic health/medical records
- Databases of billing or insurance claims
RWE is important for understanding evolving treatment practices and therapeutic outcomes in the real-world setting.5 This is of particular relevance in the setting of Multiple myeloma (MM) treatment, due to the rapidly expanding range of approved therapeutic options and the increasing complexity of treatment.2
In recent years, significant improvements in efficacy of MM treatments have been demonstrated in RCTs, and RWE is becoming important for determining how these improvements translate to real-world practice.6
RWE is also important for capturing treatment considerations taken in actual clinical practice. These considerations, not always adequately addressed by RCTs, include patient heterogeneity as well as treatment tolerability and convenience.5
For various reasons, including strict inclusion criteria, clinical trials often cannot be conducted in a way that answers all treatment-related questions and patient scenarios. Therefore, RWE has a role in supplementing the information gained from RCTs, and can be an important complementary tool to RCTs in order to gain a more complete picture when reviewing the best management options for MM patients.7
1. Berger, M. L. et al. Pharmacoepidemiol. Drug Saf. 2017; 26: 1033–1039. 2. Garrison, L. P., Neumann, P. J., Erickson, P., Marshall, D. & Mullins, C. D. Value Health J. Int. Soc. Pharmacoeconomics Outcomes Res. 2007; 10: 326–335. 3. Makady, A. et al. Value Health J. Int. Soc. Pharmacoeconomics Outcomes Res. 2017; 20: 858–865. 4. Breckenridge, A. M., Breckenridge, R. A. & Peck, C. C. R Br. J. Clin. Pharmacol. 2019; 85: 1874–1877. 5. Costello, C. et al. Future Oncol. 2019; 15: 1411–1428. 6. Richardson, P. G. et al. Blood Cancer J. 2018; 8: 109. 7. Blonde, L., Khunti, K., Harris, S. B., Meizinger, C. & Skolnik, N. S. Adv. Ther. 2018; 35: 1763–1774.
EHR, electronic health record; EMA, European Medicines Association; FDA, Food and Drug Administration; HCP, healthcare professional; MM, multiple myeloma; OECD, Organization for Economic Co-operation and Development; RCT, randomized controlled trial; RWD, real-world data; RWE, real-world evidence.
Efficacy vs. effectiveness
Efficacy vs. effectiveness: Factors impacting the feasibility and effectiveness of treatment in the real world
RCTs, the gold standard of drug assessment, assess the efficacy and safety results for a new therapeutic option. However, the patient populations that are treated in everyday practice may be somewhat different from those studied in RCTs due to the strict inclusion and exclusion criteria of RCTs.
RCTs look at “can it work”, or efficacy, through evaluating the performance of an intervention under controlled circumstances.1–3 This information provides robust evidence that a treatment does work in a select patient population. The RCT maximizes the ability to identify treatment effects independent of factors unrelated to treatment.2,3
Data from RCTs are important for regulatory approval and for generating evidence‑based guidelines for physicians and other healthcare professionals.2,3
RWE looks at “does it work”, or effectiveness, through evaluating the performance of an intervention under real-world conditions via observational studies.1–3 Comprehensive real-world/observational research can show the potential benefits and risks of an intervention in routine clinical practice in a representative heterogeneous patient population treated in a variety of clinical settings.2 Data from observational studies are gaining increasing regulatory agency interest, as well as interest from payers and health technology assessment authorities, and can provide practical information to physicians and other HCPs about patient management.1,2,9
Patient populations that are typically under-represented in RCTs include the elderly or frail, patients with comorbidities, patients with lower socioeconomic backgrounds, and ethnic or racial minorities.10–13 These characteristics have been linked to worse outcomes than those reflected in RCTs.10–13 In multiple myeloma (MM), advanced age, functional decline, and comorbidities represent components of frailty that are predictive of mortality and toxicity risk.6
Additionally, rigorous RCT protocols mean that patients will be managed under very similar, controlled conditions, and factors such as protocol-specified dose modifications may lead to better tolerability and longer durations of therapy.6 In the real-word environment, patient and physician preference and motivation may not be consistent and factors such as mobility, patient distance from hospital, number of visits required during treatment, healthcare access issues and direct/indirect costs may contribute to premature discontinuation of treatment.6
These considerations make interpretation of data from real-world analyses highly complex. Nevertheless, RWE can help provide additional insights regarding treatment of broader MM patient populations.
1. Berger, M. L. et al. Pharmacoepidemiol. Drug Saf. 2017; 26: 1033–1039. 2. Garrison, L. P., Neumann, P. J., Erickson, P., Marshall, D. & Mullins, C. D. Value Health J. Int. Soc. Pharmacoeconomics Outcomes Res. 2007; 10: 326–335. 3. Makady, A. et al. Value Health J. Int. Soc. Pharmacoeconomics Outcomes Res. 2017; 20: 858–865. 6. Richardson, P. G. et al. Blood Cancer J. 2018; 8: 109. 9. Makady, A. et al. Value Health J. Int. Soc. Pharmacoeconomics Outcomes Res. 2017; 20: 520–532. 10. Shah, J. J. et al. Clin. Lymphoma Myeloma Leuk. 2017; 17: 575-583.e2. 11. Costa, L. J., Hari, P. N. & Kumar, S. K. Leuk. Lymphoma 2016; 57: 2827–2832. 12. Ganguly, S., Mailankody, S. & Ailawadhi, S. Am. Soc. Clin. Oncol. Educ. Book 2019; 519–529: doi:10.1200/EDBK_238551. 13. Pulte, E. D. et al. Blood Adv. 2018; 2: 116–119.
EHR, electronic health record; EMA, European Medicines Association; FDA, Food and Drug Administration; HCP, healthcare professional; MM, multiple myeloma; OECD, Organization for Economic Co-operation and Development; RCT, randomized controlled trial; RWD, real-world data; RWE, real-world evidence.
Strengths and limitations
Strengths and limitations: Reviewing data sources for real‑world evidence
The utility of real-world studies is their ability to complement data from RCTs in order to provide broader insights in clinical knowledge.
Figure 1: Strengths of RWE5,7,14
With improvements in the rigor of methodology being applied to real-world studies, along with the increasing availability of high-quality, large datasets, the importance of findings from RWE is growing.7
RWE often has broader generalizability vs. RCTs, reflecting the heterogeneity of patients treated in real-world clinical practice, including patients with:
- Comorbidities
- Advanced age
- Reduced performance status
Data can validate/complement results from RCTs in real-world clinical practice:
- Effectiveness
- Tolerability
- Safety profile
RWE provides information on real-world treatment patterns:
- Dosing
- Treatment duration
- Resource use
Figure 2: Limitations of RWE6,7,17
However, it is important not to overlook the limitations that RWE presents. Data collection in real-world settings is frequently unstandardized, and accessing patient data can be challenging, particularly with growing concern for data privacy.15
Heterogeneous datasets
- Arising from a heterogeneous patient population and a variety of dataset resources, e.g. EHRs, healthcare claims
- Data may be inconsistently collected or data on key variables may be missed. This can reduce clinical validity
Findings limited to data that are randomly available – susceptible to selection bias
- Potential incompatibility of data from different sources
- Less scrutiny, and less uniformity in reporting or data processing/ collection compared with RCTs
Real-world data do not constitute a uniform assessment of efficacy under controlled conditions:
- Lack of well-characterized, balanced groups of patients in treatment arms
- Not blinded or randomized
- Potential lack of formal, defined endpoints (e.g. use of time to next therapy as a proxy for progression‑free survival due to data availability)
Selection bias
- Many positive results that were historically reported for RWE were a result of selection bias
- RCTs, through their robust design, limit the role of bias in determining the real efficacy of medicines
Additionally, many retrospective RWE publications may be unable to provide full safety information, as toxicity is best described when it is prospectively and explicitly studied.16 Establishment of a causal relationship between the treatment and safety concerns can be more difficult compared with prospective clinical trials.16
Despite its limitations, RWE helps us better understand how to manage patient subgroups who were not analyzed or included in the original clinical trials.
5. Costello, C. et al. Future Oncol. 2019; 15: 1411–1428. 6. Richardson, P. G. et al. Blood Cancer J. 2018; 8: 109. 7. Blonde, L., Khunti, K., Harris, S. B., Meizinger, C. & Skolnik, N. S. Adv. Ther. 2018; 35: 1763–1774.14. Richardson, P. G. et al. Blood 2017; 130: 3149–3149. 15. Rudrapatna, V. A. & Butte, A. J. J. Clin. Invest. 2020: 130: 565–574. 16. Di Maio, M., Perrone, F. & Conte, P. The Oncologist 2020; 25: e746–e752. 17. Kim, H.-S., Lee, S. & Kim, J. H. J. Korean Med. Sci. 2018; 3.
EHR, electronic health record; EMA, European Medicines Association; FDA, Food and Drug Administration; HCP, healthcare professional; MM, multiple myeloma; OECD, Organization for Economic Co-operation and Development; RCT, randomized controlled trial; RWD, real-world data; RWE, real-world evidence.