We reviewed all approvals for first indications for non-oncology applications based on SAT strategies submitted to FDA and EMA to summarize common factors. Briefly, we found that regulatory approvals primarily occurred in contexts involving rare diseases characterized by limited or insufficient standard of care options and a notable unmet medical need in debilitating or life-threatening conditions. When implemented, external control arms (ECAs) most frequently were derived from natural history studies, both retrospective and prospective. Criticisms of ECAs commonly revolved around issues such as an imbalance between the ECA and trial arm, leading to confounding. Additionally, concerns were raised about outcome ascertainment bias resulting from measurement errors or subjective endpoints, along with data quality issues attributed to missing data, potentially introducing selection bias.

Patient-facing labels occasionally referenced single-arm trials, suggesting their relevance in the context of communication to patients. FDA approvals more frequently included information on pivotal single-arm trials and ECAs in product labeling than did EMA approvals. This may be due to differences between the agencies in the provision product information to patients and providers. FDA’s package insert serves as a label for both healthcare professionals and patients, while the EMA provides a separate summary of product characteristics (SmPC) for providers that differs from the patient-facing package leaflet or label [59, 60]. While reviewing EMA SmPC documents was not in scope for this review, previous studies have found these documents to contain detailed information on clinical trials. [61, 62]

External control arms (ECAs), frequently drawn from registries or natural history studies, played a key role providing context to single-arm trials. Notably, our analysis found no ECAs that explicitly described using EHR or claims data. We found that in the absence of these data sources, submissions often relied on natural history studies to provide necessary context and while many NH studies did used patient-level healthcare data through retrospective chart reviews, none mentioned EHR data explicitly. Reviews did not explicitly state which such chart reviews used EHRs or whether EHRs were used to populate case report forms. Verification was not possible due to differing timelines of transition from paper to electronic records across health systems. Nevertheless, our finding is consistent with the documented gap in the availability of research-grade Real World Data (RWD) for the use of external control arms in rare diseases [63]. This scarcity poses a challenge in utilizing external control arms from electronic health records or claims data for non-oncology trials particularly in rare diseases, where they can contribute essential contextual information to the evaluation of single-arm trials. These observations deepen our understanding of the regulatory landscape surrounding single-arm trials and highlight the challenges associated with the choice of external control arms, as well as the reliance on natural history studies for context. As clinical trials increasingly utilize EHR data for various purposes and methodological approaches continue to evolve for implementing EHRs as external control arms, we may see EHRs will see increased use as ECAs in non-oncology indications. [64,65,66]

The findings of the current study align with several past findings from other reviews of regulatory submissions using RWD and/or external control arms [11, 17, 18]. Like Jahanshahi et al., we found that single-arm trials met greater regulatory acceptance in the context of rare diseases. Similarly, Jaksa et al.'s examination of the influence of external control arms (ECAs) on regulatory decisions and the importance of data quality corresponds closely with our identification of criticisms related to imbalances between ECAs and trial arms, outcome ascertainment bias, and data quality issues. Izem et al.'s focus on the contextualization of single-arm trials using real-world data (RWD) aligns with our study's emphasis on the rare disease context and the utilization of natural history studies as common external controls.

Our results are also consistent with studies noting a general increase and upward trend in the use of RWE in regulatory submissions in both the United States and European Union. In a review of NDA submissions to the FDA from 2019 to 2022, Purpura et al. found a substantial increase in single-arm trials, reported that regulators often flagged issues with endpoint objectivity, and emphasized need for increased guidance for assessing single-arm trials as fit for regulatory submission and approval [14]. In our study, we found that the majority of products approved with pivotal SATs had objective and/or large expected endpoint sizes. This is in alignment with Vaghela et al.’s recent systematic review of FDA-approved non-oncology orphan drug therapies that used RWD, which found increased regulatory acceptance of RWD studies demonstrating a large effect size [67]. Our finding that natural history studies constituted all RWD-based external control arms appears aligns with an earlier study of EMA authorizations and FDA approvals; Flynn et al. found that registries were the most commonly used data source in 2018 and 2019. [15]

Our findings align with the recent FDA guidance, which supports the use of externally controlled trials in rare diseases with well-defined natural histories and limited treatment options. Both our findings and the FDA guidance highlight the importance of high quality patient-level data. Similar to the concerns raised in the guidance, our study noted significant critiques regarding the comparability of ECAs. The high proportion of FDA approvals mentioning SAT data in product labeling mirrors the agency’s emphasis on transparency in presenting efficacy evidence​. While the EMA does not currently have dedicated guidance on externally controlled trials, the ICH E10 guideline on control groups discusses external controls, emphasizing the necessity for appropriate methodological approaches to ensure the validity and reliability of the efficacy data [68, 69]. The 2001 EMA guideline takes a more cautious stance than the FDA despite similar numbers of approvals between agencies in this review. This may suggest a need for updated guidance on externally controlled trials that reflects current European regulatory perspectives. In the absence of updated EMA guidance and relative recency of FDA guidance on SATs and ECAs, our framework provides a useful and succinct summary of key considerations that is consistent with the present regulatory landscape.

Despite the many strengths of this review, there were some limitations. First, while the development of our framework was a phased process conducted with expert input and focus groups, we did not conduct systematic reviews or structured interviews to guide its creation. Instead, we chose to test the framework with a systematic approach for novel approvals in non-oncology as they pose potentially the greatest regulatory challenges to SAT submissions. Further testing of other aspects of this framework to better understand regulatory considerations in other types of submissions.

We were unable to compare results from approvals to applications that were ultimately rejected by the FDA, because these are not publicly available. While the EMA does publish reports on authorization applications that were refused or withdrawn, this was not in scope for this study. Within our study scope, we are unable to pinpoint why certain applications using SATs as pivotal evidence were approved, while others may not have been. In addition, we based our review on unstructured medical and statistical reviewer comments, and some factors may not have been mentioned despite being relevant. Additionally, we did not collect data on the history of communications between the applicant and agency. Future studies would benefit from more detailed monitoring of communications between parties to determine whether aspects in the communication between agency and applicant influence the likelihood of a new drug or biologic application being approved.

We were also limited by differences between how the EMA and FDA review drug applications. The EMA appeared to publish approvals more consistently than the FDA, and EMA approval documents maintained a uniform format, making the analysis of European approvals more systematic. In some instances, the agencies also classified evidence differently. For example, the FDA language described single-arm pediatric extrapolation studies at the time of submission for the first indication, as pivotal evidence while the EMA considered these studies supportive. Due to these differences, we elected to remove studies from our analysis that used single-arm trials exclusively to extrapolate to pediatric patients. Discrepancies both within and across agencies in how trials were presented in review documentation may also have led to bias in the identification of pivotal vs. supportive trials.

Lastly, our analysis had somewhat limited scope. Our exclusion of oncologic indications may limit the generalizability of these results. Most applications covered conditions that were rare (including orphan drugs), had significant unmet medical need, and lacked effective SoC options. Thus, it is difficult to assess if and how single-arm trials and ECAs could be employed for conditions that are more common and have acceptable, if not ideal, SoC therapies. While traditional RCTs could be used to study new therapies for common conditions, it can be difficult to recruit for RCTs if the control arm is not as effective as treatments that are already available. A smaller control arm in the RCT along with a well-constructed ECA could be beneficial and improve efficiency and duration of trials to provide patients faster access to effective medicines. We restricted our review to initial indications, excluding supplemental applications and label expansions. Future studies should consider including approvals outside this time window, therapeutic areas, and submission types to determine if the findings are consistent.

Despite the limitations, this review is the first to directly assess regulatory responses to specific features of single-arm trials submitted as pivotal evidence for product approvals and authorizations. Our study offers the first comprehensive examination of how regulators respond to submissions employing these designs beyond the realm of oncology. This departure from the oncology-focused analyses is particularly robust for two reasons: a) single-arm trials tend to encounter greater regulatory acceptance in oncology submissions, necessitating a distinct evaluation for other therapeutic areas, and b) the landscape of available data for comparison and context differs significantly outside of oncology. Our pre-specified systematic methodology involved scrutinizing all approvals within a specified timeframe, to identify single-arm trials submitted to support approval in filings without RCTs. This methodological approach allowed us to meticulously sift through an extensive volume of regulatory data, to provide a comprehensive understanding of the regulatory landscape surrounding single-arm trials across various therapeutic domains.

Our results are consistent with the medical, regulatory, methodological, and data quality factors identified to affect regulatory acceptance of SATs in our framework. In a fast evolving regulatory landscape in the United States and Europe, our framework provides a summary that is useful early in drug development stages, allowing stakeholders to understand potential regulatory critiques that they may face in using a single arm study for pivotal evidence in non-oncology approvals.