The classic RCT

In the classic two-arm RCT design, following recruitment, determination of eligibility, and measurement of baseline variables, participants or groups are randomized to active or control study arms. In the text below, we emphasize cluster-randomized trials, but whether individuals or groups are randomized, the general study design is the same, and we do not dwell on that difference here. An intervention is delivered to the active study arm, and the control arm receives either a comparison intervention, a placebo attention control, or no new intervention. The outcome is subsequently measured in both arms after a pre-determined time interval, and the difference in the outcome between study arms is calculated to form the estimand [10, 11].

Initially, it may be helpful to envision a DRCT as two classic RCTs occurring side-by-side with overlapping recruitment and randomization steps (Fig. 1). The effect of implementation strategies on implementation outcomes is tested in one RCT, and in a second RCT, the effect of the intervention on health-related outcomes is tested. In practice, often multiple implementation strategies and intervention components are delivered at different levels. Each RCT would (usually) have a single hypothesis identified as primary, following the Consolidated Standards of Reporting Trails (CONSORT) guidelines [12]. Thus, in a DRCT, there are two primary hypotheses that are equally valued and tested in a single study that includes adequate power for both of the primary statistical models. Whether to apply correction for multiple testing in this situation is controversial and will not be discussed here [12,13,14]. Following usual trial methodology [12], the two primary statistical models are fully pre-specified before the beginning of the study, which is often demarcated by the initiation of randomization. Numerous secondary hypotheses are also studied.

Fig. 1

Type 2 hybrid study using DRCT with four arms

Rather than focusing on either the intervention or the implementation strategy, the hybrid type 2 study spreads available resources to study both. To justify doing so, Curran et al. [1] recommended specific conditions for the conduct of a hybrid type 2 study; however, they do not give specific guidance for the use of a DRCT design. A DRCT is used when investigators want rigorous evidence that has low risk of bias and quantifiable certainty indicating whether or not there are differences in implementation and health-related outcomes among two or more alternative implementation and intervention approaches that are in equipoise.

Curran et al. [2] note that the simultaneous examination of implementation strategies and interventions in a type 2 study is sometimes best served by a liberal approach that does not involve an adequately powered randomized design. We agree that type 2 studies using more liberals designs are important to the field; however, we do not classify a DRCT as a liberal design, as it stands at the opposite end of the spectrum of type 2 studies. Adherence to the tenets of an RCT (including a DRCT) requires pre-specification of intentions, masking, and reduced flexibility to make changes in the protocol and/or procedures once the trial begins [3, 12, 15]. Therefore, other designs are usually a better choice for exploratory or demonstration hybrid type 2 studies.

Examples of common implementation outcomes are reach to individual participants (number, %, representativeness), fidelity of intervention delivery, cost to implement, individual adoption of the intervention, institutional adoption of the intervention, and maintenance over time [16]. Intervention outcomes are health behaviors, health outcomes, and health-related environmental characteristics. Respective examples include smoking, incident stroke, and healthy food availability. These outcomes are often studied as counts, proportions, incidence, or means at the end of the study period or as changes from baseline. Here we assume superiority trials are being used; however, a DRCT can be a non-inferiority trial [15]. For example, a new implementation and intervention could be hypothesized to produce effects that are not inferior to an existing treatment that requires more resources.

Exposures and study arms

Figure 1 is familiar to investigators experienced in RCT designs, but it is not an ideal depiction of a type 2 hybrid DRCT because in that design each study arm is, by definition, assigned an implementation exposure and an intervention exposure. Here, we use the word “exposure” following the text by Rothman et al. [17] that defines the term broadly as a potential causal factor that can be a behavior, treatment, trait, or exogenous factor. An exposure can be naturally occurring or randomized to be part of the study arms of an RCT (as in this paper).

In Table 1, we introduce notation to represent five types of exposures that can be paired to produce a DRCT design with three study arms. More arms are possible, but at least three are required. In this paper, we focus on the three-arm design. We use “S” to indicate implementation strategy exposures and “P” to indicate intervention exposures with reference to the 7 Ps listed by Brown et al. [18] to describe different categories of interventions found in dissemination and implementation research (programs, practices, principles, procedures, products, pills, and policies). We use the words strategy (S) and program (P), rather than implementation and intervention, to differentiate exposures from arms and to escape the confusion caused by the common first letters and strong alliteration in the words implementation and intervention. Active and control exposures are denoted by “a” and “c,” respectively. The implementation strategy exposure used in the control arm (SxPc) for the test of the intervention is denoted by “Sx” rather than “Sc” to indicate that it is not the same as in the control arm for the test of the implementation (ScPa).

Table 1 Five exposures that compose 3 study arms

Although the assignment of pairs of exposures to study arms has similarities to a factorial design, that paradigm is not a good fit for a DRCT. A conventional 2 × 2 factorial design has two levels of one factor (e.g., high or low education) applied to two levels of another (e.g., young or old age) to form four experimental sets. The factors that make up the four pairs are the same (age and education) and there is only one outcome (e.g., body weight) and the factor effect can be derived from the difference between factor levels. A DRCT studies two primary outcomes: one related to implementation and one related to health. Also, the four experimental sets are not always, or even usually, composed of the same two levels of the two factors defined in the same way, and the investigators are not usually interested in the average implementation effect in groups that received either the active or the control intervention.

Core function and form, two key constructs used to describe complex health interventions [19, 20], can also be applied to implementation strategies. Core function is the fundamental purpose or desired effect (on patients, health professionals, or staff) of a set of activities hypothesized to support change. An implementation example is education to establish the knowledge base necessary for community health workers to deliver a specific intervention. The form describes how activities and other supports operationalize the core functions of the intended implementation. For example, education may take the form of hard copy materials, online videos, and/or one-to-one or group-based in-person teaching. The form can vary in dose (frequency and duration). The core functions will influence content. One or more aspects of core functions, form, and/or content will differ between the active and comparison exposures, and the core functions and content of the implementation strategies serve the intervention within a study arm.

Implementation strategies and the intervention are linked and theoretically delivered in sequence

Ideally, implementation strategies are constructed by identifying multilevel determinants that are facilitators and barriers to the successful uptake of the intervention in the target population. Thus, the implementation strategies are relevant to the core functions, form, and other characteristics of the intervention. The determinants identified are used to create implementation strategies that are hypothesized to support the reach, adoption, fidelity, and/or sustainment of the intervention [21]. Often an implementation strategy is applied at the provider level and enhances the ability of the provider to deliver the intervention to participants. The nature of this association sets up a natural order in which the implementation strategies theoretically, if not operationally, precede the intervention.

SaPa is the active arm that investigators want to evaluate as a candidate for dissemination, spread and/or scaling up

Here we assume that the investigators are interested in the usefulness of a newly developed implementation exposure (Sa) applied to a specific intervention program (Pa). The intervention can be untested, but in implementation science, often the intervention is evidence-based. A DRCT is conducted with the ultimate sustainment and spread of SaPa in mind if it is proven to produce stronger effects on intervention uptake and the health outcome than the control conditions. Therefore, the three-arm design in Fig. 2 shows SaPa in the center of the diagram serving as the active study arm in the test of both the implementation strategy and the intervention against their respective control arms. A DRCT is formed by randomizing units of the target sample (be they individuals, providers, or institutions) to one of these three arms to construct three groups.

Fig. 2

Type 2 hybrid study using DRCT with three arms

Example of active implementation strategies and intervention program (SaPa) in a hybrid type 2 DRCT examining implementation uptake and health effects

Below we walk the reader through the DRCT design using constructed examples that evaluate acceptance and commitment therapy (ACT) [22] as the active intervention program (Pa). ACT is an existing evidence-based treatment (i.e., intervention) for anxiety. The target population selected for the implementation strategy is therapists in clinics, and the target population for the intervention is adolescent patients with anxiety disorder, a population for whom evidence of ACT efficacy is emerging [23]. The active implementation strategy (Sa) is clinical supervision in a 1-h group meeting facilitated by a therapist with expertise in ACT. Over a 6-month period, therapists at the clinic meet biweekly, in-person to share situations encountered and lessons learned with each other and to get expert input on strategies for overcoming challenges that arise in the patients they are treating using ACT. These activities are in addition to the six, 20-min online training sessions of didactic content on ACT for therapists that were already being provided at all the clinics. The unit of randomization is the clinic. The primary implementation outcome variable is the percentage of eligible patients that complete at least 8 sessions of the 10-session ACT intervention. The primary intervention outcome variable is the percent change from baseline in patients’ reported anxiety symptoms on a standardized, interviewer-administered questionnaire. As recommended in the CONSORT guidelines [12], outcome variables are collected by research staff who are not implementation or intervention providers and are masked to randomization assignments when feasible.

Example of comparison of implementation outcomes in the active versus control implementation arms

The control implementation arm will include the same intervention exposure (ACT) as described above for the active implementation arm, but will have a different implementation exposure added to the pre-existing six, 20-min online sessions. The control implementation exposure (Sc) in the control arm (ScPa) could be a time-matched alternative in the form of group meetings for therapists focused on business development for the practice. Like the active arm, therapists at the clinic meet biweekly in-person, but the topics discussed are focused on issues such as networking and securing additional patients. The comparison of the percentage of eligible patients who have completed at least 8 sessions of the 10-session ACT intervention in the SaPa and ScPa arms is the estimand [10, 11]. This design holds the intervention constant while comparing two different implementation strategies. It is the same design commonly used to test the primary hypothesis in a hybrid type 3 trial [24, 25] and forms one of the two strands of a DRCT.

Features and recommendations for the study of the implementation:

1.

Using the same intervention exposure across arms. Keeping the intervention exposure constant enhances the interpretation of differences in the impact of the active versus the control implementation strategy on implementation outcomes. Nevertheless, in behavioral work, even when the same intervention is intended differences can occur across arms. For example, the fidelity of the intervention may be enhanced as a result of an active implementation strategy that creates better-trained providers who are more effective in the delivery of the intervention. In this example, enhanced fidelity caused by the implementation could mediate effects of the intervention on the health outcome. In studies in which investigators hypothesize improvement of the intervention by the implementation, it will be a priority to examine the impact of the different implementation strategies on the health outcome. Although very important, that analysis is not a requirement in a DRCT. It is an additional test of the implementation strategies, not the randomized test of the intervention.

2.

Selecting implementation strategies for the active (Sa) and control (Sc) exposures to construct a question of strong scientific merit. Investigators may want to compare two sets of implementation strategies newly introduced to a sample or a newly introduced set of implementation strategies compared to strategies that already exist in the sample (usual care). If the research seeks to improve or expand on the implementation strategies that are already in place in the study setting, then usual care is an option that can provide the control implementation strategy (Sc) to be compared with alternative or additional implementation strategies (Sa). This circumstance brings up the often-ignored point that evidence-based intervention programs already include some sort of implementation strategy in order for the intervention to have taken place previously. New hybrid research may be launched when it is determined that the implementation strategies included in the evidence-based intervention program are not adequately promoting uptake of the intervention or need to be adjusted to serve a different setting. For instance, the Kaiser Permanente bundle [26], which aims to control hypertension, includes multiple implementation strategies (e.g., follow-up visits for blood pressure management, education of healthcare staff), but new implementation strategies (e.g., delivery of a toolkit with additional training and education materials) added to those can be tested [27]. In hybrid research those implementation strategies that are part of the evidence-based intervention program are usually maintained in both study arms and not the subject of study.

A distinguishing attribute of implementation research is that if the usual care and/or new implementation exposures are known to likely have no effect on the implementation outcomes, the information gained from the research is often limited. For example, if we assume a situation in which there is no mention of ACT, or use of ACT existing in clinics prior to the research, then a usual care exposure in the implementation control arm would likely result in therapists having no increase in awareness or knowledge of the ACT intervention, and their preparation for and delivery of the ACT program would be zero (for all practical purposes). Collecting measurements on such an implementation with a zero outcome is not valuable as the outcome is essentially already known. In addition, comparisons of zero implementation endpoints in the control arm to those in an active implementation arm may be statistically significant, even if unacceptably weak. Under most circumstances, this type of implementation control group should be avoided in favor of a control that supports equipoise and provides stronger public health relevance.

Comparison of health outcomes in the active and control intervention arms

The goal in this strand of the DRCT is to determine if the active intervention, supported using the active implementation strategy, is more effective at improving the health outcome in the target population compared to a control arm. As shown in the figures, the study sample, randomization, and active arm (SaPa) are shared between strands; but the control arm, outcomes, and tests of statistical significance are different and separate as two different hypotheses are being tested. In the study of the health outcome, the decision process of choosing the control arm should not be substantially different from that used outside of implementation research to test an intervention. Investigators choose a control arm to compare to the active arm (SaPa) that maximizes relevance and health impact, given resources. Those criteria should be given prominence over maximizing effect size.

Features and recommendations for the study of the intervention:

1.

Combining effects of the implementation strategies and the intervention on the health outcome. In this strand of a DRCT, the active implementation strategy seems the natural implementation exposure to use in both intervention arms (comparing SaPa vs SaPc). This design offers the opportunity to hold the implementation strategy constant while contrasting the active and comparison interventions and appears to possess the same logic as the design recommended above for the test of the implementation outcomes. However, this appearance is misleading. It is rarely logical to have the entirely same implementation strategy used to prepare for the administration of two different interventions. The implementation strategies are typically tailored to address barriers and facilitators specific to implementing a particular intervention within the given context. Also, the active and control implementation strategies will need to be consistent with the respective core functions of the change processes that the two interventions seek to cause. As mentioned earlier, theoretically, the application of the implementation precedes the intervention, but it is useful to characterize the intervention before designing the implementation. Regardless of whether the active and control implementations are similar or different, the active implementation will be incorporated into the packaged protocol alongside the intervention should it be disseminated or spread. Therefore, in this strand of the research, it is more relevant for investigators to compare the impact of the “implemented interventions” rather than to attempt to dissect effects on the health outcome.

2.

Examples of different types of control interventions. Below we list different possible control intervention exposures (Pc) assuming that the active arm delivers ACT to adolescent patients with anxiety disorder (as in our example above):

i)

An alternative treatment for anxiety that has characteristics similar to ACT such as cognitive behavioral therapy (CBT) (as in Fig. 3).

Fig. 3

Example hybrid type 2 DRCT examining fidelity of the ACT evidence-based intervention and changes in anxiety symptoms

ii)

An alternative treatment for anxiety that shares few characteristics with ACT such as free membership in a commercial gym.

iii)

A placebo intervention that provides attention but is not hypothesized to substantially impact anxiety such as a book club.

iv)

No intervention activities.

All of these types of control interventions could be new or they could be ongoing in the research setting and, in the latter situation, would be called usual care.

3.

Implementation exposure (Sx) in the control intervention arm (SxPc). Below we discuss the selection of implementation strategies to complement the four control intervention examples shown above.

i)

Implementation exposure when control intervention consists of activities that are similar in function and form to those in the active intervention. In our first example above, the two interventions to be compared (ACT and CBT) are comprised of similar activities, setting, and staff, but are different in content. Therefore, the implementation strategy involves similar form (clinical supervision and subsequent meetings) and function (peer networking and skills building) but different content (ACT skills vs. CBT skills). In this example, the forms of the implementation as well as the amount of attention (interaction with implementation staff and investigators) given to the therapists during the implementation could be equal in time. This matching would encourage similar uptake and fidelity of the interventions and reduces or avoids the Hawthorne effect [28, 29], which poses that attention alone can change behavior. Although matching the amount of attention to targets closely is preferable, investigators are often satisfied to only approximately match attention or may choose not to address it. An attention-matched design would enhance separation of intervention effects on the health outcome from possible effects of the implementation, although as noted above, that is not a requirement.

ii)

Implementation exposure when the control intervention consists of components that are not similar in function or form to those in the active intervention. Investigators could propose to compare the active implementation (Sa) and intervention (Pa) shown in Fig. 3 to a gym membership that provides exposure to a physical activity-rich environment and help (coaching) from gym personnel (Pc). An example implementation would use navigators to distribute free gym memberships. This difference in the intervention (staff delivering ACT in a clinic versus environmental change and coaching in a gym) requires differences in the core functions, form, and content of the implementation strategies. In this example, the investigators would be unable to assume the implementation strategies would have a similar impact on mediators of the health outcome; however, this assumption is not necessary if conclusions are drawn comparing the impact of the health outcomes on the two interventions as implemented.

iii)

Implementation exposures when the control intervention is a placebo/attention control. A control intervention can be hypothesized to cause little or no effect on the health outcome, but to provide equal attention to the target sample. A book club could provide such an attention control in our example. The control implementation exposure (Sx) in the control arm (SxPc) could be activities that employ central research staff to support staff local to the clinic in the formation of a book club for clinic patients with anxiety disorder.

iv)

Sx if the control intervention is usual care/no intervention. Changes in health-related outcomes in the absence of an intervention have a more useful interpretation than changes in the implementation outcomes when there are no implementation strategies because in the latter situation essentially zero change can be expected (as discussed above). When emphasis is on testing the health outcome, there might be no new intervention activities or implementation strategies introduced. If the control intervention is usual care, there will likely be a usual care implementation already in place, and activities would continue without change. Wait listing is a type of “no intervention” control.

4.

Closing thoughts on choosing the control arm for the intervention. It may be surprising that selection of the control exposure (Sx) in the control arm (SxPc) of the test of the intervention received so much attention here, but that selection does influence the interpretation of results and can be difficult. It is our hope that the list of options presented in this section will help investigators think through the best choice for their project. The examples shown were meant to facilitate understanding, not to limit investigators in their choice of core functions, forms and content in the implementation and intervention exposures.