The multi-component COVID-19 testing implementation strategy bundle and outcome metrics will be operationalized and refined using a participatory approach engaging diverse community, clinic, and research partners through a CSAB, brainwriting premortem interviews [16] and the PRISM Fit Assessment (Aim 1). The implementation strategy bundle to optimize COVID-19 testing among underserved communities will be tested using a roll-out implementation optimization (ROIO) study design [17] (Aim 2). See Fig. 1 for the study design and timeline.

CO-CREATE-Ex participatory (Aim 1) and ROIO (Aim 2) design and timeline

The primary outcome is defined as offering the most clinically useful COVID-19 test and returning results within an optimal timeframe to enable appropriate treatment and decrease transmission, in the most acceptable, timely, and feasible manner. Secondary outcomes include increased: reach to priority communities; adoption and meaningful, sustained implementation of the testing strategy bundle; optimal COVID-19 testing experiences; and metrics that matter most to clinical and community partners such as rates of preventive care engagement. The ROIO study design allows for iterative refinement of the implementation strategies after each clinic roll-out [17]. Longitudinal data mixed-methods data collection and analysis with multiple partners and community members will inform necessary refinements and assessment of the impact of the strategy bundle on key outcomes over time.

This study will be conducted in partnership with a high-performing FQHC that serves low-income and uninsured San Diegans and has a long-standing relationship with the lead university. For this study, we will focus on clinics in Central and South San Diego. As shown in Table 1, the 4 target clinics serve primarily Latino and Black patients, and a large proportion report Spanish as their primary language. Preventive healthcare rates for commonly recommended adult preventive screenings and immunizations are low, particularly for blood pressure (M = 60.6% across clinics) and flu immunization (M = 20.4% across clinics) [18].

Recruitment of community members will occur at the FQHC clinics through collaborative efforts between promotores, and on-site research staff. Recruitment and outreach will occur on-site for patients coming to the clinic for appointments, through outreach at local community gathering locations, phone calls, and/or emails to existing FQHC patients, outreach by CSAB members utilizing their existing networks, social media posts, flyers, and/or billboards. It is anticipated that participants will represent the ethnic, racial, and sex distributions of the registered patients at each participating clinic. Promotores will be hired by the partnering FQHC and speak the languages that are most common in the local communities and clinics (Spanish and English).

Building on the phase 1 CSAB, a slightly modified membership for the current study’s CSAB will facilitate representation from communities around the 4 participating clinics for the Aim 2 implementation and evaluation activities. In addition to 4 community partners, the CSAB will include 2–3 clinical staff, 1–2 public health researchers, 1–2 policy partners, and a County Public Health Department ambassador. CSAB community partners will be selected based on their role as community weavers—that is, individuals with lived experience as a member of an underserved community who are cultural brokers between communities, health systems, and researchers to co-create community-driven public health solutions.

To refine and operationalize the implementation strategy bundle and measures of implementation and sustainment success, several participatory methods will be used. During CSAB meetings led by the Global ARC, initial strategies, measures, and data collection plans will be presented. The research team will facilitate group brainwriting premortem interviews [16] with the CSAB to gather information about what specific components of the implementation strategy bundle and measurement might be prone to fail and to co-create solutions to prevent these failures.

Aim 1 data collection will focus on evaluating the quality, extent, and content of partner engagement through the participatory CSAB activities. An established multimethod ethnographic approach [19] using a refined documentation form will be completed by trained research interns at each CSAB meeting, and a partner engagement survey based on validated measure from Goodman and colleagues [20] will be completed by all CSAB attendees after each session. Data from the partner engagement survey will be analyzed after each CSAB meeting to inform post-meeting debriefings with the Global ARC and research team. Aim 1 data (brainwriting premortem interviews, ethnographic documentation forms, engagement surveys) will be analyzed using a rapid qualitative analytic approach and simple descriptive statistics. Summarized data from these sources will be triangulate using a joint display analysis [21, 22]. This methodology has been described in more detail in Rabin et al. [19]. In addition, the research team will document adaptations made to the implementation strategies throughout the Aim 1 refinement process (and throughout Aim 2) using methods developed by Rabin et al. and that was used in the phase 1 study [19, 23].

An innovative ROIO study design [17] will be used to implement and evaluate the multilevel, and multicomponent implementation strategy bundle (Fig. 1). Four cycles of preparation, initial roll-out (3 months), and sustainment will be launched over 21 months. Refinements to the implementation strategy bundle and processes will be made prior to each new roll-out based on information collected from the prior roll-out (or the CSAB preparation for Clinic 1) and a PRISM Fit Assessment. The PRISM Fit Assessment will explore how well the proposed strategy bundle aligns with the multi-level context of Clinic 1, how likely the proposed bundle will lead to increased rapid COVID-19 testing, reach of community members, and adoption and sustained implementation. The PRISM Fit Assessment will be repeated prior to each subsequent clinic roll-out. Each PRISM Fit assessment will involve clinic staff, promotores, and clinic operational leaders completing a 21-item survey developed around the key PRISM context domains and related RE-AIM outcomes independently followed by a group review of the summary results to discuss any misalignments and additional needs for refinement in the context of the specific clinic. This process will be repeated for each clinic prior to roll-out and at the end of the study period.

Preparation phases last 3 months and include data collection from patients/community members, clinic providers/staff, and promotores, medical assistant, and onsite testing staff, rapid analysis of this information and triangulation with other testing information, presentation to and refinement with the CSAB, and finalization based on PRISM Fit Assessment data. During the initial roll-out phase (3 months), the full-intensity implementation strategy bundle is provided at the clinic. After 3 months, clinics move into continued services with vending machines and promotores, available on an as-needed basis. Walk-up testing will also decrease in intensity moving to a setup with one part-time staff operating the walk-up testing.

Quantitative and qualitative data will be collected using multiple databases and methods. Data collection methods are summarized in Table 2 with the data source name, type of data collected, origin of data and timing for data collection, and associated PRISM domains assessed. Data from the PRISM Fit Assessment will be summarized as descriptive statistics and displayed in the form of bar graphs to inform the group action planning discussion. Key implementation and clinical and public health effectiveness outcomes are summarized in Table 3. The timing for each data collection method is also noted in Fig. 1. Participants will be required to provide contact information and minimum symptom and exposure data prior to obtaining rapid tests and the type of test and timing of testing will be captured for both the walk-up onsite testing and vending machine testing along with basic demographics. Participants will also be invited to provide rapid test results and receive personalized health counseling based on their test result.

Qualitative interviews with patients/community members and clinic providers/staff will be conducted after each initial roll-out period (after 3 months of implementation) to understand their experiences with the implementation strategy bundle and identify potential areas for improvement. A total of 60 interviews with patients/community members evenly recruited from each of the 4 clinics will be conducted. Patients/community members will be identified for interviews based on their interest expressed when initially obtaining tests as part of the walk-in onsite clinic or vending machine. An additional set of interviews will be conducted with 12 providers/staff (3/clinic) identified based on their involvement in supporting the implementation or having exposure to the implementation strategy bundle.

Quantitative data from study databases will be summarized using simple descriptive statistics including frequencies, measures of central tendency, and variability along with data visualization methods, such as frequency tables, bar charts, line graphs, and scatter plots to understand patterns and characteristics of testing uptake, exposures, and symptoms and to guide adaptive implementation. Missingness in data will be examined via sensitivity analysis and imputation methods will be applied when appropriate. Parametric and non-parametric tests will be employed to compare the survey responses between the clinics while controlling for a variety of demographic and socioeconomic variables, including the FQHC clinic cluster. Data collected at the point of test distribution (collected by study coordinators, promotores, and vending machines) will be used to evaluate implementation strategy effectiveness on testing uptake. Hierarchical multilevel models will be used to ensure the inclusion of random effects, variation between clusters, variation between times within clusters, and the fixed effect of time which will be estimated independently of treatment effect so that systematic change over time will not be mistaken for effect of treatment. This is key, based on significant variations in testing uptake observed during previous COVID-19 variant waves. To account for potential within-clinic correlation, data will be reduced for each cluster to a single observation, and then subject to standard two-sample analyses. Secondary analyses will be conducted to determine the appropriateness of the test type distribution within the context of symptom onset and exposure time frame.

Rapid matrix analysis of patient/community member and provider/staff interviews and periodic reflections will be conducted to identify key concerns and opportunities for refinements. This information, along with data about the implementation of the strategy bundle and testing results, will be summarized and presented to the CSAB (two sessions after Clinic 1 roll-out and one session after each consecutive clinic roll-out). A list of possible refinements will be identified based on input from the CSAB and final decisions about refinements will be made by the community-clinical-research team. Any changes made to the strategy bundle and procedures will be documented in the Adaptation documentation database. After refinements to the multi-component strategy bundle have been completed, the PRISM Fit Assessment will be conducted with the next clinic to allow for clinic-specific further adjustments. A matrix approach will be used to triangulate data from quantitative and qualitative sources using the key domains of PRISM. A joint display will also be produced to support the integration of data sources [21, 22].

The number of participating clusters/clinics and target clinic participant enrollment for the ROIO study design was determined using cluster randomized trial sample size estimation tools. Per the ROIO study design, each clinic cluster will begin in the control group and then cross over into the intervention group at different time points and then continue with the intervention until the end of the study (see Fig. 2). Although it is anticipated that the implementation strategies as outlined by CSAB will undergo some adjustments during the implementation process, the target cluster/clinic sample size estimates are based on a stepped wedge design using serial cross-sectional evaluations of electronic medical records and test kits distributed. It is estimated that the study will have the capacity to distribute 52,000 FDA-authorized rapid antigen tests provided by the San Diego County Health and Human Services and the California Department of Public Health. The primary outcome is the increased distribution of RATs within the communities near each of the 4 clinics. This will be measured using a proxy outcome defined as the proportion of clinic patients visits to the number of RAT kits distributed per month at each participating clinic. Currently, RAT kit distribution among target clinic patients ranges from 7 to 13% (M = 9%) with approximately 430 kits being distributed per month between all four clinics. The aim is to increase distribution of test kits to 40% of clinic patient encounters per month, which would result in the distribution of 1822 kits per month during the final treatment sequence. Based on data collected from the phase 1 study, approximately 40% of testing uptake occurred among clinic patients and the remaining 60% from community members even when the intervention was clinic-based. It is anticipated that there will be a similar community uptake during this study. Based on the sequential rollout outlined in the Fig. 2 with rollout for Clinic 1 at month 4, Clinic 2 at month 8, Clinic 3 at month 12, and Clinic 4 at month 16, over the course of the entire study it is anticipated that a total distribution of 20,503 tests, assuming average distribution per clinic. Estimating that this will equal approximately 40% of total increase and the remaining uptake to occur among community members, a total uptake of 58,091 test kits is estimated over the course of the study. Using the Shiny CRT Calculator [24], it was determined that a minimum cluster sample size of 285 across 4 clinics will provide sufficient power (beta > 0.8) to determine an intervention proportion increase of 0.05 to 0.25 assuming the following: stepped-wedge design, cross-sectional sampling structure at each of 4 sequences (approximately every 3 months in the study), 0.5 coefficient of variation in cluster sizes (based on average number of patient encounters per month per clinic), intra-cluster correlation (ICC) of 0.02, for our binary outcome (test or no-test). It was also assumed that there will be an exchangeable cluster correlation structure as SARS-CoV-2 testing demand has been impacted significantly by changes in variant severity and transmissibility.

Design matrix structure: one cluster per treatment sequence