Research Information

The main clinical research assesses the blood concentration of ponatinib and treatment outcomes in patients with chronic phase chronic myelogenous leukemia (Clinical Research Registry Number: UMIN000035692). In this study, 17 participants at 13 sites in Japan were randomized. The primary outcome for the main clinical research is the blood concentration of ponatinib for 48 weeks in the major molecular response (MMR) achievement and the MMR nonachievement groups. Clinical data were collected using Electronic Data Capture (EDC), Viedoc. The main clinical research was chaired by Dr. Naoto Takahashi, Department of Hematology, Nephrology, and Rheumatology, Akita University Graduate School of Medicine; coordinated by the Clinical Research Innovation and Education Center, Tohoku University Hospital; and funded by Otsuka Pharmaceutical Co., Ltd. The Ethics Committee of Tohoku University Graduate School of Medicine in Miyagi, Japan approved our companion study (approval number 2022-1-877). Additional participant consent was not obtained because no personal identifiable information was used and no special risks or burdens were associated with participation in our companion study. The method used in our companion study is depicted in Fig. 1.

Figure 1

Methods Used in this Study.

Selection of Methods to Identify and Evaluate Risks and Plan Risk Reduction Activities

In addition to the systematic review of risk-based monitoring tools conducted by Caroline et al. [23], a collection of risk-based monitoring tools (https://ecrin.org/tools/risk-based-monitoring-toolbox) in ECRIN was employed to collect publicly available methods of risk identification and evaluation. In addition, the Pubmed database was searched for articles on newly reported methods since the study conducted by Caroline et al. [23]. We summarized the obtained information on risk identification and evaluation methods, including the intended use of the results of risk identification and evaluation, the type of risk identification, the number of risk categories that are most detailed in each method, and the type of risk evaluation, as shown in Table 1. Two typical types of risk identification are commonly used in RBA: identifying risks using abstract categories and selecting applicable risks from a list of concrete risks. In this study, we selected two methodologies that contradict each other in terms of risk identification type and the number of risk categories, based on the information summarized in Table 1 because these two types are widely used in clinical trials for risk identification. We believed that these contradicting methods would yield obviously different results. One of these methods, the risk assessment form (RAF) [17], had the lowest number of risk categories that are most detailed; in contrast, the risk assessment tool (RAT) [16] had the highest number of risk categories that are most detailed. The number of the most detailed categories in RAF and RAT is 12 and 164, respectively. The main categories of RAF are “Patient Hazards/Research Staff Hazards (Rights and Safety)”, “Study Hazards (Completion and Reliability)” and “Organizational Hazards”. RAF was used to identify risks in 12 abstract categories. The main categories of RAT are “Study organisation and governance”, “Training”, “Trial subjects rights and safety”, “Data”, “Protocol procedures”, “Study Drug/IMP”, “Safety management”, “Impact” and “Other”. RAT was used to select relevant risks from a list of 164 concrete risks. Among the categories in RAF, “Liability” and “Intellectual Property” in “Organizational Hazards” did not have any equivalents in the RAT categories. As RAF scores “Impact” and “Likelihood” on a five-point scale and RAT scores “Severity” and”Probability” on a three-point scale, the two risk evaluation methods are similar, although their number of evaluation points is different.

Table 1 Summary of the Risk Identification and Evaluation Methods.Identifying and Evaluating Risks and Planning Risk Reduction Activities

We used the RAF and RAT methods to identify and evaluate risk and plan risk reduction activities from October 3, 2018 to December 5, 2018. Risk identification and evaluation and planning of risk reduction activities were first performed with RAF and then with RAT. The activities were performed by four data managers (DMs), two clinical research associates (CRAs), and two statisticians from the Clinical Research Data Center of the Clinical Research Innovation and Education Center at Tohoku University Hospital. An RBA specialist with experience in identifying and evaluating risks and planning risk reduction activities in more than 10 clinical trials was the facilitator. The facilitator was required to participate in the study at the time of implementation, but the participation of other members was voluntary, with at least four members participating. One facilitator and one DM had experience in identifying and evaluating risks and planning risk reduction activities in clinical research. For these activities, we did not differentiate between RAF and RAT participants. The risk reduction activities involved communicating precautions and conducting training in processes and procedures to site staffs and setting alert texts to appear on the EDC screen. The alert texts were used to prevent mistakes or protocol deviations, not as queries to correct data entry errors.

Deciding on the Sites to Implement Risk Reduction Activities

The sites participating in the main clinical research were randomly assigned to the RAF or RAT groups to implement planned risk reduction activities for each group. Randomization and stratification were performed by the CRAs (n = 3) to ensure approximate balance between RAF and RAT (1:1) and avoid the biases caused by CRA. The RAF and RAT groups were equally likely to appear. The total number of participating sites was 13, and the number of CRAs was 3; thus, the CRAs oversaw five, five, and three sites. The RANUNI function of the SAS System Release 9.4 software (SAS Institute Japan Ltd., Tokyo, Japan) was used for random number generation. After the randomization, six sites were assigned to the RAF group and seven to the RAT group. However, after the assignment, the number of participating sites in the main clinical research decreased to five in the RAF group and seven in the RAT group (Table 2).

Table 2 Characteristics by Randomization Group.Implementation of Risk Reduction Activities

We prepared separate Microsoft PowerPoint files containing content explaining the risk reduction activities—which involved communicating precautions and training in processes and procedures—for the RAF and RAT groups. An audio version of the content was prepared, and two separate videos were created for the RAF and RAT groups. The risk reduction activities were performed by this video with the investigators and the site staff of every site on the CRA’s first visit. In addition, the slides were distributed to the principal investigator and site staff at the time of the CRA’s visit. As part of the risk reduction activities, each alert text appearing on the EDC screen was described by the RAF and RAT groups. All investigators were trained with regard to the content of the risk reduction activities during the investigator meeting prior to the initiation of the main clinical research. In the risk reduction activities, precautions and training were additionally conveyed in video format and an alert text was set to appear on the EDC screen in the RAF group for the risk of missing of ponatinib blood concentration measurement. In the RAT group, an additional risk reduction activity was implemented, in which the alert text was set to appear on the EDC screen for the risk of inadequate and lack of SAE registration and reporting. The deadline for data entry and the need for entering protocol deviation information into EDC are described in the case report form (CRF) completion guideline. To ensure the common risk reductions in the main clinical research, the deadline for data entry and the need for entering protocol deviation information were shared with both groups. In the RAF group, the additional risk reduction activities were implemented, in which the precautions and training of the deadline for data entry were conveyed in video format. In the RAT group, an additional risk reduction activity was implemented, in which the precautions and training of entering protocol deviation information were conveyed in video format.

Data Collection of Errors and Protocol Deviations

The implementers, implementation date, start time, and end time of the risk identification, risk evaluation, and planning of the risk reduction activities were recorded and collected.

On-site, off-site, and central monitoring were conducted in this study. On-site monitoring was conducted using partial source data verification (SDV) and source data review (SDR), targeting informed consent, eligibility of the study participants, and source documents and data up to the 4-week visit. In case of problems in on-site monitoring up to the 4-week visit, on-site monitoring was conducted after 24 weeks and after discontinuation or termination. If there were no problems for up to 4 weeks, off-site monitoring was conducted after the 4-week visit. On-site or off-site monitoring was conducted in cases of serious adverse events (SAEs).

The CRAs queried data that required correction of the electronic CRF (eCRF) and the source data during on-site monitoring. The data queried by the CRAs were used as error data. DMs also created queries after the data check. The data queried by DMs were used as error data only when the data were updated after the query date. The protocol deviation information was obtained by extracting the protocol deviation dataset from the EDC.

Analysis of Errors and Protocol Deviations

We calculated point estimates and 95% confidence intervals (CI) for means and differences of mean between the RAF and RAT groups for the following items:

Amount of error data per subject visit

Number of protocol deviations per subject visit

Amount of error data per person-time required to implement risk identification and evaluation and plan risk reduction activities

Number of protocol deviations per person-time required to implement risk identification and evaluation and plan risk reduction activities

The amount of error data was also compared between the data risk categories, which were determined using the impact of data errors, classified as high, medium, or low. The high-risk category included critical data related to the protection of human participants, eligibility criteria, informed consent, and primary endpoints. The medium-risk category included important data not related to the protection of human participants, eligibility criteria, informed consent, and primary endpoints. The low-risk category included all other data not covered by the previous two categories. Because all data were classified into high or medium risk categories, none of the data were classified in the low-risk category.