In Japan, PIs are revised due to accumulated domestic, both domestic and overseas, overseas ADR cases, up-to-date company core data sheet, literature information, epidemiological information, request from academic societies, etc. Among them, literature information, epidemiological information and request from academic societies accounted for only 2.1% (13/618) of the reasons for adding CSARs to the PI during the analysis period. These impacts on the predictive model are expected to be minimal. Therefore, we focused on PI revisions between August 2011 and March 2020, which occurred due to the accumulation of domestic cases, the most common reason for adding CSARs to the PI in Japan, as the prediction target. The features used for the construction of prediction models were created based on the information recorded in the Japanese Adverse Drug Event Report (JADER) database.

The JADER database contains domestic ADR cases. The JADER contains four Table (1) DEMO (sex, age, weight, and other patient characteristics), (2) DRUG (drug characteristics, including name and other properties), (3) REAC (types of ADRs and their outcomes), and (4) HIST (medical history). The DRUG table includes information on suspected drugs, drug interactions, and concomitant medications. As the JADER contains up to 4 months old data, we used the data released in July 2020 for our model construction to track the addition of CSAR information to the PIs during March 2020. The JADER contains cases where the onset of ADRs were recorded prior to the initial administration of the suspected drug, cases where the same ADR was reported multiple times for the same patient, and records of ADRs for over-the-counter-drugs. These data were excluded from the study population because they were unsuitable for the purpose of predicting the addition of CSARs to PIs. Herein, each tabulation was performed only for the suspected drugs.

In this study, 75 different CSARs were used as predictors for which there was an exact match between the ADR name on the CSAR section in PI and preferred terms (PTs) from the Medical Dictionary for Regulatory Activities (MedDRA). PI revisions were not announced until weeks or months after the risk investigation started. Therefore, we used data from approximately two quarters before the PI revision, when CSAR information was added as positive cases to fill the gap. Negative cases included drug–ADR pairs that met the following criteria: (1) at least one of the 75 target CSARs was reported during the analysis period, and (2) the CSAR section in the PI did not list a target ADR with the same name as that listed on March 2020. The data-extraction scheme is illustrated in Fig. 1.

Data extraction scheme for positive and negative cases. The JADER database extracted 293 positive and 22,399 negative drug–ADR pairs. JADER Japanese Adverse Drug Event Report, ADR adverse drug reaction, CSAR clinically significant adverse reaction, PI package insert, PT preferred term, MedDRA medical dictionary for regulatory activities.

Each ADR was tabulated using PTs from the MedDRA version 23.0. For some ADRs, the PTs were grouped before each tabulation. PTs included in the target disease were calculated by referring to both the cumulative number of reported ADRs for each drug in the positive group and the Standardized MedDRA Queries. As symptoms may also be associated with other diseases, only the minimum necessary symptoms were included. For example, if there was a PI revision for “hyperkalemia,” cases of both hyperkalemia and increased blood potassium levels would be added to the hyperkalemia tally, and the characteristics would be tabulated. The grouping of each ADR is shown in Table 1.

We created 34 features based on information that strengthens the signals written in CIOMS Working Group VIII and GVP Module VIII, information suggesting a causal relationship between the drug and the adverse event, and information that is focused on in pharmacovigilance activities at MAHs, including cumulative report counts that indicate the absolute amount of reporting, average number of missing values per case that indicate insufficient information about drugs or ADRs, number of re-administrations and recurrences suggesting the causality between drugs and ADRs, average and median number of days from drug administration to the onset of ADRs, and various disproportional reporting indicators. The disproportionality signal and the relative value compared to those of the other drug–ADR pairs were calculated as follows:

$$}\; = \;\frac \right)}} \right)}} \; = \; \frac}$$

$$} = \frac}/\left( } + }} \right)}}}/\left( } + }} \right)}} = \frac \right)}} \right)}}$$

$$}\;}\;}\;}\;}' \;}\;}\; = \;\frac \right)(\left| \right| - \left( \right)/2)^ }} \right)\left( \right)\left( \right)\left( \right)}}$$

here, a, b, c, and d are defined using the 2 × 2 table as follows:

a: the number of ADR cases that occurred after using the suspected drugs, b: the number of ADR cases that occurred after using all other drugs, c: the number of all other ADR cases that occurred after using the suspected drugs, and d: the number of all other ADR cases that occurred after using all other drugs. Table 2 lists the features generated based on the JADER data.

Before data aggregation, we excluded cases (1) included in JADER for which the ADR onset date preceded the first dose of the suspected drug; (2) with duplicate records with matching case IDs, ADR names, and onset dates; and (3) where the suspected drug was an over-the-counter drug (Fig. 1).

There were missing data for the date of drug administration and ADR occurrence, which resulted in 4,920 (21.7%) missing values in the “mean number of days from administration to occurrence” and “median number of days from administration to occurrence.” The “median number of days from administration to the onset of the same ADR for other prescription drugs” was used as a substitute in some models to address this issue. There were no missing data points for the other features. Each feature was transformed using standardization or quantile transformation, depending on the model used.

To predict the addition of CSAR information to PIs, we developed classification models using eXtreme gradient boosting (XGBoost) [14], light gradient boosting machine (LightGBM) [15], and support vector machine (SVM) with a radial basis function kernel (RBF–SVM) [16,17,18,19,20,21].

As some features may not contribute to the prediction, we created a model trained with all the features and another trained with selected features using exhaustive feature selection (EFS). This EFS algorithm is a wrapper approach for brute-force evaluation of feature subsets; the best subset is selected by optimizing a given performance metric given an arbitrary regressor or classifier [22]. The dataset was divided 7:3 into training and test datasets. Hyperparameter optimization was performed on the model with the highest average training score using Optuna optimization with the hyperband method, and various hyperparameter combinations were tested [23]. The Matthews correlation coefficient (MCC) was used as the model evaluation metric to measure the accuracy of a binary classification model, which is considered a balanced measure that can be used when class sizes vary. The MCC ranged from − 1 to + 1, with + 1 representing a perfect prediction and 0 representing an average random prediction. As adding CSAR information to the PI is rare and the dataset is unbalanced, we evaluated the model’s performance using MCC.

$$} = \frac} \times }} \right) - \left( } \times }} \right)}}} + }} \right) \times \left( } + }} \right) \times \left( } + }} \right) \times \left( } + }} \right)} }}$$

TP, true positive; TN, true negative; FP, false positive; FN, false negative.

All modeling and calculations were performed using Python version 3.9.3.