STRENGTHS AND LIMITATIONS OF THIS STUDY

This study will provide the comprehensive evidence on existing prognostic prediction models for survival outcomes in patients with oropharyngeal squamous cell carcinoma (OPSCC).

The results will help us to analyse and assess the quality, risk of bias and clinical utility of existing prognostic prediction models for survival outcomes in patients with OPSCC.

The results of this review will provide insight that will assist in developing and validating prognostic prediction models for OPSCC in future studies.

A highly sensitive search strategy and robust quality assessment criteria (transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) will be used to appraise existing prognostic prediction modelling studies for OPSCC.

The main limitation of this study could be the potential heterogeneity among studies included in the analysis.

Introduction

Oropharyngeal squamous cell carcinoma (OPSCC) is one of the head and neck carcinomas, which originates in tissues of the oropharynx (the part of the throat at the back of the mouth, including the soft palate, the base of the tongue and the tonsils).1–3 OPSCC represents an increasingly prominent public health concern internationally. Although OPSCC only represents 0.9% of all cancers, its incidence has been rapidly growing worldwide in recent years, with an estimated 182 666 new cases in 2020.4–6 An increased incidence of OPSCC among men under 45 years of age has been reported recently.6–9 Moreover, the death rate of OPSCC is rising by 2% worldwide per year, compared with other head and neck carcinomas,10 with an estimated 86 742 new deaths in 2020.4 Of note, OPSCC has a special feature of epidemiologic trends in different settings worldwide. Over the past few decades, OPSCC diagnosis increased especially in developed countries, including the USA and Canada,6 11–13 while South-central Asia had the highest proportion of new OPSCC cases (35.1% of global incident cases).14 Across China, there has also been an obvious increase in OPSCC in the recent decade, especially for incidence and mortality of men and in rural areas, whereas the rates of females remained stable.15

Compared with other common type of head and neck carcinomas, OPSCC is likely to be advanced (ie, with neck metastases) at the time point of diagnosis and its primary treatment is more likely to be aggressive (such as radiation therapy and/or chemoradiation), which may have devastating effects on the survival of these patients.16–19 OPSCC is a heterogeneous condition with inter-related factors significantly modifying the absolute risk of survival at an individual level.

Human papillomavirus (HPV) is considered to be the most significant risk factor for OPSCC.20–22 HPV is a major carcinogen, which meets the epidemiological criteria for OPSCC causality.23 24 Up to 70% of newly diagnosed OPSCCs are HPV positive.25 In addition, the current identified risk factors include heavy smoking and alcohol consumption.26 However, it is worth noting that HPV-positive OPSCC patients are usually confronted with decades of significantly improved quality of life, compared with the HPV-negative OPSCC group of patients.19 HPV-positive OPSCC is associated with a 58% reduction in the risk of death compared with its counterpart.27

In contemporary real-world clinical practice, interventions (treatment and/or management) are implemented after diagnosis of OPSCC, without individualised risk assessment of the absolute risk of survival. Consequently, in case of immediate start of treatment after diagnosis of OPSCC, this mode of intervention fails to identify of the proportion of patients at high risk (with a lower probability to obtain a good response) who should have received new or more aggressive therapy regimens. Meanwhile, patients at low risk will not be spared from harm of unnecessary aggressive cancer treatment and significant financial burden of cancer management. Therefore, limitations of the one-size-fits-all mode of intervention and lack of risk-differentiated decision-making are evident. In this regard, it is imperative to develop a precise and applicable prognostic prediction model for calculating the absolute risk of survival for patients with OPSCC, based on considering any relevant risk factors related to survival and individual demographical characteristics. Accurate prediction of risk of survival would then guide risk-differentiated clinical decision-making at health services level, ultimately, facilitate more personalised management of OPSCC and positively enhance the quality of life of patients.

This systematic review will identify, screen and assess all published prognostic prediction modelling studies for survival outcomes in patients with OPSCC. We aim to answer the question: What prognostic prediction models have been developed and validated for application in patients with OPSCC to predict risk of survival and inform clinicians’ therapeutic decision-making regarding the management of OPSCC. The detailed objectives of this systematic review are: (1) to systematically identify existing prognostic prediction models for survival outcomes in patients with OPSCC; (2) to qualitatively describe characteristics of identified models; (3) to quantitatively compare their performance across different clinical settings and population from different regions in the world with meta-analysis where appropriate; (4) to rigorously assess the conduct and real-world clinical utility of these prognostic prediction modelling studies.

Methods

This systematic review protocol was registered on the PROSPERO international registry of systematic reviews on 27 February 2023. This protocol for the systematic review is reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols (PRISMA-P) guideline,28 Cochrane Prognosis Methods Group Protocol Template,29 transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) statement,30 the prediction model risk of bias assessment (PROBAST) tool31 and the corresponding CHARMS checklist (checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies)32 (see online supplemental tables S1,S2).

A systematic review of prognostic prediction modelling studies for survival outcomes in patients with OPSCC will be conducted to identify eligible studies published before March 2023. The review will be guided by the recommendations of The PROGnosis RESearch Strategy (PROGRESS) Partnership, which is an international, interdisciplinary collaboration that has published a framework to improve the standards of prognosis research to improve its translational impact. The framing of the review question is presented in table 1. Formal activities for this study are scheduled to commence in September 2023 and should conclude by June 2026. Data analysis and dissemination of results will be completed in this period.

Table 1

Framing of this systematic review with key items identified by the CHARMS checklist32

Patient and public involvement

This review will be conducted completely based on published data, so approval from an ethics committee or patient consent is not required. The results will be disseminated through a peer-reviewed publication.

Ethics and dissemination

This review will be conducted completely based on published data, so approval from an ethics committee or written consent is not required. The results will be disseminated through a peer-reviewed publication.

Eligibility criteria

Table 2 shows the review question in population, index, comparator, outcome, timing, setting and study type (PICOTS) format.33 Selection of studies will be based on the eligibility criteria framed with the PICOTS system, which is a modification of the established PICO system and designed for the specific requirements of systematic reviews of prediction models with additional consideration for timing and clinical setting.31

Table 2

Eligibility criteria for the systematic review framed with the PICOTS system33

Population

Studies reporting on prognostic prediction models proposed for survival outcomes in patients with OPSCC will be included into the systematic review. OPSCC could have been diagnosed according to criteria in each eligible study included in the review. In addition, this study will include both HPV-positive and HPV-negative OPSCC.

Intervention

Prognostic prediction modelling studies (with or without external validation) and external model validation studies (with or without model updating) will be considered for inclusion into the systematic review, if they were intended to inform clinicians’ therapeutic decision-making regarding the management of OPSCC.

Outcome

The included outcome endpoints related to OPSCC, defined as the outcomes of interest in the eligibility criteria, are aligned with those agreed by consensus of systematic reviews for treatment of OPSCC and draw on published search strategies for similar review questions for prognostic models of cancers.16 17 27 34–38

The primary outcome endpoint is overall survival (OS). We choose this endpoint because it has the greatest clinical relevance and is most important for patients diagnosed with OPSCC. Furthermore, OS is an objective endpoint not susceptible to bias of the outcome assessor. In addition, disease-related mortality will be considered if possible. The secondary outcome endpoints include progression-free survival (PFS) and disease-free survival (DFS). We choose these endpoints as patients with similar survival may nevertheless have different lengths of time without disease progression or symptoms, depending on both initial treatment after diagnosis and disease characteristics.

Outcome endpoints will be assessed in hierarchical fashion in the following order: OS (and/or disease-related mortality), PFS and DFS. The timing and effect measures for each outcome endpoint will be as defined according to each eligible study included in the review. In addition, we will not require studies to have a minimum follow-up duration for inclusion in this systematic review.

Timing

Each eligible study included in the review should report on prognostic prediction models for survival outcomes occurring at any time point after diagnosis of OPSCC.

Setting

Prognostic prediction models that are designed to be used by healthcare professionals in the clinical setting, at any time point after diagnosis of OPSCC, will be considered for inclusion in the review.

Type of studies and limits

Any study design including primary research (eg, randomised controlled trial, cohort study, case–control study) or secondary research (eg, systematic review) that reports on one or more statistical models, tools or scores with at least two predictors proposed to predict an individual’s risk of a future survival outcome (prognostic prediction modelling studies) will be considered for inclusion in the review. Prognostic prediction modelling studies can be either model development, model validation or a combination. Specifically, editorial comments or letters will be excluded from the review. Eligible studies included in the review will be limited to those conducted in humans by applying The Cochrane Group’s filter for Humans not Animals filter.39

Search methods for identification of studiesDatabases

The following electronic databases will be systematically searched to identify eligible studies from their inception to present: (1) Ovid MEDLINE(R) and Epub Ahead of Print, In-Process, In-Data-Review & Other Non-Indexed Citations and Daily on Ovid and Ovid MEDLINE(R); (2) Embase Classic+Embase on Ovid; (3) Web of Science; (4) the Cochrane Library and (5) China National Knowledge Infrastructure (CNKI).

Search strategy

A highly sensitive search strategy, based on the eligibility criteria for the systematic review and combining subject indexing terms (ie, MeSH) and free-text search terms, will be designed for MEDLINE Ovid. We aimed to avoid missing any valuable relevant predictive modelling studies for OPSCC. The search strategy, specifically, subject indexing terms will be translated appropriately for the other databases.

The draft search strategy will combine concepts related to prognostic prediction modelling studies, OPSCC and survival outcomes. The updated version of a validated filter for prediction modelling studies40 will be used. For OPSCC and survival outcomes related to OPSCC, a search strategy aligned with those agreed by consensus of peer-reviewed systematic reviews of treatments for OPSCC and drew on published search strategies for similar review questions for prognostic models of cancers will be used.16 17 27 34–38 The draft search strategy is provided in online supplemental table S3. The final search strategy will be iteratively refined.

The reference lists of included model development studies and relevant systematic reviews for further studies will be handsearched for additional potentially relevant citations. The included studies will be checked for error or fraud. We will not place any restrictions on language, publication year or publication status when searching the electronic databases. Any non-English studies identified will be translated and assessed for eligibility.

Data collection and analysisSelection process

Two independent reviewers will screen and assess the abstracts of each study identified by the final search strategy. Duplicate records will be excluded using a systematic, rigorous and reproducible method using a sequential combination of fields including author, year, title, journal and pages.41 Thereafter, if the information suggests that the study meets the eligibility criteria for the review (table 2) or there is any doubt against eligibility, full texts of the studies will be independently accessed for further assessment. Any conflict will be resolved through discussion with a senior advisor (HZ), where required.

Data extraction

Two independent reviewers will extract data from eligible studies included in the review, using a standardised electronic form developed with reference to the checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies (CHARMS).32

For each eligible study, we plan to seek information on objective, source of data, participants, survival outcome(s) to be predicted, candidate predictors, sample size, missing data, model development, model performance (discrimination, calibration, clinical utility, and measures of case-mix variation), results including final multivariable models and interpretation of presented models and model validation.32 Moreover, information on diagnostic criteria for OPSCC and treatment type after diagnosis will also be extracted. Missing data will be obtained from the study authors wherever possible, in addition, if insufficient information is obtained, the study will be excluded from the review. Any disagreement will be resolved through consultation with a senior advisor (HZ), when necessary.

Data management

Covidence systematic review software will be used to manage screened records throughout the review (Veritas Health Innovation, Melbourne, Australia. Available at: http://www.covidence.org). Eligible studies included in the review will be imported into Endnote reference manager software (V.20.4.1, Clarivate Analytics, Philadelphia, USA. Available at: https://endnote.com/).

Critical appraisal

The methodological quality (risk of bias) and relevance (applicability) to the review question (tables 1 and 2) of eligible studies included in the review will be systematically assessed using PROBAST.31 This tool is structured around four key domains (participants, predictors, outcome and analysis), of which each will be rated as high, low or unclear risk of bias.

Two independent reviewers will assess the risk of bias and applicability of each eligible study included in the review. Each study will be given a rating of high, low, or unclear risk for each of the four domains. Any disagreement will be resolved through discussion and consultation with a senior advisor (HZ) to reach a consensus, where required.

Qualitative data synthesis of prognostic prediction models

All extracted data on prognostic prediction models from included studies will be tabulated to facilitate comparison of survival outcomes to be predicted, predictors included in the final model and performance measures.32 Measures of uncertainty will be reported when published or approximated using published methods.33 The characteristics of models will be tabulated to show classification measures such as sensitivity, specificity, area under the receiver operating characteristic curve (AUROC),32 were reported. Relevant analyses and visualising will be performed using R software V.4.2.1 (R Core Team, Vienna, Austria, available at: https://www.R-project.org).

Quantitative analysis and comparison of the predictive performance of prognostic prediction models

Our quantitative analysis will depend on the data available, the final number of eligible prognostic prediction models included in the review, and the type of prognostic prediction modelling studies (ie, development or validation).

We will attempt a meta-analysis by type of prognostic prediction modelling studies, if included studies are sufficiently homogenous. Clinical homogeneity will be regarded as satisfied, if the review identifies: (1) multiple validation studies for a common prognostic prediction model are identified or (2) multiple development studies where the target population to whom the model applies and survival outcomes to be predicted are considered similar or the same.

Meta-analysis and investigation of heterogeneity

Where data permit, meta-analysis will be conducted with reference to the Meta-analysis of Observational Studies in Epidemiology group guidelines.42 Where meta-analysis is feasible, performance measures such as discrimination (eg, AUROC) and calibration (eg, calibration slope) will be pooled and analysed using a random effects model,39 which provide estimates of the average performance of predictive models across the selected modelling studies. The restricted maximum likelihood and Hartung-Knapp-Sidik-Jonkman methods will be used to estimate the between-study heterogeneity and 95% CIs for the average performance.33

Statistical or clinical homogeneity will be assessed using the test, where an value ＞ 50% indicates moderate to high heterogeneity, as specified in published literatures.39 43 The test is a statistical measure used in systematic reviews and meta-analyses to assess heterogeneity among studies included in the analysis. It quantifies the proportion of total variation in effect estimates that is due to heterogeneity rather than chance. It is expressed as a percentage and ranges from 0% to 100%. A higher value of suggests a greater degree of heterogeneity. Potential sources of heterogeneity will be investigated by undertaking a meta-regression analysis. The analysis will be carried out using R software V.4.2.1 (R Core Team, Vienna, Austria, available at: https://www.R-project.org).

Subgroup analysis

Where there are enough eligible studies included in the review, we planned to conduct subgroup analyses. Subgroup analyses will be undertaken according to the type of prognostic prediction modelling studies (ie, development or validation), target population to whom the model applies, diagnostic criteria for OPSCC, whether population was treated (yes/no), treatment type after diagnosis, the follow-up duration, survival outcomes to be predicted and study quality (risk of bias).

Sensitivity analysis

If meta-analysis would be performed, we would undertake sensitivity analyses to explore the influence on effect size for exclusion of studies at lower and higher risk of bias.33

Summary of findings

Reporting and presentation of findings will be guided by the PRISMA statement28 and relevant recommendations from the TRIPOD statement.30 The grading of recommendations, assessment, development and evaluation approach will be used to determine confidence in estimates.44 45

Discussion

This systematic review will identify, screen and assess all published prognostic prediction models for survival outcomes in patients with OPSCC. All eligible models included in the review will be systematically summarised and compared for their performance across different clinical settings and population from different regions in the world with meta-analysis if feasible.

A prognostic prediction model for survival outcomes in patients with OPSCC is designed to be used by healthcare professionals in the clinical setting to inform their therapeutic decision-making regarding the management of OPSCC, at any time point after diagnosis of OPSCC. Compared with other common types of head and neck carcinomas, OPSCC is likely to be advanced (ie, with neck metastases) at the time point of diagnosis and its primary treatment is more likely to be aggressive (such as radiation therapy and/or chemoradiation), which may have devastating effects on the survival of these patients.16–19 Survival outcomes affecting the quality of life of these patients are of utmost importance. Hence, accurate prediction of risk of survival would guide risk-differentiated clinical decision-making at health services level, ultimately, facilitate more personalised management of OPSCC and positively enhance the quality of life of patients. Consequently, in case of immediate start of treatment after diagnosis of OPSCC, identification of patients with a lower probability to obtain a good response will aid in making decisions regarding management, for instance, deciding new or more aggressive therapy regimens would be delivered to this proportion of patients at high risk. In contrast, in case of a watch-and-wait strategy, differences in estimated prognostic survival risks can affect patient management regarding surveillance and treatment.

Prognosis-related research in OPSCC has been seeking to predict risk of survival after diagnosis based on routinely collected data, with a view to directing treatment and/or management efforts in real-world clinical practice. This systematic review will make an important contribution to the understanding of risk of survival for patients diagnosed with OPSCC. Moreover, each eligible prognostic prediction models included will be compared head-to-head for their performance and clinical utility in the review. From this perspective, the review will comprehensively promote the consideration of risk-differentiated clinical management of OPSCC in real-world practice. Furthermore, in case that insufficient applicable models are identified, or screened models have poor performance, and/or high risk of bias, we will provide explicit rationale and detailed guidance for development, validation and/or updating for prognostic prediction models for OPSCC. In contrast, in case that high-performance models are identified, they will be valuable to helping clinicians and patients with OPSCC understand and consider estimated risk of survival in shared decision-making, objectively and systematically.

As such, this systematic review forms the foundations of future research programmes to develop, validate and assess a prognostic prediction model for OPSCC across the four themes of the PROGRESS prognosis research framework.46 We noted that researchers have been focusing efforts on developing new prognostic prediction models to date, however, disproportionate efforts have been put into improving and ultimately implementing existing models into real-world clinical practice, which have caused a huge waste of research resources. Therefore, we strongly recommend that, in the future, researchers could optimally use information from our review. If appropriate, seeking to validate and update existing prognostic prediction models would be a better choice.47

In conclusion, this systematic review will comprehensively consider contemporary best practice and evidence of prognostic prediction modelling studies for OPSCC. This work will support risk-differentiated clinical decision-making at health services level, ultimately, facilitate more personalised management of OPSCC and positively enhance the quality of life of patients.

Ethics statementsPatient consent for publication

Not applicable.

Acknowledgments

This study was supported by the Natural Science Foundation of China Excellent Young Scientists Fund [82022064] and Merck Investigator Studies Program [100073]. The funding parties did not have any role in the design of the study or in the explanation of the data