2.1 EQUATOR Guidelines for Developing the Checklist

Our methodology for the development of the checklist has been adapted from the EQUATOR Network guidelines for developing reporting checklists [11], such as identifying the need for a checklist via systematic reviews, and around our and others’ recent work (Sections 1 and 2 of the EQUATOR Network guidelines) [3, 8, 12]. The reporting checklist was then developed following the EQUATOR toolkit, including generating a list of items and conducting a series of meetings (Section 3). We have however provided a single paper rather than follow the process recommended by EQUATOR (Section 4), which suggests a short explanatory paper alongside a longer ‘Explanation and Elaboration document’. Dissemination methods as suggested by the EQUATOR network guidelines are outlined in the discussion section.

2.2 Developing a Conceptual Framework to Provide a Foundation for the Checklist

A conceptual framework for the checklist was developed to ensure its relevance for reporting values for child HRQoL, whether the values are for individual health states (e.g., described via vignettes) or value sets, such as reporting values for all health states described by a HRQoL instrument. Given the differences between these study types, a modular approach was developed to allow flexibility for application to different study types. The modular approach also allowed us to differentiate between checklist items specific to valuation of child HRQoL and those that are important to include but are also common to reporting of adult HRQoL, thereby providing a standalone comprehensive checklist for children. An initial conceptual framework was developed by the authors to identify relevant modules, informed by existing checklists for adult HRQoL values [12] and reviews of methods for valuing child HRQoL [3, 8]. This was refined through checklist item development and testing, using an iterative process (expanded on below).

2.3 The Conceptual Model

The conceptual model for the RETRIEVE (REporting invenToRy chIld hEalth ValuEs) checklists is shown in Fig. 1. The checklists are structured using five ‘top level’ headline groupings (modules) of items. Four of the modules contain items relating to key aspects of the methods used to obtain child HRQoL values (A–D), with the fifth (E) comprising checklist items relating to the characteristics of the values themselves. The modules are not necessarily hierarchical, as decisions relevant to some modules are made simultaneously rather than sequentially and are often iterative. Figure 1 is therefore non-hierarchical. We note that there are likely to be interactions between methods decisions in each module, such as between population and anchoring or method and perspective.

Fig. 1

A conceptual framework for the RETRIEVE modular checklists for reporting values for child HRQoL. Note: the modules are intended to be non-hierarchical. HRQoL health-related quality of life

Modules A1–A3 are specific to considerations relating to child HRQoL values. The items they contain are not derived from any of the existing checklists for adult HRQoL values. Modules B2 and B3 are alternative modules that users select depending on whether the values they are considering are value sets (B1) or values for specific states or vignettes (B2). Modules A4 and C contain general methods and sample considerations. These are not necessarily specific to values for childhood HRQoL but are an important part of what users of values would need to check and developers to report. Module D relates to considerations relevant to modelling value sets for an HRQoL descriptive system, so are further relevant considerations to B1 (value sets for patient-reported outcome measures) but not B2 (direct valuation of disease-specific states or vignettes).

Checklists developed for adult HRQoL values have tended to focus on reporting the methods used to produce a given set of values, or on the clarity of reporting the final value-set model (i.e., like checklist Modules A–D described above). We considered it important that our checklist included a module focusing on the characteristics of the values, to ensure users are aware of these, and the relevant differences in values when choosing between instruments and value-sets. Including this module would help decision makers be aware of the potential implications of such differences when interpreting cost-effectiveness evidence based on them, and to encourage more complete reporting of these value characteristics by study teams (Module E).

2.4 Establishing Potential Items for Each Module

A review of items for reporting values for adult HRQoL [12] was used to identify items common to both adult and child HRQoL. Two sets of checklists were included in the study by Zoratti et al., i.e. those intended primarily for use in economic evaluation and those primarily intended for use for health utility studies (see Tables 1–6 and 7–12, respectively, in the article by Zoratti et al. [12]. Items from the latter were considered for our checklist, with potential items also identified from Table 7 from Brazier et al. [13], Table 8 from Stalmeier et al. [14], Table 10 from CREATE [9], Table 11 from Nerich et al. [15] and Table 12 from SpRUCE [10]. We did not include Table 9—MAPS [16], as that checklist is relevant to studies mapping across instruments and thus outside the scope of our checklist. Items from the included checklists provided a pool of potential items. These items were grouped by the modules in the conceptual framework by two members of the team (CB and RR) and then further independently reviewed (EL and ND).

We supplemented this pool of potential items with additional items specific to valuation of child HRQoL. The latter items were generated based on (1) methods issues relating to valuation of child HRQoL as identified by Rowen et al. [7], and (2) information from two systematic reviews [3, 8] on aspects of methods specific to valuation of child HRQoL and what was viewed as missing or unclear from the papers reporting values for child HRQoL that were included in those reviews. Combined, this process yielded a list of candidate items under each module. The original list of items, and subsequent versions created through the review process described in the following section, are available from the authors on request.

2.5 Creating an Initial List of Items for Each Module (Long Version)

A series of five meetings were held with a subset of the study team (CB, MH, ND, EL, RV, RR), where items in each module were each considered, with the objective of identifying redundancy or overlap between modules and to check for relevance. Meetings were structured, with an agenda circulated to the team by the first author (CB) prior to the meeting. Decision making was through consensus. Where gaps were identified, new items were created and/or wording clarified. Changes arose most often in the items specific to child HRQoL rather than those also applicable to adults. This collaborative and iterative process led to the creation of an initial draft checklist of 147 items grouped into five modules.

The process of eliminating redundant items and checking relevance yielded a first draft that was considered potentially usable. During this process, the conceptual model was reviewed to ensure the checklist items were grouped appropriately. The first draft of the checklist items was then distributed to the entire authorship team who were invited to comment. The commentary was compiled and the checklist items were edited accordingly (MH, CB).

2.6 Reducing Items for the Short Version

To produce the short version, all authors were asked to review the proposed items using a numbering system (1 = include, 2 = maybe include, 3 = do not include), providing specific comment on the items and to recommend revised or additional items for inclusion (if any). All responses were coded to facilitate refinement of the checklist. A first version of the short version contained 18 items, with a further 15 items as alternatives containing different wording. This was revised to 14 final items, with the format modelled on the CHEERS checklist [17], where, instead of questions, users are asked to indicate where the relevant information is located in the manuscript by page number.

2.7 Testing the Checklists: An Application to Four Studies of Child Health-Related Quality of Life Values

The checklists were evaluated using a sample of studies that report child HRQoL values. We selected four studies published between 2010 and 2021 that had been included in our earlier systematic review [8]. These papers were selected to check that the module approach worked for both value sets and vignettes, were spaced over a range of years, and featured value sets from the two most widely used child HRQoL instruments. The two papers on value sets were on the EQ-5D-Y-3L [18] and the CHU9D [19]. Two papers used vignettes [20, 21]. In each case, two members of the authorship team independently used the checklists to review and summarise the study (CB, MH, RR, KD). These reviews were compared and reported to the wider study team for discussion. Any need for refinement of the checklists was identified and implemented (MH, CB) via an iterative process.

2.8 Expert Review of the Checklists

The authors invited input from senior international health economic researchers who are part of the wider QUOKKA and TORCH project teams (‘Associate Investigators’) using an online survey. These researchers were from Canada, the UK, Australia, Spain and Singapore. Participants were asked to indicate whether items were relevant, redundant, or required wording changes. Information from the reviews was compiled and a final workshop was held (CB, MH, ND, EL) to review and address survey responses and inclusion of new items. We received six expert reviews. The reviewers commented chiefly on wording and recommended possible extra questions. The review comments were incorporated (CB, MH) and decisions on any extra question suggestions were workshopped (CB, MH, ND, EL). The final short and long versions of the checklist were then completed. After this final review, we updated the examples as described in Section 2.5.

2.9 Long and Short Versions of RETRIEVE

The resulting RETRIEVE checklists contain modules aimed at reporting methods (A–D) and the characteristics of values (E). The long version of RETRIEVE (Table 1) is populated with a total of 83 items (noting that because of the modular structure, not all items are relevant to all valuation studies) in question form with specified or open-ended response format. The short version of RETRIEVE (Table 2) has 14 items where the user notes where in the paper the information is contained, similar to the CHEERS checklist [17]. Electronic supplementary material (ESM) Table S1 contains a formatted version of the long RETRIEVE; ESM Table S2 contains examples of the use of the long and short RETRIEVE checklists; and ESM Table S3 contains a table of descriptive comments for each included item in the long RETRIEVE version. We also include editable excel versions of both versions in the ESM.

Table 1 RETRIEVE long checklist (please note that this checklist is modular and not all sections/questions will apply to all papers)Table 2 RETRIEVE short checklist (please note that this checklist is modular and not all sections/questions will apply to all papers)2.10 Differential Use of the Long and Short Versions of RETRIEVE

We considered the different needs of two broad sets of potential users of the checklist—decision makers and researchers. A longer list of items in each module was considered relevant for researchers undertaking and reporting on child valuation studies, to improve completeness of reporting. This list has more extensive descriptions of the aspects being identified and goes into more detail regarding valuation in the last module. A more concise version of the checklist was considered to be more appropriate for decision makers or other users of values wishing to assess, compare and choose between values for child HRQoL. This checklist is presented as statements to check off, similar to that used in the CHEERS checklist [17].