CDSD 1.0 (Abstract 1, online resource)Qualitative studies for content validation

Development of the CDSD was initiated in 2010 with concept elicitation interviews conducted with participants with biopsy-confirmed CeD (N = 21) and input from clinical experts (N = 5; four US gastroenterologists and one US registered dietitian specializing in CeD); Fig. 1. Concept saturation for the common and less-frequently reported symptom groups was reached by the sixth and twentieth interviews, respectively; see Online Resource Abstract 1 for full details. Eleven symptoms were spontaneously reported by participants with CeD: abdominal pain, bloating, cognitive difficulties, constipation, diarrhea, fatigue/tiredness, headache, nausea, passing gas, joint pain and skin rash. Clinical experts subsequently endorsed 10 out of 11 of these symptoms but recommended that joint pain be removed because it was not considered to be reliably attributable to CeD and gluten exposure. This resulted in a draft 10-item PRO measure that measured symptom severity using a scale of 0–10 of ‘no pain’ to ‘worst imaginable pain’ for abdominal pain and headache, or a 5-point Likert scale of ‘very mild’ to ‘very severe’ for boating, constipation, diarrhea, nausea, skin rash and tiredness. A 5-point scale of ‘not at all’ through to ‘completely’ was used to measure cognitive difficulties/difficulty thinking clearly and impact of symptoms on activities and sleep. The frequency of diarrhea (with descriptive options for stool consistency) and spontaneous bowel movements was measured using a numerical scale. This preliminary CDSD was designed to be administered via an interactive voice recognition system (IVRS) with a 24-h recall period.

The preliminary CDSD was subsequently reviewed in cognitive debriefing interviews in participants with CeD (N = 15) to confirm concept relevance and to assess any ambiguities in meaning and interpretation. These interviews identified some areas of ambiguity within the diarrhea and constipation items. Although the term ‘diarrhea’ was understood with the descriptor ‘loose stool,’ there was uncertainty around the conceptual linkage with severity. Likewise, ‘constipation’ was understood by all participants, but the term ‘successful bowel movement’ in the follow-up question was not understood. Therefore, 10 supplemental qualitative interviews were conducted in which participants’ understanding of diarrhea severity and interpretation and relevance of the constipation item were specifically explored. An initial psychometric study of the CDSD was also conducted in participants with biopsy-confirmed CeD (N = 202) recruited at three US clinical sites (Beth Israel Deaconess Medical Centre, Boston; Mayo Clinic, Minnesota; or Columbia University, New York). Passing gas, abdominal pain, fatigue and bloating were the most commonly endorsed symptoms across all 7 days of the psychometric study. This study also indicated a poor sensitivity in discriminating between different levels of symptom severity related to constipation/unsuccessful bowel movements, abdominal pain and diarrhea.

The main change introduced in response to the supplemental qualitative interviews was the replacement of diarrhea and constipation sub-items (i.e. loosest stool consistency and number of successful bowel movements) with general severity rating items. Follow-up questions on the impact of symptoms on sleep were also removed. These changes resulted in a 10-item tool, CDSD 1.0©.

Demographics and clinical characteristics of interview participants, frequency of spontaneously elicited symptoms and symptom saturation matrices for the initial and supplemental interviews and results from the psychometric study, conducted during development of CDSD 1.0© are provided in the Online Resource (Abstract 1) along with a sample copy of this earlier version of the CDSD (for illustrative purposes only).

FDA feedback on CDSD 1.0 and corresponding changes

FDA feedback (April 2012) on CDSD 1.0© and expert input (scientific advisory board and expert interviews with two US gastroenterologists and one US registered dietitian specializing in CeD, June 2012) on the supplemental interview findings and the psychometric study highlighted the need to conduct further interviews.

At this stage, the FDA did not evaluate content validity of the draft measure and stated that an adequately established and portrayed conceptual framework for CeD would be required to confirm content validity. Studies to support the development of a conceptual model for CeD were undertaken and subsequently reported by Leffler et al. in 2017 [5].

Further recommendations from the FDA on CDSD 1.0© (February 2013) included removing symptom impact sub-questions and focusing on the presence/severity of gastrointestinal (GI) symptoms typically experienced by patients (i.e. eliminating concepts not deemed to be directly associated with the GI effects of CeD and/or unlikely to improve in a clinical study). In response, cognitive difficulties, headache, passing gas and skin rash were removed and all impact and duration sub-items were removed.

The FDA also recommended replacing the word ‘fatigue’ with ‘tiredness’ and suggested that the wording related to spontaneous bowel movements, diarrhea and bloating be refined and/or definitions be included to clarify their meaning (pending further qualitative studies). In response, further clarifications were made to the wording of the CDSD. The questionnaire was also modified such that the constipation item was skipped if a patient answered ‘yes’ to experiencing diarrhea or spontaneous complete bowel movements.

The FDA also recommended that reference to CeD be removed when asking patients about a specific symptom, because patients should be asked to report on a symptom without having to make an assessment concerning its cause or origin. This change was also accepted and implemented.

For some of the response options it was unclear to the FDA whether or not patients could reliably distinguish between ‘very mild’ and ‘mild’ or between ‘very severe’ and ‘severe.’ The FDA also deemed that the scoring algorithm was inadequate because response options were transformed to a 0–10 scale that resulted in assigning a score of zero for very mild or infrequent symptoms (e.g. having diarrhea once or twice). The FDA suggested that a score of zero should be used to indicate the absence of a symptom and further assessment of the response options was recommended by the FDA to implement the necessary refinements. The FDA also indicated that the original number of questions included would be too burdensome for a daily diary and reiterated that removal of non-GI symptom items would help to reduce respondent burden.

CDSD 1.1© (Abstract 2, online resource)Qualitative studies for content validation

After modifications of the CDSD in response to the above FDA feedback, the resulting PRO measure had 6 items with response options that included a numerical scale (0–10) for abdominal pain, a 5-point Likert scale (very mild to very severe) for bloating, nausea and tiredness, and frequency options (once to ≥ 10 times) for diarrhea and spontaneous bowel movement.

These modifications were assessed in additional cognitive debriefing interviews in adult participants with CeD (N = 15). These interviews confirmed that the content of the revised PRO measure (CDSD 1.1©) was clear and comprehensible, focused on symptom concepts central to the typical participant’s experience of CeD and that this PRO measure could be appropriately interpreted and administered. Demographics and clinical characteristics of participants in these cognitive debriefing interviews are provided in Online Resource Abstract 2, along with a sample copy of CDSD 1.1© (for illustrative purposes only).

FDA feedback on CDSD 1.1©

FDA feedback on CDSD 1.1© (July 2014) suggested that additional validation evidence was necessary to use the CDSD to support product labeling of potential CeD treatments for adults and adolescents. FDA recommendations after a further review of CDSD 1.1© (March 2019) included an assessment of this PRO measure in younger adolescents (aged 12–14 years) to confirm content validity and understanding in this age group.

The FDA also suggested that the severity scales and frequency options should not be combined into a single score (in particular, that diarrhea frequency and non-stool GI severity items should be analyzed separately). The FDA again requested that evidence from participants’ interviews be provided to confirm that patients with CeD can distinguish between the response options of ‘very mild’ and ‘mild’ and between ‘very severe’ and ‘severe,’ and a reduction in response options should be considered accordingly. It was also suggested that nausea and bloating severity scores should include zero (for patients who report none of these symptoms) and abdominal pain scoring should align with nausea and bloating scores (0 to 5).

In response to a direct question, the FDA agreed that the Bristol Stool Form Scale (BSFS) [21] could be used to ensure participants’ responses to the diarrhea question align with the relevant stool descriptors.

In addition, the FDA reviewed a Weekly CDSD Severity Score algorithm in which the daily symptom scores for all items were averaged across a 7-day period (proposed for use in clinical trials) and recommended that ‘tiredness’ should not be included in this weekly severity score (to focus potential clinical trial endpoints on GI symptoms only).

EMA feedback

The EMA’s Committee for Medicinal Products for Human Use (CHMP) provided feedback on CDSD 1.1© in September 2019 and concluded that the disease concepts covered in this PRO measure may serve as a primary endpoint in CeD clinical studies. However, the CHMP pointed out that the impact of a GFD on patients’ lifestyle should also be considered in a CeD clinical trial.

The CHMP were asked whether the use of the BSFS would be suitable to ensure that patients’ response to the diarrhea question in CDSD 1.1© aligns with the relevant stool descriptors in the BSFS. The CHMP agreed that the BSFS could be used in the context of daily evaluation as this scale was readily understood by both adult and adolescent populations (as demonstrated in cognitive debriefing interviews outside of the studies reported here). However, the use of the BSFS in addition to the diarrhea question (which already included a qualifier question on ‘loose, mushy or liquid’ bowel movements) was not readily understood.

The scoring and weighting across items included in CDSD 1.1© were identified by the CHMP as an area that required further supportive evidence (e.g. the CHMP questioned the decision to cap the diarrhea count at nine episodes and suggested that all counts above nine should not be treated equally without evidence to support this decision).

Finally, the CHMP concluded that more studies may be required to confirm the adequacy of CDSD 1.1© content for adolescent patients through further concept elicitation interviews with a larger number of participants from this younger population. The CHMP did not identify any issues with regard to patients’ comprehension of CDSD 1.1©.

CDSD 2.1© (Abstract 3, online resource)Qualitative studies for content validation

In response to the FDA feedback regarding combining frequency and severity items, CDSD 1.1© was further modified to focus on symptom severity scores only. Frequency of bowel movements was removed and a supplementary questionnaire (CDSD 2.1©—Frequency Supplement) was developed to capture information on the frequency of ‘all’ bowel movements and bowel movements classified as ‘Type 6 or 7’ on the BSFS; the use of the BSFS to assess the relationship between diarrhea severity and visual stool consistency was supported by the FDA and the EMA (along with removal of the ‘loose, mushy or liquid’ descriptors). Frequency of vomiting was also added to the CDSD 2.1©—Frequency Supplement. Furthermore, the severity scoring system for all items was modified to 0 (none), 1 (very mild), 2 (mild), 3 (moderate), 4 (severe) and 5 (very severe), and further testing was conducted to assess the relevance of the ‘very mild’ and ‘very severe’ response options. The non-GI item ‘tiredness’ was retained because this was reported with a higher frequency than other non-GI items and was considered of high importance to patients; however, it was not included in the Weekly CDSD Severity Score.

After the above changes, concept elicitation and cognitive debriefing interviews were conducted with adult (N = 16) and adolescent (N = 16) participants with CeD. Targeted recruitment of younger adolescent participants (to ensure comprehension of items in this age group) was successful, with half of adolescent participants aged 12–14 years (n = 8). The inclusion criteria used were aimed at reflecting patient populations in planned CeD clinical trials (see Online Resource Abstract 3).

During an initial concept elicitation portion of the interviews, participants were asked to report on the symptoms they experienced as well as identifying symptoms that were most bothersome to them. A cognitive debriefing portion of the interview was then initiated in which participants were asked to review the modified CDSD in its entirety for content validity, along with the CDSD 2.1©—Frequency Supplement. Refinements to the CDSD were made between each of three rounds of interviews as required. Details regarding participant feedback from the concept elicitation and cognitive debriefing interview rounds are provided in Online Resource Abstract 3 along with a sample copy of CDSD 2.1© and the CDSD 2.1©—Frequency Supplement (for illustrative purposes only and not to be used without permission from the CDSD’s licensor and distributor, Mapi Research Trust [https://eprovide.mapi-trust.org/]).

Modifications based on these interviews included a minor change to further specify ‘each evening’ in the instructions for completing the questionnaire. Constipation was removed as a severity assessment because the question on bowel movement frequency included in the CDSD 2.1©—Frequency Supplement was felt to address this concept better. The response option ‘very mild’ was also removed for all items (as previously recommended in FDA feedback and confirmed as a suitable omission in these interview rounds in which several participants reported an inability to distinguish between ‘very mild’ and ‘mild’). Finally, in response to adolescent feedback, the word ‘belly’ was added in parentheses after the term ‘abdominal,’ for clarification. These final interviews demonstrated that CDSD 2.1© was readily understood by both adult and adolescent participants.

A Weekly CDSD Severity Score has been proposed for use in clinical trials in which the daily symptoms scores for all GI items are averaged across a 7-day period. As per FDA recommendations, the Weekly CDSD Severity Score does not include the ‘tiredness’ item.