The aim of this study was to conduct an evaluation of the TANDEM intervention fidelity; to assess how well the intervention was implemented according to its intended protocol. We developed a fidelity evaluation that was feasible to conduct within the context of a randomised controlled trial including a bespoke treatment fidelity evaluation framework and a validated therapeutic competency assessment15. Overall, the evaluation suggests that with proper training, guidance, and supervision, respiratory nurses and other respiratory healthcare professionals can deliver cognitive-behavioural interventions with acceptable therapeutic competency, but further training would be needed to optimise intervention delivery.

Trial facilitators adhered well to the intervention protocol, and core components were delivered with fidelity with one exception – Sets agenda for the session. In cognitive behaviour therapy (CBT), agenda setting is a collaborative process where patient and therapist agree how their session time will be spent. It highlights what is most important for the patient19 and is considered an important task for enhancing therapeutic alliance, an independent predictor of patient outcomes20. It may have been that in order to adhere to session content facilitators were reluctant to invite a patient agenda for fear of being side tracked. It might also reflect the learned clinical practice influenced by data driven templates which encourage a check-list consultation style and prioritises the healthcare professional’s consultation agenda over the patient’s21. Supporting the skill development of shared agenda setting might need more focus and on reflection, the TANDEM training programme could have addressed this task with greater emphasis, and this is noted as an important focus for future training.

Content that was delivered with high fidelity included establishing illness and treatment beliefs, breathlessness experiences and mood, which were important early activities to inform the intervention tailoring. In a separate interview study, facilitators described their increased confidence in asking participants about their beliefs and mood after training, reflecting that this was a something they might have previously avoided in clinical practice22. Facilitators asked all participants about their breathlessness and demonstrated breathing techniques with very high fidelity, yet only about half the participants were heard practicing these skills in the session. Demonstration and repetition of behaviour by a recipient are important behaviour change techniques, particularly when learning a new skill23 such a specific breathing techniques in COPD. Healthcare professionals may not frequently encourage or prompt these actions from their patients; It is not clear why this is, perhaps this relates to the point previously made about the challenges of shared agenda setting. The facilitators may have been more familiar with instructing and demonstrating the techniques but less accustomed to inviting patients to demonstrate their skills. This suggests a valuable focus for future interventions to enhance facilitator training. Encouraging these techniques more consistently could enhance the effectiveness of embedding new skills.

The therapeutic competency scores of facilitators in this study is comparable with previous research studies including palliative care nurse practitioners, newly trained in cognitive behavioural methods, assessed during month six of a trial with cancer patients17 and clinical nurse specialists (assessed using simulated patient role play after two years’ patient delivery24). In a COPD trial similar to TANDEM, respiratory nurses’ mean therapeutic competence, was rated considerably higher (44.0)25; however in that trial, two of the four nurses had completed a post-graduate diploma in CBT, so could not be considered as novice.

For specific therapeutic skills, we found higher competency for focus, pacing, collaborative relationship and interpersonal effectiveness, which likely represents the existing well-honed skills that healthcare professionals possess26. It appeared to be more challenging to reach competency in specific CBT skills such as guided discovery and eliciting key components of the model, however, higher competency was observed over time in the TANDEM trial. Evaluating both ‘early’ and ‘late’ cases, there appeared to be a trend towards higher therapeutic competency overall in the later cases which might suggest that these skills improve over time (though this observation is limited to the nine facilitators who delivered more than ten cases). Improving therapeutic competency over time has been well-documented, with evidence suggesting that experience, particularly when combined with supervision, is associated with significant improvements in CBT competence17. However, our findings should be interpreted cautiously as case numbers were limited. Therapeutic competence also appeared to strengthen ‘within cases’, as the sessions progressed, from therapeutic competency scores of 33.6 on average for Session 1 to scores >40 for Sessions 7–9, although as there were fewer later sessions to assess, this finding again needs further evidence from larger samples.

This study has many strengths including the comprehensive development of a bespoke framework, the result of a systematic collaborative process with both trial and independent researchers, to pilot, evaluate and refine the framework to ensure its optimum suitability. Recognising the complexity of tailored interventions, we made the decision to assess all audio sessions within each randomised case to ensure that content could be coded regardless of the session in which it was provided. This resulted in a substantial caseload (coding approximately 180 h of audio sessions) which required resources in addition to the trial funding (supported by the NIHR ARC North Thames). Facilitator-completed case report forms were also consulted to support the adherence analysis. This proved valuable for confirming intervention delivery details, such as identifying the leaflet provided when it was not mentioned in the audio or when audio recordings were unavailable.

Assessing fidelity through audio recordings is a robust method, representing the gold standard approach encouraged in the National Institutes of Health Behaviour Change Consortium (NIH-BCC) Framework27 and necessary to examine therapeutic competence, but is not without limitations. For sessions delivered at home, audio recordings were sometimes unclear due to background noise, interruptions from family members, or patients discussing other complex physical and social issues important to them. This challenge has been noted previously24 and was also highlighted in the patient and facilitator qualitative studies22. To address these challenges and assess this novel intervention, we complemented our bespoke framework with established methods recommended in recent fidelity literature8 Specifically, we measured therapeutic competence using the validated CFAR scale17. Employing existing measures ensures adherence to known standards and facilitates comparisons with other trials.

The fidelity assessment was completed by VW, independent of the trial intervention development, trial processes and facilitator supervision. LS provided quality assurance assessments and met regularly with VW to discuss cases and resolve any queries. However, we were unable to undertake full duplicate coding due to the lack of resources for the time-consuming task; we acknowledge this is a limitation of our study. Calls are made to ensure that sufficient resources are included for fidelity in the same way as other trial assessments such as health economics and the research community needs to agree of the importance of this task. We also recognise that, by including all sessions within each case (up to 9 per case), the number of cases is fewer that we would have ideally included (15% of the intervention group where ideally, we would have included 20%). However, due to the variability of a tailored intervention, it was not feasible to only assess certain sessions within cases.

Respiratory healthcare professionals can be trained to deliver interventions underpinned by psychological approaches. This is important given the increasing demand for psychological support particularly amongst those with long term co-morbid conditions and the challenges in obtaining support from psychological services. However, resource constraints may also hinder respiratory healthcare professionals from incorporating psychological support into their clinical care, if it takes, or is perceived to take, additional time. It is also important to highlight the training and support needed to enable clinicians to take on this role. Firstly, it is important that there is a robust training programme where people are assessed in competency before they practice. In this trial we found that, despite pre-training interviews, some individuals were not suitable to deliver the intervention even after repeat training and there should not be an assumption that this approach is suitable for all. At interview, potential facilitators for the TANDEM trial had to demonstrate a commitment to a biopsychosocial approach to treatment, which emphasises holistic, patient-centred care, ensuring that both emotional and social contexts are integrated into the therapeutic process. In clinical practice, attention should be given to healthcare professionals’ communication skills and their experience, training and continued interest in psychological approaches to care. Secondly, supervision is essential to provide support for clinicians and to ensure quality of clinical care. Although not formally evaluated in this study, qualitative findings suggested that facilitators perceived supervision as essential and its place in such care is well established22. Finally, in the trial, Facilitators were provided with a comprehensive intervention manual outlining session content, example skills delivery and background theory as well as resources to use in sessions such as prompt cards. It is likely that when using such techniques in a busy clinical service these approaches will be even more important.