The questionnaire was developed, and data collection for it took place, within the Cancer Rehabilitation Support by Cancer Counseling Centers (CARES) study. This is a feasibility study with a quasi-experimental pre/post design that aims to develop, implement, and evaluate a counseling intervention in outpatient cancer counseling centers (OCCs) that focuses on returning to work (Hiltrop et al. 2023). Data collection took place in 19 OCCs in Germany, with two study groups of cancer survivors seeking advice regarding their occupational situation. OCCs were selected from members of the Federal Working Group for Outpatient Cancer Counselling Centres (Bundesarbeitsgemeinschaft für Krebsberatungsstellen - BAK). BAK board members suggested member OCCs they considered prepared for timely participation in the study, then those recommended OCCs were selected according to geography (distribution over country, urbanity). Preselected OCCs were invited to participate in the project.

The first group of cancer survivors that participated in the study received regular counseling in the OCCs (comparison group, CG). Afterward, the second group of participants underwent the newly developed intervention (intervention group, IG). Participants in both groups were surveyed at the start of the counseling process after study enrolment (T0), 3 months after enrolment (T1), and up to 18 months (T1.2) after enrolment. Participants in the IG were also surveyed at the end of their counseling (T2), whereby T2 could either be before or after T1 because the end of the counseling was at an individual point of time. The study inclusion criteria were: having an oncological diagnosis, problems and/or counseling needs regarding the occupational situation, age over 18, sufficient German language skills, no cognitive limitations impeding participation in surveys or interviews, and informed consent. Eligibility criteria were checked by staff in the OCCs, eligible cancer survivors were then invited to participate in the study by the OCCs. T0 and T2 were handed over by the OCCs, T1 and T1.2 were sent postally to the participants by the study team. For T1 and T1.2 non-responders were reminded twice. Participants who did not respond for T1 also received an invitation to fill in T1.2. The study was approved by the Ethics Committee of the Medical Faculty of the University of Bonn (061 − 22; April 9, 2022).

The questionnaire development followed Donabedian’s approach (Donabedian 1988) that quality might include aspects on different levels. In order to cover different processes and outcomes, we designed the questionnaire based on central components of social work in cancer counseling. According to the Working Group on Social Work in Oncology in the German Cancer Society (ASO) expert panel, these central components are: 1, establishing an open communication situation and relationship building; 2, clarification and negotiation processes; 3, provision of information; 4, support and accompaniment in the process; and 5, evaluation and conclusion of counseling. The initial main categories in the German Quality of Cancer Counseling Questionnaire-Return to Work (QCCQ-W) were therefore created based on these five central measures of cancer counseling, and in consultation with experts on and practitioners of cancer counseling of the ASO (see Table 2). In a meeting, two persons from the research team and two members from the ASO discussed concepts and hypotheses as well as formulated an initial pool of 21 items in the individual categories considering existing alliance instruments (Tracey and Kokotovic 1989; Eich et al. 2018) and the five central measures of counseling. This procedure has been described as rational method in questionnaire design elsewhere (Oosterveld et al. 2019).

The items were then pretested in a cognitive think-aloud interview with a cancer survivor for testing comprehensibility and content relevance, which resulted in slight adaptations of the wording because no need for other adaptations e.g. exclusion of items or inclusion of further topics was identified. The questionnaire that was entered into the survey contained 21 items. The analysis presented here only includes 20 items, since the last item had too many missing responses (59%, “If your consultation has already been completed (if not, please proceed to the next question): My counselor and I have discussed whether my goals were achieved”). An explanation for the high number of missing values for the last item might be that most of the participants had not yet completed the counseling process when responding to the survey and therefore skipped the item. Items in the QCCQ-W were measured on a scale from 1 (= “does not apply at all” [“trifft überhaupt nicht zu”]) to 7 (= “fully applies” [“trifft voll und ganz zu”]). The QCCQ-W was measured at T1. The QCCQ-W was developed and analyzed in German.

The characteristics of the participants, listed in Table 1, were documented at T0.

Since our goal was to test the reliability and validity of the QCCQ-W based on its items, we followed classical test theory approach (Cappelleri et al. 2014). Furthermore, we expected the QCCQ-W to be multidimensional (different components of social work counseling) and reflective (items are manifestations of the different components). Therefore, according to COSMIN checklist (Mokkink et al. 2019), we decided to perform factor analyses. Exploratory factor analysis (EFA) was carried out to explore the data structure using data from the comparison group (CG) at T1. To identify whether there were sufficiently large correlations between the items for an EFA, Bartlett’s test of sphericity was run. The suitability for EFA of each item and the set of items was then evaluated using the measure of sampling adequacy (MSA; > 0.5) and the Kaiser–Meyer–Olkin measure (KMO; > 0.5 mediocre, > 0.7 good, > 0.8 great, > 0.9 superb; Field et al. 2012). Afterward, a principal component analysis (PCA) was conducted with orthogonal rotation (Varimax). The final number of extracted factors was led by the Kaiser criterion (eigenvalues > 1), inspection of the scree plot, as well as theoretical considerations (Field et al. 2012). Factor loadings > 0.4 and cross-loadings < 0.4 were considered acceptable (Field et al. 2012; Hair et al. 2014).

In order to validate the structure identified in the EFA, a confirmatory factor analysis (CFA) was conducted using data from the intervention group (IG) at T1. Model fit was evaluated by a combination of the following criteria: comparative fit index (CFI, ≥ 0.95), root mean square error of approximation (RMSEA, < 0.08), and standardized root mean square residual (SRMSR, < 0.08) (Hu and Bentler 1999; Hair et al. 2014).

To assess the construct validity of the QCCQ-W, bivariate correlations (Pearson) with other constructs measuring satisfaction with and usefulness of the counseling were performed. A significant positive correlation was expected to indicate adequate validity (Hobart et al. 2004; Mokkink et al. 2010). The instrument’s reliability in terms of internal consistency was examined using Cronbach’s α (> 0.8 good; > 0.9 may suggest redundancies) (Tavakol and Dennick 2011; Field et al. 2012). Test–retest reliability was assessed by correlating (Pearson) the QCCQ-W scores from the IG at T1 and T2.

Missing data within the items in the QCCQ-W were deleted listwise, leading to n = 229 cases in the CG and n = 216 cases in the IG. All data analyses were performed using RStudio 4.2.2 with the packages nFactors (Raiche and Magis 2022) and psych (Revelle 2023) for EFA, as well as lavaan for CFA (Rosseel 2012).