KEY POINTS The C-NICAS questionnaire's systematic translation and cross-cultural validation provide an accurate and valid French-Canadian questionnaire. A multidisciplinary panel of experts, including a patient partner, enriches the validation and the search for equivalence between the original and adapted questionnaire version. The C-NICAS-FR questionnaire is ready to be implemented in the study's designated hospital center. Nursing practice might consider training in clinical informatics.

Information and communication technologies are increasingly used in transforming the healthcare system.1 They represent complex resources whose objectives are to support clinical processes and promote synergy between the user, the delivery of care, and the context of work organization. To work in this environment, nursing informatics competencies are essential to facilitate healthcare delivery.2

Although several informatics competency assessment instruments for nurses exist, not many are from Canada. In 2012, the Canadian Association of Schools of Nursing, in partnership with Canada Health Infoway, undertook a project to develop a competency assessment scale, defined in English and translated into French, but not validated in a cross-cultural perspective.3 On the basis of this work, Kleib and Nagle4,5 developed a computer competency assessment questionnaire for nurses called the “Canadian Nurse Informatics Competency Assessment Scale (C-NICAS)” and validated it with 2844 nurses in Alberta. It comprises 21 items, including the global competency, three competency domains, and two foundational information communication technology (ICT) competencies.

The three domains of competencies are (1) “information and knowledge management” (seven items), (2) “professional and regulatory responsibility” (six items), and (3) “information and communication technology” (six items).3,4 Questions are measured using a 4-point Likert scale ranging from 1 (not competent) to 4 (very competent). Cronbach's α assesses the internal consistency of the questionnaire's overall result. Internal consistency is measured at 0.926.6 The Canadian study showed that factors such as age, work setting, educational qualifications, years of experience, computer training, and access to the Internet were associated with global competency.5

This article presents the development of the French-Canadian version of the C-NICAS with two objectives: (1) to translate the Canadian C-NICAS questionnaire and (2) to validate the French-language measurement instrument with nurses from a university hospital in the Montreal region using a cross-cultural methodology.

METHODOLOGY

When feasible, using existing and validated questionnaires is a well-founded option in a cross-cultural translation and validation context.7,8 In doing so, the original and final questionnaires in the target language should be equivalent in all respects.7 Several methodologies for translation and cross-cultural validation exist. Inspired by the principles and guidelines developed by the International Society for Pharmacoeconomics Outcomes for Research, the cross-cultural translation and validation methodological process used in this study consists of six steps and is illustrated as Supplemental Digital Content, Figure 1, https://links.lww.com/CIN/A276.9

Step 1: Forward Translation A and B Versions

The English-to-French translation was achieved by two independent, professional translators, both native French speakers and one of whom is familiar with informatics and the nursing practice.9 The expert panel comprised seven members, including an academic member, two members of the nursing practice with researcher status, a member of nursing management, the research director, the researcher in charge of the study from the target institution, and a patient partner. Although the questionnaire was geared toward nurses, including a patient partner helped enrich the cross-cultural validation process.7 Being multidisciplinary, the panel of experts allowed for multiple perspectives on the questionnaire and eliminated potential sources of bias.10 This consensual revision made it possible to reconcile differences in translation, correct vocabulary, and improve the summary version to produce an adapted version.9

Step 2: Back-Translation C and D Versions

Two native English independent translators performed back-translation. Although the Epstein et al11 study showed that back-translation only adds value when an expert panel is implemented, it is useful when the authors are not fluent in the target language. Before the second version of the expert panel, the members received a tutorial inviting them to evaluate the equivalence of the translated versions.

Conceptual, item, semantic, and operational equivalence were highlighted and considered.7

Step 3: First Experimental Version Of The Canadian Nurse Informatics Competency Assessment Scale-French

The second session of the panel of experts allowed for the revision and harmonization of the three different versions of the questionnaire. The panel members corrected translation discrepancies and ensured the conformity between the original and translated versions to produce the preexperimental C-NICAS-French (C-NICAS-FR). Simultaneously, two nursing practice researchers acted as judges by assessing the equivalencies of the identical translated versions to produce the content validity index.

Semistructured interviews were conducted with nurses. Eligibility criteria were (1) to hold a valid “Ordre des infirmières et infirmiers du Québec” license to practice and (2) to work in the university hospital in the Montreal area. The work settings included inpatient, ambulatory, mental health, and the critical care sector. The nurse was asked to rate the clarity of each question using a 7-point Likert scale, where 1 means the question is not clear and 7 means the question is totally clear. Items with averages of 4 and below were to be modified.10,12 The nurse was then encouraged to express his/her thoughts regarding reaction, opinion, ease or difficulty in understanding the questionnaire, comfort level, and any other point of view he/she desired to share.12

Step 4: Second Experimental Version Of The Canadian Nurse Informatics Competency Assessment Scale-French

Descriptive statistics were generated to describe the data related to the evaluative question on clarity. A table was used to upload and categorize voiced comments from participants and clarify the understanding provided by the nurse. The semistructured interviews were analyzed with QDA Miner and coded using a two-step method.12 The criteria that were applied to ensure scientific were credibility, transferability, and reliability. The qualitative analysis was produced by the student-researcher and reviewed by two other researchers.

DATA COLLECTION

Data from the equivalence exercise were exported to IBM SPSS Statistics version 27 (IBM Inc., Armonk, NY, USA). The content validity index is measured from the interjudge agreement measure. Cronbach's α measures the percentage of agreement when more than two observers are present. This measure was used for panelists.8 A result above 0.7 is considered acceptable. Pearson's product-moment correlation coefficient is recommended when two observers are involved in the interjudge agreement. In this case, a result between <−1 and >1 is considered acceptable.8 By comparative measurement, κ is produced based on the result obtained from panelists and judges. Concerning this coefficient, the levels of reliability are represented according to the various measures of κ, including <0 to 0.20>, none; <0.21 to 0.39>, minimal; <0.40 to 0.59>, moderate; <0.60 to 0.79>, strong, >0.90, excellent.13

Step 5: Statistical Tests

The second experimental version of the questionnaire was validated using an online survey of the 25 nurses who participated in the cognitive debriefing. Internal consistency and correlations were measured. Analyses were performed using IBM SPSS Statistics version 27.

Step 6: Canadian Nurse Informatics Competency Assessment Scale-French Final Version

Following the validation of the second experimental version and the data analyses, the panel of experts met for a third session to validate and approve the final version of the questionnaire.

Ethical Considerations

Informed consent was obtained from all participants in the study.

RESULTS Multidisciplinary Expert Panel

Several issues were reported in the second session of the expert panel. Items 3, 4, 5, 6, 8, 9, 11, 12, 15, and 16 were modified. For instance, in item 3, the phrase “performs search and critical appraisal” was changed to “carries out research and critical analysis.” In item 11, “the use of current and innovative ICTs” was changed to “the use of modern and innovative ICTs.” In the back-translation, the verb “using” was used at the beginning of each item. It was changed to “uses” by both panelists and judges.

Reliability Of The Preexperimental Version Of The Questionnaire

The content validity index score, obtained from the panelists and measured by Cronbach's α, was .742, which is considered good. As for the result obtained from the judges and measured from Pearson's product-moment correlation coefficient, the result obtained is 1 for conceptual equivalence, 1.00 for item and semantic equivalence, and 0.933 for operational equivalence, which is very good since the results are close to or equal to 1. By comparison, κ is measured according to the percentage of agreement. The panelists obtained a result of 98.5%, and the judges obtained a result of 74.1%, which is acceptable.

Cognitive Debriefing And Online Survey

Twenty-five nurses from different sectors of the target institution were eligible and agreed to participate in the study to pretest the questionnaire. The average length of the interviews was 27 minutes. Table 1 is a representation of the participants in the online survey.

Table 1 - Characteristics of Participants Characteristics Categories n % Professional designation Nurse 24 96 ICS 1 4 Education College degree 4 16 Baccalaureate degree (3 y) 20 80 Master's degree 1 4 Age, y 18–24 1 4 25–34 7 28 35–44 9 36 45–54 6 24 55-64 2 8 Years of experience Less than 5 3 16 Between 6 and 10 9 28 Between 11 and 20 8 36 More than 21 5 20 Work setting Acute care 12 40 Critical care 5 28 Outpatient 5 20 Mental health 3 12

The score for the clarity of each question ranged from 1 to 7 (1, not clear, and 7, totally clear), with a mean (SD) ranging from 4.00 (1.633) to 6.80 (0.408). Several minor changes were applied. Participants suggested seven changes to the questionnaire, including adding explanations related to domains of competency as well as items 3, 5, 7, 9, 10, and 13. For example, adding the word “computers” with systems is recommended for item 13. Many participants asked for examples when reading certain items. Most nurses (n = 20) had difficulty with item 7, specifically with the word “interoperability.” A few nurses indicated that using Google when completing the online survey would help define this word. Item 7 was revised for the online survey and validated with three participants. In addition, some nurses mentioned receiving insignificant training in computer competencies at either the university or college level.

Online Survey

The second experimental version of the C-NICAS-FR self-evaluation was sent to the 25 nurses. The average global score of the questionnaire was 60, with a standard deviation of 6.72, indicating that the nurses perceived themselves as competent. Nurses in the 35- to 44-year age group perceived themselves as more competent. Likewise, nurses with work experience between 6 and 10 years also perceived themselves as more competent. The correlation between age and the overall result was negative (r = −0.032), and the correlation with the number of years of experience was weak (r = 0.216). They are insignificant. The internal consistency measures the reliability of the overall result of the questionnaire. The result of the internal consistency of the questionnaire was 0.828.

The Final Panel of Experts

Following the qualitative and quantitative analyses, the expert panel revised the final version of the questionnaire for approval. Item 7, particularly pertaining to the topic of interoperability, and items 10 and 19 were further modified to reflect the original version of the questionnaire (see Table 2). Consensus was reached, although there was a difference of opinion among the panel of experts.

Table 2 - C-NICAS-FR Final Panel of Experts' Modifications Question Number Equivalence Dimension French-Canadian Cross-validated Version Comments Decision Q7 Semantic Fait ressortir l'importance des normes d'information, c'est-à-dire les normes de messagerie nécessaires pour assurer l'interopérabilité des dossiers électroniques. Choice of words does not adequately reflect the original version. Item is reviewed Q10 Item Respecte les exigences légales et réglementaires, les normes éthiques, les politiques et procédures organisationnelles (protection des renseignements personnels sur la santé et la vie privée). The removal of the “and” slightly modifies the understanding of the item. Adding the “and” Q19 Semantic Décrit les diverses composantes des systèmes d'information sur la santé (p. ex., affichage des résultats, documentation clinique, etc). The word « en » improves the clarity of the item. Modifying “sur la santé” for “en santé”
DISCUSSION

The translation and cross-cultural validation of a measurement instrument is a complex process. This study demonstrated that the validity of the C-NICAS questionnaire is accurate in the French-Canadian language. By using a systematic methodological approach, questionnaire items were translated. Other items were modified or improved and submitted to a multidisciplinary panel of experts to evaluate equivalences and ensure that the validated French-Canadian questionnaire respected the original version.7 Consequently, eliminating researcher, participant, or translator bias was optimized throughout each step.9,10 The literature review reflects many of the study's findings. Attention was focused on ensuring that translators removed discrepancies between the versions produced, eliminating implicit bias that a single translator approach can induce.9,10 The members of the expert panel were bilingual. The search for accurate and understandable wording encouraged a rich debate during the panel sessions. The multidisciplinary panel of experts contributed generously via their qualitative rigor, constructive exchanges, and the search for equivalence to produce a final version reflecting the original questionnaire.10 The inclusion of a patient partner in the panel of experts has enriched the validation of the questionnaire.7 Moreover, it contributed to improving the questionnaire by considering the patient's health experience.

Canadian Nurse Informatics Competency Assessment Scale-French

The cognitive debriefing of 25 nurses revealed minor changes. Out of 27 items, only item 7 received an average score of 4 out of 7. Although the average score of the items is considered good, the participants expressed their opinion on improving seven items with minor changes. In addition, during the interjudge agreement, both researchers commented on and improved six items, including general competency and items 3, 6, 7, 12, and 19. The online survey demonstrated that the C-NICAS-FR is a reliable questionnaire in the French-Canadian language with an internal consistency score of 0.828.8 This result is consistent with studies that have demonstrated reliability in pretesting.14,15 Nurses have reported needing more training in schools regarding computer competencies. In this matter, a Canadian study indicates that solutions can be implemented to train nurses to enhance informatics competency.16

Strengths and Limitations

The study has permitted to outline many strengths. The methodological approach is based on rigorous guidelines, allowing the creation of a valid French-Canadian version of the C-NICAS questionnaire.7,9,10 It helped improve the search for conceptual, item, semantic, and operational equivalencies through semistructured interviews and the online survey with nurses.7 Back-translation is essential as the authors are unfamiliar with the French-Canadian language.11

The study also has its limitations. The small number of participants used to establish the psychometric measures of this questionnaire is a limitation of this study. The C-NICAS-FR questionnaire includes an overall score of all 21 items and does not include specific scores for the three competency domains.

CONCLUSION

The C-NICAS-FR is a new assessment questionnaire for nursing informatics competencies. It represents an opportunity for French-speaking nurses to measure their informatics competencies and will be implemented in the institution targeted by the study. It is also an opportunity for healthcare organizations to address ICT training needs to ensure that nurses will have the competencies to interact autonomously across information systems and provide patient delivery care in having a satisfactory experience.

The study translated and validated the C-NICAS questionnaire into French Canadian from a cross-cultural perspective. In a context where a methodological approach requires the inclusion of participants from various cultures, the relevance and added value of the method of cultural adaptation should be considered.

References 1. Britnell M, Bakalar R, Shehata A. Digital Health: Heaven or Hell? How Technology Can Drive or Derail the Quest for Efficient, High Quality Healthcare. Swiss entity: KPMG International; 2016:33. https://assets.kpmg/content/dam/kpmg/pdf/2016/03/digital-health-heaven-hell.pdf 2. Hunter K, McGonigle D, Hebda T. TIGER-based measurement of nursing informatics competencies: the development and implementation of an online tool for self-assessment. Journal of Nursing Education and Practice. 2013;3(12): 70–80. doi:10.5430/jnep.v3n12p70. 3. Canadian Association of Schools of Nursing. Nursing informatics entry-to-practice competencies for registered nurses. Published 2019. https://www.casn.ca/2014/12/nursing-informatics-entry-practice-competencies-registered-nurses-2/ 4. Kleib M, Nagle L. Development of the Canadian Nurse Informatics Competency Assessment Scale and evaluation of Alberta's registered nurses' self-perceived informatics competencies. Computers, Informatics, Nursing. 2018a;37(7): 350–358. 5. Kleib M, Nagle L. Factors associated with Canadian nurses' informatics competency. Wolters Kluwer Health. 2018c;36(8): 406–415. doi:10.1097/CIN.0000000000000434. 6. Kleib M, Nagle L. Psychometric properties of the Canadian Nurse Informatics Competency Assessment Scale. Computers, Informatics, Nursing. 2018b;36(7): 359–365. doi:10.1097/CIN.0000000000000437. 7. Epstein J, Santo RM, Guillemin F. A review of guidelines for cross-cultural adaptation of questionnaires could not bring out a consensus. Journal of Clinical Epidemiology. 2015;68: 435–441. doi:10/1016/j.jclinepi.2014.11.021. 8. Waltz C, Strickland OL, Lenz ER. Measurement reliability. In: Measurement in Nursing and Health Research. 5th ed. New-York, NY: Springer Publishing Company; 2017: 183–207. 9. Wild D, Grove A, Martin M, et al. Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: report of the ISPOR task force for translation and cultural adaptation. Value in Health. 2005;8(2): 94–104. 10. Vallerand RJ. Toward a methodology for transcultural validation of psychological questionnaires: implications in the French language [article in French]. Canadian Psychologist. 1989;30(4): 662–680. 11. Epstein J, Osborne RH, Elsworth GR, Beaton DE, Guillemin F. Cross-cultural adaptation of the Health Education Impact Questionnaire: experimental study showed expert committee, not back-translation, added value. Journal of Clinical Epidemiology. 2015;68(2015): 360–369. doi:10.1016/j.jclinepi.2013.07.013. 12. Tremblay D, Bilodeau K, Durand MJ, Coutu MF. Translation and perceptions of the French version of the Cancer Survivor Profile-Breast Cancer (CSPro-BC): a tool to identify and manage unmet needs. Journal of Cancer Survivorship. 2019;13: 306–315. doi:10.1007/s11764-019-00752-2. 13. McHugh ML. Lessons in biostatistics: interrater reliability: the kappa statistic. Biochemia Medica. 2012;22(3): 276–282. 14. Chung SY, Staggers N. Measuring nursing informatics competencies of practicing nurses in Korea. Computers, Informatics, Nursing. 2014;32(12): 596–605. doi:10.1097/CIN.0000000000000114. 15. Olajubu AO, Irinoye OO, Olowokere AE. Competencies and barriers to the use of nursing informatics among nurses in primary, secondary and tertiary healthcare facilities in Nigeria. Journal of Health Informatics in Africa. 2014;2(1):30–41. doi:10.12856/JHIA-2014-v2-il-85. 16. Nagle LM, Kleib M, Furlong K. Digital health in Canadian schools of nursing—part B: academic nurse administrators' perspectives. Quality Advancement in Nursing Education - Avancées en formation infirmière. 2020;6(3):1–28. doi:10.17483/2368-6669.1256