The development of PDKQ questionnaire consisted of three phases: item development, content validation, and reliability test. This study was registered with the National Medical Research Register of Malaysia (NMRR ID: NMRR-20-1844-55868) and approved by Medical Research Ethics Committee, Malaysia.

The item development phase consisted of three steps: item generation, item deduction, and questionnaire formatting.

The item generation process involved a systematic and vigorous literature search of existing published English language diabetes knowledge tools. The systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) using six electronic databases: CINAHL, Medline, Google Scholar, PubMed, Sage Journals, and Science Direct. This systematic review was published in the Review of Diabetic Studies in 2021 [19]. This review included seven studies that comprised 99 items and divided into eight domains: 29 questions on disease-specific, 21 questions on nutrition, 18 questions on treatment, 12 questions on adverse effects, 9 questions on monitoring, 5 questions on physical activity, 4 questions on risk factors, and 1 question on foot care [19]. The questionnaires utilized were (1) translated and validated MDKT; (2) a questionnaire from Wee et al. and Tham et al.; (3) translated DKN; (4) translated and validated American Association of Clinical Endocrinologists (AACE); and (5) validated Theptarin Diabetes questionnaire. Items for the questionnaire were generated from the findings of the systematic review. Some items were developed based on the locally available Diabetes Medication Therapy Adherence Clinic protocol by the Ministry of Health, Malaysia (16 items) [20] and additional domain related to Ramadan (4 items).

The item deduction step involved six members of expert in three meetings. The members consisted of pharmacists who were involved in the management of diabetes patients in the ward, ambulatory care, and counseling. A nominal technique was utilized to identify the domains and prioritize the key areas of diabetes-related knowledge. Each item in the key area was evaluated in terms of relevance, representativeness, and value to educate diabetes mellitus patients. Every item was voted and comments were made to modify, keep or remove the items. Items that consistently received votes for removal were eliminated, and modifications were made to the commented items. Following extensive discussions, the reranking process was carried out until a consensus was reached, ensuring that no further changes were necessary in the questionnaire.

The first draft of PDKQ was formatted and presented as questions with multiple-choice answers, enabling the identification of specific knowledge gaps for personalized and targeted education.

The first draft of PDKQ was sent to experts selected from different states in Malaysia for content validity. The ten experts included in this study comprised consultant endocrinologists, family medicine specialists, and pharmacists who represented different states in the Working Committee of Clinical Pharmacy specializing in diabetes. The expert validation was conducted through a non-face-to-face approach. The content validation form, expert information sheet, clear information, and informed consent form were emailed to the experts. The experts evaluated the relevancy, importance, and clarity of each item for its corresponding construct on a 4-point scale (4 being either very relevant, very important, or highest clarity, whereas 1 being either not relevant, not important, or no clarity). The experts were asked to provide feedback on the questions to improve the quality of the items.

Content validity ratio (CVR) was calculated for items clarity based on the formula, CVR = (Ne − N/2)/(N/2), where Ne is the number of experts indicated ‘essential’ and N is the total number of experts. The content validity ratio was between 1 and − 1. A higher score indicated better agreement between the members of the experts on the necessity of an item in the questionnaire. Based on the Lawshe table, the items were considered acceptable if they achieved a CVR value of at least 0.62 [21]. The content validity index (I-CVI) and scale-level content validity index were calculated to measure proportional agreement. The Scale-level Content Validity Index (S-CVI/Ave) was computed on average to indicate content validity. An I-CVI of 0.78 or higher was considered to have excellent content validity, given the involvement of 10 experts [22]. For the scale-level content validity index, S-CVI/Ave of 0.9 or higher indicated acceptable content validity [23]. The second draft of PDKQ was provided after modifying and eliminating the items based on feedbacks from the experts.

Potential subjects were recruited for face validation to test the appropriateness of the questionnaire in terms of construct, language clarity, readability, and feasibility. Patients of either gender aged more than 18 years with diabetes mellitus and who understood English were included, whereas patients who had cognitive impairment or psychiatry-associated illnesses such as dementia, Alzheimer, schizophrenia, and mania were excluded. All subjects provided written informed consent.

A cognitive interview was conducted with six patients in two rounds. The second draft of the PDKQ, along with the demographic data, was administered to the subjects. Confusing questions were identified and modified to improve clarity or to be removed. Problematic, irrelevant, or unimportant questions and demographic item were identified and removed. The order of questions was rearranged. The subjects graded the relevancy and importance of each item on a 4-point scale (4 being either very relevant or very important, whereas 1 is either not relevant or not important). Content validity (I-CVI) was calculated, and a score of at least 0.83 was considered excellent validity [24]. The outcomes of the cognitive interviews were recorded. Modifications were made to grammar and word choice. Items were dropped if they lacked clarity, relevance, and importance to produce the third draft of PDKQ.

A pilot test was conducted for test–retest analysis. Patients aged 18 years and above with diabetes mellitus who understood English were recruited and consented to participate. Patients who were unwilling to consent or with cognitive impairment or psychiatric disorders were excluded. Considering the dropout rate of 20%, 36 patients were recruited [25]. The patients completed the questionnaire on Day 1 and then returned to re-administer the same questionnaire after 14 days. The correct answer for the multiple-choice questionnaire was scored as 1, while the incorrect answer was scored as 0. The total score of the first and second responses were measured.

Demographic data such as age, duration of diabetes, smoking, underlying comorbidities, and education level of the patients were collected. Data were analyzed using SPSS 22.0 (IBM Corp., New York, USA). Intraclass correlation coefficient analysis was performed to assess the test–retest reliability of the responses. Spearman rank correlation coefficient was used to analyze overall score stability. An intraclass coefficient value greater than 0.90 indicated excellent reliability, while a value less than 0.5 indicated poor reliability. A value between 0.5 and 0.75 indicated moderate reliability, and a value between 0.75 and 0.9 indicated good reliability [26].