Remember me
GENERAL ARTICLE
Year : 2022 | Volume
: 35
| Issue : 2 | Page : 48-57
A rasch analysis of three socialization and communication measures in 4th-year doctor of pharmacy students
Aryn C Karpinski1, Joseph M LaRochelle2, Kelli Qua3, Riza Memis4
1 Research Measurement and Statistics, College of Education, Health and Human Services, Kent State University, Kent, OH, USA
2 College of Pharmacy, Xavier University of Louisiana; School of Medicine Department of Pediatrics, Louisiana State University Health Sciences Center, New Orleans, LA, USA
3 Medical Education Research, Case Western Reserve University School of Medicine, Cleveland, OH, USA
4 General Directorate of Secondary Education, Republic of Turkey Ministry of National Education, Ankara, Turkey
Correspondence Address:
Dr. Aryn C Karpinski
Research, Measurement, and Statistics Program, School of Foundations, Leadership and Administration, College of Education, Health, and Human Services, Kent State University, 316 B White Hall, Kent, OH 44242
USA
Source of Support: None, Conflict of Interest: None
CheckDOI: 10.4103/efh.efh_75_21
The impact of communication and anxiety on Doctor of Pharmacy students across three measures was examined. Data were collected (N = 120) from 4th-year Doctor of Pharmacy students at a historically black college/university using the Interprofessional Socialization and Valuing Scale, the Personal Report of Communication Apprehension, and the Social Phobia Inventory. Results of Rasch Differential Item Functioning Analysis indicated statistically significant differences between each ethnic group on subcomponents of anxiety in each measure (20% Caucasian, 43% African American, 31.7% Asian, and 4.2% others). Evidence from this study shows that racial demographics affect different subscales of anxiety across doctoral pharmacy students. While some pedagogical implications exist, issues within the measures and their items must also be addressed.
Keywords: Communication, pharmacy education, Rasch analysis, socialization
Recent research in the medical professional field has started to address the education of young professionals. In particular, pharmacy education and student preparedness have begun to shed light on issues surrounding communication and teamwork. The intersection between education and practice is being bridged by this research, which examines how communication apprehension and social phobias like anxiety affect students across a variety of social situations. As many as one in five pharmacy students reported having high levels of communication apprehension.[1],[2] Additionally, it was reported that overall 1st-year pharmacy student communication apprehension was high.[3] Research has shown that different educational intervention methods have decreased overall student apprehension.[4],[5]
Differences in cultural communication styles have shifted the focus of pharmacy education research to gaining cultural competencies (i.e., address difficulties related to cultural differences[6]). Now, pharmacy education is tailored largely toward demographic aspects. For example, one study revealed that 1st-year female Asian students experience high levels of communication apprehension.[3]
Social phobias such as social anxiety have been linked to communication apprehension. An increase in social phobia and communication apprehension has been found in situations of contact with strangers.[7] This finding is of particular importance for high-contact professions such as pharmacists who may cope with social phobia. Furthermore, both social phobia and communication apprehension have been found to decrease approachability and social desirability due to increased apprehensiveness.[8] Thus, this indicates serious communication issues with persons who have high apprehension.
In particular, increased social phobia has been investigated in medical and pharmacy students. North American medical students have shown levels of distress, depression, and anxiety significantly higher than the general population.[9] The presence of this distress and anxiety was highest in the female population tested.[9] Extreme levels of distress and anxiety, represented through imposter syndrome, have also been reported in medical, pharmacy, and nursing students.[10] During pharmacy student training, anxiety was shown as a factor that impacted academic performance and test taking.[11] However, little is known about the impact of social phobia on the professional development and education of pharmacy students.
Various scales can be used to assess students' communication apprehension and social anxiety: (1) the Interprofessional Socialization and Valuing Scale or ISVS,[12] (2) the Personal Report of Communication Apprehension or PRCA,[13] and (3) the Social Phobia Inventory or SPIN.[14]
The abovementioned measures have not been specifically used to assess differences in racial/demographic group responses. Primarily, the ISVS has been used to investigate interprofessional socialization among health-care workers;[12] the PRCA-24 has been used to assess cultural communication differences;[15] and the SPIN has been used clinically to assess the spectrum of social phobia within patients.[14] A wide range of research has evidenced the validity of these measures.[13],[14],[16]
Our previous study included 4th-year pharmacy students in the United States using these scales. This study included an overall analysis of the scales and subscales used. Results from the study revealed that African-Americans had lower communication apprehension than Asian or Caucasian groups along with higher interprofessional socialization compared to Asians.[16] As being part of an “in-group” can increase comfort and lower communication apprehension through modeling and other influences,[17],[18] this experience was evidenced in the study using these scales.
In order to examine meaningful differences between demographic groups, the present study used Rasch analysis on the three measures noted above. Specifically, Rasch analysis specifies the probability of a “correct” response modeled as the difference between the person and item parameter. That is, the person parameter is the “ability” of the person, and the item parameter is defined as the “difficulty” of the item. If a person's “ability” is higher than the “difficulty” of the item, there is a higher probability of a “correct” response on an item (Bond and Fox, 2007). This allows for the researcher to gather data on specifics about the individual items on the instrument to make improvements to the scale.[19] Statistical techniques used in research such as calculating internal consistency reliability using Cronbach's α, factor analysis, point-biserial correlations, and computing a total score are commonly used to develop measures. Because we have shown these scales to have racial differences in the overall scores and subscales,[16] we conducted a Rasch analysis to better inform the specifics of these racial differences at the item level.
This study examined the items on the scales and how demographic differences led to various responses on these items. This psychometric investigation is the first to provide a more in-depth look at specific components within each measure and how different racial groups responded. Ultimately, educational implications are discussed based on differing levels of communication apprehension, interprofessional socialization, and social phobia between demographic groups.
Methods
Participants
The sample consisted of 120 Doctor of Pharmacy students in their 4th year at a historically black college/university in the United States. The sample was comprised of 68.3% females (n = 82) and 31.7% males (n = 38) with an age range of 2245 (M = 27.05, standard deviation [SD] = 3.81). Participant ethnicity was reported as 20% Caucasians, 43% African Americans, 31.7% Asians, and 4.2% others. No compensation was offered to participants and therefore there was no possible coercion. No identifiable information was collected and all data were anonymous. The original research obtained institutional review board approval, copyright permission from the original author of the ISVS, and permission to use from authors of PRCA and SPIN which are not copyrighted.
Measures
Interprofessional Socialization and Valuing Scale
The ISVS, created by King et al.,[12] assesses underlying social behaviors, beliefs, and attitudes that affect interprofessional collaboration. Previously, King et al.[12] reported strong reliability for each of the three scales on a study of 124 health-care professional students (coefficient α = 0.79-0.89). This 24-item measure uses a 7-point Likert scale to measure participants' interprofessional socialization across three components: self-perceived ability to work with others (n = 9), value in working with others (n = 9), and comfort in working with others (n = 6). The Likert scale ranged from “Not At All” (or 1) to “A Very Great Extent” (or 7) with higher scores reflecting a stronger expression of beliefs, attitudes, and behaviors endorsing interprofessional socialization. The scale also included a neutral option.
Personal Report of Communication Apprehension
The PRCA, written by McCroskey et al.,[13] explores four components of communication apprehension such as public speaking, group discussions, meetings, and interpersonal communication and has been used across a wide range of disciplines. The measure consists of 24 statements measuring their comfort or anxiety associated with each of the four components (n = 6 for each component). The measure uses a 5-point Likert scale ranging from “Strongly Disagree” (or 1) to “Strongly Agree” (or 5). Similar to the ISVS, higher scores indicated a higher presence of the attitudes explored in the measure. Therefore, the higher the PRCA score the more anxiety or apprehension a person has toward communication such as public speaking. Strong reliability of the PRCA was shown in a large-scale study of students by McCroskey et al.[13] (coefficient α = 0.93-0.95).
The Social Phobia Inventory
The SPIN consists of 17 items with three subscales, including fear (n = 6), avoidance (n = 7), and physical symptoms (n = 4). SPIN uses a 5-point Likert scale ranging from 0 (Not at all) to 4 (Extremely) with a total score ranging from 0 to 68. The SPIN has adequate test–retest reliability ranging from 0.78 to 0.89, and internal consistency ranging from 0.82 to 0.94 across participants with social phobia and control groups.[14] The SPIN established construct validity by adequately distinguishing between participants with and without social phobia. Three components were extracted using principal component analysis (PCA), which support the preliminary hypothesized subdimensions by Connor et al.[14][Table 1] shows a summary of all subscales of three measures.
Data analysis plan
Data were analyzed using SPSS 23.0 version and Winsteps® 3.91.2 version (IBM Armonk, NY United States; and Linacre JM online). SPSS was used to investigate the participants and items' descriptive statistics and to report internal consistency reliability (i.e., coefficient/Cronbach's α). In addition, SPSS was used to examine the component structure of each measure, and to determine whether or not each subscale (i.e., in SPIN fear, avoidance, and physical symptom subscales) were more appropriate than one general component. After the appropriate component structure had been determined, Winsteps® was used to investigate the degree to which the data adhere to the Rasch Rating Scale Model (RSM) (i.e., polytomous Rasch model).
Additionally, the RSM analysis included the examination of item-person fit statistics and poor fit statistics. Item-person fit statistics indicate the degree to which each item and any person meet the Rasch model expectations.[20] The item-person map was scrutinized to ensure that items and persons have well breadth across the continuum, and that items and persons are well targeted. The usage of category was examined, and based on SPSS item descriptive as well as category information from Winsteps®, multiple collapsing procedures were undertaken to investigate the most appropriate category structure for the existing data. In addition to RSM analysis, differential item functioning, or DIF, was incorporated into the analysis to assess potential item bias across groups separated by ethnicity (i.e., Caucasian, African American, and Asian). Finally, Rasch PCA was used to support the component structure for the SPIN, and global statistics was reported to represent the data fit to the model with the current component structure.
ResultsInterprofessional Socialization and Valuing Scale
The ISVS showed adequate reliability (coefficient α = 0.94) across items. Therefore, this established that the ISVS questionnaire measured its intended construct. However, the inter-item correlation matrix confirmed the presence of components as correlation values fluctuated between factors.
Self-perceived ability to work with others
After 26 iterations, global fit statistics confirmed that the data were able to fit the model (P = 0.719). The item-person map revealed a large cluster of all 9 items between − 1.33 and 0.83 logits, while the majority of persons were clustered sustainably above the mean from 0 to 6 logits. Summary statistics revealed strong congruence and minimal model misfit between item Real Root Mean Square Error (RMSE) (0.16, SD = 0.63) and Model RMSE (0.16, SD = 0.64). RMSE is the mean of measurement errors.[21] Item separation (3.87) and reliability (0.94) were strong. Item separation is measure of how items spread across measured trait.[20] Infit and outfit are two statistics which show how accurately data fit the model,[22] and mean square fit values for infit and outfit values between 0.6 (i.e., 40% less variability than the model predicted) and 1.4 (i.e., 40% more variability than the model predicted) are accepted as reasonable for surveys.[23] Summarized item infit and outfit statistics indicated good average item infit (MNSQ = 1.00) and outfit (MNSQ = 1.04). Further investigation of item fit confirmed that all fit values were <2.0.[24]
Agreement was expressed between person Real RMSE (0.93, SD = 1.68) and Model RMSE (0.87, SD = 1.71). High person mean (3.18) and range values (−0.46-6.64) indicated high endorseability of items. Person separation (1.80) and reliability (0.76) were acceptable. Person raw score reliability (coefficient α) was strong at 0.94. Average person infit and outfit statistics, only reported for 104 nonextreme persons, revealed good person fit (MNSQ infit = 1.10; MNSQ outfit = 1.08); however, high maximum infit (MNSQ = 8.28; ZMNSQ = 4.6) and outfit (MNSQ = 7.94; ZMNSQ = 4.9) values were reported.
Category structure revealed high observed counts for positive categories; however, response Categories 13 accounted for just 4% of responses. Observed averages indicated issues between Categories 1 (m = 0.85), 2 (m = −1.31), and 3 (m = 0.40) because they do not increase monotonically. Infit and outfit MNSQ values did not excessively exceed 2. Threshold is the boundary of likelihood of selecting a response category that is equal to the likelihood of selecting next higher category[20] and Andrich threshold is the term when it applies to RSM. Andrich thresholds indicated that the category endorsement was not increasing monotonically; for example Andrich thresholds for Categories 2 (−1.67) and 3 (−1.69) were disordered and indicated minimal separation. The Category Probabilities Intersect Map confirmed this by showing no distinct category intersections or shapes for Categories 1-3.
Valuing in working with others
After twenty iterations, global fit statistics confirmed that the data were able to fit the model (P = 0.689). The item-person map revealed a large cluster of all 9 items between −0.50 and approximately 0.76 logits, while the majority of persons were clustered sustainably above the mean from 0 to 3.75 logits. Summary statistics revealed strong congruence and minimal model misfit between item Real RMSE (0.14, SD = 0.37) and Model RMSE (0.13, SD = 0.37). Item separation (2.64) and reliability (0.87) were strong. Summarized item infit and outfit statistics indicated good average item infit (MNSQ = 1.04) and outfit (MNSQ = 1.06). Further investigation of item fit confirmed that all fit values were <2.0.
Agreement was expressed between person Real RMSE (0.85, SD = 1.59) and Model RMSE (0.79, SD = 1.62). High person mean (2.20) and range values (−0.79-5.82) indicated high endorseability of items. Person separation (1.87) and reliability (0.78) were acceptable. Person raw score reliability (coefficient α) was strong at 0.90. Average person infit and outfit statistics, only reported for 107 nonextreme persons, revealed good person fit (MNSQ infit = 1.10; MNSQ outfit = 1.07); however, high maximum infit (MNSQ = 6.13; ZMNSQ = 5.2) and outfit (MNSQ = 6.79; ZMNSQ = 5.7) values were reported.
Category structure revealed high observed counts for positive categories; however, response Categories 1-3 accounted for just 8% of responses. Observed averages did not increase monotonically (0.47, 0.37, and 0.03 for Categories 1, 2, and 3, respectively) which indicated issues in number of categories. Infit and outfit values indicated Category 1 misfit (MNSQ outfit = 3.58). Andrich thresholds indicated that category endorsement was monotonically increasing. However, Andrich thresholds for Categories 2 (−1.14), 3 (−0.98), and 4 (−0.77) indicated minimal separation. The Category Probabilities Intersect Map confirmed this by showing no distinct category intersections or shapes for Categories 2, 3, and 4.
Comfort in working with others
After 24 iterations, global fit statistics confirmed that the data were able to fit the model (P = 0.605). The item-person map revealed a cluster of all 6 items between −0.5 and approximately 0.5 logits, while the majority of persons were clustered sustainably above the mean from 0.25 to 2.5 logits. Summary statistics revealed strong congruence and minimal model misfit between item Real RMSE (0.12, SD = 0.71) and Model RMSE (0.11, SD = 0.71). Item separation (5.85) was high and reliability (0.97) was strong. Items spanned the range of −0.701.51 logits with the expected mean of 0. Summarized item infit and outfit statistics indicated good average item infit (MNSQ = 0.98) and outfit (MNSQ = 1.04). Further investigation of item fit revealed that Item 24 had slightly high misfit values (MNSQ infit = 2.57; outfit = 2.95).
Agreement was expressed between person Real RMSE (0.68, SD = 1.19) and Model RMSE (0.60, SD = 1.22). High person mean (1.21) and range values (−1.41-5.33) indicated slightly high endorseability of items. Person separation (1.75) and reliability (0.75) were acceptable. Person raw score reliability (coefficient α) was strong at 0.81. Average person infit and outfit statistics, only reported for 116 nonextreme persons, revealed good person fit (MNSQ infit = 1.06; MNSQ outfit = 1.04); however, high maximum infit (MNSQ = 4.24; ZMNSQ = 3.7) and outfit (MNSQ = 7.44; ZMNSQ = 5.3) values were reported.
Category structure revealed that observed counts for positive categories accounted for 78% of responses. Observed averages indicated no issues between categories. Infit and outfit values indicated no category misfit. Andrich thresholds indicated that category endorsement was monotonically increasing. However, Andrich thresholds for Categories 3 (−0.64) and 4 (0.46) indicated minimal separation. The Category Probabilities Intersect Map confirmed this by showing no distinct category intersections or shapes between Categories 3 and 4.
Interprofessional Socialization and Valuing Scale differential item functioning analysis
Uniform DIF analysis was conducted independently for all items within each component [Table 2]. This analysis was comprised of four groups: Caucasian (n = 24), African American (n = 52), Asian (n = 38), and others (n = 5). Due to others' small sample size, their results were not included in this analysis. DIF contrasts greater than 0.50 when Rasch-Welch t-tests were significant, P < 0.05, were reported.[25] First, all t-value plots for each component were investigated for response differences on item that were greater than ± 1.96 SD from the mean.
Table 2: Interpersonal Socialization and Valuing Scale differential item functioning summaryPersonal Report of Communication Apprehension
The PRCA showed adequate reliability (coefficient α = 0.96) across items. Therefore, this established that the PRCA questionnaire measured its intended construct. However, the inter-item correlation matrix confirmed the presence of components as correlation values fluctuated between factors.
Group discussion
After 34 iterations, global fit statistics confirmed that the data were able to fit the model (P = 0.628). The item-person map revealed cluster of all 6 items between −0.87 and 0.70 logits, while the majority of persons were clustered below the mean from −3.0 to 0.50 logits. Summary statistics revealed strong congruence and minimal model misfit between item Real RMSE (0.16, SD = 0.52) and Model RMSE (0.15, SD = 0.53). Item separation (3.32) was high and reliability (0.92) was strong. Summarized item infit and outfit statistics indicated good average item infit (MNSQ = 0.95) and outfit (MNSQ = 1.02). Further investigation of item fit confirmed that all fit values were <2.0.
Agreement was expressed between person Real RMSE (0.94, SD = 1.98) and Model RMSE (0.82, SD = 2.04). Low person mean (−1.93) and range values (−7.183.15) indicated low endorseability of items. Person separation (2.11) and reliability (0.82) were acceptable. Person raw score reliability (coefficient α) was strong at 0.86. Average person infit and outfit statistics, only reported for 115 nonextreme persons, revealed good person fit (MNSQ infit = 1.02; MNSQ outfit = 1.02); however, high maximum infit (MNSQ = 8.92; ZMNSQ = 6.9) and outfit (MNSQ = 9.13; ZMNSQ = 6.6) values were reported.
Category structure revealed that negative categories, strongly disagree and disagree, accounted for 74% of responses. Observed averages indicated issues between Categories 4 (m = 0.93) and 5 (m = 0.89) which do not increase monotonically. Infit and outfit values indicated misfit with Category 5 (MNSQ infit = 3.18, outfit = 3.95). Andrich thresholds between Categories 2 and 3 (0.40) and Categories 3 and 4 (0.04) indicated that category endorsement was not monotonically increasing. Additionally, the Category Probabilities Intersect Map confirmed this by showing no distinct category 3 which may suggest that this category should be removed.
Meetings
After twenty iterations, global fit statistics confirmed that the data were able to fit the model (P = 0.636). The item-person map revealed a cluster of all 6 items between −0.59 and approximately 0.54 logits, while the majority of persons were clustered below the mean from −2.50 to 2.00 logits. Summary statistics revealed strong congruence and minimal model misfit between item Real RMSE (0.16, SD = 0.34) and Model RMSE (0.16, SD = 0.34). Item separation (2.13) was high and reliability (0.82) was strong. Summarized item infit and outfit statistics indicated good average item infit (MNSQ = 1.00) and outfit (MNSQ = 0.85). Further investigation of item fit confirmed that all fit values were <2.0.
Agreement was expressed between person Real RMSE (0.99, SD = 3.20) and Model RMSE (0.95, SD = 3.21). Low person mean (−1.68) and range values (−8.73-5.33) indicated easy endorseability of items. Person separation (3.24) and reliability (0.91) were acceptable. Person raw score reliability (coefficient α) was strong at 0.91. Average person infit and outfit statistics, only reported for 114 nonextreme persons, revealed good person fit (MNSQ infit = 0.84; MNSQ outfit = 0.85); however, high maximum infit (MNSQ = 5.64; ZMNSQ = 3.4) and outfit (MNSQ = 6.54; ZMNSQ = 3.4) values were reported.
Category structure again revealed that negative categories, strongly disagree and disagree, accounted for the majority of the data – 63% of responses. Observed averages showed no major category issues (i.e., monotonically increasing). Infit and outfit values indicated no major category issues. Andrich thresholds between Categories 2 and 3 (0.83) and between Categories 3 and 4 (0.30) are disordered. The Category Probabilities Intersect Map confirmed this by showing no distinct category 3 or intersections between categories 2 and 3 and 4.
Interpersonal
After 31 iterations, global fit statistics confirmed that the data were able to fit the model (P = 0.689). The item-person map revealed a cluster of all 6 items between–0.89 and approximately 0.70 logits, while the majority of persons were clustered below the mean from − 0.50 to 0.50 logits. Summary statistics revealed strong congruence and minimal model misfit between item Real RMSE (0.16, SD = 0.46) and Model RMSE (0.15, SD = 0.46). Item separation (2.84) was high and reliability (0.89) was strong. Summarized item infit and outfit statistics indicated good average item infit (MNSQ = 0.95) and outfit (MNSQ = 1.04). Further investigation of item fit confirmed that all fit values were <2.0.
Agreement was expressed between person Real RMSE (1.00, SD = 2.28) and Model RMSE (0.88, SD = 2.33). Low person mean (−1.78) and range values (−7.10-3.08) indicated easy endorseability of items. Person separation (2.28) and reliability (0.84) were acceptable. Person raw score reliability (coefficient α) was strong at 0.89. Average person infit and outfit statistics, only reported for 110 nonextreme persons, revealed good person fit (MNSQ infit = 1.04; MNSQ outfit = 1.05); however, high maximum infit (MNSQ = 8.76; ZMNSQ = 7.0) and outfit (MNSQ = 9.68; ZMNSQ = 6.8) values were reported.
Category structure again revealed that negative categories, strongly disagree and disagree, accounted for the majority of the data – 70% of responses. Observed averages indicated a major issue between Categories 4 (m = 1.16) and 5 (m = 0.55). Infit and outfit values indicated misfit with Category 5 (MNSQ infit = 3.28, outfit = 5.31). Andrich thresholds indicated that category endorsement was not monotonically increasing and, Andrich thresholds between Categories 2 and 3 (0.45) and 3 and 4 (0.02) indicated minimal to no separation and disordered. The Category Probabilities Intersect Map confirmed this by showing no distinct category 3 or intersections between Categories 3 and 4.
Public speaking
After nineteen iterations, global fit statistics confirmed that the data were able to fit the model (P = 0.561). The item-person map revealed a cluster of all 6 items between −0.47 and approximately 0.62 logits, while the majority of persons were clustered around the mean from −0.50 to 1.00 logits. Summary statistics revealed strong congruence and minimal model misfit between item Real RMSE (0.13, SD = 0.39) and Model RMSE (0.12, SD = 0.39). Item separation (3.11) was high and reliability (0.91) was strong. Summarized item infit and outfit statistics indicated good average item infit (MNSQ = 0.99) and outfit (MNSQ = 1.07). Further investigation of item fit confirmed that all fit values were <2.0.
Agreement was expressed between person Real RMSE (0.75, SD = 1.58) and Model RMSE (0.63, SD = 1.63). Slightly low person mean (−0.10) and negative range values (−5.86-4.43) indicated easy endorseability of items. Person separation (2.12) and reliability (0.82) were acceptable. Person raw score reliability (coefficient α) was strong at 0.86. Average person infit and outfit statistics, only reported for 118 nonextreme persons, revealed good person fit (MNSQ infit = 1.08; MNSQ outfit = 1.08); however, high maximum infit (MNSQ = 7.34; ZMNSQ = 4.4) and outfit (MNSQ = 8.03; ZMNSQ = 4.9) values were reported.
Category structure seemed mesokurtic (i.e., similar to normal curve) in that the percentages of endorse categories appeared to follow and equal curve. There were no observed average issues revealed. Infit and outfit values did not indicate any misfit. Andrich thresholds indicated that category endorsement was not monotonically increasing. The thresholds between Categories 2 and 3 (0.56) and 3 and 4 (−0.46) also indicated minimal to no separation. The Category Probabilities Intersect Map confirmed this by showing no distinct category 3.
PRCA differential item functioning analysis is shown in [Table 3].
Table 3: Personal Report of Communication Apprehension differential item functioning summarySocial Phobia Inventory
The SPIN showed adequate reliability (coefficient α = 0.91) across items. Therefore, this established that the SPIN questionnaire measured its intended construct. However, the inter-item correlation matrix presented very low correlations for items measuring the same component. For example, Psychological discomfort items were not strongly correlated (r < 0.42). See [Table 4].
Fear
After 17 iterations, global fit statistics confirmed that the data were able to fit the model (P = 0.74). The item-person map revealed a cluster of all 7 items between −0.50 and approximately 0.50 logits, while the majority of persons were clustered below the mean from −2.00 to 0.00 logits. Summary statistics revealed strong congruence and minimal model misfit between item Real RMSE (0.15, SD = 0.40) and Model RMSE (0.15, SD = 0.40). Item separation (2.72) was high and reliability (0.88) was strong. Items spanned the range of −0.65-0.43 logits with the expected mean of 0. Summarized item infit and outfit statistics indicated good average item infit (MNSQ = 0.99) and outfit (MNSQ = 1.08). Further investigation of item fit confirmed that all fit values were <2.0.
Agreement was expressed between person Real RMSE (1.00, SD = 1.54) and Model RMSE (0.94, SD = 1.57). Low person mean (−2.05) and range values (−5.32-1.43) indicated easy endorseability of items. Person separation (1.54) and reliability (0.70) were acceptable. Person raw score reliability (coefficient α) was strong at 0.83. Average person infit and outfit statistics, only reported for 102 nonextreme persons, revealed good person fit (MNSQ infit = 1.01; MNSQ outfit = 1.08); however, high maximum infit (MNSQ = 4.46) and outfit (MNSQ = 7.78) values were reported.
Category structure revealed that negative categories, not at all and a little bit, accounted for the majority of the data – 83% of responses. Observed averages showed no major category issues. Infit and outfit values indicated no category issues. Andrich thresholds indicated that category endorsement was monotonically increasing. However, Andrich thresholds for Category 3 (null) indicated minimal to no separation. The Category Probabilities Intersect Map confirmed this by showing no distinct category 3.
Avoidance
After twelve iterations, global fit statistics confirmed that the data were able to fit the model (P = 0.598). The item-person map revealed a large cluster of all 6 items between − 1.00 and approximately 1.00 logits, while the majority of persons were clustered below the mean from −2.00–0.00 logits. Summary statistics revealed strong congruence and minimal model misfit between item Real RMSE (0.13, SD = 0.63) and Model RMSE (0.13, SD = 0.64). Item separation (4.74) was high and reliability (0.96) was strong. Items spanned the range of −0.91-0.83 logits with the expected mean of 0. Summarized item infit and outfit statistics indicated good average item infit (MNSQ = 1.00) and outfit (MNSQ = 1.05). Further investigation of item fit confirmed that all fit values were <2.0.
Agreement was expressed between person Real RMSE (0.82, SD = 1.10) and Model RMSE (0.76, SD = 1.52). Low person mean (-1.54) and range values (−4.72-2.11) indicated easy endorseability of items. Person separation (1.34) and reliability (0.64) were less than ideal. Person raw score reliability (coefficient α) was strong at 0.77. Average person infit and outfit statistics, only reported for 111 nonextreme persons, revealed good person fit (MNSQ infit = 1.05; MNSQ outfit = 1.05); however, high maximum infit (MNSQ = 4.57) and outfit (MNSQ = 7.28) values were reported.
Category structure revealed that negative categories, not at all and a little bit, accounted for the majority of the data −75% of responses. Observed averages showed no major category issues. Infit and outfit values indicated no category issues. Andrich thresholds indicated that category endorsement was monotonically increasing. However, Andrich thresholds for Category 3 (1.59) indicated minimal to no separation between Category 4. The Category Probabilities Intersect Map confirmed this by showing no distinct Category 3.
Physiological discomfort
After six iterations, global fit statistics confirmed that the data were able to fit the model (P = 0.840). The item-person map revealed a cluster of all 3 items between −0.20 and approximately 0.40 logits, while the majority of persons were clustered below the mean from −0.50 to −3.00 logits. Summary statistics revealed strong congruence and minimal model misfit between item Real RMSE (0.13, SD = 0.20) and Model RMSE (0.13, SD = 0.20). Item separation (1.51) and reliability (0.70) were acceptable. Summarized item infit and outfit statistics indicated good average item infit (MNSQ = 0.99) and outfit (MNSQ = 0.99). Further investigation of item fit confirmed that all fit values were <2.0.
Agreement was expressed between person Real RMSE (1.11, SD = 1.12) and Model RMSE (1.06, SD = 1.16). Low person mean (−1.74) and range values (−4.26-1.87) indicated easy endorseability of items. Person separation (1.01) and reliability (0.64) were less than ideal. Person raw score reliability (coefficient α) was low at 0.68. Average person infit and outfit statistics, only reported for 93 nonextreme persons, revealed good person fit (MNSQ infit = 0.98; MNSQ outfit = 0.99); however, high maximum infit (MNSQ = 4.37) and outfit (MNSQ = 4.66) values were reported.
Category structure revealed that negative categories, not at all and a little bit, accounted for the majority of the data –75% of responses. Observed averages showed no major category issues. Infit and outfit values indicated no category issues. Andrich thresholds indicated that category endorsement was monotonically increasing. However, Andrich thresholds for Category 2 (0.01) indicated minimal to no separation between Category 3 (0.17). The Category Probabilities Intersect Map confirmed this by showing no distinct Category 2.
DiscussionThe ISVS results indicated that overall this sample had a relatively high level of the latent traits – self-perceived ability to work with others, value in working with others, and comfort in working with others. Across all three traits, all persons were placed above the mean, indicating that this sample had above-average ability, value, and comfort in working with others. However, some issues were revealed in the category structure analysis. Due to the high level of positive responses, several categories were not endorsed substantially. For example, response categories 2, 3, and 4 (two negative responses and the neutral option) had no category structure or presence. This could be due to the lack of negative responses in general. Linacre suggests that each category should have at least ten observations and the distance between two consecutive thresholds should be at least 1.4 logits.[26] Therefore, in order to improve the measure, negative categories should be collapsed. In addition, the neutral option could also be removed in continued administrations of the ISVS.
DIF analysis of the ISVS indicated several group differences. Interestingly, across most significant items, African Americans performed higher than Asians which means that African American students have higher levels of interprofessional socialization than Asian students. DIF analysis showed that African Americans and Asians oppositely endorsed these items. For instance, on Item 3 (i.e., I have gained a better understanding of my own approach to care within an interprofessional team), African Americans responded three times above the mean, whereas Asians were two times below the mean. This pattern held across items with significant differences. Since these differences were present across all three components of the ISVS, it can be said that Asians were less positive in their endorsements. However, as a reminder, overall all groups/persons scored positively above the mean on the item-person map.
Next, the ISVS DIF analysis also indicated some differences between Caucasian and African American groups on perceived ability to work with others. On Item 3, Caucasian students were not as positive about their self-perceived ability to work with others. Caucasian students also performed higher than Asian students on all three components. Additionally, differences between African American and Asian groups appeared strong for the items assessing comfort and value in working with others where African American students scored higher than Asian students. For example, responses to “I believe that interprofessional practice is difficult to implement” indicated that the Asians were significantly less positive in their responses. The Rasch and DIF analyses revealed that specific questions had racial differences in responses. If using scales such as the ISVS to assist in evaluating competencies within interprofessional educational activities, especially communication,[27] researchers must be aware of the racial social construct differences that may impact the results.
The PRCA analysis indicated low presence of communication apprehension within this sample. For all four components, items were difficult to endorse, but persons were largely below the mean. The majority of low responses (strongly disagree and disagree) were confirmed in the category structure analysis. The lack of positive responses created some issues with category structure, especially between Categories 3 (neutral) and 4 (agree). Overall, within this sample, the level of communication anxiety in group discussions, meetings, interpersonal situations, and public speaking was low. Adaptations to this instrument could include expanding to a 7-point Likert scale, with additional research and psychometric analysis. Alternatively, higher responses (Categories 4 and 5) could be collapsed, or the neutral option could be removed from future administrations to improve the category structure.
The DIF analysis of the PRCA revealed several significant differences between groups across components. Caucasian students appeared to have significantly different responses to group discussion questions. This indicated that the Caucasian students felt more comfortable with group discussions; however, they were not as relaxed during discussions as African American and Asian (Item 6). This indicates that Caucasian students had a lower level of anxiety than the other groups. However, again, it should be noted that all groups reported low levels of anxiety. Conversely, Caucasian students reported being more relaxed answering questions during meetings than African American students. Interpersonal items revealed a stark contrast between Asian and African American students. For the item “While participating in conversation with a new acquaintance, I feel very nervous,” Asian students indicated greater feelings of anxiety. Finally, differences in public speaking comfort were seen between Caucasian and Asian students. In this case, Asian students reported higher anxiety level than Caucasian students. When looking specifically at communication, an essential component of health-care teams,[27] apprehension can hinder communication which would ultimately affect care. Understanding there may be cultural differences on specific questions about communication survey instruments must be taken into account in research within this area.
Considering the SPIN results further indicated major difference between the demographics of this sample of pharmacy students. There were significant differences among all groups on fear of authority which could have a direct impact in the classroom and the work place. Additionally, African American students have lower levels of social anxiety compared to Asian and Caucasian students in all components. The results indicated that at least a part of social phobia and anxiety are cultural and somewhat inherent.
Potential changes to these measures may enhance the applicability of the scales to all groups. Further studies with these enhancements should be conducted. The DIF for each scale revealed specific questions could be answered differently depending on the racial group. Most importantly, these results should be taken into consideration by educators and curricular developers who are trying to prepare these pharmacy students for practice. Looking at the specific behaviors and feelings of each demographic group could lead to best practices for each. For instance, African American students indicated higher anxiety in meetings than their classmates. This information could be used to create more opportunities for these students to simulate real meeting situations. As health professions schools expand on interprofessional education, racial differences and evaluation of assessment measures, such as the IVIS, should be taken into consideration and a deep dive into the differences within the scales used for their population. This would aid in studying further the racial differences and ultimately inform educational activities. As this study was done within the United States, results should not be extrapolated to other countries without further evaluation of racial differences globally.
Financial support and sponsorship
Nil.
Conflicts of interest
There are no conflicts of interest.
References
Comments (0)