Disruptive clinician behavior can damage interpersonal relationships and impair communication, information transfer, and teamwork, all of which seriously affect patient safety.1 Unfortunately, many health care workers in the medical field are victims of such behaviors.2 A book published in 2000 called To Err Is Human highlighted the importance of patient safety worldwide. That book increased interest in disruptive clinician behavior in U.S. health care institutions as a negative influence on the recruitment and retention of medical staff and the quality and safety of patient care.3 The Joint Commission, which accredits health care facilities and programs across the United States, requires facility directors to develop and implement processes to manage disruptive and inappropriate staff behavior.4 In 2008, the Joint Commission published a “Sentinel Event Alert” newsletter to inform the public about the harmful effects of disruptive clinician behavior.5 In that newsletter, the Joint Commission described zero-tolerance and clear code-of-conduct policies regarding disruptive clinician behavior as a threat to patient safety.6 Therefore, efforts to eliminate disruptive clinician behavior are actively underway in the United States. However, many medical professionals and institutions do not adequately deal with offenders and victims because of a lack of knowledge or passivity.7 As a result, disruptive clinician behavior remains a deep-seated problem that continues to negatively affect medical care and safety.

The first step in preventing or eliminating disruptive behavior is accurately understanding the actual state of such behavior. Several studies have attempted to identify types of disruptive behavior.8–11 Most of these studies used qualitative analysis based on the content of narratives obtained from interviews. However, it is difficult to eliminate the classifier’s subjectivity in content analysis. For example, at the conceptual level, “incivility” was not consistent across studies and was sometimes treated as synonymous with disruptive behavior.12,13 Rosenstein and Naylor14 described disruptive behavior as encompassing concepts such as incivility, harassment, and violence. In addition to incivility and violence, psychological aggression is generally used as a category of disruptive behavior. Other studies included job-related categories such as harassment and passive-aggressive behavior.15–19 These studies highlighted that the conceptual range of disruptive behavior was not constant. In addition, a threat was often categorized as psychological aggression8,15,17 and as bullying/harassment.20 Therefore, qualitative analyses produce variations in levels, categories, and the items encompassed in categories depending on the subjectivity of the classifier.

Quantitative research using psychometric scales offers a practical alternative that complements qualitative research. However, no generic scale for disruptive clinician behaviors is currently available. A possible reason for this is ambiguity of the definition of this behavior. Intrinsically, disruptive clinician behavior refers to a range of inappropriate behaviors among victims and perpetrators. Therefore, researchers’ focus varies across studies. However, destructive clinician behavior can be sequentially conditioned by whether or not it is physical, involves sexual assault, and is malicious. Incivility, which is a nonphysical, nonsexual, and nonmalicious form of disruptive behavior, has been the subject of numerous studies. Examples of this behavior include ignoring someone, yelling, interrupting someone who is speaking, spreading rumors, and taking credit for work that someone else has done.21 To grasp the detailed reality of this situation, a generic psychometric scale that covers a range of disruptive clinician behaviors is required. However, existing scales are limited in terms of targets and behaviors.

Guidroz et al21 developed a scale that was limited to incivility toward nurses and focused on the type of perpetrator. Only one factor reflecting incivility for each perpetrator type was extracted, except for other nurses, although several factors common to perpetrators were extracted, and there were high correlations among “general,” “physician,” and “nurse” incivility factors. Another measure that is limited to nurses is the Johns Hopkins Disruptive Clinician Behavior Scale,16 which comprehensively addresses the process of generating disruptive clinician behavior from cause to impact from a meta-perspective. Although this is an excellent approach, it cannot grasp specifics because destructive clinician behavior is treated as a single factor. The Student Experience of Bullying During Clinical Behavior scale22 used by Budden et al23 is also limited to nursing students and only includes 3 factors: physical abuse, sexual harassment, and non-violent behavior. Similarly, another scale developed by Anthony et al24 targeted nursing students, and the analysis consolidated 2 factors: hostile/rude/dismissive and exclusionary.

In terms of factor identification, the aforementioned scales only identified a single factor or a small number of factors as disruptive clinician behaviors, all of which corresponded to categories and were classifications of behaviors. Grounded Theory Approach, which is qualitative analysis methods, assumes a hierarchical structure and establishes specific subcategories under each category. Therefore, in scale development, specific behaviors (subcategories) that constitute factors (categories) should be identified through hierarchical factor analysis to grasp the detailed reality. Furthermore, during daily medical care and management, all staff working in an organization can be either victims or perpetrators of inappropriate behaviors that harm patient safety, regardless of whether or not they intend to do so. Therefore, a generic psychometric scale that is not limited to victims or perpetrators is needed.

In its development process, a psychometric scale should follow a scale construction method based on the psychological paradigm using statistical analysis. In the context of disruptive clinician behavior, specific procedures are as follows: (1) cases are collected through focus groups or open-ended questionnaires with several victims and witnesses; (2) disruptive clinician behavior is extracted from these cases, and typical behaviors are identified as items through content analysis and other methods; (3) using the items obtained, an a priori Likert scale is developed, and victims are asked to rate the items; (4) in addition to data from victims’ subjective assessments, exploratory factor analysis is used with maximum likelihood and Promax rotation (the recommended approaches). This method assumes that factors are latent behind multiple items (behaviors) and have a common influence on these behaviors. In case of reports of damage from disruptive behaviors, behaviors that tend to appear together in a given case form a common factor or category.

Psychometric scales must be validated and reliable. Although there are various checklists21,22 and scales23,24 that can measure disruptive behavior, a previous study noted that few scales have been validated and shown to be reliable using multiple criteria.25 That study suggested carefully following several steps and performing a detailed analysis of the validity and reliability of the scales to create a scale with high validity and reliability. However, in that study, the item collection was theory based and incomplete, and the constructs were multifactorial. In addition, the factors identified were types of exposure and response, and not types of disruptive behavior. This means that it was difficult to accurately grasp the reality of the disruptive behavior experienced by the reporter. Therefore, this study referenced and revised the validity and reliability checking process described by Villafranca et al,25 which involved exhaustively collecting items and identifying latent factors. To investigate disruptive clinician behaviors in a comprehensive and systematic manner, it is necessary to collect items based on victims’ voices using a bottom-up approach. The present study was based on Villafranca’s methodology25 but adopted an entirely different approach from the conceptualization and operationalization described in that method in the item collection step. In addition, many existing measures of disruptive behavior use a response format that recalls nonequally spaced frequencies (e.g., several times a year, once a month, daily), which makes it difficult to accurately reflect the frequency.26 In addition, disruptive behavior affects the victims psychologically and influences their interpersonal relationships.27–29 Therefore, it is important to grasp a real sense of how persistently victims perceive they were subjected to disruptive behaviors. Therefore, the rating method should use interval rating stages commonly used in frequency rating (i.e., never, rarely, occasionally, often, frequently, always).

As a first step in understanding the reality of disruptive clinician behavior, it is essential to quantify the victim’s actual perceptions by measuring subjective damage ratings using a uniform standard. This allows detailed categorical measurement of subjective victimization by the types of behavior and degree to which victims feel exposed to destructive clinician behavior. It can thereby accurately quantify a victim’s sense of harm from destructive behavior. In addition, a generic scale that is not application-specific will facilitate comparative studies across different databases. Therefore, this study aimed to develop an exhaustive and systematic psychometric scale to ascertain victims’ subjective evaluations of their exposure to disruptive behavior.

METHODS

We conducted 2 Internet surveys and 1 field survey using a questionnaire to develop a psychometric scale to quantify victims’ perceptions of disruptive behavior and tested the psychometric properties of the developed scale. This study was approved by the relevant research ethics board. We obtained informed consent from all survey participants.

Step 1: Open-Ended Questionnaire

An open-ended questionnaire was used to comprehensively collect items related to disruptive clinician behaviors. We targeted nurses because they are considered the most frequently exposed to disruptive behaviors based on their positions and composition (nurses, including assistant nurses, account for 43.7% of the medical staff in Japan).30 This survey was conducted using the MyVoice Inc Internet survey service, which is a company with a proven track record in marketing and survey research in Japan.

Step 2: Content Analysis

The first author (a specialist in interpersonal behavior) conducted preliminary coding and drafted codes and categories. Next, the second author (registered nurse with practical experience as a head nurse and nursing faculty member) conducted further free-text coding based on the first author’s drafts. If any improvements were identified, the first and second authors modified the codes and categories through discussion. Finally, the third author (internist and full-time medical safety manager) inspected the coding system and results and finalized the codes and categories for disruptive clinician behaviors.

Step 3: Content Validity Assessment

To assess the content validity of the identified items,31 5 full-time medical safety managers from a 751-bed acute care hospital rated each item using a 4-point scale (1, not relevant; 2, somewhat relevant; 3, quite relevant; 4, highly relevant) based on whether that item was considered relevant to the construct. Items that all raters gave a rating of 3 or 4 were judged to be relevant.

Step 4: Internet Questionnaire Survey

As in step 1, we used MyVoice Inc to disseminate a questionnaire survey to staff working in medical institutions throughout Japan. The survey used the items for disruptive clinician behavior collected and screened in steps 1 to 3. The questionnaire comprised a case situation followed by responses to items (codes) on a 6-point Likert scale (1, never; 2, rarely; 3, occasionally; 4, often; 5, frequently; 6, always).

Step 5: Exploratory Factor Analysis and Subfactor Analysis

Exploratory factor analysis was repeated until each item loading on a single factor was more than 0.300 or less than −0.300 to identify a simple structure, excluding disqualified items. In addition, subfactor analysis was conducted for the factors on which many items were loaded. In both analyses, we determined the number of factors based on the variation in eigenvalues (scree criteria) and cumulative contribution using maximum likelihood and Promax rotation.

Step 6: Item Response Analysis

Item response theory was used to confirm each item’s difficulty and discriminative power. The difficulty level represented the ease of answering a question correctly. Options corresponding to the difficulty level that returned a significant positive value were considered more likely to be selected. The discriminant power referred to whether the items identified the academic ability of the respondent; the higher the item’s score, the more likely it was that only respondents with a certain level of intellectual knowledge would answer correctly.

Step 7: Field Questionnaire Survey

Because the Internet survey targets health care providers nationwide, it collected mass data that were not influenced by a particular organizational culture. However, the population was biased because of the use of nonprobability sampling. Therefore, to obtain a more reliable population, we also surveyed employees from one medical institution before examining factor levels to clarify the usefulness of the developed scale and identify areas that should be expanded. We administered a field questionnaire survey to staff working at a 751-bed acute care general hospital in Japan, using the items finalized in step 5. The design of the questionnaire was the same as in step 4, except for the number of items. Subsequently, steps 8 and 9 (factor level) were used to analyze the Internet and field survey data.

Step 8: Internal Consistency Analysis

We verified the internal consistency by calculating the ω coefficient for each identified factor.

Step 9: Confirmatory Factor Analysis

Confirmatory factor analysis was used to test the hypothesized model’s goodness of fit. Previous studies on disruptive clinician behavior may be divided into those limited to interpersonally disruptive behavior27 and those including job-related disruptive behavior.32 In terms of interpersonal disruptive behaviors, physical violence constantly forms a separate category because it is considered severe and explicit disruptive behavior. Therefore, disruptive behavior may have a spectrum of severity (which would be identified by the difficulty level in step 6). This means that we placed insidious and minor disruptive behaviors (e.g., neglect) at the opposite pole to physical violence. In contrast, job-related disruptive behaviors can be classified according to the difference between higher and lower positions, for example, excessive management and guidance from superiors and intentional passive aggression by subordinates. Factors that contained many codes were subdivided into more specific disruptive behaviors (step 4). Bifactor analysis33 was used to test the subfactor model for these more complicated relationships. In these cases, the correlational passes between individual subfactors were set based on the severity and content of the disruptive behavior.

RESULTS Step 1: Open-Ended Questionnaire

MyVoice Inc invited nurses registered in its panel to participate by email. In response to this invitation, 712 individuals voluntarily accessed a dedicated website. Participants’ sociodemographic characteristics are shown in Table 1. Data for 478 participants (67.13%) who had witnessed or been a victim of disruptive clinician behavior were used in our analyses.

TABLE 1 - Participants’ Demographics Step 1 Step 4 Step 7 Physician 0 91 20 Nurse 712 455 39 Paramedic 0 387 21 Nonmedical professional 0 67 5 Female 653 737 49 Male 59 263 35 Unanswered 0 0 1 Age 20s, y 175 163 15  30s 189 338 21  40s 174 270 26  50s 174 188 17  60s 0 41 5  Unanswered 0 0 1 Clinic 122 248 0 Hospital (<100 beds) 58 142 0 Hospital (100–299 beds) 155 174 0 Hospital (300–499 beds) 104 204 0 Hospital (>500 beds) 109 232 85 Others 164 0 0
Step 2: Content Analysis

The 708 disruptive clinician behaviors described in the open-ended questionnaire responses were coded and classified based on their content (Table 2). This identified 47 codes as specific disruptive behaviors, which were classified into 8 categories: incivility, harassment, abusive language, physical violence, intimidation, passive aggression, instruction, and nonsupportive coercion.

TABLE 2 - Number and Percentage of Behaviors Identified in the Content Analysis No Items % Incivility 90 12.71  1 Talked behind others’ backs 22 3.11  2 Said things sarcastically 12 1.69  3 Tattled on others 4 0.56  4 Spoke ill of others 25 3.53  5 Submitted others to derision 6 0.85  6 Insulted others 21 2.97 Harassment 92 12.99  7 Altered their attitude from person to person 9 1.27  8 Made no response 3 0.42  9 Hung up the phone during a conversation 5 0.71  10 Ignored others 40 5.65  11 Harassed others 15 2.12  12 Bullied others 20 2.82 Abusive language 95 13.42  13 Behaved insensitively 17 2.40  14 Behaved in a discriminatory manner 25 3.53  15 Talked abusively 53 7.49 Physical violence 70 9.89  16 Vented their anger on someone or something 14 1.98  17 Nudged with the hand or foot 6 0.85  18 Threw objects at someone 15 2.12  19 Used violence against others 35 4.94 Intimidation 100 14.12  20 Expressed discomfort or disgust 23 3.25  21 Glared at others 3 0.42  22 Overawed others 22 3.11  23 Yelled at others 39 5.51  24 Threatened others 3 0.42  25 Intimidated others 10 1.41 Passive aggression 81 11.44  26 Neglected their duties 16 2.26  27 Adopted an uncooperative posture 13 1.84  28 Contravened an overall agreement 6 0.85  29 Provided inadequate treatment for the patient 22 3.11  30 Did not give the necessary information 5 0.71  31 Gave inappropriate and unreasonable instructions 9 1.27  32 Set policies without any explanation or consultation 10 1.41 Instruction 138 19.49  33 Treated someone with favoritism 10 1.41  34 Did not provide the required guidance 17 2.40  35 Convened unnecessarily 7 0.99  36 Provided inappropriate guidance 7 0.99  37 Talked harshly 16 2.26  38 Gave others a reprimand 25 3.53  39 Criticized others in public 25 3.53  40 Persistently scrutinized mistakes 13 1.84  41 Criticized care 3 0.42  42 Denied others’ opinions unilaterally 15 2.12 Nonsupportive coercion 32 4.52  43 Shifted the blame for negligence 9 1.27  44 Enforced participation in fraud 3 0.42  45 Coerced others into doing unreasonable work 10 1.41  46 Did not provide the necessary work support 7 0.99  47 Did not assign other jobs 3 0.42 Others 10 1.41  Made false accusations 2 0.28  Sexually harassed someone 2 0.28  Stolen property 2 0.28  Made improper entries in charts and documents 2 0.28  Made false reports 2 0.28
Step 3: Assessment of Content Validity

Content experts rated the relevance of individual items, and content validity indices were calculated. We excluded 2 items rated as 1 (not relevant) or 2 (somewhat relevant) in this rating process.

Step 4: Internet Questionnaire Survey

Health care staff for this survey were recruited using the same procedures as in step 1. Responses to the survey were closed when the number of participants with victimization experience reached 1000. Participants’ sociodemographic characteristics are shown in Table 1.

Step 5: Exploratory Factor Analysis and Subfactor Analysis

Exploratory factor analysis was repeated until a simple structure was obtained. We removed 7 items and identified 6 factors (Table 3). The first factor was “psychological aggression,” on which 15 items were loaded that were mainly categorized in the preliminary survey coding as abusive language, intimidation, and reproof. Factor 2 was “mismanagement practice,” which included 10 items classified as instruction and nonsupportive coercion in the coding system. The third and subsequent factors corresponded one-to-one with the categories in the coding system. Factor 3 was incivility with 5 items, factor 4 was “passive aggression” with 4 items, factor 5 was “ignoring” (e.g., ignoring coworkers) with 3 items, and factor 6 was “physical violence” with 3 items. The correlations between the factors were positive (r = 0.339–0.618), except for low correlations between physical violence and 3 factors: incivility, passive aggression, and ignoring (r = 0.111–0.194).

TABLE 3 - Factors and Item Factor Loadings No Items Exploratory Factor Analysis Subfactor Analysis F1 F2 F3 F4 F5 F6 F1 F2 F3 F4 Factor 1: Psychological aggression  Subfactor 1–1: Intimidation  22 Overawed others 0.851 −0.018 −0.119 −0.054 0.100 −0.029 0.714 0.060 0.168 −0.058  20 Expressed discomfort or disgust 0.564 −0.137 0.004 0.180 0.211 −0.065 0.706 −0.059 −0.145 0.245  21 Glared at others 0.581 −0.001 0.050 −0.036 0.199 0.078 0.638 0.039 0.171 −0.035  37 Talked harshly 0.868 −0.079 0.005 −0.098 0.076 −0.188 0.537 0.360 −0.083 0.010  Subfactor 1–2: Reproof  39 Criticized others in public 0.754 0.126 0.131 −0.086 −0.135 −0.068 −0.018 0.837 0.039 0.030  40 Persistently scrutinized mistakes 0.605 0.236 0.136 −0.089 −0.061 −0.070 0.013 0.788 −0.013 0.045  38 Gave others a reprimand 0.850 0.143 −0.041 −0.185 −0.092 0.016 0.044 0.504 0.447 −0.112  42 Denied others’ opinions unilaterally 0.515 0.250 0.065 0.169 −0.100 −0.152 0.043 0.480 −0.064 0.337  Subfactor 1–3: Threats