1 INTRODUCTION

The organization of health services should always consider the evaluation process of the condition in which health care is provided. This evaluation enables the identification of problems, and their results can guide healthcare managers and workers in their decision-making in an attempt to improve the quality of the provided care. This process can take place through the use of monitoring indicators and evaluation research. All of the actors involved in the healthcare process should participate in the evaluation research, including service users.1-3

The evaluation of health services from the perspective of the user, in addition to including social participation in the planning and decision-making processes, enables the identification of problems and the obtaining of information from the viewpoint of those who truly take advantage of health care and those for which the service is intended.4, 5 Within the realm of oral health, user satisfaction has been a widely used measure of results in studies on healthcare quality at the primary6, 7 and secondary care levels.5, 8, 9 Factors related to the sociodemographic profile of the user, such as factors related to the organization of the service, infrastructure, management, access, continuation of care6, 8, 10 and communication of the dental professional with the patient10, 11 have been associated with user satisfaction. In the context of the expansion of oral healthcare services in Brazil, the monitoring and evaluation of the quality and determining factors are essential in order to improve actions and services. In Brazil, secondary care is provided at Dental Specialty Centers (CEO, in Portuguese), established in the guidelines set forth by the National Oral Health Policy.12 Considering the Donabedian model, the most well-structured CEO, different conditions through which to seek out services, and user access, as well as the organization of the healthcare teams' work process, may well aid in determining different levels of user satisfaction regarding the services provided by the CEO. This study, therefore, investigated the association between user satisfaction with the services provided by the CEO in relation to the CEO’s structure and work processes.

2 METHOD

The evaluation of the CEO is conducted by the National Program for the Improvement of Access to and Quality of Dental Specialty Centers (PMAQ-CEO, in Portuguese). The PMAQ-CEO was set up in 2014 by the Ministry of Health in an attempt to evaluate the quality of the public healthcare services specialized in oral health.13

The PMAQ-CEO adopted the Donabedian theoretical model, considering the dimensions of structure, process and result in evaluating the quality of services provided by the CEO.14-16 The structure corresponds to the material resources, human resources and organizational structure. The process denotes how the user searches for health care and how this process takes place. The result represents the effects of care on one's state of health, including the level of knowledge, behaviour and user satisfaction.14, 15 Although many studies that evaluate oral healthcare services refer to the Donabedian conceptual model,2, 17, 18 few evaluate the three dimensions of quality and their relationships with user satisfaction.19-21

The conceptual theoretical model of this study was based on Donabedian15 and Swan et al.,22 considering the process, structure and result in the assessment of the quality of health services (Figure 1).15

Theoretical model of analysis based on Donabedian15 and Swan et al.22 *Equipment, Supplies, Instruments, and Ambience items were shown in the Tables S6–S8. Source: Adapted from Donabedian15 and Swan et al.22

This nationwide study is based on secondary data from two evaluation cycles of all Brazilian CEOs, conducted in 2014 (cycle 1) and 2018 (cycle 2).

In both evaluation cycles, data collection was conducted through a questionnaire divided into three modules: model I—observation of the structure of the CEO, model II—interview with managers or dentist to collect information about work processes and the organization of services and model III—interview with users to assess the self-perception of healthcare quality and satisfaction with secondary dental care services.14 The interviews and observations were conducted by a team of 85 evaluators without a link to the service, which had previously undergone training at education and research institutions through qualification training and instructive manuals. The interviews were conducted in a silent and calm location, according to the reality of each evaluated CEO. The managers were obliged to answer the interview questions, and the dentists that were available on the day of the evaluation were selected to participate in the study.

For each CEO, ten users were interviewed. This sample of users was defined by convenience, considering the feasibility of finding the service user during data collection. The users should be over 18 years of age, be undergoing treatment at the CEO and be present on the day of the evaluation. The interviewed individuals were not identified by name in order to preserve confidentiality.

2.1 Outcome measure

The outcome measure was user satisfaction, which was analysed based on the four-stage model proposed by Swan,22 which comes close to the quality standards adopted in the PMAQ-CEO. According to this model, satisfaction is defined by the following constructs: perception of service performance, confirmation of expectations regarding performance and perception of treatment, and overall satisfaction. Five quality standards were selected to evaluate user satisfaction: self-rating of CEO facilities, self-rating of dental care received, feeling of comfort during the dental appointment, satisfaction with care received by the dentist and satisfaction with the service rendered by the reception staff. The answer scale for the three first variables was ‘very good’, ‘good’, ‘regular’, ‘bad’, ‘very bad’, dichotomized into positive (very good + good) and negative (regular, bad + very bad) evaluations, considering the low frequency of dissatisfaction regarding these variables (Table S1). For the other two variables, each user rated their satisfaction with care received by the dentist and with the service rendered by the reception staff, assigning a grade from 0 (zero) to 10 (ten). This scale was dichotomized by the cut-off point 8, and users with grades 9 and 10 were considered satisfied. The definition of this cut-off point was based on the distribution of this variable in the sample and in the reasonable grouping. Most CEO users scored very high on these two questions (Figures S1 and S2). Consequently, the cut-off 8 intended to discriminate those groups that gave the highest scores, that is, very high satisfaction, from those with the lowest level of satisfaction. This same cut-off was used in a previous study that evaluated user satisfaction with primary oral healthcare services.6

The identification of homogenous subgroups with similar outcomes was performed using the latent class analysis (LCA). LCA is a mixed model that postulates the existence of an unobserved (latent) categorical variable that divides a population into mutually exclusive and complete classes. The participation of individuals in the categories (classes) is unknown, but it can be inferred from the measurement of a set of items.23 This analysis method was chosen, as the latent variable may represent a complex construct (user satisfaction) defined by a combination of measured variables. The purpose of LCA is to define the latent variable in order to identify a number of classes that describe the underlying scoring patterns in the data, estimate the prevalence of the classes, and estimate each individual probability of belonging to each class.24 The LCA helps to identify the response patterns that provide the best balance between considering all users to belong to the same subgroup and considering all existing patterns to be a relevant class on their own.

A sequence of models for the observed set of variables, containing one to three classes, was tested to determine the best fit based on the minimum value of Bayesian information criterion (BIC) and Akaike information criterion (AIC). Users who have a pattern that fits well with a certain class have a high probability (close to 1) of belonging to that group. Next, the probability of each user belonging to a subgroup (class) was estimated. From the maximum value of probability, the distribution of users in the classes was defined. The findings of LCA were shown as a supplementary file (Tables S2–S5, Figures S3 and S4), and the two-class model was chosen for both PMAQ cycles. For purposes of interpretation, the classes represent satisfied and dissatisfied with the specialized oral health service provided by the CEO. The LCA was performed using a generalized structural equation model with the logit function, considering that all observed variables were binary.

2.2 Structure

The CEO structure was evaluated considering dimensions necessary for the provision of services: equipment, supplies, instruments and ambience. The variables in each dimension were presented in Tables S6–S8. The sum of the number of items with adequate conditions of use from each dimension was obtained.25 According to the Brazilian Ordinance number 599/GM/MS, from 23 March 2006,26 which describes CEOs, the structure variables include (1) the type of CEO according to the number of dental chairs and number of oral health professionals [Type I (3 chairs; minimum of 3 dentists and 1 oral health assistant), II (4–6 chairs; minimum of 4 dentists and 1 oral health assistant) and III (7 or more chairs; minimum of 7 dentists and 1 oral health assistant)]; (2) if the CEO is a university; (3) if it also provides other specialties beyond the minimum, which correspond to the specialties of stomatology, specialized periodontics, minor oral surgeries, endodontics and dental care provided to disabled individuals; (4) if the dentist had interrupted any appointments in the last year due to the lack of supplies or instruments; (5) if dental services were interrupted in the last year due to non-working equipment; (6) the presence of the manager in the CEO; (7) manager of the CEO has complementary training; and (8) the existence of a career plan for CEO workers.

2.3 Process

The process was evaluated by variables related to the work processes described in Table 1.

TABLE 1. Variables for evaluation of the work processes in the CEOs, Brazil, 2014 and 2018 Variables Type of answer Planning and monitoring of actions The actions developed by the CEO are the result of periodic evaluations and planning No, Yes Over the last 12 months, the dentist participated in the CEO’s planning No, Yes The CEO team plans/programs its activities considering information from the outpatient information system (SIA/SUS) No, Yes The CEO team plans/programs its activities considering the information from the local epidemiological survey No, Yes The CEO team plans/programs its activities considering the aims of each specialty, established by the Ministry of Health No, Yes The CEO team plans/programs its activities considering the challenges pointed out through self-assessment No, Yes The CEO team plans/programs its activities considering the involvement of community organizations (partnership and agreements with the community) No, Yes The CEO team evaluates planned/scheduled actions No, Yes The CEO conducts team meetings No, Yes The CEO performs the monitoring of the goals established for each specialty provided by the CEO No, Yes The CEO team performs self-assessment processes periodically No, Yes Collaborative care The CEO team receives support from other professionals to aid in the resolution of complex casesa No, Yes The CEO team support the Oral Health teams from Primary Care to resolve complex cases No, Yes Feature of demand for dental care and organization of scheduling The CEO organizes its demand through referrals received from primary care dentists or attends to spontaneous and mixed demand Spontaneous/Mixed or Referred by a primary care dentist The CEO opens at lunchtime No, Yes The CEO schedules appointments on Saturday or Monday No, Yes The Basic Health Unit schedules the appointment in the CEO No, Yes The patient in the specialized consultation centre schedules the appointment No, Yes The patient himself who receives the form/referral from the Basic Health Unit schedules the appointment No, Yes Participation, social control, and channels of communication with users The CEO provides a mechanism to measure user satisfaction No, Yes The CEO provides communication channels that allow users to express their demands, complaints, and/or suggestions No, Yes Continuing Education for employees The CEO team uses Telehealth for second formative opinion, telediagnosis, teleconsulting or tele-education No, Yes The municipality promotes continuing education actions which include CEO professionals No, Yes 2.4 Covariates

The covariates were sociodemographic user profile variables: sex, age, self-reported skin colour, years of education, family income, whether the user has a paid job or beneficiary of the Brazilian Family Grant Program, location of residence (users live in the CEO municipality), and household covered by the Family Health Strategy. The Brazilian Family Grant Program is a national income transfer program. The supplementary material shows details on sociodemographic variables (Tables S9 and S10).

2.5 Data analysis

The data present an organization at two levels: CEO level (structure and process) and individual level (outcome and user profile). The databases were linked by the variable National Register of Health Facilities (CNES, in Portuguese), which identified the CEOs in the data collection instruments of the three modules in both evaluation cycles. The association between user satisfaction and variables related to the structure and work process was investigated through a multilevel logistic regression model of fixed effects and random intercepts. Initially, the variance in user satisfaction among CEOs was estimated by an empty model. A significant random intercept variance indicates the presence of unexplained differences in user satisfaction among CEOs. The Wald test evaluated the significance of random intercept, and the median rate ratio (MRR) measured the heterogeneity among CEOs, according to Austin et al.27 No variation was found among CEOs when the MRR is 1.0. However, the higher the MRR, the greater the area-level variation. The intraclass correlation coefficient (ICC) was also estimated to quantify the degree of homogeneity of the outcome within Brazilian regions. The ICC represents the proportion of the between-cluster variation (in this case: the between-CEO variation of user satisfaction) in the total variation (in this case: the between-CEO plus the within-CEO variation of user satisfaction). The adjustment of models was performed for each set of variables, referring to the structure, work process and covariables. Those statistically significant variables (p < .05) in each set were included in the sequential modelling adjustment: models 1 and 2 included variables measured at the contextual level (CEO) and sequential variables of structure and process, while model 3 (final model) added the user profile variables. The proportional change in variance (PCV) was calculated according to Merlo et al.,28 using the following formula: PCV = (variance model 1—variance model 2)/variance model 1. As the CEOs are nested in different Brazilian regions, the regression model was tested, including a level 3 (region) in the model. However, for both evaluation cycles, the between-cluster variation of the outcome (user satisfaction) was low [cycle 1 region(var(_const) = 0.0979753; cycle 2 region(var(_const) = 0.0448261)], meaning that the variation in user satisfaction was low between the regions of Brazil. In this case, it is not justified to include an additional level in the model structure or to consider explanatory variables for user satisfaction at the regional level. Additionally, the ICC values were close to zero (cycle 1 ICC = 0.0289197; cycle 2 ICC = 0.0134423). For these reasons, level 3 (Brazilian Region) was not included in the multilevel regression model, opting for the most parsimonious model, with two levels.

All of the participants of the study signed a free and informed consent form, printed in two copies. This study was submitted to and approved by the Ethics Committee for Human Research of Universidade Federal de Pernambuco (logged under protocol number 23458213.0.1001.5208).

3 RESULTS

A total of 7997 users were interviewed in 794 CEOs, and 10056 users in 911 CEOs participated in the first and second evaluation cycles, respectively. In both evaluation cycles, users who belonged to a CEO that did not have information about the structure and work process, or those who had incomplete information regarding the sociodemographic profile, were excluded. The CEOs with fewer than 10 respondents were also excluded. The flowchart of the inclusion of users and CEOs is presented in Figure 2. For most CEOs, 10 users were interviewed in the first cycle (85.2%) and in the second evaluation cycle (90.3%).

Flowchart showing inclusion and exclusion of participants in the first and second evaluation cycles, Brazil, PMAQ-CEO, 2014, 2018

The sociodemographic profile of users was similar in both evaluation cycles. In the second cycle, there was a higher percentage of beneficiaries of the Brazilian Family Grant Program and residents in areas covered by the Family Health Strategy. The mean age of the users was 41.9 (SD: 15.0) and 42.6 (SD: 15.0) years in the first and second evaluation cycles, respectively. The full results of the sociodemographic profile of the users were presented in the Table S11.

The LCA resulted in two classes. The percentage of users who were more satisfied with the CEO was 85.3% in the first and 87.1% in the second evaluation cycle.

The results of the variables included in the LCA were shown in Table 2. The percentage of users who rated the facilities, dental care received and the feeling of comfort as “very good” or “good” and who attributed grades 9 or 10 to satisfaction with the care received by the dentist, as well as satisfaction with the service rendered by the reception staff, was above 88.5% among those belonging to the same latent class defined as satisfied.

TABLE 2. Distribution of satisfied and dissatisfied users according to the variables of the self-rating of Dental Specialty Center facilities, self-rating of dental care received, the feeling of comfort during the dental appointment, satisfaction with care received by the dentist and satisfaction with the service rendered by the reception staff. 2014, 2018 First evaluation cycle – 2014

Class 1

Dissatisfied

Class 2

Satisfied

n % n % Self-rating of Dental Specialty Center facilities Regular + bad + very bad 667 46.4 770 53.6 Very good + good 510 7.8 6050 92.2 Self-rating of dental care received Regular + bad + very bad 360 100.0 0 0.0 Very good + good 817 10.7 6820 89.3 Feeling of comfort during the dental appointment Regular + bad + very bad 325 53.8 279 46.2 Very good + good 852 11.5 6541 88.5 Satisfaction with care received by the dentist Grade 0–8 801 81.5 182 18.5 Grade ≥ 9 376 5.4 6638 94.6 Satisfaction with the service rendered by the reception staff Grade 0–8 953 66.0 492 34.0 Grade ≥ 9 224 3.4 6328 96.6 Second evaluation cycle – 2018 n % n % Self-rating of Dental Specialty Center facilities Regular + bad + very bad 754 51.8 703 48.2 Very good + good 532 6.2 8008 93.8 Self-rating of dental care received Regular + bad + very bad 283 85.2 49 14.8 Very good + good 1003 10.4 8662 89.6 Feeling of comfort during the dental appointment Regular + bad + very bad 372 73.2 136 26.8 Very good + good 914 9.6 8575 90.4 Satisfaction with care received by the dentist Grade 0–8 827 78.1 232 21.9 Grade ≥ 9 459 5.1 8479 94.9 Satisfaction with the service rendered by the reception staff Grade 0–8 1010 62.4 609 37.6 Grade ≥ 9 276 3.3 8102 96.7 Note The bold values are the percentage of users who rated the facilities, dental care received and the feeling of comfort as “very good” or “good” and who attributed grades 9 or 10 to satisfaction with the care received by the dentist, as well as satisfaction with the service rendered by the reception staff.

The findings of the simple regression analysis and the adjustment steps of the multilevel regression models were presented in the Tables S12–S15. In both evaluation cycles, structure variables were associated with user satisfaction. Regardless of the user profile, those from CEOs with more favourable structure characteristics (largest amount of equipment, items of ambience and presence of management) showed a higher satisfaction. Users of type II CEO presented a lower frequency of satisfaction. In the first evaluation cycle, a similar finding was also observed when users were from CEO type III (Tables 3 and 4). Users from CEOs with managers presented a greater satisfaction with the service.

TABLE 3. Multilevel regression model of the association between the CEO features (structure and work process) and user satisfaction with secondary dental care services 2014 Variables Empty model Model 1 (Inclusion of structure + management variables) Model 2 (Inclusion of structure + management + work process variables) Model 3 (Inclusion of structure, management, work process, and user profile variables) OR (95% CI) OR (95% CI) OR (95% CI) Contextual variables (CEO) CEO structure Number of pieces of equipment 1.06 (1.03; 1.10) 1.04 (1.01; 1.08) 1.05 (1.01; 1.08) CEO Ambience 1.18 (1.10; 1.26) 1.15 (1.08; 1.23) 1.14 (1.07; 1.22) CEO type Type I 1 1 1 Type II 0.73 (0.61; 0.89) 0.71 (0.59; 0.86) 0.70 (0.60; 0.85) Type III 0.68 (0.50; 0.91) 0.67 (0.50; 0.90) 0.64 (0.48; 0.87) Service interrupted due to non-working equipment 0.67 (0.57; 0.80) 0.67 (0.56; 0.79) 0.67 (0.56; 0.81) Manager with complementary training 1.09 (0.88; 1.33) 1.01 (0.82; 1.24) 1.04 (0.84; 1.28) CEO with manager 1.49 (1.09; 2.02) 1.42 (1.05; 1.92) 1.41 (1.03; 1.92) Work process The CEO team performs self-assessment processes periodically 1.32 (1.08;1.60) 1.26 (1.03;1.54) CEO organizes their demand through referrals received from primary care dentists 1.24 (1.04; 1.47) 1.20 (1.01; 1.43) The patient himself who receives the form/referral from the Basic Health Unit schedules the appointment 0.83 (0.67; 1.01) 0.86 (0.70; 1.06) The municipality promotes continuing education actions that include CEO professionals 1.20 (1.01; 1.42) 1.19 (1.01; 1.41) Individual variables User profile Sex (female) 1.03 (0.89; 1.20) Age 1.02 (1.01; 1.02) Years of education Illiterate or functional illiterate 1 1–8 years of education 0.79 (0.57; 1.10) 9–11 years of education 0.57 (0.41; 0.80) >=12 years of education 0.68 (0.47; 0.98) The user lives in the CEO municipality 0.78 (0.57; 1.06) The Family Health Strategy covers user household