Juran model

The American engineer Joseph Juran (1904–2008) is one of the founders of quality management. Interestingly, he was more influential in Japan than in the United States at first. He visited Japan in 1950’s and consulted managers and engineers in managing for quality. Juran acknowledged that there are a number of definitions of quality, and that the meaning of quality also has a bearing on the approach to quality management. Juran’s approach to quality management first introduced in 1951 [15] comprises three managerial processes: quality planning, quality control, and quality improvement. These three processes have since come to be known as the Juran trilogy [16] and are described below.

Quality planning provides the process, methods, tools, and techniques to create a high quality product. There are 6 steps in this process: (1) Establish the project by providing the goals, direction, and materials, equipment, skills etc.; (2) identify all customers; (3) identify customer needs to inform product design; (4) develop product; (5) develop a process capable of delivering the product as it was designed, consistently, time after time; (6) develop process controls that keep the process operating at its full capability. Juran advocates the use of preventive risk analysis tools during the planning stage to inform process design (Box 1). Quality control provides stability. Quality control processes evaluate actual performance, compare actual performance to goals, and take action on the difference. After improvements have been made, a new level of performance has been achieved. Quality improvement aims to attain “unprecedented levels of performance”. At this point processes and goals are in place, though products may not all meet the goals. The approach therefore consists of “(1) discovering the causes—why do some products meet the goal and others do not—and (2) applying remedies to remove the causes”.

Juran has proposed a number of definitions for quality, two of which can be readily applied for epidemiological studies and data: ‘fitness for use’ and ‘freedom from deficiencies’ [16]. While freedom from deficiencies is a very noble goal to work towards, it may remain unattainable. The relative nature of ‘fitness for use’ on the other hand is very pragmatic and has gained traction in many fields, including data quality attributes [17] (see Box 2). The preventive nature of the Juran model lends itself well to applications in epidemiological studies. Quality control mechanisms are developed early on, during the planning stage of the study. They are closely linked to both quality goals and feedback loops. As such, the study becomes a dynamic learning system which facilitates prompt mid-course corrections. The strong focus on customers and customer needs feel artificial in research settings, although in theory research funders and society could be thought of as ‘the customers’. The Juran model has a strong focus on reducing waste and bringing systems to ever higher levels of performance. Interestingly this resonates well with the current scientific discourse around replicability, where “correctable weaknesses” in the design, conduct, and analysis of research studies are framed as a “waste” of valuable resources.

Donabedian model

The physician Avedis Donabedian (1919–2000) is considered the authority in all matters concerning healthcare quality. In his seminal 1966 paper ‘Evaluating the Quality of Medical Care’ [3] he introduced a model to assess quality of care focussing on the ‘triad’ of structure, process and outcome. Structure refers to “the conditions under which care is provided” (facilities, equipment, human resources, organizational characteristics, etc.). Process are the “activities that constitute health care” (diagnosis, treatment, prevention, etc.). Outcomes are “changes in individuals and populations that can be attributed to health care” (changes in health status, knowledge, behaviour, satisfaction etc.).

In his later work he also describes a system for quality assurance [2], defined as “activity by which we obtain information about the level of quality produced by the health care system and, based on an interpretation of that information, take the actions needed to protect and improve quality”. Proposed steps are as follows: (1) Determining what to monitor (e.g. choice of tracers); (2) Determining priorities in monitoring; (3) Selecting an approach to assessing performance (structure, process and/or outcomes); (4) Formulating criteria and standards (see Box 2); (5) Obtaining the necessary information (surveys, records, observation); (6) Choosing when to monitor (prospective, concurrent or retrospective monitoring); (7) Choosing how to monitor; (8) Constructing a monitoring system; (9) Bringing about behavior change (re-adjustments and educational/motivational activities).

The Donabedian model has gained widespread acceptance in health care. Being health-related, it can seem like a natural choice of reference for epidemiologists. In many ways it can work very well, since the implementation of epidemiological studies can also be broken down into structures (human resources and equipment), processes (data collection, data management, data analyses) and outcomes (completed questionnaires, raw data, clean data, analyses outputs such as tables and graphs). A monitoring focus on structures would only make sense for studies of extended duration (e.g. surveillance systems or repeated surveys) with long term structural investments. For one-off studies, the focus may be more on processes and outcomes, as there is little scope for corrections ‘after the fact’. In these cases, concurrent monitoring activities are arguably more important.

ISO 9000 models

The International Organization for Standardization (ISO) was founded in 1947 to reach world-wide uniformity in measurements. Thereafter it moved towards publishing standards. In 1987 ISO created the ISO 9000 family of standards for quality management, designed to help organisations ensure that they meet the needs of customers and other stakeholders while meeting statutory and regulatory requirements related to a product or service. The ISO 9000 standards have undergone a number of revisions since their creation, and over time have moved away from product inspection and towards a process-oriented approach in order to produce the sought after quality outcomes [18]. ISO 9000:1994 emphasized quality assurance via preventive actions, based on concepts of risk assessment and risk mitigation using tools such as the failure mode and effects analysis (Box 1). These concepts are very much in line with the Juran approach to quality management. The process oriented structure was introduced in ISO 9001:2000 edition, and was further developed in the ISO 9001:2015, as described below.

The process oriented approach of ISO 9001:2015 describes an organization as consisting of a series of interacting processes [18]. A process is a set of activities that uses resources (people, machines, etc.) to transform inputs into outputs. The output of one process is the input of another process, which stresses the importance of not treating each process in isolation (department, job, etc.). Each process needs to ensure it delivers (outputs) what the next process needs (inputs). The ISO 9001 Standard is designed to manage and improve processes with the following steps: (1) Identify your key processes; (2) define standards for those processes. (3) Decide how the process will be measured and evaluated; (4) document your approach to achieving the desired quality, as determined by your measurements; (5) continuously improve.

The ISO approach to quality assurance can be adapted to suit epidemiological research. However the focus on processes can feel unsatisfactory, since the data quality (per se an outcome rather than a process) should be the primary concern of quality assurance in epidemiology. One perceived advantage is that research organisations can get ISO certified by third-party certification bodies, which brings credibility to outside parties that standardised and documented processes are being followed. To qualify for ISO certification, an organisation must write a specific, step-by-step description for each process, and then demonstrate that it follows these procedures. However ISO certification has been criticised for being time-consuming and expensive (ISO standards are proprietary and certification relies heavily on external audits). Moreover, there is no evidence that ISO certification actually improves quality [19].

Box 1. Risk analysis Failure Mode and Effects Analysis (FMEA) or Hazard Analysis and Critical Control Points (HACCP) are two examples of risk analysis tools to identify potential weaknesses in a process. Procedures for conducting FMEA were described in US Armed Forces Military Procedures document MIL-P-1629 in 1949. During the 1970s, use of FMEA and related techniques spread to other industries. HACCP is the adaptation of the FMEA to the food industry The aim of these analyses is to identify all possible failures or hazards in each part of a system, during its design stage, in order to ensure that they can be prevented from occurring in the first place. Applied to an epidemiological study, this can be done by systematically questioning, for each step in the survey process (e.g. study preparation, data collection, data analysis, etc.): what can go wrong (failure modes in FMEA, hazards in HACCP)? How can this be prevented? How can we check that we are doing things right (detection in FMEA)? How can we fix things if they go wrong (mitigation in FMEA and corrective actions in HACCP)? Box 2. Standards, criteria and attributes Criteria and standards are “the tools by which the quality is measured” [2]. As such, they form the back-bone of many quality assurance approaches. However, there is no agreed-upon usage for these terms, and in fact, various contradictory definitions have been given [20] Donabedian [2] defines a criterion as “an attribute of structure, process, or outcome that is used to draw an inference about quality. […] For example a criterion of outcome could be case fatality”. Standards are defined as “a specified quantitative measure of magnitude or frequency that specifies what is good or less so. […] For example a standard for case fatality could be: no more than 0.1% for a specified procedure (or a set of procedures) in a specified category of patients Quality attributes according to Donabedian [2] are the “product characteristics [which] taken singly of in a variety of combinations constitute a definition of quality and, when measured in one way or another will signify its magnitude”. According to this definition data quality attributed in epidemiology refer to data quality framework dimensions such as relevance; accuracy; credibility; timeliness; accessibility; interpretability; and coherence [17]. These can either be attributes of the system that produced the data (i.e. the process) or of the data itself (data output/outcome) [21] Donabedian also proposes a useful link between the standard-criteria duo and quality attributes: “Criteria and standards are vehicles by which quality attributes are translated to actual measurements” Items in bold in this table and in the text can be traced back to this box as a reference