Early Warning Scores, if implemented within the clinical setting, can become an integral part of the healthcare delivery process by assessing the early signs of clinical deterioration of a patient [12, 15, 16]. When EWSs are used as a track and trigger system a patient’s EWSs can either alert the healthcare team to intervene on behalf of the patient or continue to monitor the patient’s condition [12, 17, 18]. Although the validity of EWSs to predict adverse outcomes has been established across different healthcare settings and among diverse patient populations [19,20,21], these data are not being maximized by healthcare researchers and informaticists to advance secondary research of clinical data and outcomes stored within EHR systems across multiple healthcare settings.

Researchers of individual institutions may be able to extract EWS data, clean it, and append it with patient demographic and clinical characteristics using traditional data cleaning and merging techniques; but integrating data in this format, which may exist as either structured data or free text, can be inefficient and time consuming [22]. This traditional approach to manually transform and append disparate data sources to answer a single research question can be inefficient, because the procedure is not easily reproduced by other researchers, and it does not transform heterogeneous data sources into actionable clinical information. For instance, for a researcher to have confidence in the use of a colleague's transformed dataset to extract clinical information that may be useful to answer a different scientific question, agreement with the meaning of each term used to define a particular feature must exist. Oftentimes, however, the labels or terms used by individual researchers to provide meaning to features within a dataset may differ in subtle but important ways from other researchers’ ideas. Without a standardized data dictionary that is openly accessible and includes terms agreed upon by all members of the research team and is updated overtime, use of such a data source will decrease or worse it continues to be used, but the clinical information derived is questionable. This is where ontologies can help.

Ontologies are formalized representations of the entities in a domain of discourse that are machine interpretable [23]. They are frequently used to achieve semantic integration of biomedical data [24,25,26,27,28]. The lack of semantic integration between different EWSs implemented within different EHR systems across disparate healthcare institutions is currently hampering our ability to query EWS data to identify factors relevant to different EWSs. Our goal is to achieve semantic integration of EWS data to allow easier appending of heterogeneous patient-level and environmental-level data sets to investigate relationships that extend beyond the clinical setting. For instance, examining the correlation between aspects of social determinants, a patient’s EWSs, and negative health outcomes, such as unplanned ICU readmissions, morbidity, and mortality.

Our ontology aims to represent both the structure and planned process of the National Early Warning Score [29], the six-item modified Early Warning Score [30, 31], and the quick Sequential Organ Failure Assessment [32]. The National Early Warning Score system uses inputs from seven vital function measurement datum: respiratory rate; systolic blood pressure; heart rate; Alert, Voice, Pain, Unresponsiveness (AVPU); body temperature; oxygen saturation; and supplemental oxygen use [29]. There are several Modified Early Warning Score systems that are differentiated by the number of physiological parameters used to derive the final score. For the purposes of our study, we will focus on the six-item MEWS, which uses inputs from six vital function measurement datum, respiratory rate, systolic blood pressure, urine output, heart rate, AVPU, and body temperature [31]. The final representation is of the qSOFA, which uses inputs from three vital function measurement datum, the Glasgow coma scale, respiratory rate, and systolic blood pressure [32].

Materials and methodsOntology development

We followed the Software Development Life Cycle (SDLC) framework depicted in Fig. 1 to model the tasks involved in planning, creating, and testing the Early Warning Systems Score Ontology [33]. While EWSSO is available from a public repository, we have not fully gone live with our ontology. Thus, we have not finalized the deployment, operations, and maintenance phases of the SDLC.

Fig. 1

Representation of the seven phases of the SDLC framework

SDLC phase 1 and phase 2: planning and requirements

During the planning phase of the SDLC relevant information and methods to be applied were discussed to assess the feasibility of developing and validating the EWSSO. During the design phase the team agreed to apply methods described by Arp, Smith & Spear for “Building Ontologies with Basic Formal Ontology” [34] particularly, the principles of best practice for ontology design, terms, definitions, and classifications to meet BFO requirements for ontology building. Additionally, we followed Arp, Smith, and Spear’s guidance on using Aristotelian definitions—i.e. genus-differentia definitions—which define an A as a B with the properties C, where A is the class of things being defined (e.g., national early warning score) and B is the immediate superclass (e.g., early warning scores) and C are a set of properties that make the members of A different from the members of B that are not members of A.

As an example, two Aristotelian definitions relevant to our domain are described here.

1.

early warning score = def. an information content entity that is about clinical deterioration.

2.

national early warning score = def. an early warning score that is the specified output of the national early warning score summation process that has specified input respiratory rate, systolic blood pressure, heart rate, AVPU, body temperature, oxygen saturation, and supplemental oxygen use measurement data.

In the first definition, we specify A [early warning score] as a B [an information content entity] that Cs [is about clinical deterioration]. In the second definition, we specify A [national early warning score] as a B [early warning score] that Cs [is the specified output of the national early warning score summation process that has specified input respiratory rate, systolic blood pressure, heart rate, AVPU, body temperature, oxygen saturation, and supplemental oxygen use measurement data]. If we were to replace the term early warning score in the second sentence with the definition of early warning score in the first sentence the unpacked definition for national early warning score would be an information content entity that is about clinical deterioration and is the specified output of the national early warning score summation process. The result is that we have the same meaning of national early warning score as the sentence with which we began. In keeping in line with creating high-quality ontological definitions we use singular noun terms and avoid acronyms. Aristotelian definitions are helpful for ontology development and maintenance since they bind the definitions to the taxonomy underlying our ontology.

SDLC phase 3 and phase 4: designing and buildingOBO foundry principles

The OBO Foundry Principles [35,36,37] guidelines are meant to serve as the norm for OBO Foundry ontologies and are used in assessing the ontologies that are submitted to the OBO Foundry for review. Because we have plans to submit the EWSSO for the Foundry's assessment, our ontology development process follows these principles. An assessment of how our ontology in the early development phase meets certain OBO Foundry principles is outlined in the results section of this paper. The OBO Foundry aims to be an orthogonal library of interactive ontologies [37]. To prevent conflicting representation and enable optimal semantic integration among resources using OBO Foundry ontologies, a key strategy in developing an ontology for submission to the OBO Foundry is identifying potential areas of overlap with existing OBO Foundry ontologies and prevent conflicting representations [37]. The OBO Foundry principles prescribe the reuse of relations [35], e.g., object properties but considering the reuse of classes is also a common and useful strategy [38]. To address the issue of effective reuse of individual entities or sets of individual entities, OBO Foundry contributors have developed methods of importing classes, object properties, and other entities from existing ontologies [38], called MIREOT. There are multiple implementations of MIREOT. We use the MIREOT Protégé plugin developed by Hanna et al. [39] Reusing entities from preexisting ontologies using MIREOT copies over annotation values from the source ontology. Such annotation values sometimes provide cross-references to other ontologies. E.g., the EWSSO class respiratory rate measurement datum point to the class respiratory rate [40], which in turn points to several similar terms in medical terminologies. These cross-references are coming from an outside source and not subject to our development. Our aim is to follow the strategy to prevent duplicate representation of entities in EWSSO and OBO Foundry ontologies.

Reused classes

The 4 reused classes and definitions from OBO Foundry ontologies are listed in Table 1.

Table 1 OBO foundry ontologies, reused classes, and definitionsReused object properties

We reused four object properties from OBO Foundry ontologies, listed in Table 2.

Table 2 OBO Foundry Ontologies, reused object properties, and definitions

Every early warning score is an information content entity that is about clinical deterioration. Every early warning score is specified by property is about. The Information Artifact Ontology (IAO) defines object property is about, as “a (currently) primitive relation that relates an information artifact to an entity.” In this case, the early warning score is the information artifact that is about some entity, clinical deterioration, that is “the process of the diminishing realization of the patient's vital functions.” The Ontology for Biomedical Investigations (OBI) defines object property is specified output of as “a relation between a planned process and a continuant participating in that process. The presence of the continuant at the end of the process is explicitly specified in the objective specification which the process realizes the concretization of.” In this case, the national early warning score is the continuant at the end of the national early warning score summation process that is “…explicitly specified in the objective specification which the process realizes the concretization of.”

According to BFO, reality is made up of two types of phenomena: (1) continuants, which are things like objects, qualities, and functions that last over time; and (2) occurrents, which are the activities that continuants engage in [50,51,52]. Continuants may be independent or dependent [52]. Entities we encounter daily (e.g., person, ball, pencil) are primary examples of independent continuants that also act as carriers of dependent continuants (e.g., a smile, the color of a ball, the ability to write) [52]. We mention these BFO fundamental concepts of reality because we classify early warning system scores, such as the NEWS, six-item MEWS, and qSOFA as generically dependent continuants, more specifically an information content entity, that are the specified output of a planned process (occurrent) that uses attribute specific measurement inputs to calculate a patient’s early warning score, which warns about clinical deterioration.

We created our ontology in OWL2 using the Protégé ontology editor [53] and provided visualization of our ontology using the online MIRO software [54]

SDLC phase 5: testing

To determine the usefulness of our ontology to generate and query heterogeneous vital sign data for early warning score calculation and secondary data analysis we performed the following steps.

Step 1: create synthetic datasets

The purpose for creating synthetic datasets was to simulate heterogeneous data sources of critical care patients whose degree of physiological decompensation is typically evaluated with an early warning system score, such as the NEWS, MEWS, and / or qSOFA. We created two different synthetic datasets. The first included vital sign measurement features specific to each of our early warning scores. The second dataset included other clinical and demographic features, unnecessary for early warning score calculator. This was done to test whether our process with ontological foundation could differentiate between features necessary and unnecessary for score calculation. Temperature measurement data were added as either degree Fahrenheit or degree Celsius. This was done so that the system would then calculate the early warning score with the appropriate temperature measurement scale. Additionally, a random number of values were left blank to represent missingness, common to all datasets. Our process is not meant to impute missing data. However, if a feature necessary to calculate an early warning score is missing the system will return the output as unavailable. Each dataset included 1,000 patient rows.

To create our synthetic datasets of vital measurements used in various early warning scores, we first needed a method to generate a normal distribution of values with a set median, interquartile range (IQR), and extreme range. Using a JavaScript example as a base [55] a Python script was created to generate N samples with configurable parameters (median, IQR, and range) for each vital measurement needed. For the dataset to prove useful in a comparison of early warning scores, we needed to base the vitals on a set of data pertaining to emergency / critical care. We used a Danish study by Nissen et. al. [56] of the performance of NEWS as the baseline parameters to determine most of our measurements since the Danish Multicenter Cohort focused on emergency department admissions. These patients would have distribution and ranges of EWS variables that are more indicative of the types of patients who would be subject to these types of early warning scores for deterioration assessment. The listed median and IQR for each measurement were used in conjunction with a set of extreme ranges that ensured the full range of NEWS scoring would be represented [56]. Any missing measurements were filled from similar studies regarding critical patients in European cohorts [16]. To better align the results of different early warning scores using the distinct but related measurements for consciousness, AVPU vs GCS, we generated the score for AVPU and then converted to GCS per Romanelli and Farrell [57].

Step 2: convert synthetic dataset into triples

The synthetic datasets were converted into triples in an RDF database informed by our early warning system scores ontology. We transformed the entirety of each dataset to triples.

Step 3: visualization of RDF database and SPARQL queries

Once the heterogeneous synthetic datasets were converted into triples representing each patient and their respective vital sign measurements, the results were loaded into GraphDB. We then ran a SPARQL query to GraphDB that returned all measurement data for a specific patient (patient identifier). A visualization of the triples in the RDF database, RDF graph, SPARQL query, and the answer for the SPARQL query used to extract a person’s measurement data can be seen in Fig. 2.

Fig. 2

Representation of RDF database

Step 4: calculate the early scores for each person

To calculate the early warning scores for each person, three projects were created in Python. The intention was to define the required calculations in separate modules that would be imported and utilized by the base project. One project defined each early warning score’s requirements and the formula for the score calculation. We used the guide charts for NEWS, six-item MEWS, and qSOFA scoring system methodology provided by Aseptika Limited [58], the Institute for Healthcare Improvement [59], and Omni Calculator [60], respectively. Another project contained formulas for unit conversions (e.g., converting temperature to degrees Celsius when a formula requires Celsius, but the provided data is Fahrenheit). The general flow of the application was that the base Python project would query the data from the triplestore using SPARQL, representing a person with their semantically tagged measurements. The EWS-specific module would be dynamically imported by the base project and used to calculate each early warning score that meets the minimum data requirements for the score. The calculation is possible because the data representing the measurements is semantically tagged, and the formulas for each score use those tags to determine the correct parameters. In turn, the EWS module would import the unit conversion module to perform any necessary unit conversions on the data. This approach allows new early warning scores or unit conversion formulas to be added to their respective projects without interfering with the base project or other projects created to take advantage of the centralized specification.

Step 5: validate early warning system score output

Researcher CEZ randomly selected ten patients and manually calculated the early warning system scores, based on the heterogeneous datasets, and compared to system output.