Process Mining for Quality Improvement — Case Study in Emergency Department

This is a guest article by Matthew H. Loxton, a senior analyst for healthcare at WBB. You can request an extended version of this case study with detailed recommendations from Matthew directly. An overview paper about process mining for quality improvement in healthcare environments can be found here.

Historically, Quality Improvement (QI) projects have used a combination of received workflow and observational studies to derive the as-is process model. The process model is used to target interventions to reduce waste and risk, and to improve processes that lead to gains in the target performance indicators. Process mining enables QI efforts to more rapidly discover areas for improvement, and to apply a perspective that was historically not available to QI teams.

Figure 1: Process map of ED #1 - Cumulative time

Figure 1: Process map of ED #1 - Cumulative time

Since process mining is algorithmic and uses electronic health record (EHR) data, it can be deployed at scale, and can be used to find process improvement opportunities across an entire healthcare system without undue resource requirements or disruption to clinical operations.

Approach

The case studies involved two of the busiest Emergency Departments (ED) in the U.S., and give the reader a picture of how process mining can be used as part of a long-term process improvement regime.

The WBB team used the process mining software Disco to mine ED and EHR data for two EDs for the period 06/04/2015 to 08/02/2015. Data included 2,628 cases for ED #1 and 2,447 cases for ED #2. Each case represents a unique patient transitioning through the ED to arrive at a disposition.

The WBB team also conducted interviews and facilitated sessions with various flow management application stakeholders to identify benefits and challenges, and to provide recommendations for future improvements. Interview participants related to EHR included ED directors, ED physicians, chiefs of staff, chiefs of medicine, and members of the EHR program office.

Results

The discovered process models showed a high degree of variation (see Figure 1 at the top of this article), and the team used filters to manage the process model complexity to a point where the models were useful in identifying and contrasting paths and their performance. The team obtained concurrence from the point of contact at each of the two facilities that the process model was a fair depiction of how their ED operated.

In addition to producing visual depictions of the underlying workflow and performance, a number of special cases were observed in which patient travel through the process model were unexpected and revealed opportunities for improved use of EHR, data governance, and monitoring of unusual patient transactions. For example, some processes are incomplete and do not follow the “should-be” process by omitting the Discharged status.

Among others, the team found opportunities for improvement related to data governance risks, functionality of EHR and inconsistent use of EHR status and disposition in the following areas:

1. Cases of unedited EHR labels existed in the data.

One benefit of process mining is that unknown or unexpected transitions can be identified. The activity items in the data are a combination of national terms and locally configured terms. Locally configured terms are used to describe a location or status that is required to suit local needs such as specialty wards or services unique to the local patient population or facility specialties.

When a locally configured term is created, the default name is new#, where # is the next available sequence number. The name is manually edited and renamed to be meaningful to the facility (e.g. admit to psychiatry). The process model revealed two transition states in the live data, new2, and new3. Since new2 and new3 have 9 and 28 cases respectively, it proved worthwhile to examine the cases.

The event labels stemmed from unfinished additions of new labels that had been inadvertently left in the EHR data. The discovery of these labels led to a process improvement exercise in data cleanup, and discussions regarding processes for adding or editing fields.

2. Loops in the process model due to incorrect sequence entry.

Process loops are expected in some process models, and may indicate normal functioning of the process. However, in processes that are expected to be linear and branching, such as many care flows in the ED, a process loop can indicate either clerical or clinical error, or a process issue.

Figure 2: ED #1 Process loops

Figure 2: ED #1 Process loops

In this case, the data revealed that the loops were the result of some events being entered in reverse order due to functionality in the EHR (see Figure 2 for an example).

The EHR grid view contains all the editable fields, and a user can select the disposition and status in any order. The choice and availability is not constrained or guided by business rules within EHR. As a result, the elapsed times in reports that use a formula for elapsed time based on the status timestamps may be negative, and skew EHR and productivity reports.

This discovery initiated a discussion on enhancement of the EHR and policies regarding use of the grid view. Furthermore, a review of the current reporting algorithms will be performed to ensure that negative values are not skewing or biasing data.

3. Pinball Patients with high event counts.

The distribution curve of events per case is an indicator of one dimension of complexity in a process model. Although the ED-1 distribution shows that most cases have four events, it can also be seen that a small number of variants have far more events per case (see Figure 3).

Figure 3: ED #1 Events per case

Figure 3: ED #1 Events per case

To help identify opportunity for process improvement, it is useful to examine cases that have fewer or more events than chance would predict. For ED-1, the team examined cases that had less than two events, and cases that had more than eight events.

Cases with abnormally low or high event counts may reveal clerical errors, or process gaps that do not adequately address some patient situations.

The ED-1 process model showed three variants in which there were only two events (none that had fewer than two):

  • 44 cases in which the patient was entered in error

  • 31 cases in which the patient was sent to a clinic

  • 7 cases in which the patient eloped

Cases in which patients are entered in error should be evaluated for potential training, EHR functionality, or process issues. Patient elopement is also a situation that deserves examination to see if there are delays or process issues resulting in patient dissatisfaction.

In some cases, there were an unexpectedly high number of status changes. The ED-1 process model showed 24 variants in which there were eight or more events, and two in which there were 10 events.

The following graphic shows the process model for a single case in which the patient had 10 events (see Figure 4).

Figure 4: ED #1 Pinball patient

Figure 4: ED #1 Pinball patient

Cases with both more than two standard deviations of events per variant above or below the mean merit further scrutiny to understand the causes. These cases were examined by the senior ED physician to determine root causes and any evidence of patient safety risks.

Conclusion

This case study illustrates how process mining can reveal questions and potential risks and issues that might not have been otherwise visible. The program office can examine facility processes and formulate specific and targeted questions without unnecessarily interrupting or burdening the facility staff.

Discretion must be used when evaluating elapsed time between transitions; since short times may be due to administrative bundling of tasks and long times may indicate administration being carried out after the fact. For example, short transition times such as from Admitted to Admitted to ICU, Operating Room, Admitted to Telemetry, and Admitted to Ward, showed that the events were administrative actions in the EHR, and are not due to patient movements.

Process discovery is a critical component of QI. The ability to compare accurate depictions of what was intended with what is actually being done is a central part of being able to identify variances, and to correctly target and monitor QI interventions. Traditional methods of process discovery have proven very effective, but have significant disadvantages in terms of accuracy, timeliness, and cost. Process mining enables QI practitioners to more rapidly discover as-is process maps, and thereby to identify deviations, delays, and bottlenecks. Rapid discovery of actual workflow enables faster and more targeted interventions that can increase efficiency, reduce risk, and reduce cost.

Anne Rozinat

Anne Rozinat

Market, customers, and everything else

Anne knows how to mine a process like no other. She has conducted a large number of process mining projects with companies such as Philips Healthcare, Océ, ASML, Philips Consumer Lifestyle, and many others.