Last week, we published a new article about The Added Value of Process Mining at the respected BPM analyst platform BPTrends. You find a short abstract and a link to the original article below.
Process mining, just like data mining, is a generic technology and can be applied in many different ways. This is an advantage but at the same time it makes it difficult for you to understand what exactly the added value would be for your situation. Should you be interested in process mining and learn more about it? Which kinds of processes can be analyzed with process mining? What benefits would it bring?
In this Article, we give you a framework for the most common process mining use cases, so that you can see where you fit in.
Read the full article at the BPTrends website…
What do you think about the discussed use cases? Which are the ones you find most important? Which ones have we missed? Let us know in the comments.
Process mining can not only be used to analyze internal business processes, but also to understand how customers are experiencing the interaction with a company, and how they are using their products.
“Process Mining and Customer Journeys” was the topic of the first event of the new Special Interest Group (SIG) for Process Mining in the Dutch industry association Ngi-NGN. Fluxicon is on the board of this Ngi SIG group and was co-organizing the event, which took place yesterday on 25 March 2014 in Utrecht, at the Rabobank.
More than 50 people had signed up for the event and it went great. Below is a quick summary for everyone who could not be there.
Introduction Customer Journey
Jaap Rigter from VisionWaves first introduced the topic of customer journeys. He illustrated how customers interact with a company through multiple channels, and how understanding the customer experience across these different channels is critical in understanding the customer and improving her experience.
Introduction Process Mining
I then introduced process mining using the metaphor of sailing boat journeys from 150 years ago. For the people who were already familiar with process mining I had brought the first two applications of process mining to customer journeys, which are probably not what you might think (take a look at the slides to find out).
The center of the first part of the evening was the case study presentation by Ellen van Molle and Bram Vanschoenwinkel from AE. They presented the results from a process mining mining analysis at an interim sector company, where employers were matched with employees.
By understanding how potential employees were using the job search application they could highlight the process areas, where people dropped out. Furthermore, by enhancing the data in a second iteration they were able to check hypotheses of the business such as “mostly elderly people have problems with the navigation in the system”.
The second part of the evening was an open discussion in small groups. As a starting point questions such as “What are the challenges of process mining for customer journeys?” and “What is the added value of process mining for customer journeys?” were provided to the groups. Afterwards, the results from the discussion were summarized.
Here are some of the discussion points I remember:
- One challenge is that the data need to be coupled across multiple channels / systems to get an integrated picture.
- Another challenge is that next to the analyst and the business one actually needs to involve the customer herself to understand the underlying root causes and motivations.
- While the rules for analyzing business processes are mostly well-defined, analyzing customer data is much more sensitive and privacy concerns play an important role.
- Potential benefits that were discussed are, for example, the saving of costs of customers calling the helpdesk by better adjusting the websites so that they find what they need.
- Another mentioned benefit was that by improving the customer experience, businesses can expect more revenue from their happy customers and more recommendations from their customers.
It was a well-attended and very lively event. Thank you all for coming! You can download all slides and more photos of the event at the Ngi-NGN event site here.
Photo Credit: 96dpi via Compfight cc
This is a guest post by Walter Vanherle from Crossroad Consulting in Belgium. Walter shares his experience from applying process mining to an operational process from a security provider.
If you have a process mining case study that you would like to share as well, please contact us at firstname.lastname@example.org.
Security Services companies are caught between the rising costs of operations and the downward price pressure due to direct and indirect competition. Further improvements in operational excellence together with service innovation are key in addressing these challenges.
Service delivery is always managed via agreements in the form of contractual obligations based on target performance. Not reaching pre-set targets has immediate financial implications. The service provider, therefore, actively manages these agreements in order to deliver the services efficiently, with costs/penalties managed in relation to the individual client expectation and priorities between clients.
The goal of the process mining project was to measure the performance of such a security services process and to create a reference base of Key Performance Indicators (KPIs).
The Intervention Management Process
Imagine a bank who is a customer of a security services company. Someone breaks a window and the security alarm goes off at the site of the security service provider: An intervention process is started.
The intervention management process has 2 stages (see also picture below). The first stage starts with a client intervention service request (T0). Then, the dispatching unit covering the confirmation activates the service request (T1), identifies an available agent (T2), and the agent confirms the acceptance of the mission (T3).
The second stage is the intervention itself. The execution of the intervention holds 4 subsequent steps: Effective departure to the location for the intervention (T4), Arrival at the location and start of the observations (T5), End of the Observation and documenting the intervention (T6). End of Mission (T7).
There are four KPIs that are relevant for this process. The most important one is the time from the initial client request to the arrival on site (T0-T5). Also important are the time from the client request to the confirmation (T0-T1), the time from the agent’s confirmation to the arrival on site (T3-T5), and the total time from the initial client request to the end of mission (T0-T7).
The service process execution is registered by a special service management software for security service providers by Risk Matrix Resultants. The anonymized event log held data over a period of 2 years containing all interventions for all clients. The dataset contained over 50.000 cases (missions) and 400.000 events.
The analysis below is based on the data from the missions for one client of the security service company over the timeframe of one year.
Process Mining Results
The expectation was that about 70% of the cases should follow the Straight Through Process (STP) flow with the 7 steps T0-T7 as explained above. Furthermore, the following four additional process variants were expected for the remaining 30% of the cases:
- T0-T1 (request is not confirmed)
- T0-T3 (solved, no intervention is needed)
- T0-T4 (aborted in the recording)
- T0-T7 but without T5 (no intervention is needed by accountable)
But how does the process look like in reality? Using the process mining software Disco, the real process flows could be discovered based on the data.
The process map below has been filtered to show the discovered process only for the five expected variants in the process. What stands out is that the four additional variants are almost never taken compared to the standard, STP variant, which is followed by 1518 cases. The other four expected variants are only taken by 31, 13, 2, and 20 cases, respectively.
The problem is that – unlike assumed – the process does not follow just these five expected paths. The STP variants covers 78% of the cases (this is actually slightly more than expected) but the five expected variants together only make up about 82% in total. So, the question is what is happening in the other 18% of the cases?
If we look at the full process, which has 58 variants (more than ten times as much as expected), then we get the following process map. The STP path is still visible, but there is a lot more variation. So, the question is what are these other variants and why are they there?
If we look at the unexpected variants, then it turns out that there are two types of root causes:
- actual variation in the process
- data quality problems that affect timestamps
For example, if we look at the expected variant “T0-T7 but without T5″, then we see that in addition to the sequence T0,T1,T2,T3,T4,T6,T7 (wich occurred 20 times), there are some additional patterns in the process (see process map below):
- 28 times the process went from T3 straight to T6 without T4 (no departure)
- 21 times the process went from T4 directly to T7 without T6 (no arrival)
- 34 times T3 was directly followed by T7 (no observations at all)
At the same time, there were many variations that were caused by what is called “clock drift“. In this process, many different parties were recording events on different devices (which had different clocks). As a consequence, there were often confused orderings in the process step sequence that were purely caused by such a clock difference (that is, the steps were actually be performed in the right order, but due to the different clocks they appeared inverted).
One example case, where this happens, is shown in the picture below. It seems as if T3 was performed before T2, but actually there is just a 5 sec time difference that is caused by the different clocks of the registering parties.
Such data quality problems do not only make the process variant analysis difficult, but also pose the risk to distort your KPI measuring. For example one of the KPIs was defined as the time from T0-T1. What happens now if T1 has an earlier timestamp than T0 due to the clock drift? If you just measure the time between them in Excel, you would get a negative duration that would reduce the average duration between these steps, which of course is not true.
In the intervention management Process, clock drift can occur for the transactions generated by the hand-held devices (PDAs) used by the field service agents or between the alarm generating system (T0) and the dispatching /intervention management system (T1). When the system clocks of devices are not synchronized, the recorded time stamps can shift with seconds, even minutes influencing the effective SLA timings. Using the case monitoring capacity of Disco with process filtering and visualization techniques we were able to visualize outliers quickly and suggest corrections to the transaction file compensating the irregular observations. We suppressed or eliminated the most prominent outliers from the final process mining file for more accurate performance statistics.
After cleaning the data, the SLA analysis for the KPIs (see above) was performed. We exported the durations from Disco and used a template-based Tableau Software Visualization to produce a cumulative SLA spectrum analysis. You can see such an SLA spectrum analysis for the time from T0-T5 for the year 2012 below.
SLA spectrum analysis for a partial data set
The KPIs T0-T5 and T0-T1 are particularly important, because they are linked with financial compensation. For cost optimization and predictive analytics the process sequences T3-T5, T0-T7 were analyzed. We also filtered out groups of clients with similar or different execution patterns based on their type of service contract.
Furthermore, the following process analyses were performed:
- priority accounts treatment,
- work handover patterns (preferential treatment of agents),
- correct treatment of the intervention priority classes.
Benefits and Lessons Learned
The registration process is both machine and people driven. Our experience shows that service tracking is subject to involuntary and voluntary errors and an ongoing, critical, management component. However, after overcoming these data quality challenges, we could generate many important benefits for the Security Services Company:
- Insight in the process variants helped to focus the communication to the operations teams for more accurate recording of the activities.
- Both conformance and performance analysis showed immediate money on the table (value leakage).
- The provided insight is instrumental input for business strategy and tactics corrections such as adaptations in client segmentation (priority services) and the possibility for more granular time based SLA service pricing.
- More accurate information for better planning. Recommendation for geolocation based research. Process Steps T3-T5 is the critical path in reaching target SLAs.
- More and better information in preparation and planning for client acquisition tactics. The analysis are used in pre-sales and sales campaigns.
If you want to know more about this case study, you can get in touch directly with Walter Vanherle at email@example.com.
Get process mining news plus an extra practitioner article straight into your inbox
Every 1-2 months, we create this list of collected process mining web links and events in the process mining news (now also on the blog, with extra material in the e-mail edition).
Process Mining on the Web
Here are some pointers to new process mining discussions and articles on the web, in no particular order:
Process Mining Videos
There are also many new videos. You can now watch:
To make sure you are not missing anything, here is a list of the upcoming process mining events we are aware of.
The people who have participated in our process mining trainings so far really liked what they got. Sign up now to learn from the experts and quick-start your own process mining initiatives.
Here are the next training dates:
Would you like to share a process mining-related pointer to an article, event, or discussion? Let us know about it!
Should the manager approve her own travel request? Usually, the answer is “no” and there are many other examples of where it is not desirable that the same person performs two or more activities in a process.
For example, in a Dutch government process, where citizens can ask for special support based on their income and expenses that they incurred because of illnesses and other circumstances, the employee handling the payout should not be able to change the bank account number and make the transfer at the same time. Otherwise, it would be too easy to give your own bank account number and transfer money to it.
These kind of rules are common in all companies and they are called “Segregation of Duties” (SoD), or “Four Eyes principle”. The idea is to reduce the risk of fraud by putting systems in place that help to keep people honest. These systems are called “controls” and while some controls are realised by IT others just exist in the process documentation, the business rules.
One of the main tasks of an auditor is to check whether the controls that are defined are actually working.
Process mining can be used to check compliance rules like the segregation of duties. The advantage is that while most IT-based SoD controls are implemented on the level of authorisations (for example, employees who have the ability to change the bank account number cannot transfer the money), managing authorisations is a complex task and people change roles all the time. What happens if a person first had the role, where the bank account number could be changed, and later changes into the role with the ability to transfer money? Cases that were started with the first role could be completed by the same person in the second role.
So, while an auditor will review the IT-based authorization controls, it is also interesting to check the actual process executions to see whether the controls were effective (that is, whether SoD violations did occur or not).
Three years ago, we had shown you already how to check segregation of duties with ProM. With Disco, it has actually been possible to check segregation of duties from the beginning. In this post we want to show you how.
If you want to follow along with the instructions, you can do that by simply downloading the demo version of Disco from our website here and repeating the steps that are shown below. Let’s get started!
Get the Sandbox project
You can use the sandbox example that comes with Disco. After the installation, you will be presented with the following blank screen. Click on the Sandbox… button and …
… then double-click the second data set called Process map 100% detail (or press the View details button).
This is the discovered process map of a purchasing process and you are now in the analysis view. Here, you can look at the actual process flows, use the sliders to simplify the process and change the metrics that are displayed in the process map, all based on the data that was extracted from the IT system.
Add filter for segregation of duty violations
To check for segregation of duty violations, you can add a Follower filter. This filter can be added directly by clicking the filter symbol in the lower left corner, or via a shortcut through the process map.
Imagine that this purchasing process has the SoD constraint that the activities Release Supplier’s Invoice and Authorize Suppliers’s Invoice payment should not be performed by the same person for the same case. You want “four eyes” (two different people) to look over it to make sure this is a real invoice that should be paid.
To add a follower pattern filter, you can simply click on the arc going from Release Supplier’s Invoice to Authorize Suppliers’s Invoice payment as shown below. Once you press Filter this path…
… a new Follower filter will be added to your data set. You can now further customize the Follower filter.
To check for segregation of duties in this example, make these two changes to the Follower filter:
- Tick the box Require the same value of Resource for each pair of events matched above to enable the SoD constraint. Of course you actually want that different people are performing these two tasks. However, here we are checking for violations, so we want to see whether there are cases where the person was the same.
- Change the follower pattern from directly followed to eventually followed. Because we came in through the process map short-cut the direct path is checked. However, we want to catch all violations, regardless of whether these two activities were directly performed after one another or whether a dispute was settled in between.
You could now directly apply the filter, but let us preserve the results in a new bookmark in your project, so that you can refer back to them later on.
This can be done by using the Copy and filter rather than the Apply filter button. With Copy and filter you can give a meaningful name and apply the filter to a new copy of the data set, leaving the current data set as it is. Press Create.
Inspect the results
Now it is time to inspect the results. You will see that almost 40% of the cases are violating this segregation of duties rule! To look at some concrete examples, change from the Map view to the Cases view on the top.
You can see that there are exactly 242 cases that are violating the four eyes principle here. One case, the case with the case ID 15 is shown below and you can see that, indeed, Karalda Nimwada was doing both the Release Supplier’s Invoice and the Authorize Supplier’s Invoice Payment step in this case.
You can browse through to see more examples and export all of them to Excel.
But who exactly is violating the rule most often?
To find out, you can refine the results to focus on just the two activities involved in the SoD constraint in the following way: Click on the filter symbol in the lower left corner to add another filter and add an Attribute filter from the list as shown below.
Then, only keep the two activities we are interested in at the moment (Release Supplier’s Invoice and Authorize Supplier’s Invoice Payment). Press Apply filter.
Now you see that all the other activities have been removed and you can change to the Statistics view to look at the most frequent resources.
In the Resource statistics overview, we see that just two users are involved in the SoD violations.
To take action, we can now check their authorizations or give a targeted training.
Illustration by Wil van der Aalst
We humans are great at deriving concepts. It starts when we are a kid and learn about categories like cats and dogs, or tables and chairs, just by being exposed to many examples. So, if we meet something new, we try to put it into perspective with what we already know.
People who hear about process mining for the first time need to understand how it is different from what they are already familiar with. This is why we have frequently written about how process mining relates to other technologies like data mining or BI on this blog.
However, over time these old blog posts are a little more difficult to find, and since I keep getting these questions I thought it would make sense to revisit them in an overview article.
Above, you see a picture that Wil van der Aalst uses to explain what process mining is: In essence, process mining bridges traditional process analysis techniques like modeling (which are not based on data) and data-driven techniques like data mining (which are not process-oriented).
Read on to learn in more detail how process mining relates to specific techniques and technologies.
1. Business Process Management
The BPM life-cycle shown above is often used to describe how BPM iterates through multiple phases in designing, implementing, analyzing, and then re-designing the processes.
Process mining clearly fits into the analysis phase of the BPM life-cycle. While traditional BPM approaches start with modeling the process, process mining starts by understanding the processes that are already there by discovering the actual processes from data.
Note that BPM is not about implementing BPM systems. Instead, BPM an activity, a practice, about improving processes and might not involve any technical system implementation at all.
2. Lean Six Sigma
Process mining as a technology is agnostic to the specific process improvement (or auditing, risk management, etc.) methodology that is used with it. One popular method that is used in many organizations today is Lean Six Sigma and one of the common approaches is the DMAIC (Define, Measure, Analyze, Improve, Control) approach illustrated above.
Process mining can be used in the ‘As-is’ analysis phase to identify waste and improvement opportunities much faster and more accurately than it would be possible with a manual process mapping approach, but it also provides the opportunity to repeat the process analysis and help with controlling and sustaining the change (something that is otherwise rarely feasible today).
3. Data Mining
Data mining is much older than process mining and, like shown in Wil’s picture above, rarely focusses at processes. A typical data algorithm can be used to derive rules from data about, for example, what people are buying together in a supermarket, or predict in which suburb a marketing campaign would be most effective.
Process mining, like data mining, uses data but discovers and analyzes process models to understand how the ‘As-is’ processes look like. Research-wise, process mining comes more out of the BPM community than the data mining community. There are many possibilities to combine the two areas as they are largely complementary.
4. Business Intelligence
Business intelligence tools and dashboards have to deal with many of the same challenges that process mining has to deal with when analyzing end-to-end processes. For example, data must be pulled together from multiple IT systems and process mining can often benefit from these existing data preparation routines (many of our customers use Disco based on data from the same data warehouse that also feeds their BI dashboards).
These dashboards then analyze specific, pre-programmed Key Performance Indicators (KPIs), but they do not show how the processes work. Process mining is a complementary tool that allows to analyze the processes and find out the root causes of why the KPIs are out of bound. That’s why we sometimes compare BI to a fever thermometer (showing you whether you are sick) while process mining is like an x-ray (looking inside to see what is actually going on).
While people sometimes mistake our process mining animation for simulation, you could almost say that process mining is the opposite of simulation: Process mining starts with the current behavior and automatically discovers a model to show how the process really looks like. Simulation starts with a model and allows to explore alternative ‘what-if’ scenarios.
One big challenge for business process simulation is of course that you need to have a good model of reality to start with. Here, combining process mining and simulation (taking output from a process mining tool and use it as input in a simulation tool) provides powerful opportunities to get a more accurate starting point for the simulations.
6. Big Data
There is still a lot of buzz around Big Data, and one of the challenges that I see people having with Big Data is to extract value out of it. Process mining as a new data analysis technique that is focused on processes provides clear benefits for anyone who seeks to understand their underlying business processes in the pile of data (big or small) that accumulates.
One thing that Big Data has achieved is that it has raised the awareness about how much data there is today. Ten years ago, nobody believed that they had the data to do process mining. Nowadays, when you show process mining to someone they almost instantly recognise the opportunities and think of the data they could analyze with it.
Excel and other query tools (or auditing tools like ACL and IDEA) can be powerful at answering questions that you already have. But they do not allow you to discover new things that you would have never thought of. Furthermore, just like data mining and BI tools they do not provide you with a process perspective.
For concrete examples see this article on Why Process Mining is better than Excel for Process Analysis.
What other techniques or tools would you like to see compared to process mining? Let us know in the comments!
Last year we did three webinars to help you get started with process mining — one in English, one in Dutch, and one in German.
You can now watch video recordings of all three webinars on YouTube.
The videos are about one hour long and a great starting point for everyone who is new to process mining. We cover the following topics:
- What is process mining, and why do I need it?
- How does it work?
- Process mining with Disco
- Case studies
Watch the recordings below.
Watch on YouTube.
Watch on YouTube.
Watch on YouTube.
Also people who already knew about process mining before told me that after seeing this talk they finally understood why it was useful.
Feel free to embed the video on your website or company wiki to help us spread the word about process mining!
The date has been set! This year’s Process Mining Camp will take place on Wednesday 18 June 2014, again in Eindhoven, the Netherlands.
Mark the day in your agenda and check out the new website at www.processminingcamp.com
This is the third year of Process Mining Camp, the process mining conference for practitioners, and this year we are taking it to the next level: More and more companies have gathered experience with process mining by now and the state of the art is advancing. We are busy putting together an exciting program of practice talks, and we are working hard to make the workshops even better than last year.
Last year’s camp was attended by just above 100 process miners, and since that is about the number of people we can fit in the Zwarte Doos, we expect this year’s Process Mining Camp to sell out quickly. To make sure you don’t miss out, leave your email address on the camp website and you will be the first to know when the registration opens.
See you at camp!
The BPMCon is one of my favorite BPM conferences. And it’s not just because Jakob and Bernd are great, but they always invite interesting, unusual keynote speakers and also try out new presentation formats. For example, in 2011 we could present process mining in Pecha Kucha format, which was really fun.
BPMCon 2013 – Fluxicon from camunda on Vimeo.
Last year, we got to speak about process mining again and this time the talk was recorded. You can watch the video recording above (it’s in German).
If you live in Germany and are into BPM, you should definitely think about attending BPMCon 2014 in September!
Photo 1: Dendro Poland Ltd.
This is a guest post by Zbigniew Paszkiewicz. Zbigniew describes a process mining project that he performed for Dendro, a mattress production company in Poland.
If you have a process mining case study that you would like to share as well, please contact us at firstname.lastname@example.org.
Dendro Poland Ltd., located in the Wielkopolska region, near the city of Poznań in Poland, is a medium-size production company specializing in the production of mattresses that are exported to Western Europe. Dendro is an exclusive mattress supplier for IKEA shops in Western Europe with a mattress production volume of over 2 million per year. Since its origin, Dendro Poland Ltd. has been experimenting with innovative production technologies as well as management methods to boost operational efficiency and meet the rigid quality requirements of its clients.
The process mining project was initiated by Zbigniew Paszkiewicz, Research Assistant at the Poznań University of Economics, and conducted jointly with a Dendro Poland team led by the Distribution and Warehouse Manager Justyna Tarczewska. The aim of the project was to provide insights into the warehouse processes of the company.
The operation of Dendro’s warehouse is supported by a Warehouse Management System (WMS). The WMS is used by both storekeepers and management staff. Storekeepers feed the system with data associated with their activities, such as, for example, taking delivery, organizing shipment, transporting materials to production and receiving mattresses from the inventory. The management staff monitors stock levels and supervises the storekeepers’ work.
The process mining project was launched based on two strong premises:
- Process mining could provide valuable insight into emerging managerial issues regarding the warehouse operations;
- The WMS already stored big amounts of data ready to be mined.
The project was divided into two phases:
Phase 1: Mining the data that was already available in the WMS (associated with the Product Management and Material Management processes);
Phase 2: Modification of the WMS to log additional, high quality data for refined process mining. The scope of data collected for the Product Management and Material Management processes was expanded. Furthermore, additional data about two other processes, Material Receiving and Product Shipping, was transformed and prepared to be effectively mined on demand of the warehouse manager.
Due to the limited space, only some aspects of the Product Management process analysis in the first phase of the project are described in detail in this article.
Product Management Process
The Product Management process contains activities that are required to take the product (the mattress) from the production line and to ship it to the client. Products waiting for shipment are stored in the warehouse. Products are organized in pallets which are the smallest shipment and storage units. The products are categorized into families, which are understood as mattress types. There are twenty different product families. The transport of pallets among production lines, storage areas, and shipment areas is done by storekeepers. Storekeepers work 24 hours per day on three shifts except weekends.
The Product Management process involves the following types of activities:
- Production – refers to a storekeeper who takes a pallet from the production line;
- Rest – refers to a storekeeper who puts a pallet in the storage area;
- Shipment approved – refers to a storekeeper who prepares a pallet for shipping by putting it in a shipment area;
- On fork – refers to a storekeeper who transports a pallet between production line, storage, and shipment areas;
- Shipped – refers to an actually shipped pallet from a warehouse;
- Deleted – refers to the removing of a pallet.
Each activity that is performed by a storekeeper is recorded in the WMS. Before performing any activity, the storekeeper is obliged to scan a barcode available on every pallet. The WMS keeps track of a pallet life cycle and associates each scanning with the appropriate activity. For instance, once a pallet is scanned after production (Production activity), the next recorded activity must be On fork activity and Rest activity. By choosing the option “Start shipment” in the WMS user panel, storekeepers have the possibility to perform the Shipment approved and Shipped activities. The Deleted activity is performed only in exceptional situations.
Photo 2: Storekeeper scanning pallet labels
The de jure (assumed, prescribed) model assumes the sequential execution of activities in the following order: production, on fork, rest, on fork, shipment approved, and shipped. Optionally, if a pallet is shipped first to the external warehouse and then to the client, then the process has one additional shipment activity. The Delete activity can occur at any time.
Process Business Rules
The project aim was to verify if the actual operation of the warehouse is in line with the assumed procedures and guidelines. The list below presents only a subset of rules defined for the Product Management process by the Distribution and Warehouse Manager:
- Conformance to model – Process instances must follow the de jure model;
- Work distribution – All the three shifts should perform equal amount of work. Furthermore, storekeepers are divided into two groups: (1) taking pallets from production lines and (2) shipping pallets from a warehouse. Storekeepers from one group should not be involved in activities of the other group.
- Quality assurance – All pallets shipped to a client must be checked by the quality department;
- First In – First Out (FIFO) policy – Products that were produced first must be shipped first. The FIFO rule must be satisfied for every mattress family. To conform to the FIFO rule, storekeepers must follow the recommendations that are generated by the WMS about which pallets must be handled next.
Photo 3: Dendro’s warehouse
The Product Management process analysis was performed based on 554,745 events associated with 87,660 process instances, which were recorded over a timeframe of five months. The execution of these process instances involved 55 persons.
The following information is associated with each activity in the WMS event log: activity name, activity timestamp, name of the storekeeper executing the activity, identifier of the pallet being subject of the activity, mattress family, warehouse name, an optional stakeholder comment, and an optional pallet description.
Some information was only available for a subset of activities: storage area code (Rest activity), shipment area code (Shipment approved ), recommended storage area code (Rest), and information whether the recommendation was followed by a stakeholder (Rest).
The following attributes were derived from other data available in the WMS database: stakeholder shift (day, afternoon, or night), information if a pallet was damaged, and information about whether the pallet was approved in terms of quality.
The maximum number of attributes associated with a particular activity is 12. The pallet identifier is used to group activity instances into process instances. Additionally each process instance is described with 10 attributes.
Process Mining Results
During the process mining analysis the four business rules described above were verified.
1. Conformance to Model
In Figure 1, de facto (actual) model discovered from the event log generated by Disco is presented. The presented model shows only the most frequent behavior. Overall, the model is in conformance with the assumed de jure model. The numbers assigned to activities and transitions indicate the number of process instances that appeared in the log. The darker the color of an activity and the thicker a transition line, the more frequently they were executed.
: De facto model presenting the most frequent behavior
The de facto model presented in Figure 2 now captures the full behavior that was observed in the event log. The model shows that the execution of the process is far more complex than assumed in the de jure model. In particular, many additional transitions that are not included in the de jure model appear among various activities. For example, the transition from the Shipped activity to the Rest activity, the self-loops for activities On fork, Production, Shipment approved, Shipped, and Rest are not included in the de jure model.
The de facto model indicates that not only the Shipped activity, but also Rest, On fork, and Deleted activities are closing the process. If a Rest or On fork activity is the last activity in the process instance, then this means that these process instances are still running.
Figure 2: De facto model presenting the full frequent behavior captured in the event log
The number and distribution of process variants is presented in Figure 3. Despite the relatively simple and structured process, the number of generated variants was 160. Eleven variants were categorized as allowed and desired, and they accounted for 98,82% of all the executed process instances. Process instances associated with the remaining 148 variants accounted for only 1,18% of the process instance executions. The majority of those variants were evaluated to be exceptional but controlled. Some single process instances were categorized as suspicious and needed further investigation in the form of interviews with storekeepers or the Quality Department.
Figure 3: Distribution of process variants
Overall, the Product Management process execution has been evaluated to be highly standardized and repeatable.
2. Distribution of Work
Figure 4 presents the distribution of activities over the analyzed five months. One can easily observe a regularity of work. The warehouse does not operate on weekends, which is clearly visible as well. A larger break A corresponds to the holidays that take place in Poland at the beginning of May. The two peaks (B and C) refer to the automatic actualization of statuses of a large group of pallets performed by the WMS administrator (B) and an exceptionally large shipment of products (C).
Figure 4: Distribution of events over time
The table presented in Figure 5 was generated using the Task-to-Originator ProM Framework plugin. The rows of the table correspond to different storekeepers while the columns correspond to activity types. The numbers in the table cells indicate the number of times a particular type of activity was executed by a particular person.
It can be be easily noticed that storekeepers are divided into two separate groups: Storekeepers that perform Production activities (marked with red color) are usually not involved in Shipped activities (marked with green). However, there are some rare cases where a storekeeper performs both production and shipment activities (orange color), which violates the predefined business rules.
Figure 5: Activity to person assignment
Finally, Fluxicon Disco allows the quick comparison of work distribution among the three shifts (Figure 6). The differences in the number of activities among shifts are significant. While the first shift performs 37,54% of activities, the third shift handles only the 29,12% of the activities.
Figure 6: Distribution of work among shifts
3. Quality Assurance
By filtering based on activity attributes it is possible to extract those process instances from the event log that were not accepted by the Quality Department. Exactly 12 such process instances were identified.
In Figure 7, the model that describes the execution of these 12 process instances is presented. All the 12 pallets were not shipped to a client. Ten of them were destroyed by performing the Deleted activity. The remaining two process instances were still running when the event log was created (Rest activity). This confirms a high conformance to the quality assurance rule.
Figure 7: Process instances missing quality acceptance
In Figure 7, the performance characteristics of the process are also shown. The numbers correspond to the average time of transition executions. Typically, process instances are short and transitions are performed quite fast. Only the transition from Rest to Deleted activity takes longer. This is due to one process instance, where this transition took 2 days and 12 hours, therewith raising the average.
4. First In First Out
The Dotted Chart plugin available in the ProM Framework is used for the FIFO rule conformance testing. An example of a generated chart is shown in Figure 8. The graph was created for a group of mattresses coming from a one family. Each row in the dotted chart corresponds to exactly one process instance and each dot corresponds to an activity instance. Process instances are sorted from the top according to start time. The color of the dot is associated with the activity type: green corresponds to the Production activity while red corresponds to the last Shipped activity. All other activities were excluded from this analysis.
The company efforts for FIFO assurance are clearly visible. However, some deviations from the rule are also visible, as some earlier created pallets are shipped significantly later than others that were created afterwards. In fact, some patterns of red dots (the Shipped dots) are even running in the opposite directions of the green (the Production) dots. For better visibility, yellow lines were added in Figure 8 to highlight those opposite-direction patterns.
The reason for this is the organization of stands in the warehouse: The current organization of stands forces earlier produced product pallets to be placed deeper on the stand. Access to such products requires the removal of the later produced pallets, which usually is not performed (and would not be efficient). Also the WMS provides recommendations concerning the stands, not concerning the pallets. Thus, a perfect conformance of warehouse operations to the FIFO principle will never be achieved by the company.
This shows nicely that, while process mining helps in noticing some deviations or trends, one still needs to evaluate the results in the particular organizational context. For example, the level of conformance to FIFO that can be achieved is strongly influenced by the organization of stands in the warehouse. Any interpretation of positive or negative process mining results needs to be put in context with the help of domain knowledge.
Figure 8: Dotted chart for one of the mattress families
The non-conformance of storekeepers’ behavior to recommendations generated by the WMS has been recorded for 5231 activities performed within 3665 process instances (4% of all the process instances). Only Rest and Production activities are affected by missing adherence to these recommendations. In the case of 644 process instances that did not follow the recommendation it was justified, because those instances involved damaged pallets and damaged pallets must be transported to a special storage area. The remaining number of process instances that did not follow the recommended activities is not big but influences the conformance to the FIFO rule.
The presented analysis demonstrated business value coming from process mining applied to data already available in an organization. The mining was performed on data coming from the WMS as it is, without any modifications of the system or special preparations. Even an analysis of a relatively short and structured process can result in interesting insights, especially if a larger set of attributes describing activities and process instances is available. The analysis required active involvement of the Distribution and Warehouse Manager and occasional support from the company’s IT department during the data preparation phase. No other resources were involved.
Some conformance testing questions raised by the company were not answered using existing conformance checking and process discovery methods. Those questions required the analysis of both control flow and social perspectives of the process. For instance, does the presence of the two particular storekeepers on the same shift contribute to an increase of damaged pallets? Recently proposed methods for multi-dimensional conformance analysis may be helpful here. Many conformance problems are not necessarily a consequence of storekeepers’ behavior or wrong work organization. Instead, the wrong configuration of the WMS might be an issue, for example, activities might be saved twice in the database. In such a case, process mining methods contribute to the testing of the information system itself.
Photo 4: Mattress production
Once the project is completed (the second phase of the project is still ongoing), Dendro Poland will set up a solution for on-demand process mining. Modifications made to the WMS will allow for an easy extraction of rich event logs encompassing data generated by Production Management, Material Management, Delivery and Shipment processes. Such logs can be later imported into Disco or the ProM Framework for detailed analysis performed by the Distribution and Warehouse Manager alone.