Podcast: Mining Processes to Understand What’s Really Happening in Your Business

Alberto Manuel from Process Sphere and I were invited for an interview on the PEX network in their Process Perspective Podcast Series.

The interview about Mining Processes to Understand What’s Really Happening in Your Business is 18 minutes long, and you can listen to it directly on the PEX website. For those of you who prefer to read the interview we have put up a transcript below.

Let us know what you think in the comments!

Transcript of the interview

Hello and welcome to Process Perspectives, a podcast series produced by the Process Excellence Network. PEX Network is an online and events community dedicated to Lean, Six Sigma and BPM professionals. I’m your host, Diana Davis, editor of PEX Network.com.

Coming up in today’s programme - understanding your “As Is” process faster. Process professionals can spend months mapping and trying to understand what’s really happening inside their organizations - just so that they can change it! But is there a better way?

My guests on today’s programme certainly think so and they argue that something they call Process Mining - effectively a technique for analysing data from your IT systems to find out what’s really going on with your processes - is the solution.

Dr. Anne Rozinat is an expert in process mining techniques and co-founder of Fluxicon. She has applied process mining techniques in companies like Philips Healthcare, ASML, and Philips Consumer Lifestyle. Joining Anne is Alberto Manuel, CEO of Process Sphere and an expert in Business Process Management.

Diana: Good day to you both - thank you for joining us on today’s programme.

Alberto: Hi Diana, thanks for the invitation.

Anne: It’s a pleasure to be here, thank you.

Diana: Anne, let’s start off with you…could you tell me more about Process Mining. What exactly is it?

Anne: Process mining is about bridging the gap between the ideal or documented processes and the process reality.

Imagine for example you are an operations manager at an electronics manufacturing company who is responsible for the customer service process. If a broken product is handed in by the customer, several companies like the dealer, a logistics company, the repairshop, and many different people at these companies are involved in the process.

Now imagine that there are customer complaints about the bad service, that they are waiting too long. We were working with an operations manager who was in that situation. What can you do? All you can do is looking at these individual customer cases and see what went wrong, but first of all it’s already too late - they are already complaining - and you are not fixing the root cause of the problem. The problem is in the process.

What was remarkable is that when we looked at their documented process map, they said “Yes, but that’s not really how our process works”. They didn’t trust it because - and I have seen this in other situations as well - there is often a huge gap between the ideal or documented processes and the way that processes work in reality.

So, process mining is really about bridging that gap by making the real processes visible. We do that based on data from the IT systems. Only by knowing how your real process looks like, waste can be eliminated and problems can be fixed. In the case of the customer service process, we found two major problems, of which one was a bottleneck at a forwarding company, which delayed the process on average by two weeks due to the way they were collecting their packets.

Diana: How does it work?

Anne: It’s a fact that most business processes today are supported by IT systems. ERP systems, CRM systems, Workflow systems, Order management systems, ticketing systems, and a whole lot of legacy and homegrown systems. All these systems record very detailed information about which activities are being performed, by whom, and at which point in time. These system records are like a log book in which a captain on a ship would note down events on his trip.

Process mining techniques take these IT log data and reconstructs the underlying processes out of the digital traces that are left by the real processes in the IT systems. It works in a bottom up manner: Imagine for example a simple order process, in which normally there is a quote, then the customer orders, pays, and then the goods are shipped. There will be variations of that process, for example for some customers the goods are shipped right away, because we know that these are regular customers and they will pay. In other situations, multiple proposals need to be sent, and so on.

With process mining we will extract all the thousands or millions of activity sequences of the process that really happened, for example, taking the data from last year. And that includes all the exceptions. So, we take that right out of the IT data, and - in a bottom-up manner - we can now show the complete process picture including all variations, loopbacks, and unexpected paths that the process took in reality. On top of that we can analyze the timestamp information in the log data and project the actual delays and waiting times on the process map to show where the bottlenecks are.

The advantages are that it’s quick because it’s automated, and that we get a complete, accurate picture that is objective because it’s based on facts rather than assumptions.

Diana: So it sounds like it’s a little like taking an X-ray of your processes?

Anne: Exactly! This is exactly what it is. Doctors wouldn’t think about starting an operation without taking an X-ray. Without the objective picture it’s just guesswork. And that’s not good enough. Also for business process improvement work, guesswork is just not good enough. It may not be lives that are at risk by bad processes, but the jobs of many many people depend on their companies being competitive and being able to deliver good services.

I am convinced there will be a time when process mining will be that standard X-ray procedure that makes sure you know what is going on, and to make sure you are taking the right actions.

Diana: Alberto, you’ve worked in the Business Process Management space for nearly a decade and I imagine that you’ve had a lot of experience mapping “As Is” processes. Why do you think Process Mining is the way forward?

Alberto: In order to understand what is happening regarding how business processes are being executed, companies rely on human perception of reality and this perception is a blend of tacit knowledge related with judgment, intuition, reasoning, and explicit knowledge that is related with company procedures, company policies, and culture.

The problem is that humans have different perceptions of reality. That is, if you ask people how work is done, you will get different and incomplete answers. In other words: Full of bias and affected more strongly by ease of retrieval than by the information they retrieve to explain how the process is executed.

If you add the growing complexity of todays business processes, full of exceptions, social interactions, with increasing parts of the process being outsourced: How is it possible to have an accurate snapshot of reality? How much time does it take to get it? 6 months? Can a company today wait so much time to change and to adapt to changing business conditions? Can a company take the risk of losing competitive advantage because it takes too much time to understand where change must happen?

So, the great difference of Process Mining compared with other approaches is that all that time and effort to collect information about the AS-IS does not exist. People start immediately to understand how work is done, without bias. They can jump to process improvement fast and with a clear and precise idea because its based on what enterprise systems record what is being done rather than the human perception about what is occurring.

Diana: What would you say is the key difference between traditional way of mapping processes and process mining?

Alberto: The traditional mapping approach is about drawing imprecise process models, taking too long, and in the end you don’t have a picture of what is actually happening in the company. This is the main difference.

Diana: You’ve recently had experience applying process mining at ANA airport in Portugal. Can you take me through what you did?

Alberto: Like for any other change initiative program its important to get buy-in from the stakeholders. Thus, the idea was to start with a process that the customer feels or suspects that could be improved. They had some data that was pointing to inefficiencies, meaning that the team was not headed to the typical low hanging fruit approach. At the same time the process was critical enough because we are talking about the management of IT assets that are responsible to support an Airport. Lastly, the process data from the chosen ‘Change Orders’ process (this equals Change Management under the ITIL framework) was easy to understand because the process was being managed by the Information and Communications Technologies Directorate. This means that the concept could be easily validated before being expanded to other Airport business units.

The project was performance driven and control flow driven. The two key questions were: Can we be faster and can we execute things differently? This sets up the kind of answers you are looking to and subsequently the kind of data you need to retrieve from the system that supports the process. After the data is extracted, you discover the process model and then its an interactive process of discovery, questions, answers, and all of this is led by people. Process mining is about constructing and structuring and rediscovering the knowledge about a process. Where are the bottlenecks? Where can we change the way the work is done? Where can we be faster?

One of the key buy in arguments is that it took from start to finish one single week to get the process improved. This is impossible with any other analysis and improvement approach.

Diana: Now, was it really all that quick and easy?

Alberto: I remember that when the idea was presented the first time, even before ANA airports decided to get involved, it was pretty clear that for the first time in many years ANA Airports were looking at something different, because they could understand the reality as it was. Somehow they were tired of looking at process data as they felt they were not getting closer to the answers, because the problems continued to exist.

Diana: What kind of results did you get?

Alberto: The first important change was about the process model. There was, let’s call it a worst practice of replicating workflow process model templates for each of the ‘Change Order’ categories, causing that it was necessary to execute activities that did not make sense and increased the time to execute by letting people do things that were not necessary. The process was growing in complexity and that was eliminated. The process model became much more lean to execute.

After that, three important changes were done: First, there was evidence that it took too much time to start working on a Change Order request after being submitted. The first thing that was changed here was the workforce balancing. In the past there was a rule about work distribution to different Change Orders request types that was not working. The rule was changed and brought a lot of agility. The second thing that was changed was to eliminate the noise in the process. Some requests were being submitted long before being implemented. That distracted what was necessary to look for.

Also, it was clear that it took too much time to close a Change Order request. To address that the activity sequence was changed. For some categories it was making sense to eliminate and make it parallel. That made everything simpler.

The last point was about a compliance problem that did not exist: It was concerning Change Orders that were jumping immediately to implementation (under the ITIL framework this is not allowed). What Process Mining showed was that there was a problem with data that was being recorded in the configuration management database (CMDB), which is a database that contains all relevant information about the IT assets and had to be updated manually. For that it was implemented a new procedure to reduce errors. So, in the end it was clear that there was not any problem with compliance and the problem was about the way the data was being recorded in the database.

Diana: Now this all sounds great - but how can you really trust that the picture you are building up is accurate? Isn’t there the risk that you’re relying on poor quality data or not capturing the whole picture? Anne, maybe I can come back to you.

Anne: That’s an excellent question. What is important here to understand is that although the process mining analysis is automated, it’s not just applied blindly. Usually, the process analyst who applies Process Mining works together with a domain expert, the operations manager or someone else working in the process, and these people know their processes very well. So, for example, if there is a manual step in the process that is not captured in the data, a phone call for example, then this information will be taken into account when the results are interpreted. So, common sense and process knowledge go hand in hand with the actual process mining analysis. In this way, you can fill in gaps that might be there in the data recording.

As for data quality problems, these are taken care of in the data-cleaning step. When the data are extracted and imported, they are first screened for data quality issues and problematic data are then removed or excluded from the analysis.

In fact, just the insight that there are data quality problems is often very valuable to companies already, because they recognize how important good quality data are to be able to perform data analytics today. So, knowing that these issues exist helps them to fix them, and then the next time they can get even more valuable results from the analysis.

Diana: Final question and I’ll get you both to come in on this one - if I were an organization considering using process mining to analyze my As-IS processes, what are the key things that I need to take into account to decide whether this approach would be appropriate for me?

Anne: It has a lot to do with the mindset. You have to be willing to change your opinion if the facts change. Sometimes it’s more convenient to not know about certain problems, because then you don’t have to do something about them. So, this is about the mindset, to be willing to look at the truth, and to be aware that once it is out there it’s much harder to ignore. Process mining is a technique that works very well but it needs to be applied in an organizational setting, where people are willing to take this up, and really apply it and do something with it.

Alberto: I would say the only impediment is if your company does not have data. If there is no data there is no Process Mining. Its not about if the approach is right or wrong. Process Mining is the answer to organizations that quickly want to modify their business processes and cannot be waiting for long cycles to change: Organizations that have to cope quickly to changes in the environment in which they operate, that must adapt very fast.

Diana: Anne, Alberto, thank you very much for joining me today.

Anne: Thank you very much for having us.

Anne Rozinat

Anne Rozinat

Market, customers, and everything else

Anne knows how to mine a process like no other. She has conducted a large number of process mining projects with companies such as Philips Healthcare, Océ, ASML, Philips Consumer Lifestyle, and many others.