Process Mining Challenges

Process Mining 2024

A big part of this year’s Process Mining Camp was to collect and discuss the challenges we all experience when applying process mining. Some challenges are generic. Others depend on your organization or emerge from your current skill set.

Traditionally, there has been a division between data-oriented and process-oriented disciplines.

For example, data science and BI teams have a lot of data and statistics skills. But they are not responsible for the process and often don’t understand it. On the other hand, business process experts recognize things like rework when they see them, but they are not used to leveraging the data that is available. The same applies to auditors who know how to watch out for various risks in the process but might not yet be able to handle large data sets.

In fact, process mining bridges this gap by marrying process and data-driven skills. And you, the process miners, are doing this! By using data to understand processes.

However, you can also look at your own skill gaps to improve further. For example, if your background is in BI or data science, we recommend familiarizing yourself with methodologies like process analysis or lean six sigma, and picking up skills like change management. Or are you more coming from the process analysis or audit side? Then, it can be really valuable to develop some additional data skills.

Data Challenges

Process Mining 2024

In the Data Skills workshop at this year’s camp we did just that. Together, we transformed and improved a data set step-by-step. We also discussed best practices for the data challenges that we collected the day before.

Overall, the most frequent data challenges that were mentioned fall into the following four groups.

DC1: No data or no access to the data

Some data is not captured. After all, most IT systems did not think about process mining when developing their data structures. Essential data fields might be missing completely (for example, instead of the history of status changes only the last status is captured) or partially (for example, there might not be a single identifier from beginning to end if multiple systems are involved).

Sometimes, getting access to the data can be difficult. For example, an external provider might manage the IT system, and data access is not included in your contract. Your own IT team might have no capacity to help you due to competing priorities. Furthermore, legacy systems and a lack of knowledge of all processes can make it difficult to know where to look for data in the first place.

Best practices

  • Know the data requirements for process mining.

  • To get access to the data, involve & inspire higher management by sharing process mining success stories that illustrate the benefits in monetary terms.

  • Develop a good relationship with data owners.

  • Involve data experts early in your process mining project and try to make them enthusiastic about what you are trying to do.

  • Work together with the data science team to extract and make it suitable for process mining.

  • Team up with your internal audit team as auditors can make their data extraction a priority.

  • Renegociate contracts with external providers to make sure they provide you with your data.

  • Make sure to include data requirements for new IT systems in the future.

DC2: Lack of data skills

Even if you have data, it might not be in the right format. Furthermore, it can be challenging to extract the data due to its volume or because of your data skills. You might need to merge data from different sources. Finally, the scope of your process determines the scope of your data, and you need to have an understanding of how the target data should look after your preparation.

Best practices

  • Focus first on the core data and extend it further in a second step.

  • Start with a small sample and scale up your data size later.

  • Learn to translate the process questions in the right data combination.

  • Avoid repeated copy and paste in Excel and develop data skills, making the data extraction and preparation repeatable.

  • Keep in mind that developing an event log is an iterative process (don’t try to do it right the first time).

DC3: Data quality problems

As with any data analysis technique, data quality problems can make your analysis harder or impossible. Garbage in = garbage out. Especially manually collected data is often unreliable. Furthermore, a lack of precision in the timestamps can create wrong orderings in the events.

Best practices

  • Don’t skip the data validation session.

  • Use the data quality checklist to spot (and clean) common data quality problems in process mining.

  • Understand how the data is collected during the process execution to foresee mismatches between data and reality.

  • Use the data that you have rather than waiting for the perfect data.

  • Check how much of your data remains after the cleaning step to see how representative it still is.

  • Make event data quality a priority in your data architecture.

  • Improve data quality if you can influence the data collection in the future.

DC4: Security and privacy concerns

If process mining is still new in your organization, you may encounter a reluctance to share data. Reasons can include security and privacy concerns and a fear of misusing the data. The rules regarding employee and customer data vary across different countries/entities.

Best practices

  • Make sure you consider privacy, security, and ethics questions from the start.

  • Connect to your legal team to understand which rules apply to you (e.g., GDPR).

  • If you are in Germany or another country that prioritizes workers’ rights, involve the workers’ council at the beginning of the project.

  • Minimize data to what you really need.

  • Address privacy concerns and create an ethical charter.

  • Foster a collaborative culture that helps people to speak the truth about their process.

  • Data ownership and access should be a priority. Create a data access policy and a privacy guideline for process mining, and make event data part of your data governance.

  • Anonymize data to hide sensitive information if needed.

  • Be aware that even anonymized data may be traced back to individuals (adjust your level of protection to the type of data and the situation).

  • Instead of cloud solutions, you can use a local process mining tool like Disco or ProM, where your data is not uploaded but remains in your organization.

Process Challenges

Process Mining 2024

In the analysis workshop, we learned how to separate incomplete cases and how to identify the standard process and its deviations based on patterns in the data.

Overall, the most frequent process challenges that were mentioned fall into the following four groups.

PC1: Dealing with complexity

Certain processes, such as customer journeys or healthcare processes, are particular complex. For example, clickstream sequences on websites and patient pathways are very individualized. So, they contain a lot of variants. But also other processes seem quite chaotic when you first look at their complete picture in the process mining tool.

Best practices

  • Remove incomplete cases, explore different perspectives, and separate data issues from process issues before you start with the analysis of your process.

  • To find structure in the chaos, apply simplification techniques.

  • Look at individual cases and variants to see example scenarios of the process.

  • Identify events that indicate the beginning and end of the process as well as important steps in the middle. Then, use the Milestone simplification method to visualize the process just based on these milestones (hiding the rest) to get an overview.

  • If you have a reference process (it can be just a whiteboard drawing from a subject matter expert), find those referenced milestones in the data.

PC2: Lack of process mining skills

Learning how to read the process map, how the sliders work, and which filtering mode does what is essential. Luckily, working with the process mining tool can be practiced, and you will gain more experience over time. If you are unsure, you can always look at the user guide or follow a training.

Best practices

  • Always double-check whether your process mining results are in line with your expectations. For example, if you filter completed cases based on an endpoint and end up with just 20% of the cases from the initial data set, inverse the endpoint selection to inspect the incomplete cases to see if you have missed valid endpoints.

  • To find specific activities or events, you can search them in the process map or the cases view.

  • Click on paths in the process maps that seem odd to you and look at example cases.

  • Use the synchronized animation to compare process behavior for different segments.

  • Use TimeWarp if your SLAs are based on working time rather than calendar time.

  • Make copies when you explore different analyses to preserve your current views.

  • Keep your workspace organized by renaming and re-ordering data sets.

  • Use the recipes to save and reuse filter combinations.

PC3: Lack of process mining methodology

In addition to learning how to use the process mining tool, you also need a plan for how to set up your project. Which process do you choose, and how big should you make it? We are currently putting together the essential ingredients in this 12-step project guide.

Best practices

  • Find multiple processes to mine and eyeball the data for the low-hanging fruits.

  • Take a small process to prove value fast.

  • Define an objective (important for data selection and to avoid getting lost in the data).

  • Look at the process with the most pain points or the largest business value.

  • Embed your process mining practice into your existing way of working.

  • To ask the right questions, identify the “What keeps you up at night?” and ask “If you had a magic wand and could make a wish for this process, what would it be?”.

  • Keep in mind that the questions need to be made more precise once you start answering them with the data.

  • If there are no targets for a process yet, you can use process mining for an initial measurement and define the ambition for, e.g., throughput times or other SLAs.

PC4: Missing domain knowledge

You can do a lot of exploration by just looking at the patterns in the process and inferring the most likely meaning from the names of the event labels. But ultimately, you will hit a wall and need domain knowledge of the process behind it. If you are not a subject matter expert yourself, you will need access to someone who can answer your questions.

Best practices

  • Involve stakeholders early.

  • Have a close connection with the business and select projects based on sponsorship.

  • Try to understand the process yourself before consulting the expert. Keep a list of all the questions that you encounter along the way.

  • Create a multi-disciplinary team (see the skills and roles that are needed for your process mining project here).

  • Understand the process behind the data. For example, follow the process manager for a day (shadowing) to see a “day in life.”

  • Don’t be afraid to ask “stupid” questions. Your outside perspective can be precious!

Organizational Challenges

Process Mining 2024

In the third and last workshop, we examined the organizational challenges and brainstormed best practices for addressing them.

The most frequent process challenges that were mentioned fall into the following five groups.

OC1: Stakeholder management and sponsorship

Just as important as data availability is good support from the team responsible for the process. Who do you need to involve? How do you ask the right questions? And how do you manage the right stakeholders?

Best practices

  • Identify a process owner, engage in dialog, and ask “Are you aware how the reality deviates from the expected process?”

  • Quick workshop/demo to show, not tell, advantages.

  • Discuss with auditors about the risks to look for.

  • Derive “right” stakeholders from the use case.

  • Use classical stakeholder map to classify stakeholders.

  • Involve: Budget holder, CFO/controller, IT manager, process owner (see also skills and roles).

  • Engage with stakeholders early and make them part of the project from initiation. Communication: Explain the why, how, when, and what.

  • If you sense a lack of trust, have an honest dialogue and answer questions that those hesitants might have. Show how it works, and be transparent and honest!

OC2: Business case & value

If your company is not yet familiar with process mining, they might not see the benefits (yet). Finding the proper business case can be challenging and is often a chicken-and-egg problem: How do you know how much value you can realize before looking at the data?

Best practices

  • Use experiences from the community to show process mining examples that are relevant to your organization.

  • Tailor the business case to your use case. For example, auditors can save time on audits or avoid risks or fraud cases in the future.

  • Identify company-/business issues (where KPIs are not met) and ask customers for the biggest pain.

  • Choose the process where data is easily obtained to show the advantages quickly.

  • Balance effort & reward when evaluating (accounting for the skill of the organization).

  • Close the loop after your project: Measure the impact of the change to demonstrate the value you have delivered.

  • To sustain improvement, define performance metrics and build a program to track them regularly. This helps prevent the process from falling back into old patterns.

  • While privacy is not easily quantifiable in business value, you can include it in your business case as an avoided risk.

OC3: Adopt methodology (and drive change)

Having the insights from your process mining analysis does not do anything yet! You need to change the process to realize the benefits. We discussed how to manage this change and how to expand after your initial project.

Best practices

  • “Start small” & be clear about the goals & impact. Have regular follow-ups & checks.

  • Be open about the change and leverage the change management professionals in your organization.

  • Demonstrate the value of process mining (before vs after).

  • Standardize your process mining practice before expanding.

  • Appoint process mining champions per business unit.

  • Team up with other groups, such as the BPM group, lean six sigma experts, or the data science team to develop an integrated approach and join forces.

OC4: Connecting to strategy

Connecting your process mining initiatives with your company strategy is a good way to align them with your organization’s goals. For example, process mining can support the digital transformation journey that many companies are still making.

Best practices

  • Create a clear and concrete use case that is aligned with your company’s goals.

  • Thoroughly understand the company strategy as an analyst.

  • Link your project to cost savings, efficiency, and operational excellence.

  • Link strategy with the business case.

  • Introduce and use OKR to break down company strategy (objective) into process mining tasks (key requests).

OC5: Develop a process mining capacity within the organization

How do you take the next steps after your first process mining projects? You want to skill up the organization, get new assignments, stay up to date, and build a community that shares what they have learned.

Best practices

  • To build up expertise, start small and perform multiple pilot projects.

  • Celebrate successes and use your successful projects to market the added value of process mining and get more people interested.

  • Offer awareness training for leadership positions. Make people aware of the possibilities in all layers (bottom-up/top-down).

  • Go to Process Mining Camp :)

  • If you don’t have enough internal resources, hire a consultant to help you with your first projects. Make sure you can take over what they did for you after the project.

  • Participate in hands-on process mining training.

  • Schedule meetups with peers. Document knowledge & best practices.

  • Promote process mining by nominating process mining champions.

  • Start an innovation lab.

  • Read and research about what is new in the field.

  • Sign up for newsletter.

Process Mining Café this Wednesday

Process Mining 2024

In the upcoming Process Mining Café tomorrow, this Wednesday, 10 July, at 15:00 CEST (Check your timezone here), we will review what we learned from this year’s Process Mining Camp. Come join us!

As always, there is no registration required. Simply point your browser to fluxicon.com/cafe when it is time. You can watch the café and share your thoughts while we are on the air, right there on the café website.

If you want to be reminded, you can:

— See you at the café!

Thank you for joining Process Mining Camp 2024!

Process Mining 2024

We had an amazing time at Process Mining Camp this year. Whether you were with us in Eindhoven or joined us via the livestream, thank you so much for being there with us!

Practice talks

Process Mining 2024

First, we heard from fellow campers who shared their process mining experiences from various use cases.

Gerardo Rodriguez Oviedo & Rafael Herrera Carpio from BAC in Costa Rica showed us how an initial one-man project can have a big impact. They created 4 million USD more demand deposits in just six months. The key for them was aligning objectives across business units from the customer perspective.

Lieven Bewaert from Atlas Copco in Belgium taught us that anyone can be an auditor by applying curiosity and critical thinking. His exploration uncovered a bug in the IT system and relieved an overloaded worker in the purchasing department.

Alexandros Kakakis from Online Dialog in the Netherlands took us through a website optimization project that went beyond a classical funnel view. The client wanted to understand the real customer journey paths on their site. The process mining analysis generated improvement ideas for AB Testing.

Ghada Zakhama & Caleb Borges from DHL Group in Germany showed us how their data analytics team supports other internal auditors at DHL. Reviewing data quality is as important as checking compliance to avoid the risks of bad service quality, contract penalties, and loss of reputation.

Aljoša Jankov from OTP Bank in Serbia analyzed the credit card transfer across multiple branches. He discovered that people filling in at different branches on different days caused delays in the process. Furthermore, a change in the IT system needs to remove duplicates to lighten the workload.

Erik Scroggs from Serco in the USA showed us how 100 RPA bots and 600 humans work hand in hand. Using process mining, Erik could improve the handover of work between robots and humans and quantify the benefit of automating further parts of the process.

Sign up for the camp mailing list here by the end of today to receive the public slides from the presentations tomorrow.

Discussion roundtables

Process Mining 2024

Next, we sat together in small discussion roundtables to discuss our challenges with process mining.

For an easier start, we had curated the groups along themes such as healthcare, lean six sigma, and customer journeys based on the campers’ interests and backgrounds. However, the discussions quickly broadened and covered all kinds of challenges any process miner encounters.

Each group then dug deeper to uncover the underlying issues and causes. We organized them into data, process, and other challenges.

Finally, each group reported back by sharing their main takeaways from the discussion with all of us.

Evening

Process Mining 2024

It was time to get some food and relax. We had the whole evening together to continue our discussions and learn from each other.

After dinner, we rolled out the bingo cards. We thought we had made answering some of the questions difficult or even impossible. I still can’t believe that Erik knew the first ten digits of Pi by heart. And who was it who had done process mining in secret? (Let us know where and how 😃)

With the motivation and combined smarts of our community from all over the globe, solutions were found very quickly. Thank you all for playing!

Hands-on workshops

Process Mining 2024

The next day, we started fresh and got to work. In two hands-on workshops, we practiced creating a suitable data set and learned what to watch out for before analyzing the data in the process mining tool.

Hopefully, everyone learned something new. Some of you have taken their first process mining steps. Others have sharpened their skills and learned from each other.

Your training licenses are still valid until the end of August. So, you have all summer to revisit the exercises at your own pace.

Organizational best practices

Process Mining 2024

In the third and final workshop, we revisited the remaining challenges from the first day and collectively captured best practices to address them.

There were a lot of useful and creative ideas. We also closed the loop by locating where they fit in the project cycle of the process mining methodology.

All together, we have been capturing use cases for process mining for well over a decade. This year’s talks, roundtables, and workshops make for some great additions to the community’s knowledge.

It was really fun to spend these two days with you all. Thank you!

What is next

In the upcoming Process Mining Café on Wednesday, 10 July, at 15:00 CEST (Check your timezone here) we will review the learnings from this year’s Process Mining Camp. Come join us!

As always, there is no registration required. Simply point your browser to fluxicon.com/cafe when it is time. You can watch the café and share your thoughts while we are on the air, right there on the café website.

If you want to be reminded, you can:

— See you at the café!

Livestream At Process Mining Camp 2024

Process Mining 2024 Live

Process Mining Camp starts tomorrow — We can’t wait to see you all in Eindhoven!

For those of you who are going to join us here, you are in luck: We believe that this will be our best camp ever, with two full days of process mining practice talks, roundtable discussions, intensive workshops, and lots of time with your fellow process mining campers!

For those of you who can’t join us in person, here is the next best thing: Join us around the globe for a livestream of this year’s practice talks on Thursday, June 13! Take a peek through the cracked doors of our campsite, and learn about what our speakers have experienced.

Note that we can’t take questions1, and while we try to give you a great experience, we cannot guarantee a perfect view — After all, camping is a contact sport, and we usually have our hands full with making sure that everything’s perfect for the people in the room. But we hope that you’ll join us here in spirit, that you have a good time, and of course that you can make it next year!

To watch our livestream, simply go to processminingcamp.com tomorrow, June 13th, at 13:00 CEST (check your own timezone here). No registration is needed. You will be able to watch the livestream right there.

— See you all tomorrow!


  1. Of course we can’t wait to hear from you, though: Hit us up on LinkedIn, Twitter, or of course by email↩︎

Vanessa Schindler @ Process Mining Camp 2023

You can now watch the video recordings from last year’s camp. If you have missed them, here is the first talk by Eric N. Kildea from Radwell in the United States, the second talk by Stefan Wick from the Universitätshospital Zürich in Switzerland, the third talk by Xhentilo Karaj from Euroclear in Belgium, and the fourth talk by Franck Diafouka, Ruth Zonanashvili, and Mithra Gruber from the European Medicines Agency in the Netherlands.

Our fifth and last speaker was Vanessa Schindler from T4media in Germany. Vanessa showed us what process mining adds to her toolbox as a customer journey analyst. Of course, she still uses web analytics tools like Google Analytics, but process mining helps her focus on the user’s actual behavior. Technically, the data is available without any problems: The Case ID is the user on the website, the Activity is the website’s page name, and the Timestamp is the time of the visit. What is difficult is the complexity of the user journeys: The data needs to be simplified to answer targeted questions. Watch Vanessa’s talk to see how you can better understand the customer journey based on process mining.

For this year’s camp, this is your last chance to get your ticket here.

— See you all in Eindhoven this Thursday!

Franck Diafouka, Ruth Zonanashvili, and Mithra Gruber @ Process Mining Camp 2023

You can now watch the video recordings from last year’s camp. If you have missed them, here is the first talk by Eric N. Kildea from Radwell in the United States, the second talk by Stefan Wick from the Universitätshospital Zürich in Switzerland, and the third talk by Xhentilo Karaj from Euroclear in Belgium.

Our fourth speakers were Franck Diafouka, Ruth Zonanashvili, and Mithra Gruber from the European Medicines Agency in the Netherlands. Franck shared their experience from establishing a working group on process mining across different EU organizations. Ruth explored process mining for risk assessments. Mithra then connected the various stakeholders of process mining to challenges beyond technical aspects. For example, ethical considerations are essential to avoid staff discrimination by colleagues or management. Furthermore, data protection and privacy must be ensured to comply with EU DPR and other legislation. Watch Franck’s, Ruth’s, and Mithra’s talk to see what it looks like to work on process mining at the European Union.

For this year’s camp, you can get your ticket here.

— See you all in Eindhoven on 13 June!