How are the latest trends increasing demand for Data Scientists?
February 9th, 2023
Mark Humphries is a Senior Business Consultant at IPL, a leading, independent IT & Consulting company based in Bath. In this post Mark shares his thoughts on the importance of good data and why getting it right is crucial to business success.
In my previous blog, I made the case that getting your business to 99% and beyond can be a source of sustainable competitive advantage and so should be a priority for business leaders. In this article, I will explain how to get to 99% and beyond, by managing exceptions.
Most organisations are structured according to processes, and most do a lot of repetitive work (although there are exceptions that only work on unique projects). Their activities are defined by processes, which are triggered by events and lead to predictable results. Whether you’re buying a burger on the high street or an exotic holiday online, the decision to purchase triggers processes that ensure you pay for what you ordered, that it is delivered, and that everything is lined up for the next customer. Businesses aim for consistent processes to ensure that the end result is the same – regardless of who takes the order or what day of the week it is.
However, real life has a way of throwing up surprises that are not foreseen in these processes. Even if you get your customer service team to follow a consistent script, you can’t always expect your customers to follow the same script. Processes fail because things happen in the world outside your business that weren’t foreseen in your processes.
When I lived in Germany, for example, I opened a joint bank account with my wife. At that point we generated an exception because my wife chose to keep her maiden name when we married. In those days, German banks assumed that married couples had one surname; separate surnames on a joint account were not foreseen. To open the account, my wife had to be known as Mrs Humphries. While this annoyed her, she could live with it, until the day when she triggered another process that required authentication of ID. Of course, she couldn’t produce ID to prove that she was Mrs Humphries, because she wasn’t. Luckily, the man who opened the bank account for us was available, and he was able to confirm her identity.
This is an example of an exception, and it’s something that happens daily in every business. In this case, an employee took the initiative and saved the situation. His intervention was dependent on his personal knowledge, though. Had he not been there, the bank would have had an angry customer on its hands. Just as important is this: had there been another bank that allowed husbands and wives to have different family names, we would have switched banks.
If my example doesn’t convince you, think about the frustrations that you have had as a customer because you’re asking for something that falls outside of the mainstream. The simple act of moving house will give you an insight into the number of companies that assume people only have one address and a credit history linked to that address. After all, it’s an assumption that works most of the time.
Understanding the nature of these exceptions and dealing with them will take you from getting it right 95% of the time to getting it right 99% of the time and beyond. For the record, 100% is not achievable. How many 9s you can achieve is what’s important. The thinner your margins, the more 9s you need.
Exceptions arise because process designers can’t foresee everything that’s going to happen. Reality is stranger than most of us realise, until we’re presented with evidence. Process designers typically start with a number of use cases and work them through. A use case is an example of a scenario that triggers a process. They use these use cases to explore possibilities that seem reasonable to them, based on their experience. What they don’t do at this point is to think about all the possible exceptions that might occur. If they did, they’d probably go mad, or be starting on a piece work that has no end.
In fact, it’s a good thing that they don’t attempt to foresee every possible exception, because they would waste a lot of time preparing for exceptions that will never happen. One aspect of exceptions that has surprised me over the years is how many potential exceptions never actually happen. I’ve seen process analysts tying themselves in knots trying to foresee every single possible problem, only to end up with unwieldy processes that are still caught out because something else happened that they didn’t foresee.
The good news is that managing exceptions is actually not that complicated. But first you have to acknowledge that they’re an unavoidable fact of life, and that they’re important enough that you need to deal with them. It’s not possible to engineer processes so that exceptions don’t occur. If you can’t accept this, then you should stop reading now, because the rest of this blog assumes these points to be true.
Below I describe three relatively simple techniques, which, when used together, are an extremely effective way to manage exceptions.
1. Build in exception-handling
The first technique is to build exception-handling into your processes and your organisation. You don’t have to know what is going to happen, but you do need to know what conditions should be met to proceed to the next step. When those conditions are not met, you include a step to handle the exception, you nominate people whose job it is to handle it and you create a consistent means for delivering those exceptions to them. What you don’t do, is try up front to describe exactly what the nature of the exceptions might be and what those people should do in response.
Instead, choose people who are good problem-solvers and give them the authority to use their initiative. Also consider giving them the option to overrule a control. A simple example is when an item in a supermarket is wrongly priced. The cashier cannot overrule the till and change the price, but they can call the supervisor, and if they’re happy, the price can be changed. The supervisor has the authority to overrule the process. So, the processes foresee exceptions (without prescribing a solution) and a means of passing them on to people who can deal with them. The organisation puts in place people who don’t work in the mainstream processes, but rather parallel to them, effectively dealing with all the exceptions that the process can’t deal with, and putting things back on track.
When these exceptions happen, it’s important that the process captures this information. One reason for doing this is to ensure there’s an audit trail of who is overruling your validations, and how often. While it’s probably the right thing to do, it could also be fraud.
2. Improve your processes using root cause analysis
More importantly, logging each exception enables the second technique, which is to track the exceptions and perform root cause analysis to understand why they’re happening. This will enable you to eliminate the exceptions by improving your processes appropriately. This technique is powerful because it’s fact-driven, enabling you to focus on real problems, rather than the imagination and insight of process analysts. It also enables you to solve the problems that matter most, because they happen more often.
3. Tackle the data quality problems
Possibly the richest technique that I have used though is to delve into data quality problems and to feed them back into process improvement initiatives. This is less obvious than the first two suggestions, but often even more effective. Data and process are interdependent. Processes consume, modify and generate data, which in turn steers the processes. Process designers make assumptions about the data that drives their processes.
If you can define these rules, and then search for data that doesn’t respect them, you win twice. Firstly, you can identify data that will cause process exceptions because it doesn’t respect the business rules for the process. If you can identify the problematic data, you can fix the problem before it happens. Even more valuable though, is that you get to see examples of how reality doesn’t fit your expectations. By finding and analysing data that doesn’t fit your rules, you get to understand the environment in which your business really operates, and how this is different from the way you assumed it works. If you then feed this back into your processes, again focusing on the problems that really occur rather than those that might, you have a really powerful technique.
So that’s it: three techniques that can be the difference between getting it right 95% of the time and 99+% of the time. If you foresee exceptions rather than fight them, and accept them for what they are, you can reduce your waste, boost customer satisfaction and increase your competitiveness. Furthermore, it’s a virtuous circle: once you allow for exceptions, and allow them to drive your process improvement initiatives, you can actually eliminate the most important exceptions by building ways to handle them into your processes.
Mark Humphries – Senior Business Consultant at IPL
KDR Recruitment is the home of the best data, technology and analytics jobs. For more news and views follow KDR on LinkedIn.