Now that you understand the overall ecosystem at a high level, have inventoried key processes and even have taken a look at use cases to understand the players and how they relate to each other, you need to think about what will make the operation successful. Just like in software design, as you build out components of the software, each component needs to have specific test cases written. This helps the development team understand if what they are building is directionally correct during development instead of waiting to the end where remediation becomes more expensive and more complicated. These test cases translate to success criteria in a business operation.
Many people begin designing solutions to problems too early. It is natural to try and solve a problem if you see one, of course. With that said, it is not clear how you can even begin to think about solution design if you do not already have a complete understanding of what success means.
Defining success criteria is not a daunting task on the surface and many operations managers spend too little time doing it. Defining success criteria involves thinking deeply about the output from your specific machine or your business operation as a whole and envisioning what a good outcome is. One of the more challenging aspects of defining success criteria for an operations manager is thinking about a good way to measure success.
For example, if you were implementing a “know your customer” program as part of a broader client services organization it would be easy to list as success criteria that if you “understand your customer” then your program would be deemed successful. While that is certainly true at the meta-level, what does “understand” mean in this context? Does it mean that you understand how your firm’s actions impact your client? Does it mean that you understand how outside market conditions are affecting your clients’ behavior? Chances are it means both, but you need to define a method of measuring what is in effect an intangible thing. Usually this requires a series of measurements that are layered on top of each other to indicate the desired condition.
For example, understanding how your firm’s actions impact a client might be measured by looking at the number of support calls that your organization fields from that customer and how that number changes over time or compares to the average. Does the call volume increase or decrease when you release a new version of your software, for example. If the call volumes increase, maybe the release did not solve the problem as seamlessly as you had hoped. By looking at outliers, you might be able to conclude that clients with more than 25 calls per month on average are not happy and that can trigger some behavior on your team’s part to determine if the customer is at risk.
In this example, you might look at average call volumes and determine that 10-12 calls per month per client represents normal activity but there are often spikes of up to 20 calls per month. When you back test your data you might see that your firm often lost clients that called in on average more than 25 times per month because they were not happy with the product or service. So your success criteria might be “maintain per client support call volumes below 12 calls per month on average over an extended period”. Now you can begin to diagnose how you interact with clients who already meet that criteria and bake those practices into the overall operation.
This is obviously a simplistic model in that more than likely there are many other criteria that you would need to look at together to determine if a client was at risk. Call volumes on their own are likely not all that meaningful. But call volumes are something that you can measure and by looking at history of volumes for longstanding customers as well as for customers that your firm has already lost you might be able to uncover a pattern that can be indicative of a specific client behavior and this gives you something to measure against and helps you ask good and timely questions. If you can identify three or four other behaviors that can be measured, you might be more successful at stemming erosion or maintaining a higher client satisfaction rate.
There are many other kinds of ways to measure the success of a business operation. Many operations managers try to establish benchmarks as a measurement. This can be very straight forward such as the number of widgets processed per day per person or the number of calls made to prospects. Based on historical performance the manager knows that productivity above a certain level is representative of a successful operation. Like call volumes in the previous example, these benchmarks are usually indicative of success or failure but do not determine success or failure of an operation. Importantly, what these benchmarks represent is an opportunity for the managers to ask good questions so that they can deal with or better understand exceptions.
So, success criteria can be simple or complex. They can be literal or indicative, and they should to the greatest degree possible be measurable. A good set of well-defined success criteria will help the operations manager understand what the end-state of a good business process should look like.
Developing benchmarks and success criteria can be difficult without good metrics. While there are many ways to produce quality metrics out there, what is really important is simply understanding what you have to work with. For example, many operational metrics are statistical in nature. In other words, while they will describe one dimension of an operation narrowly, they will not tell a story.
The most basic version of statistical reporting is “volumetrics”. While volume based reporting tells you what has been produced (i.e. number of widgets, phone calls or service requests) it does not tell you anything about why production was what it was or what the trend, if any, means. That is not to say that statistical reporting does not have its place. It is a foundational level of reporting that can lead to more robust reporting that can help you ask the right kind of questions. This reporting is analytical reporting.
Analytical reporting tends to be multi-dimensional and will include pattern data. Analytical reporting will focus on trends and call out outliers. It will typically include thresholds and trend lines as an overlay so you can see where your team is performing according to the norms. Good analytical reporting will include insight from the managers responsible. A report without insight does not tell much of a story.
Lastly, the combination of analytical reports can result in the creation of intelligence. Where statistical reports give you raw numbers and analytical reports help you ask good questions, intelligence tells a complete story. Intelligence will include multiple dimensions that include things like volumetrics but adds context. Intelligence will tell a story not so much about raw production but about the effectiveness of each team or the quality of the output.
For example, if there are two offices in a global company that each produces widgets and operate within their benchmarks (in other words both produce the required number of widgets per month), how does the operations managers rate their teams? Is it simply good enough that widget production goals are met? In many companies meeting productivity objectives is the goal. However, good operations managers will take a look at how effective each team is or will want to measure the quality of the output. By adding a time dimension to a productivity report, we can gain visibility into the effectiveness of each team. Most productivity reports will include a time dimension in the linear sense (i.e. XX widgets per month). An operations manager should be looking at how much time each employee takes to produce the volumes that they are reporting in the period. This is a measurement of “resource utilization”. In a pure factory we often have good measurements of individual productivity. In the services or information industry or in functional areas of a business, however, we do not.
Within a client services organization, for example, staff may be involved in many non-measurable activities such as meetings, training or report writing. Or they may be working on special projects. All of these things add value to the business but are typically not measured in any meaningful way. Worse yet, when productivity objectives are not met by an individual these items are often used as an excuse to hide a general lack of productivity. Resource utilization reports track the amount of time taken to produce workload and measure the amount of time spent on other activities. This provides insight into which employees and teams are more effective at their primary function.
Along the same lines, quality measures will tell you how well a team is operating. Where a marketing team may be measured by how many leads they generate, it is the quality of the leads that really will impact the business. In this way, adding a dimension such as conversion rates will be much more valuable than pure volumes. Most sales managers would rather have a fewer number of higher quality leads produced than tons of leads that go nowhere. Not only does that result in fewer sales, but low-quality leads generate a considerable amount of noise and inefficiencies in the sales organization, which results in higher costs for the business in order to yield the same results.
So, a successful operation includes a number of minimum traits. Its tasks are well documented. Staff time per task is measured and monitored. There are operating level agreements with upstream and downstream parties so that everyone’s roles are well understood. There are well established benchmarks and reporting goes beyond simple statistics and reports are published with consistency and at a reasonable frequency. Success criteria is well defined, understood and socialized.
But what if, like most of us, we are tasked with taking over a new operating environment where we determine that one or more of the above criteria are not met after we document the current state? In those instances, we must envision the end-state so that we can develop a way forward.
For more information about this article or our services, reach out to us at info@coeurderoy.com.