The term “Pilot Purgatory” is applied to the majority of Maintenance 4.0 pilots that never reach the deployment stage. Although we lack data relating to maintenance-focused pilots, research from McKinsey & Company suggests that less than 30% of pilots in the “IoT” category start to scale.
There were times in Presenso where we have been engaged in multiple pilots and have experienced the purgatory phenomena. Here are the most common causes:
1. Pilot Not Tied to Execution
Let’s start with the rationale for conducting a pilot. The optimal scenario for a pilot is to test a solution as the first phase of a rollout. In reality, there are multiple reasons that pilots are conducted and these are not always tied to execution. For instance, the underlying motivation for conducting a pilot may be to evaluate various alternatives to solve a problem or to build supporting evidence of an evolving strategy. It is fair to state that many pilots are simply not designed to lead to full scale deployment.
2. Success Criteria Not Achieved
An obvious reason that pilots fail is because the results are not valuable enough. In the case of Industrial Analytics, a pilot could be considered a failure if the number of correct alerts (true positives), false alerts (false positives) and missed failures (false negatives) are below the pre-set selection criteria.
The image below is an example of results of an Industrial Analytics pilot.
A pilot can also be considered a failure based on an assessment of non-functional requirements including response time, availability, maintainability and usability.
3. Poor Testing Environment
There are often challenges for the internal pilot team to secure the perquisite resources for a successful pilot. I can refer to our own experiences with AI driven Industrial Analytics pilots. To conduct a pilot, we need access to Big Data that is generated from the sensors embedded in industrial machinery
The lack of sufficient amounts of data is not always apparent while the pilot project is negotiated.
In some cases, the individuals with overall responsibility for the pilot are unaware of internal constraints and limitations that are only exposed once pilots are underway.
4. Weak Business Case
A pilot is a relatively cheap way to evaluate a solution from a technical perspective. It is also conducted on a small scale in a controlled environment. Gaining approval for funds for large scale adoption is not automatic and there are often other projects and initiatives also competing for limited funding.
There are many ways a “successful” pilot can be derailed while rigorous and extensive due diligence is conducted. For instance, given the transformative nature of Maintenance 4.0 and proliferation of new technologies, it may be difficult to forecast the long-term return from a new investment.
When a pilot conducted without the intent of immediate/short term rollout, the failure to rollout more broadly is mischaracterized as “purgatory.”
5. Lack of Internal Buy-In
A pilot represents a change from existing processes. It is common to find many employees within the organization that are protective of the status quo and are threatened by new Maintenance 4.0 practices.
Post-pilot deployment can stall even if the pilots generate positive results that are consistent with pre-defined success criteria. In many cases, the pilot owner lacks the internal support that is necessary for execution.
For instance, an executive sponsor who is not fully committed to rollout may not secure budget and resources for post pilot deployment. Across the organization, there is a need for alignment between different business groups (e.g., Operations Technology, Information Technology, Operations and Maintenance).
6. Insufficient Deployment Resources
During the pilot there is a focus on the technical aspects of the solution itself. When challenges are encountered, there may be workarounds or creative solutions which are possible given the small nature of the pilot team.
When moving out of the pilot phase there are significant organizational and resource constraints that are likely to prevent adoption. Below are some examples of internal blockers that are sometimes overlooked:
• Access to data: Sensor generated big data that is required for Maintenance 4.0 solutions is often inaccessible by plant-level employees.
• Employee Readiness: New systems and processes require employee training or upskilling which is not always feasible.
• Hardware / system constraints: When new solutions are evaluated, the Total Cost of Ownership (or TCO) is not estimated. For instance, Maintenance 4.0 technologies may require local installation and the purchase of hardware.
In some instances, when the full resource commitment required to rollout a solution is recognized, it can slow down or even halt deployment.
Guidance for Overcoming Pilot Purgatory
Not all the causes of pilot purgatory can be easily overcome, especially those that relate to organizational challenges or other resource constraints. Our guidance is deceptive simple: prepare for rollout prior to commencing a pilot. This means that resources (realistic budgets, dedicated employees and processes that relate to data governance) need to be in place so that if the decision is made to move forward to rollout, it can happen with minimal disruption.