Google the phrase “Pilot Purgatory” and you will see a raft of press articles about the state of Digital Transformation and the Industrial Internet of Things (IIoT). McKinsey and Company, Deloitte, and a series of other firms have published several articles on “pilot purgatory” in IIoT, Digital Manufacturing, and Digital Transformation. IIoT and Artificial Intelligence/Machine Learning (AI/ML) vendors often exasperatingly discuss the successful implementations they have done at the company’s home office that have not translated into wide-scale rollouts. These vendors curse “pilot purgatory.”
Given all the brouhaha, one would expect that companies deploying these pilots would feel “stuck in” pilot and that getting to a wide-scale rollout was a key challenge in front of their Industrial Transformation (IX) Program. But when we surveyed industrials, we found just the opposite. Few companies felt “stuck in” pilot mode and only a fraction of companies identified “moving out of pilot phase” to be a key challenge. So clearly, pilot purgatory is fake news! Let’s look at the facts and see how we can reconcile these different perspectives.
Industry Week has defined Pilot Purgatory:
“The Roman Catholic church defines purgatory as a place where suffering souls work out their sins before going to heaven. During this Fourth Industrial Revolution, purgatory is where pilot projects in the Internet of Things – sensors and actuators connected by networks to computing systems – languish before being taken to scale. Sometimes, it becomes their eternal resting place.”
McKinsey and Company has consistently written about Pilot Purgatory. McKinsey found in 2019 that 56% of manufacturers are stuck in pilot and reported in 2020 this number increased to 74% of manufacturers. Cisco and Deloitte note that 74% of IoT programs fail to scale.
LNS Research finds quite the opposite. We executed global surveys on Industrial Transformation in 2019, 2020, and 2021. Manufacturers and other industrial enterprises simply do not see themselves in pilot purgatory. In 2019, we found that only 13% of companies listed their status as “stuck in pilot with unclear results”. That number fell to 7% in our 2021 survey.
Further, since 2014 we have been asking in all our surveys the following question: What are the top challenges your company faces in your Industrial Transformation and/or IIoT Program? All the market noise around “pilot purgatory” led LNS Research to include a new potential answer in 2019: “Moving out of the pilot/proof of concept phase.” We had expected this to be a common choice, especially on a question that allowed respondents to select three choices. Yet, only 7% of companies listed this as even one of their top three challenges facing their program.
In fact, 57% of respondents reported their IX Program is meeting or exceeding expectations for realized benefits. These results clearly reinforce the fact that companies do not feel “stuck in the pilot phase.” Our expectations and market chatter were dead wrong about “pilot purgatory” from an IX and IIoT program perspective.
So, why the differences in thinking about “pilot purgatory?" We see four differences that matter.
The Implementation Strategy:
When asked all the steps they had in their Implementation Strategy, companies created a clear picture: Industrial Transformation is about pilots and proofs-of-concept. Plant-wide, corporate-wide, or value-chain rollouts were increasingly out of scope for the majority of IX Programs. IX Programs were only about identifying and testing solutions, not about deploying them across the organization. This alone accounts for some of the expectation gap: industrials don’t feel “stuck in” pilot because that is all they planned in the first place.
The Funding Model:
But it goes deeper than a difference in charter. When asked if plants had to pay for the IX initiative/solution for their operations we came to an insight. In 58% of cases, plants, mines, ports, or other industrial operations were expected to fund themselves any IX initiatives/solutions that could impact their operations. This funding mechanism may account for why so few companies reported they were “stuck in” the pilot phase – this is exactly how the program was structured.
In fact, LNS Research believes this funding model is core to the whole pilot purgatory phenomenon. Let’s compare the adoption and roll-out of two technologies for comparative insight: ERP and manufacturing execution systems (MES)/manufacturing operations management (MOM). Because of compelling external drivers including the necessary adoption of the Euro, Y2K (remember that?), and Sarbanes Oxley, corporations around the world generally elected to fund the adoption and roll-out of ERP centrally. Massive budgets were allocated, and significant pain was endured in those rollouts. But ERP has effectively been adopted across industrials globally. 74% of respondents to the 2021 LNS Research Industrial Readiness Survey reported having common ERP systems already in place or to be actively rolling them out.
MES/MOM never had such compelling external drivers at the corporate level. Therefore, each plant has had to fund the deployment of an MES/MOM system. Plant managers had to choose whether MES/MOM investments were more important than other potential IT and/or plant equipment investments. This significantly slowed the adoption of the technology with MOM now installed in less than 20% of plants (it also led to a diversity of systems even within one corporation.) Multiple studies of MES/MOM have shown that the only effective means to accelerate adoption and standardization around MES/MOM is through some sort of centralized funding mechanism. For example, MES/MOM penetration was accelerated when it was made part of those corporate-wide ERP rollouts discussed above. Most other plant-level systems have this plant-by-plant adoption model. Similarly, “Advanced Manufacturing” and other discontinuous improvement programs over the years have struggled with impact for the same reason: decentralized funding models lead to decentralized adoption decisions.
In fact, this may be one of the core differences between IT and OT. IT is centrally mandated and centrally funded (CRM, Microsoft Office, Cyber Security Software are just a few examples across many corporations). OT is regularly a plant-funded project.
What fuels the perception gap is that IX and IIoT are the “new ERP” in terms of scale and potential impact on the organization but have the funding model of MOM.
The history of MOM and other plant-level technologies shows that there can be any number of reasons a technology that is successful at corporate is not quickly adopted across operations. There can be wide diversity across operations even in any one corporation, particularly in the larger corporations, making solutions less universally applicable. Individual plant leaders can have multiple conflicting and compelling demands for capital budget (such as replacing aging machinery).
LNS Research also asked about the source of the IX budget. We got back a wide distribution of answers as one would expect but we did see a trend: 86% of the funding mechanisms for the centralized IX program activity were a “reallocation” from other capital and operational budgets. In an environment of ever-shrinking discretionary spending, one would expect plant managers to be careful in adopting new initiatives that would likely require incremental resources locally.
Two other insights from the survey illustrate the importance of the budget in the adoption of centrally proven technology pilots by the plants. First, IX Leaders were 2.7 times more likely to have joint funding where the IX program paid for some initiatives/solutions centrally (though that joint funding model was noted by only a small percentage of companies). Second, IX Leaders were significantly more likely to have an enterprise view of operations and four times more likely to be managing operations as a “tightly orchestrated team.” In tightly coordinated corporations’ plants are more likely to “read the tea leaves” and adopt corporate-sponsored technologies faster, even on the plant’s budget, because that is an expected corporate norm. Both steps encourage plants to participate in and roll out IX initiatives.
The Power of More™ Means Less things roll out
The single most powerful finding of all the LNS Research work on IX Programs is that Leaders are doing more in every direction. IX Leaders are:
executing a more expansive functional scope and strategy
reaching out to both customers and suppliers
getting data from more sources and making it available to more roles across the corporation
jointly funding more initiatives/solutions
Followers are always planning to close some of the gaps sometime in the future.
The Power of More™ may help account for why vendors’ perception of pilot purgatory is so disconnected from the manufacturers’ perception. IX programs, especially the better programs, are engaging in a wide array of technology. The average program has 12 distinct initiatives and is piloting 4-5 different sets of technology. While LNS Research has found “more” technology choices to be particularly advantageous to the overall health of the IX Program (because it encourages plant participation) it does make it more likely that any one technology will have only a limited rollout. A company may find three pilots deliver results but only rollout the one with the most return (at least initially). That would leave two other vendors and their associated implementation partners complaining about “pilot purgatory” while the manufacturer believes their program is progressing nicely.
Playing at IX May Be Good Enough for Many Organizations
In an earlier Research Spotlight, LNS Research identified four organizational disconnects around IX Programs that may also have a significant role in the success, failure, or stagnation of IX Programs. In looking at these organizational disconnects, one has to wonder about the real commitment many industrials have to IX. Unempowered CDOs, budgets unaligned to goals, and lack of plant engagement are all signs that the “C Suite” may be “talking the talk but not walking the walk” around IX. The four “disconnects” evidence a lack of real commitment to IX at many corporations.
This, in turn, would help account for the differences between the self-assessed perception of IX Programs and the external frustration around “pilot purgatory.” Many corporations may be happy doing something even if that something is not very much.
Transformation is another form of change that executives are “imposing” on organizations. While “pilot purgatory” may not be the key challenge, companies are struggling to achieve the full range of potential benefits being realized by IX Leaders. Transformation still requires change management across people, process, and technology to scale across an organization, which takes time.
LNS Research is not surprised to hear some of the largest enterprise software vendors and consulting organizations bemoan the lack of rollouts of IIoT and IX initiatives. These vendors and organizations built their corporations around solutions with compelling external drivers that led to a remarkably unusual occurrence: the centralized funding of a large-scale roll-out of specialized application technology. This has not been common in the OT world and is not common in today’s IIoT and IX Programs. Therefore, LNS Research believes that “pilot purgatory” is not the best way to characterize the state of Industrial Transformation. But rather, we believe an old expression sums it up better: “you get what you pay for.”
All entries in this Industrial Transformation blog represent the opinions of the authors based on their industry experience and their view of the information collected using the methods described in our Research Integrity. All product and company names are trademarks™ or registered® trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.
Vendor selection should be the last step of a Digital Transformation initiative, not the starting point. We've identified common mistakes companies...
The #MondayMusings Industrial Transformation blog series provides insight and analysis for executives from the previous week’s briefings, events, and...
Learn about the options available to quality executives aiming to make an EQMS investment in the life sciences industry.