Generative AI in the Enterprise: Now the Era of Potemkin Pilots?
A human wrote this. 😊
When it comes to generative AI use in the enterprise (and especially retail-CPG), we may have entered – and may wallow for months in -- what could be termed the Potemkin Pilot Project phase of development.
In Gartner-speak, it is a natural outgrowth of the Peak of Inflated Expectations and will precede by a few quarters a (perhaps inevitable) wave of general media click-bait disillusionment.
There are exceptions, of course. Brands with heft and vision. But outside the leading few, anecdotal data points now point to a plethora of Potemkin Pilots. I suspect there is a linking trend line. If so, that trend line will lead to two outcomes – one bad, one good.
The bad is that money and time will be wasted, little tangible business value will be found, and short-sighted business executives (of brands that can least afford to do so) will lose patience. Eyes will be rolled, consultants ushered to the door, and a budget item or two may be zeroed out for 2025.
The good is that far-sighted executives will buckle in, face the realities of what it will take to drive the enormous potential value of artificial intelligence (not just Gen AI) into their companies, and start – step-by-step – on the challenging work that will lead to a sustainable competitive advantage.
The latest scholarship suggests that it’s more myth than fact, but fellow fans of Russian history know the tale named for Grigory Potemkin, the late 18th century general, minister and lover of Empress Catherine II, the Great. The story goes that, in 1787, Catherine journeyed out to see her empire. Floating in royal style down the Dnieper River, she encountered one lovely village after another – villages that in truth were little more than stage sets built and populated (see the happy serfs!) by Potemkin and a costumed crew.
Once Catherine’s barge would pass by, Potemkin and team would dissemble the village, and rebuild it downstream. Look! Another lovely village! More happy serfs!
This gave rise to the phrase “Potemkin village,” defined as a construction that is no more than a façade, designed to lure observers into thinking that things are better than they are.
In the same way, a Potemkin Pilot Project (PPP) has little more than transitory value. It is generally limited to a single business unit (most likely IT, but also, in marketing, merchandising, store ops, HR, and at home and on weekends, in the hands of nearly every employee).
It demonstrates technology and suggests workflow possibilities – but without the painful work of workflow integration (and the requisite data integration required for workflow integration.) It does not cost much and does not venture into the difficult (and costly) questions of process transformation and change management.
But it does allow a brand to proclaim to investors and competitors that it is “doing AI.”
Evidence? Three leading indicators. And one damning number: 11 percent.
The first indicator was a conversation late last month with a wise industry insider, one who knows technology, enterprise systems, and – most important – the zeitgeist of investment inside the fast-moving consumer goods (FMCG) industry.
He pulled no punches.
“They know that they’re supposed to do AI – and so they’re all doing pilots,” he said. “But very few know where to start, how to start, or what to do to really improve the business.”
From a perspective high above the western world’s largest grocers and packaged goods manufacturers, he was seeing little that might result in a meaningful change in process or improvements in business process outcomes.
What was needed, in his estimation, was a need to “move beyond the magic,” and to spend far less time on industry innovation announcements, valuations, model sizes, parameters and tokens. but What is desperately needed, in his estimation: significantly more detailed and honest conversation about the what, the how, the structural and data dependencies of generative AI use, and the use cases that will create sustainable value today.
The second: two and very recent publications from industry sage Gary Hawkins, founder and CEO of CART (Center for Advancing Retail Technology.) Both deeply thought-provoking.
The first is Gary’s latest book, Bionic Retail. (Get it. Read it.) The second is Gary’s May 14 LinkedIn post, “The (Data) Sword of Damocles.” (Find it. Read it.)
This is insight from one who is bone-deep in his industry knowledge.
In both his book and his recent post, Gary emphasizes (and bemoans) the state of data quality and readiness across the retail industry. Especially for the emerging world of artificial intelligence.
Writes Gary in his recent post:
Data has become retail’s Damocles’ sword, and the thread holding the sword is becoming ever thinner as we accelerate up and out the exponential growth curve of tech-driven change.
A day of data reckoning is fast approaching, driven by the explosive growth of AI applications . . . as the efficacy of AI capabilities is fully dependent on the veracity of the data feeding the algorithms.
In short, he makes clear that the gains promised to retailers by AI are fundamentally dependent upon the quality of a firm’s data. Is the data clean, usable, accessible? Is it diverse, i.e., from internal and external sources, in forms both structured and unstructured? Is it received and processed in real-time, for real-time personalization and intervention? Can ever-increasing volumes of relevant data be managed and analyzed?
And: is there the foundation of a unified data architecture? Or the investment (and commitment to maintain the investment) in going through the pains of designing and implementing one?
The third is a recent, excellent and overdue paper from McKinsey: “Moving past gen AI’s honeymoon phase: Seven hard truths for CIOs to get from pilot to scale.” It is (one man’s opinion) a tacit admission by The Firm that a lot of enterprise leaders are at a loss as to what to do and how to do it.
"It is relatively easy to build gee-whiz AI pilots but turning them into at-scale capabilities is another story. The difficulty in making that leap goes a long way to explaining why just 11 percent of companies have adopted gen AI at scale . . . "
There is much goodness here. Here’s a taste: seven commonsense statements of advice -- relevant to all technology innovation, but especially relevant now:
· Eliminate the noise; focus on the signal. If every department in the company is playing with AI, resources (and attentions) are spread thinly. (There is also a potential governance nightmare, but that is a topic for another day. Ask, and focus: where, how, and how soon can AI (and generative AI) drive measurable business improvement?
· It is not about the pieces – it is about how the pieces fit together. Why this is a surprise is a surprise to me, but McKinsey notes that belabored discussions of LLM and API choices are much less important than the orchestration of interactions and integrations to drive faster-better-smarter-cheaper decisions at scale.
Companies have to manage a variety of sources (such as applications or databases in the cloud, on-premises, with a vendor, or a combination), the degree of fidelity (including latency and resilience) and existing protocols (for example, access rights.) As a new component is added to deliver a solution, it creates a ripple effect on all the other components in the system, adding exponential complexity to the overall solution.
· Get a handle on costs. Gen AI models account for roughly 15 percent of the overall cost of Gen AI implementations. The biggest expense? Change management (which generally costs 3X that of models.) And know that run costs are generally higher than build costs, with model usage and labor being the largest pieces of the former. The best implementations, in McKinsey’s experience, are those with a strong performance-management culture.
· Tame the proliferation of tools and tech. At too many firms, it is a hairball (and governance nightmare): too many platforms and technologies from too many teams pushing their own favorites and use cases. The result: Potemkin pilots everywhere, and a paucity of scaled, business-driving rollouts.
· Create teams that build value, and not just models. In short, this is a business priority and not a tech program. Development teams must embed the technology into business processes; close collaboration between technology, business, and risk teams is essential.
· Go for the right data. High-performing gen AI solutions are not possible without clean and accurate data. (see Hawkins, Gary). Invest in targeted labeling (for retrieval-augmented generation) and authority weighting; invest, most of all, in data architecture that enables access.
· Reuse it or lose it. McKinsey claims that reusable code can increased the development speed of generative AI use cases by 30-50 percent. Leaders will explore how code “modules” can be reused across three-to-five core use cases.
Trend line? Perhaps.
Call to action? Absolutely.
I am Jon Stine, 35+ years in retail business and technology. Most recently in conversational AI.
I read, I listen, I observe. I think, I write, I advise.
+1 503 449 4628.