Generative AI has taken the tech industry — and the world — by storm this past year, leaving IT leaders wondering what it will mean for their organizations in both the short and long term. With so many questions still unanswered about GenAI, Gartner used its IT Infrastructure, Operations & Cloud Strategies Conference that was held earlier this month to provide IT leaders with some clarity about the future of the technology.
One session focused on the hype cycle for generative AI, another on future implications of GenAI on infrastructure and operations (I&O).
Gartner Distinguished VP Analyst Arun Chandrasekaran led both of those sessions, beginning the first session, "Hype Cycle for Generative AI," by asking whether generative AI is a game-changer or overhyped.
"The pace of innovation in this ecosystem has been unprecedented in the past 12 months with new applications and new models being released almost every day," he said. "This has clearly overwhelmed IT leaders." That said, there is no clear-cut answer to the hype question, he said, as the research community is divided on the issue.
GenAI: Stochastic Parrot or Early Phase of AGI?
On the one hand, researchers believe that generative AI and large language models (LLMs) in particular are nothing but stochastic parrots — they know how to generate next words and sentences but have no understanding of what they're generating. This can lead to hallucinations and egregious mistakes, Chandrasekaran said.
However, there are also researchers who believe we are in the early phase of artificial general intelligence (AGI) — having human level intelligence and so are able to do tasks that humans do very well.
The potential for GenAI is going to be transformational, Chandrasekaran said, but in the short run there are a lot of speed bumps to overcome:
- The risks of generative AI models are significant and consequential.
- The true value of GenAI for the enterprise is created when organizations combine these models with their data and their domain knowledge. "Enterprise data engineering, unfortunately, is still stuck in the 20th century, and there's a lot of work that we need to do to get our data to function effectively with these models," he said.
- The return on investment and the long-term cost of ownership of GenAI applications and models are still unproven.
When will generative AI mature? "Despite our best efforts, I still think that this ecosystem is going to evolve in completely unexpected ways," Chandrasekaran said. "Which is why I have the roller coaster ride picture [see chart below], which is, prepare for the roller coaster because this is a space that's going to evolve very, very rapidly."
Within IT, there are three key business functions for which GenAI is being adopted, according to Chandrasekaran. "The first and foremost is in software engineering, where GenAI models are helping developers write code faster and even complete code within their environments," he said.
The other two domains are security, primarily for threat management, and DevOps and I&O.
GenAI Benefits and Risks
In his second session on GenAI, "The Future Implications of ChatGPT and Generative AI on I&O," Chandrasekaran discussed the benefits and risks of using the technology.
The benefits of generative AI include:
Employee productivity
Not only can employees use generative AI to do more tasks with less, but they can do them faster.
Multidomain applications
The models can generate multimodal content, meaning the same model can be deployed for multiple use cases, which simplifies the model lifecycle management.
Democratization of skills and knowledge
This, according to Chandrasekaran, is "the real promise of generative AI in the long run" as it puts enterprise knowledge at the fingertips of employees and also makes it conversational — you can interact with your data through natural language conversations.
Talent retention
By making work more satisfying and improving productivity, GenAI can help with talent retention. "These tools are also becoming a low-code, no-code solution, enabling traditional infrastructure teams, for example, to generate declarative code for managing cloud infrastructure," he said. "Or even business users, who can interact with complex systems on the back end using natural language queries."
Despite these benefits, there are a number of significant risks with GenAI.
"The No. 1 risk that customers are worried about is, 'I don't want my data to be used to train your models. I want my data to be my data. It needs to be private.'"
Other risks that worry GenAI users include hallucinations ("a complex term for saying when the model doesn't know the answer, it makes up an answer"), the lack of reference ability in GenAI ("you don't know exactly why [the models] are saying what they are saying"), and many model creators are dealing with class-action lawsuits and other legal issues.
While there are risks, there are actions to take to mitigate those risks, Chandrasekaran said:
- Loss of intellectual property: Use enterprise model APIs; prompt governance is key; review third-party service agreements.
- Hallucinations: Ground the models via RAG or fine-tuning, tools, and humans to vet output.
- Security: Implement strong data protection and guardrails for data protection and adversarial attacks.
- Explainability: Deploy solutions for explainability; prioritize data referability.
- Fairness: Use content moderation and bias detection tools; educate business and IT leaders.
- Misuse: Enforce strict access control and approval process; deploy observability tools.
- Workforce impact: Embark skills training, map career paths, and provide emotional assistance. "AI will not replace human beings. But AI will replace a human being that refuses to use AI," he said.
Chandrasekaran believes most I&O leaders will buy AI rather than build AI. "But having said that, we need to make sure that we set the bar very high in terms of what it means to do business with us," he said. "We need to understand their model training process. We need to understand the privacy and security practices that are embedded in their application. We need to understand very well their cost of ownership in the long run. We need to understand how much prompt engineering automation they can do in our environment."
Where does Gartner stand with generative AI? "In a nutshell, we believe in the long-term potential of [generative AI], but we've got to overcome obstacles in the short term," he said.
About the author
Rick Dagley is senior editor at ITPro Today, covering IT operations and management, cloud computing, edge computing, software development and IT careers. Previously, he was a longtime editor at PCWeek/eWEEK, with stints at Computer Design and Telecommunications magazines before that.
