How to Monitor AI with AI

Artificial intelligence can’t be trusted to get everything right, so you need humans in the loop. You also need AI monitoring AI for speed and scale.

2 Min Read
close-up of robot's hand holding magnifying glass

AI gaffes are sobering, whether it’s hallucinations or making dubious decisions. This is one reason why humans must be kept in the loop. However, artificial intelligence can operate at a speed and scale that is physically impossible for humans while surfacing edge case exceptions that warrant human review and oversight. This type of partnership helps ensure that AI is doing its job correctly. 

“A human can’t read and evaluate things 24/7 and take actions in milliseconds. That’s why the Turing test doesn’t apply anymore because now we’re talking about the same capability as a human but at a 100,000X improvement in scale, speed and accuracy because it’s retrieving much more information,” says Mohamed Elgendy, CEO and co-founder of AI/ML testing platform Kolena. “Large language models are being used to evaluate models before they’re deployed and as guardrails after it’s deployed.” 

For example, a business might want a simple guardrail for a chatbot that prevents the chatbot from mentioning a competitor or more complex guardrails around violence, hallucinations, and jailbreaking. In the fintech space, models are prevented from giving financial advice in fintech applications because doing so is illegal. 

“The idea of using AI to monitor and regulate other AI systems is a crucial development in ensuring these systems are both effective and ethical,” says Cache Merrill, founder of software development company Zibtek, in an email interview. “Currently, techniques like machine learning models that predict other models' behaviors (meta-models) are employed to monitor AI. These systems analyze patterns and outputs of operational AI to detect anomalies, biases or potential failures before they become critical.” 

Related:Why Prompt Injection Is a Threat to Large Language Models

The benefits of AI monitoring AI include a level of scalability humans can’t achieve, a higher level of consistency since AI does not require rest and depth of analysis based on deeper patterns and correlations that might be overlooked by human analysts. 

Continue reading this article on InformationWeek.

Read more about:


About the Author(s)

Lisa Morgan


Lisa Morgan is a freelance writer who covers big data and BI for InformationWeek. She has contributed articles, reports, and other types of content to various publications and sites ranging from SD Times to the Economist Intelligent Unit. Frequent areas of coverage include big data, mobility, enterprise software, the cloud, software development, and emerging cultural issues affecting the C-suite.


InformationWeek, a sister site to ITPro Today, is a trusted source for CIOs and IT leaders seeking comprehensive and authentic coverage of the constantly evolving world of technology and its impact on business. Our experienced and ethical journalists conduct in-depth examinations of crucial issues and the impact of global events on IT operations and strategies, helping forward-thinking executives stay at the forefront of their industries. InformationWeek also provides a platform for enterprise IT leaders and leading tech companies to share their insights and experiences through exclusive interviews, opinion pieces, and events, offering firsthand accounts of strategies, trends, and innovations.

Sign up for the ITPro Today newsletter
Stay on top of the IT universe with commentary, news analysis, how-to's, and tips delivered to your inbox daily.

You May Also Like