Authorized Intelligence cover art

Evaluating the safety and reliability of frontier models

November 5, 2026
Ep 3

This week on Authorized Intelligence, Cal Al-Dhubaib speaks with Peter Gostev, AI Capability Lead at Arena, about how frontier AI models are evaluated, where current benchmarks fall short, and what their growing capabilities mean for enterprise security. They discuss measuring real-world performance, the cost of completing tasks, and why organizations should be using AI more actively to strengthen their defenses.
What You’ll Learn

25 min
Evaluating the safety and reliability of frontier models with Peter Gostev on Authorized Intelligence

This week on Authorized Intelligence, Cal Al-Dhubaib speaks with Peter Gostev, AI Capability Lead at Arena, about how frontier AI models are evaluated, where current benchmarks fall short, and what their growing capabilities mean for enterprise security. They discuss measuring real-world performance, the cost of completing tasks, and why organizations should be using AI more actively to strengthen their defenses.

What You’ll Learn

  • Why model evaluations need to keep evolving: As the ways people use AI become more complex, benchmarks must move beyond static tests and reflect real-world tasks.
  • What makes a model useful: Performance is only part of the equation. Cost, efficiency, speed, and user satisfaction all factor into evaluating a model.
  • Why benchmark scores need context: Strong results in controlled environments don’t necessarily translate to flawless performance in the real world.
  • How organizations should approach AI security: Rather than focusing only on restricting internal AI tools, security teams should use AI to proactively test their systems and prepare for increasingly autonomous attacks.

Key Moments

  • 00:00 What does BullshitEval reveal about frontier models?

  • 04:00 How can real-world usage improve model evaluations?

  • 07:30 Why should organizations measure the cost of completing a task?

  • 10:45 What do cybersecurity benchmarks tell us about AI capabilities?

  • 13:00 How should enterprises rethink AI and security?

  • 18:00 What AI developments could reshape entire industries?

About Peter Gostev

Peter Gostev is the AI Capability Lead at Arena, where he works on evaluating the performance and capabilities of frontier AI models. He has explored new approaches to model evaluation, including BullshitEval, a benchmark designed to test whether models recognize flawed or nonsensical premises rather than confidently accepting them.

Listen & Subscribe

If you enjoyed this episode of Authorized Intelligence, subscribe wherever you listen to podcasts and leave us a rating or review. Have a question or a topic you’d like Cal to explore? Get in touch with us through authorizedintelligence.fm.

Guests
Hosts
Cal Al-Dhubaib is a globally recognized data scientist, entrepreneur, and IAPP Certified AI Governance Professional advancing the practice of Trust Engineering for AI. His work focuses on how organizations design, deploy, monitor, and govern AI systems at scale. He leads emerging tech thought leadership at Rubrik, where he focuses on AI, cyber resilience, and enterprise trust. He is an exited founder of Pandata, an AI design and development firm serving heavily regulated industries and an international speaker and thought leader.