Google DeepMind's Cryptographic AI Evaluation Pilot Targets Trust Gaps

Google DeepMind is launching a pilot to create trustworthy AI benchmarks using cryptographic double-blind testing, aiming to prevent model exposure to test questions in sensitive applications.

Google DeepMind has initiated a pilot project using cryptographic double-blind evaluation to test AI models against confidential benchmarks. The method prevents AI models from seeing test questions during training or evaluation, addressing benchmark contamination where models gain prior exposure to questions. The pilot tests Gemini Flash Lite against such benchmarks and partners with the Singapore AI Safety Institute. It targets sensitive applications like cybersecurity and government use cases where trust is critical. The project aims to eliminate prior compromises between evaluators and model providers—a problem highlighted by Anthropic’s Fable 5 benchmark delays requiring 30-day data retention. However, the pilot remains limited to specific partners and model lines, technical details are not yet public, and the project is scheduled for future deployment (August 2026). This approach could accelerate reliable AI solutions for global challenges by reducing trust barriers in high-stakes evaluations. The next step is publishing the technical report to clarify implementation scope.

*Note: Source describes this as a pilot project launching in development, not completed. Technical details remain confidential pending the August 2026 publication date.*

Source: The Decoder