Google Gemini 4 Argon 的 ARC-AGI-3 成绩受测试条件影响，数学与工程仍落后

Google Deepmind is the second major AI lab to pass 95 percent on the ARC-AGI-3 benchmark. But the results depend heavily on the testing method, and Argon trails rivals in math and software engineering, according to Epoch AI.

Google Deepmind's new frontier model, Gemini 4 Argon, scored 97.5 percent on ARC-AGI-3 at a cost of $4,800 using the "Provider Adapter Harness," a test setup that uses vendor-specific features like retained reasoning and context compaction. According to ARC Prize, Google is now the second AI lab after OpenAI to cross the 95 percent mark on what is widely considered the toughest benchmark for general problem-solving. Only GPT-6 Astra scores higher at 99.9 percent.

Like Astra, Argon needed 62 percent fewer actions than the human baseline across the 25 public games. It matched or beat the speedrun record in three games and came within 10 percent of the best time in six others. On older versions of the benchmark, Argon scored 91.3 percent on ARC-AGI-2 ($0.32 per task) and 98.5 percent on ARC-AGI-1 ($0.16 per task), based on the introductory price of $2 per million input tokens and $10 per million output tokens.

Benchmark results depend heavily on the testing setup

In the "Standard Harness," which only allows simple notes between rounds, Argon drops to 49.8 percent at low reasoning for $5,800. This gap traces back to a July 2026 dispute when OpenAI's GPT-5.6 Sol beat Claude Opus 5 on ARC-AGI-3 using two API settings.

GPT-5.6 Sol only managed 7.8 percent in the standard environment, which discards internal reasoning after each action and used an older interface lacking newer Claude API features. That meant Claude was tested under different conditions than GPT-5.6. When tested with OpenAI's own Responses API, including "Retained Reasoning" and smart context "Compaction," the model jumped to 38.3 percent and beat Opus 5 at 30.2 percent. OpenAI argued that benchmarks always measure the technical environment too. ARC Prize co-founder Francois Chollet then allowed general, publicly available API settings.

API problems also prevented Argon from completing all 55 games in the Standard Harness, according to the testers. At low reasoning, the system finished 42 games (49.8 percent), at medium 46 games (46.3 percent). At the highest level, it only completed 19 games, so ARC Prize did not publish a score. The reported results still cover all 55 games by including partial points from aborted runs.

Argon closes the gap but lags in math and coding

Research institute Epoch AI tracked Google's progress too. Argon scored 165 points on the Epoch Capabilities Index (ECI), tying Fable 5.1 and landing just behind Opus 5.5 (167) and GPT-6 Astra (166). The ECI combines over 50 benchmarks into a single capability scale, similar to how Artificial Analysis builds its main score.

Argon jumps well ahead of Gemini 3.8 Flash (157 points) and brings Google closer to the top than at any point since Gemini 3.1 Pro in February. Artificial Analysis and Vals.ai confirmed this trend before.

Performance varies by domain, though. According to Epoch AI, Argon leads several knowledge, agent, and vision benchmarks, but early estimates put it in eighth on the Math ECI and sixth on the Software Engineering Index (SWE ECI).

AI News Without the Hype – Curated by Humans

Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.