Frontier: forecasts, benchmarks, open debates

Benchmark scores, cities served by robotaxis, AGI forecasts, energy and water consumption: this page collects what moves too fast for print. The criteria for weighing each item stay in the book: the state of the art (chapter 1), the scaling laws (section 3.1), the environmental figures (section 12.5), the AGI debate (section 13.4).

Models and humans on the same tests

The comparison only makes sense where someone has measured humans too. The first table collects the tests that declare a human benchmark, with the current gap; each row also says when the run dates from, which is the information that expires first.

Human benchmarks as declared by whoever runs each test, where a declaration exists. For ARC-AGI-2 it is a panel in which every puzzle was solved by at least two people. For GPQA Diamond the benchmark is more fragile than it looks: the test's authors declare 65% for experts in their own domain, 74% discounting the mistakes they themselves recognised afterwards, and those are measures on the whole set. The 69.7% in the table is OpenAI's measure on the Diamond problems alone, the one Epoch AI reports. The 81.2% that circulates, and that this page carried until today, I have not found again in any primary source: I have removed it. OSWorld has changed version, and the 72.4% human figure was measured on the first one: the scores in the table are on OSWorld 2.1, which is why the distance column stays empty. The 81.8% is declared by Anthropic; on the official OSWorld leaderboard, updated to 17 September 2026, the best result belongs to Claude Opus 5, 77.7% on partial pass and 44.3% on strict pass, and the September models are not on it yet. WebArena and GAIA data from the AI Index 2026, chapter 2, exported in early 2026; ARC-AGI-2 from the ARC Prize leaderboard. Consulted on 16 August and again on 18 September 2026; ARC-AGI-2 and OSWorld checked again on 3 October 2026.

The overall picture in the AI Index 2026 distinguishes three groups: human benchmark surpassed on ImageNet, SuperGLUE and MMLU; reached or nearly so on GPQA Diamond, MMMU and AIME; models still below on agent tasks such as SWE-bench Verified and OSWorld. These are snapshots from early 2026, and the GPQA run in the table, from September, is by now twenty-six points past the benchmark. One reservation on SWE-bench: the report rescales it on a human benchmark whose value it does not publish, and the test's authors have never declared one, which is why on this page you find it in the second table. The ARC-AGI foundation, on the other hand, also publishes the cost alongside the score: the human panel costs 17 dollars per puzzle, GPT-6 Astra 1.12 for its 95.0%, and GPT-6.1 Sol, released on 29 September, 0.25 for 94.2%: one point lower, at less than a quarter of the cost. And the next test is no longer stuck: on ARC-AGI-3, which asks models to act in small interactive environments, GPT-6 Astra scores 62.7% with ARC Prize's standard test harness and 99.95% with the one that goes through the provider's adapter, while OpenAI has communicated 98.6% using its own interface with two settings changed. The number that compares with the other models is the first: Claude Opus 5, measured the same way, is at 30.2%, and GPT-6.1 Sol, released on 29 September, at 52.7%. When three figures this far apart circulate for the same model on the same test, the difference is not in the model, it is in the test harness.

The second table collects the most cited tests among the ones that an overall human benchmark does not cover: no one person knows all the disciplines of Humanity's Last Exam, and FrontierMath's problems take an expert mathematician hours or days each. If you read that a model "has surpassed humans" on one of these, the term of comparison was invented by whoever is writing.

From laboratory to service: the figures

Chapter 1 invites you to distinguish the demonstration from the service with paying customers. Here is where the line lies today.

Medicine. The American FDA lists 1,614 authorised AI-enabled medical devices, 1,230 of them in radiology (76.2%). The list was updated on 4 September 2026 and contains decisions up to 29 June 2026; the FDA does not publish a total, and I recounted both figures row by row on that version on 3 October 2026. The FDA itself warns that it is built by searching for AI-related terms in the authorisation documents, and that the real total is therefore higher. AlphaFold, the most cited case of AI in science, has reached 241 million predicted protein structures with version 6 of its database, October 2025, and since 16 March 2026 it also contains protein complexes, about 30 million of them, with heterodimers added on 19 May and, since 24 September, the predicted complexes for more than 2,800 viruses. In all, the database now holds more than 260 million predictions. The 2024 Nobel Prize in Chemistry linked to that work went to three people: Demis Hassabis and John Jumper for structure prediction, David Baker for the design of new proteins.

Self-driving, the service. Waymo carries passengers in 15 American metropolitan areas, if I count them one by one: the 14 cities the company listed on 1 September, when it opened Denver, San Diego and Tampa, plus Las Vegas, opened to the public on the 14th. The total is my count: Waymo does not declare it. On 25 August the company announced its entry into Germany, in Munich. Other cities are on the expansion list, among them London, New York and Washington DC, which has been on it since March 2025. Two now have a date: Tokyo, with rides open to the public in 2027 together with Nihon Kotsu and GO (announced on 14 September), and Singapore, the first market in South-East Asia, with commercial driverless rides planned for 2028 if approval comes (17 September). On rides, the latest dated figure declared by the company remains over 400,000 a week as of 2 February 2026, the day of the 16-billion-dollar round on a valuation of 126; the site's pages still say "over 500,000", with no date. On 24 September Waymo did put a date on another measure: 270 million miles driven in full autonomy.

Self-driving, the chaser. Tesla declares to the SEC (22 July 2026) that the robotaxi service is active in seven areas; the page for customers lists six of them between Texas and Florida, because in the Bay Area you travel with a safety driver on board, under a chauffeured-carrier permit. In California's registers, as of 12 August and again as of 18 September 2026, Tesla holds the permit to test with a driver and appears neither among those that may test without a driver nor among those that may operate commercially. The company publishes neither weekly rides nor fleet size: anyone comparing the two companies with a single number is comparing a published figure with an estimated one.

AGI: who predicts what

A forecast about AGI is weighed with three pieces of information: who makes it, when they made it, in what venue. The table reports all three.

Altman's dates tell of a shift: in January 2025 he wrote "we now know how to build AGI", in December 2025 superintelligence was ten years away, and in the plan signed in June 2026 the only concrete date concerns an internal company milestone, March 2028, when "a substantial fraction of research" should be carried out by AI systems. Amodei's dates teach a different check: his "1-2 years" appears identical in October 2024 and in June 2026. When you read a forecast, look at whether it contains a year or a distance, because the distance moves forward together with whoever pronounces it.

In September the register changed. On the 12th Amodei published an essay calling for development to be slowed down at once, warning that swarms of agents out of control could take over the entire internet "in 6-12 months"; Altman replied the same day, saying he agreed that the frontier needs pacing and promising access to outside evaluators. On the 13th Amodei pointed to China as the hardest dilemma in his own proposal. On the 14th Hassabis and Elon Musk signed the appeal, and the same day LeCun rejected it as a fake alarm, recalling that in 2019 Amodei argued GPT-2 was too dangerous to release. The request contains neither a date nor a threshold: it counts as a position, not as a verifiable commitment.

In the same month François Chollet, who runs ARC, brought his own AGI forecast forward: he had it at 2030, and after Astra's result on ARC-AGI-3 he says sooner, because progress is going faster than he expected. It is worth noting which figure he moved on, given what you read above about test harnesses.

LeCun has added a company to the words: he left Meta at the end of 2025 and on 10 March 2026 launched AMI Labs, with one billion and 30 million dollars of initial funding already raised, for an alternative line of research: "world models" that learn by observing the physical world instead of predicting the next word. For now they are a research thesis and a funding round.

Those who do research and do not sell models see it another way. In the survey by the AAAI presidential panel (report of March 2025, 475 respondents in all, academic community for two thirds), 76% of those who answered the question judge it unlikely that enlarging the current approaches is enough to get to AGI. And the latest large survey on timing, 2,778 researchers polled in the autumn of 2023, puts a 50% chance that machines will surpass humans at every task only in 2047. Between those who build and those who observe, fifteen years are in play, and neither side has data that settles the question.

The disagreement on scaling, with each side's facts

Ilya Sutskever, the most cited researcher among the sceptics, put his thesis on record in an interview of 25 November 2025: from 2020 to 2025 it was "the age of scaling", now "we go back to the age of research, only with big computers", because multiplying the scale a hundredfold would no longer transform capabilities. Eight months later his company signed an agreement with Nvidia to increase its own compute tenfold. The two moves fit together only in his exact formulation, and that is why it is worth quoting it in full.

Amodei, on the opposite side, declared in February 2026 that pre-training "continues to deliver gains" and that reinforcement shows the same curve. Epoch AI's public figures, checked again on 3 October 2026, meanwhile give a training compute that, in the trend on frontier models, quintuples every year - even though a single model can cost less than its predecessor - and an aggregate capability index that, since reasoning models arrived, has gained about 14 points a year in a straight line, against 6 for models that do not reason. As for the fuel, the reference estimate remains Epoch's of June 2024: the stock of public text written by humans will be reached by training datasets between 2026 and 2032. Since then no laboratory publishes the size of its own data any more, so the forecast cannot be verified yet.

In the meantime the industry has got around the problem another way: making models think for longer at the moment of the answer instead of enlarging them at the start. Epoch estimates that this jump is worth, on some tests, as much as ten times more training compute, and in the 2026 leaderboards almost every frontier model appears in distinct variants by level of reasoning.

The energy and water bill

The electricity figures are already on /en/r/hardware, updated to the IEA estimates of 16 April 2026: 485 TWh consumed in 2025 by all the world's data centres, +17% in a year, with AI data centres alone at +50% and a projection of around 950 TWh for 2030. Water and the companies remain here. For water the IEA estimate is the one in the 2025 report: 560 billion litres consumed in 2023 by data centres as a whole, with a projection of more than double in 2030 and the perimeter the book reports - two thirds in the supply chain that produces the energy, a quarter in cooling. The parallel estimate for AI systems alone, 312-765 billion litres in 2025, is on /en/r/hardware: different method and perimeter, and the two series do not add up.

The 2026 corporate reports bring a novelty of method. Google declares for 2025 total emissions growing by 81% on the 2019 base year in the perimeter it uses for its own targets, with operational emissions alone down 2% on the year: the two percentages coexist in the same report because they count different sets of emissions. Electricity consumption has risen to 43.6 TWh, +37% in a year. Microsoft declares +25% of total emissions on the previous fiscal year and has removed from the 2026 edition the table with the comparisons against the 2020 base year; the components that were falling have gone back to growing after it gave up non-additional renewable certificates. Both confirm their 2030 targets, and both have recalculated their base years.

What has changed

3 October 2026 · revision

Two new models in two weeks, and cost falls faster than the score rises. Claude Opus 5.5 came out on 22 September and GPT-6.1 Sol on the 29th. On ARC-AGI-2 GPT-6 Astra stays first, but GPT-6.1 Sol gets within a point of it at 0.25 dollars per puzzle, less than a quarter of what Astra spends. On FrontierMath, levels 1-3, the two OpenAI models are tied at 93.7%.

On Humanity's Last Exam the order depends on who measures. In Anthropic's table of 22 September, with tools, Claude Opus 5.5 scores 67.7% and GPT-6 Astra 57.2%; on Scale's independent leaderboard, without tools, Astra is first with 54.8%. I removed the sentence that had Astra behind on this test, because it held for only one of the two measures. OSWorld has moved to version 2.1, and there the only figure with both thresholds comes from the official leaderboard, on a July model.

Services, and two corrections. Waymo serves 15 areas, counted by me one by one: the previous revision had missed Tampa, opened on 1 September together with Denver and San Diego. Tokyo has a date, 2027, and Singapore 2028 if approval comes. I redid the FDA count on the September list: 1,614 devices. And I corrected two things the page had wrong: Amodei's essay speaks of "6-12 months", and Epoch's capability index grows by about 14 points a year in a straight line, without the acceleration I had written.

18 September 2026 · revision

On questions the gap is closed; on agent tasks it is closing. In five weeks GPT-6 Astra took GPQA Diamond to 96.0% and ARC-AGI-2 to 95.0%, and Claude Fable 5.1 unstuck Humanity's Last Exam, the leaderboard that in August was frozen at the models of March. On OSWorld 2.0 the best passes 77.9% of the tasks partially and 41.7% strictly: that is where the difference between answering and doing shows.

On ARC-AGI-3 the same model scores 62.7% or 99.9% depending on the test harness. It is the most useful fact of this revision, and it is not about capability: it is about the fact that a score without its test harness cannot be compared with anything. In the same revision I removed from the table the GPQA Diamond human benchmark this page had carried from the start, 81.2%, because I have not found it again in any primary source.

Those who build are asking to slow down, and do not agree on how. Between 12 and 14 September Amodei, Altman, Hassabis and Musk ended up on the same side of a request to slow down, and LeCun on the other. The request has neither a date nor a threshold, and its weak point was named by Amodei himself the day after: China.

16 August 2026 · first publication

Compared with the Italian edition of the book published in June, the gap to watch has changed in nature. On questions, even at doctorate level, the human benchmark - where it exists - turns out to be reached or surpassed; it holds up on the tasks where you have to act over a long stretch, use a computer, combine tools. And that is the ground on which agents in companies are played out.

The laboratories' forecasts have become more cautious in form: the most concrete date in circulation, March 2028, is a corporate milestone of OpenAI's - how much internal research will be entrusted to AI systems - while for AGI what circulates is moving windows and probabilities.

And the two biggest environmental reports of 2026 make the comparison with the past harder: base years recalculated by both companies, progress table removed by Microsoft. The consumption figures can still be found; the percentages you have to reconstruct yourself.

Sources

• Revision of 3 October 2026. Scores: Anthropic, Claude Opus 5.5 page, 22 September 2026, for OSWorld 2.1 and Humanity's Last Exam; official OSWorld leaderboard (data as of 17 September 2026); Scale AI, Humanity's Last Exam leaderboard (updated to 17 September 2026); ARC Prize Foundation, data files of the ARC-AGI-2 and ARC-AGI-3 leaderboards; Epoch AI, benchmark data archive (GPQA Diamond, FrontierMath, SWE-bench Verified) and trends page; darioamodei.com, "We Must Pace the Frontier" (September 2026). Consulted on 3 October 2026.
• Revision of 18 September 2026. Scores: OpenAI, GPT-6 Astra release page, 3 September 2026; Anthropic, announcement of Claude Fable 5.1 and Mythos 5.1, 1 September 2026; ARC Prize Foundation, results page for GPT-6 Astra on ARC-AGI-2 and ARC-AGI-3. Human benchmark for GPQA Diamond: Rein et al., "GPQA: A Graduate-Level Google-Proof Q&A Benchmark", for the 65% and the 74%, and Epoch AI for the 69.7% measured by OpenAI on Diamond alone.
• Waymo, blog and updates page, consulted on 3 October 2026, for the openings of 1 September (Denver, San Diego and Tampa, "14 cities" in all) and of 14 September (Las Vegas), from which the count of fifteen areas comes, for the dates of Tokyo and Singapore, the expansion list and the 270 million miles of 24 September; for Singapore also TechRepublic, 18 September 2026. The Munich announcement of 25 August is the one read on 18 September 2026. California DMV, permit lists as of 18 September 2026, consulted on 3 October 2026. FDA, list updated to 4 September 2026, counted row by row on 3 October 2026. EMBL-EBI, AlphaFold database announcements of 16 March 2026 (complexes) and 24 September 2026 (viral complexes), consulted on 3 October 2026.
• Request to slow down: Axios, 12 September 2026, for Amodei's essay and Altman's reply; CNBC, 13 September 2026, for China; international press of 14 September 2026 for the signatures of Hassabis and Musk and for LeCun's reply. Consulted on 18 September 2026.
• Not updated in the revision of 3 October 2026: WebArena and GAIA, which come from the AI Index 2026, an annual source. SWE-bench Verified has been checked again: Epoch has not yet evaluated the September models, and Claude Opus 4.7 remains first. The rows are the ones from earlier revisions, and the date beside them says so.
• Benchmarks: Stanford HAI, AI Index Report 2026, chapter 2 (data exported in early 2026); leaderboards of the ARC Prize Foundation (data as of 13 August 2026), Epoch AI (benchmarks and trends; for the text stock, "Will we run out of data?", version of June 2024), Scale AI for Humanity's Last Exam, swebench.com. Consulted on 16 August 2026.
• Services: FDA, "Artificial Intelligence-Enabled Medical Devices" list, updated to 16 June 2026; waymo.com (rides pages and blog); Tesla, Q2 2026 update filed with the SEC on 22 July 2026 and robotaxi support page; California DMV, permit lists as of 12 August 2026; EMBL-EBI for AlphaFold DB v6; nobelprize.org. Consulted on 16 August 2026.
• Statements: Sam Altman's personal blog ("Reflections", January 2025) and OpenAI site (11 December 2025, 8 June 2026); darioamodei.com ("Machines of Loving Grace", October 2024; "The Adolescence of Technology", January 2026; post of June 2026); Lex Fridman Podcast no. 475 (23 July 2025); Axios (26 May 2026); BBC Radio 4 (27 December 2024); Big Technology Podcast (5 June 2026); Brown University, report of the lecture of 1 April 2026; amilabs.xyz; Dwarkesh Podcast (25 November 2025 and 13 February 2026); Nvidia-SSI press release (27 July 2026).
• Survey: AAAI, Presidential Panel on the Future of AI Research, report March 2025; Grace et al., "Thousands of AI Authors on the Future of AI", JAIR 2025, survey October 2023.
• Energy and water: IEA, "Key Questions on Energy and AI" (16 April 2026) and "Energy and AI" (April 2025, box 5.4); Google 2026 Environmental Report (June 2026, calendar year 2025); Microsoft 2026 Environmental Sustainability Report (July 2026, fiscal year ended June 2025).
• Where a leaderboard did not date the individual run, the date indicated is the one declared by the operator for the model or for the data export.