OpenAI's GeneBench-Pro shows AI still fails most biology tests

The News

OpenAI has released GeneBench-Pro, a research-grade benchmark that measures whether AI models can handle the messy, judgement-heavy analysis that real computational biology demands. The set carries 129 problems spread across 10 domains and 21 sub-domains, reaching into genomics, quantitative biology and translational medicine.

The headline result is a reality check. OpenAI's strongest system, GPT-5.6 Sol, cleared just 28.7% of the problems at its highest reasoning setting, rising to 31.5% with Pro mode switched on. That is a steep climb from the project's early days, when the best frontier model of the time, GPT-5, sat below 5%. GPT-5.6 Sol also solved nearly six times as many questions as GPT-5.2 while burning roughly two-thirds of the tokens.

Each problem was built synthetically so that the correct analytical path is known in advance, and 82 of the 129 questions were sent to external graduate students, postdocs, industry scientists and professors for review. OpenAI is open-sourcing 10 representative questions on Hugging Face and will hand a 50-question subset to Artificial Analysis for independent, third-party testing.

Why It Matters

Benchmarks quietly steer where the whole field pours its effort. When coding evaluations became the yardstick, models grew sharply better at writing code. GeneBench-Pro tries to point that same competitive pressure at what OpenAI calls research taste: knowing which question a dataset can answer, when to revise an assumption, and when a result is actually decision-ready.

The economics on display are hard to ignore. Reviewers estimated that a single GeneBench-Pro problem would take a human expert around 20 to 40 hours. At a conservative 200 dollars an hour, that is thousands of dollars of skilled labour per question, against inference costs of only a few dollars. Today's agents remain too unreliable to replace scientists, yet even partial automation at this price gap starts to reshape the arithmetic of research.

There is a warning inside the optimism too. Frontier models still miss more than two-thirds of the problems, stumbling on the same weakness again and again: they gather observations but struggle to close the inferential loop. OpenAI expects the benchmark could be saturated by the end of the year, so this snapshot of failure may prove short-lived.

Indian Angle

India has spent the past few years generating exactly the kind of data GeneBench-Pro is built to interrogate. The government-backed Genome India programme has already sequenced more than 10,000 Indian genomes, and biobank-scale ambitions are growing. The bottleneck OpenAI describes, where the limiting factor shifts from producing data to turning it into decisions, is precisely the wall Indian genomics is now hitting.

For cost-conscious Indian drug discovery, the price signal is the story. Firms such as Biocon, Dr Reddy's and Sun Pharma, along with contract research houses like Syngene, run target-discovery pipelines where senior computational biologists are scarce and expensive. A tool that could triage hypotheses at a few dollars a problem, rather than thousands in expert time, would land hardest in markets where R&D budgets are tightest.

The caution matters as much as the promise. With models solving under a third of these tasks, Indian regulators and institutions such as the DBT and ICMR have room to set guardrails before automated analysis becomes routine in clinical and translational work. India's deep bench of bioinformatics talent is an asset here, but only if adoption stays paired with expert oversight rather than replacing it outright.

FAQ

What exactly is GeneBench-Pro?

It is a benchmark of 129 synthetically built computational-biology problems across 10 domains and 21 sub-domains. Each hands a model a messy dataset and asks it to choose an analysis, iterate, and reach a decision-ready answer, testing judgement rather than recall.

How well did the models actually do?

OpenAI's best system, GPT-5.6 Sol, passed 28.7% of the problems at its highest reasoning level, or 31.5% with Pro mode. That is up sharply from GPT-5, which scored below 5% on the earlier GeneBench.

What does this mean for Indian genomics?

India's Genome India programme has sequenced over 10,000 genomes, so the analysis bottleneck is real here. Cheap, reliable AI analysis could stretch scarce expert capacity across Indian pharma and research, though current accuracy still demands human oversight.

Where can I read the original announcement?

OpenAI published the full technical write-up on its blog, including the domain atlas and results tables, with 10 questions open-sourced on Hugging Face for anyone who wants to inspect the tasks directly.

This story was reported by OpenAI. Read the full original coverage at OpenAI.