Subquadratic's sparse-attention bet could reset AI's cost maths

The News

Subquadratic, a Miami-based startup, has come out of stealth with a bold pitch: it has cracked a maths problem that has slowed large language models for close to a decade. Its system, SubQ, leans on a sparse-attention design rather than the dense attention that powers most mainstream models, and the company argues this rewires the economics of running AI at scale.

The headline claim is cost. In third-party testing run by the data firm Appen on the RULER 128 benchmark, Subquadratic says SubQ finished the task for $8, against roughly $2,600 for Anthropic's Opus 4.6. On speed, it reports the model runs 56 times faster than systems built on FlashAttention. It also points to a 89.7% score on LiveCodeBench and 98% accuracy on needle-in-a-haystack retrieval across context windows of 6 million and 12 million tokens, far beyond the million or so tokens common today.

SubQ is built on reweighted values from Qwen, the Chinese open-source family. Its founders, chief executive Justin Dangel and chief technology officer Alex Whedon, say the trick is selectivity. "We dynamically select which ones are important," Whedon said. The firm claims tens of thousands of early-access sign-ups and more than 500 enterprise customers on its waitlist.

Why It Matters

Dense attention forces a transformer's compute to grow quadratically as inputs lengthen, so doubling the text roughly quadruples the work. That single property has shaped GPU bills, energy use and context limits across the industry. A sparse-attention alternative that holds its quality would loosen all three constraints at once.

Scepticism is warranted, and it is loud. Will Depue, an independent researcher formerly at OpenAI, warned that "public evidence does not yet justify the stronger claim". Others are warmer: Jeanine Sinanan-Singh, who directs generative AI research at Appen, said the results "could be a game changer". The last time a young lab promised to rewrite transformer economics, the claims rarely survived independent replication. The difference now is that Subquadratic has begun publishing benchmark receipts rather than asking for faith.

Indian Angle

For Indian builders, the number that matters is not 56 times faster but $8 versus $2,600. Inference cost, billed in dollars, is the hardest ceiling for rupee-funded startups, and a model that turns a four-figure run into a single-digit one changes which products are even worth attempting. Teams at Sarvam, Krutrim and a long tail of bootstrapped GenAI firms have all flagged compute spend as their binding constraint, and sparse attention attacks exactly that line item.

The 12-million-token context window matters here too. Indian enterprise work leans on long, messy documents: multi-year GST filings, vernacular contracts, sprawling KYC trails and case law across languages. Feeding all of that into one cheap prompt is worth more in India than a few points on an English coding test.

There is a note for policymakers. MeitY's IndiaAI mission has spent heavily subsidising GPU access precisely because compute is dear. If efficiency, not raw silicon, becomes the lever deciding who can ship, India's deep bench of systems engineers becomes a sharper edge than its access to scarce chips.

FAQ

What exactly did Subquadratic announce?

It left stealth with SubQ, a language model built on a sparse-attention architecture instead of the dense attention used by most rivals. The company published third-party benchmarks claiming large gains in speed, cost and context length, alongside early demand of tens of thousands of sign-ups and more than 500 enterprise customers waiting for access.

How independent are the benchmark numbers?

The headline figures come from tests run by Appen, a data-services firm, not solely from Subquadratic's own lab. That is stronger than self-reported marketing, but it is not yet peer-reviewed or widely replicated. Independent researchers, including former OpenAI staff, have urged caution until outside parties reproduce the speed and cost results across varied workloads.

Why should Indian developers care?

Because the binding constraint for most Indian GenAI startups is dollar-denominated inference cost, not ideas. If SubQ's claimed drop from roughly $2,600 to $8 on one benchmark holds even partially, it widens the range of products small Indian teams can afford to run, especially long-context tools for legal, tax and vernacular document work.

This story was reported by MIT Technology Review. Read the full original coverage at MIT Technology Review.