AI Inference Markets

Looking at inference as a market tape

The first tool is essentially an inference market tape.

It follows the same model across multiple provider routes and records the variables that seem economically relevant: quoted price, latency, reliability, provider dispersion, execution quality and observable paid-token demand.

The purpose is not to produce another model leaderboard.

I am more interested in the differences that appear after the model itself is held constant.

If two providers are both serving the same model but one is cheaper, another is faster, one has intermittent capacity and another benefits from different caching economics, then describing them simply as equivalent access to the same model hides part of the economic reality.

I started treating each route as something closer to an execution venue.

That framing changed the questions I was asking.

Instead of simply asking which provider is cheapest, I can ask whether price differences persist, whether routes become more or less attractive under load, whether latency deteriorates when demand increases, how much reliability varies, and whether routing behaviour appears to respond to those differences.

In other words, the objective of the tape is simply to make a market that is usually hidden behind an API call more observable.

Searching for dislocations

The second tool grew naturally from the first.

Once different routes were visible simultaneously, I wanted to know whether temporary dislocations existed between them.

Could exactly the same underlying inference have materially different economics depending on where and when it was executed?

At first this looked like a relatively straightforward arbitrage question.

If Provider A offers the same model more cheaply than Provider B, perhaps there is something to capture between them.

The more I looked at it, the less useful that simple interpretation became.

Quoted token price is only one component of execution.

A route can be cheaper and slower. It can be cheaper but less reliable. It can have attractive headline pricing but limited capacity. Cache behaviour can make the realised marginal economics very different from what the published price suggests.

So the object I am really observing is closer to:

model × provider × price × latency × reliability × capacity × cache behaviour × routing

That is where I think the market-structure analogy starts becoming genuinely useful.

Not because AI inference has somehow become a financial market, but because execution quality is becoming multidimensional.

The cheapest route is not necessarily the best route, in much the same way that the best quoted price in another market does not automatically tell you everything about the quality of an execution.

Some things became clearer quite quickly

A few observations have held up well enough that I now treat them as useful working assumptions.

The first is that access to the same model is not necessarily the same economic product across providers.

That sounds obvious once stated explicitly, but a lot of API comparison still collapses model and execution venue into one thing.

The second is that routing does not appear to be governed by price alone.

Reliability, capacity and latency matter, and caching can materially alter marginal economics. The effective cost of an inference therefore depends partly on the route and partly on the state of that route when the request arrives.

This also made me more cautious about apparent arbitrage.

There are moments when the provider landscape looks inefficient on a screen. Prices differ. Capacity differs. Some routes appear underutilised relative to others.

But an observable inefficiency is not automatically an accessible economic opportunity.

That distinction became one of the more useful lessons from the project.

I also tried the provider side

One branch of the research was more ambitious.

If inference capacity is fragmented and routes occasionally trade at different implied economics, could capacity itself be supplied dynamically?

In other words, could I behave as a very small provider or something loosely analogous to a market maker, bringing compute online when economics became sufficiently attractive?

I spent some time testing that idea.

It was much less promising than I expected.

The simple version of the thesis focused too much on the visible price dispersion and not enough on everything required to actually capture it.

Market access matters.

Routing behaviour matters.

Reputation and provider selection matter.

Infrastructure utilisation matters.

The economics of keeping capacity available matter.

And most importantly, seeing a price difference does not mean requests will be routed toward you in a way that allows the difference to be monetised.

That negative result changed how I think about the project.

I started with a familiar market instinct: if fragmentation creates price dispersion, perhaps there is an arbitrage.

What I found was closer to this:

fragmentation creates observable differences, but the right to capture those differences depends on market structure.

That is a much more interesting problem.

It is also much closer to how real markets work.

Where the analogy with financial markets helps

There are several concepts from traditional market structure that seem increasingly relevant to inference.

Best execution is one.

Venue selection is another.

Capacity begins to look somewhat like liquidity: a price can exist without enough accessible supply behind it to make the quote particularly useful.

Routing intelligence matters because the optimal venue can change depending on what the request values most heavily.

A latency-sensitive workload may make a different choice from a cost-sensitive batch workload.

A reliability-sensitive enterprise workflow may rationally pay more for a route that fails less often.

Caching introduces another layer because the marginal economics can depend on whether prior computation or context can be reused.

Temporary dislocations can exist, but whether they are exploitable depends on access and execution mechanics rather than simply on detecting them.

None of these mappings is exact.

I actually think the places where the analogy fails are at least as useful as the places where it works.

Where it breaks

Inference is not a fungible financial asset in the conventional sense.

A token produced by one route cannot simply be resold somewhere else.

There is no conventional order book.

Capacity is perishable.

Demand is workload-specific.

Quality can include dimensions that are difficult to reduce to a single price.

Providers may route according to rules that are only partially visible from outside.

And the value of an execution is ultimately tied to the application consuming it, not merely to the model that produced it.

That makes me cautious about importing financial-market language too literally.

The objective of this work is not to argue that OpenRouter is an exchange or that inference providers are equivalent to trading venues.

It is to use market structure as a way of asking better questions about an economic system that is becoming increasingly fragmented.

The question I am now more interested in

The project began with price.

I increasingly think routing intelligence may matter more.

If AI workloads become large enough, enterprises will not simply choose a model.

They may continuously choose between different ways of obtaining that model's output.

That decision could depend on cost, latency, geography, reliability, capacity, privacy, cache state, workload priority and perhaps eventually contractual guarantees around execution quality.

At that point, the economic unit is no longer simply:

“one million tokens from Model X”

It becomes something closer to:

“one million tokens from Model X, delivered under a particular set of execution conditions.”

That is a more interesting product.

And if that is where the market develops, some of the valuable infrastructure may sit not only in the models or the compute underneath them, but in the layer deciding where inference should be executed at any given moment and why.

I do not yet know how economically important that layer becomes.

Model providers may consolidate distribution. Pricing may converge. Capacity may become abundant enough that many of today's differences stop mattering. Large buyers may negotiate directly and make public routing markets less relevant.

Those are all plausible outcomes.

For now, the purpose of the research is narrower.

I want to understand whether inference is developing enough of a market structure that concepts such as venue selection, differentiated execution quality, temporary dislocations and best execution become useful rather than metaphorical.

The answer so far is mixed, which I think is the right result.

Some of the analogy survives contact with the data.

Some of it does not.

And the failed attempt to turn visible provider dispersion into a straightforward economic opportunity was probably one of the more useful parts of the exercise.

It reinforced something I try to keep in mind across most of my market research:

observing an inefficiency, understanding why it exists and having a practical way to capture it are three different things.

That is the part I am continuing to test.

Status

Current research · Work in progress.

The project currently includes an inference market tape and a dislocation scanner built around OpenRouter provider routes, alongside experiments into routing economics and dynamic capacity provision.

The purpose is not to claim that AI inference already functions like a mature financial market. It is to identify which elements of market structure are beginning to emerge, which ones matter economically, and where the analogy stops being useful.