The Margin Gap
Classic SaaS gross margins sit between 75 and 90 percent. That is not a range — it is an industry invariant. If you sell software, your COGS is hosting, support, and payment processing. Those are single-digit percentages of revenue.
AI-native companies do not have that luxury.
ICONIQ Capital's State of AI 2026 surveyed roughly 300 AI companies under $100 million in revenue. The average product gross margin: 52 percent. Bessemer's State of AI 2025, which skews slightly more mature, pegs LLM-native companies at 65 percent.
The gap between 52 and 80 is not a rounding error. It is the entire inference budget.
Inference Is Not Overhead
Most AI startups still classify LLM API spend under R&D or engineering. That made sense in 2023, when a company was shipping one product and burning tokens on internal testing.
It does not make sense today. Every token sent to Anthropic, OpenAI, or Google in production is a cost of goods sold. It is incurred because a customer triggered a request. If that customer did not exist, that cost would not exist.
Yet when CFOs at these companies look at their P&L, the inference line is buried inside engineering. There is no per-customer breakdown. There is no gross margin by customer. There is just one big number that grows every month.
The Agentic Multiplier
Reasoning and agentic workflows have changed the cost curve. A 2023 completion used a few thousand tokens. A 2026 multi-agent reasoning workflow — planning, tool use, reflection, validation — can burn 10 to 100 times more tokens per task.
Per-token prices are falling. Cost per task is rising.
This is the structural problem. Even if Anthropic cuts prices by 40 percent next quarter, a company that ships agentic features will see its inference bill grow because the volume per task grows faster than the price drops.
What the Distressed Companies Are Doing
A few public data points illustrate the pressure:
- Cursor reportedly ran negative gross margin, paying Anthropic roughly $650 million against $500 million in revenue.
- Perplexity spent approximately 164 percent of its 2024 revenue on AWS, Anthropic, and OpenAI combined.
- Replit was forced to introduce usage-based pricing to lift margins out of single digits.
These are not outliers. They are leading indicators.
The Board Metric That Is Coming
Right now, most boards accept "we are investing in growth" as the answer to margin questions. That answer has an expiration date.
The metric boards will ask for next — and the one that separates companies that can answer from companies that cannot — is per-customer gross margin. Not aggregate. Not blended. Per customer.
If you cannot tell your board which customers are profitable and which are burning cash, you do not have a margin problem. You have a visibility problem.
Building Visibility
The fix is not a new pricing model. It is not a new vendor negotiation. It is a data pipeline:
- Tag every LLM request with a customer identifier.
- Compute the dollar cost from the vendor's pricing model.
- Attribute that cost to the customer's revenue in the same period.
- Calculate gross margin per customer, per feature, per workflow.
Once you have that pipeline, three things change. You can price fairly. You can flag overrun before the invoice arrives. And you can answer the board question before they ask it.
TokenOps builds this pipeline. But the underlying pattern — attribute every token, reconcile every invoice — is what every AI CFO needs to implement, with or without us.
William Min is the creator of TokenOps and a Technical Product Manager at Lovie. He has 12+ years of experience building payment infrastructure and fintech products. View his LinkedIn profile.
Sources: ICONIQ Capital, State of AI 2026 (Jan 2026, ~300 AI companies). Bessemer Venture Partners, State of AI 2025. Public reporting on Cursor, Perplexity, and Replit cost structures from The Information and company disclosures.