How to reduce AI costs without hurting service quality

AI spend rarely rises because of one dramatic mistake. A prompt gets longer, history accumulates, two attempts replace one, and an expensive model starts handling easy cases as well. Then the invoice arrives. The first move is not to chase the cheapest token: it is to measure the cost of an outcome someone can actually accept and remove work that does not contribute to it.

This guide helps you find the first sensible reduction, test it on real cases and know when to stop. You do not need to know the system's code, but you do need visibility into calls, costs and a sample of outputs.

If you are short on time, do these three things

1. Allocate spend to work. Go beyond provider and model. Separate customer support, document analysis, content production and internal operations.
2. Choose one recurring job. Take twenty real cases and record cost, attempts, accepted outcome and human correction time.
3. Change one lever at a time. Remove unnecessary calls and retries first; then test model choice, output length, caching, context and batch. Keep a variant only if it passes the same quality gate.

If you cannot complete step 1 today, you have already found the first problem: spend is not attributable. Make it legible before trying to optimise it.

Where the money goes

Tokens are the units models use to read input and produce output, but an AI system's bill can include much more:

• Repeated calls after an error or rejected answer;
• Models and reasoning settings that are more powerful than the job requires;
• History, instructions and documents resent on every request;
• Answers far longer than anyone needs;
• Document retrieval, vector databases, safety checks, tools and storage;
• Human review and rework;
• Reserved capacity or infrastructure that sits idle.

This is why “we use too many tokens” is often an incomplete diagnosis.

The metric that prevents false savings

A one-cent call can cost more than a five-cent call if it has to be repeated, corrected or discarded.

Use this denominator:

Cost per accepted outcome = total technical cost plus verification and rework, divided by accepted outcomes.

For a first comparison, record:

Keep the median and the most expensive 10% as well as the mean: exceptions often create the bill.

The levers, in the order I would test them

1. Remove work that should never have started

Look for duplicate calls, loops, unbounded retries, requests launched without minimum inputs and outputs that are generated but never used. This removes pure waste without asking the model to perform with less information.

Works well when: the workflow has grown through successive additions and nobody owns the complete call count.

It is not enough when: every call is necessary and spend is concentrated in the model or context.

2. Use expensive capability only where it is needed

Split cases into simple, ambiguous and high-risk groups. Test a lighter model or reasoning setting on the first group while retaining the stronger route for the rest. A cheaper model does not save money if it creates more errors or escalations.

Do not promote routing because three examples “look the same”. Run the same test set through both routes and apply the same quality gate.

Works well when: most volume is repetitive and only a minority of cases requires complex judgement.

It fails when: the router cannot recognise difficult cases or the quality check is vague.

3. Ask for the output that will actually be used

If the system needs five fields, do not request a report and then extract the fields. Define a schema and an appropriate length. Reducing output is often safer than removing input because it changes the answer's shape, not what the model is allowed to know.

Works well when: software consumes the response or the result lands in structured fields.

It is not enough when: long documents and histories dominate the bill.

4. Reuse stable context through caching

Major providers offer prompt caching in different forms. Put stable instructions and documents first, variable data last, then inspect reported usage. Support for caching does not guarantee a hit, and write, storage and expiry costs matter.

Works well when: many requests share a large, genuinely identical prefix.

It fails when: content changes constantly, reuse is too low or caching costs more than it avoids.

5. Bring in only the context each case needs

Do not “add RAG” in the abstract. RAG, short for retrieval-augmented generation, is a way to select supporting documents before asking the model to answer. Retrieve a small number of relevant sources, keep their references and verify that required facts survive. Turn long history into facts, decisions, constraints and open items. Pass useful tool fields rather than pages of raw output.

Works well when: most of the input does not change the final decision.

It fails when: selection removes an exception, deadline or source the work needs. The Distill tests below show both outcomes.

6. Batch work that can wait

As of 15 September 2026, OpenAI and Anthropic document a 50% discount for their respective batch APIs, in exchange for asynchronous processing and provider-specific constraints. This is useful for offline classification, extraction and evaluation, not for a person waiting on an answer.

Works well when: the result can arrive later and the workload is stable enough to form a useful batch.

It fails when: someone is waiting, the batch is tiny or the delay creates more operational work.

7. Consider commitments and infrastructure last

Reserved capacity, volume agreements and running a model on your own infrastructure, often called self-hosting, may lower unit cost. They can also turn a variable bill into idle capacity, maintenance and on-call work. Consider them only after volume, load shape and quality requirements have become stable.

Distill: one accepted test and one rejected test

In the DL Frontier Lab we tested a narrow question: how much context can be removed while preserving the facts needed for the job?

For a research handover after one week, a full package contained the previous report, the handoff and the session record. The reduced variant kept only handoff and session.

• Measured size: 8,074 to 1,353 words;
• Reduction: 83.24%;
• Required facts retained: 5 out of 5;
• Verdict: accepted.

Words were a repeatable proxy for context size, not provider-billed tokens. The result shows a large input reduction on that job; it does not establish an 83.24% universal saving in money.

A second test looked even better: 4,326 to 604 words. Yet the reduced variant retained only 3 facts out of 5 and lost two operational deadlines. It was rejected on quality.

The useful finding is not “context can be compressed by 83%”. It is this: selection works when the reduced package already contains the right sources, and fails when an excluded source owns a fact the job must remember.

Read the method and its limits in the DL Frontier Lab Distill note.

What usually works best

There is no universal ranking, but there is a prudent order:

1. Unnecessary calls, retries and unused outputs;
2. Shorter, structured output;
3. Model and reasoning routing with paired evaluation;
4. Caching genuinely stable context;
5. Selective context with a fact-preservation gate;
6. Batch for non-urgent workloads;
7. Self-hosting or reserved capacity for mature workloads.

The first two usually remove waste with limited risk. Routing and caching can matter greatly at scale, but need paired measurements. Context selection has high potential and a higher quality risk. Batch offers a clear list-price benefit only for work that can wait. Infrastructure commitments come last because they change the operating model, not only the invoice.

This order keeps the most interesting technical lever from becoming a solution in search of a problem.

Shortcuts that often fail to save money

• Choosing the cheapest model for every case. Retries and rework can consume the apparent saving.
• Shortening prompts and documents without a fact gate. The removed sentence may contain the costly exception.
• Adding RAG without measuring retrieval cost and answer quality. It introduces more calls, storage and failure modes.
• Additional model training, or fine-tuning, before fixing unnecessary calls and poor inputs. It can help behaviour and specialisation, but it does not repair a wasteful workflow.
• Caching content that is rarely reused. Write or storage cost remains while the expected reuse never appears.
• Self-hosting because the API line item looks high. Total cost includes accelerators, peaks, observability, upgrades and on-call staff.
• Ignoring tools, safety checks, storage, rejected outputs and human rework. The model invoice is only one line in the operating cost.

A fourteen-day test plan

Days 1-2: export at least thirty days of usage and map provider, model, environment and workload.

Days 3-4: choose one material job and collect twenty representative cases, including difficult ones. Define the facts and errors that block promotion.

Days 5-8: run the current flow and record technical cost, calls, retries, quality and review time.

Days 9-11: test one lever while keeping everything else stable.

Days 12-14: compare only outcomes that pass the gate. Promote, revise or roll back, and leave a record of the decision.

Copy this worksheet

• Job: ___
• Monthly volume: ___
• Current technical cost: ___
• Accepted outcomes: ___
• Cost per accepted outcome: ___
• Facts that must survive: ___
• Errors that block promotion: ___
• First lever to test: ___
• Test set: ___
• Rollback rule: ___

If cost, accepted outcomes or required facts cannot be filled in, you are not ready to optimise. You are ready to establish the baseline.

A note on provider pricing and features

Provider features change. This guide was checked on 15 September 2026 against official OpenAI, Anthropic, Google Cloud and AWS documentation. Batch, caching and routing have different prices, thresholds, expiry and usage fields. Always check the current provider page before approving a business case.

Operational sources consulted:

• OpenAI Batch and prompt caching;
• Anthropic prompt caching and Message Batches;
• Google Cloud Vertex AI pricing;
• AWS Bedrock prompt caching and intelligent prompt routing.

If spend crosses several providers or cannot be allocated to real work, the AI Spend Audit service reconstructs the baseline, waste and first experiments within an agreed scope. It is an assisted request, not an automated purchase, and it does not promise a saving percentage before the data has been examined.