LLM Cost: Estimating and Reducing Your Token Bill

Why is your LLM bill unpredictable?

The same prompt, executed twice with two different models, can generate a cost difference close to 100×. Yet, the billing mechanics of large language models (LLMs) rest on four simple variables that most IT teams ignore when estimating a project. This article gives you the method to estimate your consumption before deploying, then to reduce costs without sacrificing quality.

Prerequisites: four concepts to master before any calculation

Before opening a calculator, clarify these four fundamental notions.

The token: the universal unit of measurement

A token is the minimal text fragment processed by an LLM. It's not a whole word: in practice, a token corresponds to approximately four characters, or three quarters of an English word (slightly less for French). Order of magnitude to remember:

• 1,000 tokens ≈ 750 words ≈ 1.5 pages of text
• The prices displayed by providers are always expressed per million tokens (abbreviated M tok), which avoids writing six decimals.

Input vs Output: two counters, two prices

Each API call generates two types of consumption:

• Input: everything you send to the model — the question, system instructions, conversation history, injected documents.
• Output: everything the model generates in response.

Output is systematically 3 to 6 times more expensive than input, regardless of the provider. The reason is architectural: the model reads your input text in parallel, but produces its response one token at a time, re-reading everything that precedes at each step. This sequential process ties up the GPU much longer.

Prompt caching

Prompt caching allows you to reuse a prompt prefix already sent (system instructions, fixed reference document) without recharging it at full price. The cached portion is re-read at approximately 1/10th of the normal price, meaning up to 90% savings on that portion.

TCO: don't confuse API bill with project cost

TCO (Total Cost of Ownership) refers to all expenses related to an LLM project, not just the token bill. This distinction is critical when estimating — we return to it in detail below.

What really varies your bill

Variable #1: model choice (factor up to 100×)

It's the most powerful lever. For illustration, here's the order of magnitude of pricing gaps between model categories:

Variable #2: the length of generated responses

Since output costs several times more than input, a task producing long responses will be significantly more expensive than a task returning a single word or score. This is a parameter often overlooked during initial estimation.

Variable #3: exponential growth of conversational context

An LLM has no persistent memory between calls. To maintain conversation coherence, your application must return the entire history with each turn. Direct consequence:

• At 1st message: you pay for 1 question.
• At 10th message: you pay for 9 previous exchanges + the new question as input.

Cost therefore doesn't progress linearly with message count — it accelerates. An agent chaining many steps can generate a bill well above predictions for this reason alone.

Technical prerequisites to estimate an LLM project

Before launching any estimation, ensure you have the following elements:

• Access to an API account with the intended provider(s) (Anthropic, OpenAI, Google, etc.)
• A tokenizer to precisely count tokens in your prompts. OpenAI provides Tiktoken; Anthropic offers a counter in its console.
• Official pricing pages of targeted models, checked the day of estimation.
• An estimate of monthly volume of requests (number of calls, not number of users).
• A representative example of a typical request, with its system prompt, average history and expected response.

Estimate your monthly bill: the 5-step method

1import tiktoken
2 
3enc = tiktoken.encoding_for_model("gpt-4o")
4tokens_input = enc.encode(your_complete_prompt)
5print(f"Input tokens: {len(tokens_input)}")

1Unit cost = (tokens_input / 1,000,000 × price_input) + (tokens_output / 1,000,000 × price_output)

1Estimated monthly bill = unit cost × monthly request volume

Four levers to reduce your bill

1. Choose the smallest model capable of the task

Classifying an email, extracting a date, rephrasing short text: economical models (Haiku, GPT-4.1 mini, Gemini Flash, DeepSeek V3) handle this well. Reserve high-end models for complex reasoning, multi-step analysis or critical content generation.

2. Enable prompt caching on your fixed prefixes

If your application returns the same system instructions or reference document with each call, enable caching with the provider. The cached portion is charged at approximately 1/10th of normal rate. Ideal use case: a long fixed context (product documentation, business rules base) followed by a variable question.

3. Use the Batch API for deferrable tasks

Most providers offer a batch processing API (batch): you send a set of requests, they're processed in the background (usually within 24 hours) and charged at -50% off input and output. Applicable for classifying thousands of documents, generating summaries in bulk or enriching a database. Unusable for a real-time chatbot.

4. Control conversational context size

Rather than returning the entire history indefinitely, implement a rolling summary strategy: condense the last N exchanges into a few lines and inject only the essentials. This technique alone can significantly reduce costs for long conversations.

TCO: don't budget only for tokens

The API bill is only the tip of the iceberg. A production LLM project involves other cost items often underestimated:

• Development and maintenance: developer time to build, test and maintain prompt pipelines and integrations.
• Infrastructure: vector database, application hosting, document update pipeline (essential for a RAG chatbot).
• Retries: calls that fail (timeout, parsing error) are retried — and recharged.
• Monitoring: tools for tracking quality and costs (LangSmith, Helicone, Azure Monitor depending on your stack).
• Human supervision: if your use case requires validation or human takeover on certain responses.

Managed API vs self-hosting: how to choose?

When your request volume becomes very large, the question of self-hosting an open-weight model (whose weights are public, like Llama or Mistral) naturally arises.

The practical rule: start with the API, switch to self-hosting only when your traffic is large, stable, and the API bill becomes a structural expense item. To prototype free with open-weight models, platforms like OpenRouter provide access to models like Llama or DeepSeek R1 with rate limits.

Troubleshooting

Your bill explodes without visible increase in user count → Check conversation history length injected. It's likely your application is accumulating context without truncating it. Add logging of input token count per call to identify the drift.

Prompt caching doesn't seem to work → Caching is only enabled if the exact prefix (character for character) is repeated call to call and exceeds a minimal token threshold (usually 1,024 tokens at Anthropic). Verify your system instruction isn't dynamically modified at each call.

Your initial estimation is very far from actual bill → The most frequent cause is poor estimation of output tokens (responses are longer than expected) or unconsidered conversation history in the calculation. Relaunch estimation with real averages from production logs.

Key points to remember

• The chosen model is the most impactful parameter — up to 100× difference between the most economical and most powerful.
• Output costs 3 to 6× more than input: a task generating long responses is significantly more expensive.
• Conversation history is recharged each turn: cost grows faster than message count.
• Estimation method: cost of an average request × monthly volume = reliable order of magnitude.
• Reduction levers: small model suited to task, prompt caching, batch API, controlled context.
• TCO ≠ API bill: budget development, infrastructure, retries and monitoring.
• Self-hosting: pertinent only at high stable volume — always start with the API.