TL;DR: Cost is tokens times price, with output tokens several times the price of input. Estimate input plus output per request, multiply by expected volume, and do it before designing, because the answer often rules out sending the whole document on every call.
How to approach it
Do arithmetic out loud with plausible numbers. This question is not testing whether you remember a price list, it is testing whether you reach for a calculation at all, since most people discuss LLM features for a long time without ever multiplying anything. State assumptions as approximate and keep going.
A strong answer
A token is roughly four characters of English, so about 750 words per thousand tokens. Everything is priced per million tokens, and input and output have different prices, with output typically several times more expensive because it is generated one token at a time.
Work a case. A support assistant that takes a customer question, retrieves four knowledge-base passages, and writes an answer:
system prompt + instructions ~ 500 tokens
4 retrieved passages ~ 3,000 tokens
conversation so far ~ 1,000 tokens
the question ~ 100 tokens
input ~ 4,600 tokens
output ~ 400 tokens
At an order of magnitude of a few dollars per million input tokens and perhaps three to five times that for output, one request lands somewhere around a couple of cents. Ten thousand requests a day is a few hundred dollars a day, so tens of thousands of dollars a year for one feature. That number is worth having before the design review, not after.
The levers, roughly in order of effect:
Send less input. Retrieval is where the tokens are. Four passages instead of twelve, and reranking so the four are the right ones, cuts cost and usually improves quality, because a model given twelve passages of which two are relevant does worse than one given two.
Use a smaller model for the easy path. Most requests do not need the largest model. Routing simple cases to a small one and escalating only when needed is frequently the largest single saving, and it improves latency at the same time.
Cache. Identical or near-identical requests are common in support and search. Providers also offer prompt caching for a repeated prefix, which is exactly the shape of a long fixed system prompt, and that can cut the input cost of the fixed portion substantially.
Cap the output. Output is the expensive side. A max_tokens that matches what the interface can display stops a model writing an essay into a box that shows three lines.
Trim conversation history. Naive implementations resend the entire conversation every turn, so cost per turn grows through the session. Summarise older turns or keep a window.
Two things to raise that make the answer sound operational rather than theoretical. Cost per request should be a metric you emit and alert on, per tenant, because it moves when a prompt changes and nobody notices until the invoice. And the self-hosting comparison is a real calculation rather than an ideological one: a GPU costs the same per hour whether it serves one request or ten thousand, so hosting wins at sustained high volume and loses badly at low or spiky volume, and the crossover is worth computing rather than assuming.
What interviewers probe next
"Why is output more expensive?" Input is processed in one parallel pass. Output is generated sequentially, one token at a time, each requiring a full pass through the model, so it occupies the accelerator for far longer per token.
"How does context length affect cost and latency?" Cost rises with tokens sent. Latency to the first token rises with input length because the whole prompt is processed before generation starts, which is why a very long prompt feels slow before any text appears.
"What would you measure in production?" Tokens in and out per request, cost per request, and cost per tenant or feature. Without those you cannot tell a price change from a prompt change from a usage change.
Common mistakes
Discussing an LLM feature at length without ever multiplying tokens by volume.
Pricing input and output at the same rate, which understates the bill on any generation-heavy feature.
Resending full conversation history every turn and being surprised by cost growth within a session.
Assuming self-hosting is cheaper without computing the utilisation at which it becomes so.