llm

Four options with genuinely different costs, and the requirement is usually narrower than it first sounds. Establish whether it is about the network path, the data residency, or the provider's retention terms, because those need different answers.

You have added a dependency with a latency distribution an order of magnitude wider than anything else in your stack, whose failure mode is a plausible wrong answer rather than an error.

Without one, every team implements its own retries, its own key handling, its own logging policy, and nobody can answer what the company spends or what a model change would break.

Token spend has no natural ceiling. Every retry, every retrieved document, every chain step multiplies, and nothing in the system tells anyone until the invoice arrives.

Prompts contain whatever the user typed, which in a support product is names, account numbers and medical detail. Full-text logging is a data-protection decision, not a debugging preference.