Most requests that arrive as "can you add AI to this" resolve into one of three things: a search problem, a classification problem, or a drafting problem. Naming which one you have decides the architecture, the cost and whether the feature is worth building at all.
Search problems want retrieval
If the answer already exists somewhere in your documents, tickets or catalogue, you want retrieval — chunking, embeddings and a vector store, with the model summarising what was retrieved rather than recalling it. This is the cheapest and most defensible category, because you can always show a user where an answer came from.
Classification problems rarely need a large model
Routing a support ticket, flagging a risky transaction, tagging a listing: these are classification tasks with labelled data sitting in your database already. A small fine-tuned model, or sometimes plain gradient boosting, will beat a general-purpose model on latency, cost and consistency — and it will not surprise you at three in the morning.
Drafting problems need guardrails, not accuracy
When the model writes something a human will edit — a reply, a description, a first-pass report — accuracy matters less than making the edit path obvious. The feature is a good one if the human is faster with it than without it, and that is measurable.
The part nobody budgets for
Evaluation. An AI feature without a test set is a feature you cannot change. Before we ship one we build a small, honest evaluation set from real inputs, so that a prompt change, a model upgrade or a cheaper provider can be assessed in an afternoon instead of argued about for a week.
It is also where the ongoing cost lives. Inference is a line item that scales with usage, and privacy constraints often push work on-premise. Both are cheaper to plan for at the start than to discover after launch.
