★ The anti-POC guide · LLM / RAG · Production
Most LLM features don't die in production — they never get there. The POC impresses in the demo, then stalls: no way to measure whether it's improving, per-request costs discovered too late, an agent that collapses on the first edge case. This page lists the four failure causes I keep seeing, the questions to ask any provider before signing — me included — and the cases where, honestly, you don't need an LLM.
~20%
of the work is the model. The rest: the pipeline
1
eval set minimum, or you're flying blind
60s→10s
what a well-built pipeline delivers (Upfund)
Often 0
LLM needed — classic search is often enough
Without an evaluation set — real cases, expected answers, a score — 'the AI works' is an opinion. Every prompt iteration becomes a gamble, and the project dies the day someone important hits a bad answer. The eval isn't a luxury: it's the first thing to build, before the pipeline itself.
'Let's use GPT-4' is a meeting decision, not an engineering one. Target latency, per-request cost at your real volume, data sensitivity: that triangle picks the model — sometimes a cloud API, sometimes a local model via Ollama, sometimes a small model that's plenty. You benchmark on your case, not on a public leaderboard.
A demo agent has seen ten cases; your users will bring ten thousand. Without guardrails, a deliberately scoped toolset, and a plan for 'the model got it wrong', trust evaporates at the first derailment — and it doesn't come back. The difference between demo and production is everything that doesn't show in a meeting.
A working notebook is not a feature. Until the LLM sits behind your API, inside your interface, watched by your observability and deployed by your CI, it doesn't exist for your users. That's where projects stall most often: the data team is done, and nobody owns the last mile.
Q1
If a provider can't show you how they measured answer quality on a past project, you're funding their experiments. At Upfund, the eval is what let us say '6× faster AND more relevant' — a measurement, not an impression.
Q2
A POC at 50 requests a day says nothing about production at 50,000. Demand the number at your scale before signing: it decides between cloud API and local model, and sometimes reshapes the whole architecture.
Q3
It will be. A good answer talks about guardrails, fallbacks, and what the user sees on that day. If the answer is 'with good prompting it won't happen' — hang up.
Join the waitlist: people on it hear first when a freelance slot opens up. Leave your email below, or write to me directly at alielmufti25@gmail.com.
Join the waitlistOften, no. If well-configured full-text search, business rules, or a better-designed form solves the problem, that's simpler, cheaper and more reliable. An LLM earns its place when the input is unpredictable natural language or the answer requires synthesis — that was Upfund's search; it isn't everything.
RAG first, for the vast majority of product cases: it grounds the model in your up-to-date data, costs less, and is easy to evaluate. Fine-tuning only pays off for a very specific tone, format or domain — and nothing stops you from adding it later.
Yes. I've built fully local stacks — Ollama, LanceDB, FastAPI — where confidentiality demanded it: nothing leaves your infrastructure. You give up some raw power versus the big cloud models; the trade-off should be priced on your case, and it's often very acceptable.
My day rate starts at €600/day (excl. VAT). A serious POC — your data, one use case, an evaluation — fits in a few days and tells you whether it's worth going further. Production rollout then takes weeks depending on scope; precise range after a 30-minute call.