Every week someone asks if they should fine-tune a model. 9 times out of 10, the answer is no. Here's what most companies actually need instead — and why smart architecture beats expensive training runs.
Every week someone asks me: "Josh, should we fine-tune a model?"
My answer, 9 times out of 10: No.
[Pause for the fine-tuning evangelists to fill the comments with reasons I'm wrong]
Here's what most companies actually need instead:
011. Better Orchestration
Retries, fallbacks, model routing. The unsexy plumbing that makes AI reliable. At Apptivity, we've seen more production incidents caused by missing retry logic than by model quality issues. A well-orchestrated pipeline with a base model will outperform a fine-tuned model sitting behind a single API call with no fallback.
022. A Real Evaluation Framework
If you're vibe-checking outputs manually, you're not production-ready. You're demo-ready.
Build automated evals. Define what "good" looks like quantitatively. Track regression. Without this, you have no idea whether fine-tuning even helped — you're just spending money on vibes.
033. Clean Data Pipelines
Garbage in, hallucinations out. No amount of fine-tuning fixes bad data.
We consistently find that companies wanting to fine-tune haven't even audited their training data. They're feeding contradictory documents, outdated procedures, and formatting chaos into a model and wondering why it hallucinates.
044. Cost-Aware Architecture
Route simple queries to cheaper models. Cache aggressively. $200K in fine-tuning vs. $5K in smart architecture is not a close call.
At Apptivity, we've saved clients six figures by doing architecture work BEFORE anyone mentions the words "training run."
05When Fine-Tuning Actually Makes Sense
Fine-tuning has its place — proprietary data, measurable performance gaps, clear ROI. But it's step 10, and most teams try to start there.
The companies that get the most value from fine-tuning are the ones who've already done steps 1-9. They have clean data, solid evals, cost controls, and a production system that works. Fine-tuning becomes the optimization layer, not the foundation.
If you're considering fine-tuning, ask yourself: Have I exhausted the architectural improvements first? If the answer is no, you're not ready. And that's okay — the architectural improvements will probably solve your problem anyway.