Every few weeks a founder asks me some version of the same question: “Should we be running our own models instead of paying for API calls?”
The question usually arrives after their first serious OpenAI or Anthropic invoice, and it usually comes loaded with a wrong assumption — that this is a technology decision. It isn’t. It’s a unit-economics and risk decision, which means it’s an operator’s decision. And after 10 years in startup operations and a couple of years building Dszape solo with AI, I’ve ended up with a framework I actually use, not one I read in a whitepaper.
The short version: this is not local versus cloud. Almost every serious builder I know, myself included, runs both. The real question is which workloads go where. Here’s how I decide.
The Four Questions That Decide It
Strip away the benchmarks and the Twitter discourse, and the local-vs-cloud decision comes down to four questions:
- Cost shape: is this workload high-volume and repeated, or occasional and spiky?
- Privacy and residency: can this data legally and ethically leave your machine or your country?
- Latency and availability: does this need to work instantly, offline, or when a provider has an outage?
- Quality tier: does this task need the smartest model on Earth, or just a competent one?
Score any workload against those four and the answer usually falls out on its own. Let’s take them in turn.
Cost: Per-Token Pricing vs Hardware You Already Own
Cloud APIs price per token. That’s beautiful when your volume is low — you pay pennies and someone else runs the data centre. It gets uncomfortable in exactly two situations: high-volume repeated workloads, and high-iteration creative loops.
A local model inverts the cost structure. The hardware is a fixed cost (often a laptop you already own) and every token after that is free. My MacBook Pro (M5 Max, 36 GB) runs 30B-class models comfortably, and once a model is on disk, running it ten thousand times costs the same as running it once.
The operator’s way to think about it: cloud is OPEX, local is CAPEX you’ve probably already spent. So the question is never “which is cheaper?” in the abstract. It’s “what does this specific workload cost per month on each path?”
A concrete example from my own work: I built a local AI vocal mastering pipeline for my music (autotune, processing chain, loudness — fully offline). Mastering means re-rendering dozens of times per track. Against a paid API, I would ration my iterations to control the bill, and rationed iteration is where creative work goes to die. Locally, the marginal cost of one more attempt is zero, so I experiment freely.
Meanwhile my Claude usage for coding is easily my biggest tool spend, and I pay it without blinking, because the same maths runs the other way: one prevented production incident pays for months of API calls.
The trap to avoid: don’t buy hardware to “save money” on a workload you haven’t measured. If you’re doing a few thousand tokens a day, your cloud bill is lunch money and local buys you nothing but sysadmin work.
Privacy and Data Residency: The DPDP Angle
This is the factor that’s underweighted in most US-centric writing on this topic, and for Indian startups it’s becoming the deciding one.
India’s Digital Personal Data Protection (DPDP) Act changes the calculus for any startup handling personal data. When you send customer data to a cloud LLM API, you’re transferring personal data to a third-party processor, often outside India. That’s not necessarily prohibited, but it now comes with real obligations: knowing where the data goes, what the processor does with it, and being answerable for it. “We pasted it into an AI tool” is not a data-protection policy.
A local model removes the entire category of question. Data processed on your own machine never becomes a third-party transfer. Nothing to disclose, no processor agreement to review, no residency debate.
My practical tiers:
- Customer personal data (guest records, in my case — Dszape is hotel software, so think names, phone numbers, ID documents): most protected tier. If AI needs to touch it, prefer local processing, or at minimum a cloud provider whose data handling terms you have actually read, with data minimised before it’s sent.
- Your own private material (strategy docs, financials, unreleased work): local by default. Not because cloud providers are villains, but because “it never left the machine” is the one privacy guarantee that needs no trust.
- Code and generic business text with no personal data in it: cloud is fine, and the quality gain is usually worth it.
If you’re an Indian founder and you haven’t mapped which of your AI workloads touch personal data, do it this week. It’s an afternoon of work now versus a very bad quarter later.
Latency, Availability and the Offline Question
Cloud APIs have two failure modes people forget until they hit them: round-trip latency and outages.
For chat-style use neither matters much. But the moment AI is inside a product loop or a daily workflow, they start to bite. My local TTS voice (Voxtral-4B, running via my always-on assistant setup) responds instantly and works with the Wi-Fi off. A cloud TTS call would add a round trip to every single utterance and go silent whenever the provider or my connection hiccups.
The rule I use: if the workload runs constantly in the background of your life or your product, latency and availability weigh heavily, and that favours local. If it’s a discrete task you invoke a few times a day, they barely matter.
Quality Tiers: Being Honest About the Gap
Here’s where I’ll disappoint the local-LLM romantics. There is a real quality gap between the best open models you can run on a laptop and the frontier cloud models, and it shows up precisely where the stakes are highest.
The way I think about it, in plain terms:
- Frontier cloud models (Claude and peers): the smartest tier. Long context, strong reasoning, reliable on complex multi-step work.
- Big open models on serious hardware (70B+): a tier below the frontier. Genuinely capable.
- The local laptop class (7B to 30B): competent. Comparable to cloud models from a year or two back. Excellent at drafting, summarising, classification, TTS, and creative iteration. Not what you want architecting your database.
The test I apply: what does a wrong answer cost? If a wrong answer costs you a production incident, a compliance problem, or a customer — pay for the frontier. If a wrong answer costs you one more iteration — local is free iterations forever.
I learned the expensive half of this on Dszape. My platform has real production-incident history — silently dropped database errors, billing writes that failed for weeks — and every one of those taught me that code quality is not the place to economise. (The full failure log is in my solo building journey.) The AI writing my billing engine will be the best model available, every time.
My Actual Split, as a Working Example
So that the framework isn’t abstract, here’s exactly how it resolves for me:
Cloud (frontier models, mostly Claude):
- All coding agents working on Dszape — 245+ API routes, 180+ SQL migrations, a billing engine. Maximum stakes, maximum model.
- Architecture and design reasoning, complex debugging.
- Anything long-context: whole-codebase work, long documents.
Local (Ollama on the MacBook):
- TTS voice for my assistant — constant, repeated, quality bar is “consistent and good”.
- Vocal mastering pipeline for my music — high-iteration creative work where free retries beat marginal quality.
- Drafting and private text work — my notes, early thinking, personal data.
- Fine-tuning experiments, like the model I’m training on my own writing (more on why in Fine-Tuning vs RAG vs Prompting).
Notice the pattern: cloud gets the high-stakes, occasional work; local gets the low-stakes, constant work. Stakes and volume, not ideology.
If you want the hands-on side of the local half — what fits in what RAM, quantisation, the Ollama setup — I wrote that up separately in Running Local LLMs on a MacBook.
A Decision Checklist for Your Startup
Run each AI workload you have (or plan) through this, in order:
- Does it touch personal data covered by DPDP or your customers’ trust? If yes and you can’t minimise the data, weight heavily towards local or a contractually solid cloud setup.
- What does a wrong answer cost? High cost of error → frontier cloud model. Low cost of error → keep going.
- Is it high-volume or high-iteration? If you’d run it hundreds of times a day, price it out on both paths. Local often wins on pure volume.
- Does it need to be instant or offline? If yes, local.
- None of the above decisive? Default to cloud. It’s less operational surface area, and as a startup your scarcest resource is attention, not tokens.
That last point deserves emphasis, because it’s the anti-hype part: the default for most startups should still be cloud APIs. Local models are the special case you graduate specific workloads into, once volume, privacy, or latency makes the case. Running your own models is infrastructure, and infrastructure you don’t need is just distraction with a fan noise. The same logic applies to AI adoption generally — I’ve written about when to build versus buy in Build vs Buy: When to Use AI in Your Startup.
FAQ
Is a local LLM cheaper than a cloud API?
Only at volume. A local model has near-zero marginal cost but requires capable hardware and your time. If your usage is a few thousand tokens a day, cloud APIs cost pennies and are cheaper in practice. If a workload runs constantly — TTS, batch processing, high-iteration creative loops — local usually wins.
Are local LLMs compliant with India’s DPDP Act by default?
Local processing avoids the third-party data transfer question entirely, which removes a large compliance surface. But DPDP compliance is about your whole data practice — collection, consent, storage, deletion — not just where inference runs. Local inference helps; it doesn’t absolve.
Can a startup run its production product on local/self-hosted models?
Some do, usually for narrow, well-defined tasks (classification, extraction, moderation) where a small fine-tuned model matches frontier quality at a fraction of the cost. For broad reasoning tasks in production, the frontier cloud models are still meaningfully better, and most startups should not take on model-hosting operations early.
What do you personally use for coding?
Frontier cloud models, exclusively — Claude, running as coding agents inside my repo. My production platform handles hotel bookings and billing; the cost of a wrong answer there is a real incident affecting real money. Local models never touch production code in my workflow.
What hardware do I need to start experimenting locally?
Any Apple Silicon Mac with 16 GB+ RAM can run useful 7B to 13B models. 32 to 36 GB unlocks the 30B class, which is where quality gets genuinely impressive. Start with Ollama and one model before buying anything.
I write from the trenches about building SaaS solo with AI — the economics, the incidents, the tools that survive contact with production. If this framework helped, you might like my complete AI journey or my 2026 AI ops stack, and you can read more about me here.
