Do You Need a Local Model for Privacy?

“I care about privacy, so I should run a model locally” is a reasonable instinct that’s usually an overcorrection. Running a model on your own machine is the only categorical privacy answer — nothing leaves — but it costs you capability, convenience, and a permanent bit of maintenance. Most privacy concerns don’t need a categorical answer. Some absolutely do.

This post is about telling the two apart. It’s a decision, not a setup guide.

Sort your constraint into one of four kinds

Kind 1: A vague preference for privacy. You don’t like the idea of your conversations being training data. Nothing you paste would hurt you if it leaked.

What you need: check the training setting in your account and move on. Possibly a paid tier with a no-training commitment. Local is a large amount of effort to solve a small amount of unease, and the honest advice is to spend the effort elsewhere.

Kind 2: Confidential material with no formal rule attached. Client documents, unpublished writing, internal financials, medical questions about yourself. It would be embarrassing or damaging if it leaked, but no contract or regulation names it.

What you need: a business-tier account with a no-training commitment plus a redaction habit. Redaction is dramatically underrated: most tasks need the structure of a document, not the identities in it. Replace names, account numbers, and identifying specifics with placeholders and you’ve eliminated most of your actual exposure in about ninety seconds. See what AI assistant data terms actually say.

Kind 3: A formal obligation that a contract can satisfy. Regulated data, a client agreement requiring specific handling, a security review that needs a data-processing agreement and named subprocessors.

What you need: enterprise tiers, where residency commitments, retention controls, audit visibility and DPAs exist. This is procurement, not settings. Local models are usually worse here, because “it’s on my laptop” satisfies no auditor and produces no paper trail — self-hosting for compliance means running properly managed infrastructure, which is a bigger project than buying an enterprise seat.

Kind 4: A hard prohibition on third-party processing. A duty of confidentiality that admits no processors. A client who has said, in writing, no cloud AI. An air-gapped environment. Material whose disclosure is catastrophic and unremediable.

What you need: a model running on hardware you control. This is the case where local is not an overcorrection — it’s the only thing that works, because it’s the only option that changes the category of the risk rather than reducing its probability.

Most people who think they’re in kind 4 are in kind 2 or 3. It’s worth being honest about which, because the costs differ by an order of magnitude.

What local actually buys you

The categorical guarantee. No network, no vendor, no terms to read, no policy change to worry about, no subpoena that reaches your prompts. For genuine kind-4 constraints this is the entire point and nothing else substitutes.

Independence from policy drift. A vendor’s defaults can change; a file on your disk can’t. Related to lock-in — see how locked in are you.

Offline capability. Sometimes the real requirement: a plane, a boat, a secure facility, a bad connection.

What it costs you

Be clear-eyed, because these costs are why the recommendation isn’t “everyone should do this.”

Capability. As of mid-2026 the very best models are large hosted ones. Good open-weight models running on ordinary consumer hardware are entirely adequate for summarising, drafting, reformatting, answering questions about a document you supply, and a lot of everyday coding help. They are not the frontier, and on genuinely hard reasoning or long multi-step work you will notice. That gap has narrowed a lot and will narrow further; it is not zero.

Convenience. No polished mobile app, thinner document handling, less of the surrounding product — memory, projects, voice, search grounding — unless you assemble it. Your phone won’t have it.

Maintenance. Models and software update. Something will break at an inconvenient moment. It’s a small permanent tax that a subscription doesn’t have.

Your hardware. It needs to be capable enough, and while a reasonably modern machine with decent memory handles useful models, the specifics are their own subject. We deliberately stay at the decision level here; can I run a ChatGPT alternative locally covers what to expect before you invest a weekend.

The privacy mistake almost everyone makes

Two of them, actually.

First: “open weights” does not mean private. An open-weight model served to you by a hosted provider has exactly the same data posture as a closed one — your prompt goes over the network to a company. Privacy comes from where the model runs, not from the licence on its weights. People conflate these constantly. See what are open-weight models.

Second: the leak is usually not the model. People agonise over a vendor’s training policy while pasting client data into a free consumer account, granting an assistant broad read access to their entire document store, or discussing confidential matters in a shared team workspace an admin can read. The connectors and permissions you granted are a much larger real-world exposure than the training question, and they’re entirely under your control. Fix those first; they’re free.

The pattern that works for most people

You don’t have to choose one tool. As of mid-2026 the sensible arrangement for anyone with mixed material is:

• A hosted assistant for the bulk of work — the hard interactive thinking, where capability and convenience earn their keep. On a tier whose data terms you’ve actually read.
• A redaction habit for the merely-confidential, which handles more than people expect.
• A local model for the genuinely untouchable subset — the small pile of documents that must not leave. It doesn’t need to be your daily driver to be worth having, and it only has to be good enough for the specific tasks in that pile.

That combination gets you the categorical guarantee where you need it and full capability everywhere else, which is strictly better than forcing one tool to satisfy your strictest constraint. It’s also the answer we reach most often on this site: not one tool, but the right tool per class of data.

For the broader decision about which assistant to use at all, start from our framework.