Is AI Leaving the Cloud?

Thesis

For the last few years, the story of AI has been simple: you open a browser, go to ChatGPT, Claude, Gemini, Midjourney or one of the hundreds of AI services launching every week, and type something. The request travels to somebody else's data centre, somebody else's GPUs do the work, and a few seconds later the answer comes back. We've become used to thinking of AI as a service you rent.

That assumption is starting to break down. Models are shrinking, quantisation is improving fast enough to matter, and consumer AI silicon is now good enough that a growing share of everyday AI work can run entirely on the machine in front of you. This doesn't mean cloud AI is going away. Frontier models will keep needing frontier infrastructure. It means the architecture is becoming hybrid: local handles the routine, cloud handles the exceptional, and increasingly you won't notice or care which one answered.

Switching off the Wi-Fi

The easiest way to feel the shift is to imagine switching off your Wi-Fi. Not losing your connection temporarily, but actually switching it off. No browser, no account, no API key, no subscription, no usage meter. Then asking your laptop a genuinely sophisticated question and getting a good answer from a large language model running entirely on the machine.

That's no longer hypothetical. OpenAI's gpt-oss release in August 2025 included a 21-billion-parameter model built specifically to run on a single high-end consumer machine, alongside a larger 117B variant for serious hardware. The 20B model card puts it at roughly 16GB of memory thanks to native 4-bit quantisation of its mixture-of-experts layers, and Ollama and LM Studio both ship one-command installs for it. It isn't just chatbots either. There are now credible local tools for transcription, image generation and enhancement, coding agents, and music and video generation. The cloud hasn't disappeared, but the assumption that AI has to live there is starting to.

From renting software back to owning it

This has happened before, just in reverse. For decades, software lived on your computer: you bought Microsoft Office, installed Photoshop, owned the application and ran it locally. Then SaaS flipped the model: instead of buying Photoshop you subscribed to Creative Cloud, instead of installing software on your own server you paid Salesforce, instead of maintaining infrastructure you rented it from AWS.

AI arrived almost entirely inside that rented model: ChatGPT, Midjourney, Perplexity, Cursor, Suno and Runway are all services, not products you install. There were good reasons for that: frontier models are enormous, need expensive specialised hardware and constant retraining, so centralising the compute made obvious economic sense. But that calculus is shifting as models get smaller, quantisation gets better and consumer hardware gets faster. The model doesn't have to be enormous. It just has to be good enough for the job.

The good-enough revolution

We've become fixated on frontier benchmarks: which model is smartest, which has the longest context window, which is closest to AGI. But most of what people actually ask AI to do isn't frontier research. It's summarising a document, rewriting an email, transcribing a meeting, removing a background from a photo, classifying files, writing routine code, drafting a product description.

NVIDIA's research team made a version of this argument formally in 2025, arguing that small language models are the future of agentic AI: models under roughly 10B parameters are sufficiently capable, structurally better suited, and far cheaper to run for most agent sub-tasks, with large frontier models reserved for the genuinely hard, open-ended cases. For a large slice of daily AI use, you don't need the smartest model in the world. You need one that's good enough, fast enough, and running when you need it, and that changes the economics considerably.

Why now: the denominator is moving

There's an important reality check here: local AI isn't magic, and hardware still matters. A 3B or 7B model runs comfortably on a normal laptop; a 20B model needs considerably more memory; large-scale video generation is a different proposition entirely. Watching a demo of a big model running locally on a maxed-out machine and concluding "every laptop can run everything" is the wrong takeaway.

What's actually changed is the denominator. Three things are moving together:

• Quantisation. Representing model weights in 4-bit or 8-bit precision instead of 16- or 32-bit floating point cuts memory requirements dramatically (often by up to 4x) while retaining most of the model's capability.
• Unified memory. Apple's M-series chips let the CPU, GPU and neural accelerators share one pool of high-bandwidth memory instead of being bottlenecked by a discrete GPU's fixed VRAM, which is a big part of why Apple Silicon has become a genuine local-inference platform rather than just a laptop chip.
• Dedicated AI silicon becoming standard. Gartner expects AI PCs to reach 31% of the worldwide PC market by the end of 2025, with on-device NPUs shipping as standard rather than optional.

None of that means every laptop can run everything today. It means the same laptop that could barely run useful AI locally two years ago now can, and next year's will do more again.

Privacy becomes architecture, not a compliance problem

This is where local AI gets genuinely interesting for business rather than hobbyists. For years the answer to "can we use AI on sensitive data" has been governance: enterprise contracts, data-processing agreements, private cloud instances. The risk of getting that wrong is well documented: in 2023, Samsung engineers pasted proprietary source code and internal meeting notes into ChatGPT while debugging and taking notes, prompting the company to ban generative AI tools for staff entirely. JPMorgan restricted employee ChatGPT use the same year, largely over exactly this kind of exposure risk in a heavily regulated industry.

Local models offer a structurally different answer to the same problem: don't send the data anywhere. A solicitor analysing case documents, a financial adviser reviewing client information, a pharmaceutical company analysing research data, a board working through acquisition materials. If the processing happens on hardware the organisation already controls, the data doesn't have to leave that environment at all. That doesn't eliminate security risk; local systems can still be compromised or badly configured. But it moves privacy from something you negotiate with a vendor to something built into the system's architecture from the start.

When agents start running on your machine

A chatbot answers a question. An agent reads your files, writes code, runs commands, checks the results and tries again, often for many steps in a row. Run that loop in the cloud and every intermediate step, every file it touches, potentially leaves your machine.

Run it locally and none of it has to. Tools like Cline already support pointing a coding agent at a model served by Ollama on localhost, so the agent can read your repository, write and test code, and iterate without any of it crossing the network. It's not yet a like-for-like swap with frontier cloud models on hard problems, but for routine refactors, test-writing and boilerplate it's increasingly viable. It turns the AI from a website you visit into something closer to part of the machine itself.

What a $2,000-a-year AI habit buys you

Here's a concrete version of the shift. Right now I pay for ChatGPT, Claude, Cursor, Suno and Runway: a general chat assistant, a coding tool, a music generator, an image/video generator. Individually each subscription looks reasonable: $15 here, $20 there, another $30, another $50. Collectively it adds up fast, and for an increasing number of those use cases there's now a local or open alternative. Not necessarily as capable, not always as polished. But increasingly good enough to be useful.

Ollama makes running local models remarkably accessible, and its ecosystem already lets you plug local models into coding agents and other tools while keeping the option to switch to a cloud model when you actually need the extra capability. That's the important part: the future isn't local instead of cloud, it's local plus cloud, chosen per task rather than per subscription.

The hybrid future

The likely shape isn't local AI replacing cloud AI, it's local AI handling the routine and cloud AI handling the exceptional. A laptop running a small local model for summarisation, transcription, document search, simple coding and routine email, then calling out to a frontier cloud model only when a genuinely hard problem shows up. Much like a phone doesn't send every calculation to a supercomputer, and a laptop doesn't send every spreadsheet formula to AWS.

That creates a real product opportunity around orchestration rather than raw model size: software that understands what hardware you have, what's installed locally, how sensitive the data is, and how hard the task actually is, then routes accordingly. "This runs locally," "this needs the cloud," or "local for the first pass, cloud for verification." An AI routing layer, essentially, and potentially a valuable one.

Where this leaves the cloud

Cloud AI isn't dead, and there will always be tasks that need enormous models and enormous compute that no laptop will ever match. But the question is starting to change from "where is the AI?" to "where should this particular piece of intelligence run?" Sometimes that's still a frontier lab's data centre. Increasingly, sometimes it's the laptop already sitting in front of you, and the interesting part is that most people won't notice the transition happening. The assistant will just work, Wi-Fi on or off, local model or cloud model, whichever makes sense for the task.

AI is becoming a capability, not a destination

Today we mostly think of AI as places we go on the internet: ChatGPT, Claude, Gemini, Midjourney, Cursor. Tomorrow it may become something more fundamental: like a CPU, like storage, like an operating-system capability that's simply there, embedded in the device, available to applications, available to agents, available to us. Once that happens, the question stops being "which AI service should I subscribe to?" and starts being "what can my computer do now that it has a brain?"

AI isn't just getting smarter. It's getting local.