Local or cloud AI: which should you use for work?
Neaptide · September 28, 2026 · 12 min read · Updated: October 4, 2026
Compare local and cloud AI for privacy, quality, speed and cost. Includes a worked cost example, two diagrams and a worksheet for testing your own tasks.
On this page
Local AI makes an appealing promise: your documents stay on your computer, the model works without the internet, and each request does not come with a bill. But a model that costs nothing to run can still produce expensive output. Spend long enough correcting its answers and the subscription you saved on can be gone within a few working days.
The short answer: a local language model is useful when processing on your own device or working offline is a requirement, and its quality is good enough for the task. Choose a cloud service when its model and tools produce a better working result and its data handling terms are acceptable. Combining the two makes sense when you have defined which materials may leave your infrastructure, and at what stage.
This guide compares assistants for text, documents and code using product documentation checked on 3 October 2026. We have not run a comparative model test. The cost section separates published inputs from hardware, energy and labour that you must measure in your own setting.
What exactly is local?
In this guide, local AI means a model whose computations run on your computer or a server you manage. Cloud AI means a model run for you by an external provider. The application window can look identical in either case.
A laptop running a model and a server in your office are not the same setup. The laptop, once prepared, can work offline. The server needs a network connection, though a whole team can share it. Renting a server and installing a model yourself is another arrangement: you control the software, while the provider owns the physical infrastructure. The single word “local” covers all of these boundaries, and covers them badly.
LM Studio illustrates the distinction. Its documentation says that chatting with a downloaded model and processing documents can work offline. Yet cloud models and web search in the same desktop application send requests beyond the device. So check the mode you have selected, and the workflow from end to end.
A comparison across eight practical criteria
In this table, the local option is a model on your own computer. A shared team server also requires network access, user administration and management of the combined workload.
| Criterion | Local model | Cloud service |
|---|---|---|
| Data | Processing can stay away from external providers if every stage remains on the device | The provider processes the request; retention, training and access depend on the product and its terms |
| Quality | Limited by the available model and configuration; test it on your tasks | You can choose models that would not fit on your computer; quality still needs testing |
| Offline use | Possible after downloading the required components; network tools are separate | Remote processing requires access to the service |
| Speed | Depends on hardware, model, input length and other workloads on the computer | Depends on the model, network, queues and service limits |
| Costs | Hardware, energy, setup, maintenance and answer review | Subscription or usage fees, integration, administration and answer review |
| Maintenance | You select and update models, the application and its environment | The provider maintains the model infrastructure; workflows and access remain your responsibility |
| Growing demand | Additional users share finite resources; growth may require new hardware | Expansion depends on quotas, plans and the provider’s available capacity |
| Version control | You can keep specific model files and their environment | The provider determines which versions remain available and for how long |
The table is there to help you define requirements. It cannot tell you which option is faster, cheaper or more accurate on your documents.
Where local AI is useful
Processing can stay within your infrastructure
If your team’s rules prohibit sending work materials to an external service, local processing gets the task done without that transfer: drafting a summary of internal notes, say, or categorising incoming requests. Both the model and the components reading the source files must run locally.
Control takes work. You have to restrict access to the computer and its folders, check backups and keep software updated. A model stored on your drive does nothing to protect a document from another user, malware or misconfigured synchronisation.
You can work without an internet connection
While travelling or dealing with an unreliable connection, a local assistant can continue editing text and answering questions about available materials. Prepare in advance by downloading the model, runtime components and documents. Web search, remote knowledge bases and other network dependencies will be unavailable offline. LM Studio’s documentation describes this distinction between local functions and downloading components.
You can retain a tested configuration
For a repeated operation, it can help to keep a particular model, its settings and the application. Processing similar records reliably may matter more than having the newest model every week. Keeping the files does not guarantee an identical answer on every run, though, so test behaviour together with the generation settings and the runtime environment.
Cloud models can have versions too, but the provider decides how long each one stays available. Ollama, for one, warns about retirements of certain cloud models and notes that local models already downloaded are not affected.
The trade-offs of running a model locally
The first constraint is quality on the specific task. A model that fits comfortably on a laptop may struggle with complex code, a long document or ambiguous instructions. Treat that as a hypothesis to test. It is no reason to dismiss every small model, since a well-defined task may need less capability than an open-ended conversation.
The second constraint is memory and workload. The size of the model file is only part of the memory needed in use. Context processing, working buffers and your other applications take up space as well. For language models, one consumer is the KV cache; its memory requirements depend on the architecture and storage strategy. Hugging Face cache documentation. See our guide to RAM, VRAM and context for a detailed explanation and calculations.
Hence a rule for buying hardware: an AI PC label and a TOPS figure do not replace a test of your chosen model in your chosen application. AMD’s documentation, for example, describes separate CPU, GPU and NPU execution paths using different software components. Having an NPU does not mean every application will automatically use it.
The third constraint is your time. Even a convenient application does not eliminate model selection, licence checks, updates or troubleshooting. For a personal experiment, that can be part of the enjoyment. For a team workflow, someone needs to own the work. If nobody does, recalculate the expected savings.
What the cloud offers, and where it falls short
A cloud service lets you start without buying hardware for the model. It provides access to models and tools you cannot or do not want to maintain yourself. How much that is worth depends on the task. Being able to upload a file or invoke a tool says nothing about whether the resulting answer is correct.
The trade-off is dependence on service access, limits and changes. A subscription does not necessarily provide unlimited use. An API requires spending controls. If a model changes or a required version disappears, you will need to validate the workflow again.
For a small number of varied tasks, the cloud is often the convenient place to start, because you can test the value before investing in hardware. This is about the order of doing things. We are not claiming that cloud AI is always cheaper. With sustained demand and suitable hardware already available, the calculation may look different.
Privacy: four separate questions
“Cloud AI trains on everything you upload” is too broad a claim. Anthropic, for example, says it does not use inputs and outputs from its commercial products, including Claude for Work and the API, for training by default. Feedback or separate consent can change those conditions. Consumer products have different rules.
An exclusion from training does not, by itself, tell you how long data is stored. Before deciding where to process it, ask four questions:
- Who processes the data? Only your machine, your server, or an external provider and its subcontractors?
- What is retained, and for how long? Source files, conversation history, request logs and backups are different objects.
- Can the material be used for training? Check your particular product, plan and feedback settings.
- Which additional services receive the content? Web search, scan recognition, external tools and synchronisation may have their own terms.
Even within one provider, retention can differ between API requests, uploaded files and the chat interface. No-retention arrangements may cover only particular features and agreements.
A useful local check is to open a document, ask a question and get an answer with networking disabled. This demonstrates that the tested workflow works offline. It does not prove that nothing will be transmitted when connectivity returns. Ollama, for instance, lets you disable its cloud features with OLLAMA_NO_CLOUD=1 and an application restart, but that setting does not control other programs on the computer.
What does a usable result cost?
Compare the same workload and include human time. A basic monthly formula is: a share of upfront investment + service usage + additional energy + maintenance + reviewing and correcting answers. Costs that are identical for both options can be left out, as long as you say so.
Public sources establish some inputs: Claude Sonnet 5.5’s base API costs $2/$10 per million input/output tokens; Qwen2.5-7B-Instruct Q4_K_M files total 4.683 GB. Checked on 3 October 2026. Download size does not establish hardware cost or runtime memory needs.
| Item | Local | Cloud |
|---|---|---|
| Model or API | Qwen2.5-7B-Instruct Q4_K_M: 4.683 GB of files | Sonnet 5.5: $2/$10 per million input/output tokens |
| Hardware | Actual additional purchase and useful life | Access devices and other required resources |
| Energy | Measured additional kWh × your tariff | Measurable additional use in your infrastructure |
| Setup and maintenance | Recorded hours × agreed rate | Recorded hours × agreed rate |
| Result review | Time, rejections and retries from a task log | Same accounting rules on the same tasks |
This table does not compare Qwen and Claude quality or calculate a monthly bill. That requires the model, workload and observed pilot results. Do not count existing hardware as a new purchase; record upgrades needed specifically for the model separately.
Additional monthly review cost = task count × extra minutes / 60 × hourly rate. Use measured minutes. Neither file size nor token price determines which option will need fewer corrections.
A good figure to end on is cost per accepted result: divide all costs for the compared workload, including failed attempts and corrections, by the number of results that pass your checks. If no results are accepted, there is nothing to divide by, and the process is not yet doing the job.
Test both options before buying hardware
For an initial pilot, you could use 12 representative tasks: four for extracting facts, four for writing or editing, and four for reasoning or code. It is a convenient starting set and makes no claim to statistical validity. Adjust the proportions to your work and use only materials that both options are permitted to process.
Straightforward questions alone are not enough. Add a document with a contradiction, a question the source cannot answer, and a task where an error is easy to miss. If the model extracts a delivery lead time, write down the correct value and its source first. If it edits an email, list the facts it must preserve. For code, prepare a check of the expected behaviour.
- Record the conditions. Note the model name and version, local configuration and quantisation, application, available context, tools and mode. For the cloud, record the plan or API, selected mode and date.
- Provide the same source materials. When comparing models, match access to search and tools. When comparing complete workflows, retain useful differences and describe them explicitly.
- Measure waiting time. Distinguish the first request, when the model has not yet loaded, from subsequent requests. Repeat the scenario several times; one fast run says little about a working day.
- Apply criteria defined in advance. Record critical errors, acceptance or rejection, and minutes spent correcting the output. Attractive formatting should not conceal an incorrect fact.
- Calculate costs and test a demanding condition. Try a long document while your usual applications are open, or a shared server with several users. Test offline use separately if you need it.
Tokens per second are useful for technical diagnosis, but do not measure time to a usable result. Ollama’s API, for example, reports separate durations for loading the model, processing the input and generating output. Human review and total task time you have to measure yourself.
Our local and cloud AI comparison worksheet includes pilot conditions, a task card and a results log. It is blank, with no invented model scores. For your first local setup, see the Ollama guide.
When combining local and cloud AI helps
A mixed workflow is useful when its stages have different data and quality requirements. For example, you might process an internal report locally. A person then prepares a brief approved for external processing, and a cloud model helps draft the public-facing text. The final output is checked against the source facts.
The hard part is what goes into that intermediate brief. Removing names is not enough if confidential figures, deal terms or other sensitive information remain. Transfer only material that is actually permitted to be processed externally. Otherwise, keep the task within approved infrastructure or have a person complete it.
Combining approaches has costs of its own: two sets of tools, handovers between them and review of intermediate materials. If one approach already does the job well, a second one added for the sake of the architecture is only overhead.
Where to start
Start with a local model when offline operation or processing inside your infrastructure is mandatory. First test the task on available hardware. If quality is insufficient, decide whether to change the model, process or equipment. A restriction on data does not turn a weak answer into a usable one.
Start with a cloud service when external processing is acceptable, tasks are varied and you want to test the tool’s value quickly. Compare the outcome with your current method and set a clear spending limit.
Combine the two when you have a concrete boundary: which sources stay inside, what may be transferred and who checks the handover. Consider buying an “AI computer” after making that decision, once the model, workload and output requirements are clear.