A local AI model for coding: your first run with Ollama

For your first local model, choose a short coding task whose answer you can check: explain a function, find a bug or suggest tests. Below, you will install Ollama, get a first response and try a Python function that cleans a list of names. Start with that repeatable exercise before deciding whether you need a larger model.

Ollama runs models and provides an interface for calling them. This guide uses the local Qwen2.5-Coder 1.5B variant as a small learning example. We are not recommending it as today's best model, and it promises nothing about results on large projects.

• Model card

Install Ollama and get a first answer

Follow the official installation instructions for your operating system, then start the app or service. In a terminal, run:

• official installation instructions

ollama --version
ollama run qwen2.5-coder:1.5b

The first run downloads the model, so you need an internet connection and free disk space. When the interactive prompt appears, ask a short coding question. Enter `/bye` to leave.

Do not judge speed from the first response alone, because loading the model and generating an answer are separate stages. Get a repeatable response to a small request first.

A learning task with a checkable result

Ask for a function with explicit requirements:

Write a Python function unique_names(names). It takes a list of strings, trims surrounding whitespace, and removes empty strings and case-insensitive duplicates. Preserve the order of first occurrences and their spelling after trimming. Return the code and three tests. Do not add dependencies.

Define expected results before evaluating the answer:

assert unique_names([" Anna ", "anna", "", "BOB"]) == ["Anna", "BOB"]
assert unique_names([]) == []
assert unique_names([" A ", "a", "B", "b"]) == ["A", "B"]

The checks above define the expected behaviour. They are not results from testing a model. Read the proposed code before running them in a test environment. If the function changes the order or the first name’s capitalization, point out that exact mismatch and ask for a correction.

This small exercise shows whether the model follows these particular instructions. It tells you nothing about whether the model can modify a large repository on its own.

Call the model from a program

Ollama exposes a local HTTP API. On macOS, Linux or WSL, a request can look like this:

curl http://localhost:11434/api/generate -d '{
  "model": "qwen2.5-coder:1.5b",
  "prompt": "Explain the difference between a Python list and a tuple.",
  "stream": false
}'

Here, `stream: false` requests one complete response instead of a stream of chunks. The fields are explained in the generate API documentation. This request checks that the API is reachable. It does not connect the model to every editor feature.

• generate API documentation

For editor integration, first check whether your chosen extension supports Ollama or a compatible local endpoint. Then specify the server address and the exact installed model name. Chat support does not imply support for every agentic coding tool.

What “local” means

Ollama states that it does not receive prompts or responses when a model runs locally. Cloud models follow a different processing path. Your editor, extensions and external tools may also contact their own services, so examine the whole setup.

• Ollama FAQ

Ollama has a local-only mode for working without its cloud features. Even so, do not describe your workflow as fully offline until you have checked dependency downloads, extension requests and tool behavior. Use sample code for the first trial.

If responses are too slow

Run `ollama ps` to see loaded models and how execution is split between CPU and GPU. That tells you where to look for the bottleneck.

• Check model placement

Change one condition at a time: use a smaller model, shorten the input or reduce the configured context. Close other resource-heavy applications and repeat the same task. Do not reach for the largest context window by default, since model weights are only part of memory use.

Record the model and tag, hardware, context, prompt, elapsed time and test results. “It answered quickly” says little about a configuration if nobody has checked the code.