A local AI model for coding: your first run with Ollama

A local model is easiest to try on a small piece of code: explain a function, suggest tests or find a bug. For a first experiment, a repeatable setup and a clear way to check the answer matter more than choosing the largest model.

Ollama runs models and provides an interface for calling them. This guide uses the local Qwen2.5-Coder 1.5B variant as a small learning example. It is neither a recommendation for today's best model nor a promise of reliable results on large projects. Model card.

• Model card

Install Ollama and get a first answer

Follow the official installation instructions for your operating system, then start the app or service. In a terminal, run:

• official installation instructions

ollama --version
ollama run qwen2.5-coder:1.5b

The first run downloads the model, so you need an internet connection and free disk space. When the interactive prompt appears, ask a short coding question. Enter `/bye` to leave.

Do not judge speed from the first response alone: loading the model and generating an answer are separate stages. First establish that a small request produces a repeatable response.

A learning task with a checkable result

Ask for a function with explicit requirements:

Write a Python function unique_names(names). It takes a list of strings, trims surrounding whitespace, and removes empty strings and case-insensitive duplicates. Preserve the order of first occurrences and their spelling after trimming. Return the code and three tests. Do not add dependencies.

Define expected results before evaluating the answer:

assert unique_names([" Anna ", "anna", "", "BOB"]) == ["Anna", "BOB"]
assert unique_names([]) == []
assert unique_names([" A ", "a", "B", "b"]) == ["A", "B"]

These are the author's task criteria, not measured model results. Read the proposed code before running the checks in a test environment. If it changes the order or the first string's capitalization, identify the exact mismatch and ask for a correction.

This small exercise shows whether the model follows these particular instructions. It does not establish that it can modify a large repository independently.

Call the model from a program

Ollama exposes a local HTTP API. On macOS, Linux or WSL, a request can look like this:

curl http://localhost:11434/api/generate -d '{
  "model": "qwen2.5-coder:1.5b",
  "prompt": "Explain the difference between a Python list and a tuple.",
  "stream": false
}'

Here, `stream: false` requests one complete response instead of a stream of chunks. The fields are explained in the generate API documentation. This request checks API access; it does not automatically connect the model to every editor feature.

• generate API documentation

For editor integration, first check whether your chosen extension supports Ollama or a compatible local endpoint. Then specify the server address and the exact installed model name. Chat support does not imply support for every agentic coding tool.

What “local” means

Ollama states that it does not receive prompts or responses when a model runs locally. Cloud models follow a different processing path. Your editor, extensions and external tools may also contact their own services, so examine the whole setup. Ollama FAQ.

• Ollama FAQ

Ollama has a local-only mode for workflows without its cloud features. However, do not describe your workflow as fully offline until you have checked dependency downloads, extension requests and tool behavior. Use sample code for the first trial.

If responses are too slow

Run `ollama ps` to see loaded models and how execution is split between CPU and GPU. That helps locate the bottleneck. Check model placement.

• Check model placement

Change one condition at a time: use a smaller model, shorten the input or reduce the configured context. Close other resource-heavy applications and repeat the same task. Do not automatically select the largest context window: model weights are only part of memory use.

Record the model and tag, hardware, context, prompt, elapsed time and test results. “It answered quickly” is not enough to select a useful configuration without checking the code.