How to Install and Run Ollama Locally: A Step-by-Step Guide

By StackGuide · Updated October 30, 2026

This guide is based on the open-source project Ollama by Ollama, licensed under MIT.

Ollama is the simplest way to run a capable language model on your own machine: one install, one command to download a model, and one local server that your other tools can call. You do not need an API key for local models, and once a model is downloaded, your prompts do not need to leave your computer.

What Ollama is, and who it is for

Ollama has two parts that work together. A background server loads a model into memory and serves it, and a command-line tool called ollama talks to that server. On macOS and Windows there is also a desktop app with its own chat window. The server exposes a local REST API on port 11434, including an OpenAI-compatible endpoint, so chat apps, coding assistants, and scripts can use your local model without a hosted provider.

It suits anyone who wants to try local models, build an app against a local model, or keep prompts private: students, developers, and people setting up the model backend for another tool, such as DeepTutor or Langflow. It is not a chat app with a library of hosted models, and it is not a training tool. You pick a model from the Ollama library, download it, and run it.

One distinction matters for licensing: Ollama itself is MIT-licensed, but each model has its own licence and terms. A model being available through Ollama does not make it MIT-licensed. Check the model page in the library before you use a model commercially.

Prerequisites

• A supported operating system: macOS, Windows, or Linux. The download page lists the current OS requirements, so check it if you are on an older release.
• Disk space for models: models are gigabytes each. As an example, the library lists Llama 3.2 1B at about 1.3 GB and Llama 3.2 3B at about 2.0 GB; larger models are much bigger. Leave room for more than one.
• Memory for the model you choose: Ollama itself publishes no fixed hardware minimum beyond a supported OS. The practical limit is the model: a small model runs on a modest laptop, while a large model needs plenty of RAM, or VRAM on a GPU machine, to run at a usable speed. Start small and go bigger only if the small model is too weak.
• A GPU is optional: Ollama uses a GPU when one is available (NVIDIA, AMD, and Apple Silicon are supported) and falls back to CPU otherwise. CPU-only runs work, but they are slower.
• Nothing else to install for the basic path: no Python, no Docker, and no API key are needed for local models.

Step 1: Install Ollama

Pick the path for your system. If you prefer to inspect a script before running it, download it first and read it; the commands below are the ones the official download page publishes.

Linux. Run the official install script:

curl -fsSL https://ollama.com/install.sh | sh

On Linux the installer also sets Ollama up as a service, so the server starts automatically. If the server is not running, start it by hand with ollama serve in a terminal and leave that terminal open.

macOS. Download the app from the official download page, or install with Homebrew:

brew install ollama

The macOS app installs the ollama command-line tool and starts the server in the background.

Windows. Download the installer from the official download page, or run this in PowerShell:

irm https://ollama.com/install.ps1 | iex

Verify the install in a new terminal:

ollama --version
ollama list

An empty list on a fresh install is expected. It confirms the command works and the server is responding; it does not mean anything is broken.

Step 2: Run your first model

Start with a small model. This command downloads the model if you do not have it yet, then opens an interactive chat:

ollama run llama3.2:3b

Type a prompt, press Enter, and read the reply. Type /bye to leave the chat, or press Ctrl+D. The model stays downloaded, so the next run starts much faster.

To download a model without chatting with it yet, use:

ollama pull llama3.2:3b

To see what you have installed, what is currently loaded, and details of one model, use:

ollama list
ollama ps
ollama show llama3.2:3b

When you are done with a loaded model, or want to remove one entirely, use:

ollama stop llama3.2:3b
ollama rm llama3.2:3b

Stopping unloads the model from memory; removing deletes the downloaded files, so you would have to download it again.

Browse the full catalogue at the Ollama library before picking a second model. Model names include a tag after the colon, such as :3b or :1b, and the tag matters: asking for a name or tag that does not exist gives a "model not found" error, not a similar model. Vision models, embedding models, and code models are listed separately in the library, so pick the kind that matches what you want to do.

Step 3: Use the local API from your own tools

Ollama's API runs at http://localhost:11434 by default. A chat request looks like this:

curl http://localhost:11434/api/chat -d '{
  "model": "llama3.2:3b",
  "messages": [{ "role": "user", "content": "Explain Ollama in one sentence." }],
  "stream": false
}'

The answer comes back as JSON. There is also an OpenAI-compatible endpoint at http://localhost:11434/v1, so tools built for the OpenAI SDK can usually point at Ollama by changing only the base URL and the model name. Such tools often still ask for an API key field; any placeholder text works for a local server, because Ollama does not check it locally.

That convenience is also the main security warning in this guide. By default, Ollama listens only on your own computer (127.0.0.1) and its local API has no login. Do not casually set OLLAMA_HOST to 0.0.0.0 to reach it from other devices: that exposes an unauthenticated API to your whole network, and to the internet if your router forwards the port. Anyone who can reach it could run models on your machine or delete them. If you genuinely need remote access, reach it through an SSH tunnel or a reverse proxy with authentication, and keep the port itself off the public internet.

Alternative path: Docker

Docker is useful on a server or when you want Ollama isolated from the host. This runs the server in the background, keeps downloaded models in a named volume, and publishes the API port:

docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

Then run models from the host with ollama only if you also have the CLI installed, or use docker exec -it ollama ollama run llama3.2:3b. On an NVIDIA host with the container toolkit installed, add the GPU flag to the docker run command so the container can see the GPU; without it, Docker Ollama runs on CPU. Inside a container, localhost means the container, not your host, so a second container should reach a host Ollama through the host gateway rather than localhost.

Make a custom model with a Modelfile

A Modelfile is a small text file that customises a base model: a system prompt, parameters, and other settings. Create a file called Modelfile:

FROM llama3.2:3b
SYSTEM "You are a concise assistant. Answer in short bullet points."
PARAMETER temperature 0.7

Then create and run it:

ollama create my-assistant -f Modelfile
ollama run my-assistant

Your new model appears in ollama list alongside the base model. This is the same mechanism other tools use when they offer a "custom model" option, and it keeps the customisation on your machine.

Common errors and fixes

FAQ

Do I need an API key or an account? No, not for local models. You need an account only if you choose to use Ollama's optional hosted cloud models, which are a separate, paid service from running models locally.

Are my prompts sent to Ollama's servers? No, not for a local model. The model runs on your machine, so local prompts stay local. If you use a hosted model or a hosted tool in front of Ollama, that provider's terms apply instead.

Can Ollama run without an internet connection? Yes, once the model is downloaded. The first download needs the internet; chatting with a downloaded model does not.

Which model should I start with? Start with a small general model such as Llama 3.2 3B, learn the commands and the API, and only then try a larger, vision, embedding, or code model for a specific job. Bigger is not automatically better for a small task.

How do I update Ollama? Install the new release over the existing install using the same method you installed with, and check the release notes for the version. Updating the app does not normally require re-downloading your models, but check ollama list afterwards.

What to try next

Once a small model chats well, point a second tool at it: connect it as the model backend in DeepTutor or Langflow, call it from a short script through the API, and then create one Modelfile custom model for a job you repeat. That sequence tests the three things Ollama is actually for: running a model, serving it locally, and reusing it everywhere else.

This guide was written with AI assistance and checked against the official documentation as of October 30, 2026.

Sources

• https://github.com/ollama/ollama
• https://github.com/ollama/ollama/blob/main/LICENSE
• https://ollama.com
• https://ollama.com/library