Deployment 5 minReviewed September 1, 2026

How to run an open model with Ollama

The quickest path from a model name to a local API on macOS, Windows, or Linux.

The answer

Use Ollama when you want the shortest setup path for local inference and development. It packages model download, runtime selection, and a local API behind a small command surface.

Run a model

ollama run gemma4:12b

Call the local API

curl http://localhost:11434/api/chat \
  -d '{"model":"gemma4:12b","messages":[{"role":"user","content":"Explain tensor parallelism simply."}]}'

When to move beyond Ollama

Move to a production serving engine when you need explicit batching policy, multi-GPU tensor parallelism, predictable replica scaling, detailed metrics, or strict control over model revisions and quantization artifacts.