How to Run Ollama on VPS India — Host AI Models Without GPU

Ollama makes it possible to run large language models like Llama 3, Mistral, and Phi-3 entirely on your own server — no API fees, no rate limits, no data leaving your infrastructure. The catch is that most laptops don’t have the RAM or uptime to run a model 24/7 for an app or API. That’s where a VPS comes in.

This guide covers exactly which Ollama models fit which VPS plan, the install commands, and how to expose Ollama as an API for your own apps.

Why Run Ollama on a VPS Instead of Your Local Machine

Running Ollama locally works for testing, but it stops working the moment you need the model available 24/7, accessible from multiple devices, or callable from a production app. A laptop sleeps, reboots, and loses its IP address. A VPS in India stays online continuously, gets a static IP, and sits close to users in Delhi NCR and across India for low latency.

Running Ollama on a GigaNodes VPS India also means you control your data completely — prompts and outputs never leave your own server, which matters for businesses handling sensitive customer data under India’s data protection rules.

Ollama Model RAM Requirements vs GigaNodes Plans

Step by Step — Installing Ollama on Your VPS

These steps work on any Ubuntu 24.04 VPS, including every GigaNodes plan.

1. Connect to your VPS via SSH

2. Install Ollama

3. Start the Ollama service

4. Pull a model

5. Run the model

6. Optional — add a web UI with Docker

Open WebUI gives you a ChatGPT-style interface for your self-hosted models.

Using the Ollama API

Once running, Ollama exposes a local REST API on port 11434 that any app can call.

By default Ollama listens on localhost only. To call the API from another machine, set OLLAMA_HOST=0.0.0.0 in the systemd environment file at /etc/systemd/system/ollama.service.d/override.conf, then restart the service. Put Nginx with SSL in front of it before exposing it publicly.

Performance — AMD EPYC CPU vs GPU

CPU inference on AMD EPYC 7C13 is slower than a dedicated GPU but is fully usable for personal assistants, internal tools, and low-traffic APIs. Expect roughly 5-12 tokens per second on a 7B model on Cloud S — fast enough for chat-style interaction, slower than instant for long generations. For production workloads serving many concurrent users, a GPU-backed setup is recommended; for personal use, side projects, and small team tools, CPU inference on a GigaNodes VPS India is more cost-effective than renting GPU instances.

Frequently Asked Questions