How to rent cloud GPUs cheaply for AI (2026 guide)
Disclosure: This guide contains referral and affiliate links (marked “sponsored”). They cost you nothing extra, and I’m not affiliated with these companies. Where a link gives you no perk, I say so.
Renting a GPU is usually cheaper than buying one, but only if you rent the right kind for your job and stop paying the moment you’re done. This guide walks through the three common ways to get GPU power for AI work, how to size what you need, and the habits that keep the bill small.
First: do you even need a GPU?
Plenty of AI projects never need a rented GPU. If you just want to call a popular open model from code, a pay-per-use model API is often simpler, because you pay per request and nothing sits idle. Rent a GPU when you need something an API won’t give you:
- Fine-tuning or training on your own data.
- Running a model or a custom setup that no API offers.
- Long batch jobs where you’d rather pay by the hour than by the token.
- Full control over the software stack, drivers and model files.
Size the job before you rent
The number that matters most is GPU memory (VRAM). If the model doesn’t fit, nothing else matters. A rough rule of thumb for running a model:
| Precision | Approx. memory for the weights | Example: 7B-parameter model |
|---|---|---|
| 16-bit (FP16/BF16) | ~2 bytes per parameter | ~14 GB |
| 8-bit | ~1 byte per parameter | ~7 GB |
| 4-bit | ~0.5 bytes per parameter | ~3.5–4 GB |
That’s weights only. Leave headroom for context length, batch size and framework overhead, and expect fine-tuning to need much more than inference. Quantizing (running at 8-bit or 4-bit) is the single biggest lever for fitting a model on a smaller, cheaper card.
Three ways to get GPU power
| Option | Good for | Watch out for |
|---|---|---|
| GPU marketplace (rent a machine from many hosts) | Low hourly prices, wide choice of cards, short experiments, fine-tuning | Hosts vary in reliability; storage and bandwidth can cost extra; you manage the setup |
| Model API (pay per request) | Calling open models from an app, spiky or low traffic, no setup | Limited to the models and settings offered; heavy steady use can cost more than renting |
| Plain cloud server (CPU VM) | Hosting the app, bot or database around your model | Not a GPU; don’t try to run big models on it |
Where to rent (links)
Vast.ai: GPU marketplace
- What it is
- A marketplace where many hosts list GPUs for rent by the hour, usually priced below the big clouds.
- Best for
- Fine-tuning, running open-weight models, batch jobs and experiments where price matters more than enterprise guarantees.
- Tips
- Compare the full hourly price including storage. Check each listing’s reliability info before long jobs. Save your work off the machine so you can walk away from a host.
- Skip it if
- You need enterprise SLAs or a fully managed service.
- Your perk
- None for you. This is a plain referral link.
Novita AI: model APIs plus GPU cloud
- What it is
- An API platform for popular open models, plus GPU cloud instances.
- Best for
- Developers who want to call open models from code without hosting them, then move to a GPU instance if they outgrow the API.
- Skip it if
- You’re happy with a single provider’s API already.
- Your perk
- None for you. This is a plain affiliate link.
Nous Portal: skip the GPU entirely
- What it is
- One account for many AI models, with a free tier and a paid plan with higher limits for agents and batch jobs.
- Best for
- Prompting and agent work across several models when you don’t need your own hardware at all.
- Skip it if
- You only use one model in one app.
- Your perk
- $15 off your first month on the $20 plan (new accounts only).
For the app around your model
Your model usually needs an API, a bot, a database or a small website next to it. That part doesn’t need a GPU, and putting it on a cheap server keeps your expensive GPU time for the actual AI work.
Railway: deploy the app from your repo
- Best for
- Hosting the API, bot or front end that talks to your model, without managing servers.
- Your perk
- $20 in Railway credits when you sign up.
DigitalOcean: plain, predictable servers
- Best for
- A simple Linux server for a bot, a database or a small site.
- Honest note
- The referral credit is the same standard credit you’d get without a link.
- Your perk
- Standard $5 credit for 90 days, the same as signing up without a link. Payment method required.
Habits that keep the bill small
- Shut it down. Idle GPUs are where the money goes. Stop or destroy instances as soon as a job ends, and check that storage isn’t still billing.
- Prototype small. Debug your script on a cheap card or a tiny model, then rent the big card only for the real run.
- Quantize first. If a 4-bit version of a model is good enough for your task, it may fit on a card that costs a fraction as much.
- Checkpoint often. Save progress to storage you control, so a dropped or interruptible machine costs minutes, not the whole run.
- Use an API for spiky traffic. If requests come in bursts, pay per request instead of keeping a GPU warm.
- Set a budget alert or a prepaid limit wherever the provider offers one.
Pre-rental checklist
- How much VRAM does my model need at the precision I’ll use?
- Is the price per hour all-in, or are storage and bandwidth extra?
- How will I get my data and results on and off the machine?
- What happens to my work if the machine stops?
- Have I set a reminder or script to shut it down?
Prices, hardware and perks change often. Check each provider’s current pricing page before you rent. More picks are on the deals page.