Artificial Intelligence chatbots are everywhere, but did you know you can actually run one directly on your own computer without sending any data into the cloud? In this guide, we explain how to set up a chatbot locally, using GPT-OSS as an example.
We keep things simple and beginner-friendly, making it easy for non-technical users. We also highlight why running AI locally is especially valuable for organisations focused on privacy and compliance, including teams who use Oregani to manage AI policies.
What does “run a chatbot locally” mean?
Running a chatbot locally means installing the software and model directly on your computer. Everything happens on your machine:
- Your prompts and responses never leave your device
- You decide when and how the AI runs
- It can even work without an internet connection once installed
This gives you full control and security while still benefiting from the power of AI.
Why run an AI chatbot locally?
Here are the biggest benefits for individuals and teams:
- Stronger privacy and compliance
Data never leaves your computer, making it easier to follow rules like GDPR or internal data guidelines. - No ongoing costs
You are not billed per use, as with online services. - Works offline
Useful for secure workplaces or remote areas.
For organisations, this makes local chatbots an excellent way to stay compliant while still exploring the benefits of AI. Oregani supports this by helping teams define which AI tools and setups are safe and approved.
Example: GPT-OSS via LM Studio
One of the easiest ways to get started is by using LM Studio. This is a simple desktop app that lets you download and run models like GPT-OSS. No coding is required.
Steps:
- Download LM Studio from lmstudio.ai
- Search for GPT-OSS in the app and download it
- Start chatting with your AI chatbot safely and privately
Minimum requirements for running locally
You do not need a supercomputer, but some compute is needed. Here is a friendly breakdown of the recommended specifications:
| Requirement | Minimum Specification | Recommended for Best Performance |
|---|---|---|
| Operating System | Windows, macOS, or Linux | Latest version of each OS |
| Memory (RAM) | 16 GB | 32 GB+ |
| Processor (CPU) | Modern multicore (Intel i7 or AMD Ryzen 7) | Newer generation with higher cores |
| Graphics (GPU) | 16GB of VRAM | 60GB of VRAM |
| Storage | 15 GB free space | 20 GB+ to store larger models |