One terminal per model
omni runs every model behind 127.0.0.1:10000, supervised and restarted if one crashes.
free · local · open source
Every AI model you need, running locally in one place.
Multiple models expose their API behind a single port. Set it all up in just a couple of clicks.
curl -fsSL https://raw.githubusercontent.com/nvkudva/omni-api/main/install.sh | sh
brew tap nvkudva/omni && brew trust nvkudva/omni && brew install omni
Then run it
omni init && omni up
Why omni
The next generation of apps needs more than an LLM
A virtual agent needs text, speech-to-text, text-to-speech and embeddings. Wiring them up by hand is where the weekend goes.
Which model? Will it fit?
Small, sensible defaults for every role. The playground searches for alternatives and warns before one won't fit your RAM.
Zero setup errors
omni init installs and downloads everything. Your app asks for a role like chat or tts, never a model name.
Architecture
One port in. Seven roles out.
Your app talks to one gateway. omni routes each request to the right model and keeps the engines private.
your app
OpenAI SDK
HTTP / cURL
Tauri · Electron · Python
gateway
omni · :10000
routes by path · supervises workers · restarts on crash
roles · engines
chat Ollama
decide onnxruntime
embed Ollama
image diffusers
video diffusers
stt onnx-asr
tts onnxruntime
Quickstart
Install, run, stop, uninstall
Install, pull the models, start. Then open the playground and click.
-
1
Install
Brings Python and uv if they are missing, then installs the
omnicommand. On macOS you can use Homebrew instead — each command below is its own copy button.shellcurl -fsSL https://raw.githubusercontent.com/nvkudva/omni-api/main/install.sh | shshellbrew tap nvkudva/omni brew trust nvkudva/omni brew install omni -
2
Pull the models
Installs Ollama if needed, fetches chat and embed, and sets up decide and speech. Add
--mediafor image and video.shellomni init # core models + speech omni init --media # also image and video -
3
Start and open the playground
Runs the gateway and every worker in the background. Then open
http://127.0.0.1:10000/. Stuck?omni doctorsays what to fix.shellomni up
Playground
Try every model from your browser
No terminal needed after step 3. Each role has a page to test it, swap its model and watch its logs.
- Test — chat, decide, embed, image, video, speech in one place, each with its raw JSON and a copyable cURL.
- Swap — search Hugging Face or Ollama, install, make it the default in a click.
- Control — keep a model warm, restart a worker, watch RAM and CPU.
Theme
Models
Manage models in your browser
Settings lists every model for every role. Add one, make it the default, or remove it without touching a config file.
- Add — search Hugging Face or Ollama and install with one click.
- Default — pick which model each role serves.
- Fit — see the size up front and get warned before a model won't fit your RAM.
Build
Your OpenAI client already works
Change one base_url; the Python tabs reuse the client from Chat. Full endpoint reference is in the README.
python
from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:10000/v1", api_key="local")
reply = client.chat.completions.create(
model="qwen2.5:0.5b",
messages=[{"role": "user", "content": "Hi!"}],
)
print(reply.choices[0].message.content)shell
curl localhost:10000/v1/systemone -H 'content-type: application/json' -d '{
"state": "I have asked three times and still no refund!",
"questions": {
"is_frustrated": {"type": "noul", "instructions": "Does the customer sound frustrated?"},
"topic": {"type": "choice", "instructions": "What is this about?",
"criteria": {"billing": "money, refunds", "shipping": "delivery"}}
}
}'python
client.embeddings.create(model="embed", input=["refund policy", "shipping times"])python
client.audio.speech.create(
model="tts", voice="af_heart", input="Hello from omni."
).write_to_file("out.wav")python
client.audio.transcriptions.create(model="stt", file=open("out.wav", "rb"))python
client.images.generate(model="image", prompt="a lighthouse at dawn", size="1024x1024")FAQ
Good to know
Does anything leave my machine?
No. omni binds to 127.0.0.1. The only network traffic is the first model download from Ollama or Hugging Face.
What happens when a model crashes?
Every worker is health-checked and restarted automatically. One role going down never takes the others with it.
Is the first request slow?
Yes — it loads the model. Pin the roles you use often and they stay loaded across restarts.
Which platforms are supported?
macOS on Apple Silicon runs every role. Linux serves chat and embed through Ollama, decide on ONNX Runtime, and adds speech and video with the media extra. Image generation is macOS-only.
Run your own models.
Keep them to yourself.
One install, one port, every model your app needs.