LocalAI
FreeWraps local models in an OpenAI-shaped API — point your code at a new URL
A local inference server that exposes an OpenAI-compatible API, so existing programs switch to local models by changing the endpoint rather than the calling code. Beyond text generation it covers embeddings, speech-to-text, text-to-speech and image generation, and runs on CPU as well as GPU.
Why we picked it
MIT-licensed, 49,000 stars.
Ollama, LM Studio and Jan — all in this directory — are aimed at "I want to chat with a model."
LocalAI is aimed at something else: my program already calls the OpenAI API and I want that part to
run on my own machine without touching the business code. It matches the API shape, so changing
base_url is the whole migration.
It spans more than text generation: embeddings feed a self-hosted vector search, the whisper family handles transcription, and image generation lives in the same service. Building a fully offline application, that removes a lot of glue between separate services.
To be clear about:
- If you are not building anything, Ollama is simpler. LocalAI earns its keep on API compatibility and unified multimodality; for plain local chat it is overkill.
- Configuration is heavier than Ollama's — models are defined in YAML and the first run means reading documentation.
- Running on CPU is not the same as running fast. Without a GPU, generation on a large model drags visibly.