List your AI tool on AI Orion for free. Submit a tool

Self-Hosted AI Stack

Self-Hosted AI Stack is an open-source Docker Compose project for running local models, chat, documents, voice, and AI tools on infrastructure you control.

About Self-Hosted AI Stack

Self-Hosted AI Stack is an open-source project by Lin Song. It functions as a private AI platform that can run on infrastructure you control, with local-first principles and private by default.

The project supports running local AI models, chatting with documents, and transcribing or generating speech, deploying with Docker Compose on your own server, with optional HTTPS and NVIDIA GPU acceleration.

The platform is a composable stack with components for local models, document parsing, embeddings, transcription, speech, gateways, and tools, designed to be deployed on your own hardware.

The stack provides guidance on deployment, security, backups, and everyday operation through accompanying resources like a book and readiness guide, emphasizing that deployment happens on your own Linux server.

Use cases

  • Run local models with InferCrate, powered by Ollama, and use AnythingLLM for browser-based chat.
  • Parse documents with ParseCrate, powered by Docling, and use EmbedCrate for embeddings and reranking in semantic search and retrieval
  • Transcribe audio with ScribeCrate and generate speech with SpeakCrate, powered by Kokoro
  • Route model requests through GatewayCrate, powered by LiteLLM, and connect supported tools through UplinkCrate

Key features

  • Local AI models and document chat
  • Documents and retrieval
  • Transcription and voice
  • AI gateways and tools
  • Authentication by default
  • Flexible deployment with optional HTTPS and GPU acceleration

Reviews

No reviews yet. Used Self-Hosted AI Stack? Be the first to review it.Log in to review

Frequently asked questions

Is Self-Hosted AI Stack a hosted AI service?
You deploy the software on your own Linux server, locally or in the cloud. This website provides project information and resources.
Do I need an NVIDIA GPU?
NVIDIA GPU acceleration is optional; memory and performance requirements depend on your services and models.
Can I run only some services?
Yes. Use a preconfigured lightweight stack or customize the Compose configuration.
Is the software free?
The stack's configuration is MIT licensed. Included services and models retain their own licenses.

Alternatives to Self-Hosted AI Stack

More in AI Chatbot

Comparing tools