Blazingly fast LLM inference.

Create, deploy and operate Actions using Python anywhere to enhance your AI agents and assistants. Batteries included with an extensive set of libraries, helpers and logging.

An open-source GPU cluster manager for running LLMs

Fine-tune, serve, deploy, and monitor any open-source LLMs in production. Used in production at BentoML for LLMs-based applications.

Simple API for deploying any RAG or LLM that you want adding plugins.

Inference for text-embeddings in Python