Playground for devs to finetune & deploy LLMs

MII makes low-latency and high-throughput inference, similar to vLLM powered by DeepSpeed.

Confidently evaluate, test, and ship LLM applications with a suite of observability tools to calibrate language model outputs across your dev and production lifecycle.

Get up and running with Llama 3, Mistral, Gemma, and other large language models.

Blazingly fast LLM inference.

Fine-tune, serve, deploy, and monitor any open-source LLMs in production. Used in production at BentoML for LLMs-based applications.