Confidently evaluate, test, and ship LLM applications with a suite of observability tools to calibrate language model outputs across your dev and production lifecycle.

Inference for text-embeddings in Rust, HFOIL Licence.

Seamlessly integrate LLMs as Python functions

WebAssembly binding for llama.cpp - Enabling in-browser LLM inference

Open Source LLM Engineering Platform 🪢 Tracing, Evaluations, Prompt Management, Evaluations and Playground.

MII makes low-latency and high-throughput inference, similar to vLLM powered by DeepSpeed.