Confidently evaluate, test, and ship LLM applications with a suite of observability tools to calibrate language model outputs across your dev and production lifecycle.

LLM inference in C/C++.

Open Source LLM Engineering Platform 🪢 Tracing, Evaluations, Prompt Management, Evaluations and Playground.

Lightweight alternative to LangChain for composing LLMs

Run LLMs and batch jobs on any cloud. Get maximum cost savings, highest GPU availability, and managed execution -- all with a simple interface.

A distributed multi-model LLM serving system with web UI and OpenAI-compatible RESTful APIs.