a toolkit for deploying and serving Large Language Models (LLMs).

First LLM Multi-agent framework.

Confidently evaluate, test, and ship LLM applications with a suite of observability tools to calibrate language model outputs across your dev and production lifecycle.

a chat interface crafted with llama.cpp for running Alpaca models. No API keys, entirely self-hosted!

An open-source GPU cluster manager for running LLMs

Test your prompts. Evaluate and compare LLM outputs, catch regressions, and improve prompt quality.