An open-source GPU cluster manager for running LLMs

A Device-Inference framework, including LLM Inference on device(Mobile Phone/PC/IOT)

Comprehensive set of tools for working with local LLMs for various tasks.

Seamlessly integrate LLMs as Python functions

Confidently evaluate, test, and ship LLM applications with a suite of observability tools to calibrate language model outputs across your dev and production lifecycle.

a chat interface crafted with llama.cpp for running Alpaca models. No API keys, entirely self-hosted!