Lightweight alternative to LangChain for composing LLMs

Confidently evaluate, test, and ship LLM applications with a suite of observability tools to calibrate language model outputs across your dev and production lifecycle.

a chat interface crafted with llama.cpp for running Alpaca models. No API keys, entirely self-hosted!

FlexLLMGen is a high-throughput generation engine for running large language models with limited GPU memory. FlexLLMGen allows high-throughput generation by IO-efficient offloading, compression, and large effective batch sizes.

Comprehensive set of tools for working with local LLMs for various tasks.

A method designed to enhance the efficiency of Transformer models