Run LLMs and batch jobs on any cloud. Get maximum cost savings, highest GPU availability, and managed execution -- all with a simple interface.

Locally running websearch using LLM chains

a chat interface crafted with llama.cpp for running Alpaca models. No API keys, entirely self-hosted!

Formerly langchain-ChatGLM, local knowledge based LLM (like ChatGLM) QA app with langchain.

NanoFlow is a throughput-oriented high-performance serving framework for LLMs. NanoFlow consistently delivers superior throughput compared to vLLM, Deepspeed-FastGen, and TensorRT-LLM.

A distributed multi-model LLM serving system with web UI and OpenAI-compatible RESTful APIs.