Seamlessly integrate LLMs as Python functions

Lightweight alternative to LangChain for composing LLMs

Interact with LLM using Ollama models(or openAI, mistralAI)via pure shell scripts on your Linux(or MacOS) system, enhancing intelligent system management without any dependencies.

Framework to create ChatGPT like bots over your dataset.

NanoFlow is a throughput-oriented high-performance serving framework for LLMs. NanoFlow consistently delivers superior throughput compared to vLLM, Deepspeed-FastGen, and TensorRT-LLM.

WebAssembly binding for llama.cpp - Enabling in-browser LLM inference