Run LLMs and batch jobs on any cloud. Get maximum cost savings, highest GPU availability, and managed execution -- all with a simple interface.

A distributed multi-model LLM serving system with web UI and OpenAI-compatible RESTful APIs.

Harness LLMs with Multi-Agent Programming

Lightweight alternative to LangChain for composing LLMs

NVIDIA Framework for LLM Inference(Transitioned to TensorRT-LLM)

WebAssembly binding for llama.cpp - Enabling in-browser LLM inference