SkyPilot | LLMWay – The Way To LLM

Inference Engines

SkyPilot

Run LLMs and batch jobs on any cloud. Get maximum cost savings, highest GPU availability, and managed execution -- all with a simple interface.

GitHub

Run LLMs and batch jobs on any cloud. Get maximum cost savings, highest GPU availability, and managed execution -- all with a simple interface.

Swiss Army Llama 1,009

Comprehensive set of tools for working with local LLMs for various tasks.

mistral.rs 5,216

Blazingly fast LLM inference.

wechat-chatgpt 13,340

Use ChatGPT On Wechat via wechaty

vLLM 41,431

A high-throughput and memory-efficient inference and serving engine for LLMs.

Search with Lepton 8,032

Build your own conversational search engine using less than 500 lines of code by LeptonAI.