SGLang | LLMWay – The Way To LLM

Inference Engines

SGLang

SGLang is a fast serving framework for large language models and vision language models.

GitHub

SGLang is a fast serving framework for large language models and vision language models.

FastChat 38,858

A distributed multi-model LLM serving system with web UI and OpenAI-compatible RESTful APIs.

LLocalSearch 5,940

Locally running websearch using LLM chains

Infinity 2,311

Inference for text-embeddings in Python

Text-Embeddings-Inference 3,801

Inference for text-embeddings in Rust, HFOIL Licence.

GPUStack 3,082

An open-source GPU cluster manager for running LLMs