Use ChatGPT On Wechat via wechaty

SGLang is a fast serving framework for large language models and vision language models.

NanoFlow is a throughput-oriented high-performance serving framework for LLMs. NanoFlow consistently delivers superior throughput compared to vLLM, Deepspeed-FastGen, and TensorRT-LLM.

Seamlessly integrate LLMs as Python functions

Playground for devs to finetune & deploy LLMs

An open-source GPU cluster manager for running LLMs