Use ChatGPT On Wechat via wechaty

Easily build, version, evaluate and deploy your LLM-powered apps.

A Device-Inference framework, including LLM Inference on device(Mobile Phone/PC/IOT)

Fine-tune, serve, deploy, and monitor any open-source LLMs in production. Used in production at BentoML for LLMs-based applications.

Nvidia Framework for LLM Inference

SGLang is a fast serving framework for large language models and vision language models.