Lightweight alternative to LangChain for composing LLMs
A distributed multi-model LLM serving system with web UI and OpenAI-compatible RESTful APIs.
Blazingly fast LLM inference.
Open Source LLM Engineering Platform 🪢 Tracing, Evaluations, Prompt Management, Evaluations and Playground.
A python package for Txt-to-SQL with self hosting functionalities and RESTful APIs compatible with proprietary as well as open source LLM.
NanoFlow is a throughput-oriented high-performance serving framework for LLMs. NanoFlow consistently delivers superior throughput compared to vLLM, Deepspeed-FastGen, and TensorRT-LLM.
Nvidia Framework for LLM Inference
Your email address will not be published. Required fields are marked *
Comment *
Name *
Email *
Website
Captcha: 16 + 19 = ?*
Save my name, email, and website in this browser for the next time I comment.
A distributed multi-model LLM serving system with web UI and OpenAI-compatible RESTful APIs.