Easily build, version, evaluate and deploy your LLM-powered apps.

Nvidia Framework for LLM Inference

Simple API for deploying any RAG or LLM that you want adding plugins.

Playground for devs to finetune & deploy LLMs

FlexLLMGen is a high-throughput generation engine for running large language models with limited GPU memory. FlexLLMGen allows high-throughput generation by IO-efficient offloading, compression, and large effective batch sizes.

Formerly langchain-ChatGLM, local knowledge based LLM (like ChatGLM) QA app with langchain.