Simple API for deploying any RAG or LLM that you want adding plugins.

An interactive chat project that leverages Ollama/OpenAI/MistralAI LLMs for rapid understanding and navigation of GitHub code repository or compressed file resources.

Fine-tune, serve, deploy, and monitor any open-source LLMs in production. Used in production at BentoML for LLMs-based applications.

SGLang is a fast serving framework for large language models and vision language models.

Formerly langchain-ChatGLM, local knowledge based LLM (like ChatGLM) QA app with langchain.

An open-source GPU cluster manager for running LLMs