FlexLLMGen is a high-throughput generation engine for running large language models with limited GPU memory. FlexLLMGen allows high-throughput generation by IO-efficient offloading, compression, and large effective batch sizes.

Framework to create ChatGPT like bots over your dataset.

Formerly langchain-ChatGLM, local knowledge based LLM (like ChatGLM) QA app with langchain.

Use ChatGPT On Wechat via wechaty

Interact with LLM using Ollama models(or openAI, mistralAI)via pure shell scripts on your Linux(or MacOS) system, enhancing intelligent system management without any dependencies.

A python package for Txt-to-SQL with self hosting functionalities and RESTful APIs compatible with proprietary as well as open source LLM.