Harness LLMs with Multi-Agent Programming

Formerly langchain-ChatGLM, local knowledge based LLM (like ChatGLM) QA app with langchain.

FlexLLMGen is a high-throughput generation engine for running large language models with limited GPU memory. FlexLLMGen allows high-throughput generation by IO-efficient offloading, compression, and large effective batch sizes.

an open-source NLP framework that allows you to use LLMs and transformer-based models from Hugging Face, OpenAI and Cohere to interact with your own data.

Inference for text-embeddings in Rust, HFOIL Licence.

A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.