A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.

Fine-tune, serve, deploy, and monitor any open-source LLMs in production. Used in production at BentoML for LLMs-based applications.

Simple API for deploying any RAG or LLM that you want adding plugins.

Use ChatGPT On Wechat via wechaty

A method designed to enhance the efficiency of Transformer models

Lightweight alternative to LangChain for composing LLMs