An interactive chat project that leverages Ollama/OpenAI/MistralAI LLMs for rapid understanding and navigation of GitHub code repository or compressed file resources.

Easily build, version, evaluate and deploy your LLM-powered apps.

A high-throughput and memory-efficient inference and serving engine for LLMs.

FlexLLMGen is a high-throughput generation engine for running large language models with limited GPU memory. FlexLLMGen allows high-throughput generation by IO-efficient offloading, compression, and large effective batch sizes.

A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.

Data integration platform for LLMs.