Inference for text-embeddings in Rust, HFOIL Licence.

First LLM Multi-agent framework.

a toolkit for deploying and serving Large Language Models (LLMs).

A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.

Run LLMs and batch jobs on any cloud. Get maximum cost savings, highest GPU availability, and managed execution -- all with a simple interface.

A high-throughput and memory-efficient inference and serving engine for LLMs.