A method designed to enhance the efficiency of Transformer models

Fine-tune, serve, deploy, and monitor any open-source LLMs in production. Used in production at BentoML for LLMs-based applications.

An open-source GPU cluster manager for running LLMs

Building applications with LLMs through composability

a toolkit for deploying and serving Large Language Models (LLMs).

A high-throughput and memory-efficient inference and serving engine for LLMs.