DeepSpeed-Mii
Inference Engines
DeepSpeed-Mii

MII makes low-latency and high-throughput inference, similar to vLLM powered by DeepSpeed.

GitHub
MII makes low-latency and high-throughput inference, similar to vLLM powered by DeepSpeed.

Relevant Sites

Serge 5,735

a chat interface crafted with llama.cpp for running Alpaca models. No API keys, entirely self-hosted!

Swiss Army Llama 1,019

Comprehensive set of tools for working with local LLMs for various tasks.

QA-Pilot 293

An interactive chat project that leverages Ollama/OpenAI/MistralAI LLMs for rapid understanding and navigation of GitHub code repository or compressed file resources.

mistral.rs 5,890

Blazingly fast LLM inference.

CAMEL

First LLM Multi-agent framework.

LiteChain 419

Lightweight alternative to LangChain for composing LLMs

Leave a Reply

Your email address will not be published. Required fields are marked *