Haystack
Inference Engines
Haystack

an open-source NLP framework that allows you to use LLMs and transformer-based models from Hugging Face, OpenAI and Cohere to interact with your own data.

Link
an open-source NLP framework that allows you to use LLMs and transformer-based models from Hugging Face, OpenAI and Cohere to interact with your own data.

Relevant Sites

Swiss Army Llama 1,009

Comprehensive set of tools for working with local LLMs for various tasks.

TGI

a toolkit for deploying and serving Large Language Models (LLMs).

Robocorp 509

Create, deploy and operate Actions using Python anywhere to enhance your AI agents and assistants. Batteries included with an extensive set of libraries, helpers and logging.

Infinity 1,902

Inference for text-embeddings in Python

OpenLLM 10,956

Fine-tune, serve, deploy, and monitor any open-source LLMs in production. Used in production at BentoML for LLMs-based applications.

vLLM 41,431

A high-throughput and memory-efficient inference and serving engine for LLMs.

Leave a Reply

Your email address will not be published. Required fields are marked *