A high-throughput and low-latency inference and serving framework for LLMs and VLs

Inference for text-embeddings in Rust, HFOIL Licence.

Interact with LLM using Ollama models(or openAI, mistralAI)via pure shell scripts on your Linux(or MacOS) system, enhancing intelligent system management without any dependencies.

First LLM Multi-agent framework.

Framework to create ChatGPT like bots over your dataset.

Simple API for deploying any RAG or LLM that you want adding plugins.