Harness LLMs with Multi-Agent Programming

Inference for text-embeddings in Rust, HFOIL Licence.

A high-throughput and memory-efficient inference and serving engine for LLMs.

Inference for text-embeddings in Python

Test your prompts. Evaluate and compare LLM outputs, catch regressions, and improve prompt quality.

Interact with LLM using Ollama models(or openAI, mistralAI)via pure shell scripts on your Linux(or MacOS) system, enhancing intelligent system management without any dependencies.