Building applications with LLMs through composability

Blazingly fast LLM inference.

Use ChatGPT On Wechat via wechaty

Test your prompts. Evaluate and compare LLM outputs, catch regressions, and improve prompt quality.

Interact with LLM using Ollama models(or openAI, mistralAI)via pure shell scripts on your Linux(or MacOS) system, enhancing intelligent system management without any dependencies.

Inference for text-embeddings in Rust, HFOIL Licence.