Playground for devs to finetune & deploy LLMs

Building applications with LLMs through composability

A distributed multi-model LLM serving system with web UI and OpenAI-compatible RESTful APIs.

A high-throughput and low-latency inference and serving framework for LLMs and VLs

simplifies the evaluation of LLMs by providing a unified microservice to access and test multiple AI models.

Create, deploy and operate Actions using Python anywhere to enhance your AI agents and assistants. Batteries included with an extensive set of libraries, helpers and logging.