LLM inference in C/C++.
Gateway streamlines requests to 100+ open & closed source models with a unified API. It is also production-ready with support for caching, fallbacks, retries, timeouts, loadbalancing, and can be edge-deployed for minimum latency.
Easily build, version, evaluate and deploy your LLM-powered apps.
simplifies the evaluation of LLMs by providing a unified microservice to access and test multiple AI models.
Open Source LLM Engineering Platform 🪢 Tracing, Evaluations, Prompt Management, Evaluations and Playground.
Locally running websearch using LLM chains
SGLang is a fast serving framework for large language models and vision language models.
Your email address will not be published. Required fields are marked *
Comment *
Name *
Email *
Website
Captcha: 12 - 10 = ?*
Save my name, email, and website in this browser for the next time I comment.
Gateway streamlines requests to 100+ open & closed source models with a unified API. It is also production-ready with support for caching, fallbacks, retries, timeouts, loadbalancing, and can be edge-deployed for minimum latency.