Nvidia Framework for LLM Inference

an open-source NLP framework that allows you to use LLMs and transformer-based models from Hugging Face, OpenAI and Cohere to interact with your own data.

WebAssembly binding for llama.cpp - Enabling in-browser LLM inference

Simple API for deploying any RAG or LLM that you want adding plugins.

LLM inference in C/C++.

A method designed to enhance the efficiency of Transformer models