
Inference Engines
FasterTransformer
NVIDIA Framework for LLM Inference(Transitioned to TensorRT-LLM)
NVIDIA Framework for LLM Inference(Transitioned to TensorRT-LLM)
Lightweight alternative to LangChain for composing LLMs