What is TensorRT-LLM?

TensorRT-LLM is NVIDIA's open-source LLM inference framework, released in late 2023, that compiles transformer models into highly optimized CUDA kernels for the lowest possible latency on NVIDIA GPUs. The platform strengthens enterprise