Skip to content
AI Atlas
Frameworks

Frameworks & runtimes

Libraries, inference engines and agent frameworks. Versions and stars are read from PyPI, GitHub and release pages — never estimated.

66 total

Reset
Frameworks & runtimes
FrameworkLatest versionRelease dateLicenseStarsLanguageOrganizationQualityCompare
text-generation-inferenceLarge Language Model Text Generation Inference3.3.719 Dec 2025Apache-2.010,886—Hugging Face
onnxruntimeONNX Runtime: cross-platform, high performance ML inferencing and training acceleratorONNX Runtime v1.30.010 Sept 2026MIT21,827—Microsoft
TensorRT-LLMTensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate 1.3.0rc269 Sept 2026—14,597—NVIDIA
sglangSGLang is a high-performance serving framework for large language models and multimodal models.0.5.195 Sept 2026Apache-2.035,824—SGLang project
ollamaGet up and running with Kimi-K2.6, GLM-5.2, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.0.34.010 Sept 2026MIT180,690—Ollama
mlx-lmRun LLMs with MLX0.31.322 Apr 2026MIT6,976—Apple ML Explore (MLX)
mlxMLX: An array framework for Apple silicon0.32.225 Aug 2026MIT28,386—Apple ML Explore (MLX)
llama-cpp-pythonPython bindings for llama.cpp0.3.35-cu12317 Aug 2026MIT10,615——
llama.cppLLM inference in C/C++b1091711 Sept 2026MIT127,879—ggml.ai
vllmA high-throughput and memory-efficient inference and serving engine for LLMs0.29.010 Sept 2026Apache-2.091,519—vLLM project
nanoGPTThe simplest, fastest repository for training/finetuning medium-sized GPTs.——MIT62,999——
SpeechA scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)NVIDIA NeMo Speech 3.07 Aug 2026Apache-2.018,424—NVIDIA
flash-attentionFast and memory-efficient exact attentionfa4-v4.0.0.beta309 Sept 2026BSD-3-Clause24,890—Dao AI Lab
tritonDevelopment repository for the Triton language and compilergfx950-tutorial-v2.131 Aug 2026MIT20,132—OpenAI
BentoMLThe easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!1.4.397 May 2026Apache-2.08,834—BentoML
kubeflowMachine Learning Toolkit for Kubernetes1.10.025 Mar 2025Apache-2.015,860—Kubeflow
rayRay is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.Ray-2.58.023 Aug 2026Apache-2.043,782—Anyscale
pytorch-lightningPretrain, finetune ANY AI model of ANY size on 1 or 10,000+ GPUs with zero code changes.Lightning 2.6.610 Sept 2026Apache-2.031,336—Lightning AI
axolotlGo ahead and axolotl questions0.19.010 Sept 2026Apache-2.012,461—Axolotl AI
unslothLocal UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4, MiniMax-H3, Gemma 4, FLUX and more.Large Performance Gains + Fixes9 Sept 2026Apache-2.076,034—Unsloth
trlTrain transformer language models with reinforcement learning.1.13.010 Sept 2026Apache-2.019,286—Hugging Face
peft🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.0.20.028 Jul 2026Apache-2.021,657—Hugging Face
diffusers🤗 Diffusers: State-of-the-art diffusion models for image, video, and audio generation in PyTorch.Diffusers 0.40.0: New pipelines, tensor-parallel support, improved CLI, and more20 Aug 2026Apache-2.034,497—Hugging Face
jaxComposable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and moreJAX v0.11.117 Aug 2026Apache-2.036,295—Google
transformers🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training. Release 5.17.010 Sept 2026Apache-2.0165,129—Hugging Face
pytorchTensors and Dynamic neural networks in Python with strong GPU accelerationviable/strict/1789158535: [ROCm][CD] Add gfx1250 (MI450) to the nightly wheel arch list (#196610)11 Sept 2026—102,930—PyTorch Foundation

Stars are a GitHub metric observed at crawl time (see each framework's Sources tab for the snapshot). Versions come from PyPI or the project's release feed.