EstimatedAll figures are estimates, not measurements.80 GB · 8-bit · 32k tokens540 of 658 models with a known parameter count fit
- Estimated, not measured: weights = parameters × bytes/param × 1.15 runtime overhead.
- bytes/param: 4bit = 0.5, 8bit = 1.0, fp16 = 2.0 (uniform quantization, no per-layer exceptions).
- KV cache approximated at 0.5 GB per 8 192 tokens of context, independent of architecture (GQA/MLA models need less).
- A model 'fits' when the estimate is at most the device memory minus 2 GB reserved for the OS and framework.
- Mixture-of-experts models are estimated on total parameters (all experts must be resident); active parameters are ignored.
- Device memory uses the largest configuration when several are listed (e.g. Apple silicon tiers).
| Model | Params | Quant | Est. memory | Headroom | Fits | Compare |
|---|---|---|---|---|---|---|
| nvidia/Qwen3.5-122B-A10B-NVFP4Open weightsNVIDIAestimated: 64.6B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 64.6B | 8bit | 76.3 GB | +1.7 GB | ✓ fits | |
| amd/gpt-oss-120b-w-mxfp4-a-fp8Open weightsAMDestimated: 59.5B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 59.5B | 8bit | 70.4 GB | +7.6 GB | ✓ fits | |
| ai21labs/AI21-Jamba2-Mini-FP8Open weightsAI21 Labsestimated: 51.6B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 51.6B | 8bit | 61.3 GB | +16.7 GB | ✓ fits | |
| ai21labs/AI21-Jamba-Mini-1.7-FP8RestrictedAI21 Labsestimated: 51.6B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 51.6B | 8bit | 61.3 GB | +16.7 GB | ✓ fits | |
| Jamba-v0.1Open weightsAI21 Labsestimated: 51.6B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 51.6B | 8bit | 61.3 GB | +16.7 GB | ✓ fits | |
| AI21-Jamba2-MiniOpen weightsAI21 Labsestimated: 51.6B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 51.6B | 8bit | 61.3 GB | +16.7 GB | ✓ fits | |
| AI21-Jamba-Mini-1.7RestrictedAI21 Labsestimated: 51.6B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 51.6B | 8bit | 61.3 GB | +16.7 GB | ✓ fits | |
| AI21-Jamba-Mini-1.5RestrictedAI21 Labsestimated: 51.6B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 51.6B | 8bit | 61.3 GB | +16.7 GB | ✓ fits | |
| AI21-Jamba-Mini-1.6RestrictedAI21 Labsestimated: 51.6B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 51.6B | 8bit | 61.3 GB | +16.7 GB | ✓ fits | |
| Kimi-Linear-48B-A3B-BaseOpen weightsMoonshot AIestimated: 49.1B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 49.1B | 8bit | 58.5 GB | +19.5 GB | ✓ fits | |
| Kimi-Linear-48B-A3B-InstructOpen weightsMoonshot AIestimated: 49.1B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 49.1B | 8bit | 58.5 GB | +19.5 GB | ✓ fits | |
| function-calling-v1Open weightsFireworks AIestimated: 46.7B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 46.7B | 8bit | 55.7 GB | +22.3 GB | ✓ fits | |
| firefunction-v1Open weightsFireworks AIestimated: 46.7B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 46.7B | 8bit | 55.7 GB | +22.3 GB | ✓ fits | |
| Nous-Hermes-2-Mixtral-8x7B-DPOOpen weightsNous Researchestimated: 46.7B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 46.7B | 8bit | 55.7 GB | +22.3 GB | ✓ fits | |
| Mixtral-8x7B-Instruct-v0.1Open weightsMistral AIestimated: 46.7B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 46.7B | 8bit | 55.7 GB | +22.3 GB | ✓ fits | |
| DFN2B-CLIP-ViT-L-14-39BOpen weightsAppleestimated: 39.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 39B | 8bit | 46.9 GB | +31.1 GB | ✓ fits | |
| Seed-OSS-36B-BaseOpen weightsByteDanceestimated: 36.2B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 36.1B | 8bit | 43.6 GB | +34.4 GB | ✓ fits | |
| Seed-OSS-36B-InstructOpen weightsByteDanceestimated: 36.2B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 36.1B | 8bit | 43.6 GB | +34.4 GB | ✓ fits | |
| NousResearch/Hermes-4.3-36B-GGUFOpen weightsNous Researchestimated: 36.2B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 36.1B | 8bit | 43.6 GB | +34.4 GB | ✓ fits | |
| Qwen/Qwen3.6-35B-A3B-FP8Open weightsQwenestimated: 36.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 36B | 8bit | 43.4 GB | +34.6 GB | ✓ fits | |
| Qwen3.6 35B A3BOpen weightsQwenestimated: 36.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 36B | 8bit | 43.3 GB | +34.7 GB | ✓ fits | |
| unsloth/Qwen3.6-35B-A3B-MTP-GGUFOpen weightsUnslothestimated: 35.5B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 35.5B | 8bit | 42.8 GB | +35.2 GB | ✓ fits | |
| bartowski/endless-frontier_BigBang-v1-GGUFOpen weightsbartowskiestimated: 35.5B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 35.5B | 8bit | 42.8 GB | +35.2 GB | ✓ fits | |
| cerebras/Kimi-Linear-REAP-35B-A3B-InstructOpen weightsCerebras Systemsestimated: 35.1B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 35.1B | 8bit | 42.4 GB | +35.6 GB | ✓ fits | |
| perplexity-ai/pplx-computer-qwen-3-6-35b-a3b-nvfp4-20260709Open weightsPerplexity AIestimated: 35.1B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 35.1B | 8bit | 42.4 GB | +35.6 GB | ✓ fits | |
| bartowski/Qwen_Qwen3.6-35B-A3B-GGUFOpen weightsbartowskiestimated: 35.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 35B | 8bit | 42.3 GB | +35.8 GB | ✓ fits | |
| bartowski/Qwen_Qwen3.5-35B-A3B-GGUFOpen weightsbartowskiestimated: 35.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 35B | 8bit | 42.3 GB | +35.8 GB | ✓ fits | |
| c4ai-command-r-v01RestrictedCohereestimated: 35.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 35B | 8bit | 42.2 GB | +35.8 GB | ✓ fits | |
| bartowski/Kwaipilot_KAT-Coder-V2.5-Dev-GGUFOpen weightsbartowskiestimated: 34.7B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 34.7B | 8bit | 41.9 GB | +36.1 GB | ✓ fits | |
| bartowski/thomsonreuters_Thomson-1.0-Small-GGUFOpen weightsbartowskiestimated: 34.7B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 34.7B | 8bit | 41.9 GB | +36.1 GB | ✓ fits | |
| unsloth/Qwen3.6-35B-A3B-GGUFOpen weightsUnslothestimated: 34.7B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 34.7B | 8bit | 41.9 GB | +36.1 GB | ✓ fits | |
| bartowski/XYZAILab_XYZ-Aquila-mini-GGUFOpen weightsbartowskiestimated: 34.7B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 34.7B | 8bit | 41.9 GB | +36.1 GB | ✓ fits | |
| perplexity-ai/pplx-computer-qwen-3-6-35b-a3b-mlx-20260709Open weightsPerplexity AIestimated: 34.7B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 34.7B | 8bit | 41.9 GB | +36.1 GB | ✓ fits | |
| Nous-Hermes-2-Yi-34BOpen weightsNous Researchestimated: 34.4B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 34.4B | 8bit | 41.5 GB | +36.5 GB | ✓ fits | |
| GKA-primed-HQwen3-32B-ReasonerOpen weightsAmazon Web Servicesestimated: 34.1B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 34.1B | 8bit | 41.3 GB | +36.7 GB | ✓ fits | |
| MiniMax-H3Open weightsMiniMaxestimated: 33.1B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 33.1B | 8bit | 40.1 GB | +37.9 GB | ✓ fits | |
| nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-FP8Open weightsNVIDIAestimated: 33.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 33B | 8bit | 40.0 GB | +38.0 GB | ✓ fits | |
| SynLogic-32BOpen weightsMiniMaxestimated: 32.8B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 32.8B | 8bit | 39.7 GB | +38.3 GB | ✓ fits | |
| DeepSeek-R1-Distill-Qwen-32BOpen weightsDeepSeekestimated: 32.8B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 32.8B | 8bit | 39.7 GB | +38.3 GB | ✓ fits | |
| SynLogic-Mix-3-32BOpen weightsMiniMaxestimated: 32.8B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 32.8B | 8bit | 39.7 GB | +38.3 GB | ✓ fits | |
| Qwen3 32BOpen weightsQwenestimated: 32.8B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 32.8B | 8bit | 39.7 GB | +38.3 GB | ✓ fits | |
| c4ai-command-r-08-2024RestrictedCohereestimated: 32.3B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 32.3B | 8bit | 39.1 GB | +38.9 GB | ✓ fits | |
| aya-expanse-32bRestrictedCohereestimated: 32.3B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 32.3B | 8bit | 39.1 GB | +38.9 GB | ✓ fits | |
| OLMo-2-0325-32B-InstructOpen weightsAllen Institute for AIestimated: 32.2B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 32.2B | 8bit | 39.1 GB | +38.9 GB | ✓ fits | |
| FLUX.2-devRestrictedBlack Forest Labsestimated: 32.2B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 32.2B | 8bit | 39.1 GB | +38.9 GB | ✓ fits | |
| Nemotron 3 Nano 30B A3BOpen weightsNVIDIAestimated: 31.6B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 31.6B | 8bit | 38.3 GB | +39.7 GB | ✓ fits | |
| mlx-community/gemma-4-31b-it-4bitOpen weightsMLX Communityestimated: 31.3B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 31.3B | 8bit | 38.0 GB | +40.0 GB | ✓ fits | |
| Gemma 4 31BOpen weightsGoogleestimated: 31.3B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 31.3B | 8bit | 38.0 GB | +40.0 GB | ✓ fits | |
| GLM 4.7 FlashOpen weightsZ.ai (Zhipu AI)estimated: 31.2B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 31.2B | 8bit | 37.9 GB | +40.1 GB | ✓ fits | |
| unsloth/gemma-4-31B-it-qat-GGUFOpen weightsUnslothestimated: 30.7B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 30.7B | 8bit | 37.3 GB | +40.7 GB | ✓ fits | |
| mlx-community/Qwen3-30B-A3B-Instruct-2507-4bitOpen weightsMLX Communityestimated: 30.5B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 30.5B | 8bit | 37.1 GB | +40.9 GB | ✓ fits | |
| browsesafeOpen weightsPerplexity AIestimated: 30.5B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 30.5B | 8bit | 37.1 GB | +40.9 GB | ✓ fits | |
| unsloth/Qwen3-30B-A3B-Thinking-2507-GGUFOpen weightsUnslothestimated: 30.5B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 30.5B | 8bit | 37.1 GB | +40.9 GB | ✓ fits | |
| unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUFOpen weightsUnslothestimated: 30.5B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 30.5B | 8bit | 37.1 GB | +40.9 GB | ✓ fits | |
| North Mini Code (free)Open weightsCohereestimated: 30.5B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 30.5B | 8bit | 37.1 GB | +40.9 GB | ✓ fits | |
| Hy-MT2-30B-A3BOpen weightsTencentestimated: 30.1B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 30.1B | 8bit | 36.6 GB | +41.4 GB | ✓ fits | |
| north-mini-code-1-0Cohereestimated: 30.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 30B | 8bit | 36.5 GB | +41.5 GB | ✓ fits | |
| ibm-granite/granite-4.2-30b-GGUFOpen weightsIBMestimated: 30.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 30B | 8bit | 36.5 GB | +41.5 GB | ✓ fits | |
| ERNIE-4.5-VL-28B-A3B-ThinkingOpen weightsBaiduestimated: 29.7B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 29.7B | 8bit | 36.1 GB | +41.9 GB | ✓ fits | |
| ERNIE-4.5-VL-28B-A3B-PTOpen weightsBaiduestimated: 29.4B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 29.4B | 8bit | 35.8 GB | +42.2 GB | ✓ fits | |
| granite-4.1-30bOpen weightsIBMestimated: 28.9B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 28.9B | 8bit | 35.2 GB | +42.8 GB | ✓ fits | |
| Qwen/Qwen3.6-27B-FP8Open weightsQwenestimated: 27.8B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 27.8B | 8bit | 34.0 GB | +44.0 GB | ✓ fits | |
| Qwen/Qwen3.8-27B-FP8Open weightsQwenestimated: 27.8B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 27.8B | 8bit | 34.0 GB | +44.0 GB | ✓ fits | |
| Qwen3.8 27BOpen weightsQwenestimated: 27.8B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 27.8B | 8bit | 34.0 GB | +44.0 GB | ✓ fits | |
| Qwen3.6 27BOpen weightsQwenestimated: 27.8B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 27.8B | 8bit | 34.0 GB | +44.0 GB | ✓ fits | |
| mlx-community/Qwen3.8-27B-8bitOpen weightsMLX Communityestimated: 27.4B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 27.4B | 8bit | 33.5 GB | +44.5 GB | ✓ fits | |
| mlx-community/Qwen3.8-27B-4bitOpen weightsMLX Communityestimated: 27.4B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 27.4B | 8bit | 33.5 GB | +44.5 GB | ✓ fits | |
| unsloth/Qwen3.8-27B-GGUFOpen weightsUnslothestimated: 27.3B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 27.3B | 8bit | 33.4 GB | +44.6 GB | ✓ fits | |
| perplexity-ai/pplx-computer-qwen-3-8-27b-dflash2-gguf-20260826Open weightsPerplexity AIestimated: 27.3B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 27.3B | 8bit | 33.4 GB | +44.6 GB | ✓ fits | |
| unsloth/Qwen3.6-27B-MTP-GGUFOpen weightsUnslothestimated: 27.3B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 27.3B | 8bit | 33.4 GB | +44.6 GB | ✓ fits | |
| bartowski/Fara1.5-27B-GGUFOpen weightsbartowskiestimated: 27.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 27B | 8bit | 33.0 GB | +45.0 GB | ✓ fits | |
| bartowski/Qwen3.8-27B-GGUFOpen weightsbartowskiestimated: 27.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 27B | 8bit | 33.0 GB | +45.0 GB | ✓ fits | |
| unsloth/Qwen3.6-27B-GGUFOpen weightsUnslothestimated: 26.9B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 26.9B | 8bit | 32.9 GB | +45.1 GB | ✓ fits | |
| Gemma 4 26B A4BOpen weightsGoogleestimated: 25.8B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 25.8B | 8bit | 31.7 GB | +46.3 GB | ✓ fits | |
| unsloth/gemma-4-26B-A4B-it-qat-GGUFOpen weightsUnslothestimated: 25.2B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 25.2B | 8bit | 31.0 GB | +47.0 GB | ✓ fits | |
| unsloth/gemma-4-26B-A4B-it-GGUFOpen weightsUnslothestimated: 25.2B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 25.2B | 8bit | 31.0 GB | +47.0 GB | ✓ fits | |
| cerebras/Qwen3-Coder-REAP-25B-A3BOpen weightsCerebras Systemsestimated: 24.9B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 24.9B | 8bit | 30.6 GB | +47.4 GB | ✓ fits | |
| unsloth/Qwen3.6-35B-A3B-NVFP4Open weightsUnslothestimated: 24.6B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 24.6B | 8bit | 30.3 GB | +47.7 GB | ✓ fits | |
| Voxtral Small 24B 2507Open weightsMistral AIestimated: 24.3B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 24.3B | 8bit | 29.9 GB | +48.1 GB | ✓ fits | |
| Devstral-Small-2-24B-Instruct-2512Open weightsMistral AIestimated: 24.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 24B | 8bit | 29.6 GB | +48.4 GB | ✓ fits | |
| Mistral Small 3.1 24BOpen weightsMistral AIestimated: 24.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 24B | 8bit | 29.6 GB | +48.4 GB | ✓ fits | |
| Mistral Small 3.2 24BOpen weightsMistral AIestimated: 24.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 24B | 8bit | 29.6 GB | +48.4 GB | ✓ fits | |
| mlx-community/Devstral-Small-2-24B-Instruct-2512-4bitOpen weightsMLX Communityestimated: 24.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 24B | 8bit | 29.6 GB | +48.4 GB | ✓ fits | |
| cerebras/GLM-4.7-Flash-REAP-23B-A3BOpen weightsCerebras Systemsestimated: 23.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 23B | 8bit | 28.4 GB | +49.5 GB | ✓ fits | |
| ERNIE-4.5-21B-A3B-PTOpen weightsBaiduestimated: 21.9B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 21.9B | 8bit | 27.2 GB | +50.8 GB | ✓ fits | |
| ERNIE-4.5-21B-A3B-Base-PTOpen weightsBaiduestimated: 21.8B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 21.8B | 8bit | 27.1 GB | +50.9 GB | ✓ fits | |
| ERNIE-4.5-21B-A3B-ThinkingOpen weightsBaiduestimated: 21.8B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 21.8B | 8bit | 27.1 GB | +50.9 GB | ✓ fits | |
| gpt-oss-safeguard-20bOpen weightsOpenAIestimated: 21.5B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 21.5B | 8bit | 26.7 GB | +51.3 GB | ✓ fits | |
| unsloth/Qwen3.6-27B-NVFP4Open weightsUnslothestimated: 21.2B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 21.2B | 8bit | 26.4 GB | +51.6 GB | ✓ fits | |
| gpt-oss-20bOpen weightsOpenAIestimated: 20.9B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 20.9B | 8bit | 26.1 GB | +52.0 GB | ✓ fits | |
| mlx-community/gpt-oss-20b-MXFP4-Q8Open weightsMLX Communityestimated: 20.9B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 20.9B | 8bit | 26.1 GB | +52.0 GB | ✓ fits | |
| nvidia/Gemma-4-31B-IT-NVFP4Open weightsNVIDIAestimated: 20.9B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 20.9B | 8bit | 26.0 GB | +52.0 GB | ✓ fits | |
| gpt-neox-20bOpen weightsEleutherAIestimated: 20.7B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 20.7B | 8bit | 25.9 GB | +52.1 GB | ✓ fits | |
| unsloth/MiniMax-H3-GGUFOpen weightsUnslothestimated: 20.1B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 20.1B | 8bit | 25.1 GB | +52.9 GB | ✓ fits | |
| internlm2-20bOpen weightsInternLM (Shanghai AI Laboratory)estimated: 20.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 20B | 8bit | 25.0 GB | +53.0 GB | ✓ fits | |
| internlm/internlm2_5-20b-chat-ggufOpen weightsInternLM (Shanghai AI Laboratory)estimated: 20.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 20B | 8bit | 25.0 GB | +53.0 GB | ✓ fits | |
| internlm2-base-20bOpen weightsInternLM (Shanghai AI Laboratory)estimated: 20.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 20B | 8bit | 25.0 GB | +53.0 GB | ✓ fits | |
| unsloth/Qwen3.8-27B-NVFP4Open weightsUnslothestimated: 19.9B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 19.9B | 8bit | 24.9 GB | +53.1 GB | ✓ fits | |
| internlm2-chat-20bOpen weightsInternLM (Shanghai AI Laboratory)estimated: 19.9B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 19.9B | 8bit | 24.8 GB | +53.2 GB | ✓ fits | |
| Intel/Qwen3.5-122B-A10B-int4-AutoRoundOpen weightsIntelestimated: 19.7B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 19.7B | 8bit | 24.6 GB | +53.4 GB | ✓ fits | |
| pplx-qwen-3-8-27b-dflash2-20260819Open weightsPerplexity AIestimated: 18.8B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 18.8B | 8bit | 23.6 GB | +54.4 GB | ✓ fits | |
| pplx-computer-qwen-3-8-27b-dflash2-20260824Open weightsPerplexity AIestimated: 18.8B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 18.8B | 8bit | 23.6 GB | +54.4 GB | ✓ fits | |
| nvidia/Qwen3.6-35B-A3B-NVFP4Open weightsNVIDIAestimated: 18.7B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 18.7B | 8bit | 23.5 GB | +54.5 GB | ✓ fits | |
| nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-NVFP4Open weightsNVIDIAestimated: 18.3B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 18.3B | 8bit | 23.1 GB | +54.9 GB | ✓ fits | |
| nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4Open weightsNVIDIAestimated: 17.8B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 17.8B | 8bit | 22.5 GB | +55.5 GB | ✓ fits | |
| Kimi-VL-A3B-ThinkingOpen weightsMoonshot AIestimated: 16.4B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 16.4B | 8bit | 20.9 GB | +57.1 GB | ✓ fits | |
| Kimi-VL-A3B-Thinking-2506Open weightsMoonshot AIestimated: 16.4B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 16.4B | 8bit | 20.9 GB | +57.1 GB | ✓ fits | |
| Kimi-VL-A3B-InstructOpen weightsMoonshot AIestimated: 16.4B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 16.4B | 8bit | 20.9 GB | +57.1 GB | ✓ fits | |
| Moonlight-16B-A3B-InstructOpen weightsMoonshot AIestimated: 16.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 16B | 8bit | 20.4 GB | +57.6 GB | ✓ fits | |
| Moonlight-16B-A3BOpen weightsMoonshot AIestimated: 16.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 16B | 8bit | 20.4 GB | +57.6 GB | ✓ fits | |
| bartowski/DeepSeek-Coder-V2-Lite-Instruct-GGUFOpen weightsbartowskiestimated: 15.7B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 15.7B | 8bit | 20.1 GB | +57.9 GB | ✓ fits | |
| DeepSeek-V2-LiteOpen weightsDeepSeekestimated: 15.7B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 15.7B | 8bit | 20.1 GB | +57.9 GB | ✓ fits | |
| DeepSeek-Coder-V2-Lite-InstructOpen weightsDeepSeekestimated: 15.7B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 15.7B | 8bit | 20.1 GB | +57.9 GB | ✓ fits | |
| amd/Qwen3.8-27B-Quark-AWQ-MXFP4Open weightsAMDestimated: 15.6B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 15.6B | 8bit | 19.9 GB | +58.0 GB | ✓ fits | |
| BAGEL-7B-MoTOpen weightsByteDanceestimated: 14.7B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 14.7B | 8bit | 18.9 GB | +59.1 GB | ✓ fits | |
| Phi 4Open weightsMicrosoftestimated: 14.7B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 14.7B | 8bit | 18.9 GB | +59.1 GB | ✓ fits | |
| nvidia/Gemma-4-26B-A4B-NVFP4Open weightsNVIDIAestimated: 14.4B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 14.4B | 8bit | 18.5 GB | +59.5 GB | ✓ fits | |
| Ministral 3 14B 2512Open weightsMistral AIestimated: 13.9B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 13.9B | 8bit | 18.0 GB | +60.0 GB | ✓ fits | |
| Ministral-3-14B-Reasoning-2512Open weightsMistral AIestimated: 13.9B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 13.9B | 8bit | 18.0 GB | +60.0 GB | ✓ fits | |
| FireLLaVA-13bRestrictedFireworks AIestimated: 13.4B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 13.3B | 8bit | 17.4 GB | +60.6 GB | ✓ fits | |
| gemma-4-12B-it-qat-w4a16-ctOpen weightsGoogleestimated: 13.3B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 13.3B | 8bit | 17.3 GB | +60.7 GB | ✓ fits | |
| Llama-2-13b-chat-hfRestrictedMeta AIestimated: 13.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 13B | 8bit | 17.0 GB | +61.0 GB | ✓ fits | |
| aya-101Open weightsCohereestimated: 12.9B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 12.9B | 8bit | 16.9 GB | +61.1 GB | ✓ fits | |
| Mistral NemoOpen weightsMistral AIestimated: 12.2B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 12.3B | 8bit | 16.1 GB | +61.9 GB | ✓ fits | |
| Mistral-Nemo-Base-2407Open weightsMistral AIestimated: 12.2B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 12.3B | 8bit | 16.1 GB | +61.9 GB | ✓ fits | |
| bartowski/kai-os_Grug-12B-GGUFOpen weightsbartowskiestimated: 12.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 12B | 8bit | 15.8 GB | +62.2 GB | ✓ fits | |
| mlx-community/gemma-3-12b-it-4bitOpen weightsMLX Communityestimated: 12.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 12B | 8bit | 15.8 GB | +62.2 GB | ✓ fits | |
| gemma-4-12B-itOpen weightsGoogleestimated: 12.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 12B | 8bit | 15.8 GB | +62.3 GB | ✓ fits | |
| unsloth/gemma-4-12B-it-qat-GGUFOpen weightsUnslothestimated: 11.9B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 11.9B | 8bit | 15.7 GB | +62.3 GB | ✓ fits | |
| FLUX.1-Fill-devRestrictedBlack Forest Labsestimated: 11.9B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 11.9B | 8bit | 15.7 GB | +62.3 GB | ✓ fits | |
| FLUX.1-Krea-devRestrictedBlack Forest Labsestimated: 11.9B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 11.9B | 8bit | 15.7 GB | +62.3 GB | ✓ fits | |
| FLUX.1-Kontext-devRestrictedBlack Forest Labsestimated: 11.9B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 11.9B | 8bit | 15.7 GB | +62.3 GB | ✓ fits | |
| FLUX.1-devRestrictedBlack Forest Labsestimated: 11.9B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 11.9B | 8bit | 15.7 GB | +62.3 GB | ✓ fits | |
| FLUX.1-schnellRestrictedBlack Forest Labsestimated: 11.9B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 11.9B | 8bit | 15.7 GB | +62.3 GB | ✓ fits | |
| Intel/Qwen3-Coder-Next-int4-AutoRoundOpen weightsIntelestimated: 11.8B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 11.8B | 8bit | 15.6 GB | +62.4 GB | ✓ fits | |
| KaLM-Embedding-Gemma3-12B-2511Open weightsTencentestimated: 11.8B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 11.8B | 8bit | 15.5 GB | +62.5 GB | ✓ fits | |
| Nous-Hermes-2-SOLAR-10.7BOpen weightsNous Researchestimated: 10.7B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 10.7B | 8bit | 14.3 GB | +63.7 GB | ✓ fits | |
| Llama-3.2-11B-Vision-InstructRestrictedMeta AIestimated: 10.7B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 10.7B | 8bit | 14.3 GB | +63.7 GB | ✓ fits | |
| GLM-4.1V-9B-ThinkingOpen weightsZ.ai (Zhipu AI)estimated: 10.3B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 10.3B | 8bit | 13.8 GB | +64.2 GB | ✓ fits | |
| GLM-4.6V-FlashOpen weightsZ.ai (Zhipu AI)estimated: 10.3B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 10.3B | 8bit | 13.8 GB | +64.2 GB | ✓ fits | |
| Kimi-Audio-7BOpen weightsMoonshot AIestimated: 9.8B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 9.77B | 8bit | 13.2 GB | +64.8 GB | ✓ fits | |
| Kimi-Audio-7B-InstructOpen weightsMoonshot AIestimated: 9.8B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 9.77B | 8bit | 13.2 GB | +64.8 GB | ✓ fits | |
| Qwen3.5-9BOpen weightsQwenestimated: 9.7B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 9.65B | 8bit | 13.1 GB | +64.9 GB | ✓ fits | |
| glm-4-9b-chatOpen weightsZ.ai (Zhipu AI)estimated: 9.4B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 9.4B | 8bit | 12.8 GB | +65.2 GB | ✓ fits | |
| academic-ds-9BOpen weightsByteDanceestimated: 9.4B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 9.37B | 8bit | 12.8 GB | +65.2 GB | ✓ fits | |
| FLUX.2-klein-base-9BRestrictedBlack Forest Labsestimated: 9.1B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 9.08B | 8bit | 12.4 GB | +65.6 GB | ✓ fits | |
| FLUX.2-klein-9b-kvRestrictedBlack Forest Labsestimated: 9.1B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 9.08B | 8bit | 12.4 GB | +65.6 GB | ✓ fits | |
| FLUX.2-klein-9BRestrictedBlack Forest Labsestimated: 9.1B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 9.08B | 8bit | 12.4 GB | +65.6 GB | ✓ fits | |
| bartowski/tencent_UI-Mate-9B-GGUFOpen weightsbartowskiestimated: 9.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 9B | 8bit | 12.3 GB | +65.7 GB | ✓ fits | |
| black-forest-labs/FLUX.2-klein-9b-kv-fp8Open weightsBlack Forest Labsestimated: 9.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 9B | 8bit | 12.3 GB | +65.7 GB | ✓ fits | |
| black-forest-labs/FLUX.2-klein-base-9b-fp8RestrictedBlack Forest Labsestimated: 9.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 9B | 8bit | 12.3 GB | +65.7 GB | ✓ fits | |
| bartowski/Fara1.5-9B-GGUFOpen weightsbartowskiestimated: 9.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 9B | 8bit | 12.3 GB | +65.7 GB | ✓ fits | |
| black-forest-labs/FLUX.2-klein-9b-fp8RestrictedBlack Forest Labsestimated: 9.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 9B | 8bit | 12.3 GB | +65.7 GB | ✓ fits | |
| unsloth/Qwen3.5-9B-GGUFOpen weightsUnslothestimated: 9.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.95B | 8bit | 12.3 GB | +65.7 GB | ✓ fits | |
| Qianfan-VL-8BOpen weightsBaiduestimated: 8.8B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.81B | 8bit | 12.1 GB | +65.9 GB | ✓ fits | |
| internlm3-8b-instructOpen weightsInternLM (Shanghai AI Laboratory)estimated: 8.8B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.8B | 8bit | 12.1 GB | +65.9 GB | ✓ fits | |
| granite-4.1-8bOpen weightsIBMestimated: 8.8B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.79B | 8bit | 12.1 GB | +65.9 GB | ✓ fits | |
| amd/Qwen3-VL-8B-Instruct-w8a8-llmcompressorOpen weightsAMDestimated: 8.8B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.77B | 8bit | 12.1 GB | +65.9 GB | ✓ fits | |
| Qwen3 VL 8B InstructOpen weightsQwenestimated: 8.8B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.77B | 8bit | 12.1 GB | +65.9 GB | ✓ fits | |
| VibeVoice-ASROpen weightsMicrosoftestimated: 8.7B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.67B | 8bit | 12.0 GB | +66.0 GB | ✓ fits | |
| Molmo2-8BOpen weightsAllen Institute for AIestimated: 8.7B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.66B | 8bit | 12.0 GB | +66.0 GB | ✓ fits | |
| aya-vision-8bRestrictedCohereestimated: 8.6B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.63B | 8bit | 11.9 GB | +66.1 GB | ✓ fits | |
| Intern-S1-miniOpen weightsInternLM (Shanghai AI Laboratory)estimated: 8.5B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.54B | 8bit | 11.8 GB | +66.2 GB | ✓ fits | |
| GKA-primed-HQwen3-8B-ReasonerOpen weightsAmazon Web Servicesestimated: 8.5B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.5B | 8bit | 11.8 GB | +66.2 GB | ✓ fits | |
| GDN-primed-HQwen3-8B-InstructOpen weightsAmazon Web Servicesestimated: 8.5B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.5B | 8bit | 11.8 GB | +66.2 GB | ✓ fits | |
| LFM2.5-8B-A1BOpen weightsLiquid AIestimated: 8.5B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.47B | 8bit | 11.7 GB | +66.3 GB | ✓ fits | |
| allenai/olmOCR-2-7B-1025-FP8Open weightsAllen Institute for AIestimated: 8.3B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.29B | 8bit | 11.5 GB | +66.5 GB | ✓ fits | |
| UI-TARS 7BOpen weightsByteDanceestimated: 8.3B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.29B | 8bit | 11.5 GB | +66.5 GB | ✓ fits | |
| olmOCR-2-7B-1025Open weightsAllen Institute for AIestimated: 8.3B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.29B | 8bit | 11.5 GB | +66.5 GB | ✓ fits | |
| Qwen2.5-VL-7B-InstructOpen weightsQwenestimated: 8.3B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.29B | 8bit | 11.5 GB | +66.5 GB | ✓ fits | |
| UI-TARS-7B-DPOOpen weightsByteDanceestimated: 8.3B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.29B | 8bit | 11.5 GB | +66.5 GB | ✓ fits | |
| UI-TARS-7B-SFTOpen weightsByteDanceestimated: 8.3B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.29B | 8bit | 11.5 GB | +66.5 GB | ✓ fits | |
| Stable-DiffCoder-8B-InstructOpen weightsByteDanceestimated: 8.3B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.25B | 8bit | 11.5 GB | +66.5 GB | ✓ fits | |
| Seed-Coder-8B-ReasoningOpen weightsByteDanceestimated: 8.3B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.25B | 8bit | 11.5 GB | +66.5 GB | ✓ fits | |
| Stable-DiffCoder-8B-BaseOpen weightsByteDanceestimated: 8.3B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.25B | 8bit | 11.5 GB | +66.5 GB | ✓ fits | |
| Seed-Coder-8B-BaseOpen weightsByteDanceestimated: 8.3B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.25B | 8bit | 11.5 GB | +66.5 GB | ✓ fits | |
| Seed-Coder-8B-InstructOpen weightsByteDanceestimated: 8.3B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.25B | 8bit | 11.5 GB | +66.5 GB | ✓ fits | |
| Qwen3 8BOpen weightsQwenestimated: 8.2B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.19B | 8bit | 11.4 GB | +66.6 GB | ✓ fits | |
| DeepSeek-R1-0528-Qwen3-8BOpen weightsDeepSeekestimated: 8.2B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.19B | 8bit | 11.4 GB | +66.6 GB | ✓ fits | |
| mlx-community/Qwen3-8B-4bitOpen weightsMLX Communityestimated: 8.2B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.19B | 8bit | 11.4 GB | +66.6 GB | ✓ fits | |
| granite-guardian-3.3-8bOpen weightsIBMestimated: 8.2B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.17B | 8bit | 11.4 GB | +66.6 GB | ✓ fits | |
| granite-3.0-8b-instructOpen weightsIBMestimated: 8.2B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.17B | 8bit | 11.4 GB | +66.6 GB | ✓ fits | |
| stable-diffusion-3.5-largeRestrictedStability AIestimated: 8.1B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.15B | 8bit | 11.4 GB | +66.6 GB | ✓ fits | |
| ERNIE-Image-TurboOpen weightsBaiduestimated: 8.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.03B | 8bit | 11.2 GB | +66.8 GB | ✓ fits | |
| ERNIE-ImageOpen weightsBaiduestimated: 8.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.03B | 8bit | 11.2 GB | +66.8 GB | ✓ fits | |
| NousResearch/Hermes-3-Llama-3.1-8B-GGUFOpen weightsNous Researchestimated: 8.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.03B | 8bit | 11.2 GB | +66.8 GB | ✓ fits | |
| Hermes-3-Llama-3.1-8BOpen weightsNous Researchestimated: 8.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.03B | 8bit | 11.2 GB | +66.8 GB | ✓ fits | |
| NousResearch/Meta-Llama-3-8BOpen weightsNous Researchestimated: 8.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.03B | 8bit | 11.2 GB | +66.8 GB | ✓ fits | |
| Llama-3.1-8B-InstructRestrictedMeta AIestimated: 8.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.03B | 8bit | 11.2 GB | +66.8 GB | ✓ fits | |
| Meta-Llama-3-8BRestrictedMeta AIestimated: 8.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.03B | 8bit | 11.2 GB | +66.8 GB | ✓ fits | |
| NousResearch/Meta-Llama-3.1-8B-InstructOpen weightsNous Researchestimated: 8.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.03B | 8bit | 11.2 GB | +66.8 GB | ✓ fits | |
| NousResearch/Meta-Llama-3-8B-InstructOpen weightsNous Researchestimated: 8.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.03B | 8bit | 11.2 GB | +66.8 GB | ✓ fits | |
| NousResearch/Meta-Llama-3.1-8BOpen weightsNous Researchestimated: 8.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.03B | 8bit | 11.2 GB | +66.8 GB | ✓ fits | |
| Llama 3.1 8BRestrictedMeta AIestimated: 8.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.03B | 8bit | 11.2 GB | +66.8 GB | ✓ fits | |
| Salesforce/Llama-xLAM-2-8b-fc-rOpen weightsSalesforceestimated: 8.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.03B | 8bit | 11.2 GB | +66.8 GB | ✓ fits | |
| Hermes-2-Theta-Llama-3-8BOpen weightsNous Researchestimated: 8.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.03B | 8bit | 11.2 GB | +66.8 GB | ✓ fits | |
| mlx-community/Llama-3.1-8B-Instruct-4bitOpen weightsMLX Communityestimated: 8.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.03B | 8bit | 11.2 GB | +66.8 GB | ✓ fits | |
| Meta-Llama-3-8B-InstructRestrictedMeta AIestimated: 8.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.03B | 8bit | 11.2 GB | +66.8 GB | ✓ fits | |
| Hy-MT2-7BOpen weightsTencentestimated: 8.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.03B | 8bit | 11.2 GB | +66.8 GB | ✓ fits | |
| aya-23-8BRestrictedCohereestimated: 8.0B params × 1.0 B × 1.15 + KV cache for 32768 tokens | 8.03B | 8bit | 11.2 GB | +66.8 GB | ✓ fits |
Headroom = device memory − 2 GB reserve − estimated need. Sorted by the API (fitting models first). Compare the models you shortlist with the + buttons. Devices with these memory sizes: hardware listing.