Which open-source AI model should you use in 2026? For a manageable first evaluation, shortlist Qwen3.8-27B and Gemma 4 for multimodal applications, gpt-oss-20b for local text reasoning, and Olmo 3.1 when training transparency matters. For infrastructure-scale coding and agent workloads, investigate DeepSeek-V4.1-Flash, GLM-5.3-Flash, Kimi K3, MiniMax-M3, and Mistral Small 4. Their licenses, memory requirements, and deployment complexity differ substantially.

There is no single winner across every task. A model that solves difficult repository issues can be unnecessarily expensive for extracting invoice fields. A compact model that runs locally can be the better product choice even when a larger model leads a public benchmark.

Research checked: October 8, 2026. This guide compares published model cards, technical documentation, and release announcements. Recommendations are an editorial shortlist for evaluation, not results from a hands-on benchmark performed for this article. It focuses on language, reasoning, coding, and multimodal understanding; dedicated image-generation, music, and speech-synthesis models deserve separate comparisons. If the terminology is new, start with my beginner’s guide to AI and large language models.

Open-source AI vs. open-weight models: what is the difference?

People often search for “open-source AI models” when they mean models whose trained weights can be downloaded and run independently. That is useful shorthand, but it hides important distinctions.

Open weights give you access to the learned parameters. Depending on the license, you may be able to run inference, adapt the model, and redistribute derivatives. They do not automatically give you the original training data or a reproducible training pipeline.

The Open Source Initiative’s Open Source AI Definition goes further: it requires freedoms to use, study, modify, and share, together with the preferred form for modification, including parameters, code, and sufficient training-data information. A permissive license on weights alone does not establish that the complete AI system meets this definition.

For practical selection, separate three questions:

  1. Can I download and self-host this exact checkpoint? An API announcement is insufficient.
  2. Does its license permit my intended use? Internal experimentation, selling an assistant, and operating a model API may have different terms.
  3. How much of the training process can I inspect or reproduce? Final weights, training recipes, intermediate checkpoints, and data documentation offer different levels of transparency.

This article therefore includes both permissively licensed weights and prominent models under custom licenses, while labeling them explicitly. Olmo receives separate attention because training openness is central to its design.

Best open-source and open-weight AI models: comparison table

The following table is a starting point for selecting candidates, not a ranked leaderboard. Model names link to the developer’s exact checkpoint or official family documentation. On a phone, scroll the table horizontally to inspect every column.

Model or familyParameter scalePublished contextInput capabilitiesWeight license
Qwen3.8-27B27B language model262,144 native; extensible to 1MText, images, videoApache 2.0
Gemma 4E2B, E4B, 12B, 26B A4B, 31B128K small; 256K mediumText, images, video; audio on selected sizesApache 2.0
gpt-oss21B / 3.6B active; 117B / 5.1B activeCheck model and serving configurationTextApache 2.0
DeepSeek-V4.1-Flash552B backbone plus memory/components; 8B active prefill / 16B decodingUp to 1MText, imagesMIT
GLM-5.3-Flash320B total / 18B active1,048,576 in configurationNative multimodalMIT
Mistral Small 4119B total / 6.5B active, model-card convention256KText, imagesApache 2.0
Kimi K32.8T total / 104B active1,048,576Text, images, videoCustom Kimi K3
MiniMax-M3Approximately 428B / 23B active1MText, images, videoCustom MiniMax Community
Qwen3.8-2.4T-A95B2.4T total / 95B active262,144 native; extensible to 1,010,000Text in the released checkpointCustom Qwen3.8-Max
Olmo 3.1 32B Think32BCheck checkpoint configurationTextApache 2.0

How to read these numbers: B means billion and T means trillion parameters. Mixture-of-experts (MoE) models activate a subset per token, but their full weights still need storage and a suitable memory/offloading plan. Parameter totals can include different embedding, vision, memory, and prediction components. Context is measured in tokens, not words; advertised extensions may require specific settings and additional memory. A provider can expose a smaller limit than the underlying model.

The leading model families, explained

Qwen3.8: compact multimodality and a much larger flagship

Qwen3.8-27B is a practical candidate for developers who want image and video understanding alongside text, coding, and reasoning. Its native context is 262,144 tokens; the advertised million-token capacity is an extension, not the default native window. This particular checkpoint uses Apache 2.0. It was released in August 2026. See the Qwen3.8-27B model card and official release history.

The much larger Qwen3.8-2.4T-A95B has downloadable weights, but its released checkpoint is text-only and always uses thinking. Hosted Qwen3.8-Max features such as vision or built-in service tools should not be attributed to these weights. Its custom license includes separate authorization for certain model-service or independent coding/office-assistant businesses above a $50M twelve-month revenue threshold. Inspect the checkpoint card and Qwen3.8-Max license for the full conditions.

Qwen3.8-Flash-Next is an experimental architectural preview, rather than a complete Qwen4 release. Unlike the 27B checkpoint, its custom license requires separate authorization for commercial model-service or independent coding/office-assistant businesses, without the flagship’s $50M threshold; an internal-use exemption exists. See its model card and Qwen Community license.

Gemma 4: a family spanning edge devices and larger machines

Gemma 4 provides E2B, E4B, 12B, 26B A4B, and 31B variants. All support visual inputs; native audio is available on E2B, E4B, and 12B, rather than every size. The smaller models have 128K context, while medium variants support 256K. Video understanding means processing visual frames, not generating a video. Google’s Gemma 4 overview explains these differences.

The 26B A4B variant is an MoE with approximately 25.2B total and 3.8B active parameters, using Apache 2.0. The “effective” count in E2B/E4B also differs from the full stored parameter count. Check the exact Gemma 4 checkpoint card rather than estimating memory from a short product name.

For a local application, evaluate the smaller variants first. For document and image workflows, compare the appropriate Gemma checkpoint with Qwen3.8-27B using the same inputs and output requirements.

gpt-oss: configurable text reasoning with local options

OpenAI’s gpt-oss-20b and gpt-oss-120b are open-weight text models under Apache 2.0. Their actual parameter counts are approximately 21B and 117B, with 3.6B and 5.1B active respectively. They support configurable reasoning effort and tool-oriented workflows. They are separate downloadable models, rather than the proprietary models served in ChatGPT. See the 20b model card and 120b model card.

The developer describes the supplied MXFP4 configuration as running 20b within 16GB of memory and 120b on an 80GB GPU. Treat these as documented deployment targets, not a promise for every context length and batch size. The required Harmony conversation format is another important integration detail: use a runtime that implements it correctly.

Start with 20b when local text reasoning is the priority. For scanned documents or image analysis, choose a vision-capable model or add a separate extraction stage.

DeepSeek-V4.1-Flash: long-context reasoning at infrastructure scale

Released in September 2026, DeepSeek-V4.1-Flash combines text and image understanding with up to a million-token context. Its MIT-licensed architecture uses a 552B backbone, additional Engram memory, and other components; Hugging Face lists approximately 763 billion stored parameters for the released checkpoint. Active counts differ between prefill and decoding. The model card, technical report, and release changelog describe the design.

This is a candidate for substantial engineering and input-heavy agent systems. It is not a laptop-sized model because only a small subset activates per token. Evaluate its reasoning modes, serving implementation, and long-input latency with your workload before budgeting a deployment.

GLM-5.3-Flash: native multimodality and agent-oriented inference

GLM-5.3-Flash is a native multimodal GLM-5-series release with approximately 320B total and 18B active parameters under MIT. Its configuration specifies 1,048,576 positions. The architecture combines sparse and linear attention, and reasoning settings allow different effort levels. See the GLM-5.3-Flash card, configuration, and developer explanation.

Consider it alongside DeepSeek and Mistral for tool-using applications where throughput and multimodal inputs both matter. Verify support for your exact media format in the serving stack. A model’s native capability does not ensure that every third-party endpoint exposes it.

Mistral Small 4: reasoning, vision, and coding in one checkpoint

Mistral Small 4, announced in March 2026, combines instruction following, reasoning, coding-agent capabilities, and image understanding. The Apache-licensed checkpoint has 119B total parameters and a 256K context window. Its model card reports 6.5B active; the announcement uses different counting conventions for active parameters.

“Small” is a family name, not a hardware guarantee. It still needs much more weight memory than a dense 7B model. Shortlist it when a unified model and permissive weight license are useful, then measure the cost of your reasoning and visual workloads.

The larger Mistral Large 3 remains another downloadable Apache-licensed option, with 675B total, 41B active parameters, and 256K context.

Kimi K3: a very large multimodal model with custom terms

Kimi K3 offers text, image, and video inputs with a 1,048,576-token context. Its approximately 2.8T total and 104B active parameters put it firmly in the infrastructure-scale category. The Kimi K3 card and developer article describe its long engineering and knowledge-work focus.

Its downloadable weights use a custom license. That includes conditions for some Model-as-a-Service businesses and large commercial products; internal use and certain hosted access have explicit exemptions. Read the actual Kimi K3 license against your intended product.

A hosted evaluation can help establish whether the capability fits your tasks before planning a self-hosted installation. Record the provider and version because hosted tooling can affect the result.

MiniMax-M3: long-context coding and office workflows

MiniMax-M3 combines text, images, and video with a million-token context. Its approximately 428B total and 23B active parameters, sparse attention, and enabled/adaptive/disabled reasoning modes make it another candidate for large coding and document workflows. See the MiniMax-M3 card and official announcement.

Its model-weight license is custom, even though some associated software has permissive licenses. Commercial terms include credit and notification requirements, with prior authorization above a stated revenue threshold. Consult the MiniMax Community license. Do not infer weight permissions from the license on a separate inference kernel.

Assess it with realistic office tasks: extracting an ambiguous table, reconciling several documents, and editing a presentation outline with explicit constraints.

Meta Llama: an established ecosystem with a custom license

Llama remains relevant for its established integrations and existing deployments. Meta’s official inventory lists Llama 4 Scout and Maverick as multimodal MoE models, alongside earlier text-focused Llama releases. The 17B figure used in their names refers to active scale, not the entire checkpoint; Scout and Maverick are much larger in total weight count. See Meta’s official model inventory.

Access involves Meta’s license and acceptable-use terms. Evaluate those conditions separately from Apache- or MIT-licensed alternatives. An established Llama deployment may be worth retaining if it already meets your quality, latency, and operating requirements; a newer release alone is not a migration case.

Olmo 3.1: training transparency as a selection criterion

Ai2’s Olmo 3.1 32B Think belongs to a family designed to support research into how language models are built. Alongside weights, Ai2 publishes training code, data resources, intermediate stages, and training details. Its Olmo overview and Olmo 3 release explanation explain the broader open training process.

The Olmo 3.1 32B Think card documents the specific reasoning checkpoint and recommended inference settings. Instruct and Think variants serve different purposes, so compare the appropriate version rather than treating the whole family as one model.

Choose this route when understanding or modifying the training process matters alongside inference quality. Transparency can be valuable even when another model offers better results on a particular application benchmark.

Which model should you choose for your use case?

Local assistants and private document workflows

Start with a checkpoint that fits your actual machine: smaller Gemma variants, gpt-oss-20b for text, or a quantized Qwen3.8-27B if your memory budget allows. Use a short representative prompt first, then increase document length. Check system RAM as well as VRAM when offloading to the CPU.

Local inference can keep inference inputs on your own hardware, but privacy also depends on the surrounding application. Embedding services, telemetry, document storage, browser tools, and logs can still transmit data. Map those paths before calling a complete system private.

For retrieval-augmented generation (RAG), test the retrieval pipeline independently. If it retrieves the wrong policy passage, a larger generation model may simply produce a more confident wrong answer. A good test includes an answerable question, a conflicting source, an outdated source, and a question the documents cannot answer.

Coding assistants and software agents

Compare Qwen3.8-27B and gpt-oss for approachable initial experiments, then evaluate larger DeepSeek, GLM, Mistral, Kimi, or MiniMax deployments when the workload justifies them. Include at least one task from your real codebase: a bug with a failing test, a feature spanning several files, and a refactor that must preserve behavior.

Score a coding agent on the complete result: tests passed, unintended files changed, new dependencies introduced, tool calls, and wall-clock time. A correct-looking code snippet is a weaker test than an executable change that resolves a real issue. If your app connects to tools through an API, my GraphQL guide provides related background on API design.

Multimodal document analysis

Match the input to the model. Image support does not necessarily mean native audio support; video support can mean selected frames rather than continuous audiovisual understanding. Gemma, Qwen, and the larger multimodal families offer different combinations.

Evaluate tiny text in screenshots, rotated scans, multiple-page tables, charts with similar colors, and screenshots containing misleading instructions. Require the output to identify the source page or region when possible. A model that summarizes a document fluently may still misread a single digit that matters.

Research, fine-tuning, and training experiments

Start with Olmo when data and training-process transparency are central. For application adaptation, permissively licensed Qwen3.8-27B, Gemma 4, gpt-oss, or Mistral checkpoints can also be candidates, subject to their exact artifacts and your infrastructure.

Before fine-tuning, establish a baseline with better prompting and retrieval. Fine-tuning can help with format, style, and task behavior; it does not automatically turn a model into a continuously updated factual database. Keep a separate held-out test set so improvements on training examples do not masquerade as general gains.

Hardware requirements: how much RAM or VRAM do you need?

The first memory estimate is simple:

Weight storage in bytes ≈ total stored parameters × bits per parameter ÷ 8

This is a lower-bound planning calculation. Quantization metadata, mixed-precision layers, vision encoders, runtime buffers, and the key/value (KV) cache add memory. Disk size and live inference memory are also different quantities.

Stored parameter countIdeal 4-bit weight payloadIdeal 8-bit weight payloadIdeal 16-bit weight payload
8B4 GB8 GB16 GB
14B7 GB14 GB28 GB
27B13.5 GB27 GB54 GB
120B60 GB120 GB240 GB
1T500 GB1,000 GB2,000 GB

These are decimal GB and weight payloads only, not recommended GPU capacities or measured model footprints. They assume a uniform precision that real checkpoints may not use. A 27B model with a theoretical 13.5GB payload can need appreciably more memory once loaded, especially with long prompts.

Three factors frequently change the deployment decision:

  • Context and concurrency: several simultaneous conversations retain more state than one short prompt. Budget the actual context and batch size you expect to serve.
  • MoE storage: active parameters estimate part of per-token computation, not the full weight footprint. Offloading experts trades accelerator memory for transfers and latency.
  • Backend compatibility: a quantized checkpoint needs support for its architecture, precision format, and hardware. A file fitting in RAM does not mean it will run fast enough.

Start at a modest context limit, measure peak memory, and increase it deliberately. Leave headroom for the operating system and serving stack. For Gemma specifically, Google’s memory-planning documentation provides checkpoint-aware estimates and explains the additional context cost.

How to deploy and evaluate an open-weight model

Choose a runtime that supports the exact checkpoint

For approachable local experimentation, review the Ollama quickstart. For control over local quantization and hardware backends, inspect llama.cpp. For concurrent server workloads, consult vLLM’s serving documentation.

Support changes quickly. Check the current architecture compatibility, chat template, quantization format, and media-input support before downloading hundreds of gigabytes. Prefer a documented checkpoint over an ambiguously named community upload, and record the artifact revision you use.

Run a small, explicit first request

For example, after installing a compatible vLLM environment and checking gpt-oss hardware requirements, the official gpt-oss card documents serving the exact checkpoint. A constrained local starting point is:

vllm serve openai/gpt-oss-20b --host 127.0.0.1 --port 8000 --max-model-len 4096

From a separate terminal with curl, send a short request. This example uses a POSIX shell; on Windows PowerShell use curl.exe with equivalent quoting, or send the same JSON through your preferred HTTP client.

curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"openai/gpt-oss-20b","messages":[{"role":"user","content":"Explain mixture-of-experts in three sentences."}],"max_tokens":512}'

The 4,096-token limit bounds this experiment; it does not describe the model’s maximum capacity. This is a documentation-based example, not a model inference run performed for this article. Runtime versions, GPU support, and available memory still determine whether it starts. Keep the initial server local; a public deployment also needs authentication, request limits, monitoring, and a deliberate network configuration.

Build a repeatable evaluation rather than trusting one demo

Create a small test pack before comparing models. Forty representative tasks can reveal more about your application than a broad leaderboard:

  1. Ten ordinary tasks: the common questions or transformations your users perform.
  2. Ten difficult tasks: longer inputs, ambiguous requirements, or multiple constraints.
  3. Ten failure cases: absent evidence, conflicting documents, malformed data, or misleading instructions.
  4. Ten operational checks: JSON validity, tool-call arguments, output length, language requirements, and recovery after an error.

Use identical inputs and the correct template for each model. Record model revision, precision, context limit, reasoning budget, sampling settings, system prompt, retrieval corpus, and tool permissions. Repeat unstable cases and inspect the actual failures rather than reporting only an average score.

For structured outputs, define success mechanically where possible. Does the JSON parse? Are required fields present? Does a quoted invoice total equal the source? For coding, run tests in an isolated environment. For open-ended writing, use a documented human rubric instead of silently treating another model’s judgment as ground truth.

Measure task success, time to first token, end-to-end latency, peak memory, and cost per successful task. The cheapest token price can still be expensive if failed attempts trigger repeated retries. Select the smallest deployment that consistently meets your acceptance criteria.

Benchmarks, long context, and licenses: common traps

Why benchmark scores do not establish one universal winner

A reported score depends on the benchmark version, prompt, reasoning budget, sampling, tool access, and agent scaffold. Comparing a bare model with a model-plus-tools agent can lead to the wrong conclusion. A longer reasoning run may improve accuracy while increasing latency and cost.

Use SWE-bench for repository-level evaluation context and LiveCodeBench for coding methodology. To rerun supported language-model evaluations with recorded settings, inspect the EleutherAI evaluation harness. Developer-reported numbers are useful signals, but your own task pack should decide deployment.

A million-token window is capacity, not guaranteed comprehension

Test whether a model can retrieve and reconcile evidence across a long input, including material in the middle. Measure whether it preserves exact numbers and follows instructions after irrelevant sections. Longer inputs can also increase prefill time and memory pressure.

Compare long-context prompting against selective retrieval. Feeding an entire archive is not automatically better than retrieving a few accurate passages. The right choice depends on whether the task needs broad cross-document synthesis or a precise answer from a small source region.

For a closer look at adapting an application to its own information and tasks, read my RAG vs fine-tuning comparison.

Commercial use depends on the exact license

Apache 2.0 and MIT are permissive software licenses, but notice and other applicable obligations still need attention. Custom weight licenses can introduce conditions based on revenue, product category, redistribution, or attribution. Review the full terms of the exact downloaded release.

Do not assume that all models from one organization share a license: Qwen3.8-27B, Qwen’s flagship, and Flash-Next demonstrate why checkpoint-level checking matters. Similarly, a permissive kernel license does not replace MiniMax-M3’s model license. Hosted-provider terms are another layer on top of the underlying weights.

Announced models are not always downloadable

As of the research date, Mistral Large 4 is in API preview with weights promised later in October. Reflection Beam is also announced with weights and supporting materials expected later. Both belong on a watchlist, rather than in a comparison of checkpoints you can download now.

A practical starting recommendation

Choose one accessible checkpoint, one realistic task, and one measurable definition of success. Add a second model only after you can explain what failed or what costs too much. For most application developers, that produces a better decision than downloading every new release.

Keep a short deployment record with the model revision, license, runtime, precision, and evaluation results. Revisit it when a release solves a demonstrated problem, when workload demands change, or when serving support improves. The strongest model for your product is the one that delivers reliable results within your operational constraints.

For the application layer around a tool-capable model, see my Python research-agent tutorial with citation checks and execution limits.

The model cards linked throughout this guide provide current specifications. The resources below build the underlying knowledge needed to evaluate and deploy models as the market changes. Books teach durable foundations rather than certifying today’s rankings.

Books

Use Read More to explore each book’s summary, topics, recommended background, and author biography. The Buy link opens the official publisher’s page.

Build a Large Language Model (From Scratch) By Sebastian Raschka
Build a Large Language Model (From Scratch)
By: Sebastian Raschka
Buy
Hands-On Large Language Models By Jay Alammar and Maarten Grootendorst
Hands-On Large Language Models
By: Jay Alammar and Maarten Grootendorst
Buy
AI Engineering By Chip Huyen
AI Engineering
By: Chip Huyen
Buy
Generative Deep Learning, Second Edition By David Foster
Generative Deep Learning, Second Edition
By: David Foster
Buy

For code and exercises accompanying Raschka’s book, visit the author’s LLM companion hub. Purchase, library, and subscription access vary.

Videos and courses

Select a play button to watch inside the site’s video modal. Start with 3Blue1Brown for visual foundations, follow Raschka for implementation, use Stanford CS336 for deeper study, and watch the DeepLearning.AI workshop for practical evaluation.

Attention in transformers, step-by-step — 3Blue1BrownAttention in transformers, step-by-step — 3Blue1Brown
Build an LLM from Scratch: Working with text data — Sebastian RaschkaBuild an LLM from Scratch: Working with text data — Sebastian Raschka
Stanford CS336: Overview and Tokenization, Spring 2025Stanford CS336: Overview and Tokenization, Spring 2025
Deep Dive into LLM Evaluation — DeepLearning.AIDeep Dive into LLM Evaluation — DeepLearning.AI

Continue with the 3Blue1Brown attention lesson and exercises, Raschka’s full LLM video series, or the official Stanford CS336 recordings and assignments.

For a structured course on the model lifecycle, adaptation, evaluation, and deployment, see Generative AI with Large Language Models from DeepLearning.AI and AWS. Course access and certificate options vary; the public evaluation workshop above is a separate resource.

Documentation, model sources, and evaluation tools

Frequently asked questions about open-source AI models