Llama 3.1 to Llama 4: A Migration Guide for Business

TL;DR: Llama 3.1 to Llama 4 for a normal business

If you run a 10 to 200 person firm, here is the short version, and it is not what most vendors will tell you.

• Model choice is the least important decision you will make. The workflow around the model, the data plumbing, the evaluation, and the human approval gates are where the value and the risk live. An implementation partner earns its keep there, not on picking a model.
• Llama 4 is a Mixture-of-Experts (MoE) model. Both released variants activate 17 billion parameters per token: Scout is 17B active out of ~109B total (16 experts), Maverick is 17B active out of ~400B total (128 experts). Only a fraction of the model runs per token, which is what makes MoE cheaper to serve than a dense model of the same total size.
• For almost every SMB, self-hosting Llama makes no economic sense. The point where owning GPUs beats per-token API pricing sits in the billions of tokens per month, roughly a thousand times more than a 40-person agency will ever generate. A hosted API, or a managed open-weight endpoint, is almost always the right answer.
• Open weights become right for an SMB under specific conditions: a hard data-residency or client-confidentiality obligation, a carrier or regulator contract that forbids third-party processing, or genuinely high volume in one narrow repetitive task. Even then, the answer is usually a managed open-weight endpoint, not a server in a closet.
• Do not choose a model from a benchmark table. Llama 4's own launch is the cautionary tale (see below). Choose with a small evaluation on your own data.

A note on scope. This page was originally about Llama 3.1 and is now mostly about the move to Llama 4, so read it as a migration guide: what changed, whether it matters for you, and how to decide. The accounting and insurance sections below are general information, not legal, tax, or professional advice; a licensed professional must review anything client-facing in those fields.

What changed from Llama 3.1 to Llama 4

Llama 3.1 launched in July 2024 as Meta's most capable open-weight model. The 405B dense model competed with the GPT-4 generation, and the smaller 8B and 70B variants became the base for thousands of production systems. For about nine months it was the open-source default.

Llama 4 arrived in April 2025 and changed the architecture. The headline shift is dense to Mixture of Experts (MoE). In a dense model like Llama 3.1 405B, all 405 billion parameters run for every token. In an MoE model, the network is split into many "experts," and a router sends each token to only a couple of them. Llama 4 Scout and Maverick each activate about 17 billion parameters per token even though their totals are far larger. That is the whole efficiency trick: near-large-model quality at small-model serving cost.

For a business, the practical takeaways are simpler than the architecture:

• If you are on Llama 3.1 today and it works, you do not have to move. "It still works" is a valid reason to stay.
• If you are choosing now, Llama 4's efficiency and long context are real improvements, but they only matter if your workflow needs them.
• The migration effort is rarely the model swap. It is re-validating your prompts, your evaluation, and your guardrails against the new model's behavior.

The Llama 4 models: Scout, Maverick, and the one that never shipped

• Llama 4 Scout (meta-llama/Llama-4-Scout-17B-16E-Instruct): 17B active / ~109B total, 16 experts. Meta positions it as fitting on a single H100 with Int4 quantization, and advertises a very large context window (up to 10 million tokens as a headline figure; usable context in practice is bounded by your hardware memory, see the deployment section). Best thought of as the efficient workhorse.
• Llama 4 Maverick (meta-llama/Llama-4-Maverick-17B-128E-Instruct): 17B active / ~400B total, 128 experts. The higher-quality variant for reasoning and multimodal work, still open-weight, heavier to serve.
• Llama 4 Behemoth: previewed as a ~2-trillion-parameter model, delayed repeatedly through 2025, and never publicly released. By 2026 Meta's frontier work had moved toward closed-weight models. Treat Behemoth as vaporware for planning purposes, do not architect around a model you cannot download.

Both released models are natively multimodal (text and image in) and open-weight under Meta's community license. "Open-weight" means you can download and run the weights; it is not the same as a fully open-source license, and there are use restrictions worth reading if you are at real scale.

Why you should not choose a model from a benchmark table

Here is the most useful thing this page can tell you, using Llama 4's own launch as the example.

At release, Meta submitted a version of Maverick to the LMArena leaderboard (a site that ranks models by blind human preference). That version, labeled Llama-4-Maverick-03-26-Experimental and "optimized for conversationality," scored an ELO of 1417 and placed second, just behind Gemini 2.5 Pro. Impressive, except the version that ranked so highly was not the one Meta released. When the public weights were tested on the same leaderboard, they landed around 32nd, below models that were already months old. LMArena publicly stated that Meta's interpretation of its submission rules "did not match what we expect from model providers," and it changed its policy in response (April 7 to 8, 2025).

The lesson is not "Meta bad." The lesson is that a benchmark number can be true and useless at the same time. A score comes from a specific version, a specific prompt style, and a specific test that may look nothing like your work. Benchmarks like MMLU and HumanEval are also largely saturated, top models cluster within noise, so a table that ranks them to two decimal places is measuring the test, not the models.

The practical alternative: a small evaluation harness on your own data

For a business, the right way to pick a model is to test candidates on your actual work. You do not need an ML team for this. This is a HowTo you can run in an afternoon:

1. Collect 50 to 200 real examples from the workflow you want to automate: real invoices, real support tickets, real listing descriptions, paired with the correct or acceptable output.
2. Write down your acceptance criteria in plain language. What does "good enough" mean? For an invoice, maybe "correct vendor and amount, correct GL category, flags anything ambiguous."
3. Run each candidate model (Scout, Maverick, a hosted frontier API, whatever you are considering) on the same examples with the same prompt.
4. Score the outputs against your criteria. A simple pass/fail per example plus a note on failures is enough to start.
5. Pick the cheapest model that clears your bar, and keep the harness. Re-run it whenever a model updates or your data drifts.

Fifty labeled examples from your own workflow will tell you more than every benchmark table on the internet combined. Building this harness is exactly the work an implementation partner should be doing with you, and it is the honest core of a model decision.

The honest self-host question for an SMB

Every vendor selling GPUs or "private AI" wants you to believe that self-hosting an open model saves money. For a small or mid-sized firm, it almost never does. Here is the math, with current figures.

Self-hosting is a fixed cost. To serve Llama 4 Scout or Maverick you rent GPUs by the hour whether you use them or not:

A hosted API is a variable cost. You pay per token: Llama 4 Scout runs roughly $0.10 per million input tokens and $0.30 per million output; Maverick roughly $0.20 and $0.60 (provider pricing, verify current rates). A firm processing a few million tokens a month pays single-digit dollars. Even 50 million tokens a month is tens of dollars.

Now the crossover. To justify the cheapest single-GPU box above ($1,360/month) against API pricing ($0.30 per million tokens), you would need to be generating on the order of 4 to 5 billion tokens a month. A 40-person insurance agency generates a rounding error against that. Self-hosting to "save money" at SMB volume means buying a full-time infrastructure problem, on-call, security patching, GPU availability, to save nothing.

So when is open-weight the right call for an SMB?

Three honest conditions:

1. A hard confidentiality or data-residency obligation. A carrier contract, a client agreement, or a regulation that forbids sending data to a third-party API.
2. A regulator or contract that bans third-party processing of the specific data involved.
3. Genuinely high volume in one narrow, repetitive task, where per-token pricing does add up.

And even when one of these applies, the answer is usually the third path: a managed open-weight endpoint (Amazon Bedrock, Google Vertex AI, Azure AI Foundry, or an inference provider). You get open weights and data controls without running a cluster. Open weights, someone else's operational headache. That is the option most SMBs actually want when they say "we need to self-host."

SMB deployment patterns by vertical

These are illustrative deployment patterns, not case studies. They describe what these workflows look like in a typical small firm. They do not reference specific Gaper clients, and they contain no invented metrics or outcomes.

Marketing agency: client reporting and campaign QA

Firm profile: 15 to 60 people, running paid and lifecycle campaigns across Google, Meta, and a few analytics tools. Before: an account manager pulls performance data across platforms every month, writes client-ready commentary, and eyeballs for anomalies, hours per client. After: data pull and first-draft commentary become agent-drafted; anomaly flagging is agent-owned; the client-facing narrative is human-owned. The approval gate: a human reads every client-facing paragraph before it sends. Model choice: a hosted frontier API, because prose quality for client-facing writing matters more than saving fractions of a cent, and the volume is low and the stakes are commercial, not regulatory. What breaks: a confident narrative attached to a metric the agent misread, which is why a person reads the paragraph. What it takes: connectors to the ad and analytics platforms, a brand-voice sample, a couple of weeks to first value. This is squarely AI agents for marketing territory.

Accounting and bookkeeping firm: categorization and close prep

Firm profile: 10 to 80 people, running QuickBooks or NetSuite for a book of clients. Before: staff categorize transactions, chase documents, and prep the month-end close by hand. After: categorization and document intake become agent-owned behind a confidence threshold; close entries are agent-drafted, human-approved; anything material stays human-owned. The approval gate: an exception queue plus a reviewer sign-off before books close. Model choice: a managed open-weight endpoint, because client financial data carries a real confidentiality obligation, and a managed endpoint gives you data controls (and, where needed, a business-associate-style agreement) without a GPU in the office. What breaks: a silent misclassification that compounds across a quarter and surfaces at review, the reason the exception queue exists. What it takes: ledger integration, a labeled sample of the firm's own transactions, licensed review of the workflow. See AI agents for accounting; a finance agent like AccountsGPT fits this shape.

Insurance agency or broker: submission and document intake

Firm profile: 10 to 100 people, processing submissions, ACORD forms, carrier documents, renewals, and certificates. Before: CSRs re-key data from PDFs into the agency management system and assemble renewal and certificate paperwork. After: intake and field extraction become agent-owned; renewal and certificate drafts are agent-drafted, human-approved; anything touching coverage terms is human-owned. The approval gate: hard-coded field validation against the source document, plus a licensed producer signing off on anything that touches coverage limits, effective dates, or terms. Model choice: a hosted API is fine for extraction, but if a carrier contract forbids third-party processing of insured data, move to a managed open-weight endpoint in the agency's own cloud, a data-residency decision, not a cost one. What breaks: a hallucinated coverage limit or effective date reaching a binding document, which is an errors-and-omissions exposure, not an inconvenience. This is the vertical where the discipline below (what we refuse to automate) matters most. What it takes: AMS integration, the carrier's document formats, and legal review of the validation rules.

Real estate brokerage or property management: listings, disclosures, and tenant communication

Firm profile: a brokerage or a property manager with a few dozen doors to a few thousand. Before: agents prepare listing and disclosure packages, abstract leases, triage maintenance requests, and draft tenant communication by hand. After: lease abstraction and maintenance triage become agent-owned; listing copy, disclosures, and tenant messages are agent-drafted, human-approved; anything public-facing is human-owned. The approval gate: a compliance review on all public-facing and tenant-facing language. Model choice: a hosted API, because volume is low and the risk is not in where the model runs, it is in what it writes. What breaks: fair-housing exposure in a generated listing or tenant message, a legal risk with real penalties that an LLM can produce with no obvious signal that anything is wrong. That is precisely why every public-facing line gets human compliance review.

Notice these reach different answers: hosted frontier API for marketing, a managed open-weight endpoint for accounting, a conditional move to a private endpoint for insurance, and a plain hosted API for real estate. If a vendor tells all four that the answer is "self-host Llama," they are selling you infrastructure, not solving your problem.

What we refuse to automate

Naming the lines is what makes the rest trustworthy, and in these verticals the lines are not optional:

• No coverage terms without a licensed producer. In insurance, anything touching a limit, an effective date, or policy language is human-owned and human-signed.
• No public-facing or tenant-facing copy without compliance review. Fair-housing and advertising rules apply to what the model writes, not to who wrote it.
• No books closed on unreviewed AI categorization. An exception queue and a human sign-off stand between the agent and the ledger.
• No hallucinated value reaching a document of record. Field validation against the source is hard-coded, never model-judged.

How to actually deploy Llama 4 (for the technical reader)

This section is for whoever on your team, or your implementation partner, sets it up. If that is not you, skip to the FAQ.

Local testing with Ollama

Good for trying the model on a laptop before committing. Note that even quantized, Scout is a large download and needs substantial memory; a laptop is for testing, not production.

# Install Ollama (macOS/Linux)
curl -fsSL https://ollama.com/install.sh | sh

# Pull and run Llama 4 Scout (verify the exact tag against ollama.com/library/llama4)
ollama run llama4:scout

# Serves an OpenAI-compatible API at localhost:11434; point your OpenAI SDK at that base URL.

Self-serving with vLLM

For teams that have a genuine reason to self-host (see the conditions above). Verified against vLLM's Llama 4 support notes; check the current docs, as flags change.

pip install vllm

# Serve Llama 4 Scout. Corrected from the earlier version of this article:
# real HF model ID, and max-model-len set to a realistic value, not 128K on a
# "10M context" claim you cannot fit in memory anyway.
python -m vllm.entrypoints.openai.api_server \
  --model meta-llama/Llama-4-Scout-17B-16E-Instruct \
  --tensor-parallel-size 8 \
  --dtype bfloat16 \
  --max-model-len 131072 \
  --port 8000

# Notes:
# - The advertised 10M-token context is memory-bound. Set --max-model-len to what
#   your hardware and workflow actually need; 128K is already large for most SMB work.
# - Meta reports Scout fits on a single H100 with Int4 quantization. Full bfloat16
#   serving needs more GPUs (hence tensor-parallel-size); size this to your setup.

Managed open-weight endpoints (what most teams should use)

If you need open weights but not a cluster, run Llama 4 on Amazon Bedrock, Google Vertex AI, Azure AI Foundry, or an inference provider. You get an OpenAI-compatible endpoint, data controls, and no GPUs to babysit. For most of the SMB conditions above, this is the deployment path.

Fine-tuning

Most SMBs do not need to fine-tune; good prompting and retrieval over your own documents usually get you there faster and cheaper. If you genuinely need the model to learn a format or a domain vocabulary, fine-tune a small variant, and read RAG vs fine-tuning first, because retrieval is often the right answer instead.

How Gaper shows up

Gaper is an AI-native implementation partner. For an SMB, the work is not "which model," it is everything around it:

• Choosing the deployment path honestly. Sometimes that means telling you not to self-host, and not to over-buy. If a hosted API is right, we say so.
• Building the evaluation harness on your data so the model choice is evidence, not a benchmark screenshot.
• Wiring the workflow into the tools you already use (your ledger, your AMS, your ad platforms, your PMS), with a quarterly cadence to re-check quality as data and models drift.
• Setting the approval gates the vertical requires, and serving it behind an OpenAI-compatible layer so you are never locked to one model.
• Handing you something you own. Not rented engineers, a governed workflow your team runs.

Where we have a product it maps cleanly: AccountsGPT for finance and bookkeeping workflows, Stefan for marketing operations. Where we do not, we build a supervised agent to fit. For the build decision, see build vs buy AI agents, AI agent development cost, and AI agent ROI; on keeping client data governed, AI agent data privacy for enterprises; and for the broader path, how to become AI-native.

Book a free AI assessment and we will tell you honestly whether you need Llama at all, and map the one workflow worth automating first. Free, 30 minutes, no obligation. Or talk to an AI architect.