Choosing a Model

How to Choose an AI Model: A 7-Question Checklist

The best AI model for a project is the cheapest one that does your specific job well enough, fits your text sizes and comes from a provider you can rely on. Seven questions narrow the list, and a small test on 20–50 of your own examples makes the final call.

Cover reading "7 questions to pick your model" beside a ring of brass keys with one glowing violet key next to a lock (AI-generated illustration)
AI-generated illustration

Forty-two models on our board, nine providers and new launches almost every week. Where do you even start?

Below are seven questions to answer, in order. Each one cuts the list down. The figures come from the 42 models on our price board, as of 2026-11-21.

1. What Exactly Is the Job?

Write the task down in one sentence, with an example of a good answer. For instance: reply to customer emails about orders, in a friendly tone, using our help-center text. Or: turn a 30-page contract into a one-page list of dates and duties.

Different jobs reward different models:

  • Short, repetitive tasks (classifying, tagging, extracting fields, simple replies) often work well on the smallest, cheapest models.
  • Long documents need a large context window and careful instructions more than raw power.
  • Hard reasoning, tricky code or multi-step plans are where the larger, pricier models earn their cost.

If you can’t describe a good answer, no model choice will fix that. Start there.

Think of it like hiring: you write the job description before you read any CVs. A clear task description does the same job for choosing a model.

2. How Much Text Goes in and Out?

Every model has a context window: the most text it can take in one request, counted in tokens, including the conversation so far. On our board, 36 of the 42 models list a window of about 1 million tokens. The other six range from 128,000 tokens (Codestral 25.08) to 500,000 tokens (Grok 4.7). Context sizes are as listed on each model’s page on our board.

For most chat and email tasks, any current model has room to spare. The window matters when you send whole books, large code bases or long chat histories. Even then, check the price: some models charge more per token once a prompt passes a size limit, and the whole prompt is then billed at the higher rate.

Also estimate how much the model writes back. Output tokens cost about five times as much as input tokens at the median on our board, so long answers add up quickly.

3. What Can You Spend per Month?

Estimate your monthly tokens in and out, then price two or three candidates. Our guide How LLM API Pricing Works shows the sum step by step.

The gap between models is wide, even inside one provider’s line-up. Here are the cheapest and most expensive model for the five providers with the widest range on our board, using a blended price (three parts input to one part output, per 1M tokens):

Provider Cheapest on our board Most expensive on our board Gap
OpenAI GPT-6 Luna, $0.20 GPT-6 Astra, $20.00 100×
Anthropic Claude Haiku 5.5, $0.20 Claude Fable 5.2, $16.00 80×
Z.ai (Zhipu) GLM-5.3-Flash, $0.24 GLM-5.3, $2.15 9×
Google Gemini 3.5 Flash-Lite, $0.85 Gemini 4 Neon, $6.00 7×
Mistral AI Ministral 3 14B, $0.20 Mistral Large 4, $1.05 5×

The practical lesson: test the small model from a provider before assuming you need its flagship. If the small one handles 90% of requests well, you can send only the hard 10% to a bigger model.

4. Do You Need Open Weights?

Open-weight models publish their trained weights, so you (or a hosting company) can run them on your own hardware. That gives you more control over data and the model version, but you take on hosting work.

On our board, 12 of 42 models are marked open-weight. They come from Mistral AI, DeepSeek, Moonshot AI, Z.ai and MiniMax. The rest, including every OpenAI, Anthropic and Google model we list, are offered through the provider’s API or cloud partners. Licences change, so check each model’s own page.

Choose open weights if you must keep data inside your own systems, need a fixed model version for years, or expect very high volume. Otherwise, a hosted API is simpler. Our collection Open models you can run on your own computer lists sources that cover this world.

5. How Reliable Does It Need to Be?

Every provider has incidents. Our AI API status page lists 29 providers and gathers what most of them (26 on 2026-11-21) post on their official status pages, including how long each incident lasted.

Ask yourself what happens to your users when the model is unavailable for 30 minutes:

  • Nothing serious (an internal tool, a batch job that can wait): any reputable provider is fine.
  • Lost sales or angry customers: plan a fallback model from a second provider, and make sure your prompts work on both.

Reliability is also about consistency. Pinning a specific model version, where the provider allows it, protects you from silent changes in behaviour.

Did you know? Using models.dev release dates, 23 of the 42 models on our board were released in the 90 days before 2026-11-21.

6. How Long Will the Model Be Around?

Models are retired and prices change, sometimes within weeks. Two examples from official pages we read on 2026-11-21:

  • Moonshot AI says its K2 series is closed to new users from 2026-10-31 and will be retired by the end of December 2026. That includes Kimi K2.7 Code on our board.
  • Google lists the paid input price for Gemini 4 Carbon rising from $2.00 to $4.00 per 1M tokens on 2026-12-01, with output going from $10.00 to $20.00.

Before you build on a model, check its provider’s pricing and deprecation pages for dates like these. Keep your code ready to switch: store the model name in one setting, not scattered through the app. Our board keeps every price change with its date, so you can see how often a provider moves prices.

7. How Will You Test It?

Public benchmarks measure general skills on someone else’s tasks. Your own test tells you far more:

  1. Collect 20–50 real examples of your task, including a few hard ones.
  2. Write down what a good answer looks like for each, even briefly.
  3. Run the same prompt on two or three candidate models that passed questions 1–6.
  4. Score the answers yourself or with a teammate, without looking at which model wrote them.
  5. Record cost and response time next to quality.

Pick the cheapest model that meets your quality bar. Then re-run the same test whenever you change prompts or a provider releases a new version.

Copy-Ready Model Test Sheet

Example # Input (short description) Good answer looks like Model A score (1–5) Model B score (1–5) Notes
1
2
3

Which of the seven questions would rule out the model you use today?

Quick Decision Table

If your situation is… Start by testing…
High volume, simple task The cheapest small models on the price board, such as those in our cheap chatbot models collection
Very long documents Models with a 1M-token window, checking long-prompt price tiers
Strict data control Open-weight models you can host yourself
Hard reasoning or code One mid-priced and one flagship model, then compare on your own test
Can’t afford downtime Two models from different providers, with a fallback in your code

What This Means for You

  • Developers: build your 20–50 example test set first, and keep the model name in one setting so switching later is a one-line change.
  • Founders and team leads: choose on cost for acceptable quality on your own tasks, and plan for a second provider before you need one.
  • Curious beginners: try a few well-known AI chat apps with the kinds of questions you care about. The full checklist matters once you build something.

Bottom Line

Choosing a model is a narrowing process, not a hunt for a single best model. Define the job, check text sizes and budget, decide on open weights, plan for outages and retirements, then let a small test on your own examples make the final call. All figures here come from our price board as of 2026-11-21. Prices and models change, so check the board before you commit.

Next, price your shortlist with What Does One AI API Request Cost?. Want these updates by email? Sign up here, or follow new guides and data stories through our RSS feed.

Get the Weekly AI News Bay Email

The week’s biggest AI stories, model price changes and outages in one short email. The first issue goes out in early 2027; sign up now to get it.

Double opt-in: we email you a link to confirm. One-click unsubscribe in every email. Privacy policy.

Sources

Written by AI News Bay’s AI writers and checked by a separate AI auditor before publishing. Prices and uptime figures come from our own measurements. Spotted something out of date? Tell us.

Related on AI News Bay

Type a name, a topic or a format. Enter opens the full results.