When to Use Agentic AI – and When Not To (A Job-by-Job Matrix)

ByProf. H

Almost every question we get is a version of which AI is best. It is the wrong question, and it has a wrong answer waiting for anyone who asks it.

The right question is smaller and much more useful: for this particular job, does the machine need to answer, or does it need to act? That single fork decides which tool wins, how much it costs you, and how badly it can fail.

Here is the matrix we actually use, the physics underneath it, and the numbers – ours and other people’s – that put it there.

Matrix of five job types mapped to generative, agentic, or neither, with the reason for each
The whole article in one picture. The rest is why each row says what it says.

The one-line difference

Generative AI answers. You ask, it produces, it stops. You are in the loop on every turn, which means every mistake meets a human before it travels.

Diagram comparing generative AI as a single pass from user to model to answer, versus agentic AI as a plan-act-read-adjust loop
The difference in one picture: a straight line versus a circle. Cost, speed and failure modes all follow from which shape you chose.
Chart of our timed agent run: 76 seconds to a working draft, 6 minutes 43 seconds total, 6 of 7 features passing
Our own run, timed. The number that matters is not 76 seconds — it is that the two failures were invisible to reading the code.

Agentic AI acts. You give it a goal; it plans, does something, reads what came back, and decides the next move. The loop is the product. It is also the risk: mistakes now compound before anyone looks.

Everything below follows from that one structural difference. We wrote the longer definition in Generative AI vs Agentic AI; this page is about choosing.

The physics: why task length decides everything

The most useful idea we have read on this is Toby Ord’s, in Is there a half-life for the success rates of AI agents? He models agent failure as a constant hazard rate – a fixed probability of failing per minute of work.

If that is right, the consequence is brutal and clean: success declines exponentially with task length, and every agent can be described by a half-life, the task duration at which its success rate halves. Longer tasks are chains of subtasks, and failing any one link fails the chain.

Curve showing agent success rate decaying exponentially with task length, marked at one, two and three half-lives
One half-life in, you are at 50%. Three half-lives in, you are at 12.5%. Nothing about the model got worse – the task just got longer.

The empirical companion to this is METR’s time horizon measurement: the human-expert task length at which an agent succeeds half the time. It has been climbing quickly, and headlines have enjoyed that number enormously.

What METR itself says the number is not. Read their caveats and the picture changes shape: the measurement covers software, machine-learning and cybersecurity tasks, in low-context settings rather than professionals working inside a familiar project, and measurements above 16 hours are unreliable with the current task suite. Most importantly, METR states plainly that it does not measure how long an agent can act autonomously. A time horizon is a capability ceiling, not a permission slip.

What we measured ourselves

Two runs of our own, both timed, both written up in full elsewhere on this site.

The agent run. One brief, one coding agent, a real browser test at the end. From brief to a draft that opened and ran: 76 seconds. Total session: 6 minutes 43 seconds. Final file: 326 lines, zero external requests. Six of seven features worked on the first drive-through.

Then the two failures, which taught more than the six successes:

  • The code was not wrong, it was fragile. A navigation highlight used a detection band about 5% of the viewport tall. Reading the code, it looks correct. It only fails when a human actually scrolls. No amount of the agent re-reading its own work would have caught it.
  • Then our test lied. After the fix it still failed – because we were testing in a background browser tab, where animation frames stop and scroll events never fire. The logic had been right the whole time. We nearly had the agent rewrite working code to satisfy a broken measurement.

Full write-up: the timed agent benchmark, and the four-pass method that came out of it in how to verify AI-generated code.

The generative run. Same discipline, different mode: a six-line brief into a normal assistant, then one revision command. Brief to refined draft, 20 minutes; the cut took the piece from 610 words to 455 while keeping every fact and the one concrete scene. No loop, no autonomy – a human judging each turn. Method: how to write a blog post with AI.

The pattern across both: the agent won where a machine could check the machine – the browser either renders or it does not. The assistant won where the only meaningful check was a person deciding whether a sentence was any good. That is the entire decision rule, and everything in the matrix is a restatement of it.

The third data set: 27 people, same brief

Last semester we graded 27 student websites built with AI agents against a 100-point rubric. Average score: 50.5. The interesting number was not the average but the spread between the top seven and bottom seven students:

Category Top 7 vs bottom 7 What it means
Functionality 3.6× Whether the thing worked was where people separated
Design 3.2× Judgment, still human
Accessibility 2.6× Nobody does what the rubric does not force
Written content 1.7× The model levelled the prose almost completely

Read the top and bottom rows together and you get the sentence this whole page exists to deliver: AI levels the writing. It does not level whether the thing works. Which is exactly why the two modes divide the way they do – and why the gap between people is now mostly a gap in verification.

The matrix, explained

The job Use Because Watch for
Drafting, rewriting, tone Generative The only real verifier is a person reading it. A loop adds nothing to judgment. Sameness. Your facts and your one real detail are what make it yours.
Research, summarising Generative, with sources The failure you fear is a confident wrong fact, which a loop will happily carry forward. Check the primary source yourself. Always.
Code you intend to run Agentic The machine can verify the machine – it runs, reads the error, retries. Verification is free. Fragile-but-plausible code. Drive it in a real browser, not a background tab.
Repetitive multi-step chores Agentic Same steps every time, cheap retries, checkable output. Keep each run short. Length is the enemy, per the half-life above.
Long open-ended projects Split, then agentic A four-hour goal is a chain of failures waiting to happen; four one-hour goals are not. If you cannot cut it into checkable pieces, it is not agent work yet.
Anything irreversible Neither, alone Payments, sends, deletions, anything with a legal or financial edge. Keep the final click human. Every time.

Four questions before you hand a job to an agent

  1. Can a machine tell whether it worked? If yes, agents are strong. If the only judge is taste, they are not.
  2. How long is the task, honestly? Every extra minute is another draw against the half-life. Cut it shorter than feels necessary.
  3. What does one failure cost? A wasted minute, or a sent email you cannot recall? The second is not agent territory.
  4. Will you actually check the result, or just glance at it? Be honest. The most common failure in all of this is a human who has quietly stopped reading.
Useful background on why every vendor is shipping agents at once. CNBC Television (3.39M subscribers), June 8, 2026. Verified Aug. 20, 2026.

What to do with this

This week, pick one task and route it deliberately:
1. Write the task down in one sentence. If the sentence has an “and” in it, split it.
2. Ask question one above. Machine-checkable goes to an agent; taste-checkable goes to an assistant.
3. Whatever you use, verify by running it, not by reading it.
4. Keep the last irreversible click for yourself.

Now price it

You have chosen the mode. The companion piece turns that into a monthly number — including the discounts most people never claim.

What this should cost you →

If you want to go deeper

“Will the ceiling keep rising?” METR’s horizon has been climbing quickly, and if it keeps climbing the matrix shifts rows downward over time. But the caveats do not move with it: the measurement is still software-shaped, still low-context, and still not a statement about safe autonomy.

Four questions to ask before delegating to an agent: machine-checkable, task length, cost of failure, will you check
Print this one. If any answer worries you, keep the work in the assistant and stay in the loop.

“What does all this cost me per month?” That is the other half of choosing, and it is now published: what AI really costs you per month – the same matrix, priced, including the discounts most people never claim.

“How do I actually verify agent output?” Four passes, including the one nobody does: suspect your own test. The verification method.

Frequently asked questions

When should you use agentic AI, and when not to?

Use an agent when the work is a loop it can close by itself: it can act, see the result, and correct. Code that runs, files that get sorted, repetitive multi-step chores with checkable output. Do not use one when the task is long and open-ended, when a single wrong step is expensive, or when the last action is irreversible – payments, sends, deletions. In those cases a generative assistant with you judging every turn is both safer and usually faster.

What is the actual difference between generative and agentic AI?

Generative AI answers: you ask, it produces text or an image, and it stops. Agentic AI is given a goal and runs a loop – it plans, takes an action, reads what came back, and decides the next step. The loop is the entire difference, and it is also where both the power and the failure modes come from.

Why do AI agents fail more on longer tasks?

Because failure compounds. Toby Ord’s analysis models agent failure as a constant hazard rate – a fixed chance of failing per minute of work – which produces an exponentially declining success rate as tasks get longer, so every agent effectively has a half-life. A long task is a chain of subtasks where failing any one fails the whole thing.

How long a task can an AI agent actually finish?

METR measures a 50% time horizon: the human-expert task length at which an agent succeeds half the time. It has been rising fast. But METR is explicit that this is measured on software, machine-learning and cybersecurity tasks, in low-context settings, and that measurements above 16 hours are unreliable with the current suite. It is not a measure of how long an agent can safely run unattended.

Are benchmark scores like SWE-bench a good guide?

Only as a ceiling. Scores are usually reported as a single run on a curated set. Reliability under repetition is far lower than headline pass rates, which is why an agent that scores in the eighties can still feel unreliable on your own work. Treat benchmarks as evidence that something is possible, not as a promise that it is dependable.

Which tasks are agents genuinely best at today?

Work where the verification is cheap and automatic. Writing and running code is the clearest case: the compiler or the browser tells the agent immediately whether it failed, so the loop closes without a human. That single property explains most of where agents succeed.

Do I need to pay for an agent to try this?

No. Start by handing the same task to a normal assistant and doing the loop yourself for one afternoon. If you find you are the only thing in the loop that reads an error and retries, that is the signal an agent would help. If you are mostly judging quality, it would not.

Sources

  • METR, Task-Completion Time Horizons of Frontier AI Models – definition, methodology and stated limitations (metr.org/time-horizons, accessed Aug. 20, 2026)
  • Toby Ord, “Is there a half-life for the success rates of AI agents?” – constant hazard rate model, arXiv:2505.05115
  • Our own timed agent run, Aug. 11, 2026 (76 seconds to working draft; 6 of 7 features; two documented failures)
  • Our own timed generative run, Aug. 6, 2026 (20 minutes; 610 to 455 words)
  • Our grading of 27 student projects against a 100-point rubric, summer 2026

Keep reading

About the author

Prof. H is a professor of AI and robotics, a former Samsung researcher, and the author of dozens of textbooks on AI and automation. He writes hands-on, no-hype guides. More about the author →

Similar Posts