Going from generative to agentic AI is less about a better model and more about what you build around it: state, tools, and a loop that knows when to stop.
Same model, different job
Most agentic systems run on the same large language models that power ordinary generative AI. The difference isn't in the weights. It's in how the model is used.
A generative model behaves like a function. You send it some context, it returns the most likely continuation, and then it forgets the whole exchange. An agent wraps that same function in a loop that keeps state, calls tools, checks the results and decides whether another round is needed.
What generative AI does well, and where it stops
Generative AI is very good at producing content: drafting text, translating, summarising, writing a snippet of code, answering a question. For a large share of everyday tasks that is all you need.
Its limits show up as soon as a task takes more than one step:
- It has no memory of its own. Anything it should know has to be sent again with every request, and all of it has to fit in the context window.
- It can't touch anything. On its own it cannot query a database, call an API or run the code it has just written.
- It doesn't check its work. It produces one answer, with no way of knowing whether that answer was right and no way of trying again if it wasn't.
What agentic AI adds
An agentic system makes the model the decision-maker inside a larger program. Four parts are usually involved:
- Planning: splitting a goal into smaller tasks and deciding their order.
- Tool use: calling APIs, running SQL queries, executing code in a sandbox.
- Memory: the state of the current session, plus longer-lived knowledge such as user preferences.
- Reflection: checking each result, against a schema or a test for example, and changing course when something fails.
Take a travel request. A generative assistant suggests a few flights. An agent searches for flights, checks the visa requirements, compares the options with your calendar, books the ticket once you confirm, and rebooks you if the flight gets cancelled.
The trade-offs in production
None of that extra capability comes free. The table summarises what typically changes when you move from one to the other. Exact figures depend on the model and the task.
| Generative AI | Agentic AI | |
|---|---|---|
| Latency | One model call, usually a second or two | Several calls in a loop, often 5 to 30 seconds or more |
| State | None; the context is resent each time | Session state plus long-term memory |
| Errors | Contained in a single answer | Can compound: one wrong early step misleads every step after it |
| Cost | Predictable per request | Depends on how many steps a task ends up taking |
| Debugging | Log the input and the output | Trace every step and every tool call |
What changes for chatbots
For chatbots this is the biggest shift since they started using language models at all. A generative chatbot answers questions about your order. An agentic one looks the order up, sees that it is stuck in customs, opens a ticket with the carrier and messages you when the parcel moves again.
That changes how the bot has to be built. A conversation no longer ends when the answer is sent, so the bot has to keep track of open tasks. It has to verify who you are before it acts on your account. And it needs a clean handover to a human for anything it isn't allowed to decide, such as a refund above a certain amount.
How production systems are usually put together
In larger deployments the model is kept away from side effects. A common setup has three layers:
- A model endpoint whose only job is to pick the next tool and produce its arguments as structured JSON.
- A separate worker queue, Celery with Redis for example, that actually executes those calls outside the model's request path.
- Validation in between, which checks every call against a schema and a list of permissions before anything irreversible runs.
The short version
Generative AI is the engine that produces language. Agentic AI is the machinery that points that engine at a goal. If you are building one, expect to spend less time on prompts and more on tracking state, tracing steps, and making sure one failed step can't spoil the whole task.
Next in the series: Part 3 looks at how agents are built on the inside: planning models, memory and orchestration.
