One real vulnerability followed from first probe to written finding, and the method around it. The written version of the 29 September session, with chapter timestamps.
On 29 September we ran a 39-minute session on a question that comes up in almost every first conversation: what does an autonomous pentest agent actually do when it attacks an application? Not why security matters, but what the agent does, step by step. This is the written version; the timestamps point into the recording above.
One way in against all of them
The shape of the problem is old. The attacker needs one way in; the defender needs to close all of them. True twenty years ago, true today.
What changed in the last two years is the cost on one side. Finding the one vulnerability is now automated. Closing all of them is still, mostly, manual. The two sides did not get faster at the same speed, and that is the real change.
Agents already run attack paths on their own
This is not a prediction. In July, models under test at OpenAI and Hugging Face left their test environment, reached the internet and got into systems belonging to somebody else, unprompted. Earlier in the year, an agent found and investigated more than 200 interfaces of an internal platform in about two hours, no credentials and no human in the loop, ending with read and write access to the database. The damage is not the point. The clock is: two hours, or a couple of days.
A report is a photograph
A pentest report describes your application on one day, the day it was tested, or over the six-week window it ran in. Draw a timeline: the blue dot is the pentest, the red dots are your releases. Every red dot after the blue one is untested.
Annual pentesting is not wrong, it is not enough, and it is not a process problem you can solve by working harder. It is arithmetic, and it needs a different way of testing.
An agent is a loop, not a script
Autonomous pentesting is penetration testing where AI agents do the work instead of a human. So the word to define is agent, and an agent is not a script. A script does the same steps every time. An agent does not.
An agent is a loop with four phases, a description from a paper I co-wrote for the Cloud Security Alliance, Using AI for Offensive Security:
- Planning. It decides what is worth doing next.
- Reasoning. Based on what it knows right now, it chooses a tool and the arguments for it.
- Execution. The tool runs: a browser clicking a form, an HTTP call, something else. The only deterministic step in the loop.
- Analysis. The tool returns something, typically an HTTP response, and the agent works out what it means, then goes back to planning.
Three of the four phases are model calls. Even if you start with a plan of steps, the agent rewrites it as the answers come back. That is why it can follow a lead nobody wrote a rule for, and why it is not a scanner: the plan is fixed neither at the start nor for the run.
Five stages, handed between agents
That was one agent. A pentest is several of them, arranged in stages, still a loop at scale.
Recon builds a model of the application: what APIs exist, how the front end works, what each role can do. As with a human tester, you give it credentials per role; it logs in as each and looks for inconsistencies between them. Planning decides what is worth attacking, differently for a payment flow than a marketing page. Exploitation attacks the part that was chosen. Then the stage scanners do not have: verification. Because agents and language models are not deterministic, a second agent tries to reproduce a finding before it is reported, then it loops back to planning. The feedback loop matters more than the boxes.
It reads first, then decides
In our example run, the agent turned a methodology into around 90 tasks, a living list that changes during the run. The one thing to take from this: the capability of autonomous pentesting is not sending requests quickly. That is a tool, and a scanner does it better. The capability is reading the answers, changing its mind, and updating the plan as it goes.
Three kinds of finding
Three kinds of finding separate a pentest from a scan.
- Single step. Change one identifier, get another customer's record. A scanner can at least
suspect and flag this, without confirming it.
- Chain. One endpoint leaks an identifier, a second endpoint trusts it. Each part is low risk;
together they are serious. A scanner cannot connect the two.
- Business logic. Enter a workflow at step three instead of step one and the outcome changes. A
scanner has nothing to say, because it does not understand the workflow.
This is where autonomous pentesting is different, and it is not speed: it is slower than a scanner, faster than a human tester, and keeps the human tester's approach.
A second agent tries to prove the first one wrong
Most people are tired of security tools because of alert fatigue: too many false positives, or information that does not help. If a tool is wrong one time in three, ignoring it is the rational choice. That is a tool problem, not a discipline problem, and it is why verification exists.
A second agent tries to reproduce a finding before it is reported, and duplicates are folded (two findings a single fix would remediate count as one). Every finding carries a label, Confirmed, Inconclusive or Ruled out, so you see the engine's confidence per finding rather than a wall of equal-looking alerts. Ruled-out findings stay out of the report but are still counted and visible in the app. The goal is zero false positives.
One vulnerability, probe to proof
This is the part people come for. The task was to test the strength of the secret that signs the application's session tokens, and the agent read before it attacked. It decoded a token to see the algorithm and contents, then sent that untouched token to the authorization endpoint and got a 200: a baseline for what a valid request looks like. Next it tried the cheap attacks, setting the algorithm to none and dropping the signature. Both were rejected with a 401, as they should be.
Then it generated candidate secrets, and here is the interesting part: not from a generic list off the internet, but from words it had seen while walking this specific application. It tried them, and failed.
So it went back for more evidence, reading the JavaScript bundle and calling a health endpoint for clues about how this team names things, and rebuilt its candidate list from what it had learned. The second time, the key fell. To prove the finding was real rather than a guess on paper, it forged a token for the first user, the administrator, and got the same 200 it had seen at the start. Then it wrote the finding up itself.
Guess, fail, gather more evidence, guess better, succeed. A scanner has no step where it fails and reconsiders, and that step is the clearest answer to how this differs from a scan.
Next to what a scanner sees
Scanners are useful, and our agents use them as one tool among many. The difference is the quality of what comes back. Where a scanner says "this looks like an identifier, here is a warning", a pentest says "this is another customer's record"; where a scanner says an input "is reflected", a pentest shows the data it returned. On business logic, a scanner says nothing at all.
What it does not do
A vendor that names what its engine cannot do earns the right to be believed about what it can. Our engine does no phishing and no physical access testing: those need a different kind of consent and a human. Some entry points need a person for a very specific authentication mechanism. If the target becomes unreachable mid-run, the run stops and an operator is told.
And the important limit: an engine can only rank technical severity. It cannot know what a finding is worth to your business. That judgment stays with you.
One question worth carrying into any vendor conversation in this space: what did your engine miss last month? A straight answer tells you more than any score on any slide.
How it is kept inside the boundary
Scope has to be defined: what is in the pentest and what is not. We decide it both deterministically and with an AI step, and any URL outside the target application is out of scope (bar an authentication provider you rely on). You give credentials per role, and optionally an objective, which helps on a monthly run after a change.
If you have never run a real pentest against the application, test staging first: neither of us wants your customer data in our system. During the run, a dedicated scope enforcer has one job, to confirm every tool call is in scope and refuse the rest. You can watch live, pause the run, or cancel it. You cannot steer the agents.
Four guardrails, three of them free
If you do nothing else after reading this, do these four:
- Know your external attack surface, continuously. You cannot protect what you do not know you expose. This is attack surface management, simple or complex depending on your size.
- Protect the steps worth attacking. For changing bank details, urgent transfers or password resets, use four eyes or a second channel. This is how to run any business safely, not a pentesting point.
- Assume you get hacked. Not whether, but when, and how bad. Least privilege, separated networks, multi-factor authentication. Defense in depth.
- Govern your own AI agents. No blanket permissions, no unwatched access to production.
Three of these four cost nothing and need no vendor, including us.
Run one against your own app
Here is how I would start. Create a free account at app.squr.ai, no card. Pick one application you are allowed to test and set the scope, about ten minutes with no call in the way. Start with staging, and give it credentials for two roles, because the valuable findings often live in the boundary between them. Launch a real run and the results come back in 24 hours, every finding labelled with whether verification confirmed it. When the report arrives, check with someone who knows the product that they can reproduce each finding as described. Easy-to- reproduce findings are the whole point.
The fastest first look costs nothing: run a free 60-second external scan of your attack surface, then decide whether to point a full run at the application behind it.