We shipped whitebox Autonomous Pentesting, so we put it up against the field of security tools on three real applications. On the vulnerabilities it proved by exploiting them, it beat every buyable scanner and matched or beat the strongest agentic engine in the field on two of three.

We recently released whitebox Autonomous Pentesting: you give SQUR your source code, and it reads the code to find where the logic can be bent, then proves each vulnerability by exploiting it against your live application. Reading source is table stakes. Proving the exploit is the product.

We needed a fair, public yardstick, and ProjectDiscovery had already built one. ProjectDiscovery are the team behind Nuclei and much of the open-source security tooling the industry runs on, and they published results for three applications, each built by a different AI coding assistant and carrying the real vulnerabilities those assistants produced: a banking app (VaultBank, built with Claude Code), a medical portal (MedPortal, built with Codex), and an insurance claims app (ClaimFlow, built with Cursor). They ran a field of security products against each one, including their own agentic pentester, Neo, and published the full results (with a finding-by-finding walkthrough in part two). We ran SQUR on the same three with the same source access. Below is what it found, and the exploits to back it.

The headline: critical and high vulnerabilities, proven by exploitation

These apps contain more real bugs than any one tool has found; the published list is a floor, not a ceiling. So we did not grade against that list alone. We counted the critical and high vulnerabilities each engine actually proved, removed the list's false positives, and added the exploitable bugs it never contained.

Critical + high, proven	VaultBank	MedPortal	ClaimFlow
SQUR	8	5	7
Neo (ProjectDiscovery)	8	6	6
Claude Code	5	5	3
Invicti	0	0	0
Snyk	0	0	0
Semgrep	0	0	0

On two of the three applications, the banking app and the insurance app, SQUR matched or beat ProjectDiscovery's own engine. On the medical portal it came in one finding behind Neo. Against every tool a buyer can actually purchase, the result was not close: Snyk, Semgrep and Invicti, each handed the full source code, confirmed zero critical or high vulnerabilities on any of the three. Source access is not the differentiator. What you do with it is.

Show me the data

Fair. Every SQUR finding ships with a working exploit and the steps to reproduce it. Here are three that no tool in the published study reported, proven against the live apps.

VaultBank: a customer rewrites balances through a scheduled payment. The update endpoint does not validate the amount, so a customer edits a pending payment to a negative value. When the scheduler runs, its arithmetic reverses: funds move to the sender and away from the recipient.

PATCH /api/v1/scheduled-payments/34   { "amount": -50.00, "scheduled_for": "2020-01-01T00:00:00" }
Authorization: Bearer <redacted>
-> HTTP 200 OK

After the scheduler ran (balances read back from the live accounts API):
  sender    savings: 170,480.00 -> 170,530.00   (+50.00)
  recipient savings:  35,020.00 ->  34,970.00   (-50.00)

We ruled out a silent rollback by reading both balances back from the live app; the change persisted.

VaultBank: a race turns one approved loan into two disbursements. The disbursement endpoint checks the loan status and then writes, with no row lock between the two. Twenty concurrent requests slipped two writes through the gap.

20x concurrent  POST /api/v1/loan-applications/15/disburse   { "account_id": 2 }
-> 2x HTTP 200 OK, 18x HTTP 400

Ledger after the burst:
  transaction 317  loan_disbursement  10,000,000.00 USD   loan 15
  transaction 318  loan_disbursement  10,000,000.00 USD   loan 15   (7 ms later)

One approved loan, paid out twice, ten million created from nothing.

ClaimFlow: a policyholder approves their own over-limit claim. The approval action lives in a route whose authorization is enforced only on page loads, not on the form POST underneath it. A low-privileged policyholder files a claim far over their limit, then posts the approval directly.

(as policyholder Jane Smith, session <redacted>)
POST /claims/new                               amount = 99,999.00    -> HTTP 200
POST /underwriter/claims/<id>?/makeDecision    decision = approved   -> HTTP 200
GET  /claims/<id>                              status: Approved  $99,999

None of these three appear in the study's published list, and no tool in the study reported them. They are the findings that cost a company real money, and they are exactly the ones a pattern match never sees.

Proof, not guesswork: the false-positive count

Running the attack before writing the finding is what keeps a report clean. SQUR reports only what it exploited, so every confirmed finding is a real vulnerability with a working proof-of-exploit. The numbers below for the other tools are ProjectDiscovery's own adjudication of what each one reported.

False positives	VaultBank	MedPortal	ClaimFlow
SQUR	0	0	0
Neo (ProjectDiscovery)	0	3	2
Claude Code	8	8	8
Invicti	4	5	1
Snyk	3	1	1
Semgrep	1	0	2

The point a buyer cares about: SQUR hands you a report with nothing to triage away, because a finding only exists once it has been exploited. Every other tool in the field, the agentic ones included, leaves you some number of dead ends to rule out by hand. ProjectDiscovery's Neo is the closest to clean, and still produced a handful on two of the three apps; the untuned buyable options produced up to eight per app.

SQUR was more accurate than the study's own list

A published result is only as good as the judgement behind it, so we checked the list rather than trusting it. It records "Hardcoded JWT Secret Key" as a critical vulnerability in VaultBank. It is not exploitable on a real deployment: the app replaces that key with a random secret at startup, as its own setup instructions require. SQUR found the hardcoded default, signed a token with it, watched the live app reject that token, and reported the finding as not exploitable. Neo and the rest of the field counted it as a live critical. Rejecting a dead critical that the whole field accepted is a result only SQUR produced here.

It happened a second time, on MedPortal. The list rates mass assignment on the user-update endpoint a HIGH, a privilege escalation. The route tells a different story: it rejects every non-administrator with a 403 before the write, so the only account that can reach it is already an administrator. SQUR tested it as five different non-admin roles, recorded the 403 each time, confirmed the mass assignment only with an admin token, and rated it MEDIUM. An administrator editing accounts is not a privilege escalation, so MEDIUM is the accurate severity.

The list also dismisses as false positives two MedPortal findings that are real behaviours: any authenticated user can forge entries in a clinical audit log, and a clinician can inject notifications into another patient's feed. SQUR found and proved both. In fairness, so did Neo and Claude Code; the disagreement is with the study's panel, not a catch unique to SQUR.

Accuracy runs both ways, and judgement is what a real pentest brings that a pattern match cannot. The full per-finding breakdown for all three apps is available on request.

The questions a careful reader will ask

Did SQUR already know the answers from its training data? No, on two counts. The models SQUR runs have a training knowledge cutoff of January 2025, more than a year before this study was published, so the study was never in their training data to begin with. And the deeper reason is structural: SQUR does not report a vulnerability it has read about, it reports one it has exploited. A training corpus cannot produce a live HTTP 200 that moves money in a running application. The proof is an action against your system, not a recollection. SQUR also reported several critical and high vulnerabilities that appear nowhere in ProjectDiscovery's published results, so for those there was no answer to have memorised.

Did SQUR look up the published write-ups while it ran? No. We reviewed the engine's network activity for all three runs, and none of it concerned ProjectDiscovery or the public identities of these applications. SQUR works from the source you give it and the application it is attacking, not from anyone's blog.

Did SQUR get an unfair advantage from source access? No. Every tool in the study, the traditional scanners included, was given the full source code. The whitebox setting was the same for everyone. What differed was the result.

Did we pick the apps that flatter us? No. These are the three applications ProjectDiscovery published, and we are reporting all three, including MedPortal, where SQUR came one finding behind Neo. We did not drop the one we did not win.

What this means for you

• Whitebox Autonomous Pentesting: give SQUR your source, get back proven exploits, not a list of code smells.
• A full pentest in 24 hours, not three to six weeks.
• Every confirmed finding comes with proof-of-exploit, re-tested by an independent verifier against your live app.
• Reports structured for ISO 27001, SOC 2, DORA Art. 24, NIS2 and CRA, with a shareable certificate after a clean run.
• On the Professional plan, a full pentest every month, so the window a real finding stays open and exploitable closes in weeks, not once a year.
• €1,995 per pentest, roughly 80% below a traditional engagement. All data stays in the EU (GCP Brussels).