How to test a vulnerability reporting tool in 14 days

Most free trials end the same way. You sign up, click around with the sample data, get pulled into other work, and the trial expires before you have tested anything that matters.

A vulnerability reporting tool is easy to judge if you test it on your own data, against a few clear questions. Here is a two-week plan that does that.

Key takeaways

• Use your own scan exports, not the vendor's demo data. Bring at least one messy one.
• Check the ranking against real-world evidence, not just severity scores.
• Send one report to a real reader (a client or your manager) and watch what they ask.
• Test the second scan, not just the first: proving fixes is where most of the value is.

Before you start: pick your test data

Gather three things:

1. A typical export from the scanner you run most.
2. A difficult export: a large one, one from a second scanner, or one where you know the scanner left CVEs out.
3. Two scans of the same client, about a month apart, from the same scanner. You will need these for the proof-of-fix test.

Remove anything you are not allowed to share with a third party, and read the tool's data retention terms first. How long does it keep your raw uploads?

Days 1 to 3: first report

Upload the typical export and read the report as if you were the client.

• Did it read everything? Compare the host and finding counts with the scanner's own summary. A tool that silently drops rows is a tool you can't trust.
• Is the top of the list right? The first ten findings should be the ones you would fix first. Check that anything on the CISA KEV list is near the top, whatever its CVSS score.
• Can you see why? For each top finding, the report should show its evidence: KEV listing, exploit likelihood (EPSS), end-of-life status. "Critical" alone is not a reason.
• Is anything made up? If the tool uses AI to write, check that every CVE, host name and count in the text matches the export. Facts should come from the data, not from the model.

Days 4 to 6: the hard export

Now upload the difficult one. This is where tools separate.

• Does it handle a different scanner's format without you converting anything?
• If the export is missing CVEs, does the tool map the scanner's own check IDs back to CVEs, or does it just show less?
• Does a large file finish in a reasonable time, and does it tell you what it skipped and why?

Days 7 to 9: a real reader

Send one report to the person who would really receive it, under your own branding if you are an MSP.

• Do they understand the first page without your help?
• Do they ask "what should I do first?" If so, the report didn't answer it.
• Is the fix list something your team can act on: specific steps, per system?

Days 10 to 12: the second scan

Upload the later scan of the same client. This is the test most trials skip, and it matters most for monthly work.

• Does the report show what was fixed, what came back and what is new since the last scan?
• Is it honest about gaps? If the second scan didn't log in, issues should show as not rechecked, not fixed.
• Can findings become tickets in your PSA, with due dates, without retyping?
• Can you record a finding the client won't fix, with a reason and an end date?

Days 13 and 14: decide

Answer four questions in writing:

1. Did it save time compared with how you build reports today? Roughly how much per report?
2. Would you send its report to a client or auditor without editing it?
3. Did it get the priorities right on your own data?
4. What does it cost at the number of reports you produce in a normal month?

If three of the four answers are yes, it is probably worth paying for. If not, you have lost nothing but two weeks, and you know exactly what to look for next time.

Try this plan on Scan Hub

We built Varaxon Scan Hub to pass this test. It is scanner-agnostic, ranks findings by CISA KEV, VulnCheck KEV and EPSS before severity, maps scanner check IDs to CVEs when an export leaves them out, and shows what was fixed since the last scan. Code decides the facts, and AI only writes the explanations from them. Raw uploads are deleted within 7 days.

Every new account gets a 14-day free trial of Core: up to 10 reports and 3 team seats, with no card needed and no automatic charge at the end. Not ready to sign up? The free pre-flight check needs no account at all. Start a free trial or see sample reports.

Originally published at https://blog.varaxontech.com on October 28, 2026.