Building an AI-Powered HackerOne Analyst

At Deriv, we continue to enhance our security workflows by combining human expertise with AI and automation. We have operated a public HackerOne bug bounty program since 2015, available at https://hackerone.com/deriv, and maintain a strong long-term partnership with the vendor and their platform. Many submissions reveal vulnerabilities, including zero-day issues specific to a product. Rapid triage is essential, since any delay increases operational risk and affects our ability to respond effectively.

The Problem

Security operations run continuously, and the HackerOne community does the same, so reports can arrive at any time. Critical submissions require immediate triage and rapid coordination with the appropriate teams. Our existing HackerOne to Slack integration provided basic notifications but lacked attachments, context, and detail, which forced us to log in to the platform for full review. This made the off-hour triage slow and inconvenient. The process also lacked support for custom triage paths and awareness of business or technical context, which reduced consistency and increased manual effort. As a result, meeting response targets and communicating quickly with researchers was more difficult.

In short, it was not enough. I wanted to reduce our mean time to respond (MTR) and make it clear when immediate action was required.

Hackerone Hai

HackerOne provides an agentic AI system called Hai, introduced in early 2024, which supports vulnerability management by combining generative models with HackerOne's vulnerability intelligence. Hai summarises reports, prioritises risks, identifies likely root causes, and suggests remediation paths, acting as a copilot within the platform. We needed to bring these capabilities into our own workflow to automate the initial stages of triage, deliver fast contextual insights, and reduce manual effort. While Hai is available through the graphical interface, it also exposes an API, which enables direct integration and aligns exactly with our operational requirements.

Building Harry: The AI HackerOne Analyst

I developed Harry, an AI-powered agent that performs the initial analysis and triage of HackerOne reports. Harry is triggered whenever a new report is received and supports analysts with high-quality summaries and structured context. It provides insights, recommends next actions, and allows authorised users to interact directly through Slack, enabling real-time triage and improving both speed and consistency.

The team can access these insights at any time. The system delivers the right level of information for both junior engineers and senior analysts, helping them make fast and informed decisions.

Expected Outcomes and KPI Impact

Harry was introduced to improve triage performance, responsiveness, and overall efficiency. The objectives included:

• Meeting or exceeding our HackerOne response efficiency targets
• Reducing triage time, potentially from hours to minutes
• Accelerating responses to critical issues and security incidents
• Improving accuracy in report prioritisation

These improvements support our key performance indicators, including stronger SLA adherence, greater operational efficiency, and reduced dwell time for vulnerabilities. We expect Harry to deliver measurable gains across both security responsiveness and team productivity.

Data Flow and Architecture

Harry operates on the data supplied through the HackerOne API. When a new report ID is detected by the Slack integration, the Slack bot passes the event through authorisation and validation layers to the central processing engine. The engine retrieves the full report context from HackerOne, including the description, attachments, metadata, and researcher profile history. This gives Harry the same visibility available to a human analyst, with faster access to structured information. The system then uses the HackerOne Hai, Eraser, and Mermaid APIs to generate contextual insights, visualisations, and recommended actions. All outputs are returned directly to Slack, allowing analysts to work without switching tools.

Development

I used a rapid prototyping approach focused on implementing and testing core functionality early. The system includes a set of fully or semi-automated capabilities, accessible through Slack:

• Automated triage: Performs the initial evaluation of a report and provides a valid or invalid determination with a confidence score.
• Hacker evaluation: Generates an AI-driven assessment of the reporting researcher based on their history, profile, and prior submissions.
• Full report access: Retrieves attachments and embedded content, which can also be downloaded directly from Slack.
• Attachment image analysis: Processes screenshots and translates non-English content.
• Follow-up actions: Supplies three recommended next steps generated by Hai, selectable directly in Slack.
• Diagram generation: Uses Eraser and/or Mermaid to create visual attack paths and interaction flows, which are especially useful for visual thinkers like me.
• Ask Harry: Supports natural language queries for real-time clarification and additional insights. You can even ask Harry to explain the report as if you were ten years old.
• Update status: Allows authorised users to change report status or apply penalties for invalid or spam submissions.
• Weekly reporting: Produces a summary of valid reports and bounties paid within the program.

These features showed strong potential during early internal simulations, and the integration points and trigger logic were refined throughout development. To build the AI-powered agent, I used AI-driven tooling across the workflow. Visual Studio Code with the Cline extension supported code generation, refactoring, and rapid prototyping. My role was to direct the solution, review architectural decisions, validate generated logic, and ensure stability. Regular hands-on code reviews were essential to maintain quality, correct inconsistencies, and avoid model drift.

Prompt tuning became a key part of prototyping. Small adjustments to phrasing or context produced noticeable changes in output quality, so prompts were iteratively tested, refined, and validated. Precise guidance and continuous oversight were required to keep the model focused and prevent over-engineered responses.

Security requirements were integrated from the beginning. All code included defined security controls and underwent an internal security review before deployment. Once the core framework was established, each capability was expanded and refined until the system performed reliably and met its intended purpose.

Harry in Action

Once integrated into the workflow, Harry automatically processes new HackerOne report notifications. When a report arrives, the Slack bot triggers analysis, and Harry retrieves the full report details through the HackerOne API. It evaluates the content, reviews the reporter's profile, and generates contextual insights within seconds.

Initial prompts are sent to Hai to assess both the report and the researcher. Hai provides a likely valid or invalid determination with a confidence percentage and includes an explanation of its reasoning. Harry enhances this with additional context by linking the reporter's username to their public profile and presenting the confidence score, helping engineers quickly assess the researcher's credibility.

The results are delivered directly in Slack, enabling engineers to make informed decisions without switching tools. For more complex submissions, Harry generates diagrams using Eraser and Mermaid to visualise attack paths and interaction flows. Each analysis includes a summary, a confidence rating, and recommended next steps. Engineers can query Harry for clarification, request additional information, or update report status directly from Slack. This workflow reduces Mean Time to Respond and improves the consistency of triage.

The following diagram illustrates the end-to-end workflow, showing how a new HackerOne report moves through Slack, Harry, and the AI engine before reaching the security engineer for review and resolution.

Below is an example of Harry in action, analysing a fictitious report using fully simulated data.

Results

Harry enables our security team to manage triage more efficiently, and we no longer require external triage support. The system enhances speed and consistency by providing rapid analysis while keeping human oversight in place.

Many bug bounty submissions come from new or low-signal researchers. Harry helps us manage this efficiently by using Hai's confidence scoring as an indicator of potential report validity. This allows the team to focus time and effort on the reports that matter most while maintaining full visibility across every submission.

HackerOne provides a mature AI capability with Hai. By integrating through its API, along with Eraser and Mermaid, we created a private Slack workspace where reports, diagrams, and contextual insights are presented in one place. All AI processing occurs within the HackerOne environment, ensuring consistent privacy controls. Harry incurs no additional AI cost because it uses the native Hai implementation.

Eraser integration requires a paid subscription and processes diagram content in its cloud environment. Mermaid is free and typically renders diagrams locally. Any organisation adopting these tools should review how each service processes and stores submitted data to ensure alignment with internal security and privacy requirements.

Harry has been running in production for three months and has already delivered measurable improvements in triage efficiency, prioritisation accuracy, and overall workflow collaboration. If you would like to try it, the source code is available here: https://github.com/deriv-security/Hackerone-Analyst

This is an example of effective human-in-the-loop automation, where AI enhances productivity and decision-making while retaining expert oversight. It transforms a time-consuming task into a continuous, scalable workflow.

Future Plans

We plan to refine Harry based on team feedback and build a closed learning loop that improves reasoning accuracy and contextual awareness over time. Additional insights and metrics dashboards are planned to provide real-time visibility into program performance and triage trends. We also intend to incorporate automated AI validation of reported issues, likely through an additional agent, allowing Harry to assess potential validity with greater certainty before presenting evidence for human review. As the HackerOne Hai roadmap evolves, we will continue to adapt Harry to take advantage of new capabilities and ensure it scales with our security operations.