Double your signal, double your fun

Noise from invalid or low-impact reports makes it difficult for customers to maintain healthy programs. These reports create a burden for programs and reduce the time that can be spent on reports that matter.

While we maintain the highest signal to noise ratio in the industry, we didn’t want to stop there. We asked ourselves, what would it take to double signal? While we know that eliminating all noise is improbable, we wanted to aim high, so we’ve set ourselves a target to reach 90% signal - a standard that hasn’t been seen on any other platform in our industry.

Introducing Human-Augmented Signal

Human-Augmented Signal improves the signal of programs as reports flagged with a high noise probability are reviewed by HackerOne security analysts. After our system utilizes various criteria to automatically classify all incoming reports, reports with potential noise are forwarded to HackerOne security analysts for review.

This human-in-the-loop review guards against false positives and further trains our classifiers over time. There are 2 paths a report can take:

1. If an analyst dismisses a report, the report will appear in your inbox as “Not Applicable” and a notification won’t be generated in your inbox.

2. If an analyst accepts a report, the report will appear in your inbox as usual.

Over the past several months, we’ve been testing this feature with a small group of beta programs such as New Relic, Wordpress, and Mapbox. Our initial results are showing a 30 to 40 percentage points signal increase for customers using Human-Augmented Signal. Wordpress, one of our early beta testers of this feature, is seeing an increase of 31.37% in signal since they started using Human-Augmented Signal.

With the help of our beta customers, we’ve been able to improve and validate our new process for Human-Augmented Signal. We’ve made sure to build a feature that doesn’t disturb existing processes while still increasing the value customers receive from their program. New Relic has been an early tester for this feature and gave us the following feedback:

"One of our biggest challenges with running a bug bounty program is sifting through the noise. With the addition of HackerOne's Human-Augmented Signal, we have a higher degree of confidence in the reports we evaluate, reproduce, and triage”

Ian Melven, Director of Product Security New Relic

Ensuring smooth public launches

It’s common for public launches to result in an immediate surge in volume as the program is advertised to a fresh pair of eyes. Though the public launch of a bounty program is exciting, it’s a demanding time for any security team. Human-Augmented Signal lessens the noise for these teams and enables launches to be much smoother. Last week, Showmax, one of the early beta testers, launched their bug bounty program to the public. With Human-Augmented Signal enabled, Showmax had a 60.48% reduction in noise compared to our original system.

What should I do?

As we believe that every company should be able to run a vulnerability disclosure program, we are providing Human-Augmented Signal for free. We think every program could benefit from this service, but since it requires granting HackerOne temporary view access to a subset of your reports, we need programs to explicitly opt-in by going to Settings > Program > Signal. For more information on this feature and what it can do for you, click here.

After opting-in, you’re all set. The filter will start reviewing any new reports, and we will automatically filter out the reports that need to be reviewed by HackerOne staff, resulting in less invalid reports in your inbox.

While we continue our progress towards 100% Signal, program managers with a desire to get there right away are encouraged to inquire about our fantastic managed services.

"The WordPress security team is staffed by volunteers, who donate their time because they love open source ideals and know that keeping WordPress secure is important. The Human-Augmented Signal program at HackerOne has helped us to respect those volunteers valuable time by reducing the noise on our program by over thirty percent! For us that means more time focusing on helping to keep WordPress users secure and less time responding to invalid reports. Thank you HackerOne!”

Aaron D. Campbell, WordPress Security Team Lead

We’re eager to hear your feedback. Please let us know if you have any questions or comments regarding Human-Augmented Signal.

This feature has been brought to you by Miray, Willian, Maarten, Alejandro, Saida, Jeroen, Siebe Jan, Ivan and Martijn.

HackerOne is the #1 hacker-powered security platform, helping organizations find and fix critical vulnerabilities before they can be criminally exploited. As the contemporary alternative to traditional penetration testing, our bug bounty program solutions encompass vulnerability assessment, crowdsourced testing and responsible disclosure management. Discover more about our security testing solutions or Contact Us today.