In this article, we'll assess two services, both freely accessible to consumers: Google One and HaveIBeenPwned.com, utilized by both 1Password and Mozilla.

Google One

First Google One. Regular Google users can get a one-time scan for free, while continuous dark web monitoring is also available to everyone for a fee (in addition to VPN services, among other things). How does Google get this data? According to its website, they utilise an unnamed supplier.

Mozilla Monitor / HIBP

HIBP is a big name in the world of dark web monitoring. This is a website by Australian Troy Hunt that has been notifying people of information leaks around their accounts for years. Over the years, this website has been able to develop into the industry standard in dark web monitoring. HIBP deserves credit for that: without their work, the subject of leaked data would never have had the attention it has today.

Interestingly, HIBP engages in a passive form of data collection. It does not independently extract data sources from the dark web, but receives them from actors in its own network. This, of course, has implications for the data. By definition, it is less complete than it could be in an active search.

The results

In our specific example of a real user, the Google One scan yielded 17 hits, with 11 of them containing a password. The scan from HIBP yielded very different results: 'only' 11, of which 6 potentially included passwords. Why 'potential'? HIBP indicates which data types have been leaked for each database, but not which ones are specific to your account. With Mozilla Monitor, which utilizes HIBP, we gain deeper insight into the detected data: 8 data leaks, allegedly including passwords 4 times.

Clearly, Google outperforms HIBP/Mozilla, suggesting it is the superior service. However, unfortunately, the situation is not as straightforward as you might think. All 11 passwords identified by Google are identical. This could be attributed to the repeated use of a specific password. The data indicates that this is indeed at least partly the case: the same password has been used for LinkedIn (Leak year 2012), Dropbox (2012), and MyHeritage (2017). Furthermore, Google reports an additional 8 instances of this password being leaked from 'compilations'. These are databases compiled by hacker groups, the original source or sources of which cannot be determined.

These compilations can be interesting, but should be critically evaluated. We see that Google One certainly involves some circular reporting. This concerns the lists Collection #1, #2, #4, #5 and the 2019 Antipublic Combo List, all of which were published between 25 January and 6 February 2019. In other words, the same information is being published multiple times and creating “noise”. Don't get us wrong. Any dark web monitoring involves a degree of circular reporting. Indeed, with compilations, it cannot always be ruled out that the source is unique after all, precisely because the original source is not yet known. However, for the aforementioned lists, the fact that circular reporting is involved is so widely known that it is preventable.

Curiously, Google misses the leaked password at Nitro that Mozilla reports, raising questions about the source of Google's data procurement.

The method to these services

Despite earlier compliments on HIBP's pioneering work, a critical note must be made here. This is because HIBP's methodology ensures that dark web scanning is, by definition, not conducted. After all, HIBP receives the data and does not engage in active scanning of dark web forums. It's important to clarify that HIBP does not claim to perform such scans, but services like 1Password and Mozilla, which utilize HIBP data, do raise the impression of up-to-date information. For example, 1Password writes "With Have I Been Pwned integration, you'll know as soon as any of your logins are compromised." Mozilla Monitor positions itself as a "notification service about data breaches".

Google One's exact methodology cannot be determined because Google utilizes an unnamed third party. It's possible that the underlying methodology indeed involves a form of dark web scanning. However, when we compare Google's results with our own database, we observe that, at most, only the surface is scanned: aside from some commonly known hacker forums, Google's scope appears to be limited. This disparity may also stem from our alternative methodology. For Google One, at most, only a few dark web forums are superficially scraped, but with our bots, we gain access to more advanced datasets.

The value of dark web scanning

So the depth of these services is limited. Does that make it worthless? The answer is: no, and certainly not for consumers. Indeed, it raises awareness about leaked passwords and data. At the same time, there are critical points to consider. For instance, Google One includes noticeable instances of circular reporting that could be mitigated with a more thorough assessment of its own data. Additionally, 1Password and Mozilla imply that their data is recent, which is simply not feasible based on the HIBP method.

Certainly, the reliability and timeliness of the data is where the analyzed services fall short. Indeed, not only are the aforementioned leak databases limited in completeness, reliability, and timeliness, but they also lack information on threat indicators and compromised devices. While individual consumers may be willing to tolerate this risk, it is unacceptable for professional organizations to rely on such data. Consequently, some organizations may mistakenly believe that they have a comprehensive and up-to-date understanding based on data from Google and HIBP. However, the analysis above demonstrates that this belief is unfounded.

‍