https://www.zdnet.com/article/have-i-been-pwned-goes-open-source/ * Edition: + Asia + Australia + Europe + India + United Kingdom + United States + ZDNet around the globe: + ZDNet France + ZDNet Germany + ZDNet Korea + ZDNet Japan Search What are you looking for? [ ] Go * Videos * Windows 10 * 5G * Best VPNs * Cloud * Security * AI * more + TR Premium + Working from Home + Innovation + Best Web Hosting + ZDNet Recommends + Tonya Hall Show + Executive Guides + ZDNet Academy + See All Topics + White Papers + Downloads + Reviews + Galleries + Videos + TechRepublic Forums * Newsletters * All Writers * + Preferences + Community + Newsletters + Log Out * * + What are you looking for? [ ] Go * Menu + Videos + Windows 10 + 5G + Best VPNs + Cloud + Security + AI + TR Premium + Working from Home + Innovation + Best Web Hosting + ZDNet Recommends + Tonya Hall Show + Executive Guides + ZDNet Academy + See All Topics + White Papers + Downloads + Reviews + Galleries + Videos + TechRepublic Forums * * + o Preferences o Community o Newsletters o Log Out * us + Asia + Australia + Europe + India + United Kingdom + United States + ZDNet around the globe: + ZDNet France + ZDNet Germany + ZDNet Korea + ZDNet Japan Have I been Pwned goes open source Want to find out if someone's stolen your user IDs and passwords? Then you can use "Have I Been Pwned," and now the code behind it is being open sourced. * * * * * * * * Steven J. Vaughan-Nichols By Steven J. Vaughan-Nichols for Linux and Open Source | May 27, 2021 -- 19:34 GMT (12:34 PDT) | Topic: Security The question isn't "Does someone have your user IDs and passwords?" I guarantee you someone has. Don't believe me? Check for yourself at Have I Been Pwned (HIBP). I'll wait. Now, do you believe me? Open Source * Linux Foundation announces new open-source software signing service * Top open source, tech-smart, board-ready women executives * Red Hat's survey results on the state of enterprise open-source software * The open-source RISC-V is prompting chip technology breakthroughs (ZDNet YouTube) * Top five open source Linux server distributions (TechRepublic) People check the free HIBP site at a rate of almost 1 billion requests per month. It collects data from all the many personal security breaches that happen every week or two. Last year alone we saw dozens of data breaches. Moving forward, HIBP will now also receive compromised passwords discovered in the course of FBI investigations. Why is the FBI getting involved? Because Bryan A. Vorndran, the FBI's Assistant Director, Cyber Division, said, "We are excited to be partnering with HIBP on this important project to protect victims of online credential theft. It is another example of how important public/private partnerships are in the fight against cybercrime." The FBI passwords will be provided in SHA-1 and NTLM hash pairs; HIBP doesn't need them in plain text. They'll be fed into the system as they're made available by the Bureau. To do that, HIBP is adding on a new, open-source program, Pwned Passwords, to let the data flow easily into HIBP. HIBP founder Troy Hunt, security expert and Microsoft Regional Director, explained he's open-sourcing the code because "The philosophy of HIBP has always been to support the community, now I want the community to help support HIBP." HIBP is written in .NET and runs on Azure. With a billion searches a month, I'm sure Hunt can use all the help he can get. He started planning to open-source HIBP in August 2020. Hunt quickly discovered this wasn't easy. He wrote: I knew it wouldn't be easy, but I also knew it was the right thing to do for the longevity of the project. What I didn't know is how non-trivial it would be for all sorts of reasons you can imagine and a whole heap of others that aren't immediately obvious. One of the key reasons is that there's a heap of effort involved in picking something up that's run as a one-person pet project for years and moving it into the public domain. I had no idea how to manage an open-source project, establish the licensing model, coordinate where the community invests effort, take contributions, redesign the release process, and all sorts of other things I'm sure I haven't even thought of yet. This is where the .NET Foundation comes in. The .NET Foundation isn't part of Microsoft. It's an open-source independent 501(c) non-profit organization. Hunt's starting with the Pwned Password code because it's relatively easy. The reasons for this include: 1. It's a very simple codebase consisting of Azure Storage, a single Azure Function, and a Cloudflare worker. 2. It has its own domain, Cloudflare account, and Azure services so it can easily be picked up and open-sourced independently to the rest of HIBP. 3. It's entirely non-commercial without any API costs or Enterprise services like other parts of HIBP (I want community efforts to remain in the community). 4. The data that drives Pwned Passwords is already freely available in the public domain via the downloadable hash sets. Thus, Hunt could "proverbially 'lift and shift' Pwned Passwords into open-source land in a pretty straightforward fashion which makes it the obvious place to start. It's also great timing because as I said earlier, it's now an important part of many online services and this move ensures that anybody can run their own Pwned Passwords instance if they so choose." Hunt hopes "that this encourages greater adoption of the service both due to the transparency that opening the code base brings with it and the confidence that people can always 'roll their own' if they choose. Maybe they don't want the hosted API dependency, maybe they just want a fallback position should I ever meet an early demise in an unfortunate jet ski accident. This gives people choices." At one time Hunt had considered selling HIBP. With this open-source move, this no longer appears to be the case. The HIBP code is being kept on GitHub. It's licensed under the BSD 3-Clause license. The overall plan is: 1. There's an authenticated endpoint that'll receive SHA-1 and NTLM hash pairs of passwords. The hash pair will also be accompanied by a prevalence indicating how many times it has been seen in the corpus that led to its disclosure. 2. Upon receipt of the passwords, the SHA-1 hashes need to be extracted into the existing Azure Blob Storage construct. This is nothing more than 16^5 different text files (because each SHA-1 hash is queried by a 5 character prefix), each containing the 35-byte SHA-1 hash suffix of each password previously seen and the number of times it's been seen. 3. "Extracted into" means either adding a new SHA-1 hash and its prevalence or updating the prevalence where the hash has been seen before. 4. Both the SHA-1 and NTLM hashes must be added to a downloadable corpus of data for use offline and as per the previous point, this will mean creating some new entries and updating the counts on existing entries. Due to the potential frequency of new passwords and the size of the downloadable corpora (up to 12.5GB zipped at present), my thinking is to make this a monthly process. 5. After either the file in blob storage or the entire downloadable corpus is modified, the corresponding Cloudflare cache item must be invalidated. This is going to impact the cache hit ratio which then impacts performance and the cost of the services on the origin at Azure. We may need to limit the impact of this by defining a rate at which cache invalidation can occur (i.e. not more than once per day for any given cache item). That said, as Hunt admits, this is very much a work in progress: "I don't have all the answers on how things will proceed from here." But, with the help of you, the FBI, and the .NET Foundation, HIBP promises to be more useful than ever. Related Stories: * Emotet botnet harvested 4.3 million email addresses. Now the FBI is using Have I Been Pwned to alert the victims * Have I Been Pwned to release codebase to the open-source community * The biggest hacks, data breaches of 2020 Related Topics: Cloud Security TV Data Management CXO Data Centers * * * * * * * * Steven J. Vaughan-Nichols By Steven J. Vaughan-Nichols for Linux and Open Source | May 27, 2021 -- 19:34 GMT (12:34 PDT) | Topic: Security Show Comments LOG IN TO COMMENT * My Profile * Log Out | Community Guidelines Join Discussion Add Your Comment Add Your Comment More from Steven J. Vaughan-Nichols * [][shutterstock-361393196] Networking Internet 2021: Here's what the year will (and won't) bring * [][shutterstock-1496925578] Networking Best internet provider in Nashville 2021: Top ISP picks * [][image-4] Internet of Things Google Fuchsia OS finally rolls out * [][shutterstock-1031967217] Networking Best internet provider in Atlanta 2021: Top ISP picks Please review our terms of service to complete your newsletter subscription. [ ] By registering, you agree to the Terms of Use and acknowledge the data practices outlined in the Privacy Policy. You will also receive a complimentary subscription to the ZDNet's Tech Update Today and ZDNet Announcement newsletters. You may unsubscribe from these newsletters at any time. [ ] You agree to receive updates, alerts, and promotions from the CBS family of companies - including ZDNet's Tech Update Today and ZDNet Announcement newsletters. You may unsubscribe at any time. By signing up, you agree to receive the selected newsletter(s) which you may unsubscribe from at any time. You also agree to the Terms of Use and acknowledge the data collection and usage practices outlined in our Privacy Policy. Continue Newsletters See All See All Related Stories * 1 of 3 * * [][oil-pipeline-prices-co_] DHS releases new cybersecurity guidelines for pipelines after Colonial attack The new rules will force pipeline operators to report any cybersecurity incidents to CISA and hire cybersecurity coordinators who can be on call 24/7. * [][pub-drone] Singapore sends out drones to watch over reservoirs Drones programmed to monitor water quality and activities initially will be deployed over two reservoirs, before another four are added to the roster later this year, and will slash ... * [][istock-518282086-1] This phishing attack is using a call centre to trick people into installing malware on their Windows PC BazaLoader phishing emails tell people to dial a phone number to cancel a phoney subscription - and that's when cyber criminals posing as 'customer support' guide victims towards ... * [][istock-1155540861] Criminals love cryptocurrencies. Should you? Crypto currencies have been all the rage for more than a decade. Great fortunes have been made and hundreds of millions if not billions has been paid to criminals thanks to their anonymity. ... * [][istock-1212062404] Big changes to 1Password in the browser as it adds biometric unlocking 1Password should provide a faster login experience with new support for Touch ID and Windows Hello biometric authentication. * [][screenshot-2021-05-26-a] Fake human rights organization, UN branding used to target Uyghurs in ongoing cyberattacks The ethnic group is being targeted in spy campaigns under the guise of the United Nations. * [][facial-recognition] Human Rights Commission calls for a freeze on 'high-risk' facial recognition Until protections around the use of such technologies are in place, the Australian Human Rights Commission has asked for a moratorium on the use of biometrics, including facial recognition, ... * [][gettyimages-1058360160] Various Japanese government entities had data stolen in cyber attack: Report Japanese government data stored in Fujitsu software has reportedly been accessed and stolen by hackers. * [][australian-government-p] Colonial Pipeline attack used to justify Australia's Critical Infrastructure Bill Home Affairs has touted the benefits of the pending Critical Infrastructure Bill while confirming the government has considered the merits of a mandatory reporting requirement ... ZDNet Connect with us (c) 2021 ZDNET, A RED VENTURES COMPANY. ALL RIGHTS RESERVED. Privacy Policy | Cookie Settings | Advertise | Terms of Use * Topics * Galleries * Videos * Sponsored Narratives * Do Not Sell My Information * About ZDNet * Meet The Team * All Authors * RSS Feeds * Site Map * Reprint Policy * Manage | Log Out * Join | Log In * Membership * Newsletters * Site Assistance * ZDNet Academy * TechRepublic Forums