https://insidegovuk.blog.gov.uk/2022/07/04/opening-up-our-code-and-logic-for-algorithmic-decision-making/ Skip to main content [govu] GOV.UK Blog Inside GOV.UK Organisations: Government Digital Service Search for: [ ] [Search] https://insidegovuk.blog.gov.uk/2022/07/04/ opening-up-our-code-and-logic-for-algorithmic-decision-making/ Opening up our code and logic for algorithmic decision-making Posted by: Alessia Tosi, Data Scientist at GDS, Max Froumentin, Senior Developer at GDS and Dr Matthew Gregory, Lead Data Scientist at GDS, Posted on: 4 July 2022 - Categories: Data man at the computer reading the Algorithmic Transparency Standard on GOV.UK At GDS, we work in the open - whenever we can, we share our work: ideas, decisions, designs, source code, our successes and failures. We do so for the sake of transparency and to give back to the web design community we borrowed so much from. One of the tools we designed in the open is the related content recommender. Many pages on GOV.UK include a "Related content" box that suggests other pages that the user may be interested in. The editor of a page is able to manually specify what content is suggested on it, but by default that content is automatically generated by our related content recommender, which is based on the node2vec algorithm. On average, about 2% to 3% of visits to a GOV.UK page click on a recommended content link. In the context of almost 4 million average visits per day (May 2022), this makes for quite a popular tool on the site. GOV.UK only captures users who consent to analytics cookies. Screenshot of View your Annual Tax Summary page on GOV.UK with the Related content section highlighted by a red box showing 5 related links: Check when you can expect a replay from HMRC Check your tax with the official HMRC app Tax return for Self Assessment Check your payroll calculations manually Register for Self Assessment But, what if a user wanted to see how content was recommended to them on a page? Sure, they could look at the code on GitHub and read our blog posts on related links, but some technical details about the data we use or the assumptions we make to train the model are missing or not accessible to a non-technical reader. 'Algorithmic transparency' means being open and clear about how algorithms and any product or application underpinned by them support decisions. This transparency is essential to build people's trust. Participating in the algorithmic transparency pilot Our team was enthusiastic about putting a newly-developed algorithmic transparency template to the test. This was a pilot organised by the Central Digital and Data Office (CDDO) and the Centre for Data Ethics and Innovation (CDEI). The template forms part of the Algorithmic Transparency Standard, itself part of the government's National Data Strategy and National AI Strategy. It aims to improve transparency by standardising the way information is collected about how the public sector uses algorithmic tools. The template is a structured series of questions that we answered in order to document: * what our algorithmic tool does and why it is used (in non-technical terms) * technically, how our tool works, what alternatives we have considered * what data it uses * any consideration around equality impact, risk monitoring and ethical assessment Our answers were then collected by the CDDO and CDEI team and mapped against the algorithm transparency data standard, and was subsequently published on GOV.UK as our output transparency pilot report. Completing the template was a useful exercise, not just in terms of the output report that may benefit the public, but also for us, the developers and maintainers of the algorithmic tool. It has helped us reflect upon the approach and clarify assumptions. It also helped us spell out the decision-making process that the tool follows, and justify its use. This will be invaluable knowledge for future colleagues and for future iterations of the tool. What's next? Since the recommender was introduced in 2019, we've iterated it multiple times to adapt to changes in the content on GOV.UK and to improve the relevance of the suggested related content. For instance, the ever-increasing amount of content would cause it to crash, as it exhausted the memory available on the machine running it. We therefore had to improve the performance to make it run on a reasonably-sized machine. We also worked on suggesting better recommendations by testing variations on the original algorithm. We are looking forward to keeping our algorithmic transparency report up-to-date with any upcoming improvements and changes. We want to demonstrate how we value transparency and openness and set a precedent for how this would work with algorithms. We encourage all teams who develop or use algorithms in the public sector to complete the algorithmic transparency template. In fact, we recommend it. Read our transparency pilot report for related content. Subscribe to Inside GOV.UK to find out more about our work. Incident report: Unable to subscribe and unsubscribe from email alerts on 30 to 31 May 2022 Sharing and comments Share this page * Twitter * Facebook * LinkedIn * Email Leave a comment Cancel reply Enter your comment [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] Name [ ] Email We only ask for your email address so we know you're a real person [ ] [ ]Email me if someone replies to my comment [ ] [ ] [ ] [ ] [ ] [ ] [ ] D[ ] Submit comment By submitting a comment you understand it may be published on this public website. Please read our privacy notice to see how the GOV.UK blogging platform handles your information. Related content and links Inside GOV.UK A blog about running and improving the GOV.UK website, for anyone who is working on GOV.UK or interested in how the digital home of the UK government works. Written by the GOV.UK Team The GOV.UK roadmap GOV.UK Roadmap with the icon Read the GOV.UK roadmap to find out what the GOV.UK team are working on. It shows what we've recently shipped, what we're working on and what we're looking into. Subscribe to Inside GOV.UK to keep up to date with our work Categories Categories[Select Category ] Standards and guidelines Read GOV.UK standards and guidelines on blogs, URLs, service requests, Welsh, logos and more. Sign up and manage updates * Email * Atom Follow GOV.UK * @GOV.UK * @GDSteam * Flickr Recent posts * Opening up our code and logic for algorithmic decision-making 4 July 2022 * Incident report: Unable to subscribe and unsubscribe from email alerts on 30 to 31 May 2022 17 June 2022 * What we've learned from one year of Real User Monitoring data on GOV.UK 13 June 2022 * Iterating the GOV.UK proposition: the first of our improvements for 2022 6 June 2022 * Celebrating Global Accessibility Awareness Day on GOV.UK 19 May 2022 Comments and moderation Read our guidelines. Useful links * All GOV.UK blogs * All GOV.UK blog posts * GOV.UK * All departments * Accessibility statement * Cookies All content is available under the Open Government Licence v3.0, except where otherwise stated (c) Crown copyright