https://github.blog/2022-03-31-how-github-does-take-home-technical-interviews/ * Back to GitHub.com * Changelog * Resources * Engineering * Community * Product * Security * Open Source * Enterprise * Education * Policy * Company Search by Keyword [ ] Search * Engineering How GitHub does take home technical interviews We believe our technical interviews should be as similar as possible to the way we work at GitHub. How GitHub does take home technical interviews Andy McKay By Andy McKay March 31, 2022 * * * There are many ways to evaluate an engineering candidate's skills. One way is to ask them to solve a problem or write some code. We have been striving for a while to make this experience better at GitHub. This blog post talks about how candidates at GitHub do the "take home" portion of their interview--a technical challenge done independently--and how we improved on that process. We believe the technical interview should be as similar as possible to the way we work at GitHub. That means: * Writing code on GitHub and submitting a pull request. * Using your preferred editor, operating system, and tools. * Using the internet for documentation and help. * Respecting time limits. In order to make this process seamless for candidates, we automate with a GitHub app called Interview-bot. This app uses the GitHub API and existing GitHub features. First, candidates get to choose the programming language they'll use to take the interview. They'll get an email asking them to take the interview at their convenience by signing into Interview-bot. Each interview is aimed at being similar to the day-to-day problems that we solve at GitHub. These aren't problems to trick or test obscure knowledge. Interviews come with a clear set of instructions and a time limit. We place this time limit because we respect your time and want to ensure that we don't bias toward candidates who have more time to invest in the solution. The exercise is contained in a repository in a separate organization on GitHub. When the candidate signs in, we make a new repository, grab a copy of the exercise and copy the files, issues, and pull requests into the repository. This is done as a copy and not a fork or clone because we can alter the files in the process to fix things up. It also allows us to remove any Git history that might hide embarrassing clues on how to complete the exercise. Diagram showing that the candidate exercise is copied from "based repository" to "candidate repositor"Diagram showing that the candidate exercise is copied from "based repository" to "candidate repositor" The candidate is given access to the repository, and a timer starts. They can now clone the exercise to their local machine, or use GitHub Codespaces. They are able to use whatever editor, tooling, and operating system they want. Again, we hope to make this as close as possible to how the candidate will be working in a day to day environment at GitHub. When the candidate is satisfied with their pull request, they can submit it for review. The application will listen for the pull request via webhooks and will confirm that the pull request has been submitted. Screenshot of pull request confirmation that candidate will receive Screenshot of pull request confirmation that candidate will receive At this point, we anonymize the pull request and copy it back to the base repository. Diagram that shows how the anonymized candidate response is sent back to the base repositoryDiagram that shows how the anonymized candidate response is sent back to the base repository The pull request contains the code changes and comments from the candidate. To further reduce bias, the system anonymizes the submission (as best as it can) by removing the title. The Git commits and pull request will display Interview-bot as the author. To the reviewer, the pull request comes from Interview-bot and not the candidate. Sample pull request from interview-bot, showing anonymized ID rather than candidate nameSample pull request from interview-bot, showing anonymized ID rather than candidate name The pull request includes automated tests and a rubric so that interviewers know how to mark it and each submission is evaluated objectively and consistently. The tests run through GitHub Actions and provide a base level for the reviewer. For each language, we've got teams of engineers who review the exercise. Using GitHub's code review team feature, an engineer at GitHub is assigned to review the code. To mark the code, we provide a clear scorecard on the pull request as a comment. This clear set of marking criteria helps limit any personal bias the interviewer might have. They'll mark the review based on given technical criteria and apply an "Approve" or "Request changes" status to give the candidate a pass or fail, respectively. Finally, Interview-bot tracks for changes on the pull request review and then informs the assigned staff member so they can follow up with the candidate, who hopefully moves on to the next stage of the GitHub interview process. At the start of the interview process, Interview-bot associates each candidate with an issue in an internal repository. This means that staff can track candidates and their progress all within GitHub. Sample status update from Interview-botSample status update from Interview-bot Using the existing GitHub APIs and tooling, we created an interview process that mirrors as closely as possible how you'll work at GitHub, focused on reducing bias and improving the candidate's experience. If you're interested in applying at GitHub, please check out our careers page! Tags: * remote work More on remote work GitHub's Engineering Team has moved to Codespaces Technical interviews via Codespaces Codespaces is a great tool for technical hiring exercises and helps level the playing field for candidates. Ian Olsen Solving the innersource discovery problem Imagine you're in an organization with over 2,000 repositories across several different product lines. It can be daunting task to find the right project. Zack Koppert Measuring enterprise developer productivity In a recent paper written by Nicole Forsgren and her colleagues, "The SPACE of developer productivity: There's more to it than you think," there is an irony that is hard... Kevin Alwell Related posts Engineering An update on recent service disruptions Over the past few weeks, we have experienced multiple incidents due to the health of our database. We wanted to share what we know about these incidents while our team continues to address them. Keith Ballinger Keith BallingerMarch 23, 2022 Engineering GitHub Availability Report: February 2022 In February, we experienced one incident resulting in significant impact to multiple GitHub services. Jakub Oleksy Jakub OleksyMarch 2, 2022 Product * Features * Security * Enterprise * Customer Stories * Pricing * Resources Platform * Developer API * Partners * Atom * Electron * GitHub Desktop Support * Docs * Community Forum * Training * Status * Contact Company * About * Blog * Careers * Press * Shop * GitHub on Twitter * GitHub on Facebook * GitHub on YouTube * GitHub on LinkedIn * GitHub's organization on GitHub * (c) 2022 GitHub, Inc. * Terms * Privacy