https://arxiv.org/abs/2208.04259

Skip to main content
Cornell University
We gratefully acknowledge support from the Simons Foundation, member
institutions, and all contributors. Donate
 
arxiv logo > cs > arXiv:2208.04259
[                    ]

Help | Advanced Search

[All fields        ]
Search
arXiv logo
Cornell University Logo
[                    ] GO
quick links

  * Login
  * Help Pages
  * About

Computer Science > Software Engineering

arXiv:2208.04259 (cs)
[Submitted on 8 Aug 2022]

Title:First Come First Served: The Impact of File Position on Code
Review

Authors:Enrico Fregnan, Larissa Braz, Marco D'Ambros, Gul Calikli,
Alberto Bacchelli
View a PDF of the paper titled First Come First Served: The Impact of
File Position on Code Review, by Enrico Fregnan and 4 other authors
View PDF

    Abstract:The most popular code review tools (e.g., Gerrit and
    GitHub) present the files to review sorted in alphabetical order.
    Could this choice or, more generally, the relative position in
    which a file is presented bias the outcome of code reviews? We
    investigate this hypothesis by triangulating complementary
    evidence in a two-step study.
    First, we observe developers' code review activity. We analyze
    the review comments pertaining to 219,476 Pull Requests (PRs)
    from 138 popular Java projects on GitHub. We found files shown
    earlier in a PR to receive more comments than files shown later,
    also when controlling for possible confounding factors: e.g., the
    presence of discussion threads or the lines added in a file.
    Second, we measure the impact of file position on defect finding
    in code review. Recruiting 106 participants, we conduct an online
    controlled experiment in which we measure participants'
    performance in detecting two unrelated defects seeded into two
    different files. Participants are assigned to one of two
    treatments in which the position of the defective files is
    switched. For one type of defect, participants are not affected
    by its file's position; for the other, they have 64% lower odds
    to identify it when its file is last as opposed to first.
    Overall, our findings provide evidence that the relative position
    in which files are presented has an impact on code reviews'
    outcome; we discuss these results and implications for tool
    design and code review.
    Data and materials: this https URL

          This paper has been accepted at ESEC/FSE '22 (30th ACM
Comments: Joint European Software Engineering Conference and
          Symposium on the Foundations of Software Engineering)
Subjects: Software Engineering (cs.SE)
Cite as:  arXiv:2208.04259 [cs.SE]
          (or arXiv:2208.04259v1 [cs.SE] for this version)
          https://doi.org/10.48550/arXiv.2208.04259
          Focus to learn more
          arXiv-issued DOI via DataCite
Related   https://doi.org/10.1145/3540250.3549177
DOI:      Focus to learn more
          DOI(s) linking to related resources

Submission history

From: Alberto Bacchelli [view email]
[v1] Mon, 8 Aug 2022 16:44:14 UTC (663 KB)
Full-text links:

Access Paper:

    View a PDF of the paper titled First Come First Served: The
    Impact of File Position on Code Review, by Enrico Fregnan and 4
    other authors
  * View PDF
  * TeX Source
  * Other Formats

license icon view license
Current browse context:
cs.SE
< prev   |   next >
new | recent | 2022-08
Change to browse by:
cs

References & Citations

  * NASA ADS
  * Google Scholar
  * Semantic Scholar

a export BibTeX citation Loading...

BibTeX formatted citation

x
[loading...          ]
Data provided by:

Bookmark

BibSonomy logo Reddit logo
(*) Bibliographic Tools

Bibliographic and Citation Tools

[ ] Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
[ ] Connected Papers Toggle
Connected Papers (What is Connected Papers?)
[ ] Litmaps Toggle
Litmaps (What is Litmaps?)
[ ] scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
( ) Code, Data, Media

Code, Data and Media Associated with this Article

[ ] alphaXiv Toggle
alphaXiv (What is alphaXiv?)
[ ] Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
[ ] DagsHub Toggle
DagsHub (What is DagsHub?)
[ ] GotitPub Toggle
Gotit.pub (What is GotitPub?)
[ ] Huggingface Toggle
Hugging Face (What is Huggingface?)
[ ] Links to Code Toggle
Papers with Code (What is Papers with Code?)
[ ] ScienceCast Toggle
ScienceCast (What is ScienceCast?)
( ) Demos

Demos

[ ] Replicate Toggle
Replicate (What is Replicate?)
[ ] Spaces Toggle
Hugging Face Spaces (What is Spaces?)
[ ] Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
( ) Related Papers

Recommenders and Search Tools

[ ] Link to Influence Flower
Influence Flower (What are Influence Flowers?)
[ ] Core recommender toggle
CORE Recommender (What is CORE?)

  * Author
  * Venue
  * Institution
  * Topic

( ) About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and
share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have
embraced and accepted our values of openness, community, excellence,
and user data privacy. arXiv is committed to these values and only
works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community?
Learn more about arXivLabs.

Which authors of this paper are endorsers? | Disable MathJax (What is
MathJax?)

  * About
  * Help

  * Click here to contact arXiv Contact
  * Click here to subscribe Subscribe

  * Copyright
  * Privacy Policy

  * Web Accessibility Assistance
  * arXiv Operational Status
    Get status notifications via email or slack