[HN Gopher] Researchers seeing little evidence of benefit from c...
       ___________________________________________________________________
        
       Researchers seeing little evidence of benefit from co pilots
        
       Author : chairmansteve
       Score  : 16 points
       Date   : 2024-09-28 22:04 UTC (56 minutes ago)
        
 (HTM) web link (www.cio.com)
 (TXT) w3m dump (www.cio.com)
        
       | nbbnbb wrote:
       | I don't have any formal data to prove this available without
       | losing anonymity and probably getting sued by my employer but the
       | introduction of them at my organisation correlates directly to a
       | measurable rise in bugs and incidents. From causal analysis, the
       | tools themselves are not directly responsible as such despite
       | having limited veracity, but people trust them and do not do
       | their jobs properly. There is also a mystique around them being
       | the solution for all validation processes which leads to
       | suboptimal attention at the validation stage on the hope that
       | some vendor we already have is going to magically make a problem
       | go away like they said they would at the last conference. I
       | figure at this point the gain might be a negative on a social and
       | human perspective the moment the idea was commercialised.
       | 
       | Urgh. I can't wait to retire.
        
       | wanderingbit wrote:
       | This finding bewilders me, because my copilot (I use
       | Sourcegraph's Cody) has become an essential part of my dev
       | productivity toolset. Being able to get answers to questions that
       | would normally break me out of flow mode by simply Option + C'ing
       | to open up a New Chat has been a productivity boost for me.
       | Getting it to give me little snippets of code that I can use
       | helps keep me in flow mode. Getting it to do a first pass on
       | function comments, which I then edit, has made it much easier to
       | get over the activation energy barrier that usually holds me back
       | from doing full commenting.
       | 
       | I can't say if the bug count is higher or not. Maybe it is higher
       | in terms of total number of bugs I write throughout my coding
       | session. But if bug count goes up 10% then the speed with which I
       | fix those bugs and get to a final edit of my code is 30% or 40%
       | faster, so the bug count is not the right metric.
       | 
       | Maybe the differentiator is that I am a solo-dev for all this
       | work, and so the negative effects of the copilot are only
       | experienced by me. If I were in a 10 person team, the bugs and
       | the weird out of context code snippets would be magnified by the
       | 9 other people, and the negative effects would be strong. But I
       | don't know.
        
       ___________________________________________________________________
       (page generated 2024-09-28 23:00 UTC)