[HN Gopher] Cargo-mutants:zombie: Inject bugs and see if your te...
       ___________________________________________________________________
        
       Cargo-mutants:zombie: Inject bugs and see if your tests catch them
        
       Author : todsacerdoti
       Score  : 82 points
       Date   : 2025-04-10 20:10 UTC (3 days ago)
        
 (HTM) web link (github.com)
 (TXT) w3m dump (github.com)
        
       | matthewfcarlson wrote:
       | I wonder if there's something similar to run on a c codebase.
        
         | zhaoweny wrote:
         | There's mull[1], which is based on LLVM toolchain and can
         | accept any language that transforms into LLVM bytecode.
         | SQLite[2] also does mutation testing, by compile and mutate
         | generated assembly code.
         | 
         | [1]: https://mull.readthedocs.io/en/0.26.0/ [2]:
         | https://sqlite.org/testing.html#mutation_testing
        
         | BadBadJellyBean wrote:
         | There are a few. See: https://github.com/theofidry/awesome-
         | mutation-testing
        
       | rao-v wrote:
       | This seems like a near perfect use of coding LLMs and a useful
       | way to implement reinforcement learning.
       | 
       | "Add a major bug to this file that is not covered by existing
       | tests" vs "Find the bug in this file" vs "write a sensible test
       | in this file that protects against this type of bug"
        
         | infogulch wrote:
         | Somehow that reminds me of how diffusion models are trained.
        
       | J_Shelby_J wrote:
       | Neat.
       | 
       | Is this to be used in addition to the tools mentioned in this
       | talk: https://youtube.com/watch?v=qfknfCsICUM
        
       | ramon156 wrote:
       | Need to replace the :zombie: with a
        
       | stblack wrote:
       | At RustConf 2024 in Montreal, Cargo-mutants' creator Martin
       | Pool's presentation was excellent. One of the best sessions of
       | the conference.
       | 
       | https://www.youtube.com/watch?v=PjDHe-PkOy8
        
       | kilroy123 wrote:
       | Really cool! I wish there was such a thing for JavaScript.
       | 
       | I say this as a so-so software engineer. I badly wish there was
       | more emphasis on increasing software quality. There is so much
       | the industry could do to radically improve quality, such as tools
       | like this.
       | 
       | I know the incentives just aren't there, but still.
        
         | MaoSYJ wrote:
         | Good news it is a thing in js!
         | 
         | https://stryker-mutator.io/
         | 
         | There was some hype about it some years ago.
        
       | IshKebab wrote:
       | This is called mutation testing and is very common in formal
       | silicon verification.
       | 
       | https://en.wikipedia.org/wiki/Mutation_testing
       | 
       | A downside of naively changing source code is that you have to
       | recompile the code for every mutant, which can be very slow
       | (especially for Rust!). Obviously the right thing to do is to
       | decide at runtime whether to insert a bug or not for each
       | mutation point.
       | 
       | I had a brief skim through the help for cargo-mutants and it
       | looks like it takes the naive approach which is rather
       | unfortunate.
        
       ___________________________________________________________________
       (page generated 2025-04-13 23:00 UTC)