Post B4ZuLBxE6WKGj7u5ei by IngaLovinde@embracing.space
 (DIR) More posts by IngaLovinde@embracing.space
 (DIR) Post #B4Z0jgUic4z2xjfhTc by whitequark@social.treehouse.systems
       1 likes, 1 repeats
       
       i'm at a loss of words after reading a paper about reformatting code using an ML model that has a measured statistical quantity A_c which says how often the reformatted code behaves the same as the originalthe "ideal" (their choice of words) case is 64.2%
       
 (DIR) Post #B4Z0lFr4d3V8v9sQHw by lumi@snug.moe
       0 likes, 0 repeats
       
       @whitequark is this satire? this sounds crazy
       
 (DIR) Post #B4Z0rJpekrmQze6bHU by whitequark@social.treehouse.systems
       0 likes, 0 repeats
       
       @lumi no it's published in the proceedings of IEEE/ACM https://upload.whitequark.org/1774306843-Duetcs_Code_Style_Transfer_through_Generation_and_Retrieval.pdf
       
 (DIR) Post #B4Z0xEPnoAAFvjpdR2 by lumi@snug.moe
       0 likes, 0 repeats
       
       @whitequark i was more so asking if the paper was satire, but i guess looking at it answers my questions as well ​:neocat_shocked:​
       
 (DIR) Post #B4Z1ORCKOGlvOTVC9A by whitequark@social.treehouse.systems
       0 likes, 1 repeats
       
       this isn't satire, this is real research published by IEEE/ACM
       
 (DIR) Post #B4Z7B5aG2U9iJriHZo by snowyfox@deadinsi.de
       0 likes, 1 repeats
       
       Figure in question seems to be about "model performing in its ideal conditions"The author's actual opinion is implied in the Results:"After inspecting the compilation checking module, we found that DUET CS achieves 55.8% computational accuracy, which is a practical metric for a code generation system. This result shows that more than half of the output code are compilable and implement the same function as the input code. The user canuse this check as an optional layer of the pipeline to guarantee grammar correctness....We found that even the non-compilable outputs display around 60% similarity to the ground truth, which means even if DUET CS cannot always produce grammar-correct code, it can still provide valuable information to help user to transfer code style....Notice, that generally the task of generating the exact same code as ground truth is very hard, especially when the code length is rather long (˜47 lines)."
       
 (DIR) Post #B4Z7Fudo9RejpSfq7c by snowyfox@deadinsi.de
       0 likes, 0 repeats
       
       That last one is a funny statement because it's laughably easy for a human to maintain the execution of a function after a style refactor. You would reprimand a junior if they couldn't do that
       
 (DIR) Post #B4ZE0bel9XniHhle7M by porglezomp@mastodon.social
       0 likes, 0 repeats
       
       @whitequark i can imagine a few cases where reformatting a code could change behavior (mostly related to language constructs that capture source locations) so I think I would be willing to accept as low as 99.99%
       
 (DIR) Post #B4ZE0bswIp8wzgOyDg by whitequark@social.treehouse.systems
       0 likes, 0 repeats
       
       @porglezomp you'll love Fig. 6
       
 (DIR) Post #B4ZE0c6PUjv1fShjDU by mntmn@mastodon.social
       0 likes, 0 repeats
       
       @whitequark @porglezomp long live the new flesh
       
 (DIR) Post #B4ZE2q6AQdinefeTzs by whitequark@social.treehouse.systems
       0 likes, 0 repeats
       
       @porglezomp there's explanatory text that says the issue with the identifier "found" is that it's rarely used
       
 (DIR) Post #B4ZE7xqD4LuJ7GdEH2 by mirabilos@toot.mirbsd.org
       0 likes, 0 repeats
       
       @whitequark I cannot even
       
 (DIR) Post #B4ZsFixiGkDDS9OdUG by xgranade@wandering.shop
       0 likes, 0 repeats
       
       @whitequark @porglezomp I'm spitting out my drink at j++ ­→ j--. Holy shit.
       
 (DIR) Post #B4ZsFjCbNO7cCKMWh6 by sabik@rants.au
       0 likes, 0 repeats
       
       @xgranade @whitequark @porglezomp I think reversing the `j` for loop is actually wanted by them? It's labelled "ground truth", and it is a potential valid optimisation
       
 (DIR) Post #B4ZsFjNwhDCClVfaNM by IngaLovinde@embracing.space
       0 likes, 0 repeats
       
       @sabik @xgranade @whitequark @porglezomp but they also changed the boundaries! "Input" checks all values from 2 to i+2 inclusive; but "ground truth" just trows i+2 iteration out.
       
 (DIR) Post #B4ZtuXnWmYyieef1BQ by sabik@rants.au
       0 likes, 0 repeats
       
       @IngaLovinde @xgranade @whitequark @porglezomp `i` starts from 1 in the "ground truth" version
       
 (DIR) Post #B4ZuLBxE6WKGj7u5ei by IngaLovinde@embracing.space
       0 likes, 0 repeats
       
       @sabik @xgranade @whitequark @porglezomp ah I see, so the new i is just the old one + 1