Post AV3geL53GRo5MtBP04 by randomwalker@mastodon.social
 (DIR) More posts by randomwalker@mastodon.social
 (DIR) Post #AV2nzCbyS5M5hjW9sO by randomwalker@mastodon.social
       2023-04-26T12:42:58Z
       
       0 likes, 2 repeats
       
       People have been posting glaring examples of ChatGPT’s gender bias, like arguing that attorneys can't be pregnant. So @sayashk and I tested ChatGPT on WinoBias, a standard gender bias benchmark. Both GPT-3.5 and GPT-4 are about 3 times as likely to answer incorrectly if the correct answer defies gender stereotypes — despite the benchmark dataset likely being included in the training data. https://aisnakeoil.substack.com/p/quantifying-chatgpts-gender-bias
       
 (DIR) Post #AV3geL53GRo5MtBP04 by randomwalker@mastodon.social
       2023-04-26T13:15:45Z
       
       0 likes, 1 repeats
       
       OpenAI mitigates ChatGPT’s biases using fine tuning and reinforcement learning. These methods affect only the model’s output, not its implicit biases (the stereotyped correlations that it's learned). Since implicit biases can manifest in countless ways, OpenAI is left playing whack-a-mole, reacting to examples posted on social media.
       
 (DIR) Post #AV3gePSayr14x4EGFk by randomwalker@mastodon.social
       2023-04-26T14:54:43Z
       
       0 likes, 0 repeats
       
       This is the latest in the AI Snake Oil book blog by @sayashk and me. Writing this blog alongside the book has been really fun. I'll probably do something like this for all future books! Thank you to everyone who subscribed. https://aisnakeoil.substack.com/