Post ASEJb4sPdoGAlVPZNQ by ben@mastodon.adida.net
(DIR) More posts by ben@mastodon.adida.net
(DIR) Post #ASEJb4sPdoGAlVPZNQ by ben@mastodon.adida.net
2023-02-01T03:13:48Z
1 likes, 3 repeats
So OpenAI just released a detector of AI-generated text, I assume because of concerns in education / homework.https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text/Maybe this is good?No, it's very bad.They claim 26% true positives, 9% false positives. Assume 10% of submitted homework is chatgpt generated, you get the classic counterintuitive outcome of poor predictive power: if a homework is flagged, there's a 3:1 chance it's *human* generated.This is going to cause a lot of harm. It should be immediately recalled.
(DIR) Post #ASEJb5QnZxlWU9V1yS by zleap@qoto.org
2023-02-01T08:27:06Z
0 likes, 0 repeats
@ben How does plagiarism detection software work normally?
(DIR) Post #ASEZXTenAISjjDLUa8 by Cmastication@mastodon.social
2023-02-01T11:25:43Z
0 likes, 1 repeats
@zleap @ben normally it matches literal strings. e.g. “these two sentences came from this source”
(DIR) Post #ASEemQxVyhQxTW3QMy by zleap@qoto.org
2023-02-01T12:24:29Z
0 likes, 0 repeats
@Cmastication @ben Ah, that makes sense,
(DIR) Post #ASEgZOWnu82k50TEv2 by shriramk@mastodon.social
2023-02-01T12:24:27Z
0 likes, 0 repeats
@Cmastication @zleap @ben That's not how MOSS, the most widely-used checker, works.
(DIR) Post #ASEgZOzsA3INXA4SEC by Cmastication@mastodon.social
2023-02-01T12:38:43Z
0 likes, 0 repeats
@shriramk @zleap @ben looks like MOSS uses fingerprinting which is a computationally efficient way to find white space invariant string matches. https://yangdanny97.github.io/blog/2019/05/03/MOSS
(DIR) Post #ASEgZPTeNL7B1W0Eds by zleap@qoto.org
2023-02-01T12:44:31Z
0 likes, 0 repeats
@Cmastication @shriramk @ben Sounds interesting, so from this I would guess it would look at a string such asThe cat sat on the mat., or The quick brown fox jumps over the lazy dog. From these generate a hash, is this similar to how say md5sum works, as I could write one of the above in a text file, save and generate a md5sum from that, this would be unqie, if you want back in and changed a lower case letter to upper case, or added a comma, it would change the file, and the md5sum would be different. We could then compare the two checksums to see if they match or don't match. Or am I completely off track here. Given I am not remotely an expert in this.
(DIR) Post #ASEpukTwNImvpxO9Oy by zleap@qoto.org
2023-02-01T14:29:15Z
0 likes, 0 repeats
@Cmastication @shriramk @ben I have never understood what white space means.
(DIR) Post #ASEqlaZByDWyQv2bPU by Cmastication@mastodon.social
2023-02-01T14:38:46Z
0 likes, 1 repeats
@zleap @ben @shriramk spaces, tabs, line feeds. Stuff that’s white on a printed page.
(DIR) Post #ASEquNU3ysG2Sppha4 by zleap@qoto.org
2023-02-01T14:40:23Z
0 likes, 0 repeats
@Cmastication @ben @shriramk Ah thanks for clearing that up.
(DIR) Post #ASExgDOlNgaYoFY1Bo by shriramk@mastodon.social
2023-02-01T15:31:56Z
0 likes, 0 repeats
@Cmastication @zleap @ben But I don't this MOSS would be very effective without the language-sensitive parts. Those do a critical bit of normalization to make the fingerprinting actually "work". And that part is definitely not "matching literal strings".
(DIR) Post #ASExgDphlW8i9o9XBQ by emeryberger@discuss.systems
2023-02-01T15:39:28Z
0 likes, 0 repeats
@shriramk @ben @zleap @Cmastication agreed, the big wins for MOSS are arguably just language-specific tokenization, removing white space and comments, and canonicalizing all variable names. Fingerprinting is easily foiled, including automatically. See our Mossad paper for more - https://arxiv.org/abs/2010.01700
(DIR) Post #ASExgEKttX5piYkRo8 by Cmastication@mastodon.social
2023-02-01T15:43:20Z
0 likes, 0 repeats
@emeryberger @shriramk @ben @zleap solid work.
(DIR) Post #ASExgEj0RuNKvK1hNg by emeryberger@discuss.systems
2023-02-01T15:46:50Z
0 likes, 0 repeats
@Cmastication @shriramk @zleap @ben I don’t think ChatGPT (yet) knows enough to foil MOSS at scale (as MOSSAD can - taking one source and producing hundreds of variants that all defeat MOSS) but an appropriately tuned prompt would go a long way.
(DIR) Post #ASExgFFGVyBCXN7SfA by Cmastication@mastodon.social
2023-02-01T15:48:10Z
0 likes, 0 repeats
@emeryberger @shriramk @zleap @ben that makes sense to me. I was being a bit cheeky with my example.
(DIR) Post #ASExgFfqv7RlrpYh6W by emeryberger@discuss.systems
2023-02-01T15:55:57Z
0 likes, 1 repeats
@zleap @shriramk @ben @Cmastication There is current ongoing work by Scott Aaronson at OpenAI and other groups (this looks promising: https://arxiv.org/abs/2301.10226) on effective watermarking strategies
(DIR) Post #ASGvXSIrmW7hqnGExk by WiseWoman@fediscience.org
2023-02-02T11:22:52Z
0 likes, 0 repeats
@shriramk @Cmastication @zleap @ben I would be careful calling it the most widely used text-matching software. Thrre are many systems with many variants on how they work.
(DIR) Post #ASGvXSqtjzLTYLBQ0W by shriramk@mastodon.social
2023-02-02T13:29:31Z
0 likes, 1 repeats
@WiseWoman @Cmastication @zleap @ben I am sorry, you are of course absolutely correct. I meant "for software in courses", and forgot the broader context.Incidentally, Prof. Weber-Wolff, your Nature article on plagiarism detectors [https://www.nature.com/articles/d41586-019-00893-5] is one of the readings in my introductory computer science course, tied to an assignment about document comparison [https://cs.brown.edu/courses/cs019/2022/docdiff.html#%28part._dd-rcs%29]. That article is more relevant today than ever!
(DIR) Post #ASHnj74rv3Xz1HimnI by BarrenPlanet@c.im
2023-02-02T10:26:58Z
0 likes, 1 repeats
@ben The software only uses two metrics: perplexity and bounciness. Those of us who write crisply and simply, with a steady rhythm, can generate false positives.Essentially, it works on the implicit assumption that all humans write in a loose, conversational style. This assumption is unsafe for formal and/or academic texts.