Post AcEsS89ETfJ2PfPNjM by clive@saturation.social
(DIR) More posts by clive@saturation.social
(DIR) Post #AcETV9vpntMBxbUNvM by clive@saturation.social
2023-11-27T12:58:15Z
0 likes, 0 repeats
In this paper in Nature, the researchers find that essays generated by ChatGPT were rated higher for quality than human-written onesInteresting on its face, but …… what really drew my eye was this finding: human students are more likely to use language that conveys “speaker attitude“I wonder how much of that is due to ChatGPT’s relatively low temperature (to minimize weird results) and/or prompting?https://www.nature.com/articles/s41598-023-45644-9 (Via @jasemrau)1/2
(DIR) Post #AcETVEiC22Pqmk93r6 by clive@saturation.social
2023-11-27T12:58:15Z
0 likes, 0 repeats
I’m also intrigued by that point about ChatGPT using more “nominalizations” in its text Here’s a little chart showing what nominalizations are (https://law.rwu.edu/sites/law/files/downloads/Academic%20Success/Nominalizations%20Handout%20-%20With%20Logo.pdf)I personally find their overuse a mark of dull, wordy style — not sure why ChatGPT has picked up on them. Maybe they’re used too often in the human-authored training data? Possibly …2/2
(DIR) Post #AcEVZcdTE2Ka3B9gcS by atthenius@fediscience.org
2023-11-27T13:21:23Z
0 likes, 0 repeats
@clive First impression: both my kids were taught to regurgitate this dry, formulaic, easy to grade style of writing to facilitate the proliferation of standardized tests that goes by the pneumonic RACES in all core subjects. I can see machines excelling at that. https://www.purdue.edu/science/K12/races.htmlBut I see the authors are based in Germany. But the grading the emotional nuance of the language in English?Learning new tools is good but fwiwI’d rather my kids get taught R than how to use a calculator.
(DIR) Post #AcEViIbd44WoXGQoRU by PaulGrahamRaven@assemblag.es
2023-11-27T13:23:02Z
0 likes, 0 repeats
@clive "Rated highly" on what metric, what set of standards? the sort of writing that _Nature_ considers publishable, for instance, is a very particular subgenre of the academic style, which -- as countless complaints from no-academics attest -- is not greatly received outside of the sacred grove. That these models can reproduce the turgidity of the OA papers they were presumably trained on is no surprise; whether that reflects well on them (or on academic writing) is another question entirely.
(DIR) Post #AcEfILPj08LFFz7roO by Transportist@mastodon.social
2023-11-27T15:10:20Z
0 likes, 0 repeats
@clive yeah, I think it’s from their training set for scientific and formal writing, which is worse than we would like. You can of course tell it to write in whatever style you want, and it does well.
(DIR) Post #AcEjJpFxvrLluzWU64 by clive@saturation.social
2023-11-27T15:55:29Z
0 likes, 0 repeats
@Transportist Yep yep
(DIR) Post #AcEjQDCnns7Fvr5Y92 by clive@saturation.social
2023-11-27T15:56:26Z
0 likes, 0 repeats
@PaulGrahamRaven I think it was rated according to standard student-essay-grading heuristics With all the limitations that that implies
(DIR) Post #AcEjWKCCj5phLkivKa by PaulGrahamRaven@assemblag.es
2023-11-27T13:25:17Z
0 likes, 0 repeats
@clive (Throwing no shade, here: I just came off a ten-year stretch in the ivory tower, and I understand the utility of academic prose, even as I lament the particular form it has taken as a result.)
(DIR) Post #AcEjWL0tgcNlsyRPLU by clive@saturation.social
2023-11-27T15:57:05Z
0 likes, 0 repeats
@PaulGrahamRaven yeah, I have the same thoughts about it — it has its utility, and also a lot of knock on problems
(DIR) Post #AcEjpURWqryodJ1w9Y by sewblue@sfba.social
2023-11-27T16:01:08Z
0 likes, 0 repeats
@clive I'm an engineer who writes and QC's technical documents for a living, often instructions for processes that can kill if done wrong.I spend hours training younger engineers to cut the fluff out of their language. It can be very difficult sometimes to untrain what "write a 5 page paper" on a topic light in content has done for our collective writing styles. Verbosity is rewarded over clarity of thought. "Specific to the specified" when "use" will suffice. Clarity must win, so I teach them to play a game to see how many words they remove from a sentence and have it make sense. What I see in Chat GPC is fluff manufacturing. The actual knowledge and thought goes into the prompt. The error rate in reading Chat GPC is going to be through the roof due to overly complex sentence structure, with too many concepts packed into a sentence. Or no concepts mentioned at all, just fluff to bulk up the word count.
(DIR) Post #AcEmGK5vdPUqGUqjrs by Coocho@lingo.lol
2023-11-27T16:28:25Z
0 likes, 0 repeats
@clive @jasemrau as long as the task of the essay is to smoothly restate the obvious, GenAI will naturally perform better than secondary school level language learners. The problems begin when any degree of original thinking is involved, for instance, at the university level, when we no longer train the formal writing skills but also pay attention to the content…
(DIR) Post #AcEnm3Q3GjO5lfxMVE by atthenius@fediscience.org
2023-11-27T16:45:21Z
0 likes, 0 repeats
@clive trying to get word count under the limit -> nominalizations out, concise writing in.
(DIR) Post #AcEpyGpZQkfgkLqUEq by clive@saturation.social
2023-11-27T17:09:58Z
0 likes, 0 repeats
@atthenius I feels like there ought to be much more creative ways to use LLMs in education apart from just generating essaysIe generating opposing views of a single point, synthesizing sources, generating examples of different ways of explaining a point All subject to the risks of LLMs bullshitting bien surAnd more to your point — man, just have the kids *read* more more more That’s how you learn style and substance
(DIR) Post #AcEsS89ETfJ2PfPNjM by clive@saturation.social
2023-11-27T17:36:06Z
0 likes, 0 repeats
@sewblue Alas, yeah, that makes senseBoiling things down to precise and clear prose ought to be the goal we teach kids ... quite the opposite
(DIR) Post #AcEsXb8a7fDHloi9HU by clive@saturation.social
2023-11-27T17:37:13Z
0 likes, 0 repeats
@atthenius yes!
(DIR) Post #AcEsctLLrQJtgPCyiu by clive@saturation.social
2023-11-27T17:37:40Z
0 likes, 0 repeats
@Coocho @jasemrau Yes, precisely!