[HN Gopher] Inside the university AI cheating crisis
___________________________________________________________________
Inside the university AI cheating crisis
Author : pseudolus
Score : 59 points
Date : 2024-12-15 23:17 UTC (1 days ago)
(HTM) web link (www.theguardian.com)
(TXT) w3m dump (www.theguardian.com)
| HomeDeLaPot wrote:
| How do students that cheat with AI pass the exams?
| tkgally wrote:
| A lot of university courses don't have exams. Rather, the
| students are graded based on projects, papers, or
| presentations.
|
| I've been teaching at a university for more than 20 years, and
| only a few times have I given students a final exam. I used to
| assign final papers, but I stopped doing that in the spring of
| 2023, when ChatGPT was becoming widely known and I couldn't
| decide how to deal with its use by students.
|
| The big issue, in my opinion, is that AI can be used
| productively and ethically in education; it can be used to
| cheat; and there's a huge gray area between those two extremes,
| where there doesn't seem to be any consensus yet.
|
| For example, suppose a student uses Claude to brainstorm
| topics, then chooses one from among those topics and researches
| it in depth, then does some more brainstorming with ChatGPT
| based on what he or she found, and then jots down what he or
| she wants to say as bullet points. Finally, Gemini is used to
| write a paper that presents the information in those bullet
| points in a logical and well-formed manner, and the student
| checks the paper and makes revisions before turning it in. Is
| that okay?
|
| When I've discussed this issue with humanities faculty, they've
| generally regarded that kind of AI-assisted writing to be
| cheating. Science faculty, however, have been more receptive to
| it, as they care more about the accuracy and originality of the
| ideas than about how the words are strung together. My sample
| was small, though, and I am sure that there are different
| opinions on both sides.
|
| I retired from my full-time university post in March 2023. Now
| that I'm teaching only one class a semester, I have each
| student do a final one-on-one presentation and interview with
| me, which makes up a large part of their grade. I'm able to do
| that because I'm teaching only that one class. If I had a full
| teaching load with a hundred students or more total, there's no
| way I could interview each one individually.
| eddsolves wrote:
| A few universities scrapped exams during Covid and didn't bring
| them back, focusing on coursework instead
| ars wrote:
| If you use AI exclusively I assume they won't. But I feel like
| most people who use it, use it to save some time - like they
| have the general idea of what they want to say, and just need a
| little help polishing it into a final form.
|
| Like that joke about how you write a summary, use AI to expand
| it, then the recipient uses AI to get a summary. It's no longer
| a joke - it's really happening.
|
| I've also seen students use AI to understand material - instead
| of reading the material, they feed it through AI to explain it
| to them. This is either a mediocre student who'll have a hard
| time if they ever need to do original research, or it's a bad
| teacher. Either way they'll do OK on the exams.
| gixco wrote:
| I graduated before LLMs but after WolframAlpha and I
| essentially cheated my way through calculus I and II. Lenient
| grade weighting made 90s on the homework and 60s on the exams
| enough to slide through with a C. Funnily enough, now that it's
| over a decade later and I know more about myself and my
| neurodivergent patterns, I feel much more able _and_ interested
| in actually learning calculus. I 'm looking forward to seeing
| the pedagogical changes that result from LLMs enabling this
| sort of trickery for all subjects.
| mackross wrote:
| We've built a solution in conjunction with a university to this
| problem that is pretty low effort to implement, but very few
| professors can be bothered to even try it out (the apathy and red
| tape is unreal). Honestly, it has been disheartening that
| distribution is so tough, as the results have been great for
| those who are using it.
| fardinahsan wrote:
| Could you share the product and some details about it? I'm
| curious.
| lcnPylGDnU4H9OF wrote:
| > very few professors can be bothered to even try it out
|
| Do you happen to know if there is some overlap between these
| professors and professors who also refuse AI-detectors? Apathy
| could be the reason but I wonder how much of it is driven by
| cynicism encouraged by the inefficacy of AI-detecting tools.
| michaelmior wrote:
| As a professor would be interested in a solution to this
| problem, I'd be curious to see hear more details. FERPA issues
| can sometimes make it difficult to adopt solutions where
| student info of any kind needs to be sent to a third party.
| alins wrote:
| Would you please share more details about this?
| a_bonobo wrote:
| TurnitIn's cheating detection tools have always been garbage -
| and I say that as an ex-academic who was 'forced' to use those
| tools extensively (the PDFs submitted by students are only
| visible in the Turnitin portal with the 'flags' to the right).
|
| Even before the 'AI detection' tool (IMHO snake-oil) the features
| around copy-pasting/plagiarism were so bad I always ignored them.
| The tool would flag things like commonly used coverpage forms.
| That universities pay a fortune for those tools is an indictment
| of the industry.
| papreclip wrote:
| The one student who got caught for plagiarism when I was a TA
| was busted by turnitin. Maybe the tool isn't perfect but it
| caught this student who had painstakingly replaced every single
| word in a copied paper with a synonym. I'm talking "perhaps the
| utility is not flawless, but the use of this utility allowed
| for the apprehension of a pupil who..." etc.
|
| I was surprised at the level of effort
| KittenInABox wrote:
| I'm always shocked at how much students will put in effort
| simply not to do an assignment given. I guess this is just
| part of what you will get when you attach cultural and
| economic worth to a set of hoops to jump through [even if the
| intent is to have institutions of knowledge, sadly I think
| many students just see it as a way to get economic
| stability].
| christianqchung wrote:
| Unrelated to the cheating part, if you don't see American
| colleges primarily as a path to economic stability, I think
| you are probably either rich that it doesn't matter or
| haven't seen the bottom 25% outcomes of college graduates
| who didn't treat it as a financial opportunity/investment
| and are struggling.
| falcor84 wrote:
| More likely that they used something like quillbot or
| grammarly for automatic paraphrasing.
| papreclip wrote:
| This was about 15 years ago
| ffsm8 wrote:
| grammarly is pretty old, Not sure when it became
| available, but it was already pretty well established as
| a high quality writing assistant by ~2012.
|
| It had various tools and heuristicsv to suggest different
| phrasings and better words that were pretty incredible if
| compared to anything else back then
| joe5150 wrote:
| So it either only caught a single cheater among some unknown
| number, or only a single student cheated? Neither case seems
| to justify the expense and hassle of subjecting everybody to
| this process, not to mention everything else wrong with
| TurnItIn.
| michaelmior wrote:
| As a current academic who uses Turnitin regularly, I find it
| works fairly well. There are a lot of false positives, but it
| also regularly identifies actual plagiarism for me.
| carabiner wrote:
| I am enormously glad I went to college before these tools
| existed. I don't know how I would handle myself with access to
| them.
| ars wrote:
| It depends on your goal: Do you just need that piece of paper
| with your name on it? Or are you going to school to really
| learn material?
| mmooss wrote:
| What we do depends on much more than our goals. There's also
| stress, time pressure and workload, emotional problems, money
| issues, competitive pressure, fears (of failure, of
| disappointing important people, etc.), exhaustion, etc.
|
| People don't do bad things because that's their goal but
| because of these issues. If you give humans an easy way to do
| bad things, many people under pressure will do it.
| mupuff1234 wrote:
| You also know that you probably shouldn't eat that extra
| cookie and yet...
| carabiner wrote:
| We all start out with good intentions. Do we all know if we
| would follow them to the end?
| proteal wrote:
| I did undergrad before chatGPT and now I'm back in school for
| an MBA at a school where the school pays for all students to
| have access to GPT4o. Maybe because we're all older the tools
| are less of a problem, but I would say they have been a net
| positive. GPT3 was ass and students could easily tell when a
| classmate used it for their group contributions. Profs seems to
| be about 6-12 months behind the curve in terms of the tech. All
| profs are aware and thinking deeply about how AI is changing
| how they teach.
|
| Group papers are usually made into an outline at first and then
| we divvy up the responsibilities. If you use AI, most kids only
| care if your work sucks. AI consistently scores like an 80-85%
| (profs sometimes submit and blindly grade the responses), but
| almost always misses the core teaching points of classwork.
|
| In my program, grades don't matter (so employers can't stack
| rank us). People who make extensive use of AI are really only
| cheating themselves. If you're a big AI user, other kids
| generally know and try to avoid forming groups with you if they
| know in advance. You learn better when learning alongside
| others and if all someone does is dump AI slop in the google
| doc, you're wasting my time in addition to yours.
|
| I use AI to flesh out points, especially on assignments I don't
| care about. It can help for idea generation and "connecting the
| dots" between ideas, but I always edit the output because the
| AI makes stylistic choices I don't like. It's definitely an
| accelerator for me when writing papers. I stand behind all the
| papers I've submitted, though some have sucked (regardless of
| AI usage or not).
|
| In undergrad, when my priorities were less about learning and
| more towards dating/partying, I definitely would have abused
| this tool. At the end of the day, using the tool mainly cheats
| your own learning. I hope they transition to talking about
| using LLMs like one uses gambling - a little is fine here and
| there but if it's all you do that's a problem.
|
| I'm not sure what to do about elementary age kids, because the
| AI easily writes "better" essays. At least in college I could
| do better than an AI if I applied myself. But in sixth grade?
| Good luck me. Cats out of the bag now and we should be really
| empathetic to the younger generations. Imagine getting slammed
| with TikTok->Pandemic->ChatGPT in the span of like 6-7
| formative years. They are growing up differently and I
| certainly have no clue what we need to do to help them be
| successful.
| carabiner wrote:
| If I could wave a wand to wish away all LLM's I would do it
| in a heartbeat. More harm than good.
| histriosum wrote:
| > In my program, grades don't matter (so employers can't
| stack rank us).
|
| Can you expand a bit on how that works? I have limited
| academic experience, so I'm fascinated. Does everyone end up
| with a 4.0 if they pass, or..?
| michaelmior wrote:
| > I hope they transition to talking about using LLMs like one
| uses gambling - a little is fine here and there but if it's
| all you do that's a problem.
|
| If I hire someone to manage my money, I don't want them to do
| any gambling. Although given that investing is somewhat of a
| gamble, I at least want to set the terms and have them to
| disclose to me exactly how they're gambling.
| indymike wrote:
| This is an easy to solve problem: oral exams / interview exams.
| gedy wrote:
| Agreed, but you know someone will shout bias or whatever
| metal_am wrote:
| That's how grad degrees are done. But there's simply not enough
| time to do this at an undergrad level with potentially hundreds
| of students.
| bnchrch wrote:
| Unless... You have an AI listening and evaluating their oral
| exam.
|
| I know it sounds like a farce. Because it is. But it might
| also be a proper solution.
| michaelmior wrote:
| Not really in my experience. That's how final PhD
| examinations are typically done. Outside of that, it's pretty
| rare. That said, PhD students are also regularly interacting
| with faculty members and discussing their work so it would be
| difficult to get by without actually knowing things.
| User23 wrote:
| Today's first year undergraduate is roughly as literate as a
| sixth grader was a century ago. College was never meant to be
| a mass remedial education program and by attempting to be one
| it's failing at its core mission.
|
| All this nannyware of various sorts is just more evidence,
| proof even, of how few of those students should even be
| there.
| UncleMeat wrote:
| Rough when you have 200 students.
| tverbeure wrote:
| Rough, but perfectly doable. It's how pretty much all my
| finals were done way back when. Classes of 200 and more.
| michaelmior wrote:
| Depends on the institution. If you have one faculty member
| for all those students and no TAs (yes, this happens) then
| it's going to be rather c hallenging.
| mupuff1234 wrote:
| Or just regular written exams.
| BrenBarn wrote:
| That's mentioned towards the end. The problem is it requires
| much more time and labor to give oral exams.
| somnic wrote:
| Or just any normal proctored exam?
| kelsey98765431 wrote:
| > "What if I'm doing an MA, or in a job, and everyone got there
| just by cheating..."
|
| This is absolutely the problem here. The work is done, the
| answers are right, the report is the best anyone has seen, but oh
| no a fancy new tool was involved so its CHEATING! This is so
| friggin stupid. When i get my work done I am responsible that it
| comes out right. When its right, thats just it. - it's right. No
| matter how i got there it's fine. But wait, what if i used some
| freelance platform to farm out my job to another person then turn
| in their work? Well that's breach of NDA and my terms of
| employment. So what, are the student over here committing wage
| theft of some kind even though they are the ones paying for
| tuition? Make this make sense to me, you can't. School is a job
| you pay to do so you can learn to get good at a job you get paid
| to do. Somehow school ends up being a bad investment and we get
| mad at the students? Schools are broken and useless and at this
| point i would prefer my doctor was great at using chatgpt and
| checking the results carefully over being taught at an
| educational institution.
| scandox wrote:
| The purpose of most educational productions is not to produce
| some correct thing but to undergo a process of learning
| required to produce the thing.
| timenotwasted wrote:
| And now academia needs to adapt since they can't just assign
| out all the learning to the students. All this is doing is
| exposing how broken the entire concept of higher-ED learning
| as been for quite some time.
| ARandumGuy wrote:
| Why do we teach kids how to do basic arithmetic by hand? After
| all, calculators are basically ubiquitous at this point. Why
| waste the time teaching a skill that's not necessary as an
| adult?
|
| The reason is because doing arithmetic by hand teaches kids how
| math actually works, while using a calculator just teaches them
| how to find the answer. This base-level understanding is
| crucial for understanding more advanced math. But it's also
| important for making mental estimations, or just having a basic
| grasp on how the world works.
|
| This is why students using ChatGPT is a problem. If you never
| put in the actual work in the first place, you never gain any
| actual understanding. A doctor "great at using ChatGPT" won't
| know how to check the results carefully if they've relied on
| ChatGPT for their entire education. Having a baseline skill set
| is crucial for evaluating LLM output, and for handling
| situations that LLMs can't handle. And there will _always_ be
| situations LLMs can 't handle, regardless of how advanced genAI
| becomes.
| lcnPylGDnU4H9OF wrote:
| This seems like it's replying to a misunderstanding of their
| comment.
|
| > A doctor "great at using ChatGPT" won't know how to check
| the results carefully if they've relied on ChatGPT for their
| entire education.
|
| They _didn 't_ rely on ChatGPT for their education; they
| relied on ChatGPT for finishing homework. If they go over the
| generated essay and make sure it doesn't have any
| inaccuracies and they _consistently do so correctly_ , what's
| the problem?
| falcor84 wrote:
| > Why do we teach kids how to do basic arithmetic by hand?
|
| I think kids waste way too much time doing hard arithmetic by
| hand. As a particularly egregious example, I visited a
| Montessori primary school where they demonstrated what is
| essentially a complex single player board game, for
| calculating square roots by hand. It's a process whereby a
| 5th grader would take ~10 minutes to calculate the root of a
| 3 digit number, by moving dozens of pins around. It looked so
| silly, and while other Montessori equipment offers
| geometrical intuition, this one seemed to offer none.
| Squaring with trial and error would have been so much easier.
|
| Other more typical examples are of kids spending hundreds of
| hours practicing long multiplication and long division of
| e.g. 4 digits, and then getting punished and stressed out
| about having made silly mistakes. While in the real world, no
| one in their right mind would choose to perform these
| calculations by hand any more, and would instead use a tool
| incapable of making arithmetic mistakes.
|
| I don't have a clear solution, but would tend towards an
| educational system that teaches only the very basics of
| performing each mathematical operation (probably up to the
| level the average literate adult would do mentally), and then
| focus the rest of the time on problems where the students are
| the ones posing the question, using a calculator/computer to
| solve it, and then check whether the answer makes sense in
| the original context.
|
| EDIT: For those curious about it, I just found a blog post
| demonstrating the logic behind this square root peg board.
| The author of the blog argues that it does help build
| understanding, but from what I saw, I would politely
| disagree.
|
| https://borisreitman.medium.com/square-roots-the-
| montessori-...
| ARandumGuy wrote:
| Obviously there's a balance to be struck. There are
| certainly schools and teachers that teach arithmetic
| poorly, and there's a practical limit to the amount of
| hours spent doing equations before it becomes tedious
| busywork.
|
| But I still think it's important that kids are taught how
| to do arithmetic by hand. And part of that will involve
| repetition, both to help reinforce and practice skills, and
| so educators can more accurately gauge a student's ability.
| You can't just bypass the need to sit down and do the work,
| even if the exact amount of work necessary is debatable.
| christianqchung wrote:
| This argument can be extended downwards infinitely. You can
| argue against teaching addition and writing of any form
| with this mindset.
|
| I also think this argument means that theoretically someone
| with a 6th grade education and a 12th grade education
| should have the same economic output, as they both know how
| to read and add/multiply numbers. In real life the reason
| someone drops out before/during high school is always
| extenuating life circumstances, so the outcomes are much
| worse for them. But in a world where that didn't happen, do
| you think the outcomes would be essentially identical?
| User23 wrote:
| > what is essentially a complex single player board game
|
| Perhaps this sounds silly, but I think that's actually
| deeply interesting. There is a profoundly deep relationship
| between calculation and games. This applies not just to
| ordinary arithmetic, but also things like game theoretic
| proofs to prove the soundness of logical inference rules.
| Exposing children to that relation, even with a toy case,
| sounds like it could be fruitful.
| FigurativeVoid wrote:
| But this was always the case. When I was studying chemical
| engineering, it was an open secret that all the students were
| cheating. There was a giant google drive with all the solutions
| manuals, old tests, old programs, lab reports, whatever you
| wanted.
|
| That's why pencil and paper tests were worth 80% of the grade.
| michaelmior wrote:
| > Well that's breach of NDA and my terms of employment. So
| what, are the student over here committing wage theft of some
| kind even though they are the ones paying for tuition?
|
| Students are typically agreeing to submit original work. It's a
| breach of the terms they agreed to, so I'm not sure it's so
| different.
|
| > i would prefer my doctor was great at using chatgpt and
| checking the results carefully over being taught at an
| educational institution.
|
| What is checking over the results carefully going to do if your
| doctor has never been taught medicine?
| codingwagie wrote:
| Maybe this will finally liberate humanity from educational
| prestige dominating life outcomes.
| mmooss wrote:
| What about education?
| dhosek wrote:
| We'll be free of that as well.
| spudlyo wrote:
| I'd never let school get in the way of my education!
| codingwagie wrote:
| :)
| User23 wrote:
| If these students cared about education they wouldn't be
| using AI to avoid learning how to do the work.
| krunck wrote:
| My wife is an English prof at a local University. She is tired of
| being put in the position of being an AI cop. She is tired of 10%
| students whose work was flagged wasting 50% of her office hours
| trying to convince her that their work is real.
|
| For her literature classes - where the class grade is composed of
| a participation grade and a written work grade - she is strongly
| considering getting rid of all written homework. Instead all
| writing work will be _hand written_ in class.
| gizajob wrote:
| I've been saying for a couple of years now that academic essays
| are going to have to go back to being handwritten. It seems
| like possibly the only solution.
|
| The job of essays isn't to produce a Word document of text that
| can get a grade above 70 - it's to demonstrate and improve the
| student's writing and thinking. The computer seems to have now
| completely messed this up.
| mmooss wrote:
| We need validation of GPT detection tool accuracy (false/true &
| negative/positive), and in what circumstances, or they are
| useless. Without independent information about whether something
| is GPT-written [1], there is no way to verify it, so they can't
| be verified on submitted student papers where the info is from
| people (students) with very strong incentives to lie (not wanting
| to be caught).
|
| Here are a couple references in the article, but I would like to
| see a review of this question:
|
| > Researchers at the University of Reading recently conducted a
| blind test in which ChatGPT-written answers were submitted
| through the university's own examination system: 94% of the AI
| submissions went undetected and received higher scores than those
| submitted by the humans.
|
| > One study at Stanford found that a number of AI detectors have
| a bias towards non-English speakers, flagging their work 61% of
| the time, as opposed to 5% of native English speakers (Turnitin
| was not part of this particular study).
|
| [1] That needs to be defined: Every word? mostly? GPT-written
| then edited? What about ideas from the humans and writing by GPT?
| First draft by human and editing by GPT - what about using it as
| a grammar checker? For suggestions on clarifying some tricky
| passages? ?
| ramoz wrote:
| Prompts need to be integrated into grading/eval of where students
| are mentally. Like - cite/attach your prompts to the paper, code,
| whatever.
| cpcpLLM wrote:
| I think that there will be people that will only learn to copy
| LLM generated text, and they will be not able to look like people
| that learn.
| vander_elst wrote:
| Why cannot this be solved by written exams?
| MrMcCall wrote:
| In an world with honorable human beings raised in Wisdom, we
| would understand that cheating only hurts ourselves in the long
| run, but that is not our world majority's ethical zeitgeist at
| the moment.
|
| The lack of ethics, like all selfishness, propagates
| inefficiencies into the greater society, as exemplified here for
| our overworked and underpaid teachers.
|
| Our son has become quite a good chess player over the past 6ish
| years, during which online chess cheating via engines has
| proliferated. I explained to him that he is completely
| responsible for being able to honestly say, "No." in answer to
| the question, "Have you ever cheated in online (or over-the-
| board) chess?".
|
| Having a decision matrix that leads to ethical action confers a
| specific peace that accompanies knowing that one has done nothing
| wrong. In a world full of liars and cheats, _being_ honest is its
| own very special and quite rare power, looks to me.
| babkayaga wrote:
| there is no crisis. stop teaching students to write trash, using
| ai or not. if the essay is good, it should get an A.
| from-nibly wrote:
| Just came to the same conclusion with my professor father in
| law. If AI can actually do a good job at writing some
| assignment then people shouldn't be bothering with that thing.
| If it can't it will not matter that it was written by AI it
| will just be bad.
|
| Also our universities are failing us if the only thing we get
| at the end is a boolean piece of paper (or even one represented
| by a floating point number).
|
| This is like servers at McDonald's complaining that the
| customers are cheating with trash cans because they aren't
| eating the meals they buy, just throwing them in the trash. Why
| would they do that?!?!? Because someone told them they need to
| go to McDonald's "for the experience"?
| itishappy wrote:
| Is content the only thing that matters? What if the whole class
| submits the exact same essay? What if the next class submits
| the exact same essay too?
___________________________________________________________________
(page generated 2024-12-17 23:01 UTC)