[HN Gopher] DetectGPT: Zero-Shot Machine-Generated Text Detection
___________________________________________________________________
DetectGPT: Zero-Shot Machine-Generated Text Detection
Author : wwarner
Score : 36 points
Date : 2023-01-31 19:12 UTC (3 hours ago)
(HTM) web link (arxiv.org)
(TXT) w3m dump (arxiv.org)
| skissane wrote:
| I had a go at using ChatGPT to write some (not very good)
| fiction. I give it my story idea, and it gives me a few
| paragraphs. I edit its output, keeping the stuff I like, deleting
| things I don't, rewording some things, adding a few sentences of
| my own. Then I say "Now write a new paragraph about how <new
| story element>". I read what it says and then I say "Change what
| you just said so that...". Then I paste that new paragraph into
| my document at the appropriate point. So the text I'm
| constructing is neither purely ChatGPT nor purely me, it is a mix
| of my ideas and its ideas, my words and its words. I wonder if
| this tool would detect such a text as ChatGPT-generated or not?
| drstrangevibes wrote:
| [dead]
| lostoldsession wrote:
| Is there really something of value to be had by fighting a war
| against the coming tide of AI that can spit out whatever you ask
| for?
|
| To me it looks like old world professors calling the council
| because the abacus is under threat by the calculator, and we need
| to ensure the students are absolutely not using the calculator
| for their abacus studies. This all being despite the fact that
| society at large is moving as fast as it can to leave the abacus
| for calculators.
|
| I can't help that but feel that people have lost the forest for
| the trees. So transfixed on the small steps that they have
| completely lost track of why we take all those small steps, and
| the utility that they ultimately provide.
|
| If you want to study whatever subject and become an expert,
| history or chemical engineering, cool, go ahead. I'm still gonna
| chose my AI consultant over you in 5-10 years.
| gitfan86 wrote:
| It is valuable for SPAM filters, and other fraudulent activity.
| If you get an e-mail from someone you know and it says "I need
| itunes gift cards right away!" most people know that is a scam.
| But if you get an e-mail that appears to be following up on a
| previous conversation, you are much more likely to click on a
| link in that e-mail.
| rosywoozlechan wrote:
| I doubt spammers will be the only ones leveraging AI
| generated text. I'm sure legit marketing people and even
| administrators and support staff will benefit from using AI
| written responses. So you're going to filter out valuable
| emails as well.
| mehlmao wrote:
| There are no valuable emails from 'legit marketing people',
| especially so if they're using AI-generated messages to hit
| you with a firehose.
| mistymountains wrote:
| Typically, outside experts are useful for out of distribution
| inference about a particular situation. Fine-tuned generative
| AI is self evidently not good at generalizing outside the scope
| of its training data, and becoming reliant on it for this task
| is foolish. Signed, AI researcher.
| sublinear wrote:
| > I'm still gonna chose my AI consultant over you in 5-10
| years.
|
| AI will never replace experts.
|
| Someone still has to curate the firehose of information
| returned. What's the point then?
|
| AI can only interpolate a response from internet sources, and
| claims of plagiarism are already coming hard. The internet was
| never a good source of expertise.
| skor wrote:
| these kind of comments "I will hire an AI instead of a human"
| are all quite ironic.
|
| the only things you will find out from asking an AI consultant
| is what occurred to _you_ to ask it.
|
| Interacting with a human will always give you more ideas,
| opinions and .. more questions.
|
| And you both can use the AI consultant, and get that important
| thing done.
| maxutility wrote:
| > This approach, which we call DetectGPT, does not require
| training a separate classifier, collecting a dataset of real or
| generated passages, or explicitly watermarking generated text. It
| uses only log probabilities computed by the model of interest and
| random perturbations of the passage from another generic pre-
| trained language model (e.g, T5).
|
| Very interesting, though the passage above makes me wonder how
| robust it is to different models or even finetuned variations on
| models, as even GPT-3(.5) has evolved quite a bit over recent
| releases since its initial introduction, and there is likely to
| only be a greater and greater proliferation of models over time.
| [deleted]
| rogers18445 wrote:
| The approach to feed text through a language model and then
| obtain some kind of vector/hash from it will be leveraged to
| fingerprint writing and track people.
| flandish wrote:
| In addition, this can feed a sort of "recaptcha" loop that
| trains AI to notice AI and create new text less likely to be
| noticed...
| wwarner wrote:
| "Unlike human-written text, model-generated text tends to lie in
| ar- eas where the log probability function has negative curvature
| (e.g., local maxima of the log probability)."
| [deleted]
| pelasaco wrote:
| > For example, students may use LLMs to complete written
| assignments, leaving instructors unable to accurately assess
| student learning.
|
| I'm one student using GPT to write my written assignments, and I
| cannot understand what is the issue with that. Even though it
| enhances the quality of my text, I cannot just copy, paste,
| submit and profit. I still have to understand what is being
| written. My workflow is normally the following:
|
| - I ask for an outline to the essay. I use this outline as guide,
| remove some suggestion, add others that I think are important,
| rephrase some. I could as well google for similar written essays
| and do the same.
|
| - Then I ask for the topics, please describe X, Y.. I read it, as
| a regular source. I understand it, and rewrite it. I add stuff,
| remove stuff, add references. Normally I endup rewriting
| everything
|
| - Push the text through Grammarly, rephrase some stuff, check for
| plagiarism and submit it. I have to read and re-read it, many
| times.
|
| I have the impression that I've got much more effective avoiding
| the mechanical part of the task of writing. In another hand my
| course has some stupid enforcement like 15k words/essay. So they
| are almost asking me to do some bullshit around the topic instead
| of going straight to the point. I simply automate the burden. As
| any software developer would do. I however study philosophy.
|
| So i guess, it will help the universities and education
| institutions to rethink their traditional way of evaluate written
| assignments, which probably will be positive. It helps me
| definitely to write better.
| spikej wrote:
| The problem is that people aren't doing what you're doing, and
| ARE just copy/pasting whatever gets spit out. Using it to
| enhance your wording + thoughts is absolutely the way to make
| use of these tools.
| janalsncm wrote:
| From a technical standpoint, this is interesting. Like all other
| methods I've seen, it requires access to the original model
| logits, so it can't be used in closed models like GPT-3 except by
| OpenAI themselves.
|
| But I would classify this into the category of dangerous
| research. There is a conflict of interest: overstating the
| effectiveness leads less technical people to give undue trust to
| what is ultimately a statistical decision. And it shouldn't
| require repeating how serious the consequences of this decision
| are.
|
| The most likely outcome is that methods like this are used to
| diffuse responsibility. The next step is for institutions to buy
| into one of these classifiers and then to create a punitive
| "policy".
| canjobear wrote:
| You can get all the necessary log probabilities from the GPT-3
| API.
| wwarner wrote:
| Seems to me you might be able to guess without the weights by
| manually changing a word or two and seeing how much gpt wants
| to change.
| janalsncm wrote:
| Hmm seems you may be right. You can get the top 5 out of the
| box but you'll need to file for an exception to get any more
| than that.
| The_Colonel wrote:
| I can't judge the capabilities of this tool, but I can't help but
| be skeptical on the mid-long term prospects. The fakes (text,
| audio, video) will improve to the point of being virtually
| unrecognizable from real material, no matter how good your fake
| detection is.
| wwarner wrote:
| Can some chess players detect when an opponent is using
| stockfish? Yes, if they're Magnus Carlsen and memorized a
| particular line of play. This is a bit like that. But yes, I
| suppose when these tools become so fluent that they basically
| never need to use the same pattern twice, we're toast.
| The_Colonel wrote:
| AFAIK top chess players can recognize they're playing against
| an engine relatively easily. The engines are so far ahead
| that they often play moves which are "surprising" even for
| Carlsen, i.e. lines which may seem suboptimal or just unknown
| - this would normally indicate a weak player, but engines
| then crush even Carlsen with ease.
|
| But these engines are optimized to win, not to mimic humans,
| so it's a different story from ChatGPT.
| jsf01 wrote:
| It's typically not the moves alone that indicate engine use
| to the other player. The timing is also incredibly important.
| If it takes you 5 seconds to recapture a piece, where that's
| the only viable move, but it also takes you 5 seconds to come
| up with an insane brilliancy that's the result of a deeply
| calculated line, cheating is a likely explanation.
| gillesjacobs wrote:
| I found a great way to fool these detectors: piping output
| generative models.
|
| 1. Generate text by promoting ChatGPT.
|
| 2. Rewrite / copyedit with Wordtune [1], InstaText [2] or Jasper.
|
| This fools GPTZero [4] consistently.
|
| Of course soon these emotive, genre or communication style
| specialisations will be promptable too by a single model too.
| Detectors will be integrated as adversarial agents in training.
| There is no stopping generative text tooling, better adopt and
| integrated it fully into education and work. Resistance is
| futile.
|
| 1. https://www.wordtune.com/
|
| 2. https://instatext.io/
|
| 3. https://www.jasper.ai/
|
| 4. https://gptzero.me/
___________________________________________________________________
(page generated 2023-01-31 23:00 UTC)