[HN Gopher] ChatGPT generated a puzzle game
___________________________________________________________________
ChatGPT generated a puzzle game
Author : danieltait
Score : 430 points
Date : 2023-03-06 08:39 UTC (14 hours ago)
(HTM) web link (puzzledpenguin.substack.com)
(TXT) w3m dump (puzzledpenguin.substack.com)
| erikstarck wrote:
| You guys are funny. An AI generates a game, comes up with the
| rules, writes the code and designs the web page for it. Your
| reactions:
|
| - Bah, it's not very fun.
|
| - It's been done before.
|
| - It took too long to make.
|
| Seriously. Let me repeat that. An AI generates a game. It comes
| up with the rules for the game. It even writes the code and
| designs the web page for it!
|
| Come on! This is amazing!
| bob1029 wrote:
| It is amazing.
|
| I worry that a lot of otherwise brilliant developers are going
| to get blindsided by this stuff.
|
| The current models are impressive in strong, quantifiable ways.
| They are only going to become more powerful from this point.
|
| Consider the current state of affairs: ChatGPT supports a 4K
| context size. Leaked foundry pricing indicates models that can
| handle 32K context size. 32K tokens is enough for your entire
| brand manual or several days worth of call center transcripts.
| Many products could have the most important parts of their
| codebase completely loaded into just the prompt.
|
| I would say you should at least try the OpenAI playground (or
| equivalent technology) to understand what is possible right
| now. I had no clue where we were at until ~3 weeks ago. I
| wouldn't wait until 2024 on this one anymore.
| olalonde wrote:
| Agreed. LLMs are on par with the invention web or the
| smartphone in terms of how much impact they'll have (possibly
| more). It's weird to see so many HNers being so dismissive of
| them. I've been using ChatGPT daily (mostly to ask
| programming related questions) and it's like having a new
| super power.
| moring wrote:
| You'll have to add one reaction to that list: "it didn't
| generate the game because something like that was in the
| training data".
|
| Nevermind that this perfectly describes 90% of software
| development.
|
| I'm actually wondering to what extent these responses are
| fueled by fear of being replaced by AI.
| beepbooptheory wrote:
| I know its getting popular, but I really doubt 90% of
| software development is generated from trained models...
|
| I don't understand what's so at stake with this that you feel
| like people are afraid. It's fun and amazing it can spit out
| stuff like this, and if you are a good developer
| experimenting with this stuff you already know its inarguably
| a novel and useful utility, if still limited in some ways.
|
| But where is the fire? Why does everything got to devolve
| into one vague culture war or another? Shouldn't you welcome
| good faith critique? If only for the fact that these things
| can still be improved, and how can you hope to improve them
| if you smother and dismiss every suggestion that these models
| might be less than perfect.
| moring wrote:
| I think you are fighting a strawman here. These models are
| far from perfect, and critique is certainly needed to
| improve them.
|
| That being said, I don't think saying that the model
| outputs information from its training set adds a lot to the
| discussion because -- and that's my point -- the same is
| true for human software developers most of the time (yes,
| the 90% is a made up number). This isn't meant at all to
| criticize the skills of the developers, but rather point
| out that most of our work is just much less interesting
| than we'd like it to be, and could be automated.
|
| Also, _I_ don 't think developers will be replaced by AI,
| just like they were not replaced by code generators, build
| scripts, IDE auto-complete, IDE rename all usages, and so
| on. What might happen is that they will no longer have to
| write mind-numbing boilerplate code, which IMHO is a good
| thing.
| xxs wrote:
| How do you relate: "An AI generates a game", and then "It's
| been done before." Obviously it didn't generate a game but it
| copied, which is not "amazing".
|
| On a flip note: the game is fun enough.
| chpatrick wrote:
| The HN AI cope is real.
| matsemann wrote:
| Did it come up with the rules, or did it just rehash an
| explanation for an existing game from its training data?
| speedgoose wrote:
| The model weights aren't that big, even compressed with the
| best algorithm possible.
|
| For such an obscure game I would guess it came up with the
| rules. It's difficult to prove though.
| tempestn wrote:
| The "it's been done before" one is pretty relevant. It means
| the model didn't actually generate the game, but likely pulled
| it more or less straight out of its training data. It's still
| very cool that you can ask it for something and it can
| basically mine the entire (2021) internet for it, but it's not
| the same as being able to create something really new.
|
| I've noticed the same thing testing it on various coding
| questions. It's extremely good at problems that have solutions
| online. And given stackoverflow, that's a lot of problems. If
| you manage to hit it with something that it hasn't seen before
| though, even if it's conceptually very straightforward, it
| tends to just generate a mix of boilerplate and nonsense.
| Kiro wrote:
| That's not my experience at all. Copilot consistently creates
| implementations that are very specific to my app and manages
| to understand the context and problem surface spanning many
| files. It's not just getting a standard problem and pulls an
| answer from Stack Overflow.
| muskmusk wrote:
| >The "it's been done before" one is pretty relevant.
|
| But is it? 99.999% of software development has been done
| before. Even if you do something that is legitimately new
| (like creating a chatbot that can generate code on demand).
| Then your solution will still contain more than 99% code that
| is just a repeat of things that have already been done.
| shynrou wrote:
| Exactly. When the first news came out about it's ability to
| "understand" code, find bugs and improve uppon it, I tested
| it with some snippets of mine. It just gave boilerplate best
| practices you find on 100 of blogs, but was not able to make
| meaningful contribution. It claimed to have introduced a
| feature while only having found another way to write the same
| snippet. On other things it straight up invented variables &
| functions that didn't exist.
|
| As long as the task is in it's training set, it can give you
| a decent answer, but it can't code it just mimics doing so...
| xxs wrote:
| There are tons of examples from the training set that are
| awful. Most people will just eat them.
| TrapLord_Rhodo wrote:
| ChatGPT would be so amazing as a pair programmer if it
| didn't invent functions.
|
| It's perfect for what is a python function for doing X. But
| it's honestly 50/50 whether that function even exists.
| olalonde wrote:
| HN went from "AGI is impossible" to "anything below AGI is
| worthless" real quick.
| postalrat wrote:
| The only thing that needs to change for HN to care is for the
| AI to start writing this stuff in Rust.
| ahsood wrote:
| The second point on your reactions "It's been done before" is
| very crucial.
|
| That defeats the point of your argument that "AI generates a
| game. It comes up with rules for the game".
|
| No, it doesn't. It plagiraised the game and pretended to come
| up with it. It just used a random puzzle game that it had on
| its training set.
|
| It's like asking it to write a poem and getting the same exact
| poem from a random google search. It didn't come up with it. It
| just copied it. It's not as amazing as you say it is.
|
| Also if you look I comments you can see that it's not even just
| one game. There are several games that are exactly like that.
| Which means more probability of having it in a training set.
| Kiro wrote:
| > It's like asking it to write a poem and getting the same
| exact poem from a random google search.
|
| No, it's not. A better comparison would be a poem that feels
| the same as an existing one and using the same prose but with
| its own words. Or any musical plagiarism dispute where the
| song is clearly different but similar enough that it needs to
| be decided by court. ChatGPT is not just copypasting a puzzle
| game here.
| bottled_poe wrote:
| It remains to be seen whether the lawyers agree with you.
| Kiro wrote:
| I'm not saying it's not plagiarism, just that it's not a
| copypaste.
| codebym wrote:
| It is fairly typical of HN to err on the side of cynicism.
|
| Correct me if I'm wrong, but ChatGPT is a very fancy auto-
| complete function. It's has no ability to create from scratch,
| just the ability to recompile and recontextualise any of the
| many existing pieces it has in its library.
|
| It's unlikey that this game or its rules are truely original,
| ChatGPT will have just plucked it from the library, perhaps
| given it a new name.
| postalrat wrote:
| Can you come up with a question to ask it that would prove or
| disprove your theory?
| hgsgm wrote:
| What is "scratch"?
|
| "Great artists steal".
|
| Art is defined by remixing the life experience of the author.
|
| ChatGPT's ability to create art is only limited by it's input
| (text corpus) while humans have images, sound, smell, touch,
| etc.
| beepbooptheory wrote:
| Each day I walk through my war torn and impoverished little
| village, looking for food and water. I keep in my pocket a
| small postcard of a beautiful tropical beach I've never
| been to. When I get home, I use the bits of supplies I've
| found to make my little paintings or write my little
| stories.
|
| The GPT version of me can only remix the world I am already
| in, so this version mostly paints dark landscapes and
| violent imagery, however much I prompt myself to draw
| something pretty, it always comes out looking a little
| macabre.
|
| The regular old person me, on the other side, is plagued by
| the human afflictions of desire and fantasy. This version
| of me can only paint the beach on my postcard, but because
| of my desperation, focus, and need, these paintings become
| larger and more fantastical than I thought I could imagine,
| but in that, they provide me and my loved ones comfort and
| escape and novelty.
|
| The human version of me makes statistically improbable
| things, but, to me, is still a plausible human, and the one
| I'd rather be at least.
|
| All just to say, maybe there is a more qualitative
| difference here than you think.
| overspeed wrote:
| People draw an arbitrary line of difference in the way we
| treat AI programs and human outputs.
|
| A human imagining orcs and one horned horses has
| 'fantastical, larger than life' imagination but AI
| generation drawing people with strange hands is
| 'incorrect'.
|
| These are not one to one examples but the point stands
| that with enough suspension of belief, people are more
| likely to take on human creations at face value than AI
| when they know the source.
| beepbooptheory wrote:
| Sure I think I agree. My point is just you can imagine
| the post-apocalyptic artist with a human brain painting
| the beach or just painting the dark landscapes; but we
| can only imagine the GPT brain painting the dark
| landscapes, in so far as that is majority of its day to
| day dataset, thus the statistically likely output.
|
| This suggests a qualitative difference between the two
| when it comes to "creating" or "generating" that feels
| far from trivial, even if you want to say the AI can make
| "good" things, whatever that means to you.
| sinuhe69 wrote:
| Yes, in some sense it's. But if the game already exists and the
| AI just parroted it, it's significantly less impressive, isn't
| it?
| wildmanx wrote:
| > An AI generates a game.
|
| But it hasn't! This is just another step in the BS storm coming
| out of the latest AI hype. The language model has reproduced
| something that has existed before and was likely part of its
| training data. That's cool, but it's far from what's being
| claimed here.
|
| We really need to get better at fact checking this stuff. And
| with "this stuff" I mean the output of LLMs and other AI
| frameworks as well as the claims about it. And with "we" I mean
| society as a whole and our industry in particular. Let's keep
| the hype in the drawer. The general population can be hyped up
| about sth, but we should know better, so instead of joining the
| hype, let's keep a cool head and educate people about what this
| is and what it isn't.
| pharmakom wrote:
| yeah its more akin to using Google to find a game on github
| and copy pasting it.
| isaacremuant wrote:
| I don't see how your comment contributes to the discussion. It
| seems aimed at shutting it down only allowing praise.
|
| The scope of the creation and whether it actually produced
| something novel is quite important to the discussion and part
| of the claim (although the author is very open to be proven
| wrong, in the article).
|
| Your claim n the second to last paragraph is false. That's
| relevant. This is HN.
| popeshoe wrote:
| It's still incredible you can coerce a language model to produce
| this, but it's not a new game, I've had it on my phone for a few
| years:
| https://play.google.com/store/apps/details?id=com.rohitpailw...
| latexr wrote:
| A while ago another poster thought ChatGPT invented good
| jokes.[1] All of them were ripoffs, which took less effort to
| verify than it takes to make a new post.
|
| I get people are excited about a chatbot which doesn't suck,
| but ideally it wouldn't turn off critical thinking skills.
|
| [1]: https://news.ycombinator.com/item?id=34744921
| ducktective wrote:
| Nice find!
|
| Seems to be similar to a game called Kakuro. This [1] repo even
| contains a similar rule:
|
| > The algorithm exceed the rules that the sum over a row must
| equal to the value on the left and the sum over a column must
| be equal to the value on the bottom of the cells with the
| diagonal and one or two numbers
|
| [1]: https://github.com/MarioBonse/KakuroSolverCSP
|
| [2]: https://github.com/topics/kakuro
| hgsgm wrote:
| That's the _first_ thing ChatGPT said at the start of the
| whole process. It created a game like Kakuro.
| samwillis wrote:
| And herein lies the issue with ChatGPT, it can generate
| functioning code, but can also lie through its none existent
| teeth about it. Using ChatGPT (or Co-Pilot) can feel like pair-
| programming with a very talented developer who loves to
| bullshit.
| andrepd wrote:
| "Very talented developer"? Sorry, I don't think googling my
| prompt and replying with the top stackoverflow answer (or a
| mashup of the top answers) counts as a talented developer.
|
| Anecdotal, but I've not yet had any success in producing any
| non-trivial code with ChatGPT. It has, however, produced
| copious amounts of bullshit with plausible variable names...
| :)
| olalonde wrote:
| > can also lie through its none existent teeth about it
|
| Ironically, it seems to me that you are anthropomorphizing
| ChatGPT a bit too much here. It has no reason to lie so I
| think it's more likely that it just doesn't know such game
| exists. It probably came up with it independently or doesn't
| have a strong memory of it. In some respect, it would be even
| more impressive if it was actually "lying through its teeth"
| because it would imply the AI had some kind of hidden agenda.
| furyofantares wrote:
| Similarly I don't think it makes sense to say it "knows"
| anything at all. I would be more comfortable saying Wolfram
| Alpha knows things than saying an LLM does, but I'm not
| comfortable with either.
|
| I'm not sure I'm comfortable with "remembers" either. My
| gut says I want to say I'd be more comfortable with that
| word for a web cache, but due to my understanding of human
| memory as constructive maybe I should be more comfortable
| with that for an LLM than for any other software.
|
| Happy to be convinced otherwise.
| rhtgrg wrote:
| ChatGPT does indeed know nothing at all. Proving this is
| quite easy, it was trained on text generation and can
| generate paragraphs quite well, so if you ask it to tell
| you about Harry Potter's family tree it will do well.
|
| However, it will fail immediately when you ask it to
| print an ASCII chart of Harry Potter's family tree,
| because it does not actually "know" anything, and it will
| make all sorts of odd connections.
|
| The clearest observation I can make of ChatGPT's success
| is that the general public is quite ill-informed and
| easily impressed by theatrics, both lessons we've already
| learned from politics.
| [deleted]
| neilv wrote:
| ...and plagiarize like crazy, while lying about it. :)
| sebzim4500 wrote:
| Is this actually in the training data though? I couldn't
| find a textual description of the rules though Google, so
| I'm not convinced.
| sekai wrote:
| Is it plagiarizing if ChatGPT knows about chess rules,
| analyzes every single chess strategy, then create a similar
| game?
| yowzadave wrote:
| ChatGPT does not know chess rules, nor can it "analyze" a
| chess strategy. ChatGPT has digested the conversations of
| many many people who have talked about chess, and can
| reproduce a transcript that sounds similar to this corpus
| of conversations.
|
| But it is not synthesizing an understanding of the game
| of chess.
| skeaker wrote:
| Except that there is demonstrable evidence that indicates
| GPT actually does have some level of understanding via
| internalized world models (of Othello in this case, not
| chess, but the idea is the same):
| https://thegradient.pub/othello/
| dbrgn wrote:
| Could it be plausible that ChatGTP processed some text
| describing this exact game, where it is claimed that the
| idea is novel (because it was, at that time)? Since ChatGPT
| does not understand the concept of novelty, it would simply
| "learn" that the rules are novel, and then repeat the
| rules, still claiming that they're novel. After all, that's
| the information it was trained with :)
| ChatGTP wrote:
| ...maybe this is the key to success and ChatGPT is here to
| show us the way! :)
|
| Ride those coattails and take other peoples ideas as your
| own!
| jstummbillig wrote:
| I am confused at to how this would be "the issue" with
| ChatGPT. Being wrong and not being aware of it is not a
| unique concept. At least with ChatGPT it is fair to assume
| there is no hidden agenda and no need to worry about ill
| will. If anything that makes it less of an issue, compared to
| humans.
| satisfice wrote:
| No hidden agenda? It has an agenda and it is not honest
| about it. That's a hidden agenda. You don't know what is
| "motivating" ChatGPT. Neither does ChatGPT. But it has been
| given motivation. It has been designed to write in a
| certain way. Its design prevents it from learning or
| honestly engaging in serious discussions. It's not any sort
| of unbiased equation.
|
| More dangerous than ChatGPT is the sheer gullibility of
| many people putting it to use.
| malaya_zemlya wrote:
| Its agenda is predicting the next input token.
| mcphage wrote:
| Yes, and that agenda has severe consequences, like
| "confabulates constantly". Just because it's simple to
| state doesn't mean the consequences are simple or
| innocent.
| hgsgm wrote:
| When a meteor strikes, and causes a mass extinction, is
| it "guilty" or just "bad"?
| mcphage wrote:
| How many stockholders does a meteor have?
| jasfi wrote:
| It looks like it sometimes, even though that may not be the
| case. I've had times when I've corrected ChatGPT, and yes,
| it knows that what it told me was wrong. It then goes on to
| tell me more along the lines of what it seemingly already
| knew what was right.
|
| This obviously isn't the intention of the software, it's
| just an LLM after all, but there's something missing in the
| experience when it comes to working with code. Hopefully
| this sort of issue can be corrected.
| ldhough wrote:
| I wonder if this could partially be a result of training
| on code found in question/answer environments like Stack
| Overflow. It sees "How do I do X, here's what I've tried"
| with broken code and then an answer "This is incorrect
| because Y, here is the correct answer" with the correct
| code.
|
| Intuitively it makes sense to me that broken code would
| often be very close to questions about how to achieve
| something in code.
| ssss11 wrote:
| "An" issue with it is that we may come to rely on these
| AI's outputs as assumed correctness or truth. If we have to
| double check everything they produce then that's not great
| either.
| latexr wrote:
| > At least with ChatGPT it is fair to assume there is no
| hidden agenda and no need to worry about ill will.
|
| Is it? Even if it's fair to assume that _now_ , we have no
| idea if that will remain true or when the shift will
| happen.
|
| The CEO of OpenAI is the same scammer who scanned eyeballs
| in return for a non-existing cryptocurrency[1] and the
| company itself is criticised all the time[2].
|
| [1]: https://www.buzzfeednews.com/article/richardnieva/worl
| dcoin-...
|
| [2]: https://techcrunch.com/2023/03/01/addressing-
| criticism-opena...
| skeaker wrote:
| Yes, it is fair to assume that and in cases like these it
| will continue to be for the perceivable future. The AI
| does not stand to gain anything by about lying about a
| simple puzzle game, and neither does the CEO. Even if the
| CEO somehow did, it would be a disproportionately
| colossal amount of effort to tamper with ChatGPT in this
| specific instance. And that's also assuming that the CEO
| himself has all the knowledge and tools needed to do all
| of it himself, which I doubt.
| mattlutze wrote:
| There is an issue with how people are personifying ChatGPT
| and assigning it agency.
|
| Some want to talk of these LLMs as approximating an
| intelligent actor. If that's the case, then we also need to
| assign metaphors for things like deceit and coercion. We
| also need to consider assignments of novelty to what's
| generated and think of their rights as quasi-sentient, etc.
|
| Some want to talk about them as probabilistic text token
| generators, which brings the benefit of not being
| intelligent or independent actors at all really but also
| then comes with the issue of intellectual property theft in
| training them on information not licenced for reproduction
| or commercial use.
|
| The industry prefers to thread the needle between these as
| the former case brings some pretty wild conversations and
| the latter may mean lawsuits.
| samwillis wrote:
| Ok, so maybe not the issue with ChatGPT, but with peoples
| understanding of its limitations. It can generate text and
| code from instructions, but it's limited in its logical
| analysis of what it's "saying". In this case it was asked:
|
| > And to the best of you knowledge this type of puzzle does
| not currently exist?
|
| and it responded:
|
| > As far as I am aware, this specific type of puzzle with
| the given rules and mechanics does not currently exist in
| the puzzle game genre. However, there may be similar games
| out there that share some similarities with this puzzle.
|
| That response is not generated (as far as I am aware) by
| any form of logical analysis or understanding, it's just
| generated text based on its training and prompting. It was
| asked to come up with something "new", and will continue to
| claim that as it was part of its prompts.
|
| So yes, this may not be a failing of ChatGPT, but of users
| understanding of it. You cannot take what it states as
| "fact" as anything other than potential BS. But it is an
| incredible tool for using to generate text and code.
|
| We are still early in its development though, who knows
| where it will be in 18 months time!
| sebzim4500 wrote:
| Is it actually wrong though? Will the rules of 'summer'
| be in it's training data anywhere? AFAICT they aren't
| described on the google play page, although you can
| easily figure them out by the screenshots.
| vsareto wrote:
| I feel like you can compensate with more complicated
| prompts. Or even different prompt categories (like
| negative prompts, but for programming it might be a list
| of constraints). Like this interface:
| https://github.com/AUTOMATIC1111/stable-diffusion-webui
| but for code
| kqr wrote:
| This is a very good comment. ChatGPT uses language so
| fluidly it's easy to interpret as there being more
| substance than there is.
|
| Looking at the response the way you suggest, it's clear
| it's given a boilerplate answer that would seem likely
| given the context it has found itself in.
| prox wrote:
| Exactly, as soon as were to butcher my english, a certain
| amount of credibility will be incurred, even if the
| communicators are aware of it. It could probably
| eloquently explain the workings of a Retro Encabulator
| fluidly and you'd nod a few times and thinking it's fine.
|
| If words not right said for listen like now, think you
| might not be smart as is tho.
| Jensson wrote:
| ChatGPT will reverse that, if you sound smart it is
| likely nonsense generated by an AI.
| chongli wrote:
| With humans we can demand that people cite their sources.
| If they fail to do this, they run the risk of being accused
| of plagiarism. ChatGPT, on the other hand, plagiarizes all
| day long and never cites sources. That is why it's an
| issue.
|
| And as for whether ChatGPT has an agenda or not, that is
| beside the point. People can and do use it as a tool for
| plagiarism while trying to hide behind a layer of plausible
| deniability provided by the "black box" of the model. This
| cannot be allowed to continue. This is why we need to push
| back, just as the GP is doing.
| mynameisvlad wrote:
| I mean, sure, you can demand it. And people are just
| going to make up sources. It's not like they have a gun
| held up to their head to ensure that demand is followed.
|
| > People can and do use it as a tool for plagiarism while
| trying to hide behind a layer of plausible deniability
| provided by the "black box" of the model. This cannot be
| allowed to continue. This is why we need to push back,
| just as the GP is doing.
|
| This is absolutely preposterous. People are going to lie
| and plagiarize whether they have a chat bot do it for
| them or not. The existence of a chat bot isn't going to
| be the make or break in this equation and if anything,
| the people using it for that purpose should be rightfully
| vilified rather than the tool.
| gattilorenz wrote:
| > People are going to lie and plagiarize whether they
| have a chat bot do it for them or not.
|
| The difference is, with a chatbot it might not even be a
| conscious act, the chatbot is doing it for you and you're
| not aware that it's happening.
| unyttigfjelltol wrote:
| > And people are just going to make up sources. It's not
| like they have a gun held up to their head to ensure that
| demand is followed.
|
| The consequences actually are quite serious. A person
| falsifies work product once in an academic or
| professional setting and their career is severely
| impacted. This is why people are "surprised" to encounter
| such a BS generator operating under the trademark of a
| reputable company.
| mynameisvlad wrote:
| It's not the tool that's at fault in that case, it's the
| person doing that falsification. The person would have
| faked their sources and made shit up without ChatGPT
| there.
|
| It's almost as if you ignored everything I said, cherry-
| picked a random part, then went on a tangent about a
| different part of my comment. All without actually
| comprehending what the things you replied to said.
| jtxt wrote:
| We can help it look for and use sources.
|
| I've had it generate search terms that could be used to
| verify "facts" in is answer. Then I'd give it the page
| results and have it adjust and source it's answer using
| that.
|
| Have not tried it yet, but perhaps Bing's implementation
| is a step in that direction?
| gattilorenz wrote:
| One of "the issues" is that you are led to believe that
| since there is no agenda and this is AI, its result must be
| true and you don't need to double-check whether they are.
| And of course, since it did invent a name for the game (or
| a new function name, or [insert your example here]), it's
| even harder to google to cross-check if it's actually new
| or if it's essentially telling you bullshit or inciting to
| plagiarism.
| IIAOPSW wrote:
| Bullshit is far more insidious than a lie, for a lie is
| wrong and will come to light, but bullshit is uncorrelated
| with truth and may even be coincident with it. Thus
| bullshit can go unnoticed far longer.
| Technotroll wrote:
| In this case I think I'd give ChatGPT the benefit of the
| doubt. It is possible to invent something that already
| exists, and it has happened on several occasions trough-out
| history. A great example is the history on who was really
| first at inventing the telephone. In the end Alexander Graham
| Bell got the patent, but perhaps Elisha Gray was actually
| first? Historians remain divided on the topic.
|
| For instance, I once found what I thought was an ingeniously
| original idea about about how TV is really just a kind of
| reflection of reality akin to Plato's Cave. I immediately got
| started writing a thesis about it, but I didn't have to
| search for long on the topic before I found an entire book
| written on this way of thinking about television. I wasn't
| really disappointed, because in the back of my head I knew
| that it had too be too good to be true that I'd be first with
| such a great idea. In any case I kept working with the
| thesis, and I still did got a good grade on it despite the
| idea not being revolutionary.
|
| The questions I now wonder about is, can ChatGPT forget? Or
| could it be that ChatGPT was never exposed to this game, but
| could still infer it through other game rules, such as those
| for Soduko? Which I guess opens up another rabbit hole on if
| or how AI can be creative. Which I guess opens up another
| rabbit hole on how creativity works in general.
| Jensson wrote:
| > Or could it be that ChatGPT was never exposed to this
| game, but could still infer it through other game rules,
| such as those for Soduko?
|
| There is no way, the game type is centuries old, you can
| read this giant wikipedia articles about games like this.
|
| https://en.wikipedia.org/wiki/Magic_square
|
| ChatGPT "inventing" this is like thinking it invented
| chess.
| Felminor wrote:
| Chatgpt should be able to cluster things and see were
| clusters could be, collect everything necessary for that
| theoretically cluster and the human could evaluate it.
| mynameisvlad wrote:
| This is not a magic square, though. All rows and columns
| _explicitly do not_ add to the same number.
| Jensson wrote:
| Yes, but the non magic square is inspired by the magic
| square and such games are everywhere. Just buy a random
| puzzle book and you find pages and pages of puzzles with
| "make the numbers add up to these columns and rows",
| because they are very easy to make.
|
| Point about magic square is that every culture invents
| games like that, it is one of the most basic puzzle ideas
| humans have, I don't see how ChatGPT can't have that in
| its training set.
| mynameisvlad wrote:
| For someone trying to show that a chat bot could not
| possibly have generated this specific game on its own
| because it already exists, you _kind of have to show that
| it already exists_.
|
| All that you've done is shown that similar types of
| puzzles exist. Which, I mean, is kind of the point of a
| generative AI.
|
| "Games like this" exist. Does _this specific game_ exist?
| mhitza wrote:
| One game I can think of being very similar, is a game
| within a game. Dungeons & Diagrams puzzle within Last
| Call BBS [0]. In that game you can place or remove walls
| for them to add up to the numbers shown per row/column.
| That game has another layer of strategy built on top, as
| there are certain "dungeon patterns" you could observe
| that would in theory guide you through completion. I
| myself haven't noticed any patterns when I've tried the
| game the first time, and just relied on the numbers
| shown. (Guess that's why I've only played 3-4 levels)
|
| [0] https://steamuserimages-a.akamaihd.net/ugc/1858314357
| 3725211...
| maxpro wrote:
| https://play.google.com/store/apps/details?id=com.rohitpa
| ilw...
|
| this comment thread started with this link, this is
| exactly the same game
| Jensson wrote:
| Much more advanced versions of it exists, and has existed
| for a long time, for example Kakuro is before computer
| games. Magic sum is just a special case of it. Finding a
| discussion with those exact rules are probably a bit
| hard, search engines aren't good at searching for that,
| but given how common these games are and how many game
| design discussions and ideas there are online, a game
| where "block out these numbers to make these sums" is
| surely to exist somewhere. The poster above even found
| the exact same game, although that wasn't described in
| text, but someone probably described it in text
| somewhere.
|
| https://en.wikipedia.org/wiki/Kakuro
| mynameisvlad wrote:
| Once again, to show that's one thing is blatantly copying
| another thing, you _kind of have to show that thing
| exists already_. Kakuro is also a similar game with its
| own unique rules that only somewhat overlap with this
| one.
|
| It's not enough to say "a lot of games with similar rules
| exist" and if anything, that just shows that a generative
| AI is good at what it does: break down the rules of a
| game and make modifications to make what is potentially a
| new game.
|
| If you can show an example of this exact game having
| existed for centuries, then you have a point. But showing
| that magic squares and similar games exist... just shows
| that magic squares and similar games exist, not that the
| algorithm incorrectly said this is a new game.
| Jensson wrote:
| The discussion was probability of ChatGPT having invented
| it, the probability that description for such a game is
| in ChatGPT's dataset is extremely high. We have examples
| of that exact game existing (the top post of this
| thread), and we know from my links that there are
| countless texts about puzzles like this out there,
| although they aren't exactly the same.
|
| > It's not enough to say "a lot of games with similar
| rules exist" and if anything, that just shows that a
| generative AI is good at what it does: break down the
| rules of a game and make modifications to make what is
| potentially a new game.
|
| No it doesn't, even if that is the case it just shows
| that it adds random variations. Since we only see the
| trimmed subset of ideas it generates that people found
| good enough to post, the smart one is the person.
|
| You would need to prove that ChatGPT actually
| consistently generates working puzzle ideas that are
| novel to convince anyone that it actually does so.
| Extraordinary claims require extraordinary evidence, so
| all I need to do is find plausible explanations to how
| ChatGPT found it, you would need much better evidence to
| convince people it actually did make a novel game.
| mynameisvlad wrote:
| > The discussion was probability of ChatGPT having
| invented it, the probability that description for such a
| game is in ChatGPT's dataset is extremely high.
|
| If this were the case, it would have been trivial for you
| to find a game with its written rules described and which
| match the one generated.
|
| You have done nothing but say that is the case. You
| haven't actually _proven_ that's the case.
|
| ChatGPT can't magically infer the rules of the game from
| screenshots, and you have only shown that similar games
| exist and have existed for centuries. But that is not the
| same as saying that this specific game has and that
| ChatGPT just pulled it out of its dataset.
|
| _That_ is the extraordinary claim that you don't have
| evidence for but are acting like it's right there
| obviously out in the open for everyone to see.
| Jensson wrote:
| > If this were the case, it would have been trivial for
| you to find a game with its written rules described and
| which match the one generated.
|
| Search engines doesn't work like that. You are basically
| asking me the equivalent of proving that a photo isn't
| depicting a ghost. No, I can't prove that, I can however
| come up with examples showing how the photo could have
| been created even if it wasn't a ghost.
|
| If you want to prove that ghosts are real you need plenty
| of photos from lots of angles and situations, or videos,
| and from many sources to show that it isn't all made up
| by a single person. The equivalent of that would be if
| they had made ChatGPT generate 100 different working
| games for example, that would be much more believable.
| But a single case of a game that already exists and has
| countless texts describing similar games? It just looks
| like random chance that got handpicked or plagiarism.
|
| This isn't a court trial, I am not going to sue ChatGPT
| for plagiarism here, it is just a discussion whether it
| is reasonable to believe ChatGPT can generate novel
| puzzle games.
|
| Edit: But do note that since ChatGPT can find such ideas
| that are hard to find with a search engine, that makes
| ChatGPT very useful in a way search engines aren't. So I
| am not saying it doesn't add value. Just that people seem
| to say ChatGPT does a lot of thing that it doesn't seem
| to be able to do.
|
| Edit again:
|
| > That is the extraordinary claim that you don't have
| evidence for but are acting like it's right there
| obviously out in the open for everyone to see.
|
| Yes, you think it is obvious that ChatGPT is capable of
| very creative and productive thinking. But most people
| don't think that, to them that is an extraordinary claim.
| I'm not here to convince you, I'm here to explain to you
| why you aren't convincing anyone with what you say.
| People like you were convinced by articles like this
| before the discussion even began.
| mynameisvlad wrote:
| > Search engines doesn't work like that. You are
| basically asking me the equivalent of proving that a
| photo isn't depicting a ghost. No, I can't prove that, I
| can however come up with examples showing how the photo
| could have been created even if it wasn't a ghost.
|
| The claim was that it pulled the game out of its dataset.
| If this were the case, I would argue it would absolutely
| be trivial to find them. It's not some concept that can't
| be described in words or would be hard to quantify. The
| rules have been provided, and, assuming they were
| plagiarized from somewhere else, would be listed verbatim
| or close to it.
|
| If a student plagiarized on their work, whether in
| written form or in code, it's been trivially easy to find
| the exact work that was copied from. It generally takes
| me a few seconds of searching to find it.
|
| This is the same. If these rules existed in a dataset,
| then it should be equally easy to pull them up and prove
| the plagiarism. If all you can find is similar puzzles,
| you can't just throw your hands up and say "yep, gottem".
| That's just not how this works.
| Jensson wrote:
| > The claim was that it pulled the game out of its
| dataset. If this were the case, I would argue it would
| absolutely be trivial to find them. It's not some concept
| that can't be described in words or would be hard to
| quantify. The rules have been provided, and, assuming
| they were plagiarized from somewhere else, would be
| listed verbatim or close to it.
|
| ChatGPT uses word vectors, it wont use the same words but
| variants of the words. You can't search for that. Cases
| where word vectors only maps to single words with no
| variations for every word are very rare, so ChatGPT is
| very good at plagiarising things without reproducing
| exactly, it just rarely fails at it.
|
| > If a student plagiarized on their work, whether in
| written form or in code, it's been trivially easy to find
| the exact work that was copied from. It generally takes
| me a few seconds of searching to find it.
|
| No it isn't, they just change the words and rewrites it
| until it no longer looks the same. ChatGPT is trained to
| rewrite texts like that to avoid triggering trivial
| plagiarism detectors. They train it to produce the same
| text, but with different words, producing exactly the
| same text is punished.
| mynameisvlad wrote:
| > No it isn't, they just change the words and rewrites it
| until it no longer looks the same. ChatGPT is trained to
| rewrite texts like that to avoid triggering trivial
| plagiarism detectors. They train it to produce the same
| text, but with different words, producing exactly the
| same text is punished.
|
| Do you think students plagiarizing don't do the exact
| same thing? Clearly someone has never actually dealt with
| plagiarized work. This is plagiarizing 101. The structure
| remains the same even if they use synonyms. Considering
| it's trivially easy to find _in code_ which is magnitudes
| harder to pull off, I would still argue it should be easy
| as pie to find this supposed set of rules.
|
| Your point is not very credible without proof of this
| game existing and ChatGPT pulling it from this source.
| Without showing this supposed proto-game having existed
| with rules the ChatGPT can pull from, then all you've
| done is wave your hands around and yelled "similar games
| exist so this can't possibly be uniquely generated" and
| that's not a very compelling argument.
| Jensson wrote:
| > Do you think students plagiarizing don't do the exact
| same thing? Clearly someone has never actually dealt with
| plagiarized work. This is plagiarizing 101. The structure
| remains the same even if they use synonyms.
|
| You rewrite the structure of the text, you don't just use
| synonyms. ChatGPT is capable of rewriting text to a
| different structure while keeping the meaning, I hope you
| are aware of that.
|
| Anyway, even if you just change the words to synonyms it
| wont be easy to find in a search engine. Search engines
| aren't very good at finding matches to synonyms. Google
| tries, but in doing so they fail to find more specific
| texts like scientific publications or documentation, so
| no search engines aren't good at finding plagiarism.
|
| Edit: And you make it sound like most plagiarism is
| found. No, that isn't the case, most plagiarism is not
| found out because it is a very hard problem to solve.
| Only the most blatant cases are caught. For humans that
| is reasonable, for AI we can be stricter since there
| isn't a humans career at stake.
| mynameisvlad wrote:
| > Anyway, even if you just change the words to synonyms
| it wont be easy to find in a search engine.
|
| Got it, so you've never actually dealt with plagiarized
| work. You should have just led with that.
|
| I have literally said, from actual experience, that this
| is the case. But I guess discarding that and pretending
| it was never said and that the opposite is true is I'm
| sure an easier position to hold.
| Jensson wrote:
| Do you believe you never missed any plagiarised work
| examples? You caught some people doing X, and then you
| declare that catching people who do X is trivial. But
| plenty of people get away with doing X so we know that it
| isn't easy to catch.
|
| For students they are probably easier to catch since they
| use the same tools you do, they use a search engine to
| find an article and plagiarises that. But ChatGPT takes
| deep discussions from reddit or stack overflow, I can't
| find those with a search engine.
| mynameisvlad wrote:
| If it's as blatant as copying the entire game, you'd
| think it would be easier for you to find the game it
| copied. By your own account, this is an example of an
| obvious case of plagiarism. You were dead set on it, 100%
| sure.
|
| Yet here we are. Dozen comments later and still no
| written set of rules produced which definitively shows
| that it was copied.
|
| Come back when you actually have that and maybe we can
| continue this conversation.
|
| > But ChatGPT takes deep discussions from reddit or stack
| overflow, I can't find those with a search engine.
|
| Where do you think the answers come from? It's not like
| Google has a massive index island around Reddit and SO.
| Jensson wrote:
| I tend to exaggerate my claims a bit, yes. But you
| exaggerate your claims as well, for example you claim
| that if it had copied the rules it would be easy to find
| an example, that isn't true at all. Many examples of
| plagiarism goes unnoticed for years, until someone who is
| familiar with the original work points it out. I know
| examples where the person was found out during his thesis
| defence, he had plagiarised his entire PhD work from
| papers in another language and nobody noticed until years
| later, not even all the peer reviewers of the papers.
|
| So maybe these rules are described in Japanese? Most
| similar games comes from Japan, Kakuro, Sudoku etc. Would
| your plagiarism detection method of Googling it find a
| Japanese source? I doubt it. But ChatGPT transcends
| language barriers, it can translate to English just fine.
| Technotroll wrote:
| Hm, well in that case I may well be wrong. Thanks for the
| info!
| zuppy wrote:
| from my understanding, anybody please correct me if i'm
| wrong, ChatGPT can not really invent anything, it can
| just generate text based on probabilities obtained from
| the mountain of source documents used for training it. it
| does not think in the same way we do, it is just amazing
| at writing coherent phrases (and very simple code).
|
| there's a quite long article from Stephen Wolfram about
| how it works and this is why I belive it can't do that:
| https://writings.stephenwolfram.com/2023/02/what-is-
| chatgpt-...
| idiotsecant wrote:
| What does it mean to say that it can't invent anything?
| If, for example, I ask it to make a new poem with no line
| previously recorded in the english language it will do
| so. If I google that poem to test it's originality I
| won't find a match. It seems to me it just made something
| novel, right?
| panarky wrote:
| When humans write new literature, or design new games,
| are we simply remixing elements of language and game
| mechanics that we've seen before, or is there something
| more going on?
| BoiledCabbage wrote:
| Who else's experiences can we pull from but our own? It
| can be anything else.
| BurningFrog wrote:
| You may be splitting the wrong hair here.
|
| However it generates a text, that text may describe what
| for practical purposes is a new invention.
| _nalply wrote:
| Re forgetting: we should be careful not to anthropomorphize
| ChatGPT.
|
| In principle, ChatGPT cannot forget. It is trained on data
| and this training will stay as long as it didn't get
| deleted or destroyed. In other words, in all cases of
| someone having made ChatGPT tell something, it should be
| possible to repeat this. Perhaps in some case it will be
| effectively impossible for some rare combination of prompt
| and random seed, so one could say, ChatGPT forgot
| something. But this is not the same as people forgetting
| something.
|
| Or during the training something was not considered
| important, but this is not forgetting, this is ignoring.
| juujian wrote:
| The funny thing is that it is neither lying, nor inventing
| something new. What OpenAI did pretty well was collect
| data. And wouldn't you know it, the folks who developed
| that new puzzle describe it as what it is---a new kind of
| puzzle. So now in the training data you have a combination
| of puzzle, sudoku, and new/novel. And wouldn't you know it,
| by asking for a new puzzle, based on sudoku, you make
| ChatGPT dig for that kind of text. If ChatGPT really had a
| novel idea, I would not expect it to be this coherent---
| after all, logic and coherence are not a constrain on how
| language models work, just what words are likely to occur
| next. That is why it is being compared to entry level
| college writing, because that is how an excited student
| writes who hops from topic to topic.
| djmips wrote:
| But how is it different from humans? I can't tell you how
| many times now that I've come up with what I thought was
| a really cool idea but upon web searching found it was
| already invented/discovered etc. In fact before the
| Internet I had come up with my own algorithms and only
| upon the Internet existing did I find they were already
| discovered years earlier. There's no way that I was
| regurgitating something I had read in that case.
| juujian wrote:
| It is different to what you do. If I tell you that this
| is already a thing, you might go back to the drawing
| board, and do something from scratch. Maybe do some
| abstract drawing with numbers for brainstorming. A
| language model is not able to do this, the starting point
| for a language model is always the training data. That is
| why there is so many instances where you see some wrong
| (or correct) response from ChatGPT and when the other
| person corrects this, the model just agrees to whatever
| the user says. That is the right thing to do according to
| language etiquette, but it has nothing to do with what is
| true and right. (It invokes the image of a sociopath
| manager trying to sell you a product---they will find a
| way to agree with you to close the deal.)
|
| I don't know what introspective is, but I know it when I
| see it. People around me genuinely come up with new
| concepts---some of what they came up decades ago with is
| now ubiquitous---and the sources is often not language.
| It comes from observing the world with your eyes, from
| physical or natural mechanisms. If you want to put it
| into the language of models: we just have so much more
| data to draw on. And we have a good feedback mechanism.
| If you invent a toy, you can build it and test it.
| Language models only get second hand feedback from users.
| They cannot prototype stuff if the data isn't out there
| already.
| jacobr1 wrote:
| One implication though, is that LLMs can currently come
| up with novel mixes of existing ideas. It might be a good
| blender, integrating different pieces into a new whole.
| travisjungroth wrote:
| There's a difference between coming up with a puzzle then
| finding out it already exists versus finding a puzzle and
| saying you came up with it.
|
| If I told you "We need a brand new, never-never-before-
| seen puzzle for our next game release." and you searched
| Google for "brand new, never-before-seen puzzle", found a
| puzzle game with those words in its marketing copy and
| pitched it to me, that would be some combination of
| unintelligent and dishonest behavior. Like, surprisingly
| so. It's different from forgetting some puzzle you played
| with as a little kid and thinking you made it up, or
| creating a puzzle you'd never seen but has been made
| before.
| not2b wrote:
| But ChatGPT is not a person, it is a text generator. By
| asking it to generate a new puzzle, you are prompting it
| to find text in its training data showing someone
| describing a new puzzle, and it is going to speak in
| their voice. It's going to emit sentences that were
| influenced by what the puzzle developer originally wrote,
| and that person correctly said that it was new.
| urbandw311er wrote:
| This. people who can ground themselves in what ChatGPT is
| (an auto completion text predictor) are able to best
| understand the origins of its output.
| [deleted]
| visarga wrote:
| It is a dilettante, it has not reached the level of
| "talented" in anything. It knows many things about many
| things and nothing in depth. Test it on your specialisation,
| you will see it make absurd mistakes and hallucinations. Try
| it on a domain you know less about - it looks perfect.
| jokethrowaway wrote:
| It depends on the language. I assume they trained it more
| on the most popular ones
|
| It's pretty good in JS, it sucks in Rust
| visarga wrote:
| Yes, just a couple of days ago I asked it to write a
| Pytorch Lighning module for me, it looked great at first
| sight. But it mixed up the dimensions and made other hard
| to see bugs. It was frustrating to fix, almost the same
| effort with writing the damn thing manually.
| kilgnad wrote:
| Bro in this case the human is generating made up garbage.
|
| That game is NOT the same game. It's similar but the games
| are different.
| de6u99er wrote:
| Great find. I would be amazed if a language model like ChatGPT
| could come up with a novel idea.
| omnicognate wrote:
| Google Trends shows a small number of searches for "sumplete"
| going back to 2004 [1]. Not sure how to find what the results
| might have been, though.
|
| [1] https://trends.google.com/trends/explore?q=Sumplete (search
| "Worldwide" and extend the time range)
| kilgnad wrote:
| Except this game is different. It's similar but different.
|
| Your game involves addition. chatGPT is using subtraction.
| pmontra wrote:
| "If you can think about it somebody already did it and it's on
| the internet."
|
| Loose quote from I don't remember who, early 90s.
| ChewFarceSkunk wrote:
| [dead]
| SilasX wrote:
| If anyone is interested in talking about the puzzle app itself, I
| think I found a generated puzzle with two valid solutions (posted
| on the author's substack -- not sure if it's the same as the
| submitter). Unfortunately, it doesn't let you step back or even
| restart once you've solved.
|
| Here's the screenshot -- of the four remaining sixes, either of
| the two diagonally-opposite pairs should work (top-left and
| bottom-right or top-right and bottom-left):
|
| https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi...
| sebasv_ wrote:
| Very cool! Although I would love to be proven wrong, I am still
| suspicious that this is actually a code sample from an existing
| game it picked up somewhere during training. Seeing how chatgpt
| struggles with basic logic I would be surprised if it can
| actually generate a new and workable game of logic.
| Kiro wrote:
| The way I understand these models (and please correct me if I'm
| wrong) is that they predict every single word one at a time out
| of billions and billions of possible paths. So for the model to
| actually reproduce code you would either need to bait it really
| hard (reducing the possible paths by pushing it into a corner)
| or be impossibly "lucky". It can't just copy-paste something
| accidentally since it has no "awarenesses" of the source
| material the last neuron was created from.
| Jensson wrote:
| It is trivial to encode text in neural networks, these models
| try hard to avoid that by punishing it in training but they
| still encode a lot of text word for word. The most famous
| example is "Fast inverse square root", it gives you the exact
| same thing with same comments etc.
| Kiro wrote:
| When you say it's trivial to encode text in neural
| networks, what does that mean for LLMs? What makes it
| decide to encode certain texts or not? Isn't it just one
| big network of neurons?
|
| The prompt I've seen for it to verbatim reproduce the fast
| inverse square root from Quake was: //
| fast inverse square root float Q_
|
| When I ask ChatGPT to give me code for a fast inverse
| square root it doesn't reproduce it at all but gives me an
| implementation that looks completely different.
|
| So, my original thought was that the prompt above with the
| characteristic Quake III Q_ naming is enough to push it
| into a corner where the path is reduced to just one
| possibility (with that path being the words in the code
| itself) and not that it merely copypasted the code from an
| encoded version of it. I.e. it still predicts it word-by-
| word but with only one possible way for each step. This is
| just be my naive take on it though but I really want to
| understand.
| Jensson wrote:
| > my original thought was that the prompt above with the
| characteristic Quake III Q_ naming is enough to push it
| into a corner where the path is reduced to just one
| possibility
|
| This is what people mean when they say it copy pastes
| things. It doesn't literally go to the source code, press
| ctrl-c and then ctrl-v that to you, nobody believes it
| did.
|
| And the model does this a lot, as I said the reason it
| doesn't do that all the time is that they train it not
| to. And the quake code example got such a big deal that
| they started to hard code it to not return that, but that
| doesn't mean it never does that for other things, just
| that this particular example is now "fixed".
| Kiro wrote:
| Alright, that makes sense. I still think it's an
| important distinction and just looking at this thread it
| certainly seems that people think ChatGPT is just
| copypasting things. Like if it ctrl-c/ctrl-v all this
| from a single tutorial on how to make a puzzle game.
|
| The Quake code example still works in Copilot so I don't
| think they did anything about it, but only if I use the
| Q_ trick.
| ldhough wrote:
| I think they did something specific to this example to
| make it no longer generate the code verbatim. A few
| months ago when I tried it "fast inverse square root" was
| enough to get the exact function with comments and no
| explanation. A week ago that gave me the same function
| with no comments and
|
| "give me C code for calculating the fast inverse square
| root of a number, do not explain yourself"
|
| again resulted in the original function, comments
| included. Today it generates a completely different
| result and interestingly opts to explain the origin of
| the technique despite the directive for it to not explain
| itself.
| TMWNN wrote:
| Astounding!
| articsputnik wrote:
| amazing
| samwillis wrote:
| > Sumplete was created in a few hours by ChatGPT
|
| I think this is an interesting note at the end, it's clear that
| the whole "conversation" hasn't been posted, and not clear how
| many prompts were needed to finish it. A proficient developer
| would be able to develop this same app to the same level (once
| they have the idea) in a couple of hours without ChatGPT too.
|
| It would be super interesting to see a full screen recording of
| the process (or a similar one). What was the total word count of
| all prompts, how does that compare to the code size?
|
| Don't get me wrong, this is incredible and inspiring (I'm
| regularly using ChatGPT and Copilot). But I think the post
| doesn't critically analyse the process well enough.
| Lewton wrote:
| Yes, I think it's kinda rude (in a way) to cut out 99% of the
| ChatGPT conversation, because it gives people the impression
| that ChatGPT is even more magical than it actually is
| ChatGTP wrote:
| I thought the article recently about the guy who used
| Mindjourney (https://arstechnica.com/information-
| technology/2023/02/viral...) to generate a bunch of images
| which people on Instagram loved was fascinating.
|
| It was quite amazing how much actual manual work was going
| into some (not all) of those images in the end.
|
| A lot of processing in photoshop etc. I actually started to
| think that really, he was doing art, just not using a brush.
| davidkunz wrote:
| From my experience, most time is spent in the generate-copy-
| paste-compile-error cycle. Especially for strict/strongly-typed
| languages, checking for programming errors can be done by a
| machine, so this process can be automated (not for logical
| errors, of course).
| Towaway69 wrote:
| Isn't that what IDEs already do?
| raincole wrote:
| > A proficient developer would be able to develop this same app
| to the same level (once they have the idea) in a couple of
| hours without ChatGPT too.
|
| I mean... the whole point is that you don't need to be that
| proficient to use ChatGPT.
|
| If ChatGPT is much better than a proficient developer, you and
| I wouldn't be surfing HN, but on the street protesting for
| universal income.
| ozten wrote:
| New growth marketing technique!
| johlits wrote:
| I wonder which JS framework it will prefer when coding larger
| stuff.
| phist_mcgee wrote:
| One that causes the most flame wars on discussion boards.
| alexmolas wrote:
| Amazing game! Solving it sounds like a LeetCode exercise.
| xxs wrote:
| Solving it in general is quite trivial - solving it 'fast' is
| another story. I can write a backtracking algorithm or a
| constant propagation one on spot. Yet, I am not quite sure how
| efficient the latter would be, the former is obvious a brute
| force one.
| sebzim4500 wrote:
| It's almost certainly NP complete, so an asymptopically fast
| solution would be one hell of an achievement.
| Jensson wrote:
| It is subset sum, it is definitely NP complete.
| Synaesthesia wrote:
| This game was on Island of Dr Brain.
| codewiz wrote:
| Is this the game you're referring to? There are some
| similarities, but it's closer to Sudoku than Sumplete.
|
| https://youtu.be/gMnEBo0UICY?t=3495
| Synaesthesia wrote:
| It is the one, and you're right it's not the same game,
| although similar.
| askiiart wrote:
| I'm sure someone has already said this, but there's another HN
| post about it here: https://news.ycombinator.com/item?id=35010035
| DanSmooth wrote:
| Looks like a variant of Kakurasu [0] but instead of given numbers
| (1 to X) they use random numbers.
|
| [0] https://www.puzzle-kakurasu.com/
| m3kw9 wrote:
| It's cool it can do that but the game looks mundane as F
| ilitirit wrote:
| I don't understand why people don't think this is noteworthy.
|
| Would it be noteworthy if it this game (whether or not it existed
| before) were designed and implemented by someone who has very
| limited experience with software development in a few hours?
| Well, the fact that ChatGPT did this means _the former is now a
| very real possibility_.
|
| That you can go from conceptual inception of a framework to a
| fairly complete product in a few hours with very little
| experience is a big deal.
| mabbo wrote:
| I think the trouble is that ChatGPT's output made it sound as
| though it had _invented_ the game, which is noteworthy on its
| own.
|
| That it also coded it up itself is pretty amazing, but it's
| overshadowed by the false _more amazing_ claim that it invented
| it.
| cuttysnark wrote:
| > That it also coded it up itself is pretty amazing
|
| I haven't tried to replicate the author's journey, but in my
| experience it requires a considerable amount of hand-holding.
| e.g. Functions will be stubbed out, but contain no logic.
|
| I was able to guide it through a "playable" flappy bird, bit
| it took several revisions where I pointed out what was
| wrong/needed to be done before it truly returned a functional
| error-free prototype.
|
| It felt like pair-programming with a promising and apologetic
| junior dev.
| xxs wrote:
| > why people don't think this is noteworthy.
|
| Because the game has existed for centuries. ChatGPT claiming
| ownership is nothing new, either. The current top post even
| links an android app that does the same.
| hgsgm wrote:
| So what is the criticism here?
|
| Lichess and chess.com both exist. does the existence of one
| make the other worthless? unimpressive?
| schwartzworld wrote:
| If one of them claimed to have invented chess, then that
| claim would indeed be worth criticizing.
| sebzim4500 wrote:
| I haven't been able to find any reference to this puzzle
| other than the app on the play store linked elsewhere in this
| thread. ARe you sure that
|
| i) This exact puzzle predates the app ii) A description of
| the rules appears in the training set
| yieldcrv wrote:
| I made a game with chatgpt too
|
| I've been experimenting with the depth of "prompt engineering"
|
| I basically prompted it to procedurally generate rooms and fill
| them with NPCs with a common goal, on its own it persists the NPC
| names, genders, talking style, and the goal of the puzzle even
| though each region of the puzzle are procedurally generate
|
| I will absolutely slap an 8-bit GUI on top of that
| djha-skin wrote:
| > Sumplete was invented and coded by AI ChatGPT
|
| A common misconception. Legally, the game was invented by the OP
| through the use of prompt engineering. The op retains copyright
| for it. A machine cannot hold invention rights or copyright
| anymore than a camera can claim copyright for a photo "it" took
| with your help.
| hn92726819 wrote:
| No. The world does not revolve around the U.S. copyright
| system.
|
| OP might be able to claim copyright, but OP did not invent the
| rules or write the code.
| jordanpg wrote:
| These are hardly settled legal issues. This is uncharted
| territory. It will be years before appellate courts nail down
| these kinds of questions.
| pfedak wrote:
| I would have much preferred if the author spent more time giving
| an objective assessment of what ChatGPT had actually accomplished
| at each step.
|
| The "Labyrinth Sudoku" description feels like a classic language
| model speciousness. It doesn't actually work: you can't fit the
| digits 1-9 if you can't use all the cells, and it hasn't modified
| sudoku rules in a way that makes paths relevant. Maybe you can
| come up with a way to salvage it, but ChatGPT didn't.
|
| The initial rules for Sum Delete should be read with this in
| mind: it sounds reasonable, but there's no reason to trust that
| it can make a puzzle at all, let alone a good one. Also,
| unsurprisingly, the provided puzzle isn't solvable (the 25 column
| requires use of the 9 in the first row).
|
| Similarly, I'd love a critical analysis of the initial code. Did
| it guarantee solvability? An awful lot can be swept under
| "improving the design".
| [deleted]
| pimlottc wrote:
| I thought the first image was the actual game, I clicked on it to
| try to play :P It's always nice to make sure readers don't
| confuse a screenshot and an actual interactive widget. The worst
| is when it's a screenshot of a video player, with a play button
| and everything.
| hgsgm wrote:
| > No problem. I spent the next few hours iterating the game with
| ChatGPT - adding new features and improving the design.
|
| People are mad that headline over-exagerrates the impressive
| performance.
| aitchnyu wrote:
| I'm also impressed by the author, who didnt set a bar for the AI
| and call it a day.
| bobmaxup wrote:
| I tried following the same prompts:
|
| The game:
|
| """
|
| Sure, I can give it a try! Here's a logic puzzle game that I came
| up with:
|
| Title: Pathfinder
|
| Objective: The objective of the game is to fill a grid with
| numbers from 1 to N (N is determined by the size of the grid),
| such that each number appears exactly once in each row, column,
| and diagonal, and also such that the numbers create a continuous
| path throughout the grid.
|
| Rules:
|
| The grid is a square grid of size N x N. The grid is divided into
| different regions, each representing a different terrain or
| landscape (e.g. forest, desert, ocean, mountain, etc.). Each
| region has a specific starting and ending point, represented by
| two numbers on the edge of the grid. The numbers in the grid must
| form a continuous path connecting the starting and ending points
| of each region, without ever intersecting or overlapping with
| itself. The numbers in the path must also satisfy the rules of
| Sudoku, meaning that each number must appear exactly once in each
| row, column, and diagonal of the grid. Example: Here's an example
| of a Pathfinder puzzle for a 5x5 grid: | | |
| | 9 | | | | | | | 7 | | | | 3 | |
| | | | | | | | | | 2 | | | |
|
| """
|
| First: As an AI language model, I am not able to
| provide a working code with HTML and JavaScript.
|
| Then, even through multiple tries, responses would kind of just
| end when the Javascript reached the display section of the
| algorithm, and everything before that was not really great either
| (hallucinated/pseudocode functions, etc).
|
| Example: // Generate the puzzle board and
| regions function generatePuzzle() { let size
| = 5; // Change this to adjust the size of the grid
| let board = new Array(size); for (let i = 0; i <
| size; i++) { board[i] = new Array(size);
| for (let j = 0; j < size; j++) { board[i][j]
| = 0; } } let regions =
| generateRegions(size); // Generate a path that
| connects all of the regions let path =
| generatePath(regions, size); // Fill in the grid
| with the numbers from the path let index = 0;
| for (let i = 0; i < size; i++) { for (let j = 0;
| j < size; j++) { if (path[i][j] !== 0) {
| board[i][j] = path[i][j]; } else {
| board[i][j] = ""; } }
| } // Display the board and regions on the page
| let grid
|
| Maybe if the game was better, the code would generate. I am not
| sure. Overall, I really think there is more work here than this
| article and others of its genre portray.
| Sunhold wrote:
| If it ends early, just ask it to continue. If there is
| something wrong with the code, ask it to fix it.
| martopix wrote:
| I asked ChatGPT to be my dungeon master for a tiny text-based
| dungeons&dragons game. It took a bit of prompt engineering (it
| would initially believe me if I unilaterally said I won dice
| rolls and even the whole game). The result was pretty
| astonishingly good.
| kleiba wrote:
| A lot of the "it's been done before / plagiarism" arguments here
| assume that the game has been part of the training data. Do we
| know that for sure?
| flutas wrote:
| Uh... Yes?
|
| https://imgur.com/a/0k9aXGZ
| ChatGTP wrote:
| It's a bit of a problem that we don't know that...
| sebzim4500 wrote:
| It is really hard to find this out. There is no search engine
| for this data AFAIK, and it isn't all indexed by Google.
|
| Even if it was, what would we search for?
|
| I was hoping someone would find a reference on wikipedia or
| something, which would clearly be part of the training set.
| Narishma wrote:
| There's no way to know that, is there?
| sdfgdfghj wrote:
| Is it safe to assume that anything which comes out of ChatGPT is
| not novelty but an existing concept or existing concept with some
| nuances?
| ChatGTP wrote:
| Kind of, existing things only with 0.8 points of randomness
| injected for "creativity", I'm overly simplifying of course but
| it is kind of like that.
|
| I really recommend trying to get through this if you can:
| https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...
|
| It raises some questions for me about how far this current
| model can actually scale / evolve without getting a bit too far
| out to be useful for human use. But it can (sometimes) connect
| things in ways that people have likely never seen before.
|
| Maybe if you just clicked the button all day, you'd wind up
| with some really impressive new games people have never seen
| before.
| fellerts wrote:
| No, because ChatGPT doesn't understand "concepts". It helps to
| understand that all it does is take your prompt and generate
| the most likely words that will satisfy the user. This is how
| it was trained.
|
| That doesn't mean it won't regurgitate preexisting material
| (ask it to recount the constitution, for example), but it can
| definitely connect words in way that describes novel ideas.
| mercer wrote:
| yes, just like everything that is novel
| azatom wrote:
| It depends how good your layer is.
|
| If the internet is America the continent, where are lots of
| humans work, you can call the colonization as a genocide to
| native americans, or stealing the land with honor.
| kqr wrote:
| Can we ask the same thing about humans?
| dmichulke wrote:
| > .. but the solution must be unique
|
| The solution is not unique, I just found one puzzle where I have
| 8----8 | | | | 8----8
|
| Everything is solved except the 8s.
| codewiz wrote:
| Previous discussion:
| https://news.ycombinator.com/item?id=35010035
| madars wrote:
| Puzzle link: https://sumplete.com/
| asicsp wrote:
| Discussed 2 days ago:
| https://news.ycombinator.com/item?id=35010035
| [deleted]
| rashidae wrote:
| I love the game!
| codewiz wrote:
| Please publish the entire conversation with ChatGPT that led to
| the game. You can use this Chrome extension:
| https://sharegpt.com/ .
| jphoward wrote:
| The game in easy/intermediate is very fun and satisfying! However
| it doesn't seem to scale too well beyond this - it feels unless
| you get 'lucky' with one row/column having a very low total, you
| end up with many many degrees of freedom for each move, which
| scale, I suppose, quadratically?
|
| Sudoku at the hardest difficulty also has this, but usually the
| errors manifest a bit more quickly. It feels in this mode you
| could end up finding your error 'at the end' and have an
| absolutely diabolical time debugging your error?
| fsckboy wrote:
| don't tell us, tell chatGPT to incorporate your feedback!
| Lewton wrote:
| > it feels unless you get 'lucky' with one row/column having a
| very low total,
|
| very large totals gives you enough information to
| validate/eliminate numbers too
| once_inc wrote:
| People often forget that sudoku has a number of rules which
| make the game deterministic:
|
| Sudoku Rule: Don't Guess
|
| Sudoku Rule: Unique Solution
|
| You aren't supposed to guess at any point, instead solving the
| puzzle by determining a number's position by logical deduction.
| You aren't supposed to just jot something down and hoping for
| the best.
|
| Each sudoku puzzle should have one single valid solution. This
| has been used in the past by a devious creator that built his
| sudoku in such a way that the final n numbers to be filled led
| to two deduction paths; one leading to a single solution, the
| other to two possible solutions. As such, the only valid
| solution is the path where uniqueness remained.
|
| I didn't look at the code for Sumplete, but using similar rules
| when building the puzzle seems obvious.
| cyborgx7 wrote:
| >This has been used in the past by a devious creator that
| built his sudoku in such a way that the final n numbers to be
| filled led to two deduction paths; one leading to a single
| solution, the other to two possible solutions.
|
| I don't see how this differs from there being three
| solutions, thereby not following the unique solution role.
|
| Like, what is functionally the difference between picking the
| path that leads to one solution rather than two, and just
| picking one of the two solutions at the end?
| xxs wrote:
| I tried advanced (7x7) three times, it was quite doable. An
| obvious optimization would be showing the current sum of the
| rows/columns. Master (9x9) features negative numbers too, which
| is quite funny.
| mavhc wrote:
| Advanced can be easily solved with some logic, odd vs even,
| eliminating impossible combinations.
|
| Master I've no idea how to solve
| yesenadam wrote:
| I tried 9x9 and it seemed impossible at first but I solved
| it in 5-10 minutes. I want to try larger ones!
|
| Keep fixing digits one at a time with things like parity,
| e.g. if the numbers are all odd and sum to an odd number,
| you know an odd number of them are needed. If a total is
| very high or low, there's often obviously only one way to
| make it. For totals not quite so high, if you choose the
| few highest numbers, and by trying ways to reduce the total
| by the excess, it becomes evident what the numbers are.
| ("Get your hands dirty.")
|
| Often a row/column contains e.g. two 7s, and you know one
| of them is needed, but not which until other information
| comes in. (Having an "OR" label/symbol to mark those would
| be nice I guess.)
|
| e.g. One of the columns I had: total 46, column is
| 16,-2,12,-2,14,-2,14,16,1. Eliminate the 1 by parity. I try
| adding the largest numbers, 16,16,14, which add to 46...and
| somehow I can see/intuit that there's no other way of
| making 46, as the -2s can only adjust downwards. So you
| green-circle the two 16s, leave the two 14s blank, as you
| don't know which is needed yet, and cross out all the
| others.
| elliotec wrote:
| Fascinating. I wonder how soon it'll be before it can do the
| devops for you too.
| manojlds wrote:
| AIOps is a thing right. And ChatOps as well, though it didn't
| get lot of traction.
|
| I am sure someone is already working on a ChatGPTOps
| jedberg wrote:
| I just tried it. This was my prompt:
|
| create a terraform template to create a brand new aws account
| from scratch, and then set up and admin account and two users
| named alice and bob.
|
| This is what it output:
|
| https://gist.github.com/jedberg/dd31783e13521a69aa8923cb5142...
| (which at quick glance looks correct)
|
| It then said: " Note that you will need to have appropriate
| permissions and credentials to create a new AWS account, and
| this template assumes that you have those."
|
| So I asked it how to get those permissions and it gave me step
| by step instructions on creating a new account, adding a
| payment method, and getting the root keys.
|
| Then I asked it how to run the terraform and it gave me the
| commands to do it.
|
| So I'd say yes, it can do DevOps too!
| sushiburps wrote:
| Asked ChatGPT the following "I have a static HTML file I'd like
| to host in AWS S3 using a CloudFront distribution. I'd also
| like to manage the domain's DNS in Route 53. Can you write me
| the Terraform to do this?"
|
| https://i.imgur.com/GW5rUpL.png
| vorticalbox wrote:
| it could probably generate some terraform if you explained the
| task to it
| oofnik wrote:
| Can confirm that this is indeed possible
| anukin wrote:
| Yeah it definitely requires an AI to make sense of some of
| those k8s deployments
| cxromos wrote:
| > And herein lies the issue with ChatGPT, it can generate
| functioning code, but can also lie through its none existent
| teeth about it.
|
| now i'm really worried :)))))
| ChatGTP wrote:
| Well it's not "lying", it's just making stuff up, which is what
| it has to do to be "original" or createive, else every time you
| clicked generate it would give you the same answer, same code
| etc.
| romwell wrote:
| I have always been very skeptical about AI being just a pile of
| hype, but this is the straw that broke the proverbial camel's
| back for me.
|
| I don't know _what_ the AI will or will not be able to do, but
| what 's clear is that things are not going to be the same any
| longer.
| [deleted]
| csomar wrote:
| In the short-medium future the path of least resistance will be
| auto-completion. You can achieve faster with AI than without
| AI. People will also start "mastering" AI prompting in the same
| way they mastered Google search in the past.
|
| New Jobs:
|
| - AI Prompt Engineer.
|
| - AI Prompt Analyst.
|
| - AI Prompt Architect.
| ChatGTP wrote:
| I recently read the first half of this:
| https://writings.stephenwolfram.com/2023/02/what-is-
| chatgpt-... and as a programmer I can quite easily understand
| it, not completely but I immediately had similar thoughts to
| you...even if you are joking :)
|
| If you kind of understand how the prediction works, you're
| likely able to become quite good prompting it to generate
| mostly functional things.I really believe a lot of the
| articles we see here are people doing just that,
| understanding how the tool works, then getting 80% of the way
| to something functional , then using actual programming
| knowledge to make things work.
|
| I do think as others have said, this puzzle game has already
| existed so nothing was really invented, but...how would you
| know that if you didn't know the name or the format of the
| original puzzle game.
| courseofaction wrote:
| I have a narrative generation system in development - the
| hardest part of developing it is not just stopping and talking
| to the characters for hours. They surprise me time and time
| again. This feels like science fiction.
| cosmin800 wrote:
| No, it didn't
| progx wrote:
| Battlefield developer could look now for a new job. Their
| "innovation" could be done bei ChatGPT as well ;-)
| aldanor wrote:
| Potential QoL improvements for the game itself (maybe ChatGPT
| would have suggested them if you asked it?)
|
| - Ability to mark cells as "locked" (i.e. "definitely present").
| It's hard to play it on a larger grid since you have to keep it
| all in your head - Built in calculator of some sort where you
| could multi-select a few cells and it would show their sum
| somewhere. Otherwise it's too much arithmetics to do manually
| djrockstar1 wrote:
| > Ability to mark cells as "locked" (i.e. "definitely present")
|
| Try double clicking
___________________________________________________________________
(page generated 2023-03-06 23:01 UTC)