[HN Gopher] How did you do on the AI art Turing test?
___________________________________________________________________
How did you do on the AI art Turing test?
Author : sieste
Score : 34 points
Date : 2024-11-22 19:30 UTC (3 hours ago)
(HTM) web link (www.astralcodexten.com)
(TXT) w3m dump (www.astralcodexten.com)
| sieste wrote:
| The only way I can explain people getting 98% accuracy on this is
| being familiar with the handful of AI artists submitting their
| work for this competition.
| Izkata wrote:
| It's a google form with no apparent time limit. It wouldn't
| surprise me if some people could do this (think of it like how
| older special effects in TV/movies look dated), but most likely
| they did an image search on each one and got one wrong.
| RcouF1uZ4gsC wrote:
| I don't think that is AI art Turing test.
|
| An AI art Turing Test would be interactive with me telling it
| what to draw and what changes to make and see if what is
| producing the art is human or AI.
| egypturnash wrote:
| This species of test would also need a multi-day turnaround
| period on each image. And/or a video stream of the work being
| drawn.
|
| "Changes" are an interesting one, honestly as a professional
| artist who has to pay her rent, there is certain complexity of
| change beyond which I am likely to say "look, we're going to
| need to renegotiate the budget on this if you want this much of
| a change from the sketch you already approved", or even "no".
| equestria wrote:
| Eh, this is pretty unfair. That's a test of how good humans are
| at deceiving other humans, not a of how hard it is to distinguish
| run-of-the-mill AI art from run-of-the-mill human art in real
| life.
|
| First, by their own admission, the author deliberately searched
| for generative images that don't exhibit any of the telltale
| defects or art choices associated with this tech. For example,
| they rejected the "cat on a throne" image, the baby portrait, and
| so on. They basically did a pre-screen to toss out anything the
| author recognized as AI, hugely biasing the results.
|
| Then, they went through a similar elimination process for human
| images to zero in on fairly ambiguous artwork that could be
| confused with machine-generated. The "victorian megaship" one is
| a particularly good example of such chicanery. When discussing
| the "angel woman" image, they even express regret for not getting
| rid of that pic because of a single detail that pointed to human
| work.
|
| Basically, the author did their best to design a quiz that humans
| should fail... and humans still did better than chance.
| bena wrote:
| Also impressionism is probably one of the most favorable art
| styles for AI. The lack of detail means there are fewer places
| for AI to fuck up.
|
| A street with cafe chairs and lights, that's like an entire
| genre of impressionist paintings.
| bongodongobob wrote:
| I think it's fair. It's the same thing humans do with their own
| art. You don't release the piece until you like it. You revise
| until you think it's don't. If a human wants to make AI art,
| they aren't just going to drop the first thing they generated.
| They're going to iterate. I think it's just as unfair to
| include the worst generations, because people are going to
| release the highest quality they can come up with.
| majormajor wrote:
| > It's the same thing humans do with their own art.
|
| How so? Humans distributed all those "I filtered them out
| because they were too obvious" AI ones that _aren 't_ in the
| test too. So they passed someone's "is this something that
| should get released" test.
|
| What we _aren 't_ seeing is human-generated art that nobody
| would confuse with a famous work - which of course there is a
| lot of out there - but IMO it generally looks "not famous" in
| very _different_ ways. More "total execution issues" vs
| detail issues.
| rurp wrote:
| Based on what I've empirically seen out in the world most
| people posting AI art are not using the same filtering as the
| author of this test. Plus the human choices used probably
| skew more towards what people think of as classic AI art than
| all human art as a whole.
|
| The test was interesting to read about, but it didn't really
| change my mind about AI art in general. It's great for
| generating stock images and other low engagement works, but
| terrible as fine art that's meant to engage the user on any
| non-superficial level.
| equestria wrote:
| > I think it's fair. It's the same thing humans do with their
| own art.
|
| No, hold on. The key part is that you have a quiz that
| purports to test the ability of an average human to tell AI
| artwork from human artwork.
|
| So if you specifically select images for this quiz based on
| the fact that _you, the author of the quiz_ can 't tell them
| apart, then your quiz is no longer testing what it's promised
| to. It's now a quiz of "are you incrementally better than the
| author at telling apart AI and non-AI images". Which might be
| fine, but is a lot less interesting, right?
| cicdw wrote:
| This article has an implicit premise that the ultimate judge of
| art is "do I/people like it" but I think art is more about the
| possibilities of interpretation - for example, the classics/"good
| art" lend themselves to many reinterpretations, both by different
| people and by the same person over time. When humans create art
| "manually" all of their decisions - both conscious and
| unconscious - feed into this process. Interpreting AI art is more
| of a data exploration journey than an exploration of meaning.
| ARandumGuy wrote:
| That's one of my problems with AI art. AI art promises to bring
| your ideas to life, no need to sweat the small stuff. But it's
| the small details and decisions that often make art great!
| Ideas are a dime a dozen in any artistic medium, it's the
| specific way those ideas are implemented that make art truly
| interesting.
| cicdw wrote:
| I couldn't agree more; I love what you said in your other
| reply: "AI art punishes the viewer who looks closer"
| jncfhnb wrote:
| Eh. That's an artificial goalpost. Realistically, it's a tool
| in the toolkit.
| dpig_ wrote:
| Agreed. AI art subtracts intentionality.
| Izkata wrote:
| 66% here. I was pretty much scrolling through and clicking on
| first instinct instead of looking in any detail.
|
| Interestingly I did a lot better in the second half than the
| first half - without going through and counting them up again I
| think somewhere around 40% in the first half and 90% in the
| second half. Not sure if it's because of the selection/order of
| images or if I started unconsciously seeing commonalities.
| ARandumGuy wrote:
| I feel like the comment made by the author's friend captures a
| lot of my feelings on AI art. AI art is often extremely detailed,
| but that detail is often just random noise. It's art that becomes
| worse and makes less sense the more carefully you look at it.
|
| Compared that to human art. When a person makes a highly detailed
| artwork, that detail rewards looking closely. That detail forms a
| cohesive, intentional vision, and incentivizes the viewer to
| spend the time and effort to take it all in. A person could spend
| hours looking at Bruegel's Tower of Babel paintings, or Bosch's
| "The Garden of Earthly Delights".
|
| Overall, I've never felt the need to spend the time looking
| closely at AI art, even AI art that I couldn't tell was AI right
| away. Instead of rewarding close inspection with more detail, AI
| art punishes the viewer who looks closer by giving them
| undecipherable mush.
| morkalork wrote:
| It's exactly what isn't captured in the training data. The AI
| knows what the final texture of an oil painting looks like but
| it doesn't, know if what it's creating isn't possible from the
| point of view of physical technique. Or, likewise, it doesn't
| see the translation from mental image to representation of that
| image that a human has. It's always working off the final
| product.
| card_zero wrote:
| That makes it sound like impressionism. But the phony details
| have a more intense bullshitting quality, like the greebles on
| a Star Wars spaceship.
| doawoo wrote:
| There's a lot of thought that goes into things like the
| greebles on a spaceship, like the shape language, the values
| and hues, etc.
|
| Impressionism might seem "random" like what a model would
| output, but the main difference is the human deciding how
| that "randomness" should look.
|
| The details on a model generated art piece are meaningless to
| me, no one sat down and thought "the speckles here will have
| to be this value to ensure they don't distract from the rest
| of the piece."
|
| That's more what I look at when I digest art, the small,
| intentional design choices of the person making it.
| lupire wrote:
| Hmm? Impressionism is noted for extreme _lack_ of detail,
| that still is suggestive of something specific, because the
| artist knows what details your brain will fill in. (8bit
| pixel art is impressionistic :-) )
| majormajor wrote:
| The gate picture has the same problem as as the cat one that he
| didn't filter out. There's a lot going on, and the lighting
| does seem to be one of the somewhat-inconsistent issues IMO,
| but it's just generally weird about why there's cats of all
| different sizes, why some of the smallest cats have the same
| coloring as the biggest ones, but some don't, what's going on
| with the arms of the two darker cats on the right, why aren't
| the sides of the throne symmetric, etc.
|
| Everything is consistent in terms of "the immediate pixels
| surrounding it" but the picture as a whole is just "throw a LOT
| at the wall."
|
| It passes the "looks cool" test but fails the "how likely would
| a human be to paint that particular composition" test.
| QuadmasterXLII wrote:
| so much of the value of art, which Scott has actually endowed on
| these AI generated pieces, is the knowledge that other people are
| looking at the same thing as you.
| arjie wrote:
| Generative AI is so cool. My wife (a creative director) used it
| to help design our wedding outfits. We then had them embroidered
| with those patterns. It would have been impossible otherwise for
| us to have that kind of thing expressed directly. It's like
| having an artist who can sketch really fast and who you can keep
| correcting till your vision matches the expression. Love it!
|
| I don't think there have been any transformative AI works yet,
| but I look forward to the future.
|
| It's unsurprising to me that AI art is often indistinguishable
| from real artists' work but famous art is so for some reason
| other than technical skill. Certainly there are numerous replica
| painters who are able to make marvelous pieces.
|
| Anyway, I'm excited to see what new things come.
| Barrin92 wrote:
| It would have been interesting to know how much time most people
| spent per picture because if you look at the quoted comment from
| the well scoring art interested person mentioned:
|
| _" The left column has a sort of door with a massive top-of-
| doorway-thingy over it. Why? Who knows? The right column doesn't,
| and you'd expect it to. Instead, the right column has 2.5 arches
| embossed into it that just kind of halfheartedly trail off._"
|
| You can find this in almost every AI generated picture. The
| picture that people liked most, AI generated with the cafe and
| canal, the legs on the chairs make little sense. Not as bad as in
| non-curated AI art, but still no human would paint like this.
| Same for the houses in the background. If you spend say a minute
| per picture with AI art you almost always find these random
| things, even if the image is stylized, unlike human art it has a
| weird uncanniness to it.
| 12_throw_away wrote:
| Oh come on. I guess I missed the part in the "Turing test" where
| a human filters out 99.999% of the machine's output prior to the
| test.
| hombre_fatal wrote:
| On the other hand, if you think you can identify AI art, that
| only means anything if you can do it on the hard cases, not the
| easy ones.
|
| It's also a reminder to not believe images/video that you see
| online. It's not the low quality slop that's going to fool you.
| beepbooptheory wrote:
| Duchamp rolling in his grave about this post!
| maldusiecle wrote:
| Fine art is a matter of nuance, so in that sense I think it does
| matter that a lot of the "human art" examples are aggressively
| cropped (the Basquiat is outright cut in half) and reproduced at
| very low quality. That Cecily Brown piece, for example, is 15
| feet across in person. Seeing it as a tiny jpg is of course not
| very impressive. The AI pieces, on the other hand, are native to
| that format, there's no detail to lose.
|
| But those details are part of what make the human art interesting
| to contemplate. I wouldn't even think of buying an art book with
| reproductions of such low quality--at that point you do lose
| what's essential about the art work, what makes it possible to
| enjoy.
| egypturnash wrote:
| It's interesting that the impressionist-styled pieces mostly
| fooled people. I think this is because the style requires getting
| rid of one of the hallmarks of AI imagery: lots and lots of
| mostly-parallel swooshy lines, at a fairly high frequency.
| Impressionism's schtick is kind of fundamentally "fuck high-
| frequency detail, I'm just gonna make a bunch of little
| individual paint blobs".
|
| One of the other hallmarks of AI imagery was deliberately kept
| out of this test. There's no _shitposts_. There 's _one_ , as an
| example of "the dall-e house style". It's a queen on a throne
| made of a giant cat, surrounded by giant cats, and it's got a lot
| of that noodly high-frequency detail that _looks_ like something,
| but it is also a fundamentally goofy idea. Nobody 's gonna pay
| Michael Whelan to paint the hell out of this and yet _here it
| is_.
| d--b wrote:
| I think what gave AI away the most was mixed styles. If one part
| of the painting is blurred, and another part is very focused, you
| can tell it's AI. People don't do that.
|
| I got all of Jack Galler's pictures wrong though. The man knows
| how to do it.
| throwawayk7h wrote:
| You can't really draw many conclusions from this test, since the
| AI art has already been filtered by Scott to be ones that Scott
| himself found confusing. So What do any of the numbers at the end
| of this really mean? "Am I better than Scott at discerning AI art
| from human" is about the only thing this test says.
|
| If you didn't filter the AI art first, people would do much
| better.
| jncfhnb wrote:
| AI Art can be hard to identify in the wild. But it still largely
| sucks at helping you achieve specific deliverables. You can get
| an image. But it's pretty hard to actually make specific images
| in specific styles. Yes we have Loras. Yes we have control nets
| (to varying degrees) and ipadapter (to lesser degrees) and face
| adapters and what not. But it's still frustrating to get
| something consistent across multiple images. Especially in
| illustrated styles.
|
| AI Art is good if you need something in a general ballpark and
| don't care about the specifics
___________________________________________________________________
(page generated 2024-11-22 23:00 UTC)