[HN Gopher] We ran over 600 image generations to compare AI imag...
___________________________________________________________________
We ran over 600 image generations to compare AI image models
Author : kalleboo
Score : 87 points
Date : 2025-11-11 17:26 UTC (5 hours ago)
(HTM) web link (latenitesoft.com)
(TXT) w3m dump (latenitesoft.com)
| Dwedit wrote:
| You can always identify the OpenAI result because it's yellow.
| Bombthecat wrote:
| And mid journey because it's cell shading:)
| Hoasi wrote:
| Also because it's mid :)
| jstummbillig wrote:
| > If you made it all the way down here you probably don't need a
| summary
|
| Love the optimism
| LogicFailsMe wrote:
| I skipped to the end to see if they did any local models.
| spoilers: they didn't.
| CWuestefeld wrote:
| Honestly, I think it was misfounded. As an photographer and
| artist myself, I find the OpenAI results head-and-shoulders
| above the others. It's not perfect, and in a few cases one or
| the other alternative did better, but if I had to pick one, it
| would be OpenAI for sure. The gap between their aesthetics and
| mine makes me question ever using their other products (which
| is purely academic since I'm not an Apple person).
| Retric wrote:
| How many of those result did you actually look at? I thought
| it did ok with the cats, but check the other images and
| OpenAI strait up failed to do the prompt a large fraction of
| the time.
| sema4hacker wrote:
| Are artists and illustrators going the way of the horse and
| buggy?
| LogicFailsMe wrote:
| No, but this _is_ the beginning of a new generation of tools to
| accelerate productivity. What surprises me is that the AI
| companies are not market savvy enough to build those tools yet.
| Adobe seems to have gotten the memo though.
| somenameforme wrote:
| In testing some local image gen software, it takes about 10
| seconds to generate a high quality image on my relatively old
| computer. I have no idea the latency on a current high end
| computer, but I expect it's probably near instantaneous.
|
| Right now though the software for local generation is
| horrible. It's a mish-mash of open source stuff with varying
| compatibility loaded with casually excessive use of
| vernacular and acronyms. To say nothing of the awkwardness of
| it mostly being done in python scripts.
|
| But once it gets inevitably cleaned up, I expect people in
| the future are going to take being able to generate
| unlimited, near instantaneous images, locally, for free, for
| granted.
| pkroll wrote:
| Did you test some local image gen software in that you
| installed the Python code on the github page for a local
| model, which is clearly a LOT for a normal user... or did
| you look at ComfyUI, which is how most people are running
| local video and image models? There are "just install this"
| versions, which eases the path for users (but it's still,
| admittedly, chaos beneath the surface).
| bnj wrote:
| I've been waiting for solutions that integrate into the
| artistic process instead of replacing it. Right now a lot of
| the focus is on generating a complete image, but if I was in
| photoshop (or another editor) and could use AI tooling to
| create layers and other modifications that fit into a
| workflow, that would help with consistency and productivity.
|
| I haven't seen the latest from adobe over the last three
| months, but last I saw the firefly engine was still focused
| on "magically" creating complete elements.
| Hoasi wrote:
| > Adobe seems to have gotten the memo though.
|
| So far Adobe AI tools are pretty useless, according to many
| professional illustrators. With Firefly you can use other
| (non-Adobe) image generators. The output is usually barely
| usable at this point in time.
| jonathanstrange wrote:
| Artists no, illustrators and graphic designers yes. They'll
| mostly become redundant within the next 50 years. With these
| kind of technologies, people tend to overestimate the short-
| term effects and severely underestimate the long-term effects.
| Bombthecat wrote:
| Yes and now. IKEA and co didn't replace custom made tables,
| just reduced the number of people needing a custom table.
|
| Same will happen to music, artists etc. They won't vanish. But
| only a few per city will be left
| consumer451 wrote:
| "AI won't replace you, but someone who knows how to use AI will
| replace you" appears to be too short a phrase.
|
| There is no better recent example than _AI comedy made by a
| professional comedian_ [0]
|
| Of course, this makes sense once you think about it for a
| second. Even AGI, without a BCI, could not read your mind to
| understand what you want. Of course, the people who have been
| communicating these ideas with other humans up to this point,
| are the best at doing that.
|
| [0] old.reddit.com/r/ChatGPT/comments/1oqnwvt/ai_comedy_made_by
| _a_professional_comedian/
| jrflowers wrote:
| > There is no better recent example than AI comedy made by a
| professional comedian
|
| To clarify, the "comedy" part of this "AI comedy" was written
| entirely by a human with no assistance from a language model.
|
| > For anyone interested in my process. I wrote every joke
| myself, then use Sora 2 to animate them.
| consumer451 wrote:
| Exactly.
|
| Apologies if I wrote my original comment poorly, but that
| was I was trying to communicate.
|
| Not only was this person able to write good comedy, but he
| knew what tools were available and how to use them.
|
| I previously wrote:
|
| > "AI won't replace you, but someone who knows how to use
| AI will replace you." ...
|
| The missing part is "But a person who was excellent at
| their pre-AI job, will replace ten of you."
|
| An analog that just popped into my head is the nearly
| always missed part of the quote "the customer is always
| right" ... "in matters of taste."
| rgmerk wrote:
| For some applications.
|
| Photography didn't make artists obsolete.
|
| For that matter, the car didn't make horse riding completely
| obsolete either.
|
| For artists, the question is whether generative AI is like
| photography or the car. My guess, at this stage, is
| photography.
|
| For what it's worth I think the proponents of generative AI are
| grossly overestimating the utility and economic value of meh-OK
| images that approximate the thing you've asked for.
| sylos wrote:
| I've seen cover art on a lot of magazines already replaced
| with AI images. I suspect, for the time being, that a lot of
| the low hanging art fruit will be destroyed by image
| generation. The knock on effect is less art jobs, but more
| artists. In the vein of your analogy, it removes the gas
| station attendants that fill your tank.
| kg wrote:
| When there's a need for something with specific traits and
| composition at high quality, I've yet to see a model that can
| deliver that, especially in a reasonable amount of time. It's
| still way more reliable to just hand a description to a skilled
| illustrator along w/references and then go back and forth a bit
| to get a quality result. The illustrator is more expensive, but
| my time isn't free, so it works out.
|
| I could see that changing in a few years.
| Theodores wrote:
| Horse and buggy isn't quite the analogy, I think it is more
| like the arrival of junk food, packed with sugar, salt and
| saturated fats. You will still be able to find a cafe or
| restaurant where a full kitchen team cooks from scratch but
| everything else is fast food garbage.
|
| Maybe just the advent of the microwave oven is the analogy.
|
| Either way, I am out. I have spent many days fiddling with AI
| image generation but, looking back on what I thought was 'wow'
| at the time, I now think everything AI art is practically
| useless. I only managed one image I was happy with and most of
| that was GIMP, not AI.
|
| This study has confirmed my suspicions, hence I am out.
|
| Going back to the fast food analogy, for the one restaurant
| that actually cooks actual food from actual ingredients, if
| everyone else is selling junk food then the competition has
| been decimated. However, the customers have been decimated too.
| This isn't too bad as those customers clearly never appreciated
| proper food in the first place, so why waste effort on them? It
| is a pearls and swine type of thing.
| kevin009 wrote:
| Everyday I generate more than 600 image and also compare them, it
| takes me 5 hours
| alienbaby wrote:
| Interesting experiment, though I'm not certain quite how the
| models are usefully compared.
| th0ma5 wrote:
| This seems to imply that the capabilities being tested are like
| the descriptive words used in the prompts, but, as a category
| using random words would be just as valid for exercising the
| extents of the underlying math. And when I think of that reality
| I wonder why a list of tests like this should be interesting and
| to what ends. The repeated nature of the iteration implies that
| some control or better quality is being sought but the mechanism
| of exploration is just trial and error and not informative of
| what would be repeatable success for anyone else in any other
| circumstance given these discoveries.
| fsniper wrote:
| Is it me or ChatGPT change subtle or sometimes more prominent
| things? Like ball holding position of the hand, face features
| like for head, background trees and alike?
| qayxc wrote:
| It's not you. The model seems to refuse to accurately reproduce
| details. It changes things and leaves stuff out every time.
| yapyap wrote:
| Using gen. ai for filters is stupid, a filter guarantees the same
| object but filtered, a gen. AI version of this guarantees nothing
| and an expensive AI bill.
|
| It's like using gen. ai to do math instead of extracting the
| numbers from a story and just doing the math with +, -, / and *
| whoaoweird wrote:
| It was interesting to see how often the OpenAI model changed the
| face of the child. Often the other two models wouldn't, but
| OpenAI would alter the structure of their head (making it
| rounder), eyes (making them rounder), or altering the position
| and facing of the children in the background.
|
| It's like OpenAI is reducing to some sort of median face a little
| on all of these, whereas the other two models seemed to reproduce
| the face.
|
| For some things, exactly reproducing the face is a problem -- for
| example in making them a glass etching, Gemini seemed unwilling
| to give up the specific details of the child's face, even though
| that would make sense in that context.
| sasper wrote:
| I've noticed that OpenAI modifies faces on a regular basis. I
| was using it to try and create examples of different haircuts
| and the face would randomly turn into a different face --
| similar but noticeably changed. Even when I prompted to not
| modify the face, it would do it regardless. Perhaps part of
| their "safety" for modifying pictures of people?
| SweetSoftPillow wrote:
| https://www.reddit.com/r/ChatGPT/comments/1n8dung/chatgpt_pr...
| fsniper wrote:
| It's also changing scene features too. Like removed background
| trees.
| captainclam wrote:
| It looks to me like OpenAI's image pipeline takes an image as
| input, derives the semantic details, and then essentially
| regenerates an entirely new image based on the "description"
| obtained from the input image.
|
| Even Sam Altman's "Ghiblified" twitter avatar looks nothing
| like him (at least to me).
|
| Other models seem much more able to operate directly on the
| input image.
| frotaur wrote:
| It's crazy that the 'piss filter' of openAI image generation
| hasn't been fixed yet. I wonder if it's on purpose for some
| reason ?
| CamperBob2 wrote:
| They don't want you creating images that mimic either works of
| other artists to an extent that's likely to confuse viewers (or
| courts), or that mimic realistic photographs to an extent that
| allows people to generate low-effort fake news. So they impose
| an intentionally-crappy orange-cyan palette on everything the
| model generates.
|
| Peak quality in terms of realistic color rendering was probably
| the initial release of DALL-E 3. Once they saw what was going
| to happen, they fixed that bug _fast_.
| asadotzler wrote:
| LOL. If you believe that, let me tell you about this bridge
| I've got.
| gs17 wrote:
| It's interesting to me that the models often have their "quirks".
| GPT has the orange tint, but it also is much worse at being
| consistent with details. Gemini has a problem where it often
| returns the image unchanged or almost unchanged, to the point
| where I gave up on using it for editing anything. Not sure if
| Seedream has a similar defining "feature".
|
| They noted the Gemini issue too:
|
| > Especially with photos of people, Gemini seems to refuse to
| apply any edits at all
| minimaxir wrote:
| Nano Banana in general cannot do style transfer effectively
| unless the source image/subject is a similar style as the
| target style, which is an interesting and unexpected model
| quirk. Even the documentation examples unintentionally
| demonstrates this.
|
| Seedream will always alter the global color balance with edits.
| basch wrote:
| Something like a style transfer works better in Whisk. Still
| quirky and hit and miss.
| dwringer wrote:
| I've definitely noticed Gemini's tendency to return the image
| basically unchanged, but not noticed it being worse or better
| for images of people. When I tested by having it change aspects
| of a photo of me, I found it was far more likely to cooperate
| when I'd specify, for instance, "change the hair from long to
| short" rather than "Make the hair short" (the latter routinely
| failed completely).
|
| It also helped to specify which other parts should _not_ be
| changed, otherwise it was rather unpredictable about whether it
| would randomly change other aspects.
| basch wrote:
| Not only does it return the image unchanged, but if you are
| using the Gemini interface, it confidently tells you it made
| the changes.
| mattmaroon wrote:
| I have had that problem with nano banana but when it works I
| find it so much better than the others for editing an image.
| Since it's free I usually try it first, and I would say
| approximately 10% of the time find myself having to use
| something else.
|
| I'm editing mostly pics of food and beverages though, it
| wouldn't surprise me if it is situationally better or worse.
| jrflowers wrote:
| I like that they call openai's image generator ground breaking
| and then explain that it's prone to taking eight times longer to
| generate an image before showing it add a third cat over and over
| and over again
| treesciencebot wrote:
| we build our sandbox just for this use case, fal.ai/sandbox. take
| the same image/prompt, and compare across tens of models.
| beezle wrote:
| Found OpenAI too often heavy handed. On balance, I'd probably
| pick Gemini narrowly over Seedream and just learn that sometimes
| Gemini needs a more specific prompt.
| Scrapemist wrote:
| Seedream is the only one that outputs 4k. Last time I checked
| that is..
| stevage wrote:
| I wish they'd used a better image than the low contrast mountain,
| which rarely transformed into anything much.
| justhw wrote:
| I came to the same conclusion as the authors after generating
| 1000s of thumbnails[1]. OpenAI alters faces too much and smoothes
| out details by default. NanoBanana is the best but lacks high
| fidelity option. SeeDream is catching up to NanoBanana and
| sometimes is better. It's been too long since OpenAI's gpt-img-1
| came out, hope they launch a better model soon.
|
| [1] = https://thumbnail.ai/
| chanw wrote:
| Hey. We'd love to fund thr generations for free for you to try
| Riverflow 2 out if you're up for it. Riverflow 1 ranks above them
| all and 2 is now in preview this week.
| angry_albatross wrote:
| The shortcut to flip between models in an expanded view is nice,
| but the original image should also be included as one of the
| things to flip between, and should be included in the side by
| side view.
___________________________________________________________________
(page generated 2025-11-11 23:00 UTC)