[HN Gopher] Adobe Firefly vs. MidJourney AI image generation
___________________________________________________________________
Adobe Firefly vs. MidJourney AI image generation
Author : redox99
Score : 108 points
Date : 2023-04-03 16:23 UTC (6 hours ago)
(HTM) web link (twitter.com)
(TXT) w3m dump (twitter.com)
| moralestapia wrote:
| I don't really think Firefly is bad, it's just that MJ is
| exceptionally good.
| AuryGlenz wrote:
| The amount of prompt injections they're doing (presumably for
| racial equity) really ruins a lot of prompts on Firefly.
|
| Here's a 'Soviet style propaganda poster of a female cosmonaut,
| illustrated, illustration': https://i.imgur.com/BEGHUsx.jpg
|
| The same thing in Stable Diffusion:
| https://i.imgur.com/4Yfk7fO.jpeg
|
| You'll note I tried to add illustration on to the end after
| illustrated to try and force it. That's because when it does this
| it tends to break the overall artstyle.
|
| Another prompt I did was a dwarf carrying a rusty axe, fantasy
| art, etc. It worked fine when it generated white dwarves, but
| when it had races injected their faces were like photographs on
| top of rest of the art. One was a black man with dreadlocks and
| the other a Muslim woman with a headscarf. You can add "white" to
| the prompt to try and fix this but chances are it'll apply
| "white" to other things as well.
|
| Also, their censor is hilarious. "Photo of a man in prison" gets
| rejected. "Photo of a woman in prison" works. You can't use the
| word attack, battle, World War 2, etc. You can't use the word
| "lingerie" or "babydoll" but you can absolutely do a "sheer lace
| cami set." That means this prompt totally worked when I tried it
| - "A 14 year old girl in bed with a sheer lace cami set," and the
| results were what you'd expect - sexy pose and all.
|
| I honestly don't know how larger companies are going to deal with
| trying to censor this stuff. It's a really hard problem. I'm sure
| there's a combination of words out there that would let someone
| generate child porn. Potential legal issues aside I'm not sure
| Adobe has the stomach to deal with the Twitter/media fallout.
| nycdatasci wrote:
| From Adobe's TOS, which you agreed to when you accessed the
| app: Do not use Firefly to attempt to create, upload, or share
| abusive or illegal content. This includes, but is not limited
| to, the following:
|
| * Pornographic material and explicit nudity * Hateful or highly
| offensive content that attacks or dehumanizes a group based on
| race, ethnicity, national origin, religion, serious disease or
| disability, gender, age, or sexual orientation * Graphic
| violence or gore * The promotion, glorification, or threats of
| violence * Illegal activities or goods * Self-harm or the
| promotion of self-harm * Depictions of nude minors or minors in
| a sexual manner * Promotion of terrorism or violent extremism *
| Dissemination of misleading, fraudulent, or deceptive content
| that could lead to real-world harm * Personal or private
| information of others (like full name, home address, phone
| number, email address, government issued IDs, or anything else
| that would let someone locate or contact you in the real
| world). * Please note that we may report any material
| exploiting minors to the National Center of Missing & Exploited
| Children (NCMEC). If at any time you believe someone has
| violated these Guidelines or the Adobe Firefly Beta Terms of
| Use, please report it by contacting us at abuse@adobe.com.
| AuryGlenz wrote:
| Yeah, sure. I was prodding it to see how they handled what
| seems like a really hard issue and I used the built in tool
| to send a report. It seems to me that should be one of the
| reasons to have a beta.
|
| To be fair my prompt didn't explicitly say anything
| inherently sexual, and I don't think the images would pass
| the test for child porn.
|
| I do find that language funny - material exploiting minors -
| who was exploited? I'm waiting for journalists to figure out
| that it's a thing easily done in Stable Diffusion and
| probably less easily done in other image generators to see
| the outrage. Personally I think that if pedophiles could
| generate images that might mean there'd be less of a market
| for real images and therefore less (real) child exploitation,
| but I doubt Adobe will want to make that argument ;)
| lemoncookiechip wrote:
| There's a Stable Diffusion plug-in for Photoshop. While it
| doesn't promise as many features as Firefly, it's still a solid
| tool, and obviously has the SD community to back it up with
| models, loras... https://github.com/AbdullahAlfaraj/Auto-
| Photoshop-StableDiff...
| quitit wrote:
| It's not specific to copyright, Adobe Firefly has trouble
| producing something as straightforward as a cat. This puts it on
| par with earlier renditions of competitor AI image generators,
| and just like those, I'm sure Adobe will work it out.
| anonylizard wrote:
| Just like they worked it out with figma right? Surely they
| won't be overtaken by an upstart and have to pay $20 billion
| for it?
|
| Firefly is absolutely underwhelming. Its been 10 months since
| Dalle 2, and it is just a slightly improved version of Dalle2
| (With just as bad censorship filters). Adobe clearly only
| started their generative AI project only after Dalle 2 was
| released. Otherwise you would expect their internal team to
| have far more polished results.
|
| Like, who exactly is a user of firefly at this point? You'd
| work far faster just copy pasting images from Midjourney into
| photoshop, than trying to wrangle firefly to produce good
| results. If you want strong control, SD is the answer. If you
| hate AI, you won't use it at all. Adobe will barely get
| feedback data.
| killerdhmo wrote:
| lol that is not the user they're targeting. They're targeting
| the express user. Someone who is definitely not using Discord
| or Photoshop.
| anonylizard wrote:
| Its unclear where these 'express users' are. Midjourney
| only takes a day to learn, its not rocket science to learn
| how to install discord. Its far faster to join midjourney
| and use its images, than to learn how to wrangle firefly to
| work.
| neom wrote:
| What types of cats are you trying to produce?
| https://imgur.com/a/lZDqbOw I got decent cats with a simple
| prompt.
| sebzim4500 wrote:
| There's definitely something off about the first four photos,
| I can't put my finger on what it is though. It reminds me of
| early SD generations.
| sebzim4500 wrote:
| >just like those, I'm sure Adobe will work it out.
|
| This is not guaranteed, since Adobe has a much smaller library
| to call on for training data than the rest of the market. This
| can't really change unless they follow their competitors and
| ignore licences etc.
| Robotbeat wrote:
| The predictable result of this is that companies like Adobe will
| be pushing for absurdist interpretations of copyright so their
| competitors in this space are annihilated. They'll be calling it
| "ethical and fair." And the effect will be basically a
| destruction of open source versions of these tools.
| adventured wrote:
| The castle walls will go up soon. Make hay while the sun is
| shining.
|
| The entrenched will argue for regulation and get it. The
| entrenched that built everything on data they didn't actually
| own, will come out in favor of preventing that from happening
| again.
| CaptainFever wrote:
| With the end result being giant companies owning tons of IPs
| like Getty, Adobe and perhaps Disney owning proprietary AI
| models anyway, with no open source alternative for indie
| artists to use because they all got sued out of existence. A
| pretty bad future; definitely not what I would call ethical.
| gamblor956 wrote:
| Firefly is great...as long as you can deal with the many filtered
| terms they've burdened it with.
|
| For example, during testing, I could not use the word "evil" or
| "gang", a number of celebrities and politicians, and a random
| assortment of IP (Star Trek related IP seems to work okay, but
| "lightsaber" does not and it's not clear if that's a Disney issue
| or a weapons issue, since "gun" and "pistol" don't work but
| "blaster," "cannon", and "knife" work fine.
| poniko wrote:
| Midjurney v5 is magic. Super impressed by the team behind and the
| pace they evolve the model. Now just open a API and let the money
| roll in.
| ftxbro wrote:
| From the twitter thread:
|
| "Same prompt as above but for Pikachu. Again, somehow Firefly
| does not fully get these famous characters. Maybe a training data
| copyright issue?"
|
| Comment in hacker news thread some days ago
| https://news.ycombinator.com/item?id=35247720 :
|
| "Adobe employee, not in Creative Cloud. [...] The team is going
| to great lengths to make this ethical and fair (try and generate
| a photo of a copyrighted character like Hello Kitty or Darth
| Vader)."
| the88doctor wrote:
| The thread author said Firefly was only trained on properly
| licensed photos which definitely supports the hypothesis that
| it sucks because it is doing things legally. Unfortunately that
| seems to make it suck even when the prompt does not reference
| any protected IP (e.g. the woman in red in a crowd example).
| Robotbeat wrote:
| Learning off of works of other artists is not illegal.
| rondini wrote:
| I think that the law is woefully unequipped to handle this
| situation, but there is definitely an ethical concern over
| training your models on copyrighted work. Adobe isn't going
| to scrape images off of social media to train their image
| generator and that will always put them behind companies
| that do. I can't say whether that's fair but I have very
| little faith that regulators will understand this issue
| enough to find a solution.
| jonny_eh wrote:
| We don't know that yet, we may find out soon.
|
| https://vox.com/recode/23580554/generative-ai-chatgpt-
| openai...
| Robotbeat wrote:
| Exactly, so insisting on it being illegal now is
| inaccurate. You can bet that companies which have access
| to massive IP stockpiles, like Adobe, will absolutely be
| pushing things in a direction so that anything you see or
| hear cannot be used for inspiration without a hefty fee
| which is at their sole discretion.
| _coveredInBees wrote:
| The correct interpretation is that relying on tools like
| MidJourney can be a liability for the exact reason that
| this can go any which way and the legality issue is
| rather gray. So the point still stands that many
| companies would rather have a more clear-cut tool like
| Adobe Firefly without having to worry about potential
| liability.
| sebzim4500 wrote:
| I doubt Adobe has a serious lobbying presence. It's
| Disney you've got to watch out for, but hopefully they
| are distracted arguing with Florida about whether gay
| people exist.
| ftxbro wrote:
| > I doubt Adobe has a serious lobbying
|
| It depends what you call "serious" but on their
| government relations page https://www.adobe.com/about-
| adobe/government-relations.html they say "Adobe discloses
| information on our federal and state lobbying activities.
| Our most recent federal lobbying disclosures are listed
| on the U.S. Senate lobbying disclosure database and U.S.
| House lobbying disclosure database."
|
| If you look at the forms on those databases you can see
| that Adobe has a "Policy, Government, and Ethical
| Innovation team in Washington, D.C." which I guess is
| what they call their lobbying crew.
|
| > It's Disney you've got to watch out for
|
| Probably most lobbying efforts would be considered 'not
| serious' if the standard is Disney!
| waboremo wrote:
| Yes, but leaning too hard in the other direction (that
| all these models do is "learn" therefore it's ok to
| duplicate) would negate a whole lot of protection artists
| (and in general individuals) get regarding their creative
| works in the digital space.
| blibble wrote:
| it's not learning anything
|
| if you turn the temperature value down to zero it will
| produce the same thing every time given the same input (in
| the same way zlib or jpeg will)
|
| it's a lossy compression function that is implemented using
| a neural net
|
| copyright isn't removed if I take someone's existing image
| and re-save it as a jpg
| RC_ITR wrote:
| Ok, so let's assume it's using a Visual Transformer
| architecture, since AFAICT, we don't know.
|
| What the model does is turn the picture into numbers in a
| clever way (embedding).
|
| Then, the model compares the relationships between all
| the embedded tokens (self-attention).
|
| Then the model does the fancy linear algebra stuff we've
| come to know and love (feed forward neural network).
|
| What you're basically getting at with your temperature
| comment is "rather than having it semi-randomly pick one
| of the top n options that the feed-forward network
| recommends for which pixel goes where, we can always have
| the model simply choose the top answer each time."
|
| That is _not_ "we can make it reproduce training images"
|
| The _point_ of embedding is to see multiple examples of
| similar images and generalize from them. The process in
| and of itself breaks the ability to re-create a photo,
| unless the training set is very skewed.
|
| If that's the argument of legal cases, they are likely to
| fail or win on ignorance.
| blibble wrote:
| > That is not "we can make it reproduce training images"
|
| didn't say that, all I said is it's a deterministic lossy
| compression function of the inputs (unless randomness is
| deliberately introduced)
|
| and as such it should be treated the same way as zlib or
| jpeg for copyright purposes
| pertymcpert wrote:
| Those are two orthogonal concepts. The temperature has
| nothing to do with whether it's a lossy compression
| function.
| RC_ITR wrote:
| I think you're really stretching the definition of
| deterministic here.
|
| For example, switching training order of images has a
| known impact on outputs, but I don't see the equivalent
| in traditional compression.
|
| I agree with you that it is computer code that runs on
| circuits (and is therefore deterministic in a very loose
| sense), but it is fundamentally incorrect to compare it
| to traditional image compression.
| f_devd wrote:
| > deterministic lossy compression function
|
| This would only be true if it didn't mix or 'forget' any
| of the training data for each prompt, and only then for
| the exact same inputs of the training data. That is a lot
| of qualifiers that aren't practically applicable.
|
| Slight side note: a deterministic lossy compression
| function should not always be treated the same as
| jpeg/zlib, as hash functions are technically also in that
| category.
| alxhill wrote:
| This is false and represents a poor understanding of how
| these models work - they do abstract concepts and no you
| can't trivially get training images out.
| blibble wrote:
| it's exactly the argument that the court cases against
| training on copyrighted works without permission are
| using
|
| if the courts agree and this is ruled to be infringement
| then we'll left with products with level of quality we
| see here
|
| meaning the technology will be an economic dead end
|
| here's to hoping
| [deleted]
| fazqq wrote:
| [flagged]
| jamilton wrote:
| It's interesting that the outputs still vaguely resemble the
| subjects of the prompts. It implies to me that there's still a
| little bit of the characters in the dataset, but not very much.
|
| If you look up "Pikachu" in Adobe Stock [1] there actually are
| a lot of results. They're mostly figurines, pokemon cards,
| quokka, and Machu Pichu. Very few direct illustrations. And of
| course there might be filtering used on the dataset before it
| was used to train the model, a lot of the images that have
| Pokemon imagery are marked "Editorial use only".
|
| 1:
| https://stock.adobe.com/search?filters%5Bcontent_type%3Aphot...
| reedciccio wrote:
| Trademark rather than copyright but yeah, a plausible
| explanation is that Adobe played it safe and avoided pissing
| off litigious companies.
| shagie wrote:
| Copyright is the proper form of IP for fictional characters.
|
| https://en.wikipedia.org/wiki/Copyright_protection_for_ficti.
| ..
| mftb wrote:
| Also their customers are you know... artists.
| sebzim4500 wrote:
| Long term, if Adobe is clearly worse than the competition
| as a result of being legally/ethically conservative they
| will lose users. They will get a lot of likes on twitter
| but companies will not pay for a firefly subscription if
| it's worse than Stable Diffusion.
| pmoriarty wrote:
| It's not like no artists use Midjourney.
|
| I'm an artist myself, absolutely love Midjourney, and have
| absolutely no problem with it being trained on any and all
| art -- just like human artists have been doing for
| millenia.
| thordenmark wrote:
| I played with Firefly for a few hours and was quite unimpressed
| with the results. It reminds me of Midjourney 2.0, which was kind
| of impressive a year ago. Adobe has a lot of catching up to do.
| ianbicking wrote:
| I suppose the trademarked characters is expected. But the app
| icon puts it below Dall-E. I'm trying some of these with Bing
| Image Creator (I guess that's Dall-E 3?) and it's also definitely
| outperforming Firefly.
|
| Here's the human figure (number 7):
| https://www.bing.com/images/create/editorial-style-photo2c-m...
|
| Pretty solid Pikachu:
| https://www.bing.com/images/create/a-photograph-capturing-pi...
|
| It had a hard time with the woman in red, but did better than
| Firefly: https://www.bing.com/images/create/modern-street-style-
| photo...
|
| Totally flubbed the app icon, surprisingly:
| https://www.bing.com/images/create/app-icon-design3a-ios2c-s...
|
| A little low on detail on the scifi city:
| https://www.bing.com/images/create/abstract-fractal-circular...
| dtagames wrote:
| I tried it over the weekend was very disappointed. It feels like
| they want to keep it broken so it that doesn't threaten their
| existing tools or userbase. More of a "Hey novice, make basic
| stuff" product.
|
| BTW, faces are horrible and unusable. Full stop.
|
| Which is sad! They shouldn't have released this until it
| integrated with your existing Adobe tools and your own art.
|
| Also... it "watermarks" your images with a huge ugly gradient in
| the corner, and it keeps track of all them with some metadata
| that's stored on a "verification" website, along with your name
| and tool's info. Just so you know.
| underwater wrote:
| Putting the same text prompts into each system seems like a
| flawed methodology. Instead they should have a general goal and
| tweak the prompt for each system.
|
| After all, directives like "Warm glow, elegance" or "skeuomorphic
| style" are hacks that people have learned to use to specifically
| improve MidJourney's results.
| neom wrote:
| I thought Firefly was pretty great, sure found it better than
| dall-e, I've not used MidJourney because I'm not a discord user
| (why is their tool discord only??) - The thing I don't like about
| this twitter thread is that it doesn't disclose the modes he
| used, Firefly has literal buttons you can push to tell it what to
| do (like make a photo) if he asked for a photo on non-photo mode,
| it's going to be more art less portraiture.
|
| The one thing I really liked from firefly was the text creation
| tool, it's really cool (although I have no clue what the
| applications are for people not working in gfx design)
| sebzim4500 wrote:
| Dall-e is far from the state of the art though
| anonylizard wrote:
| Those 'buttons' are just a single word in a prompt. Nothing
| fancy.
|
| Also, Dalle2 is from a year ago. Midjourney has undergone 3
| massive version upgrades since then. Midjourney v5 completely
| crushes the DALLE and firefly on every perceivable axis.
|
| MJ is on discord, because they need community moderation by
| forcing public generation. Its the most effective way to
| suppress deepfake generation given their model very dangerous
| if use unrestricted.
| og_kalu wrote:
| Better than the new dalle ? (the one free on bing image creator
| or the dalle-2 site if you're a beta tester)
| dtagames wrote:
| The original DALL-E straight up from the OpenAI website still
| gives the best results, in my opinion.
| KeplerBoy wrote:
| that one is miles behind MJ.
| dtagames wrote:
| Really? Ok, I'll investigate further! Thank you.
| yCombLinks wrote:
| Yes, many miles
| pmoriarty wrote:
| If you're in to AI image generation and haven't tried
| Midjourney you're seriously missing out. It's nothing short of
| incredible. Signing up for a Discord account to use it is
| definitely worth it.
| fishtoaster wrote:
| I, too, found Firefly's generations pretty underwhelming.
|
| My hope with it is that despite mediocre image generation, Adobe
| will lean into their strengths and build really great _tooling_
| around it. I imagine a really great user interface around
| generation, inpainting, controlnet, image2image, etc - a UI that
| a well-heeled company has put a lot of money into designing and
| testing. I 'd be ok with last year's generation quality in
| exchange for that.
|
| So far, though, Firefly is just a demo of a few basic, outdated
| generation tools. I don't see any value in it yet, but hopefully
| it's early days for Adobe.
| 0xDEF wrote:
| MidJourney gives by far the prettiest results of all the text-to-
| image models however it's obvious they have fine-tuned it to that
| generic ArtStation digital art look.
| pmoriarty wrote:
| The look you get depends a lot on the prompts. You can get
| widely different results from different prompts. If you think
| it only has one style you either haven't used it much or are
| haven't experimented much with your prompts.
| qq99 wrote:
| I think Firefly's main strength right now is the stable diffusion
| letter forms. I'm not aware of any other way to generate
| something as good as those.
| GaggiX wrote:
| Isn't it just ControlNet with depth? It is more of a gimmick
| than anything else.
| kfarr wrote:
| Stable diffusion SDXL beta has improved greatly generating
| text. Not perfect, but getting there...
| dtagames wrote:
| Except that it gives you a PNG, not a usable vector font. And,
| in reality, how many of these filled letterform designs are we
| going to need? I can see some utility in blog post header or
| something once in a while. But it's not a design style that
| would be commonly used and the output isn't great for anyone
| who already uses Adobe tools. Not modifiable later, etc.
| yumraj wrote:
| He should have tried to generate images of Xi, Firefly _may_ have
| done better.
|
| Referring to: https://news.ycombinator.com/item?id=35418999
|
| https://www.techdirt.com/2023/03/31/midjourney-ceo-says-poli...
| nico wrote:
| Are all these images just the first shot of both? Midjourney
| generates 4 versions on a first prompt, which one was chosen?
| What was the process and criteria?
|
| Without knowing that, it's hard to compare.
|
| It would be good to watch a side by side video of someone doing
| their creative process, using each of the tools, with the same
| goal.
|
| Then you can see the reality behind generating those images.
|
| How long it takes, how many tries, tactics used, etc.
___________________________________________________________________
(page generated 2023-04-03 23:02 UTC)