[HN Gopher] A preliminary analysis of DALL-E 2 (Marcus, Davis, A...
___________________________________________________________________
A preliminary analysis of DALL-E 2 (Marcus, Davis, Aaronson)
Author : gibsonf1
Score : 64 points
Date : 2022-05-02 14:00 UTC (9 hours ago)
(HTM) web link (arxiv.org)
(TXT) w3m dump (arxiv.org)
| drcode wrote:
| Yes, DALL-E2 clearly gets confused when dealing with multiple
| objects, where each object is given specific attributes.
|
| However, it does answer SOME of these types of prompts correctly:
| It often correctly handles the simplest case of two objects with
| one specific property each.
|
| My prediction is that since it can handle SOME of these cases
| already, it means this problem has shown itself to be tractable,
| and future ML researchers will be able to chip away at the
| deficiencies with additional effort.
|
| I'm confident the next versions of these AI systems will handle
| the prompts given in this paper with ease.
| jandrese wrote:
| From what I've read elsewhere DALL-E2 can't handle overly
| long/complex requests. Pretty much everybody who has tried it
| has discovered the same thing. Keep the requests simple.
| ravi-delia wrote:
| I bet a lot of the issue is just the CLIP encoding. It's doing
| a ton of work getting text prompts into vector form, and it's
| incredible it does so well as is, but past a point you're
| squeezing blood from a stone. It's just not a full language
| model! Bump up the size, or figure something out about the
| space it's working in, I bet you'd see huge improvements.
| joering2 wrote:
| > Caption: a red ball on top of a blue pyramid with the pyramid
| behind a car that is above a toaster.
|
| Have you ever heard humans talking like that? I bet you each
| person would also draw it differently. I consider myself human
| and yet I don't grasp a concept of a "car that is above toaster".
| What do they mean? Like hovering? Or just standing on the top?
| Cars aren't UFOs, they don't hover.
|
| The whole paper seems like they tried to break the system with
| tasks even humans would have problems grasping. What's the
| point??
| theptip wrote:
| It's like an IQ test. I think it's a good way to probe the
| "cognition" (if you will) of the model.
|
| Sure, humans would probably differ in how they draw that
| picture. IQ 150 humans would almost all get the positional
| relationships correct though, even if the precise
| interpretation of "above" might differ. The fact that none of
| the images were correct under any interpretation of the
| relationships shows something meaningful, namely that this
| model can't understand long chains of relationships.
| blueyes wrote:
| This is not very substantial work. They used the API, gave it
| more challenging questions on 14 prompts, and judged whether it
| met expectations. I would have expected something much more
| intelligent from Aaronson, although this kind of light-weight
| analysis is par for the course with Marcus, who has been fighting
| his rearguard action against large parametric models for many
| years, and is not above motivated reasoning. It's like one of
| those open letters that people get roped into, only hosted on
| arXiv.
| karpierz wrote:
| I wouldn't describe these as challenging, the prompts are
| fairly simple if you're capable of understanding pretty basic
| concepts.
|
| > A pear cut into seven pieces arranged in a ring.
|
| > A couple in formal evening wear going home get caught in a
| heavy downpour with no umbrellas
|
| The work successfully demonstrates that the API lacks basic
| conceptual understanding by example.
| yboris wrote:
| I don't see much of a downside to a light, quick exploration of
| a new tool and a writeup shared publicly. While this could have
| been a (very long) blog post, it seems like arXiv is a good
| place for this "report of findings".
| freediver wrote:
| In this case the only thing "light" is the paper itself and
| manner in which the authors scrutinized the work.
|
| The prompts themselves were very complex and thought through,
| probably a result of a lot of cherrypicking to find
| weaknesses in the model.
|
| Still, it is impressive how DALL-E dealt with them even in
| cases where it got it wrong.
| standardly wrote:
| I agree. The title is "A very preliminary analysis of DALL-E
| 2". There is really nothing offensive about this publication.
| fullshark wrote:
| It's nice to see some examples that haven't been filtered by
| the creators as being the best ones, to show how capable it
| really is. This paper (at least to me) makes me even more
| impressed by DALL-E 2, it hasn't parsed the input exactly but
| it's still incredibly impressive imo.
| syspec wrote:
| Sorry for the n00b question, but how do I see the actual paper?
| gibsonf1 wrote:
| https://arxiv.org/pdf/2204.13807
| sophiebits wrote:
| Press the "Download PDF" button.
| MarcoZavala wrote:
| Strilanc wrote:
| I really liked this paper. On the one hand, all they did was come
| up with some prompts and show the results. On the other hand, it
| does a really good job of being clear about exactly what they
| queried and why and how much selection they are applying (eg.
| they mention accidentally discarding a result once; this kind of
| thing was always very unclear in people's twitter posts of DALLE2
| images). Also the paper highlights things that the model is
| clearly still struggling with, which researchers can then focus
| on improving.
| jmmcd wrote:
| But I expected more clever reasoning and insight into what
| specific properties of prompts cause failures, in particular by
| trying variants of prompts. And more discussion of the weird
| style issue that causes "old man talking to his parents" to
| always be in a specific web-comic style.
| astrange wrote:
| They look a lot like the art from the copyright-free Japanese
| clip art site irasutoya.
|
| https://latelyjapanese.com/culture/20201214/do-you-know-
| abou...
| londons_explore wrote:
| Are these people just trying to pad their citation count?
|
| It really looks like nothing more than an hour's work to write
| this 'paper'.
| orbifold wrote:
| Some academics are experts in also being science influencers.
| In those cases I actually know their non-scientific work better
| than their actual scientific contributions. That is I know
| Scott Aaronson from Shtetl optimised not from any particularly
| memorable scientific accomplishment, which he might well have.
| jmmcd wrote:
| He's surely a leading light in complexity, but since it's not
| my field I also know him more from the blog.
| shannifin wrote:
| Because many of the examples seem to show that the system easily
| misunderstands certain grammatical implications (prepositional
| relations, etc), I wonder if it would benefit from of different
| sort of input system in which the relationships of the objects in
| the desired image are more concretely defined, something more
| like a tree graph perhaps? Or perhaps this would be very
| difficult to implement?
|
| At the very least, seems it would be useful for users to edit a
| given output so that it can be continually tweaked, keeping what
| they like about an image (e.g. object layout, proportions, color
| scheme) while editing other aspects.
| soraki_soladead wrote:
| Gary Marcus here again to throw cold water on anything deep
| learning. The negative tone when not succeeding in tasks it
| wasn't designed or advertised for continues to be disappointing.
|
| > To the extent that the goal is to develop artificial
| intelligence that can be trusted in safety-critical applications
| (Marcus & Davis, 2019), a much higher standard must be applied.
|
| I have seen no mention of that being the intended use of DALL-E
| (2). In fact, the most common use case I've seen described is in
| replacing Fiverr-type tasks: quick graphic design.
|
| That said, the results are actually encouraging given that it
| _wasn't_ designed to succeed here:
|
| > Nevertheless, for 5 out of the 14 prompts, at least one of the
| ten images fully satisfied our requests.
|
| Some of the authors' interpretations could be argued against, as
| well. For instance, in example 10, "An old man is talking to his
| parents":
|
| > In none of these images did DALL-E successfully infer that
| image should show an old man with two even older people
|
| Several of the images appear to show exactly that? How is the
| author judging "even older"?
| ravi-delia wrote:
| As annoying as the constant goalpost moving is in AI, it's also
| true that how these models fail is much more interesting than
| how they succeed. That's where all the information is. Dalle-2
| is better at some things than others. Why? What's a limitation
| of the model, and what's an accident of architecture? I'm
| reminded of GPT-3's difficulty rhyming and doing math, which
| seems to be a result of the tokens it was trained on, compared
| with trailing off and inconsistency which may be more
| foundational.
| visarga wrote:
| > Why? What's a limitation of the model, and what's an
| accident of architecture?
|
| The limitation is caused by the CLIP model they used to
| encode text and images. It's a separate model only generating
| an embedding, it's not using attention and pairwise
| interactions on the whole sequence. This causes Dalle2 to be
| bad at handling multiple objects with multiple attributes.
| There is no reason the complete prompt could not be related
| to the generated image instead of an embed, thus correctly
| stacking the coloured cubes and assigning the right age to
| each person mentioned in the prompt.
| soraki_soladead wrote:
| I completely agree. However, here are some things a more
| useful study would cover:
|
| - Describe the task creation process, why they are
| appropriate for measuring X, etc. Preferably drawing from
| similar studies in people instead of ad hoc. Lots of work has
| already gone into measuring these things. They may have to be
| modified for DALL-E but it would be a better starting point.
|
| - Show several variations of the same task and outputs. For
| example #11, maybe the issue is that DALL-E has a poor
| understanding of milk sizes as expressed but not size
| relationships in general. Example #13 with pizza sizes is
| ambiguous by the authors own interpretation yet they deem it
| a failure. It would be trivial to construct dozens of similar
| examples to give a more holistic understanding.
|
| - Narrow the scope of the prompts if you want to see how the
| model understands a particular relationship. Many of the
| tasks include multiple ancillary statements.
|
| - Discuss prompt engineering in more depth. We already know
| that these models are sensitive to the formulation of the
| prompt. What did the authors try / not try?
|
| - Replace the authors' individual opinions of the outputs
| with crowdsourced opinions from mturk or even Twitter. As I
| noted above, example #10 is not as clear cut as the authors
| suggest.
|
| - Measure how well people do at the same task as a baseline:
| draw something that aligns with a given text prompt and
| compare to DALL-E, maybe with a crowdsourced opinion of which
| is a more accurate interpretation. Even if they're just stick
| figures this would be interesting to see.
|
| As-is, this article doesn't really add anything substatial
| about the model's capabilities to the conversation.
| garymarcus wrote:
| what goalposts changed? i pointed out same limits in 2001,
| 2012, 2018, 2019, 2020.
|
| please actually read my work and please don't make stuff up.
| rel2thr wrote:
| I didn't get a negative tone from the paper, it seemed mostly
| positive to me , just calling out a handful of areas ( like
| relative positioning ) where Dall-e fails .
| chrisco255 wrote:
| I'd be curious to compare the results of Dall-E 2's output vs
| a group of human artists each individually given the exact
| same text prompt (with no follow up clarification allowed)
| and asked to produce 3-5 drafts.
| joshcryer wrote:
| For me, I read the introduction and how they discuss how
| impressive DALL-E 2 is, and then they show very simple
| mistakes it makes and I want to be protective of DALL-E 2. I
| want to say "you did a good job!"
|
| That the paper provides "a clearer picture of what remains to
| be done" is very hard to accept, as all it does is show edge
| cases which are subjective at best. If anything the picture
| is less clear as they don't even try to form a hypothesis why
| DALL-E makes mistakes like these. One thing in particular I
| have noticed is that DALL-E has trouble producing action
| images. It may be because it has no sense of temporality and
| in those cases it could serve to run the parameters a bit
| longer using the same scene. But I am not an AI scientist so
| what do I know.
| vletal wrote:
| > ... it is reasonable to question whether DALL-E 2
| constitutes progress toward solving the deep challenges of
| commonsense reasoning, comprehension, reliability, and so
| forth that would be needed for a truly general-purpose AI ...
|
| I like how they pose this question as a bait to the abstract
| yet they do not even attempt to answer it. Instead they focus
| on general shortcomings of the model.
|
| Moreover, there is no proper conclusion which would discuss
| the findings.
|
| Given how outspoken Gary Marcus is on Twitter - criticising
| current advances in DL I would expected him to do a much
| better job publishing a document about it.
| soraki_soladead wrote:
| Many of the perceived failures of DALL-E 2 are subjective and
| ambiguous (by the author's own interpretation in some cases!)
| and speak to the author's negative biases which are well
| known outside of this publication.
|
| To be clear, I'm not defending DALL-E 2. I'm criticizing a
| poorly written paper that was published to Arxiv to lend
| further credibility for a Twitter audience to substantiate a
| claim that the DALL-E 2 authors have not made but that Gary
| Marcus has a vested interest in perpetuating:
|
| > How much does DALL-E have to do with AGI? Maybe not so
| much, after all... A lesson in caveat emptor: -
| https://twitter.com/GaryMarcus/status/1521120022298464256
|
| This should have been a blog post or Twitter thread like the
| dozen or so other experimentations people have done with the
| system.
| joshcryer wrote:
| Yikes, that Twitter feed: "Thoughts and prayers for the
| deep learning fanboys," as if ones admiration for DALL-E
| 2's achievement is something to be mocked... I feel like
| this sentiment is the way we're going to get I Have no
| Mouth, and I Must Scream. There's just something in there
| about negative AI minimalists or AI alarmists (same coin
| different side).
| pixl97 wrote:
| Heh, so many of these people's reaction to 'we have not
| created AGI' seems to be mocking and pessimism that we ever
| will.
|
| My response to 'we have not created AGI' is
|
| "Thank goodness".
|
| I don't think we're ready for that yet.
| jmmcd wrote:
| Marcus is only one of three. From what we know if Aaronson,
| I doubt that this was a Marcus-driven paper with the others
| just along for the ride. But I agree some of the intro has
| a Marcus ring to it.
| garymarcus wrote:
| How quickly they forget. 45 minutes after the DALL-E n release,
| Sam Altmann tweeted "AGI is gonna get wild"; Greg Brockman
| posted multiple tweets around the idea of AGI and walls broken.
| That invites a critical analysis.
|
| Ad hominem against me won't remedy the limitations that we and
| others have observed.
| hervature wrote:
| I think you are being disingenuous by omitting the preceding
| claim.
|
| > Whether results of this kind should be considered as
| successes for the program - what is the proper measure to use
| in evaluating success - depends on the intended use of the
| program. If the goal is to generate candidate images that a
| graphic artist will choose from, or choose from and edit, then
| the system can reasonably be measured in terms of the quality
| of the best result out of ten or out of one hundred.
|
| They basically admit your Fiverr use case is valid. But say,
| that it should not be used "in safety-critical applications"
| which is neither a grand claim nor controversial. It is
| probably the most blase claim because, as you point out, no one
| is expecting this to be used in safety-critical applications.
| From an economics point of view, the Fiverr use case seems
| pretty strained to me. If you've ever watched street art, some
| dazzling things can be done in under 10 minutes. Unless the
| DALL-E gets it correct on the first shot +99% of the time,
| someone sifting through images is probably just as costly as
| paying for Fiverr. What this paper elucidates to me is that
| even historical figures are off-limits which, in my
| expectation, is a non-trivial use case.
| axutio wrote:
| > someone sifting through images is probably just as costly
| as paying for Fiverr
|
| The difference being that anyone can sift through images and
| identify good ones, whereas few people can create good
| images. The amount of time it takes to complete the task may
| be the same, but the number of people who can do it greatly
| increases.
| soraki_soladead wrote:
| If someone says, "Cars may be great at transportation to and
| from work but we should be cautious to use them as a
| counterbalance for a trebuchet." I don't think it's
| disingenuous to take issue with the second part of the claim
| and to not give the author credit for the first part of the
| claim.
| hervature wrote:
| Right, unless the creator of the car's mission is to use it
| as a counterbalance for a trebuchet. From [1]:
|
| OpenAI is an AI research and deployment company. Our
| mission is to ensure that artificial general intelligence
| benefits all of humanity.
|
| [1] - https://openai.com/about/
| ridruejo wrote:
| The paper complains that some of the answers are not perfect but
| half the humans I know would have gotten them wrong... (thinking
| about the octopus one). I just find this technology fascinating.
| Imagine once you can extend this to video ("Star wars but the
| Jedis are the bad guys", "breaking bad but an 80s sitcom" ...
| Netflix should be investing in this...)
| vintermann wrote:
| This is a very serious paper about a very serious topic, AI and
| AGI and how the...
|
| Who am I fooling? They're clearly just messing around and having
| fun. I don't begrudge them, though.
| minimaxir wrote:
| This paper notes that average output can be flawed...which is the
| case with _all_ AI-generated content and why it 's important to
| note the importance of cherry picking/selection bias whenever
| posting it publicly.
|
| Fortunately, AI Image generation has helped visually communicate
| how AI content generation isn't quite sci-fi magic where you
| always get what you want with zero ambiguity. Yet.
| theptip wrote:
| It's an interesting point - cherry-picking muddies the water in
| an academic context. However it's how these models will
| actually be used in the real world (with a human "editor" in
| the loop) so in some sense you care more about "best of N"
| performance. Or maybe even "best performance from someone that
| knows how to AI-whisper" since these are the new content
| creators that will produce the best output with these models
| (or their descendants).
|
| This is really a case where the real-world market will provide
| an assessment regardless of the sort of academic assessment of
| quality in the OP.
| AbrahamParangi wrote:
| To quote the generally incredible Scott Aaronson with a somewhat
| more optimistic perspective:
|
| _Judged against where AI was 20-25 years ago, when I was a
| student, a dog is now holding meaningful conversations in
| English. And people are complaining that the dog isn't a very
| eloquent orator, that it often makes grammatical errors and has
| to start again, that it took heroic effort to train it, and that
| it's unclear how much the dog really understands._
| benlivengood wrote:
| The original paper examines some likely causes for failure to
| encode relational details between subjects, e.g. the corgi with a
| red hat and green necktie or vice versa chosen seemingly at
| random. The model seems to learn about contrasting subjects but
| not track particularly which attributes belong to which subjects.
| This is, as far as I can tell from the original paper, inherent
| in the training and would require different data sets or more
| parameters to learn a greater number of subject-attribute
| associations in each image.
|
| https://arxiv.org/abs/2204.06125
| pjs_ wrote:
| Honestly try getting a bunch of PhDs in computer science to solve
| the same task and I think you will get dramatically worse
| results, not only artistically but also in terms of
| comprehension...
|
| The toaster/pyramid/ball one makes no sense to me - "with the
| pyramid behind a car that is above a toaster"?
| dukeofdoom wrote:
| How do you try this out? Is there a free alternative you can try?
| I would like to make some backgrounds for a platformer game.
| dukeofdoom wrote:
| How do you try this out? Is there a free alternative you can try?
| I would like to make some backgrounds for a platformer game.
| Imnimo wrote:
| It would be an interesting experiment to see where along the path
| some of these details are lost. For example, in the experiment
| where they ask for a pear cut into 7 slices, and Dall-E often
| gives the wrong number. First check if you can train a classifier
| on CLIP embeddings of the caption to recover the specified
| number. If so, check if you can do the same for the image
| embedding the prior produces. Then check if you can do it for
| unCLIP's internal feature maps, and so on. Similar experiments
| could be conducted for other lost attributes like object
| relationships (on top, behind, etc.) or specific identities
| (Lincoln, Washington, etc.) .
| visarga wrote:
| Very good points, but I would start with the obvious one - the
| contrastive objective of CLIP forces the prompt representation
| to a single embedding vector. So it acts as a bottleneck,
| limiting the number of object-attribute pairs it can
| simultaneously encode. And of course the solution is to use an
| autoregressive setup like Dalle-1, but then they can't have the
| same diffusion model decoder for image generation.
|
| I bet they consciously prioritised the artistic applications
| over exact semantics. The CLIP embedding space has nice
| properties, it's tempting to use it. From my experience
| semantic similarity based on embedding dot products is much
| easier to do than exact semantic matching. Three is similar to
| four and red similar to blue in embedding space.
| andybak wrote:
| Can someone remind of the link that turns arxiv papers into
| vaguely readable HTML?
| jwilk wrote:
| https://www.arxiv-vanity.com/ but... "This paper doesn't have
| LaTeX source code, so it can't be rendered as a web page."
| tonyg wrote:
| https://ar5iv.labs.arxiv.org/
| Q6T46nT668w6i3m wrote:
| The criticism that autoregressive language models with billions
| of parameters don't acquire basic logics (e.g., propositional,
| quantificational) is valid and, in my opinion, extremely
| interesting. Unfortunately, the subjectivity of their evaluation
| undermines that critique. The subjectivity is especially weird
| since these characteristics can be objectively evaluated (e.g.,
| finding contradictions).
___________________________________________________________________
(page generated 2022-05-02 23:01 UTC)