[HN Gopher] Implementation of Imagen, Google's text-to-image neu...
       ___________________________________________________________________
        
       Implementation of Imagen, Google's text-to-image neural network, in
       PyTorch
        
       Author : CesarERM
       Score  : 343 points
       Date   : 2022-05-26 03:05 UTC (19 hours ago)
        
 (HTM) web link (github.com)
 (TXT) w3m dump (github.com)
        
       | TekMol wrote:
       | What is the reason Google published their research details about
       | Imagen?
       | 
       | Why don't they just keep their findings to themselfes and build
       | products on top of them?
       | 
       | Public companies can't do stuff just for the fun of it, right? So
       | there must be some commercial reasoning behind it?
        
         | toxik wrote:
         | It's simply flag planting. If they don't do it, some other
         | player will.
        
         | mzs wrote:
         | You realize this doesn't include the trained data, right?
        
         | pyinstallwoes wrote:
         | Absolute power, corrupts absolutely. Perhaps it's a game
         | theoretic approach to leveling the playing field.
        
         | yaroslavvb wrote:
         | Researchers like to talk about and show off their work outside
         | the company. If you don't let them, they get unhappy and leave.
        
         | minimaxir wrote:
         | Publishing high impact research gives credibility to the ML
         | teams, which helps recruiting and prestige.
         | 
         | It's less cynical, more incentive alignment.
        
           | hackernewds wrote:
           | Also good for society. Less evil, more nice people. Respect
           | Google for these traits.
        
             | benoketamad wrote:
             | Oh, they don't even want to create/release a usable image
             | search... Respect, of course.
        
               | Filligree wrote:
               | Didn't they run into legal trouble with that?
        
               | Geonode wrote:
               | They could have won that fight, but they don't care
               | enough. I use Yandex for images now, it's very good.
        
               | lostmsu wrote:
               | Care to link?
        
         | 29athrowaway wrote:
         | Because the competitive advantage is also in the training
         | datasets and ML infrastructure.
        
         | nl wrote:
         | > Public companies can't do stuff just for the fun of it,
         | right? So there must be some commercial reasoning behind it?
         | 
         | Yes they can actually.
         | 
         | If you are a shareholder you can either sue (unlikely to
         | succeed) or vote against the board. That's pretty much the only
         | recourse.
        
           | pmoriarty wrote:
           | _" If you are a shareholder you can either sue (unlikely to
           | succeed) or vote against the board. That's pretty much the
           | only recourse."_
           | 
           | Or you could sell or just threaten to sell your shares.
           | 
           | Buying more of the company's shares to take it over is
           | another option.
           | 
           | Good luck doing that with Google.
        
         | DeathArrow wrote:
         | They did it out of goodness of their hearts. :)
        
         | blueblob wrote:
         | I think it is because the proprietary part for them is the
         | data, not the particular algorithm. They benefit more from
         | other people making advances on their technology because they
         | have the data to get more benefit than anyone else. If they
         | kept it to themselves, they would get no "free" advancement. So
         | they trade-off the secret of the technique in the hope that
         | others will advance the technique, making their data more
         | valuable.
        
         | dotnet00 wrote:
         | I think there are three main reasons why:
         | 
         | - being open is kind of just how things in ML generally work
         | right now, it's in stark contrast to things like chemistry or
         | physics where paywalls are pretty common
         | 
         | - it's a matter of clout, ML is moving ridiculously quickly,
         | with work from just 5 years ago being considered outdated in
         | terms of capability, if you don't publish, someone else will
         | and they'll get the credit. This likely also matters for the
         | researchers since they get credit too. In a sense this is just
         | publish or perish culture from academia.
         | 
         | - it's also somewhat about hiring, which is related to the
         | clout. By putting out this kind of research, they're attracting
         | talented engineers to consider working for them. This of course
         | is pretty relevant to the rest of their business, especially
         | given how heavily Google leans on AI to handle moderation.
        
         | nmfisher wrote:
         | Two reasons: 1) Even though it's all technically very
         | impressive, so far there's not a huge amount of
         | commercialization potential here. OpenAI is charging for its
         | GPT-3 model but its revenue is probably negligible next to the
         | hardware costs (sunk + ongoing) to train it in the first place,
         | let alone the researcher salaries they're paying 2) Most of the
         | stunning examples are cherry-picked. These things fail much
         | more often than they are willing to admit, and they're probably
         | assuming (correctly) that not enough people are willing to pay
         | for something that only-sorta-kinda-works 1/3 of the time, when
         | you're holding it the right way.
        
           | urthor wrote:
           | It's also because these companies hire away staff all the
           | time.
           | 
           | Rule 1 of working in this field, recruit a mid level member
           | of the other company's research group biannually to get the
           | latest gossip.
           | 
           | It's impossible to keep a 1 page or shorter "algorithm"
           | secret, when the creators are geniuses and they hop jobs
           | every year or so.
           | 
           | Fellas with more IQ than games in a baseball season, they
           | just don't forget.
        
           | adamhowell wrote:
           | I'm currently working fulltime on AI-powered design suite
           | Accomplice (https://accomplice.ai) and if you ask me on a
           | good day I would tell you I do think there's already huge
           | commercial potential. On a bad day, though ;)
           | 
           | My current approach is a "model marketplace"
           | (https://accomplice.ai/models) where the most popular open
           | source text-to-image models (VQGAN+CLIP, Disco Diffusion,
           | DALL-E Mega coming soon...), sit alongside the most popular
           | open source style transfer models, and then finally I have
           | the ability for a user to finetune their own models using a
           | simple drag-and-drop tool (https://accomplice.ai/no-code-
           | model-training).
           | 
           | Using this approach a user has enough models to try or train
           | that they can have a higher hit rate. For example, Accomplice
           | currently has finetuned models for photo realistic people (ht
           | tps://accomplice.ai/models/f58bfa91-bb18-406f-a0e1-db00fcf...
           | ), watercolor backgrounds (https://accomplice.ai/models/91b8a
           | 080-faca-4ff4-8b11-64b0789...), etc...
           | 
           | So theoretically if there were a searchable marketplace of
           | 100s of different finetuned models people could choose from,
           | they would use it much like an iStockPhoto and be able to
           | create the kind of images they want instead of just
           | downloading them.
           | 
           | But it's of course a constant work in progress. Slowly
           | growing though and lots of promising stuff ahead!
        
             | nautilus12 wrote:
             | Why would people do this when aspiring artists are
             | practically giving their real photos and paintings for free
             | on places like deviant art? Why further commoditize
             | something that's already been commoditize to practically
             | free?
        
               | adamhowell wrote:
               | Well, I'm focusing on business use cases - stock images
               | and how Accomplice could be useful for marketing and
               | content creation.
               | 
               | i.e. The ability to easily take your logo and stylize it:
               | https://accomplice.ai/@adam/iterations/2bcc90ad-3237-486a
               | -8d...
               | 
               | Create a photorealistic avatar whenever you need it: http
               | s://accomplice.ai/@adam/iterations/988b7d54-dc39-43b1-b5.
               | ..
               | 
               | Easily remove the background of a photo:
               | https://accomplice.ai/models/97746c4b-c6f0-49cb-
               | ae1b-859716b...
               | 
               | Upscale a photo: https://accomplice.ai/models/bd4619ee-82
               | 02-4cf0-a04e-291820f...
               | 
               | Etc etc. AI can make all this stuff easier. And you have
               | a sense of ownership over what you create. All in one
               | place where you can collaborate on all of it with your
               | team. I feel like that's valuable. It's certainly a tool
               | I've always wanted.
               | 
               | But, also, as a bit of an aside - if the Googles and
               | OpenAIs of the world are just going to bite every
               | artist's style anyway with a mostly black box service and
               | training set... it feels like the option for an artist to
               | train/finetune their own model, promote it and possibly
               | make money off of that is worth trying.
        
               | pmoriarty wrote:
               | I'm an artist, and I'd absolutely love to use something
               | like this to inspire me or to give me something to
               | continue working with on my own.
               | 
               | In one sense it's kind of like a much "smarter" photoshop
               | filter, where it can make your own art/photos look more
               | like what you want (ex: Van Gogh, Dali, Picasso, or
               | combinations of those, or something completely
               | weird/new/different).
               | 
               | You could also train the models on your own work and have
               | it generate art in your own style that could inspire you
               | or could be useful to you either as a base to work from
               | or that you could take interesting elements from to
               | create new art.
               | 
               | Similar things can be done in music, by the way, and that
               | would be really useful to musicians too.
               | 
               | Poets could use something like this to create poetry,
               | novel writers to write novels, etc..
               | 
               | This is really an improvement on the collaboration
               | potential between humans and computers -- which is
               | probably why it's called "Accomplice".
        
             | nmfisher wrote:
             | I meant commercialization potential for companies like
             | Google, where anything less than a hundred million is
             | probably a failure :) Hopefully for the non-Googles of the
             | world (i.e. you), there's a good pathway forward!
        
             | 1024core wrote:
             | Your headshots model's outputs look creepy. The eyes are
             | off; e.g. the young african girl. You may want to tune your
             | loss functions to fix that.
        
             | TekMol wrote:
             | I wanted to try your site, but after clicking on the link
             | in the email, it tries to send me through some
             | redirectingat.com link which is blocked by my ad blocker.
             | 
             | What does that link do?
        
               | adamhowell wrote:
               | Update: Ah, found the "Tracking" setting in Sendgrid that
               | I thought I had already turned off. Off for sure now.
               | Thanks again for the heads up!
               | 
               | The confirmation link should only be going to
               | accomplice.ai unless Sendgrid is doing some link tracking
               | that I've just forgotten about. Could you forward that
               | email to adam at accomplice dot ai if you get a chance.
               | Thanks for letting me know!
        
               | jermaustin1 wrote:
               | That will get turned back on automatically in an
               | "update".
               | 
               | Same thing happened to me multiple times across multiple
               | platforms: SendGrid, Mandrill, MailJet, MailGun. I always
               | turn off the tracking (enabled by default on all of
               | them), but magically its back on a few weeks/months
               | later. I've given up finding a solution and just revisit
               | my settings every few months to check on it.
        
             | gbasin wrote:
             | Cool project! Will follow along
        
           | gfodor wrote:
           | Seems exactly false. DALL-E 2 seems to end much of the
           | illustrator industry and if the endless array of Twitter
           | posts from early adopters are any indication it works great.
        
         | curiousgal wrote:
         | What would the product be? My experience is that most ML papers
         | are very brittle, for every cool result/example you see there's
         | a plethora of nonsense spit by the models.
        
         | frozenport wrote:
         | 1. This finding is hard to monetize, especially with ROI that
         | Google typically does (for example a an app that makes $500 a
         | month isn't worth it)
         | 
         | 2. Deploying models in a cost effective way is hard
         | 
         | 3. Lessons learned from building this model can indeed be
         | monetized and many of them may be kept secret.
        
         | dekhn wrote:
         | Google started publishing for several reasons but the primary
         | one was recruitment (showing off was a secondary goal). The
         | mapreduce, GFS, and bigtable papers played an important role in
         | attracting an early generation of distributed computing/high
         | performance computing people from around the valley and
         | CMU/MIT, who helped build the second really successful versions
         | of the web search engine (retrieval and ranking), ads serving
         | (the auction, the logs joining pipeline), etc.
         | 
         | The other reason is that the leaders at Google at the time
         | believed that we would achieve the singularity faster if Jeff
         | Dean periodically sent ideas back 10 years in time to Doug
         | Cutting.
        
         | quickthrower2 wrote:
         | It could be classed as a way to attract talent. It is basically
         | then a very expensive fusball table.
        
       | jonathankoren wrote:
       | Of course the implementation isn't the issue. It's the training
       | data and the compute machines.
       | 
       | Open source is pretty meaningless here
        
         | emadm wrote:
         | Images actually used our open LAION 400m dataset plus 400m of
         | their own.
         | 
         | On compute there is more than enough compute available to open
         | source now via LAION and Eleuther AI to train these models,
         | will just a bit of time.
        
         | jfoster wrote:
         | In fact, this kind of reverses things, doesn't it?
         | 
         | Open source is built on the assumption that you can do more
         | with source code than with binaries. In the case of AI models,
         | the computed weights of models are what's valuable, and the
         | source code used to achieve them is less useful.
        
           | aero-glide2 wrote:
           | How much would it cost in training to match dall-e 2?
        
             | abrichr wrote:
             | https://twitter.com/alexjc/status/1347458546636619778?s=21&
             | t...
             | 
             | > The blog post says 256 GPUs for 2 weeks, so:
             | 
             | > DALL*E would cost $131,604 to train on AWS, assuming a
             | p3.16x-large at market rates. Could be as low as $40k if
             | you already paid for reserved instances.
        
       | dang wrote:
       | Recent and related:
       | 
       |  _DALL-E 2 open source implementation_ -
       | https://news.ycombinator.com/item?id=31228710 - May 2022 (152
       | comments)
       | 
       | Also:
       | 
       |  _X-Transformers: A fully-featured transformer with experimental
       | features_ - https://news.ycombinator.com/item?id=27089208 - May
       | 2021 (37 comments)
       | 
       |  _Text to Image Generation_ -
       | https://news.ycombinator.com/item?id=26615791 - March 2021 (88
       | comments)
        
       | tomatowurst wrote:
       | What's it going to take to train this on porn? This is something
       | that can be crowdfunded.
       | 
       | Look deepnude is a thing and somebody is making money off it:
       | https://app.deepnude.cc/upload
        
         | chirau wrote:
         | Lol. Not the same thing obviously but your question just
         | reminded me of @robotpornaddict, a neural net that watches porn
         | and tries to describe what it sees.
         | 
         | https://twitter.com/robotpornaddict
        
         | hackernewds wrote:
         | They specifically mention in the Images documentation that one
         | reason they haven't publicized this yet is due to porn or deep
         | fakes risk. Assuming this works very well already.
        
         | 29athrowaway wrote:
         | How can this be the most upvoted comment?
         | 
         | What you describe is already illegal on many jurisdictions.
        
           | lajamerr wrote:
           | Which part is illegal?
           | 
           | Are you talking about porn? Or using potential copyright
           | material in an infringing manner? Or perhaps the possibility
           | of "deepfake" porn?
        
             | 29athrowaway wrote:
             | In some jurisdictions, deepfakes of any kind are treated as
             | a form of defamation. In others, adult deepfakes
             | specifically are illegal.
        
               | pmoriarty wrote:
               | Which jurisdictions are those?
        
               | tomatowurst wrote:
               | Korea. Hardcore pornography is also illegal there.
               | 
               | Keep in mind this is a country where if you leave a bad
               | review after you get scammed by someone with evidence, it
               | is defamation. So not quite leadership the world needs in
               | this industry.
               | 
               | Really sad to see ppl on HN flagging all of my comments
               | on this thread. I mean it's not like you can't find
               | celebrity deepfakes including Kpop.
               | 
               | The cat is out of the bag and its only going to get
               | better and faster from here whether some
               | cultures/jurisdictions take offense or not.
        
         | DeathArrow wrote:
         | And what is going to take to generate videos, not pictures?
        
           | a2800276 wrote:
           | Maybe if you study hard enough, one day you'll have what it
           | takes to make a real girl.
           | 
           | https://youtu.be/9qd04u2Yj44
        
           | tomatowurst wrote:
           | To run we must learn to walk first. To walk we must learn to
           | be erect. To be erect we must master crawling.
           | 
           | I don't think the leap is too crazy if we are talking short
           | moving pictures without sound. However, when sound gets
           | involved, this is where it would become very tricky.
        
             | pmoriarty wrote:
             | _" when sound gets involved, this is where it would become
             | very tricky"_
             | 
             | I don't think a neural net would have much trouble
             | generating moans in sync to the motion.
             | 
             | Why pay humans for all that fake moaning when an AI could
             | do it?
        
               | tomatowurst wrote:
               | actually you are right, sound generation here wouldn't be
               | as hard as I originally thought.
        
             | comex wrote:
        
         | porn_rec_throw wrote:
        
           | [deleted]
        
         | londons_explore wrote:
         | This tech is actually pretty bad at human faces - it all
         | appears unnatural and distorted. You get things like the
         | occasional extra earlobe...
        
           | Al-Khwarizmi wrote:
           | Human faces were excluded on purpose from the DALL-E 2
           | training set in order to prevent misuse. I suppose the same
           | will be the case here (or at least, in public-facing versions
           | of the models).
           | 
           | Given how it renders dog faces, I don't see why it wouldn't
           | be good at human faces too, if trained for it.
        
           | jstummbillig wrote:
           | While the "it will never do as well as humans at this one
           | thing I feel strongly about" bias is still a common one, even
           | on hn, at this point I am fairly certain that we will soon
           | all have to live with the fact that we are no longer all that
           | special in regards to, well, absolutely everything.
           | 
           | One of my more interesting realisations over this development
           | is that being human is apparently a religion to a lot of
           | otherwise secular humans.
        
             | pmoriarty wrote:
             | _" we are no longer all that special in regards to, well,
             | absolutely everything"_
             | 
             | If/when general AI comes about, maybe so.
             | 
             | Until then all we've got are a bunch of highly specialized
             | tools/helpers/slaves that may be good (in some sense) at
             | one thing and awful at pretty much everything else.
             | 
             | You could argue that humans themselves are an ensemble of
             | such highly specialized parts that are more than the sum of
             | their parts in some ineffable way that's more of a "we know
             | it when we see it" than something that's formalizable.
             | Machines/computers lack that... so far.
        
       | eminence32 wrote:
       | I wonder if this implementation was trained on the same dataset
       | that trained DALL-E2, would it produce results of comparable
       | quality.
        
         | tomatowurst wrote:
         | I have to clarify I am visioning DALL-E-2-PORN.
         | 
         | Couldn't you just scrape porn to get copious amount of dataset?
         | Scope would be narrower and thus require less classification
         | and in general it has common themes.
         | 
         | I'm more concerned how expensive it will be to train it on GPU
         | instances. We are looking at A6000s right? That's like $15/hr.
        
           | Karawebnetwork wrote:
           | Sure, you could use images you do not have the legal rights
           | to if you do not release anything at the end / just leak the
           | final result. But this is a lot of effort and resources
           | invested for something that would effectively never be shared
           | on the clear web.
        
           | ghotli wrote:
           | "just"
           | 
           | All the complexity wrapped up into one word.
        
             | tomatowurst wrote:
             | You do realize web scraping has gotten very cheap at scale
             | and easy now too. It's an after thought for me, I'm more
             | concerned with the economics of training, it can't be cheap
        
               | visarga wrote:
               | You don't realize how hard it is to make an inclusive,
               | cleaned up dataset. Take a look at this Notion from
               | BigScience detailing their workgroups. Three of them are
               | related to preparing the dataset.
               | 
               | https://bigscience.notion.site/10743770aae24ff3bdc1b938cf
               | 454...
               | 
               | And this is just for a text-only scrape.
        
               | dotnet00 wrote:
               | In my experience, scraping the data is the easy part.
               | Once you've scraped it you've got to get rid of all the
               | garbage, which is where the issues arise, especially if
               | you're just blindly scraping everything you can find.
               | 
               | For example, in a generative model I'm working on, I have
               | a dataset consisting of ~5M images just blindly scraped
               | from a website. After filtering, this drops down to ~500k
               | images, yet a model trained on that performs worse than
               | one trained on a set of 30k curated images (picked based
               | on a manually evaluated list of "known good" artists),
               | where filtering brings it down to ~18k images. The larger
               | dataset, while containing more information, also contains
               | more errors, many of them pretty hard to filter out.
        
               | tomatowurst wrote:
               | Which is why porn is such a great dataset for
               | crowdsource:
               | 
               | - lots of people are stimulated by it
               | 
               | - lots of people want DALL-E-2 for porn
               | 
               | - and lots of people are willing to work towards that
               | common goal
               | 
               | The beauty of this is that people are just going to keep
               | coming and coming to it.
               | 
               | Like I'm trying to be mature and serious about this.
               | What's it going to take?
               | 
               | - Community responsible for scraping dataset, generating
               | image dataset from moving pictures, upscaling said
               | dataset.
               | 
               | - Crowdsourced labeling, cleaning, normalizing dataset
               | 
               | - Crowdfunding to train, host, and publish.
               | 
               | All of the above are not easy by any means but its much
               | more achievable than trying to build a generic DALL-E
               | that can create anything.
               | 
               | Really my point is that the scope of the dataset has
               | narrower range in terms of desired output where as
               | DALL-E-2 casts a far wider net.
               | 
               | Specialization is key here.
        
               | dotnet00 wrote:
               | I was writing up a whole explanation for how that
               | couldn't possibly work, but tbh you aren't really wrong
               | with that, after all captchas these days do similar work.
               | 
               | I guess the primary concern with a porn model would be
               | the ethics of it, which might turn off any company from
               | helping out on training resources (for example, Google's
               | TPU Research Cloud requires you to follow their code of
               | ethics on AI, which would be very difficult to do with a
               | topic as sensitive to people as porn).
        
               | fluoridation wrote:
               | Sincerely asking: why is this an issue of ethics? What
               | about algorithmically generating porn would be unethical?
        
               | turtledove wrote:
               | You are going to need to figure out how to filter out
               | copyrighted works and images for which you don't have
               | permission (eg, someone uploading a picture of an ex
               | partner).
        
               | tomatowurst wrote:
               | If its in public domain, there is no implicit right to
               | privacy. It's going to be super difficult to claim
               | derivative of copyrighted material going forward.
        
               | turtledove wrote:
               | Sure, but how much of that material is in the public
               | domain? I'd assume not a lot.
        
               | mpalmer wrote:
               | Web scrape... what, exactly? How will "web scraping" lead
               | to high quality results?
        
               | visarga wrote:
               | Already does. It works by magic, we put images on one
               | side and text on the other side and then power it up. Let
               | it simmer for a few months at megawatt power levels.
        
               | Kleto wrote:
               | Reddit has plenty of nsfw categories.
               | 
               | And tags.
               | 
               | Insta has tags.
               | 
               | Plenty of others have nsfw and tags
        
               | tomatowurst wrote:
               | if you have to ask, the answer would be irrelevant.
        
               | mpalmer wrote:
               | I certainly don't want to waste the time of someone who
               | knows what they're talking about.
        
               | minimaxir wrote:
               | Training an AI requires high quality, clean/normalized
               | data, which is very difficult to do at millions/billions
               | of data points and is _very_ frequently done incorrectly
               | with silent failures.
        
               | tomatowurst wrote:
               | There's a section of the internet where you can easily
               | find _billions_ of images or can be generated from moving
               | pictures. Even upscaled. I really didn 't expect to have
               | to spell it out.
               | 
               | hint: they are all about one thing and there are a lot of
               | eager volunteers to help on those websites. It would be
               | easy to "normalize/clean/classify" because the pictures
               | would have a consistent theme, thus reducing the amount
               | of parameters.
               | 
               | We are not trying to generate elephants getting railed on
               | a SpaceX rocket flying in oil painting style (although
               | I'm sure theres people into that and its not my place to
               | judge), we are just trying to remove the human cost out
               | of this necessary evil.
               | 
               | I can't believe nobody is investing in "DALL-E-2-4-PORN".
               | This sounds like an X amount of money thrown at a hugely
               | sticky product that can be iterated (with the current
               | trend in hardware) to the point where it literally
               | generates billions of dollars in revenues for ages to
               | come (no pun intended).
        
               | thorum wrote:
               | Does anyone actually want porn of fake, AI generated
               | "people"? Seems like most of the demand would be
               | deepfakes of real people, which is both highly unethical
               | and a good way to get your business sued out of
               | existence.
        
               | sterlind wrote:
               | furry porn? vore? transformation? selfcest? there's a ton
               | of niche fetishes that aren't really doable IRL, and only
               | exist as art. generated images would be a game changer.
        
               | Kleto wrote:
               | Why not?
               | 
               | Do you prefer known porn actress over non known ones?
               | 
               | I would say that I might know a handful of names but
               | search for them rarly.
        
               | aqme28 wrote:
               | There's also seemingly unlimited content of real people
               | produced consensually by those people. What can DALL-E
               | generate that you can't find on pornhub or onlyfans??
        
               | Kleto wrote:
               | They cost money.
               | 
               | They involve real woman (ethical concerns).
               | 
               | They might not show exactly what you are looking for.
               | 
               | Like the good scene is to short or the quality is too bad
               | or there is only one video.
               | 
               | Also variation.
               | 
               | ML porn should be a good thing. Only thing Im not sure is
               | about people creating pedo porn. But even that is better
               | than real pedo porn :|
        
               | badwolf wrote:
               | https://davidmack.medium.com/what-i-learned-from-
               | building-an...
        
       | 29athrowaway wrote:
       | I guess the stock photos industry will be disrupted by this.
        
         | karmasimida wrote:
         | How big is getty? I think this is a billion dollars business.
        
         | seydor wrote:
         | They can make more profit if they don't need to pay artists
        
         | benstrumental wrote:
         | Definitely. With the right training set, video game assets too.
         | Imagine being able to generate N variations of any asset in any
         | video game style...
        
         | quickthrower2 wrote:
         | r/wtfstockphotos will be full of this
         | 
         | And medium posts about React hooks and Go generics are gonna be
         | full of "Animated Drake meme but with Rick Astley" kind of fun.
        
       | throwaway12245 wrote:
       | I did the pip installs and installed Cuda. I changed prompt and
       | ran the sample code. It ran to completion. How do I save the
       | image from trainer.sample?
        
         | visarga wrote:
         | You'd have to train it first.
        
       | [deleted]
        
       | bufferoverflow wrote:
       | How much would it cost to train something like this? Is there
       | even a good dataset for it?
        
         | dirtyid wrote:
         | I feel like the fact big porn hasn't poached talent and jumped
         | all over this suggest at least 10s of millions. That said some
         | for profit no-rules deepfake service for disinformation and
         | illegal content has to be in the works.
        
           | tomatowurst wrote:
           | There's a company in Montreal that makes that in a month and
           | also has access to copious amount of said datasets on their
           | servers. It may or may not be that they already have
           | engineers on it. We have no way of knowing since its a
           | private company
        
         | Yenrabbit wrote:
         | There is a dataset of 5 billion image-text pairs (laion-5b)
         | scraped by various parties. This can then be filtered and used
         | to train these models. Cost is a bit of an issue but there are
         | orgs that have provided compute for open model training. And
         | Imagen is nice because the text encoder part is already
         | available and doesn't need more training, so it would just be
         | the diffusion model components being trained. I'd guess we'll
         | see a biggish training run starting in a few weeks.
        
           | visarga wrote:
           | I hope so. It's a bit cruel to show off and then lock it
           | away.
        
           | dharma1 wrote:
           | When you say a bit of an issue, how much are we talking?
        
             | Yenrabbit wrote:
             | Four or five figures I'd guess? I'm not clued up on
             | costs/performance for TPU stuff to give a better estimate,
             | but guessing at a week on a 256 TPU pod, call it $30k?
        
               | Voloskaya wrote:
               | You are off by an order of magnitude at least. 256
               | TPUs-v4 (not pod), would cost you around 20k$/day. They
               | actually used 512 TPUs (256 for base model + 128 for each
               | of the two superresolution models).
               | 
               | Assuming an average training time of 1 week as you said,
               | that gives us about 280k$.
               | 
               | It's also most likely trained for longer than a week, the
               | base model for Dalle-2 was trained for 100-200k GPU
               | hours, so between 2-4x longer than that, we can guess
               | this is roughly similar.
               | 
               | You also never successfully train everything first try,
               | so all in all, to replicate this work just from the
               | paper, we are talking about at least 500k$.
        
               | rockemsockem wrote:
               | While that is what they did, they also used a batch size
               | of 2048 while training. This is just to speed training
               | up, not a hard requirement. It's easy for Google to
               | justify more money on compute to save engineer iteration
               | loops.
               | 
               | I'll have to read the paper for more details, but it
               | would almost certainly cost less (and take longer) to
               | train a model like this in a more resource constrained
               | situation than Google faces .
        
               | tomatowurst wrote:
               | Couldn't a bunch of us shell out $5000~$50,000 and do
               | this ourselves? Create a non-profit shell corporation
               | outside US jurisdiction, issue shares, raise funds and
               | open source the result?
               | 
               | The shares would simply be votes towards future training
               | dataset endeavors as no profit would be booked here. Say
               | you buy 5000 out of 500,000 shares, that would give you
               | 1% voting power in what dataset to train.
        
               | joshvm wrote:
               | TF Research Cloud. Especially if multiple people could
               | use it with eg model checkpoints:
               | 
               | > 5 on-demand Cloud TPU v3 devices, 5 on-demand Cloud TPU
               | v2 devices, and 100 preemptible Cloud TPU v2 devices for
               | free for 30 days
               | 
               | So up to 7k hours on demand and 70k pre-emptible
        
               | bufferoverflow wrote:
               | Go right ahead, spend $5-50K of your own money and let us
               | use it for free.
        
               | bergenty wrote:
               | Who is going to be liable for the flood of child
               | pornography that comes out of that setup?
        
       | tbalsam wrote:
       | Good to see LucidRains get the love he (rightly) deserves. He's a
       | beast!
       | 
       | As a thank you to him -- he also does for work for
       | commission/etc, check hus GitHub page for more info. I'm not
       | fiscally or currently otherwise directly linked to him too
       | closely, I've just hung around a while and think he deserves far
       | more credit than he gets. This is literally the smallest piece of
       | the pie of what the man does across several subdisciplines, send
       | him a thank-you please, if possible!
        
         | [deleted]
        
         | emadm wrote:
         | Second this, one of the biggest contributors to open source AI
         | and someone everyone should sponsor.
        
       ___________________________________________________________________
       (page generated 2022-05-26 23:02 UTC)