[HN Gopher] Animated Drawings
___________________________________________________________________
Animated Drawings
Author : jonbaer
Score : 837 points
Date : 2023-04-13 19:51 UTC (1 days ago)
(HTM) web link (fairanimateddrawings.com)
(TXT) w3m dump (fairanimateddrawings.com)
| josh_carterPDX wrote:
| Guess I know what I'm doing with the kids this weekend. This
| looks awesome. Thank you!
| upmostly wrote:
| Small anecdote: I discovered the codebase for this project last
| week and assumed that it had already been advertised and
| popularised. Apparently that's just happening now.
|
| I deployed the Docker container and got it working fine and I'm
| already thinking of plenty of cool use cases for the code.
|
| I get the impression that a few startups / apps are going to be
| launched off of the back of this. A few ideas:
|
| - Animated story book using GPT (I'm working on)
|
| - Animated greetings cards using photos (works really well)
| hjessmith wrote:
| Glad to hear it was easy for you to get up and running, but
| where did come across the repo? It's been in 'stealth release'
| mode for a few months now, but nothing was publicly announced
| until today.
| upmostly wrote:
| A friend told me about the project last week, so I did some
| Googling and discovered the GitHub repo by searching for
| AnimatedDrawings and Facebook.
| animatethrow wrote:
| I want to animate video game characters with as much automation
| as possible because hand drawing frame-by-frame takes forever and
| ever. Is this my best bet?
| defaultcompany wrote:
| You could look at Adobe Character Animator.
| Jaxkr wrote:
| I've been working on a project exactly for this:
| https://dreamlab.gg/.
|
| Currently it only allows for animating a character generated
| from a prompt. However, we're working to use null text
| inversion [0] combined with inverse DDIM (used for pix2pix) to
| allow animating a pre-drawn character.
|
| [0]: https://arxiv.org/abs/2211.09794
| hjessmith wrote:
| It currently only outputs .mp4 and .gif files. You could
| probably use these as game assets, but I don't think it would
| be optimal.
|
| The rendering code utilizes an MVC framework. If you need the
| animation output in a .obj format or something, it's as simple
| as creating a new View that supports it. Feel free to request
| this in a Github issue if you want me to prioritize it.
| finikytou wrote:
| this is amazing. how about animating live from a video stream in
| AR :D?
| self_awareness wrote:
| The demo website is frozen and I can't click anything.
|
| > [Report Only] Refused to create a worker from 'blob:https://ske
| tch.metademolab.com/923050e3-9390-4b78-9349-8159a...' because it
| violates the following Content Security Policy directive:
| "worker-src 'none'".
|
| > [Report Only] Refused to create a worker from 'blob:https://ske
| tch.metademolab.com/d94425ea-c1b7-424f-a898-90708...' because it
| violates the following Content Security Policy directive:
| "worker-src 'none'".
|
| > [Report Only] Refused to create a worker from 'blob:https://ske
| tch.metademolab.com/923050e3-9390-4b78-9349-8159a...' because it
| violates the following Content Security Policy directive:
| "worker-src 'none'".
|
| > Failed to load resource: net::ERR_UNKNOWN_URL_SCHEME
|
| > [Report Only] Refused to create a worker from 'blob:https://ske
| tch.metademolab.com/d94425ea-c1b7-424f-a898-90708...' because it
| violates the following Content Security Policy directive:
| "worker-src 'none'".
|
| > Failed to load resource: net::ERR_UNKNOWN_URL_SCHEME
|
| > Uncaught (in promise) DOMException: The play method is not
| allowed by the user agent.
| Oras wrote:
| This is incredible! Fantastic presentation and easy to follow
| code.
| hjessmith wrote:
| You're the first stranger to ever compliment my coding style. I
| feel all warm and fuzzy now.
| totetsu wrote:
| I remember doing something similar to this with many manual steps
| with Photoshop and aftereffects many years ago.
| mholt wrote:
| Very fun. Is there something like this but which can capture the
| motion from a webcam (without separate motion capture process as
| described in the readme) and project that onto the drawings?
| hjessmith wrote:
| As written now, it expects 3D motion data. Extending it to
| support 2D motion data is doable, but will take time. Feel free
| to open a github issue requesting it if you'd like me to
| prioritize it.
| umaar wrote:
| This is a great project and it was fun playing around with it.
|
| When Meta started discussing this in 2021, the HN sentiment [1]
| was quite...cautious, making connections to how Meta can't be
| trusted with kids drawings. Has something changed?
|
| [1] https://news.ycombinator.com/item?id=29580619
| blitzkrieg3 wrote:
| If I had to guess it's because this one is unbranded and
| released as open source
| aierou wrote:
| We've moved on to a different moral panic
| hutzlibu wrote:
| I mean it gets exhausting. We had Covid (and Q), Musk taking
| over Twitter, war in Ukraine, fear of war in Taiwan, ...
| accelerated climate change and so on. There are real limits
| to how many things one can be upset about.
| devsda wrote:
| The terms of usage seems to remain the same and the demo page's
| privacy policy points to the common Facebook privacy policy.
|
| I hope the terms are limited to the demo and there's no
| telemetry within the source.
| hvidevold wrote:
| Made this using the tool: https://github.com/emnh/mantary-
| platformer/blob/main/mantary...
|
| I used the website, downloaded the mp4 and postprocessed to walk
| in place using a shell script: https://github.com/emnh/mantary-
| platformer/blob/main/animscr...
|
| The character was generated using stable diffusion.
| hjessmith wrote:
| Hey! That's my project!
|
| Code and dataset are here:
| https://github.com/facebookresearch/AnimatedDrawings
|
| And a browser-based version of it is here:
| http://sketch.metademolab.com/
| SanderNL wrote:
| This is great fun. Thank you for giving away this work and so
| accessibly. I got it working in minutes.
| sva_ wrote:
| Would be nice to add a queue. Went through all the steps and
| then it said the servers are busy losing progress.
| greggsy wrote:
| This is really cool, and refreshingly simple in the current
| landscape of generative tooling.
|
| I'm interested to hear your thoughts about the inevitable
| monetisation that will occur once someone wraps this into a
| horrible app with ads?
|
| Maybe someone else can answer the related question of how the
| respective app stores deal with attribution-based code
| licenses?
| doubtfuluser wrote:
| This is really a fun project!
|
| Is it actually possible to use the code and create an app for
| sale out of it. The MIT license says so I think, just wondering
| if anything else would speak against it.
| hjessmith wrote:
| I don't know who would speak out against it. The MIT license
| certainly allows for it, and I don't actually own it... I
| just wrote it :)
| ac50hz wrote:
| +1
|
| This is a pleasant take. I may borrow the sentiment
| expressed here to help me describe some similar
| distinctions.
| upmostly wrote:
| I'm working on an app called Monstdraw.com that uses this
| code if you're interested in collaborating? My email is in my
| bio.
| tealpod wrote:
| Thank you Thank you Thank you, my kid loving it. I will be
| happy to volunteer and help in anyway I can.
| qainsights wrote:
| Cool project. Just tried with my kids. They enjoyed it. Is it
| possible to download the video?
| Knucklebones wrote:
| On the online demo, you can also download by clicking 'share'
| at the end (there's an option in the menu that shows up).
| [deleted]
| hjessmith wrote:
| I assume you're referring to the demo at
| sketch.metademolab.com? The server is returning an .mp4 file
| that is displayed by your browser, so you should be able to
| download it. On Chrome, if I maximize the video and click the
| three vertical dots, I get the option to download the video.
| qainsights wrote:
| Thanks. Yes, I figured it out :)
| the-mitr wrote:
| very cool project!
| cstrat wrote:
| This is AMAZING, my 3 year old draws the cutest pictures and I
| can animate them!
|
| One thing with the browser version, it all works fine up until
| the animation and then its flipping my image upside down? Could
| this just be the photo metadata rotation/orientation??
| eurekin wrote:
| Imagine future job market, where a 5 year old will already be
| super familiar with the T-posing, which currently mostly CG
| artists now about
| hjessmith wrote:
| It's not metadata, you would see that when you upload the
| photo at the beginning.
|
| I'm guessing that the nose key point is located below the
| shoulders in your child's drawing?
|
| That's a known issue caused by how we 'apply' poses onto the
| characters. It's fixed in the GitHub code but not in the
| browser version. Try moving the nose keypoint to the top of
| the characters head and that should resolve it
| cstrat wrote:
| Ahh yeah that is it, haha thanks :D cant wait to show her
| tomcam wrote:
| Amazingly good demo. You rock.
|
| Also you're apparently very, very fit and flexible. I pulled a
| hamstring just watching the dance.
| hjessmith wrote:
| It's equally important to stretch before coding and stretch
| before dancing.
| tomcam wrote:
| I'VE BEEN DOING IT WRONG THE WHOLE TIME
| marai2 wrote:
| I'm definitely going to start stretching before coding
| now! Maybe even 15 minutes of yoga before debugging.
| mind-blight wrote:
| Honestly, that would be so good for your health. You'd go
| into your 30s, 40s, and 50s with much less MSK pain.
| liveoneggs wrote:
| my kids and I had a lot of fun with the sketch version while
| killing time in a hotel room a while ago.
| DiscourseFan wrote:
| I hate to say it, but I think this could have a great application
| in horror games.
| Elias-Braun wrote:
| I have a big smile in my face right now : D
| yegle wrote:
| Oh wow, this is the animation I saw on Disney Cruise's Animator's
| Palate, but better! https://disneycruise.disney.go.com/onboard-
| activities/animat...
| simple10 wrote:
| This is amazing!
|
| It's a bit buried in the repo readme files, but the author
| @hjessmith used Rokoko Video [1] to create the animation files
| (BVH) from a phone video. Then he used Animated Drawings to make
| the drawings replicate his motion captured from the video.
|
| Both Rokoko and Animated Drawings were new to me. I know what
| I'll be doing with my kids all weekend! Thanks for sharing the
| project.
|
| [1] https://www.rokoko.com/products/video
| hjessmith wrote:
| If you post your creation to social media, please use
| #FAIRAnimatedDrawings. I'd love to see what you create!
| nmfisher wrote:
| Completely tangential point - Rokoko Video is fine for a free
| tool, but if you need something better and you're willing to
| pay, there are other options. I spent a couple of days
| evaluating various video-only mocap and in the end went with
| DeepMotion.
|
| Rokoko Video wasn't very reliable (at one point I was standing
| perfectly still, but it thought I was on one leg twirling like
| a ballerina). Although it's very expensive, DeepMotion has been
| great (also if you were disappointed by DeepMotion 12+ months
| ago like I was, I'd suggest revisiting - it's leaps and bounds
| ahead of the earlier version I tried out).
|
| move.ai looks very good too, but needs multiple cameras which
| was overkill for my use case. Plask seems to have regressed
| (the platform seems like it doesn't even work half the time)
| and Radical's aggressive tactics really turned me off.
| simple10 wrote:
| It looks like DeepMotion has a free tier. Did you just need
| more credits or is there some additional feature only
| available on the paid tiers?
| yantrams wrote:
| Edit: My bad it has to do with my Adblocker. Its all good now.
| Cheers
|
| Super cool! Thank you for bringing this to the world. Kids are
| gonna love this. I'm unable to get it running on the web demo
| unfortunately though :( Post image upload, I'm redirected to
| https://sketch.metademolab.com/canvas with a blank screen with
| the grid image. Without any error message, it isn't obvious if
| the issue is with the images I uploaded ( which probably is the
| case ). Will try the Python library and play around.
|
| PS: That silly dance thingy gif is so hilarious
| hjessmith wrote:
| Glad you got it working.
|
| And glad you appreciate my sick dance moves :)
| yantrams wrote:
| :) Can't wait for my 4 year old kiddo to wake up so I can
| tell her how her drawings come to life in the night and dance
| around if she behaves haha
|
| Thank you again (and Meta too) for making this happen. This
| is gonna be put a smile on millions of kids I tell you and
| make them draw more.
| Franzeus wrote:
| Very cool tech and amazing dance! :)
|
| We are working on a startup (https://dibulo.com) which provides
| children an off-screen coloring experience combined with a
| digital reward. We have so many ideas and using pose-estimation
| is one of them. Definitely gonna play around with it!
| s1mon wrote:
| It's cool that this is possible, but it's missing some very
| important subtleties which make professional animation look
| "right" or even "real". As it is, there's no sense of compression
| or snap at the right parts of the movement. 2D character
| animation has a lot of exaggerated distortions of objects to make
| them look like they land on the ground or bounce. The sense of
| weight and inertia is critical.
|
| These aren't necessarily great, but this page[0] does touch on
| some of the basic concepts. "Squash and stretch" is a key thing
| missing. Also while in the demo you're dancing not walking, a
| walk cycle is a very basic thing which is often done poorly by
| even supposedly professional companies. [1] It's really important
| for the "contact" part to seem like it's happening. Getting feet
| to plant themselves on the ground and look like they're not
| swimming or sliding around is a challenge.
|
| [0] https://idearocketanimation.com/13721-12-principles-of-
| anima... [1]
| https://www.angryanimator.com/word/2010/11/26/tutorial-2-wal...
|
| [my father worked in animation for much of his career and would
| complain bitterly about badly done animation, so I guess I can't
| unsee things with his critical eye]
| timini wrote:
| did you get paid to make this?
| hjessmith wrote:
| Yes. I did most of this as a postdoc at FAIR, Meta's AI
| research group.
| liendolucas wrote:
| Out of all the ML projects submitted to HN this is one of the
| funniest and most entertaining I've seen so far! Also very neat
| demo and presentation in the website! KUDOS to the author(s) of
| the implementation and paper!
| sebastianconcpt wrote:
| Best use of computing power ever :D
| tomcam wrote:
| Analysis: true
| Waterluvian wrote:
| Phenomenal. For what it's worth, I think this would be a killer
| app for my phone. My kids would easily spend hours drawing and
| animating their creations.
|
| I'd pay money for this.
| Galanwe wrote:
| I would pay for that as well. All it needs is some kind of
| Android app.
| noduerme wrote:
| Get them one of these. They'll actually get the experience pf
| animating, broken down into bare terms...
|
| https://the80sand90s.com/articles/etch-a-sketch-animator
|
| that little toy I had as a kid did more to shape my career as
| than even hypercard.
|
| FTA (and relevant): _> What truly made this the best toy ever
| made was the amount of patience and perseverance a kid needed
| to create these animations, just like real animators. This is
| also why it would later fall from popularity._
| mojomark wrote:
| Holy crap - I totally forgot that I had one of those and
| spent an obscene number of hours creating on it. Thanks for
| the memory-blast post!
| noduerme wrote:
| Hah! Cool to meet another kid who grew up with that. I wish
| there were more actual _devices_ like that now, with their
| own weird UIs and knobs and buttons, instead of everything
| being an app mixed in on a tablet with endless other apps.
| When you think about it, what a far-out thing for a company
| that made Etch-A-Sketches to engineer and sell...
| ShadowBanThis01 wrote:
| That thing seemed really cool, but even at the time I passed
| on it because there was no way to save your work externally.
| egonschiele wrote:
| So great to see more work on pose research! My ultimate dream is
| to have a tool where I can give it a cartoon character and a set
| of pose photos, and have the tool pose the character in those
| poses. This feels one step closer to that.
| hjessmith wrote:
| You might check out the technical paper, the background section
| lists some relevant related works on non-photorealistic pose
| estimation.
|
| One work that seems particularly pertinent to your use case is:
|
| Gesture3D: Posing 3D Characters via Gesture Drawings
| http://www.cs.ubc.ca/labs/imager/tr/2016/Gesture3D/
| dtjohnnymonkey wrote:
| I have a bunch of my kids drawings that I want to turn into
| digital art paintings. Is there any product or service for this?
| polygamous_bat wrote:
| Have you tried the browser app? https://sketch.metademolab.com/
___________________________________________________________________
(page generated 2023-04-14 23:01 UTC)