[HN Gopher] CLIPasso: Semantically-Aware Object Sketching
___________________________________________________________________
CLIPasso: Semantically-Aware Object Sketching
Author : lnyan
Score : 75 points
Date : 2022-02-14 09:18 UTC (1 days ago)
(HTM) web link (clipasso.github.io)
(TXT) w3m dump (clipasso.github.io)
| mcphage wrote:
| Like most people, I've got piles of old photos that I'd like to
| be able to label who is who in the person. For some family
| photos, I've doodled sketches of the photo, labelled the sketch,
| and attached it to the back. But I've often thought there had to
| be a way to get the computer to take the photo and generate the
| sketch from it. This seems exactly like it!
| yeldarb wrote:
| Continually amazed by how versatile CLIP is. I'm keeping a
| compendium of cool use-cases I run across here[1]. I'll add this
| to it today!
|
| [1] https://blog.roboflow.com/openai-clip/
| chrisweekly wrote:
| Thanks for sharing, @yeldarb! Your blog is even more
| interesting to me than the OP.
| pizza wrote:
| related: I had an idea to make a CLIP paint.NET image editing
| plugin. You would make a new layer, add some green ('towards')
| and red ('away from') splotches to the layer, and give a prompt
| to CLIP. Wherever there was green, CLIP would 'nudge' the image
| along the direction of the gradients towards the semantic goal,
| keep it constant where there was nothing, and nudge it in the
| opposite direction where there was red. And you could do this
| over and over again, w/ different CLIP prompts all the while
| editing the semantic masks as you like. The end result would be a
| feedback process to use CLIP + feedback as an extremely creative
| and controllable image manipulation tool. But then I realized
| that I didn't know how to make a paint.NET plugin :) so here is a
| free idea for anybody who does
| yeldarb wrote:
| It doesn't use CLIP but check out NVIDIA's Canvas[1] (powered
| by GauGAN[2] which has a pretty poor UI but is usable in your
| browser).
|
| [1] https://www.nvidia.com/en-us/studio/canvas/
|
| [2] http://gaugan.org/gaugan2/
| pizza wrote:
| Yep, seen Canvas - probably where I Picasso'd (read: stole
| :^) ) the original idea! The nice thing about making it a PDN
| plugin based on CLIP is that it could plug into the existing
| PDN plugin system, PDN is pretty solid image editing
| software, and also CLIP could make it extremely versatile -
| no more limitation to primarily landscapes - including the
| fact that there are newer and vastly better CLIP tools coming
| out, e.g. diffusion model based CLIP - but I don't think I
| had actually seen the GauGAN UI before, thank you!
___________________________________________________________________
(page generated 2022-02-15 23:02 UTC)