[HN Gopher] How to Train an AI Image Model on Yourself
___________________________________________________________________
How to Train an AI Image Model on Yourself
Author : aberoham
Score : 161 points
Date : 2025-01-31 16:46 UTC (6 hours ago)
(HTM) web link (www.coryzue.com)
(TXT) w3m dump (www.coryzue.com)
| ge96 wrote:
| What I want is to be able to feed in a bunch of videos and
| generate an animatable (from talking) 3D face from that data. I
| suppose you in theory only need 3 images (front and sides). But
| mapping pixels to motion is interesting (facial expressions).
|
| There wouldn't be depth data so it would be inferred from shadows
| ilaksh wrote:
| Replicate has Hunyuan video training now.
| https://replicate.com/blog/fine-tune-video
|
| Also, Kling 1.6 Elements works pretty okay if you use the same
| person/face for each element.
|
| Kling also has lip sync.
|
| Or this lip sync with replicate:
| https://replicate.com/bytedance/latentsync
|
| Or there is HeyGen or D-ID or Synthesia, or tavus.io for full
| interactive digital twins.
| ge96 wrote:
| Thanks
| timdiggerm wrote:
| Why do you want to do that?
| ge96 wrote:
| My case is not directly nefarious, for example an old popular
| YouTuber that streamed in the early 2000s taking their
| content and making a model of them for personal use like a 3D
| chat bot but with that person's quirks
|
| Edit: when I say "nefarious" I mean you can use that tech to
| impersonate someone (eg. political reason) but for my case
| it's more the creeper type cloning someone for personal use
| eg. Replika
|
| Tangent, the holo vtubers industry is interesting since they
| build up these characters with some unique persona/theme and
| then people follow that specific model, they could make
| themselves into an AI easily since it's a rigged 3D asset but
| of course it would be boring compared to the real thing
| GaggiX wrote:
| >they could make themselves into an AI easily since it's a
| rigged 3D asset but of course it would be boring compared
| to the real thing
|
| The most popular vtuber on Twitch is an AI tho
| ge96 wrote:
| You talking NeuroSama? I haven't kept up with it in a bit
|
| I'm not sure if that's truly AI since the Turtle drives
| her
|
| Edit: if the source was open I'd believe it
| GaggiX wrote:
| >I'm not sure if that's truly AI
|
| It has always been a LLM. There is no human typing at
| insane speed to the TTS.
| ge96 wrote:
| I'm referring to live interception of messages which I
| guess has to be done to be compliant with Twitch's terms
| -- there is a human there
|
| edit: but yeah the fact that so many people interact with
| her shows generated content can keep people occupied
| manishsharan wrote:
| This is fantastic but now you need to train a model to detect AI
| generated images from actual photos. Then of course , a model to
| beat the detector model and then a model to catch the model that
| beats the detector model and so on.
|
| Thank you from people holding NVDA.
| beng-nl wrote:
| You may have re-invented GANs :-)
| petercooper wrote:
| Replicate does make this particularly easy while still being
| somewhat developer focused. I've used it for a few people in our
| group chat so we can make silly in-joke memes and stuff and the
| results are quite stunning. Replicate then offers the model up
| over a simple API (shown in the post) if you wanted to let people
| generate right from the chat, etc. Replicate is worth poking
| around a bit more broadly, too, they have some interesting models
| on there (though the pricing tends not to be very competitive if
| you were going to do it at scale.)
| deadbabe wrote:
| I'm imagining something where an influencer trains AI to make and
| post images of themselves on social media, then the influencer
| dies but the AI keeps going forever.
| ge96 wrote:
| The impact is kind of interesting, how do you know someone's
| legit, the person doing basejumping or whatever
|
| Thanos/NFTs: where did that take you? right back to me
|
| Thinking hardware with built in chain interface for proof
|
| Oh man dating apps too
|
| That's true love though, two people meet up IRL they're both
| like wtf who are you
| isoprophlex wrote:
| I did this for our beloved, dead cat... On replicate, too. I
| loved the results, until at one point I suddenly got really
| creeped out about the thing I was doing.
| ryandvm wrote:
| This is going to be big business I think. I have probably sent
| hundreds of thousands of emails, texts, chats, etc. It would be
| well within the realm of possibility to train an LLM on a loved
| ones communications corpus and allow you to chat with "them"
| after they're gone.
|
| Possible? Yes. Convincing results? Probably. Good idea? I doubt
| it.
| mipmap04 wrote:
| Oh man, I did this with my dad's voice after he died and set
| up a thing where I could talk with an LLM-backed assistant
| and have it respond in his voice and mannerisms. It was a
| very weird coping and grief period and I ultimately hit a
| point where I got really weirded out about what I was doing.
| slig wrote:
| I remember seeing it here on HN that someone did that with a
| group chat and it would reply as each friend.
| knicholes wrote:
| This is exactly what I'd want to do for my "smart urn."
| mystified5016 wrote:
| Forget an urn, I want my digital ghost to haunt a furby.
| TeMPOraL wrote:
| Code golf task: implement the whole pipeline above in
| minimum amount of (existing as of now) ComfyUI nodes.
|
| Extra challenge: extend that to produce videos (e.g. via
| "live portrait" nodes/models), to implement the digital
| version of the magic paintings (and newspaper photos) from
| Harry Potter.
|
| EDIT:
|
| I'm not joking. This feels like a weekend challenge today;
| "live portraits" in particular work fast today on a half-
| decent consumer GPU, like my RTX 4070 Ti (the old one, not
| Super), and I believe (but haven't tested yet) even
| training a LoRA from a couple dozen images is reasonably
| doable locally too.
|
| In general, my experience with Stable Diffusion and ComfyUI
| is that, for fully local scenario on _normal person 's
| hardware_ (i.e. not someone's totally normal PC that
| happens to have eight 30xx GPUs in a cluster), the
| capabilities and speed are _light years_ ahead of LLM
| space.
|
| Just for comparison, yesterday I - like half the techies on
| the planet - got to run me some local DeepSeek-R1. The 1.58
| bit dynamic quant topped at 0.16 tokens per second. It's
| about the same as it takes a SD1.5 derivative to generate
| me a decent-looking HD image. I could probably get them
| running parallel in lock-step (SD on GPU, compute-bound;
| DeepSeek on CPU, RAM-bandwidth bound) and get one image per
| LLM token.
| portaouflop wrote:
| I think that was 1:1 a black mirror episode
| PaulDavisThe1st wrote:
| Episode title was "Be right back"
| waspleg wrote:
| Literally a Black Mirror episode.
| oskarkk wrote:
| This reminds me of paintings in Harry Potter.
| ramoz wrote:
| FYI: For Flux, there is a lot more power in the text-encoder &
| you can prompt with more meaningful and comprehensive sentences.
| Thus, less of the traditional comma separated & concise phrasing
| we saw in stable diffusion.
|
| You should do the same with your training images. Caption
| everything you do not want the model to remember as "you" (what
| you're doing, wearing, accompanied by, accessories, etc).
| DoodahMan wrote:
| is something like this possible to do with video yet?
| thefourthchime wrote:
| I did this a while back, though it was pictures of my wife in
| lingerie.
|
| - I asked grok to generate a list of racey prompts. - Has
| replicate generate them via script. About 10-20% are very poor, I
| filtered those out manually. - It also has NSFW guardrails, but a
| simple retry or word juggle gives you a chance to get around it.
|
| I think I spent $10
| Der_Einzige wrote:
| There is a parallel "underground" AI research world of stuff
| like this, with it's hub on "civit.ai" instead of huggingface.
|
| Often the innovations from that world are ahead of mainstream
| AI research by years. You should see what coomers did for LLM
| sampling in order to get over issues with "slop" responses just
| for their own pervy interests. This is a full several years
| before the mainstream crowd ever cared.
| ok_dad wrote:
| Porn has always pushed the boundaries of media on the
| internet. I don't know why people are surprised! Since sex is
| something nearly everyone does, it would make sense that a
| lot of human progress were the result of trying to integrate
| sex and whatever new tech is out there at the time. I am sure
| a hundred years ago some inventors were pushing the
| boundaries of motors in sex toys, and in another hundred
| years some other inventor will be pushing the boundaries on
| putting sex in holograms.
| DrSiemer wrote:
| It's kind of annoying that some of the best models out there
| have a tendency to produce very not safe for work results.
|
| Look mom, I can make some cool astrology images for you!
| Whoops, that's boobs. That too. And this one. Ehh, hold up, I
| need to add a pile of negative prompts first...
| wongarsu wrote:
| Sketching nude humans is a huge part of how human painters
| learn. Because surprisingly clothed humans are just nude
| humans with some fabric over them, and the fabric can make
| it harder to tell what's going on.
|
| Even if we assumed equal amounts of effort, it wouldn't be
| surprising if a large corpus of nude images in the training
| data improved model results.
|
| But maybe we should have better negative prompt presets for
| different levels of decency
| njx wrote:
| Thank you for sharing. Is there any model that can help train
| convert pictures into cartoon or flat vector illustration?
| Our_Benefactors wrote:
| Look into using img2img in stable diffusion
| m463 wrote:
| I had set up automatic1111 a while back, and I believe the webui
| let you your image generation have a starting image. It's kind of
| fun to have a cartoon of yourself based on an image.
___________________________________________________________________
(page generated 2025-01-31 23:01 UTC)