[HN Gopher] Show HN: Shortbread - Create AI comics in minutes
___________________________________________________________________
Show HN: Shortbread - Create AI comics in minutes
Just go to the link and click on "Start Creating". No signing in
required. I built shortbread to help anyone to create comics /
manga series. The onboarding process helps you kick start a page
from 60%, then you can use your creativity to bring it to 1000% in
a fully-controllable editor. Tech stack: GPT 3.5 Turbo - the
comic script generation. It handled everything from layout,
character, scene, SD prompts, to dialogue. SD 1.5 - We put up SD
servers on GCP. For every comic we generate one large image and
crop it into panels. Per the experiments of u/Deathmarkedadc on
Reddit, this massively helps with consistency. The models are
trained on anime scenes tho, and might not be so great with
animals. Frontend: Next.js 13 on Vercel, React + Typescript. We
built the entire editor from scratch to compose the comic (images,
panels, speech bubbles, text) like a webpage. This allows you to
edit and republish your comics like a website. You can dynamically
generate panels as well. Try resizing a panel into a long narrow
box and generate. Backend: Firebase. Sample comics: a japanese
couple sits at dinner table. The husband told the wife a secret
(link
https://create.shortbread.ai/viewer/debdf25c-3f95-492a-952a-...)
An army of male soldiers fighting against an army of female
soldiers in ancient china
(https://create.shortbread.ai/viewer/4566613c-7146-4ed7-9b8d-...)
a team of girls play volleyball against a team of boys
(https://create.shortbread.ai/viewer/aafc2f61-d008-4f3f-aa8f-... )
Next steps: - More pages - Fine panel-level control. Poses,
control net, etc. - Multi-character. - Different styles. -
Allows you to control character design. I'm Fengjiao Peng, founder
and chief engineer at Shortbread. I was previously a webtoon
artist. We want to build this into something you can create entire
comics series / manga / webtoons with. Criticism and suggestions
welcome!
Author : Fengjiao
Score : 120 points
Date : 2023-10-06 15:56 UTC (7 hours ago)
(HTM) web link (shortbread.ai)
(TXT) w3m dump (shortbread.ai)
| atleastoptimal wrote:
| I think the flow should be
|
| I give summary => site gives me panel descriptions to choose from
|
| I pick descriptions = > Site creates a "skeleton" layout option
| for each panel. At this point I can modify the skeleton or save
| them
|
| I get skeleton layout => Site makes it look pretty
|
| With each stage modifiable
|
| I think this is very cool and well made.
| canadianfella wrote:
| [dead]
| ianbicking wrote:
| Very cool. I like the editor, though I wish I could drag the
| bubbles without clicking on them first.
|
| There's not a lot of plot in a single page, but even with that I
| wish I had more control, and for longer pieces I would absolutely
| want a lot more control. I'd want to see some basic text
| previews, and be able to control both the large scale and fine
| scale of the progression. I'd want to be able to control the tone
| of the piece, clarify points that GPT might not be picking up on,
| override choices, etc. I might tweak some dialog... but most of
| the changes I envision are before dialog, about how scenes are
| broken up, or the basic premise of the story/world.
|
| Many of the other features you list (outside of more pages, of
| course) feel less important than the story building itself. (But
| that's also coming from my personal interest in the story
| design.)
| brucethemoose2 wrote:
| This is all super cool.
|
| Some random suggestions:
|
| - I dunno what diffusion framework you are using, but the
| AITemplate (for GPUS) or diffusers JAX (for TPUs) backend can
| massively increase your diffusion throughput.
|
| - Alternatively, I believe HuggingFace already has a JAX backend
| for Stable Diffusion XL, so you could run a model with much
| better support for large resolutions/inpainting massive images at
| a similar (?) speed.
|
| - There are schemes for area prompting and subject "subset"
| prompting in stable diffusion, as well as using images as input.
| As an example of how y'all might use this, you could generate a
| image for Character A, an image for Character B, encode them.
| specify that the character A prompt latents go on the left side
| of the image, and the character B prompt latents go on the right
| side of the image. And of course you can add to these area
| prompts, like "jumping" on the left side and "ducking" on the
| right side of the image. There's also a way to specify which
| prompts/encoded images belong to which subjects instead of
| manually cutting out areas, see:
| https://github.com/BlenderNeko/ComfyUI_Cutoff
| Fengjiao wrote:
| Woaaa love these inputs. Thank you! Wasn't aware of the JAX
| backend will check it out. Right now we're on SD 1.5. We tried
| SDXL but found the quality improvement to be marginal. Yes to
| area prompting/regional control to help people create more
| complex scenes. I need some design thinking first since it's
| easy to over build and spit out something super complicated.
| Immediate next step is to def add controlnet.
| brucethemoose2 wrote:
| > JAX
|
| Yeah, check out their post:
| https://huggingface.co/blog/sdxl_jax
|
| I dunno how expensive TPU instances are these days, but the
| performance is insane!
|
| > We tried SDXL but found the quality improvement to be
| marginal.
|
| Yeah, the vanilla HF diffusers pipe is unimpressive to me.
|
| Try playing with this though, turn on FreeU and specify an
| anime style: https://github.com/MoonRide303/Fooocus-MRE
|
| I have never gotten such high quality results from simple
| prompts, even in cloud models like Midjourney/GPT4. The
| question is how to port even part of that magic over to the
| diffusers pipeline...
| brucethemoose2 wrote:
| Also, VoltaML has a good reference GPU AITemplate SD 1.5
| implementation:
|
| https://github.com/VoltaML/voltaML-fast-stable-
| diffusion/tre...
|
| The speed jump is massive on my desktop GPU, probably even
| more dramatic on cloud hardware, and it may support some
| things (weight swapping/lora swapping/resolution
| changing/controlnet) better than JAX.
| Fengjiao wrote:
| My issue previously with these prebuilt backends is that
| you can't tweak it like sdwebui does, but to make our thing
| work it took a thousand tweaks. Can look into this first to
| see how customizable it is.
| brucethemoose2 wrote:
| VoltaML is a relatively vanilla diffusers-based backend,
| so its not a hairy monster to hack like you may have seen
| with SAI-based UIs (like Comfy, Fooocus and Automatic)
|
| The AITTemplate code is a lightly modified version of
| Facebook's example dynamic AIT script, to get rid of
| small issues like VRAM spikes: https://github.com/faceboo
| kincubator/AITemplate/tree/main/ex...
|
| InvokeAI is also diffusers based, but they seem to mess
| with the pipeline a bit more.
|
| Anyway, all that may be better as a reference for
| interesting features rather than a backend to try and
| adopt.
| AiCredentials wrote:
| [flagged]
| Fengjiao wrote:
| Author here. Just go to https://shortbread.ai and click on "Start
| Creating". No signing in or anything required.
| Fengjiao wrote:
| Author here - Didn't expect the amount of traffic rn. The wait
| might be EXTRA long. We're trying hard to spin up more servers!!
| colesantiago wrote:
| Was looking for something like this, i'm definitely signing up!
|
| I'm not a comic book artist by any means but now I can add this
| to my skills list with this.
|
| Is there any pricing yet on this and is this backed by YC or
| bootstrapped?
| Fengjiao wrote:
| - I intended for anyone to be able to use this - so def no
| worries - No pricing yet, everything is free since it's early
| beta, but I'm thinking a subscription for a bunch of credits
| style like Midjourney. - Yes, we are YC
| jrflowers wrote:
| I love "as seen on y combinator"
| Fengjiao wrote:
| hahaha sorry the urge of putting on a nice looking logo
| overtook me
| [deleted]
| HanClinto wrote:
| This is really impressive, well done!!
| riffraff wrote:
| This is cool, but it appears the consistency issue with this
| stuff is not solved yet, e.g. in every panel clothing and armors
| on the dame character are different.
|
| Perhaps a more simplified style might work better.
| alex_c wrote:
| Very cool! I was briefly toying with something similar just last
| week.
|
| Main challenge I see is character consistency. I really like the
| way you set up the prompts, but even so:
|
| _Outfit - a simple black sleeveless gi with white pants, a black
| belt tied around his waist_
|
| In two consecutive panels, the output swaps the colors (first
| panel gets it right, second panel has white gi, black pants).
|
| Curious how you'll tackle this challenge!
| Fengjiao wrote:
| Hi Alex, great catch -> We didn't solve consistency, but we saw
| that if you regenerate a few times, you usually get something
| that's visually similar. Right now AI artists all do loads of
| postprocessing - using AI, so later we might have a "smear"
| feature that inpaints the inconsistent part. Let me know if you
| have thoughts on this
| nottheengineer wrote:
| People can generate a few good samples of what they want a
| character to look like and then interrogate clip to get a
| more detailed prompt that makes it more consistent. It
| increases the prompt sizes a lot, but I don't think there's
| an easy way to solve this.
|
| Maybe you could build a UI that semi-automates this process?
| mentos wrote:
| Could eventually use GPT4vision to review the output of the
| art to see if all the panels are consistent?
___________________________________________________________________
(page generated 2023-10-06 23:00 UTC)