[HN Gopher] Pigment Mixing into Digital Painting
___________________________________________________________________
Pigment Mixing into Digital Painting
Author : tlarkworthy
Score : 353 points
Date : 2024-12-29 23:15 UTC (23 hours ago)
(HTM) web link (scrtwpns.com)
(TXT) w3m dump (scrtwpns.com)
| arcticfox wrote:
| not an artist but this is so cool. Way better than the
| comparisons.
| miiiiiike wrote:
| This is great. I paint miniatures and once you get past the
| basics and start using glazes for filters/blends software isn't
| great for prototyping your paint schemes.
| amarcheschi wrote:
| I was already impressed but at the bottom of the page they show
| they wanna make an addon to color blender fluids accurately.
| Paint me impressed. Paint me impressed but with pigment blended
| accurately
| Kaijo wrote:
| The addon you're referring to is already available:
| https://github.com/rlguy/Blender-FLIP-Fluids/wiki/Mixbox-Ins...
|
| It works as a kind of experimental extension supplied with the
| paid FLIP Fluids addon. Note that the fluid solver built into
| Blender is also an implementation of FLIP, but "FLIP Fluids" is
| a separate product. I've played with fluid simulation in
| Blender quite a bit. The bundled FLIP is very limited, FLIP
| Fluids is great, and the color mixing is amazing!
| amarcheschi wrote:
| Oh, I thought it was referring to the free blender
| implementation. Cool nonetheless
| amjoshuamichael wrote:
| Personally, I don't care about the library, and just want the
| pure math implementation for shaders and such. For that, they
| have the algorithm available here in GLSL, vec3 mixbox_lerp is
| the function you want:
|
| https://github.com/scrtwpns/mixbox/blob/master/shaders/mixbo...
|
| I don't have time to whip up a demo, but it seems like it's
| pretty well optimized and it looks really nice in the examples.
| Great work from these folks.
| guipsp wrote:
| Beware of the licence however
| twobitshifter wrote:
| Cool, could see this being useful as a palette with a combination
| of the normal mode. I don't see a clear way of controlling the
| amount of paint in the demo, if i want 3:1 red to white how would
| i do it?
| kuschku wrote:
| Looks awesome, sadly it's All Rights Reserved + CC-BY-NC, so it
| won't be possible to implement it in Krita, GIMP or Blender
| natively.
| echelon wrote:
| I'd be curious to learn why Secret Weapons (the makers of this
| tool) didn't raise $100M in venture funding two or three years
| ago, before the start of the AI video boom.
|
| They were first on the scene with EbSynth [1], which was the
| first widely used AI-powered video-to-video style transfer and
| keyframe interpolation system. EbSynth gave rise to comedians
| like Joel Haver [2] and lots of others like him.
|
| Several other companies [3] used EbSynth to base their technology
| on and wound up raising large seed rounds. Major musicians used
| EbSynth either directly or through these other companies to make
| music videos [4,5].
|
| Secret Weapons was in a great position to conquer AI video and AI
| creative tools, but they seemingly slept on it.
|
| [1] https://ebsynth.com/
|
| [2]
| https://www.youtube.com/watch?v=SY3y6zNTiLs&list=PLKtIcOP0Wv...
| (playlist with all of his animations)
|
| [3] https://kaiber.ai/ and others
|
| [4] Linkin Park https://www.youtube.com/watch?v=7NK_JOkuSVY
|
| [5] Magdalena Bay https://www.youtube.com/watch?v=dXLCHvRsgRQ
| bloomingkales wrote:
| This video was from three years ago:
| https://www.youtube.com/watch?v=6FN7fKlJcPE
|
| Didn't know we were that far along 3 years ago?
| echelon wrote:
| EbSynth was a sleeper hit and these guys could have built
| something huge. This predates Sora, Runway, D-ID, and
| everything else in the scene.
|
| https://www.youtube.com/watch?v=sP3u09_YFxk
| refulgentis wrote:
| Super quick browse...but...IMHO, aren't video diffusion models
| and ebsynth apples and oranges?
|
| ebsynth looks like final cut pro plugins for masking and
| tracking. I could see the projected customer base being
| similar, but the skillset & resources required from your
| engineering team is significantly different.
|
| > Secret Weapons was [set] to _conquer_ AI video and _AI
| creative tools_
|
| Oh goodness, I think we both agree that's a bit much :)
| neom wrote:
| "Colors in painting software do not act like real pigments. Why?
| Because there was no practical way to implement true pigment
| mixing into digital painting." - not sure that is accurate -
| digital tools were developed such that they colour mix not colour
| blend. I'm not sure it's so much as there was no framework for
| it, so much as if I want to blend 2 colours I'd mix them then use
| a blend mode on a layer. Traditionally most people working with
| pro imaging tools had a background in colour theory, I can't
| imagine working on imaging and not knowing blue and yellow is
| green, so if I want green on my mixed yellow and blue, I'd either
| draw on with green, or draw with a blend mode if I want the
| _exact_ green. Doesn 't make this any less cool, just saying'
| pedrovhb wrote:
| That's very interesting!
|
| My first thought, looking at the webpage: "Huh, that's neat. I
| didn't know that painting software didn't even attempt to do
| color mixing beyond naive interpolation, though I guess it
| figures; the physics behind all the light stuff must be fairly
| gnarly, and there's a lot of information lost in RGB that
| probably can't be just reconstructed."
|
| Scrolling down a bit: "Huh, there's some snippets for using it as
| a library. Wait, it does operations in RGB? What's going on
| here?"
|
| Finally, clicking the paper link, I found the interesting bit:
| "We achieve this by establishing a latent color space, where RGB
| colors are represented as mixtures of primary pigments together
| with additive residuals. The latents can be manipulated with
| linear operations, leading to expected, plausible results."
|
| That's very clever, and seems like a great use for modern machine
| learning techniques outside the fashionable realm of language
| models. It uses perceptual color spaces internally too, and
| physics based priors. All around very technically impressive and
| beautiful piece of work.
|
| It rhymes with an idea that's been floating in my head for a bit
| - would generative image models, or image encoder models, work
| better if rather than rgb, we fed them with wavelength data, or
| at least a perceptually uniform color space? Seems it'd be closer
| to truth than arbitrarily using the wavelengths our cone cells
| happen to respond to (and roughly, at that).
| raincole wrote:
| As far as I know, most painting apps mix color in sRGB space
| instead of the linear RGB space. Which means they're even
| "worse" than naive interpolation.
| subb wrote:
| Consider that just after the cone cells, there are other cells
| doing some computation / aggregate of cone signals. Don't
| forget that color is a brain construct.
|
| For those reasons (and others), there's often a strong
| disconnect between stimuli and perception, which means there's
| no such thing as a perceptual uniform color space.
| geon wrote:
| The eyes even do edge detection before sending signals to the
| brain.
| brookst wrote:
| Is this correct? I thought edge detection was done in the
| primary visual cortex.
| subb wrote:
| Sidestepping what is defined as an "edge", quite a lot of
| work is done in the retina, including differential
| computation across cones - some "aggregator" cells will
| fire when it detect lines, movement, etc.
|
| You can read on ganglion cells, bipolar cells and
| amacrine cells and see that a lot of preprocessing is
| done before even reaching the brain!
| code_biologist wrote:
| Here's a detailed 17 min video presentation of the approach,
| from SIGGRAPH Asia 2021:
| https://www.youtube.com/watch?v=_qa5iWdfNKg
|
| Very fun stuff if you work with both digital (additive) and
| physical (subtractive) colors.
| pjlegato wrote:
| AI and machine learning aren't necessary at all. You 'just'
| have to empirically measure a few constants that describe and
| bound the various nonlinearities of different real pigments,
| and then plug them into a relatively straightforward paint
| mixing equation.
|
| Paints have predictable mathematical properties in terms of
| what color they produce when mixed; they just mix nonlinearly,
| which is counterintuitive for people who have not practiced
| mixing paint a lot.
|
| Photoshop and the other comparison programs on the page
| illustrate the linear mixing that most people intuitively
| expect.
| NKosmatos wrote:
| Wow, this is really cool color mixing! It's strange how come they
| haven't been bought (yet) by a big company like Adobe.
| rgovostes wrote:
| It's ancient history now but in 2012, the team behind the Paper
| drawing app for iPad shared interesting details about their color
| mixing: https://www.fastcompany.com/3002676/magical-tech-behind-
| pape...
|
| In short, they first implemented realistic pigment mixing
| behavior. But in user testing it turned out non-painters are not
| good at getting the color they want by mixing; you usually end up
| with brown. So they had to make an _un_ realistic mixing
| algorithm that felt more intuitive.
| ricardobeat wrote:
| That article is gold.
|
| While this work is impressive, two things to consider:
|
| - Mixing oil paints is a learned skill. They don't behave
| intuitively, so this will only going to be useful for people
| who already paint in physical media (or want to learn that
| specifically)
|
| - ArtRage has arguably been the most realistic painting
| software for over a decade, It has a _real color mixing_ mode
| that has to be enabled (probably not the default for the
| aforementioned reasons), and gives much closer results [1] than
| what they have in the samples page. Yet they barely show it in
| the comparisons.
|
| [1] https://i.ibb.co/T0GwbDV/artrage-mixing.jpg (right: real
| color blending on)
| dcrazy wrote:
| Thank you, I knew I'd used real color mixing in multiple apps
| for years. Secret Weapons is making a categorically false
| claim to have developed the first ever true-to-life color
| mixing algorithm.
| blacklion wrote:
| I remember that in Windows 3.0 time there was "Fractal Designer
| Painter" software, which had different engines to precisely
| emulate different types of painting materials: pastel, oil, water
| color, gouache. It imitaded mixing, stroke, pressure, etc. It
| looks incredible realistic in these times when 800x600x24bit
| screen was a luxury. And it was very slow on i486 computer.
|
| As I learned several minutes ago now it is Corel Painter.
|
| I wonder, is it still good and how it (both old and new
| algorithms, I allow that it was simplified down the road!)
| compares to this new library.
|
| Unfortunately, authors doesn't compare themselves with Corel
| Painter.
| kevin_thibedeau wrote:
| I have the 2015 version. It still works better than ever on
| modern PCs.
| codingdave wrote:
| Yep, it has changed hands a few times, but Painter still
| exists, gets updated yearly, and does everything I saw
| described in the article. I've been using Painter since the 90s
| for digital art specifically because of how well the color
| mixing works and how well it matches real art tools - not just
| various paints, but pastels, pencil, pens, etc. Tons of
| options.
|
| Now, all that doesn't come as a library for your own apps -
| that is definitely new and sounds like a great thing. But the
| implication that it has never been done before it quite wrong.
| bfrog wrote:
| Doesn't Krita do this?
| samlinnfer wrote:
| This has been around for a few years now, but it is doomed by a
| non-commerical/restrictive license.
|
| https://krita-artists.org/t/implementing-mixbox-mixing-for-k...
| jsheard wrote:
| Doomed for open source projects perhaps but they are open to
| commercial licensing, Rebelle uses it.
| rendaw wrote:
| As does ArtRage?
| https://www.deviantart.com/artrageteam/art/Realistic-
| Color-B...
|
| MyPaint also has it though, and that's open source:
| https://mypaint.app/blog/2019/01/26/MyPaint-2.0-alpha/
| (https://github.com/mypaint/libmypaint/pull/56 mentions using
| 7 primaries for blending), and another older thread about
| other methods: https://community.mypaint.app/t/real-color-
| blending-wip/390 . MyPaint IIUC is integrated into Gimp, so I
| assume Gimp can also do it using the MyPaint brush engine.
|
| Edit: I was responding to TFA's claim "there was no practical
| way to implement true pigment mixing into digital painting.
| Until now." but this page isn't dated - seeing how many other
| discussions reference MixBox blending could this need a
| [20XX] tag? I don't think it's new.
| lidavidm wrote:
| There's a PR implementing a similar approach. Seems it could
| use help to work out some kinks in the color mixing.
|
| https://invent.kde.org/graphics/krita/-/merge_requests/1783
| https://krita-artists.org/t/can-we-get-mixbox-on-krita/64201...
| moralestapia wrote:
| You cannot IP natural phenomena so that license is void.
| GuB-42 wrote:
| The license is for the library, not for the algorithm. You
| can't license algorithms, only their implementation, and
| that's the case here.
|
| The algorithm could be patented, I guess, but it doesn't seem
| to be.
|
| The algorithm is not the natural phenomenon of pigment
| mixing. It is about how to mix RGB values in a way that
| imitates natural pigment mixing, which is not something
| natural as nature doesn't work in RGB.
| a1o wrote:
| The implementation looks fairly small unless I am missing
| something
|
| https://github.com/scrtwpns/mixbox/blob/master/cpp/mixbox.c
| p...
|
| Like, it looks like just decompress that lut and have it
| decompressed in the project and the rest is just
| interpolation algorithms.
| morningsam wrote:
| I was wondering what the catch was :-)
|
| GPLv3 reimplementation linked in that thread:
| https://github.com/joeedh/pigment-painter
| bj-rn wrote:
| There is also spectral.js which is MIT:
| https://github.com/rvanwijnen/spectral.js
| andybak wrote:
| I added a comment to the issue here:
| https://github.com/scrtwpns/mixbox/issues/15
| mkl wrote:
| Previous discussions, from 3 years ago:
|
| https://news.ycombinator.com/item?id=30116316 (71 points, 15
| comments)
|
| https://news.ycombinator.com/item?id=30033611 (60 points, 13
| comments)
|
| Has anyone tried Rebelle, the commercial painting app that uses
| this method?
| Arcanum-XIII wrote:
| Rebelle is very good. It's fast, work fine at high resolution
| and so on. Less feature rich than painter (but that's a good
| thing)
|
| What's still missing : the proper feel for dry materials (like
| charcoal) and a deep integration with the surface of the
| support. Right now it's more or less a light depth map -
| nothing as gritty as wood, harsh paper (or even cloud ones)
|
| My favorite at the moment.
|
| And yeah, color mixing has been very good for a while there.
| ChrisMarshallNY wrote:
| That's great!
|
| That seems like something that Adobe would want to borg.
| Daub wrote:
| As someone who teaches both oil painting and digital painting, I
| find this project fascinating. They have certainly identified two
| of the key difference between digital and real paint:
|
| - RYB mixing (e.g. yellow painting plus blue paint = green). I
| tested it on a mix of alizarin crimson plus hansa yellow to
| produce a cold orange (e.g. blood orange).
|
| - Relative behavior of specific pigments. For example, a Prussian
| blue is almost black when applied thickly, but very chromatic
| when applied as thinly over white. A cerulean blue is pretty much
| the same in those two states. I tested this on a Prussian and the
| app performed well.
|
| The app seems to make the assumption that the artist will always
| want the paint to mix. In a glaze, a thin transparent layer of
| paint is applied over dry paint. Effectively the result is a
| filter (the result will always be darker). To a degree, this can
| be faked using paint applied in a layer the blend mode of which
| has been set to multiply. It might be nice to add a drying brush
| to the tool set which selectively dries the paint to prevent
| mixing. Maybe also a mode switch to selectively different modes
| of physical application such as glazing, scumbling and dry
| brushing. While we are at it, why not add a Tonk mode using which
| thick paint can be lifted off the surface without smudging.
|
| When I paint digitally, I mix the incredible nuance of real
| paint... even its smell. When I paint with oils I miss the
| incredible flexibility of the digital process (blend modes,
| compositing, even the humble undo). Never the twain...
| magicalhippo wrote:
| > It might be nice to add a drying brush to the tool set which
| selectively dries the paint to prevent mixing.
|
| The effect of thinning out the paint is effectively letting
| more light passing through the paint and reflecting off the
| canvas instead.
|
| So in the case of drying out, seems it would be a good fit for
| simply painting on a new layer, and having the code reflect off
| the combined layers below.
|
| That would make it easier to change the paint/glaze on top, as
| well as not having to commit to drying the layer underneath.
| swayvil wrote:
| I think that the colors here (in this screen-world) are 99%
| mental. Like everything else here. Which is to say, it is not the
| configuration of the dots or their color that significate, it is
| the meanings and memories that they evoke. 1% of the game happens
| on the screen and the other 99% happens inside your head.
|
| Which makes the whole thing a bit solipsistic when you think
| about it.
| printrrr wrote:
| Rebelle does this.
|
| Here it is in 2021:
|
| https://www.escapemotions.com/blog/rebelle-5-meet-color-pigm...
|
| And here it is currently:
|
| https://escapemotions.com/products/rebelle/manual/starting-p...
| chefandy wrote:
| I use Rebelle (which uses this) for all of my digital painting.
| It's _excellent_. They do need a new UI framework and they say
| they're working on it.
| CapsAdmin wrote:
| While this is really cool and should be an option in all painting
| programs, a part of me wonders if this complicates the learning
| process of painting?
|
| After all, mixing paint colors is something you have to learn
| because it's not very intuitive.
|
| Almost everywhere else in life, when you mix A and B, you expect
| a linear transition to happen. By adding salt to water, you are
| increasing its saltiness, it's not expected that at some point,
| the water becomes sour before it becomes salty again.
| itronitron wrote:
| I think it would simplify the transition for a person moving
| between physical media and digital, and vice versa. The mixing
| examples they show on the webpage for the Mixbox algorithm look
| real to me and make the mixing available in other programs seem
| incorrect.
|
| edit: I'll add though that Mixbox doesn't seem to take actual
| physical pigment characteristics into account and that can
| radically impact pigment mixing.
| mdtrooper wrote:
| I have been reading the license, it is CC by-nc then it is not
| free software.
| woolion wrote:
| Why isn't this more widely used? First, we have to understand
| that digital painting is a fundamentally different beast than
| traditional, in how you can use the tool, what you need to know,
| and how you learn it.
|
| In traditional, you tend to have some rules that make it easy to
| get good colors. One method is to restrict your paints to a few
| primary colors and burnt sienna (or another low saturation
| brown). Then by mixing these colors you get the colors you want,
| in a way where these colors have a good relationship with the
| others. So a simple rule is for instance that you get your green
| by putting lemon yellow with some black and a bit of sienna to
| lower the saturation. The relationship is almost 'free' from the
| properties of the physical pigments if you follow such simple
| rules.
|
| What's the deal about 'relationship'? Colors are not treated
| individually, they are perceived in context. In real life,
| lightning will unify a scene by changing colors. However, there
| is a semantic superposition that affects it. Grass is green in
| your mind, so if the color you see is a light blue/purple, the
| brain will adjust its understanding to match it. This is where
| many illusions come from (most famously the white and gold/blue
| and black dress meme come from).
|
| This is a big deal to learn painting, but can be almost
| completely side-stepped by learning how to mix colors.
|
| In digital however, artists got used to the fact that color is
| generally treated as HSV and adjusted their approach. So the most
| commonly taught approach to manage colors is to work in black and
| white, only dealing with value, which is the most essential
| component of color. Values are the primary issue because this is
| fundamentally what implies to our brain what are the normal
| vectors of light, and so carry almost all 'volume information'.
| Then, you can apply colors via some layers set to multiply, color
| and overlay mode. Note that it has some antecedent in the
| traditional world, called 'grisaille' technique.
|
| So, you can color pick anywhere you want on the color wheel
| without mixing. The impact of color mixing is actually pretty
| small. The other technique is called gamut masking; you're only
| allowed to pick colors on a subset of the color wheel that is in
| a good HSV relationship with the chosen color scheme. The thing
| is, even though this is not perceptually accurate, this is good
| enough.
|
| The main impact of color mixing is that if working with low
| opacity brushes, the colors will look 'muddy'. This problem is
| mostly side-stepped by avoiding it altogether with high-opacity
| brushes, and using other layer blending modes.
|
| Because now most digital artists have learnt through these
| techniques, bringing techniques that work best coming from the
| traditional world is not very useful, to say the least. In my own
| practice, traditional and digital are indeed almost entirely
| separate activities, at every level. The fundamental, theoretical
| understanding of all aspects is the same, but it doesn't really
| matter very much; almost all other artists I ever discuss with do
| not have such an understanding, and build their practice from a
| more 'human'-centric foundation (technique and artistic
| concepts).
|
| I'm of course interested in the space, and it is a bit similar to
| asking why would people would use bash or plain text; it's not
| necessarily that these are the best tools, but there is much more
| to it.
| mg wrote:
| What is the core of the problem?
|
| Is it that when you have two colors like RGB(0,0,255) and
| RGB(255,255,0) there is no easy formula to calculate what a mix
| of these two colors would look like in the real world?
|
| And therefore one needs a lookup table? And since 16Mx16M is a
| large number, a complete lookup table is impractical?
| mncharity wrote:
| I've an oddball interest in exploring what science education
| content might look like if someday crafted with implausibly
| extreme expertise and effort. Given that first-tier physical-
| sciences graduate students not uncommonly have foundational
| misconceptions around color, and color is commonly taught in K-2,
| I was exploring how one might emphasize spectra when teaching
| color, in preK to whatever. With interactives like mousing over
| hyperspectral images, a light/image manipulation workbench, and
| apropos topic, physically realistic Paint (with fluorescence,
| controllable lighting, glitter, blackbody, fire, make your own
| pigment, collage). Now community around that oddball interest has
| been hard to come by, especially no longer being around MIT. So
| in support of a "find your people - build your village" 2025 New
| Year's theme, if anyone knows of potentially overlapping
| interests...
___________________________________________________________________
(page generated 2024-12-30 23:02 UTC)