[HN Gopher] The Night Watch: The Missing Pieces
___________________________________________________________________
The Night Watch: The Missing Pieces
Author : BjoernKW
Score : 151 points
Date : 2021-06-24 09:33 UTC (13 hours ago)
(HTM) web link (www.rijksmuseum.nl)
(TXT) w3m dump (www.rijksmuseum.nl)
| Daub wrote:
| Art teacher speaking. Cropping is surprisingly easily to detect
| in an aesthetic image. Its kind of like reading a half-finished
| book. Aesthetic images are designed for an defined format. If
| that format is changed nearing completion, the geometry feels
| wrong. It is even easier in paintings, where there are overt
| brush marks. The structure of a brush mark is strongly framed by
| the format of the canvas.
|
| Anyone interested in the recreation of famous incomplete
| paintings, should check out Manet's The Execution of Emperor
| Maximilian. This is a painting Manet gave up on, and chopped up
| into smaller studies.
|
| https://www.nationalgallery.org.uk/media/33986/n-3294-00-000...
|
| There are sketches that indicate the format he was aiming for,
| but Manet was a famous experimenter, so we will probably never
| know.
| bruce343434 wrote:
| I was hoping for a before/after comparison picture, but I can't
| find it.
| appleflaxen wrote:
| For anyone else who wants context before watching:
|
| this is about the use of AI to reconstruct missing pieces of a
| Rembrandt painting ("Night Watch") using a contemporaneous copy
| from Lundens. After the Lundens painting was made, the Rembrandt
| original was cropped aggressively (~30 cm on a side) in order to
| fit onto a particular wall where it hung.
|
| Now that it is considered a masterpiece, they are trying to
| computationally estimate what those cropped sections looked like
| using Lunden's copy.
| DFHippie wrote:
| A vaguely similar anecdote:
|
| My grandfather, a taciturn, unsentimental Quaker mechanical
| engineer, was given the job of preparing a Methodist church to be
| a Quaker meetinghouse. The church had tall stained glass windows,
| but Quaker meetings are un- or minimally adorned. Well, Dan was a
| competent, efficient fellow. In short order the meeting was fit
| for occupancy again. The stained glass windows were gone except
| for a rose window. Some time after some of the Methodists came by
| asking what had become of the tall stained glass windows. They
| were memorial windows and they wanted to keep them as memorials.
| No one knew. Questions went around. Eventually someone asked Dan.
| He said they were under the gravel in the parking lot. What! Why
| are they there!? He just said, "they were ugly."
|
| I hope the spare pieces of the Night Watch weren't ugly, and if
| they were, the job of fitting them to the space wasn't given to
| some 18th century Dan.
| hutzlibu wrote:
| Or the elderly women, who "restored" a historical painting.
|
| https://en.wikipedia.org/wiki/Ecce_Homo_(Mart%C3%ADnez_and_G...
| MomoXenosaga wrote:
| It is entirely possible that at the time the Nightwatch was
| considered ugly. Let us not forget Rembrandt died an
| impoverished man.
|
| How art is appreciated changes with the times.
| coyotespike wrote:
| To be fair, Rembrandt was famous and very well-employed in
| his own day, he just spent money like it was going out of
| style. Had he saved a little he'd have died rich.
| stefanpie wrote:
| Robert Erdmann also gave a keynote talk at PyCon US 2021 about
| how they used Python for most of digitization and processing of
| the Night Watch (aka Operation Night Watch). Fascinating talk
| with tones of technical details.
|
| Link:
| https://www.youtube.com/watch?v=z_hm5oX7ZlE&list=PL2Uw4_HvXq...
| ElViajero wrote:
| This is a cool project. The part that I am not so sure about is
| why don't they just use an artist to paint the missing part? What
| is the advantage of using an AI?
|
| My guess, even that I hope not, is that people thinks that AI is
| going to be more impartial, more accurate, less biased. The
| reality is that an AI, as used in this example, is going to be
| over-fitting.
|
| An artist can understand Lunden and Rembrant art and use that
| knowledge to produce something closer to the intention of the
| artist and the copy.
|
| Cool project, but a dangerous precedent if people thinks that AI
| provides unbiased reality, that using AI you can show "how it
| really looked" as something more accurate that any human con
| achieve. Constructing something from missing data is always
| guesswork, whenever the intelligence is natural or artificial.
| treve wrote:
| They mention in the video that they could ask a person to make
| the changes with Photoshop, but the reason they give is: "Then
| it would be that artist's work, not Rembrandt".
|
| That statement rubs me the wrong way a bit though, because
| whether you ask a person or a computer model to copy
| Rembrandt's style, in each case it's still a copy, and I would
| argue in each case there's still a(n) artist(s) mimicking
| Rembrandt, just using different tools.
|
| > My guess, even that I hope not, is that people thinks that AI
| is going to be more impartial, more accurate, less biased. The
| reality is that an AI, as used in this example, is going to be
| over-fitting. > Cool project, but a dangerous precedent if
| people thinks that AI provides unbiased reality, that using AI
| you can show "how it really looked" as something more accurate
| that any human con achieve. Constructing something from missing
| data is always guesswork, whenever the intelligence is natural
| or artificial.
|
| I think this is an excellent take.
| floatrock wrote:
| I'm not gonna anthropomorphize the AI and say "the AI is the
| artist", but the AI is definitely an art _style_.
|
| I see it as just an old-meets-new mashup, craftfully executed.
| Any artist/technique you use isn't going to be the original
| Rembrant, so the choice of what artist/technique you use is
| just a question of aesthetics and taste.
|
| Here we're just exploring a new technique of the 21st century.
| ffhhj wrote:
| Let's make those paintings Ships of Theseus: recreate the missing
| parts around the paintings up to universe scale.
| shadeslayer_ wrote:
| Anyone else thought this was about the Night's Watch in the Game
| of Thrones books/novels?
|
| I'll see myself out..
| astrange wrote:
| Could've at least thought it was about a Dutch translation of
| the Terry Pratchett book.
| defaultname wrote:
| Very interesting watch.
|
| What sort of NN would be used to warp two compositionally similar
| images to match each other in structure? In the example the copy
| painting had almost all of the same general components, laid out
| with fairly significant differences, and they warped it to match
| quite precisely.
| danbruc wrote:
| Without a neural network one would probably use optical flow
| [1] and I guess you could train a neural network to predict
| optical flow and then use that flow to align the two inputs. I
| have no answer to your specific question, what kind of neural
| network architecture would be best suited but given the task it
| seems reasonable to assume some form of convolutional neural
| network.
|
| [1] https://en.wikipedia.org/wiki/Optical_flow
| jvanderbot wrote:
| I'm taking your question without context, so sorry if it
| doesn't make sense: Warping images to match based on structure
| / features is common in much of image processing and _requires_
| no NN since we know how to do it. (But may be enhanced by NN).
|
| The basic idea is:
|
| 1. Find features in images A and B (SURF / SIFT are common).
| However, maybe a NN could find interesting features here?
|
| 2. Match features so you know what feature from A is where in
| B. Again, a NN could potentially do this "better".
|
| 3. Compute a homography (translation between planes of
| features) or procrustes transform (or other types of
| transformations)
|
| 4. For each pixel (p) in the target frame (B), take the inverse
| of the transformation to find the pixel of the original frame
| (A) (which is usually a fractional pixel because it won't line
| up)
|
| 5. interpolate between nearby pixels in (A) to get the "true"
| pixel value at (p) in (B). Perhaps a NN could improve this
| interpolation? We've seen this in super resolution, for
| example.
|
| You can easily reproduce these steps with just a few function
| calls in OpenCV. Try it out. https://opencv.org/
|
| Up to step 3 is very helpful to answer questions like: "How
| much did the camera move between captures of image A and B. So
| now you can solve structure from motion, visual odometry, etc.
| This is typically how quadrotors navigate with cameras.
| danbruc wrote:
| Would SIFT or SURF features actually work in this scenario,
| that was also one of my thoughts, but then I considered that
| the inputs are not photographs of the same objects from
| different angles but a painting and a manual reproduction of
| it. I would be really interested in knowing whether those
| features are robust enough under the distortions in such a
| scenario. And how far could one push this, could you still
| align the Mona Lisa with a good pencil drawing of it?
| jvanderbot wrote:
| I have no idea! Careful feature selection is very
| important. If you only have edge information, then
| extracting edges in both images seems like a reasonable
| pre-processing step.
| dominicjj wrote:
| That is brilliant. I saw the Night Watch at the end of a busy day
| and it was pretty much the only thing I had time for in the Rijks
| Museum. I will have to return to Amsterdam and see the expanded
| version.
| BrandoElFollito wrote:
| The first time I went to the Rijks Museum I almost missed it.
|
| I saw on the left side of the main gallery (this is from
| memory, 30 years back) a small painting (say, 40x20 cm) which
| had a name similar (or the same) as The Night Watch and I
| thought that it is a great deal for such a nice but average
| painting.
|
| Then I saw some people standing in the middle of the gallery
| watching something and I discovered the painting :)
| BjoernKW wrote:
| The impact also depends on the direction from which one
| approaches the painting.
|
| If you enter from the room to the right hand side of the
| Night Watch it's almost easy to miss, not least because the
| ship models in the rooms that precede it are quite
| impressive, too.
|
| From the Great Hall and the Eregalerij (Gallery of Honour)
| the effect is positively awe-inspiring. I've once heard
| someone describe the Rijksmuseum as a cathedral of
| Enlightenment. The Night Watch is the centrepiece that
| cathedral was built around.
___________________________________________________________________
(page generated 2021-06-24 23:01 UTC)