[HN Gopher] Piano on Paper
___________________________________________________________________
Piano on Paper
Author : thunderbong
Score : 194 points
Date : 2024-03-08 18:10 UTC (1 days ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| elwell wrote:
| Obviously a bit rough, but a start. This is the kind of content I
| like on HN.
| mayureshsatam wrote:
| yeah! it was a hella challenging to detect touch.
| tmountain wrote:
| I had an idea for a similar project. Basically, a tabletop
| drum kit. In this case, you'd tap on a few objects on the
| table that make distinct sounds and then map that back to a
| virtual drum kit in your DAW using midi. I never got around
| to building it, but I am still keen on the idea.
| timlod wrote:
| That already exists in some forms, most notably perhaps in
| Dubler 2. I'm also working on an open source version of
| something similar, albeit quite involved and aiming at
| professional real-time performance and drumkit augmentation
| (if all goes well to share on HN this month for the first
| time :))
| mayureshsatam wrote:
| Hey that is cool! lemme know if you need any help, send
| mail if required!
| iefbr14 wrote:
| Reminds me of virtual laser keyboards, eg:
| https://nerdknowbetter.com/best-virtual-laser-keyboards
| userbinator wrote:
| I'm surprised they actually exist instead of being a rendered
| design concept that I saw many years ago, and are apparently
| common and numerous enough that someone would write a listicle
| about them. They don't seem to cost more than a good regular
| keyboard either, but I've found reviews saying that the typing
| experience is about as uncomfortable as you'd expect from
| tapping on a hard surface.
| robinsonb5 wrote:
| The typing experience is
| <chyrosran22>Hhhhhhhideous</chyrosran22>[1]
|
| [1] https://www.youtube.com/watch?v=1Cy1dPlo8No (NSFW)
| latchkey wrote:
| Previously:
|
| _The pianist who learnt to play on a paper piano_
| https://news.ycombinator.com/item?id=21823936 (December 18, 2019
| -- 1 points, 0 comments)
|
| It is a video of a guy in 2019 who couldn't afford a piano, so he
| drew it out on paper and learned that way and became an award
| winning musician.
| macintux wrote:
| Obviously in no way comparable, but you reminded me that many,
| many years ago, when I was taking typing in high school, I'd
| sit in class "typing" in mid-air (hands at my side)
| transcribing what the teacher was saying.
|
| It was a little annoying to people around me, but ended up
| being very effective.
| dhosek wrote:
| There was a scene like this in the Talented Mr Ripley (1999).
| a1o wrote:
| This is pretty cool work, love the concept. I am curious if there
| are fast, easy to use and efficient ways to also detect the full
| hand. Maybe by placing the camera from above would make it easier
| to process but at the cost of complicating the setup.
| proee wrote:
| When I was in 3rd grade we learned to type on a "keyboard" that
| was printed on a sheet of paper. I suppose it's a novel way to
| teach kids to type with a budget of zero.
| dhosek wrote:
| When I was in high school in the 80s, there was a piano class
| at the high school and the students had non-sounding, non-
| moving plastic piano keyboards to practice on at their desks.
| Nowadays, they get 60-key keyboards with headphones.
| devsda wrote:
| I know that this is better than nothing, but as I understand,
| precise fingering is an important part of learning piano.
|
| Is there a risk of kids picking up wrong habits on a paper
| compared to a (cheap) real one ?
| xandrius wrote:
| I guess it's always better bad habits than no habits at all.
|
| The context in which this is done is rather important. Also,
| it might prevent having a ton of lost and worthless keyboards
| in everyone's attic :D
| persnickety wrote:
| I have a long password which I don't remember. I can type it on
| my laptop, but not recall it.
|
| Last holiday season I unexpectedly needed to access my master
| keyring, but all I had with me was my phone. Which doesn't have
| a laptop-sized or -shaped keyboard.
|
| No amount of conscious typing helped me remember. As a last-
| ditch effort, I found squared paper and a pen, drew a full-
| sized keyboard, and "typed" my password in on that.
|
| It took a couple of attempts, but I did eventually reverse-
| engineer my own password from my own muscle memory by carefully
| observing what my hands are doing.
| zarzavat wrote:
| You should write it down. I had such a password that only my
| fingers knew, not my head. Then I went a couple of months
| without using it. When I wanted to type the password again, I
| had completely forgotten it. Without even knowing the first
| letter, I had no hope of guessing it.
| p1mrx wrote:
| I think this idea would work better using 3D printed keys. It's
| possible to print flexible springs, and filament can be used as a
| hinge pin. The computer just needs to recognize which keys have
| moved.
| mayureshsatam wrote:
| yeah! but main motive was to make it as free as possible.
| Someone like me who does not have much of money to buy a piano
| but to play one see this as absolute win with trade offs ofc!
| p1mrx wrote:
| I just made this piano key; the cost of filament is around
| $0.15: https://files.catbox.moe/kvop4m.webm
|
| So it's in the ballpark of a couple dollars per octave. I
| agree that's not very competitive with paper.
| mayureshsatam wrote:
| That's cool, Sounds good. I bet you can sell keyboards with
| that sound.
| pama wrote:
| I've often wondered about the reverse. Get the finest grand piano
| action, like the ones used in the Fazioli, and create a minimal
| mechanical unit that has just that piece and the pedals (with the
| action mechanics that connect the left/right pedals to the keys)
| and then nothing else but a couple of sensors and a couple fast
| USB-C. The Kawai Novus series did an OK action and left pedal,
| but then they connect it to a whole furniture and their sound
| synthesis and vibrations are imperfect. With modern powerful GPU
| the sound should be possible to become near perfect and easy to
| tune to one's preference.
| SamBam wrote:
| I am no master pianist, but I imagine much of the feel of such
| an instrument comes from the physical weight of the keys, and
| the feel as they strike the strings. Although good quality
| keyboards have "weighted" keys, they don't feel exactly the
| same. I think it would be hard to reproduce this exactly.
| pama wrote:
| I agree. Perhaps I was unclear. I was hoping that someone can
| use the exact action of the grand piano, but skip the
| strings, soundboard and so on.
| ksherlock wrote:
| craigslist is full of free upright pianos and occasionally
| free baby grands. Of course if you show up with a chainsaw
| instead of 5 men and a dolly...
| pclmulqdq wrote:
| The top-end Yamaha electric pianos have a near-complete
| replica of the action inside of them. If you adjust the
| volume exactly right, you almost replicate the feel of
| playing a real piano.
| ta988 wrote:
| GPUs may not be great for latency.
| pama wrote:
| You may have a point. Perhaps naively I imagined that if
| competitive gamers can handle it, they may be useful enough
| for piano. Currently the CPU software is lacking in quality
| (though improving over time) but has acceptable latency
| compared to grands (which also have latency due to the
| mechanism and the location of the sound source).
| throw5323446 wrote:
| Kinda like a silent piano? There are aftermarket options like
| PianoDisc
|
| https://en.wikipedia.org/wiki/Silent_piano
| pama wrote:
| Yes but without the need for the full furniture. One should
| be able to miniaturize everything else but the action. I
| think such compact designs will eventually appear but the
| difference in real action versus very good hybrid digital
| actions is of highest interest to musicians who typically
| have access to real pianos. Roland, Yamaha, Kawai, and others
| have been moving in that direction over the years and the top
| models by Yamaha and Kawai have passable actions though they
| still have a large piece of furniture around them.
| mastazi wrote:
| Very high end digital pianos have an action that is quite
| similar to a real grand.
|
| This is a relatively cheap digital piano action
| https://www.youtube.com/watch?v=S_VT0IpM66k
|
| These are 2 versions of a mid-high range digital piano action
| https://www.youtube.com/watch?v=rJEAKI2jBLU
|
| This is a very high end digital piano action
| https://kawaius.com/wp-content/uploads/2021/06/Kawai-NV10S-M...
|
| That very same action is also used in acoustic pianos, and the
| acoustic version looks like this (you can see the main
| difference is that the hammers have felt)
| https://snadenspianos.com.au/wp-content/uploads/2018/05/MG_9...
|
| And this is another (more traditional) acoustic grand piano
| action https://www.youtube.com/watch?v=v5O1Kfd_4Fw
| bradleyishungry wrote:
| The kawai vpc is pretty much that, right? You don't need fast
| usbc for audio and it's the best action on a midi controller
| tgv wrote:
| A large part of the experience comes from the actual sound.
| GPUs don't matter. Mozart has some very good physically modeled
| pianos that respond very naturally. But to get that to sound
| like a real piano in a real space is a greater challenge than
| reproducing the feel of Fazioli.
| yakito wrote:
| I've previously used PianoVision, and it worked quite well. The
| concept seems similar, now augmented with VR. I anticipate that
| VR pianos will evolve to such an extent that they'll replace
| traditional pianos in many scenarios (like teaching/learning)
| https://www.meta.com/experiences/5271074762922599/
| josephg wrote:
| > they'll replace traditional pianos in many scenarios
|
| I'm not convinced. Totally VR pianos are obviously a bad idea
| just like totally VR keyboards for typing are rubbish for
| anything more complex than a wifi password.
|
| And as for learning using VR - well, I'm not convinced by
| things like pianovision. How do you learn a piece with that?
| With sheet music I can play the music at my own speed, when and
| how I want. I can go back and play a measure a few times to
| experiment with timing. Speed up and slow down, and develop my
| own ear for how I want it to sound. If I'm feeling cheeky, I'll
| improvise extra notes in the song, or take some notes out if
| I'm not feeling it. At an instrument, sheet music is a polite
| suggestion.
|
| I can imagine playing something like pianovision and getting
| good at pushing the buttons at the right time like guitar hero.
| But being a human shaped CD player feels like a very small part
| of what makes a musician.
| superb_dev wrote:
| Getting good at pushing the buttons is exactly it. Just being
| able to hit the note that you want to hit on an instrument is
| a skill
| m3at wrote:
| I agree for pianos, because the mechanical feedback is part
| of what makes playing enjoyable.
|
| I'm not as sure for keyboards in VR! There has been a lot of
| research on non-invasive brain computer interface (BCI),
| including predictive systems that guess what you _want_ to
| type, instead of where you 're actually typing (example from
| 8y ago [1]).
|
| Simpler gesture recognition is already on the market (like
| this one, from 2020, for the apple watch [2]). And now bigger
| VR players are investing in the tech [3]. I expect useful
| brain interface to be integrated in common VR devices in a
| couple of generations.
|
| [1] Air Keyboard: Mid-Air Text Input Using Wearable EMG
| Sensors and a Predictive Text Modeland a Predictive Text
| Model; https://digitalcommons.dartmouth.edu/cgi/viewcontent.c
| gi?art... [2] https://mudra-band.com/ [3]
| https://www.androidcentral.com/gaming/virtual-
| reality/zucker...
| andoando wrote:
| The weight of the keys are an extremely important factor.
| Pianos and even good digital pianos have hammer action keys.
|
| You have no way of control the volume with this.
|
| I don't think digital will ever replace a real piano.
| andoando wrote:
| Wanted to add:
|
| This isn't to say that something like this can't make its own
| cool instrument, but as to learning the piano, no matter how
| good the tech is, its always going to be an imitation of the
| real thing. Which isn't to say that its useless for learning
| piano either.
| ludston wrote:
| What? Why would they? Digital piano sounds still can't
| reproduce the richness of dynamics and texture of an acoustic
| piano. Practising on a digital keyboard and then performing on
| a real piano is jarring enough, let alone using an instrument
| with zero haptic feedback. This is pure gimmick.
| two_handfuls wrote:
| This requirement is logical but also.. ouch
|
| > Hand with all fingers.
| abrichr wrote:
| Congratulations on shipping!
|
| You might find this interesting, a comment on
| https://news.ycombinator.com/item?id=35376138
| (https://news.ycombinator.com/item?id=35379983):
|
| > I had been playing with the idea of creating a browser-based
| virtual piano for when I'm travelling and don't have access to a
| real piano but have my laptop with me. The idea would be to point
| the webcam down at the table between me and the laptop, and play
| on the table as if a piano were there. Then use the mediapipe
| framework [1] to capture finger positions, and use those to
| update a virtual environment like the one you have here.
|
| > I put it on hold due to the significant engineering required,
| but it seems you have already implemented (and open sourced!) the
| browser-based piano simulation component.
|
| > A quick scan through your repo indicates that this is all
| implemented in Python. I see that you are using mujoco_wasm [2].
|
| [1] https://google.github.io/mediapipe/
|
| [2] https://github.com/zalo/mujoco_wasm
| ipeev wrote:
| Where is the file touch_detection_model.h5?
| mayureshsatam wrote:
| well actually you need to train the model then the file will be
| generated
| phlakaton wrote:
| I'm glad you're finding ways of getting into music inexpensively.
| But my advice is: haptics are super important in music
| interfaces! Spend the extra effort to make something that you can
| satisfyingly press and which can give your fingers some
| mechanical feedback. Your playing will be sooooo much more
| rewarding.
| mayureshsatam wrote:
| yeaa you are right, but those haptics cost money :(
| zerojames wrote:
| This is a delightful project. Congratulations on shipping it! I
| would be curious to see ways to do this where the piece of paper
| had lines so you could visually see on the paper which key was
| which.
|
| Doing this would be tricky, though. On first thought, an object
| detection model that detects each key from a perspective, then
| saves the coordinates, may work. Then you would calculate the
| Intersection over Union between the finger annd each region. This
| may only work with a top-down perspective, which might be less
| practical in real-world use.
|
| In any case, I always miss my keyboard when I am traveling. A
| project like this feels a lot more practical than the "portable"
| travel keyboards.
| mayureshsatam wrote:
| yess I am currently working on that too!
| alwinaugustin wrote:
| Appreciate your work. Happy coding !!
| SonOfLilit wrote:
| Mayuresh, I'd like to donate a keyboard instrument to you. Please
| choose between an AKAI MPK Mini and a Hohner Melodica and email
| me at my nick on gmail with a shipping address.
|
| Also, if you consider this effort much cheaper than a $100
| instrument, maybe I could hook you up with a programming job?
___________________________________________________________________
(page generated 2024-03-09 23:02 UTC)