[HN Gopher] Show HN: Controlling 3D models with voice and hand g...
       ___________________________________________________________________
        
       Show HN: Controlling 3D models with voice and hand gestures
        
       I'm sharing my project to control 3D models with voice commands and
       hand gestures:  - use voice commands to change interaction mode
       (drag, rotate, scale, animate)  - use hand gestures to control the
       3D model  - drag/drop to import other models (only GLTF format
       supported for now)  Created using threejs, mediapipe, web speech
       API, rosebud AI, and Quaternius 3D models  Githhub repo:
       https://github.com/collidingScopes/3d-model-playground  Demo:
       https://xcancel.com/measure_plan/status/1929900748235550912  I'd
       love to get your feedback! Thank you
        
       Author : getToTheChopin
       Score  : 75 points
       Date   : 2025-06-03 14:45 UTC (8 hours ago)
        
 (HTM) web link (github.com)
 (TXT) w3m dump (github.com)
        
       | getToTheChopin wrote:
       | Here's a quick video demo showing how it works:
       | https://x.com/measure_plan/status/1929900748235550912
        
         | pvg wrote:
         | https://xcancel.com/measure_plan/status/1929900748235550912 for
         | the tweetless
        
       | xnx wrote:
       | Sounds very cool, but I could not make sense of the on-screen
       | instructions. Some images or animations would go a long way to
       | explain the controls.
        
         | getToTheChopin wrote:
         | Sorry about that, the instructions need to be improved.
         | 
         | Does this video demo help?
         | 
         | https://x.com/measure_plan/status/1929900748235550912
         | 
         | If it makes it clearer, I'll upload it to the github repo
         | directly
        
           | xnx wrote:
           | That video did help. I think I was thrown off by two things:
           | 1) I was expecting 3D controls with more direct mapping (e.g.
           | rotating my hand rotates the model). This is more like
           | gesture mouse controls. 2) Some of the controls were too
           | subtle. The scaling between my gesture size and effect on
           | screen was smaller than I expected.
           | 
           | Great area to develop though. There's so much untapped
           | potential in applying Mediapipe.
        
             | getToTheChopin wrote:
             | Thank you for the feedback. I'll continue to work on it!
        
       | ednite wrote:
       | Great job! Looks very useful for interactive content creations
       | and product showcasing. Definitely will testing it more. Thanks
       | for sharing.
        
         | getToTheChopin wrote:
         | yes I'd love to go further with this concept so that 3D / CAD
         | designers could easily present their models during video calls.
         | 
         | thank you!
        
       | corysama wrote:
       | I understand you need your face in the videos for the demos. But,
       | want to mention that you should make sure your system works with
       | your hands in your lap. As shown, the user is going to experience
       | "gorilla arm" fatigue very quickly.
        
         | getToTheChopin wrote:
         | Good points, maybe a second camera (phone?) pointed downwards
         | at the tabletop would be good for that. Then the user can rest
         | their hands in a "normal" position.
         | 
         | Thank you for the feedback!
        
       | doakes wrote:
       | Very cool! I like the different modes. I've always been
       | fascinated with this space and products like Leap Motion:
       | https://www.youtube.com/watch?v=zXghYjh6Gro
        
         | getToTheChopin wrote:
         | It's a shame that computer vision tech like Leap / Eyetoy /
         | Kinect didn't have lasting power. So much fun to build with
        
       | danvoell wrote:
       | Awesome, nice work! This type of tech opens up a world of
       | physical games.
        
         | getToTheChopin wrote:
         | I made a demo game where you need to dodge the evil bouncing
         | DVD logo by moving your body:
         | https://x.com/measure_plan/status/1924830500541157570
         | 
         | I'm working on a couple other body movement concepts and hope
         | to share soon :)
        
           | hulium wrote:
           | See also Webcam Mania: https://webcam.sulat.net/
           | 
           | It uses a bit simpler approach, only detecting movement, but
           | it works good enough for such games.
        
       | ugh123 wrote:
       | Amazing! Maybe use specific finger positions/gestures to trigger
       | a rotation and scale functions (index finger up and within a
       | bounding box of the model perhaps for rotation, similar for pinch
       | to fingers to scale).
        
         | getToTheChopin wrote:
         | I'll try it, thank you! I separated them into completely
         | different interaction modes to avoid misfires, but there's
         | definitely room for efficiencies
        
       ___________________________________________________________________
       (page generated 2025-06-03 23:00 UTC)