[HN Gopher] Show HN: AI climbing coach - visualize how to climb ...
___________________________________________________________________
Show HN: AI climbing coach - visualize how to climb any route based
on your body
I made SABR - an AI model that helps you visualize the
beta/technique on any route, based on your body parameters. You can
input a video of you climbing any route, in any orientation or
lighting condition (it's truly versatile!). SABR then creates a
virtual avatar of your body shape and uses it to climb the route
you're climbing. Then, you can compare/contrast. You can see the
demo here: https://www.youtube.com/watch?v=cnvNPWoYZz4 Will be
open sourcing the model, backend, and frontend codebase soon!
Author : smandava
Score : 310 points
Date : 2024-05-06 08:09 UTC (3 days ago)
(HTM) web link (climbing.ai)
(TXT) w3m dump (climbing.ai)
| gus_massa wrote:
| Does the AI create the 3D model from a static camera?
|
| Can I choose the color of the avatar? (It's not an important
| feature, but people will love it.)
|
| Does it work with natural walls? (I guess the artificial walls
| have "handles" with an standard size that makes guesing the scale
| easier for the computer.)
| smandava wrote:
| - it creates a 3D model of you based on the first second of
| your body in the video. How the video is recorded does affect
| how the 3D avatar looks.
|
| - don't have that functionality but its really easy to
| implement.
|
| - ideally, it works for anything. I will attach some outputs of
| it climbing outdoor routes. The model doesn't know what a wall
| means. It just has seen enough data of people climbing that it
| can somehow correlate certain features in videos to certain
| human movements.
| gus_massa wrote:
| My guess is that for the computer it's easier in an
| artificial wall becasue features are standarized.
|
| (Disclaimer: I never climbed a wall, but from time to time I
| fall in the https://www.youtube.com/@AlbertOkay rabit hole.
| In particular this video looks relevant for my question
| https://www.youtube.com/watch?v=onXrJ5sX9BU (but I din't
| watch it yet.))
| smandava wrote:
| Yeah it performs best with indoor bouldering videos with
| spread out holds and great lighting. I will post examples
| of it climbing natural walls in the paper. Will send soon.
| smandava wrote:
| Note:
|
| Will attach more example outputs and make a detailed document
| about how the model was built and the research behind it. If this
| is interesting to you, feel free to sign up to the waitlist on
| www.climbing.ai (and make sure to sign up for our Discord!)
|
| I originally planned on open sourcing the model, data, weights,
| and code. Only a few people (me and a couple friends) have access
| to the hosted model on the web app.
|
| If enough people are on the waitlist, I will consider releasing
| access. This is very expensive to run so was only considering
| open sourcing it.
|
| Note: the model works well sometimes, but most of the times it
| does not. This is an early research preview. Please tamper any
| expectations. A really good general model requires millions of
| videos and much more training time so it is really prohibitively
| expensive. As mentioned before, if someone has the compute, all
| they have to do is scale the existing dataset and training
| pipeline (which I will publish open-source in the coming weeks).
| ethbr1 wrote:
| High-level details about how you tackled the various sub-
| problems would be useful.
|
| I assume, at minimum, there's: - Segmentation
| of moving elements - 3d reconstruction from 2d video
| - Reverse kinematics - Identification of holds -
| Search for potential climbing path - Generation of
| kinematics for path
| hirako2000 wrote:
| Likely looking to see a 100k emails list before really
| thinking about all these problems to solve and well.
|
| Worst case able to build this, scenario, author builds
| something else entirely.
| ethbr1 wrote:
| Afaict, author already built a proof of concept.
| icpmacdo wrote:
| I've been working on the same thing, there is an interesting
| quote from back in the day and its either make your program
| faster or wait 6 months for the computers to get faster. I look
| at it the same way now with Key Point Analysis and Pose
| Estimation libraries.
|
| I think OSS is best for this category right now.
| crabmusket wrote:
| *temper expectations!
| smandava wrote:
| typo :) thanks
| jfengel wrote:
| I look forward to playing with this. It seems astonishing.
| smandava wrote:
| I will be open sourcing the codebase and model. Due to compute
| and training dataset size constraints, the model is not exactly
| "production ready". I have (and am) cherry picking rare
| examples of it where it is decent. You can play around with it
| when I open source it but truly good results require multiple
| millions of dollars (but it really is a linear scaling of
| compute and dataset).
| goodmattg wrote:
| A few thoughts as I've done academic research & built products in
| this area:
|
| - if you're using SMPL body parameters this will have to stay
| research / open-source - is this leveraging some sort of
| monocular depth estimation to estimate the wall in 3D space?
| Also, do you have assumed camera parameters, or is that also
| estimated? If there isn't any depth information, this will be
| highly inaccurate on any cliff routes, but still useful on flat
| wall climbing.
|
| Overall, a good idea (that I've also thought about building as a
| climber) - the tricky part that I'm impressed you have a solution
| to is path planning up the wall. Even assuming a flat wall with
| no depth estimation, it's still looks effective.
| syntaxing wrote:
| Not OP, but why does it have to be open sourced? Copy left
| license?
| goodmattg wrote:
| commercial license - the research group formed a corporate
| entity that licenses the body model and all derived work
| (SMPL-X, etc.): https://meshcapade.com/SMPL
| ethbr1 wrote:
| > _the tricky part that I 'm impressed you have a solution to
| is path planning up the wall._
|
| I'm assuming this is evolutionary / brute force of some nature,
| given OP's comments about it being expensive to run.
| smandava wrote:
| Yes, this will be open source.
|
| This is an end-end system that just takes in video frames.
| Camera parameters are one of the things that is predicted. It
| gives promising results for a wide variety of environments
| (cliffs, diff types of bouldering walls, diff outdoor walls,
| etc.), though not always accurate. Path planning is also part
| of the end-end system. Will share more details in the paper.
| goodmattg wrote:
| sweet can't wait to read it! I'll also be at CVPR this year
| if you're presenting
| prions wrote:
| It looks really interesting, but as an experienced climber I'm
| not sure if just watching a video of my avatar climbing would
| really help with skill acquisition.
|
| Also, this claims that the wall type or video quality doesn't
| matter, but I have a hard time understanding how the model would
| be able to understand that a small crimp could possibly be dual
| textured and therefore has only a few specific ways of
| approaching it.
|
| So it seems that this is more for visualizing a climb (which is a
| skill most climbers should develop) and not really for dialing in
| some sort of microbeta for a problem.
| etrautmann wrote:
| Agreed - so much about the detail of how you would climb
| something comes down to details that would be hard to measure
| with a camera, like textures, your estimate of friction, etc.
| Very cool idea though, looks fun to test.
| sandworm101 wrote:
| And this is deep into "sport climbing", borderline gym rat
| territory imho. It doesn't model all the other core aspects
| such as protection, rope management, exposure and rest stops.
| I imaging if you pointed this at a real cliff and recorded
| several assents it would quickly become a blurry-twitchy mess
| as all the movements not touching the rock spoiled the data.
| Maybe for bouldering, but not for real climbing.
| soperj wrote:
| What makes rope climbing "real"?
| lambdaone wrote:
| There are lots more variables to consider, particularly
| in lead climbing, even when you have a bolted route. And
| trad climbing is even more complex than that.
| sandworm101 wrote:
| Trad lead is climbing. Basically everything else is some
| sort of simulation.
| et-al wrote:
| It's 2024 and some people are gonna gatekeep that if
| you're not shoving cams in some bigwall granite, it's not
| climbing?
| sandworm101 wrote:
| Such differences are actually at the heart of climbing.
| Is taking a helicopter up to the top climbing? Of course
| not. So from day one it is not about getting to the top
| but about following invented rules governing _how_ you
| get to the top. How about a bolted-on ladder? Or pre-
| placed protection (bolts)? Clean trad, leaving the rock
| as you found it, is generally seen as the highest form.
|
| (Climbing rock without ropes may be more "pure" but is so
| dangerous that it should never be idolized.)
| Intermernet wrote:
| Cams? Real climbers use stones wrapped with hawser!
| lambdaone wrote:
| This one-upmanship is very much part of climbing culture.
| From the top down, the hierarchy is: free soloists
| (climbing at height without any protection), trad
| climbers (carry the rope up with them, set their own
| protection), sports lead climbers (carry the rope up with
| them, bolts are set into the wall), top rope climbers
| (rope already there dangling from the top). Right down
| the bottom, you have aid climbing, when you use equipment
| to haul yourself up on a rope.
|
| And then there's indoors vs. outdoors, with some
| dedicated outdoor climbers regarding anything done in a
| climbing gym as "not real climbing".
|
| Most people don't take this literally, and it's generally
| considered to be a standing joke in climbing. Sadly,
| though, some people take it very seriously.
| sandworm101 wrote:
| Aid climbing isn't respected on smaller or well-
| established cliffs, but most understand that it has a
| place. All the great routes started as aid routes, only
| being climbed "free" years later. Aid is also a very
| useful skill in the rain or rescue situations when
| friction disappears.
| logtempo wrote:
| I don't see the shape of the holds being a big problem. With
| some help from indoor companies and hold makers, figuring out
| which hold model is on the wall should be possible.
|
| As for the usefullness of the software, I'm sceptical too as it
| don't really solve a problem. But maybe I'm not seeing it and
| it could be good for beginners :) A good improvement would be
| adding a comparison between you and the model in term of body
| position and fluidity of movements.
| party_possum wrote:
| The idea of incorporating actual hold data and "recognizing"
| specific holds is interesting, but I'm not sure it completely
| solves the problem.
|
| The "Boss" from Pusher is arguably the most famous climbing
| hold ever made. For a decade or more, every gym had one, but
| they were all unique. Lots of them had micro chips that
| became critical to usage of the hold. Some had decent texture
| and some were glassy smooth from years and years and years of
| use. A lot of the accidental variation in new holds has gone
| away as the industry has standardized around a handful of
| industrial fabricators like Aragon, but even over the course
| of a single indoor boulder problem's life, the accumulation
| of chalk, sweat, and shoe rubber can have a significant
| impact on how a hold climbs.
|
| I guess the real question is, do these changes just make
| routes _harder_ or do they make them fundamentally
| _different_? Do they actually change the set of moves that
| constitutes the easiest way to the top? To be honest, I 'm
| not entirely sure. But it's something interesting to think
| about.
| alexcp_ wrote:
| Exactly, holds will evolve as they get used and more
| polished, even indoors. Climbing a Moonboard with a new set
| of holds is quite different than climbing on one with older
| more polished holds, even if it's the exact same problem
| and the same holds.
|
| It's an interesting project and it could be fun to watch,
| but it's completely useless.
| ethbr1 wrote:
| Couldn't you reverse-reason about that?
|
| To me, the customer here would be climbing gyms, offering
| a service to climbers. 1. Set up camera
| on routes 2. Record all climbs 3. Reason
| through hold details 4. Generate potential
| movements 5. Show climbs vs ghost movements
| 6. Feedback to tune model
|
| 3 being accomplished by reasoning "if a movement should
| be possible using the identified hold, but no one
| successfully does it, the hold must be misidentified or
| have different properties."
| alexcp_ wrote:
| But what is the point? Finding the optimal movements that
| are needed to complete a climb is not useful if you are
| not strong enough to execute it.
| ethbr1 wrote:
| The point in this thread seemed to be "real world holds
| have different properties, and that defines possible
| approaches to holds."
|
| To which I pointed out that, with enough data, you could
| reason backwards to figure out their properties.
|
| Assuming that's solved, if the question is "What is the
| point?" then I'd answer the same point as golf swing
| analysis -- structured comparison feedback for continual
| improvement.
|
| "Have you thought about trying X move at Y point?" or
| "You're trying X move at Y point, but here's how you
| differ from someone successfully doing it" both seem
| useful feedback.
|
| And essentially what's manually generated now, from
| someone watching and then providing feedback.
|
| With regards to strength, hell, if you wanted to get
| fancy you could also deduce a specific user's strength,
| comparing their moves against others' moves on the same
| features.
| jfengel wrote:
| Huh. I recognize it but didn't know its name. (I don't know
| any of the names.) Route setting sounds fascinating.
| Zanfa wrote:
| > I don't see the shape of the holds being a big problem.
| With some help from indoor companies and hold makers,
| figuring out which hold model is on the wall should be
| possible.
|
| Even if you know the exact hold model and it's in pristine
| condition, it's basically impossible to tell how it's gonna
| work from a single angle on video at a distance. Even tiny
| variations in angle of the wall and rotation of the hold on
| the wall can completely change how you use it.
| Snackchez wrote:
| What could be interesting is if you could compare your attempts
| to the avatar climbing and receiving feedback afterwards. This
| would effectively be a step up from simply recording your send
| attempts.
| petsfed wrote:
| I suspect that, absent that information about the exact right
| way to grab a hold, or the exact way to put a foot on a hold,
| you'll be limited to beta _suggestions_ , which is fine, I
| think. It'd be like having a group of other climbers nearby to
| suggest different beta, even if you don't have any friends.
|
| So, in terms of solving complicated beta faster, I see real
| utility to this.
|
| It is very interesting that since the AI climber is trained on
| actual climbers, it could, in principle, provide beta to climb
| consistent with your own style. If you train the bot
| exclusively on footage of yourself, it would return movement
| based on your style. If your style is finessy-all-backstep-all-
| the-time (aka The Edlinger), it can provide beta consistent
| with that. If your style is to square up and pull (otherwise
| known as The American), it can provide beta consistent with
| that instead.
| smandava wrote:
| Correct
| bmj wrote:
| (Long time climber here)
|
| > It is very interesting that since the AI climber is trained
| on actual climbers, it could, in principle, provide beta to
| climb consistent with your own style. If you train the bot
| exclusively on footage of yourself, it would return movement
| based on your style. If your style is finessy-all-backstep-
| all-the-time (aka The Edlinger), it can provide beta
| consistent with that. If your style is to square up and pull
| (otherwise known as The American), it can provide beta
| consistent with that instead.
|
| I would think this is actually a Bad Thing. It's very easy to
| get stuck trying to make a sequence fit your style of
| climbing. The better approach (especially for long term skill
| acquisition) is a willingness to learn new styles. That's to
| say that every sequence is only solvable via one particular
| style, but I think long term development is hindered if you
| approach every crux with the one thing you are good at.
|
| > So, in terms of solving complicated beta faster, I see real
| utility to this.
|
| I can agree with this. But, to the point that others have
| made, I do wonder what this and the availability of beta
| videos for many, many routes and blocs does to climbing skill
| overall. Perhaps I'm just a grumpy old man, but, particularly
| when bouldering, sorting out the beta should be part of the
| journey toward eventually sending. Last fall, I visited Hueco
| Tanks after a six year absence. I suppose I was a bit
| disappointed to see so many people watching YouTube beta
| videos of nearly every problem they tried.
| petsfed wrote:
| >I can agree with this. But, to the point that others have
| made, I do wonder what this and the availability of beta
| videos for many, many routes and blocs does to climbing
| skill overall.
|
| That's a fair concern. That said, there are certain
| sequences that I'm physically incapable of doing without a
| dedicated multi-month program of stretching beforehand.
| Turns out falling on a thumbs-down jam is close enough to a
| shoulder dislocation that I maybe should've done some PT
| about it. The extreme is, obviously, the Edlinger vs.
| American example, but I think the middle ground actually
| addresses people's peculiar body geometry and/or range of
| motion. Alex Honnold's exact hip or elbow position might
| not be as meaningful for Ashima, even if they're on the
| exact same route.
|
| Such nuances have made a difference in some cases (the one
| that springs to mind is Todd Skinner's observation that
| Steve Petro's hips sagged just a little mid-crux on Fiddler
| on the Roof, which, until corrected, had prevented Petro
| from nabbing the first ascent of one of the hardest cracks
| ever climbed to that point). Probably a net-bad for folks
| projecting Midnight Lightning or similar, but definitely
| useful for somebody looking to repeat Silence or whatever.
| smandava wrote:
| To clarify:
|
| Of the full distribution of possible video qualities one can
| take on a modern phone camera, the vast majority of video
| qualities will be fine for the AI to understand fine details.
| Obviously, if you somehow or for some reason, take a video with
| really bad quality, it will not give you what you want.
|
| Same explanation goes for the walls. If you take a video of
| just a really dark wall with really bad holds, it is probably
| won't give you what you want either.
| piterrro wrote:
| Nice! Cant wait to try it!
| aconz2 wrote:
| Very interesting, do wish we could play with it since it is a
| show hn, maybe repost when it is open sourced/usable. Looking
| forward to reading about how it works. How does it know which
| holds are on? How does it know what the top is? Are those inputs
| or inferred or is it just end to end completing likely body
| positions one after the other?
| DanBC wrote:
| Be interesting to see if climbers like it - I'd love to see Wide
| Boys testing it out.
| greenie_beans wrote:
| cool technical project, but as a climber, part of the fun is
| solving the problem with the help of my own brain and the brains
| of the other climbers around me.
| kettunen wrote:
| One of the most fun part of climbing is the problem-solving
| aspect of it. Figuring your way to get up the wall/boulder. Why
| would you want to get rid of that?
| rokkamokka wrote:
| As with any tutor, this could (possibly) help beginners.
| _fs wrote:
| Do you enjoy climbing with others? Personally, I enjoy seeing
| the different beta on a climb, especially between different
| body types. It is interesting seeing unique approaches to the
| same problem.
| hacksoi wrote:
| You can use it after you do the climb to see if there is a more
| optimal solution.
| NegativeK wrote:
| Where the fun is is an opinion.
| crazygringo wrote:
| What about when you try to solve the problem and you get stuck
| and can't? Or solve it but in a bad way that requires way too
| much exertion?
|
| The point is to try to solve it yourself, then compare with an
| expert solution, and therefore learn how to improve.
|
| If you're just blindly trying to problem-solve through trial
| and error without ever comparing against expert feedback,
| you're going to learn climbling _extremely_ slowly...
| Tronno wrote:
| Getting stuck is part of the game. It could take several
| sessions or it could take several years. It's what makes a
| climb truly hard, a "project".
|
| It's an essential part of the sport - the satisfaction of
| using one's own body and mind to overcome the seemingly
| impossible.
|
| If you never get stuck, how could you possibly experience it?
| crazygringo wrote:
| Games are no fun if they're too hard.
|
| When faced with a challenge, you want to figure out as much
| as you reasonably can, and then learn what you were
| missing.
|
| Struggling at the same problem for several sessons, or god
| forbid _years_ , sounds like misery to me. I'd rather use
| that time productively learning, rather than struggling for
| the sake of it, because I refuse to learn from others.
| silentsea90 wrote:
| Very cool!! I am unfortunately very far from an AI expert. Can
| you share how you built this? Did you have to train your own
| model, or is it an open source model that you finetuned for
| climbing?
| smandava wrote:
| It is a new type of biomechanics AI model. More details in our
| paper which is coming soon!
| justinnhli wrote:
| I have had a similar idea for the past few years: less an AI
| model that takes in a photo and generates a video, more a
| movement planning tool given a 3D model of the wall.
|
| Echoing some of the other comments, I think it would be more
| interesting to not just see a single avatar climbing the problem,
| but to see many possible approaches to the same problem, even for
| climbers with the same body type. This way, even skilled climbers
| that are stuck on a problem can consider alternate
| beta/techniques that they may find easier to execute.
| gazelle21 wrote:
| Wow, this is great!
|
| I do BJJ and wonder how it could AI models could help. I find
| watching tape to help, not only myself but everyone who I know
| has recorded their matches has found it helpful.
| blobbers wrote:
| Fun! How does it know what route you're climbing, if you can't
| climb it? >_> Sorry if that's an obvious question.
|
| Does it just tell you how it would climb things you can already
| climb?
| hangonhn wrote:
| This is really cool from a technology point of view but the main
| attraction of climbing for me is figuring out how to "solve" the
| climb. Trying to solve a crux with a buddy is half of the fun for
| me.
| eigelc wrote:
| And a quarter of the other half is asking people in the gym how
| to do it.
| crazygringo wrote:
| Right, but for people in their first couple of years of
| climbing, it can be really helpful to try to solve it, then see
| what the AI coach says, examine the diff, and thereby improve
| your solving and climbing skills.
|
| I have definitely discovered in my life that I learn probably
| 10x faster in situations where I can try something, and then
| _immediately_ compare it with an expert solution. Rather than
| just trying and trying and trying on my own.
|
| And once you're experienced in climbing (like you probably
| are), then of course you don't need to compare anymore. The joy
| is in the solving. But expert solutions are _really_ helpful to
| _get_ to the point where you can consider yourself experienced,
| so much more quickly.
| et-al wrote:
| With bouldering, the set of moves is fairly limited and what
| actually prevents someone from pulling harder comes down to
| flexibility, core strength, and contact strength.
| crazygringo wrote:
| OK, but the submission and conversation here isn't about
| bouldering.
|
| This is about wall climbing where there's lots of choice
| for moves.
| IanCal wrote:
| I'm not sure what definition you're using here but the
| example video on the main page is bouldering.
| et-al wrote:
| Well if we're talking about "wall climbing", the set of
| moves don't expand that much as one progresses. Your
| technique needs to be more efficient and your transitions
| and balance on point, but it's not like there's a
| esoteric 13a move that a 10d climber hasn't seen. Sure
| the first time someone dyno'd on a roped climb was a sea
| change.
|
| Bouldering arguably has a larger set of moves than "wall
| climbing", but again the physicality of this sport is the
| crux for most people. One can study and maybe drill all
| the "advanced" moves, but without the flexibility and
| strength to execute on microholds, that knowledge is
| useless.
| alexcaza wrote:
| Cool idea! My friends and I have chatted about building something
| similar, though with a coaching focus. Specifically for use with
| spray walls/kilterboards/etc.
|
| It could be interesting to combine something like this with an
| analysis tool. Analyse a climbers attempts or successes and
| compare it to the beta figured out by the model. Then offer tips
| on body positioning or technique based on your
| weight/height/strength.
| programmarchy wrote:
| Not a climber myself but this is super impressive. Curious about
| what techniques you're using. Is this based on a GAN model?
| party_possum wrote:
| It would be really interesting to build an instance of this model
| trained on world cup footage for a few reasons.
|
| - Like all of these things, your training data matters and the
| internet is awash with videos of people climbing badly. A lot of
| people specifically post "I can't climb this, what am I doing
| wrong?" videos. World cup climbers are, by the nature of the
| competition, extremely talented and technically proficient
| climbers. Even when they fail, they fail in smart interesting
| ways.
|
| - There's lots of high quality video footage out there. Heck, the
| problems are even set with visual clarity in mind which would
| help when parsing that footage. There's potentially enough video
| to train instances on individual climbers. You could run side by
| sides like "How would Tamoa climb this and how would Janja climb
| this?".
|
| - World cup problems are stylistically distinct. They involve
| lots of moves "typical" climbers will never ever encounter. Many
| climbers will look at a typical gym problem and think "I have an
| idea of how to climb this" but will look at a world cup problem
| and just think "????????". An app that told you how a problem
| like that should be climbed might be useful.
|
| There are drawbacks too.
|
| - World cup climbers are outliers, whose physical ability
| (strength, flexibility, etc.) give them access to kinds of
| movement that other climbers just don't have. No amount of
| "knowing the sequence" will get me up a climb that requires a
| full bat hang (look it up) because I just don't have the ankle
| strength to do the movement.
|
| - World cup "style" is only commonly used at high level comps and
| in very large commercial gyms. It's probably not extremely
| relevant to a typical climbing session.
|
| - World cup problems are very hard. Mostly v10 and up? It would
| be hilarious to watch a model trained on genetic monsters
| crushing the world's hardest boulder problems try to tell a
| doughy office worker (me) how to climb v2.
| nl wrote:
| This was exactly my thought. See
| https://www.instagram.com/p/Cvg3bJkp30a/ on a super-hard slab
| problem at Innsbruck 2023 (Sorato Anraku couldn't do it, so you
| know it's hard...)
| takk309 wrote:
| Could this be used to grade routes? It would be cool if a photo
| or short video of a route could be used to better assist in
| grading climbing routes. Grading is a massively subjective thing,
| to some extent, so this could be super tricky.
| et-al wrote:
| I only watched the video, but without taking in account the
| texture of the holds, the grading aspect would be limited.
|
| One also learns to just recalibrate according to regions. E.g.
| climbing 11d at one gym, versus 12a at another.
| mordechai9000 wrote:
| Also, how will the AI know when it is appropriate to inflate
| the grade to make the climbing seem harder, and when it is
| appropriate to sandbag so I can pretend a route is easier for
| me than it actually is?
| takk309 wrote:
| Easy, it'll just grade everything 5.9+!
| latchkey wrote:
| Of course everyone on HN is focused on the climbing / beta /
| technique aspect of this.
|
| It is ok that it is a solution looking for a problem. There is
| obviously no 'business' or 'product' here. It isn't like there is
| a payment link on the page or anything.
|
| What I'd like to see the comments focus on is that we should just
| be happy that someone is making the effort to learn more about AI
| and building tooling around it. Experimentation is king.
|
| They've put their work out into the open (soon to be open sourced
| even!), not to be criticized over whether it is useful or not,
| but just that they created something that could spawn other
| interesting things that solve real world use cases.
|
| Huge kudos for doing this work.
| iosguyryan wrote:
| really good example of the culture of climbing too!
| iosguyryan wrote:
| If I could "yes and" this... route setters could benefit from
| this tool to simulate layouts against a number of body types
| and mobility.
| ungreased0675 wrote:
| Yes, and help setters create more interesting routes. Maybe a
| layout with multiple ways to solve it, but all the
| approximate same difficulty?
| latchkey wrote:
| I spent 8 years lead climbing at Mission Cliffs in SF 2-5
| days a week (5.12 range was my top). They had excellent route
| setters there that I spent many hours just watching between
| climbs.
|
| The route setters would effectively do what you're saying,
| while they were setting the route. They knew exactly what
| would work and what wouldn't. The holds/wall are really what
| dictate things and if you're not able to climb it... it isn't
| really their problem.
|
| As far as I can tell, while this project is cool, it really
| has nothing to do with climbing or route setting. It is an AI
| project where the developer just kind of made up some task
| and had AI follow it.
| party_possum wrote:
| Climbing route setting (when done at a high level) is constantly
| in conversation with itself. Many climbers don't realize that
| there are trends or fads in movement styles that sweep across the
| industry. This has only picked up steam in an era of social
| media.
|
| Just like movie dorks will happily spend hours explaining how an
| individual shot in a movie is actually an insider reference to
| another movie, and as a result a statement of intent for the
| movie as a whole, professional route setters will talk your ear
| off about the way one of their problems embraces or rejects
| specific kinds of movement trends of the last 6 months.
|
| That intentional rejection is interesting. Many route setters,
| especially for competitions, are in constant search of novelty.
| One kind of perfect problem is something that looks confusing and
| impossible, up until you see it done, at which point it seems
| almost obvious. It's the feeling of solving a sudoku. But
| critically, they _want_ climbers to be initially confused.
|
| I wonder if AI might actually be better than humans at sequencing
| these kinds of problems. Humans bring so much context and
| experience and expectation to the process that we are easily
| tricked. AI just looks through a few terabytes of video and says
| "What about this?".
| chefandy wrote:
| > I wonder if AI might actually be better than humans at
| sequencing these kinds of problems
|
| Probably a deeply unpopular take here, but without knowing
| anything about climbing routes, I'm gonna say _no_. I 'm not
| saying that they won't have excellent quality output that might
| even solve problems that human output can't, but the process of
| creating something is meaningful, even commercially. Surely
| this will be useful in some respects, but I just don't buy the
| idea that humanity is destined to passively consume automated
| algorithm-generated utility products-- _especially_ creative
| ones-- no matter how smooth, cheap, and clever they might be.
| JadeNB wrote:
| > the process of creating something is meaningful, even
| commercially.
|
| That's true, but why does it mean that the answer to the more
| or less objective question "will AI actually be better than
| humans at sequencing these kinds of problems?" (As stated
| it's not really objective, but one could easily come up with
| metrics like, say, total time to a correct solution, or time
| spent observing the route or other climbers, or ....) One can
| imagine other, related questions that are less objective
| (like "will it be a good idea to integrate this AI assistance
| into climbing competitions?"), but, to me, the answer to the
| (implicit) original question has nothing to do with whether
| or not the activity is meaningful, or with humans' destiny
| one way or the other.
| chefandy wrote:
| Sure, they may well be quantitatively better. If you were
| to create metrics to measure the number of mistakes or
| weird spots or overly annoying things, it's quite possible
| that the output from the algorithm could score better than
| human output, and the throughput would obviously be
| incomparable. But, whether or not something is
| qualitatively better is far more subjective-- it's
| influenced by our culture, our experiences, and everything
| else that creates the lens through which we see the world.
| Something's origin absolutely affects the way people
| experience it, be it a physical object, story, experience,
| etc. Don't get me wrong-- I realize there's real value in
| affordable quantity with with mediocre quality-- how many
| restaurants in the world are McDonald's? But then again,
| how many restaurants in the world _aren 't_ McDonald's? If
| McDonald's could sell you a ribeye comparable to Capital
| Grille, I'd be astonished if it put a dent in Capital
| Grille's bottom line. Applebees, however...
| JadeNB wrote:
| Right, those are important considerations, but I don't
| think that they're the _same_ consideration. If you ask
| whether a celebrity chef is better at cooking a certain
| dish than I am, then the answer (no matter the dish, for
| I 'm not much of a chef) is almost certainly "yes." If I
| answered instead no, that the provenance of a dish
| matters, that the celebrity-chef culture is bad for fine
| dining, and that my wife would rather have the dish made
| by me, then I think I'd be regarded as missing the point
| of the question, even if my objections were true on their
| own.
| chefandy wrote:
| Well, I think that celebrity vs husband is not a good
| analog for human vs generated. I also think that in this
| case, origin has broader connotations than provenance, in
| that it includes context. What if the celebrity chef was
| your wife's brother? What if you had a degenerative
| neurological disease? What if it was the dish you cooked
| for her on your second date? Anything can seem very cut
| and dried if you impose artificial restrictions and
| examine it in a vacuum. And, because you _can 't_ easily
| define and quantify things like context, the engineering
| and tech worlds tend disregard them because they don't
| really factor into the engineering equation... and that's
| why designers exist.
| travisjungroth wrote:
| I think your arguments against AI being better at
| creating routes reduce to:
|
| 1. You don't like what that would mean about the destiny
| for humanity.
|
| 2. A human making it makes it inherently better.
|
| 3. If it's lower quality, it's lower quality.
|
| I get why this would lead to strong beliefs. But these
| arguments aren't very _convincing_.
| chefandy wrote:
| I think you probably need to re-read what I wrote. I'm
| not against AI at all-- I use it all the time. I just
| don't think it's going to be qualitatively better than
| human creative output because how something was made
| matters to people. I also think the tech world thinks
| they're a lot better at judging creative output than they
| are.
| petsfed wrote:
| Well, the tricky bit here is that the route setter, a human,
| is the one actually solving the problem. So the problem as
| set is (and must be) a human creation first. This is
| especially true in outdoor climbing, where the first ascent
| process might involve the installation of anchor fixtures, or
| the removal of poorly-secured features for safety. You'd need
| some pretty wild sensor suites to correctly differentiate
| between a really good hold, and a dangerous flake that will
| peel off the wall if the slightest force is applied to it.
| The AI just generates potential solutions to the problem once
| the holds are found/placed. Certainly, there's some
| interesting conversations about how satisfying it is to solve
| a rubick's cube using somebody's algorithm vs. just figuring
| it out, but its not like the computer is inventing a rubick's
| cube.
|
| Embedded in your comment is the idea that AI might create
| boulder problems or routes in climbing gyms, and the human
| (or eventually robot) just follows that plan in bolting the
| holds to the wall. I expect that for a long time, AI
| generated climbing routes would rarely be good, but would
| consistently be physiologically impossible, feature
| uninteresting movement, or be too easy.
|
| Its easy enough to shotgun holds up onto the wall based on
| some imagined sequence, the real skill of route setting is to
| (as the GP pointed out) figure out what's physically possible
| and also fun and challenging.
| chefandy wrote:
| > You'd need some pretty wild sensor suites to correctly
| differentiate between a really good hold
|
| Ah, I see.
|
| > The AI just generates potential solutions to the problem
| once the holds are found/placed
|
| Yeah and I think that's really going to be the sweet spot
| for generative tools for the forseeable future.
|
| > Its easy enough to shotgun holds up onto the wall based
| on some imagined sequence, the real skill of route setting
| is to (as the GP pointed out) figure out what's physically
| possible and also fun and challenging.
|
| Right right. I have a feeling that making a more convincing
| substitute is primarily a matter of having less access to
| data than say, paintings and photography which are
| certainly not _less_ nuanced than this creative task. But
| as I said, a lot of people care about how something was
| made, too. I 'll bet that's going to be a much bigger
| factor, at least in marketing, than many realize in the
| near future.
| ethbr1 wrote:
| > _Embedded in your comment is the idea that AI might
| create boulder problems or routes in climbing gyms, and the
| human (or eventually robot) just follows that plan in
| bolting the holds to the wall._
|
| This would follow the exact path image GenAI evolved
| through.
|
| Step 1: Teach a model to recognize objects from noisy data.
|
| Step 2: Reverse-feed that model random noise and force it
| to hallucinate that noise back into likely objects.
|
| As there's probably physics simulation at some point in
| this particular scenario, there'd probably also be step 3
| of simulating a climb through the generated path to
| validate feasibility / specific qualities.
|
| It doesn't sound impossible.
| petsfed wrote:
| Its certainly not impossible, but that physics simulation
| is the biggest obstacle that I can see.
| rockfishroll wrote:
| Spotting a refugee from rockclimbing.com on hacker news was
| not on my bingo card for the day. But I guess if I'm here
| (writing novels about route setting) then I shouldn't be
| surprised other people are too.
| petsfed wrote:
| Outside of a short-lived usurper on instagram selling pet
| food dishes, I am still the only petsfed on the internet,
| since 1997.
|
| What's really wild to me is how somebody would recognize
| a mid-tier poster from a website I thought effectively
| defunct for nearly 10 years now.
| rockfishroll wrote:
| It just goes to show how impactful online communities are
| capable of being. rockclimbing.com was in it's heyday
| right as I was discovering climbing. I was a bored kid
| constructing my entire identity around climbing and there
| was no other place to do that outside of the gym. No
| mountain project. No youtube. No social media. I spent a
| lot of hours lurking those forums. There are only a
| handful of users I could still name, but I bet I would
| recognize a _lot_ of them.
| pierrec wrote:
| You mean AI might be better at setting or at reading? I feel
| like your question is getting misinterpreted.
| i_am_a_squirrel wrote:
| If anyone would actually use this (I don't think they would,
| because the puzzle, and talking about beta is half the fun of
| climbing)...
|
| ...then this would be very valuable data.
|
| Eventually you could train AI to generate routes, and then you
| could fire all of your master setters!
|
| Climb setters should unionize and copyright their art before it's
| too late (like for painters and software developers and
| musicians)
| Temporary_31337 wrote:
| I feel like I'm the right audience for this - a not very good
| climber with unusual distribution of limbs. But I'm not
| interested - part of the fun in climbing is on sight climbs and
| solving problems yourself.
| 31337Logic wrote:
| Thank you for posting this. :-)
| johnfn wrote:
| Man, where is the hacking spirit? OP posts a very cool,
| impressive hack and all everyone can talk about in the comments
| is how it actually isn't interesting to them.
| swores wrote:
| It's great if someone wants to use the "hacking spirit" to
| help themself climb, it doesn't obligate everyone else to
| want to use the same technique; and frankly it's (slightly)
| more interesting hearing people say why they would or
| wouldn't use it than to have a bunch of comments just saying
| "awesome that you're doing this" and ignoring whether they'd
| find it interesting themselves or not.
| SOLAR_FIELDS wrote:
| I would say (opinion) that it's vastly more interesting to
| hear whether other people would use Thing I Built. I can
| nerd snipe myself all day on projects. By that I mean I can
| easily endlessly entertain myself building things, both
| useless and useful. Whether they are interesting AND useful
| for other people is quite a large driver for me both for
| personal and business projects
| jajko wrote:
| If somebody solves the 'hacking' part for you, a lot of fun
| disappears. Yes they probably had tons of fun, but you lose
| some of it.
|
| Figuring a route for the first time is by far the most
| rewarding part of climbing (although I love all of it). Same
| with kids - if they grok something on their own, what a
| reaction and reward compared to being told how to do it.
| anvil-on-my-toe wrote:
| It's cool as an academic exercise, but there are things in
| life where the struggle is kind of the point. The messy
| learning process, asking other humans for help, is better for
| your growth.
| rideontime wrote:
| In 2024, the "hacking spirit" aligns well with not using an
| app to do everything.
| perfect_wave wrote:
| Came here hoping to see something like this. Glad to see it's
| the top comment. Part of the beauty of climbing is solving a
| puzzle and figuring things out.
|
| Edit: Unsurprised to see the demo on a gym route instead of
| actual rock.
| smandava wrote:
| Will post more videos on outdoor rocks soon! I just chose
| that video for the demo because it was easy to explain the
| route
| kyawzazaw wrote:
| gym route is also important and heavily used
| carabiner wrote:
| If you ever Moonboard, you'll see the app gives easy access to
| beta videos from Instagram. That's because many people want
| help on really hard problems. OP app is like those videos.
|
| If you _never_ feel like you need help from watching someone
| else climb, then I think you are not trying hard enough
| problems.
| cloudripper wrote:
| As an experienced, very poor climber myself, I enjoy the
| process of problem solving. However, knowing many experienced,
| very good climbers - studying beta is how many of them excel.
| As an aside, the term beta also came from the use of beta tapes
| by climbers to record themselves climbing so they could study
| the minutiae of their movement to find improvements. In higher
| levels of bouldering especially, a nuanced and firm
| understanding of beta is everything - so careful analysis of
| your movement can help identify areas of improvement.
|
| I believe this is consistent with most elite (or elite-
| aspiring) athletes from many sports.
|
| While I personally enjoy the problem-solving aspect of climbing
| (when I rarely do get out), I absolutely see the value in this
| project (and other climbing apps that thrive on beta sharing)
| buzzwordninja wrote:
| To clarify and define the time period: beta as in Beta Max
| video cassette tapes.
| surume wrote:
| The AI analysed my body and said coldly, "Just take the
| elevator."
| amelius wrote:
| Here is an interesting post about how body mass affects
| climbing ability.
|
| Even if you have a great BMI and muscular build, but if you are
| heavy, then at some point physics starts working against you.
|
| https://www.reddit.com/r/bouldering/comments/595a2m/how_much...
|
| > The basic physics of strength/weight is that strength is
| reliant on the cross sectional area of muscles (number of
| fibers) and therefore increases quadratically, whereas weight
| is a function of volume and increases cubically. So the amount
| of strength you can gain as your mass increases will always lag
| behind the amount of mass gained. This is why insects are so
| strong compared to their bodyweight, and why the creatures with
| the highest strength/weight ratio's are all small.
|
| > If we analzye male pro climbers we'll see that the heaviest
| of them max out at around 170-180 lbs (and there are VERY few
| of those, the majority being between 130 and 150), and that's
| not a fluke. In order to be a great rock climber you need
| strong forearms, and strong lats, and that's about it. Muscle
| mass, or fat mass basically anywhere else is mostly just
| pulling you off the wall (a strong core helps too but it
| doesn't need to weigh a lot).
|
| > (...)
| lambdaone wrote:
| I'm sceptical about the usefulness of this for a number of
| reasons, including the points raised by others about hold
| recognition etc.
|
| It's also difficult to see how the system would work out the 3D
| arrangement of holds. It can clearly try to infer 3D
| relationships from video of an existing climb, but it would be
| hard for it to work out the 3D position or orientation of any
| holds not used by the climber in the video.
|
| These two put together make solving the climbing problem even
| more difficult, because appropriate body positions and moves are
| often very sensitive to even tiny differences in shape, relative
| position, and orientation of holds; and occluding volumes,
| aretes, cracks and so can make the problem even harder.
|
| But as a simplified demo, this is cool, and I salute it.
| giantg2 wrote:
| Probably going to show me the best route to the couch.
| eps wrote:
| Your demo video is monetized and showing 30 seconds of ads. Might
| not be a bad idea to switch the ads off.
| dathos wrote:
| Pff, they're sharing their work for free and you complain about
| a 30s ad?
| eps wrote:
| I assume it's an oversight rather than an actual attempt to
| earn few pennies.
| alanzhuly wrote:
| This is really neat. Do you plan to expand this to other sports
| or activities as well?
| smandava wrote:
| Maybe! Depends on the reception.
| nsagent wrote:
| Great job! A lot of folks from my CS PhD program are really into
| climbing (including me), but none of us has conducted research
| that explored climbing. Rather climbing is our respite from
| research.
|
| Anyway, I think this could work really well as an app. Good luck.
| smandava wrote:
| Thanks! Still unsure if we will commercially pursue this
| specific idea or not. This was moreso meant to be a public
| progress marker from our broader work pursuing virtual avatar
| simulations. If enough interest is there, we will think about
| it. But, for now, this (will be) a paper and some open sourced
| code.
| colehud wrote:
| Saw tennis on your LinkedIn - is SABR a nod to the Sneak Attack
| By Roger?
| smandava wrote:
| Ha! That was unintended. "You can't connect the dots looking
| forward; you can only connect them looking backwards." -
| someone somewhere (likely Steve Jobs)
| ropable wrote:
| I'm genuinely curious how the model classifies holds. I could
| (hypothetically) see this doing something like assigning more
| utility to certain holds (e.g. a steep sloper that is impossible
| to use as a handhold but is can be used as a smearing foothold)
| and thereby giving bad/implausible beta. That being said, it
| seems like an obvious shortcoming and I'm very interested in
| learning more!
| nl wrote:
| I have a side project in a related space. For mine I'm syncing
| climbing between different videos automatically. I find this
| pretty useful because it highlights difference.
|
| Some examples:
|
| https://www.instagram.com/p/Cvg3bJkp30a/
|
| https://www.instagram.com/p/Cpuv2zvptdW/
|
| It's actually syncing in 3D so is fairly independent of camera
| angle (as the second video shows)
| teapourer wrote:
| Looks cool! It definitely looks technically challenging,
| especially for expert climbers who have a variety of creative
| moves at their disposal. But I can definitely see this being
| useful for novices.
| tuatoru wrote:
| But does it do Hammersmith Broadway?
| monroewalker wrote:
| This is cool and all but I can't help but wonder if it'd be even
| cooler to instead show the beta for climbing out of the
| negativity found in the comments here. /s
|
| But really though, awesome project and congrats on getting it to
| this point! I'm excited for the day when something like this can
| be used to critique form
| bryanrasmussen wrote:
| I guess it's just how my mind works but I thought this will make
| a great part of some stories I'm working on, only adapted for cat
| burglars to figure out the optimal route on a building
| beforehand.
|
| Maybe also good thing for the next Mission Impossible movie if
| they ever make one.
| kriro wrote:
| Very nice. Please do open it up as much as possible. I'm curious
| about the data pipeline. I love bouldering and have thought about
| different ML ideas in that domain. I'm also an avid beta breaker.
| Finding alternative solutions somehow tickles my hacker-brain. So
| my first idea was intended beta vs. alternatives but I was only
| thinking in 2d and images :) Another idea I've played around with
| is more objective grading but labeling is kind of hard especially
| with gyms rotating routes so I thought about using Kilterboatd
| data and transfer learning it or maybe unsupervised approaches.
|
| Congrats on a cool idea and most importantly the execution.
| raptorraver wrote:
| Cool project but please delete it. You're taking the fun part out
| from bouldering and just leaving us the labour ;)
|
| Edit: apprently I wasnt the first to make this joke. I emphasise
| that this is Really a cool project. Great job!
| RunSet wrote:
| https://i.imgur.com/S7ugEXC.mp4
| nachteilig wrote:
| Very cool project. I will try this out when you release it
| some_random wrote:
| This is rad and I can't wait to try it out for myself
| copypasterepeat wrote:
| Very cool idea but I wonder how well this can possibly work in
| practice. So many bouldering problems hinge on small things, such
| as how a little crimp hold is oriented or how deep the groove in
| it is. It can be hard to determine this even in person until you
| get up close or try to grab it.
|
| Additionally, this model seems to take into account the body type
| but not other relevant factors, such as climber's strength and
| flexibility/mobility. Even in the demo, that suggested move of
| putting the right foot high and pushing off the jug up and to the
| right could not be done by a lot of new climbers. This parallels
| what I've personally seen when watching others climb. For
| example, some climbers are exceptionally strong and can basically
| campus their way up a route. Watching their beta is irrelevant to
| me. I don't have the same strength to pull those moves.
| ericmcer wrote:
| This has always been a dream, because there is a "perfect" way
| for your body to climb a route but it would be insanely complex
| to calculate, especially when factoring in every tiny
| physiological aspect like ankle strength and hamstring
| flexibility.
|
| This is a really cool step though.
|
| Maybe a more useful variant with the current tech would be
| isolating a single move and seeing every variation the AI can
| think up.
| southernplaces7 wrote:
| Considering the notoriety of AI models in making confident but
| blatant mistakes or simply lying about things they don't have
| real answers to I don't know how smart it is to trust one with
| something in which bad advice can kill you very quickly and
| immediately.
| ericzakariasson wrote:
| can't wait to try haha
___________________________________________________________________
(page generated 2024-05-09 23:01 UTC)