[HN Gopher] Andrej Karpathy (Tesla): CVPR 2021 Workshop on Auton...
___________________________________________________________________
Andrej Karpathy (Tesla): CVPR 2021 Workshop on Autonomous Vehicles
[video]
Author : vpj
Score : 96 points
Date : 2021-06-21 16:27 UTC (6 hours ago)
(HTM) web link (www.youtube.com)
(TXT) w3m dump (www.youtube.com)
| childintime wrote:
| The video: https://youtu.be/NSDTZQdo6H8
| dang wrote:
| Thanks! Maybe it's best if we change the URL to that from
| https://twitter.com/vpj/status/1407000737423368197.
| ejdyksen wrote:
| That video is a screen capture from another video (which was
| screen capped from a livestream), but the original has much
| better audio quality.
|
| Here's a direct link:
|
| https://www.youtube.com/watch?v=eOL_rCK59ZI&t=28293s
| Robotbeat wrote:
| He made a very good argument for vision-only, but it seems like
| training actually uses radar data to help calibrate vision
| measurements, so it seems to me there's value in making some
| vehicles still contain radar (say, one out of 10) even if it's
| not used for controlling the vehicle directly at drive time.
|
| Also, the sensor resolution issue he mentioned could be addressed
| by using a higher resolution radar sensor.
|
| I find the list of 221 triggers to be interesting. In principle,
| the NHTSA or NTSB could help contribute lists of triggers to
| companies to validate their training sets on.
|
| Every time there is a fatal airliner accident, the NTSB does a
| safety investigation and airliners get a little bit safer each
| time. In the same way, each fatal accident in a vehicle with this
| kind of autonomy could end up being captured by these triggers,
| improving safety over time in a sort of mixture between expert
| human analysis and ML.
|
| (Nobody does this for all regular car crashes because fatal car
| crashes happen every day! And you're not going to retrain human
| drivers about some new edge case every day, although you can for
| vehicles like this.)
| avs733 wrote:
| Most fatal car crashes are investigated, but by the police not
| engineering experts. The invetigational motivation is legal and
| liability focused not improvement focused.
|
| You sparked a happy delusion in my mind...training drivers to
| the same level we train pilots. Can you imagine drivers having
| regular check rides?
| eddanger wrote:
| I would love to see re-certification of "professional"
| drivers. Almost daily I encounter taxi or semi-truck
| operators driving at the limit of what is acceptable.
| osipov wrote:
| > so it seems to me there's value in making some vehicles still
| contain radar (say, one out of 10) even if it's not used for
| controlling the vehicle directly at drive time.
|
| That's not how neural networks work. You start by training them
| with a radar, then you deploy them without the radar. Neural
| nets make the radar irrelevant post-training. This is the
| entire point of Karpathy's pitch.
| andyxor wrote:
| I like Andrej from his PhD research days and awesome blog posts
| but this is a series of disasters in the making, that is until
| FTC steps in after more people die from "self-driving" accidents
| under interesting and unexpected circumstances.
|
| The whole vision vs. LIDAR stuff is a distraction as long as
| Tesla "AI" doesn't have common sense.
|
| It literally doesn't know what it's doing, and the tail of "edge
| cases" is infinitely long.
|
| It would be more honest to show the cases where it missed,
| thankfully there is no lack of them in "FSD beta" videos on
| YouTube.
| [deleted]
| RosanaAnaDana wrote:
| >common sense
|
| What a ridiculously useless term.
| [deleted]
| gameswithgo wrote:
| a better phrase might be "ability to do higher order
| reasoning to decide what to do in a novel situation"
| andyxor wrote:
| exactly, who needs common sense when you can sell the bright
| vision of self-driving future.
| Fricken wrote:
| Try using common sense to figure out what he meant by that.
| option wrote:
| There are two fundamental reasons, in principle, vision alone
| can do it: 1) Humans do it with vision alone 2) You can
| actually predict lidar's output with vision alone. So many
| systems out there actually use lidar to generate more labeled
| data to make lidar unnecessary
| kaba0 wrote:
| In principle. But we do have 3 billion neurons for that with
| ridiculous number of interconnections - even if an order of
| magnitude less connection would be enough, we are still far
| away from that amount of computations. And that is actually
| the easy part of the problem - mapping an image to a depth
| map. The hard part is interpreting it for which we have a
| complete inner universe built up, with context-dependent
| logic. FSD is simply a really hard problem that we are not
| even getting close to. We are at a fancy robot vacuum level.
| andyxor wrote:
| besides vision, humans also have this thing called brain, and
| reasoning, and instincts, and being able to tell if the
| object in front of them is e.g. a roof of an overturned truck
| vs. empty space, etc, etc.
|
| the key word is "etc" which expands into infinite tail, which
| no big data training on farms of GPUs would ever help with.
|
| Humans and other animals have an ability to understand the
| scene and generalize from prior experiences to infinite set
| of new and unexpected circumstances, the "common sense" these
| dumb curve-fitting models are fundamentally lacking.
| darknavi wrote:
| > humans also have this thing called brain, and reasoning,
| and instincts, and being able to tell if the object in
| front of them is e.g. a roof of an overturned truck vs.
| empty space, etc, etc.
|
| Assuming the car is reasonably good at this, it has the
| advantage that it can see in every direction at once.
|
| I don't think self-driving cars will ever be perfect, but I
| think they will quickly become less-lethal than the average
| human.
| kaba0 wrote:
| For the average case, perhaps. But we are not doing too
| badly for the average case either.
|
| Let's see how well does a Tesla react to exceptional
| cases where actual decision making is required, few data
| sample is available, etc. Statistics can be misleading
| with rare events.
| semi-extrinsic wrote:
| There are many things humans do with vision alone that
| machines can't. For instance see if a person standing 50 ft
| away is looking at you or at the house behind you.
|
| The human vision system operates at a bitrate equivalent of
| well beyond 500 gigabits per second. Resorting to "humans can
| do it with vision alone" is only a sufficient argument if
| your computer vision system can match that.
| [deleted]
| bluepanda928752 wrote:
| With what kind of reliability can LIDAR data be predicted
| with vision nowadays?
| bluepanda928752 wrote:
| Tesla's decision not to use the LIDAR as a safety feature (i.e.
| having reliable high-resolution data about things the car can
| collide with) is so incredibly indefensible, since solving the
| last 1% of this using only vision likely requires a general
| artificial intelligence
|
| Prediction: Tesla will be the last of all major auto
| manufacturers to get to L5 autonomy. Time interval between when
| Tesla L5 FSD is finally available and when humanity is destroyed
| by the general AI it runs on will be very awesome and also very
| short
| SheinhardtWigCo wrote:
| They're betting that they can use a massive feedback loop to
| train a set of neural networks to the point where they are as
| accurate as LiDAR without actually firing any lasers.
|
| Even if you believe this goal is possible to achieve at some
| point in the future, I think the argument falls apart when you
| consider that it will take years, probably decades, for a pure
| vision approach to catch up to where Waymo is _today_ in terms
| of safety. (They have cameras too.)
|
| That Tesla can't afford to fit expensive LiDAR sensors to all
| of the cars it sells is Tesla's problem. Regulators won't give
| a shit that pure vision is "better" in theory. They will simply
| compare Tesla's crash rate in autonomous mode with that of
| Waymo and other AV operators, and act accordingly.
| bluepanda928752 wrote:
| I understand why they made the "no-LIDAR" bet early when the
| LIDARs were completely unpractical for a production consumer
| car
|
| However, nowadays it starts to look that 100% reliable depth
| estimation from cameras might actually require a human-level
| AI to work and also solid-state LIDAR technology is becoming
| cheap enough and integrateable into normal cars, but Tesla
| can't really change their stance on this without admitting
| that FSD options they already sold would not actually become
| FSD within the lifetimes of these vehicles. I suspect this
| might also be the reason why Karpathy looks more and more
| nervous with each new talk
| practice9 wrote:
| > I think the argument falls apart when you consider that it
| will take years, probably decades, for a pure vision approach
| to catch up to where Waymo is today in terms of safety. (They
| have cameras too.)
|
| On what set of metrics do you think Waymo is safer? IMO it's
| too early to compare and cherry-picked proofs both from Waymo
| and Tesla are not really representative.
| DSingularity wrote:
| Definitely. Especially with car companies like NIO strapping
| in LIDAR to their upcoming models.
| ra7 wrote:
| > Prediction: Tesla will be the last of all major auto
| manufacturers to get to L5 autonomy.
|
| Tesla is also the only company to claim to target L5 autonomy.
| Everyone else, including Waymo, is strictly targeting L4 and
| say L5 autonomy is not possible or realistic. L5 is a pipe
| dream.
| nickik wrote:
| I resonantly had an argument on here where somebody insistent
| that breaking because of over-passes were issues with vision
| system. Seem pretty clear that it is the resolution of the radar,
| not the shadow of the bridge that causes the issue. Good to get
| some more insight into this.
|
| This is the right thing to focus on, as it is by far the largest
| issue with Autopilot on the highway. Multiple people who do
| testing of these system that false positives on some highway
| overpasses are the biggest usability issue.
| nightski wrote:
| It's interesting that an academic conference now feels like a
| marketing op for industrial research labs more than anything. His
| claims about how accurate their vision system is and how it is
| exceeding other sensors is not verifiable in any way to the
| public. Given how well qualified he is I am sure he is not wrong!
| Andrej is brilliant. But this is an academic conference right?
| This isn't open science, it's a discussion about an engineered
| system. I'm afraid this is the future of ML research (which CV is
| so heavily dependent on now). Long gone are the days of reading a
| paper and understanding the approach. Now you need the data and
| model which may not even be computationally feasible without
| millions of dollars in hardware. This isn't Tesla's fault or
| anything, it just makes me sad.
| aeternum wrote:
| In the talk, he gave clear examples with detailed position +
| velocity graphs where the vision system detected obstacles
| sooner and with less jitter than the radar system. Specifically
| the overpass where radar triggers erroneous braking, and the
| pulled over truck where radar detects the obstacle
| significantly slower.
| stefan_ wrote:
| That's a strawman. No one is looking to build FSD with radar
| sensors that have shipped on cars for 20 years now for things
| like adaptive cruise control. LIDAR is what vision only is
| compared to.
| ketamine__ wrote:
| I've read claims that they are desperately trying to hire.
|
| https://mobile.twitter.com/TaylorOgan/status/140705191831739...
| DSingularity wrote:
| Maybe it's an unfortunate side effect of their success.
| Almost all early members of the team are now multi
| millionaires if they held their stock.
| jowday wrote:
| Anecdata but both of the people I know that worked on
| Autopilot quit within 18 months of starting, citing extreme
| overwork and Musk micromanaging things. This lines up with
| that.
| karpathy wrote:
| For the record these are some blatant & false FUD attempts.
| sam_goody wrote:
| tldr: Tesla uses vision alone, and has dropped radar and the
| other sensor. He makes a very decent argument why.
|
| (Surprisingly, he basically ignores night driving.)
| nickik wrote:
| The list of triggers contains things like 'motorcycles at
| night', so it seems its all in that dataset.
| dogma1138 wrote:
| Ironically this is what Tesla criticized Mobileye for.
|
| I still think that this is far the best demonstration of
| autonomous driving to date https://youtu.be/A1qNdHPyHu4
| marricks wrote:
| Wow that demo has it all.
|
| - car stalled in it's lane - complicated intersections -
| people exiting cars in it's lane - car going over into it's
| lane
|
| If I had an hour of driving that I'd be stressed.
| halotrope wrote:
| Wait until you see the Jerusalem 40 minute video. Munich
| traffic is tame in comparison: https://youtu.be/kJD5R_yQ9aw
| I really don't understand why mobileye gets so little
| recognition. They might be quietly winning the self driving
| race.
___________________________________________________________________
(page generated 2021-06-21 23:00 UTC)