[HN Gopher] GM's Cruise alleged to rely on human operators to ac...
___________________________________________________________________
GM's Cruise alleged to rely on human operators to achieve
"autonomous" driving
Author : midnightdiesel
Score : 63 points
Date : 2023-11-04 20:59 UTC (2 hours ago)
(HTM) web link (www.nytimes.com)
(TXT) w3m dump (www.nytimes.com)
| mlinhares wrote:
| The title isn't news at all as every single trustworthy
| autonomous driving solution MUST HAVE human operators somewhere
| to take over but the actual article is a good summary of Cruise's
| current situation and I'd guess the competition as well.
| dventimi wrote:
| Where is the article does it even support the title? I'll
| reread it but I didn't see anything about human operators.
| potatolicious wrote:
| > _" Half of Cruise's 400 cars were in San Francisco when the
| driverless operations were stopped. Those vehicles were
| supported by a vast operations staff, with 1.5 workers per
| vehicle. The workers intervened to assist the company's
| vehicles every 2.5 to five miles, according to two people
| familiar with is operations. In other words, they frequently
| had to do something to remotely control a car after receiving
| a cellular signal that it was having problems."_
|
| Title of the post should be edited though since it's not the
| headline of the piece and this information, while
| interesting, isn't the main thrust of the article.
| Animats wrote:
| That's a terrible disengagement rate. Cruise claimed in
| 2020 "Cruise, for comparison, clocked 831,040 miles with a
| disengagement rate of 0.082 (per 1000 miles)" [1]
| Something's not right here.
|
| [1] https://www.engadget.com/2020-02-27-waymo-
| disengagement-cali...
| AlotOfReading wrote:
| Companies measure multiple disengagement rates for
| different purposes. The DMV numbers are usually safety
| rate numbers, as in "if a human hadn't intervened there
| may have been an accident or near miss". The specifics
| vary company-to-company, and they'll have a large
| document somewhere laying out exactly what the criteria
| are. The numbers in the article are some other metric,
| though I have no idea what. I'm a bit skeptical that it's
| the average over their entire ODD, given that it's much
| higher than my own experiences and most of their vehicles
| were running around the outer city at night, where they
| seemingly did okay.
|
| It could reflect some particular ODD (e.g. downtown at
| rush hour) where the vehicles didn't do nearly as well,
| or something else entirely.
| creer wrote:
| It's buried pretty deep all the way at the end.
| KennyBlanken wrote:
| I thought HN very strictly required the title to be the
| original article's title unless the title was really, really
| bad?
| donsupreme wrote:
| > staff intervened to assist Cruise's vehicles every 2.5 to
| five miles
| dventimi wrote:
| https://archive.ph/2023.11.04-050448/https://www.nytimes.com...
| haltist wrote:
| It's not an allegation. It's the same as using human feedback for
| tuning large language models. There are no autonomous cars
| currently regardless of what is written on the marketing
| brochures. In various "emergency" situations the cars phone home
| and ask a human operator to take over the controls.
| throwaway5959 wrote:
| The latency on that has to be massive.
| jonhohle wrote:
| Why? UAVs are piloted remotely, video games are played
| remotely with sub-100ms latency.
|
| Getting a remote driver connected might a while, but
| afterwards it seems like a mostly solved (in practice)
| problem.
| notahacker wrote:
| UAVs don't have to deal with traffic (the thought of
| driving a vehicle with the latency and intermittent
| connectivity of my drone horrifies me) and when someone
| dies in a video game they respawn...
| throwaway5959 wrote:
| I wasn't talking about signal latency, I was talk about the
| time it took for an operator to sign on and take control.
| haltist wrote:
| It's good enough for the routes that Cruise uses in the city.
| hartator wrote:
| There is an emergency every 2-5 miles?
| alkonaut wrote:
| That sounds pretty low if it's city driving or poor
| conditions. I know some of the trial cities are basically
| easy mode (wide streets, almost never snows..) but still.
| KennyBlanken wrote:
| First off: not even close. Waymo has a disengagement rate
| of 0.076 per 1,000 miles.
|
| Second: You're shifting the goalposts from the grandparent
| comment's assertion that these interventions are to be
| expected in an "emergency", when the frequency of the
| interventions shows they're clearly not "emergency"
| interventions but part of normal operation.
| l33t7332273 wrote:
| > It's the same as using human feedback for tuning large
| language models
|
| It isn't remotely the same. This would be like if human
| operators typed some of chatGPTs answers
| haltist wrote:
| OK, you must know more than I do about how human feedback is
| used for tuning large language models.
| ra7 wrote:
| This is completely incorrect. Remote operators cannot "take
| over controls" at all and hence cannot help in any "emergency"
| i.e. safety critical situation (e.g. preventing a crash). All
| they can do is _assist_ the vehicle with things drawing a path
| to get around a parked vehicle, instructing it to do a multi-
| point turn when it's stuck and so on.
|
| What the article says is that Cruise vehicles need some sort of
| assistance every 2.5 to 5 miles (I highly doubt this number is
| accurate). Not that they're getting into emergency situations
| that frequently.
| haltist wrote:
| Do you work at Cruise?
| ra7 wrote:
| No.
| haltist wrote:
| Then you wouldn't know if I was completely incorrect or
| not.
| ra7 wrote:
| I do because I know self driving companies have talked
| about how remote operations work. It doesn't involve
| taking control of the vehicle.
|
| Here's a Waymo engineer explaining how they can't
| joystick a car:
| https://www.reddit.com/r/SelfDrivingCars/s/2ujFLZoLbo
|
| And here is Zoox's video about their teleoperations:
| https://youtu.be/NKQHuutVx78?si=4PDnG0gQm6lEnp9v
|
| No reason to believe Cruise is doing any different. If
| you have evidence of the contrary, please share it.
| haltist wrote:
| So you are 100% certain that remote operators can not
| take over the car?
| ra7 wrote:
| Based on what I know, yes. Why would they want to do it
| with the latencies involved? It's not a reliable
| solution, so it's not used in any safety critical path.
| RobotToaster wrote:
| Relevant XCKD https://xkcd.com/1897/
| batmansmk wrote:
| Having to be remotely operated every 2.5 to 5 miles seem to
| defeat most of the economics of self driving cars.
|
| Back of the napkin math, cars drive at an average of 18mph in
| cities, so every 10-20min. Let's assume it takes over for 1min,
| and that you need remote drivers not too far for ping purposes,
| so at the same hourly rate. To guarantee you'll be able to take
| over all demands immediately, due to the birthday paradox, you
| end up needing like 30 drivers for 100 vehicles? It's not that
| incredible of a tech...
| spondylosaurus wrote:
| Yeah, the driver-to-passenger ratio is still way less efficient
| than a train or even a bus.
| pj_mukh wrote:
| Just FYI, Most autonomous car companies have backup drivers.
|
| Its the disengagement rate that drives the number of operators
| you need per driver and therefore the economics. Theoretically,
| this rate should be improving steadily at all these companies.
|
| Cruise seems to have a bad disengagement rate _right now_ (
| <5miles seems really low), but methinks nytimes might be
| partaking in some obfuscation here.
|
| Waymo's should be much better already. Curious by how much
| though.
| cheriot wrote:
| Wages can fall off a cliff within modest distances. To use
| unemployment rate as a proxy for driver pay, Bakersfield, CA
| 7.5% and San Francisco, CA 3.5%. Go a little farther to Los
| Vegas 5.7% and one can avoid California's minimum wage.
| alex_young wrote:
| https://web.archive.org/web/20231104212102/https://www.nytim...
| causality0 wrote:
| I thought being a social media moderator and being constantly
| exposed to violence, racism, and child pornography was bad.
| Having your whole day being a series of "quick, don't let these
| people die!" moments seems like the worst tech job on earth.
| wolverine876 wrote:
| > Two months ago, Kyle Vogt, the chief executive of Cruise,
| choked up as he recounted how a driver had killed a 4-year-old
| girl in a stroller at a San Francisco intersection. "It barely
| made the news," he said, pausing to collect himself. "Sorry. I
| get emotional."
|
| ...
|
| > Cruise's board has hired the law firm Quinn Emanuel to
| investigate the company's response to the incident, including its
| interactions with regulators, law enforcement and the media. /
| The board plans to evaluate the findings and any recommended
| changes. Exponent, a consulting firm that evaluates complex
| software systems, is conducting a separate review of the crash,
| said two people who attended a companywide meeting at Cruise on
| Monday.
|
| After the first [edit: the first performative charade, about
| little girl in a stroller], why should we trust the second isn't
| also a performative charade? What independence or credibility
| does some hired law firm have, that the company itself does not?
| How about using an independent third party?
| wmf wrote:
| Independent third parties don't work for free and if you pay
| them (by your logic) they're no longer independent. The best
| you can probably hope for is a government investigation.
| wolverine876 wrote:
| There are ways to do it. Non-profits don't need payment,
| always, and their mission isn't profit. For example,
| companies have worked with environmental non-profits on
| internal climate change and other issues.
| raldi wrote:
| Can you explain what you mean by "after the first"?
| simonw wrote:
| Presumably they meant that the first paragraph they quoted
| looked to them like a "performative charade".
| wolverine876 wrote:
| Yes. I'll clarify.
| raldi wrote:
| What would it have looked like if it had been sincere?
| ciabattabread wrote:
| Cruise is trying to save itself from getting shut down by GM. I
| guess it would look slightly better for optics if the GM board
| hired them instead of Cruise's board. But it's the same money,
| and it's GM's decision at the end.
| pj_mukh wrote:
| Hmm? I saw it exactly the opposite. A lot of people in the
| autonomous driving industry are driven by exactly what Vogt
| describes (little girl in the stroller etc.). See also Chris
| Urmson of Waymo fame's TED talk, he talks about a similar
| motivation[1].
|
| Its a fallacy everyone conveniently ignores. The woman the
| Cruise car ran over was actually first hit by a human driver
| _who is still at-large_ , not a peep about him. The press kinda
| just accepts this as the "cost of doing business".
|
| The way I see it, Vogt sincerely believes autonomous cars will
| make things safer from the #2 killer of Children under 19
| (outside of guns) by a _wide_ margin [2] and therefore
| accelerated the rollout past what was safe. I see no evidence
| otherwise.
|
| [1]
| https://www.ted.com/talks/chris_urmson_how_a_driverless_car_...
|
| [2] https://www.nejm.org/doi/full/10.1056/nejmc2201761
| sroussey wrote:
| We have become so desensitized to human deaths due to cars
| even though those numbers are higher than violent acts of
| terrorism et al that actually kill far fewer people each
| year.
|
| Many people have to be killed AT ONCE for it to be news
| worthy these days.
| evbogue wrote:
| This is the same whacky theory I've been spreading about Tesla
| self-driving for a year or so. "Imagine Tesla self-driving is
| like some dude driving your car via videogame on the other side
| of the world."
|
| Most people are pretty sure my theory is wrong. I have absolutely
| no evidence this is true, it's just some crazy idea that popped
| into my head one day.
| cheeselip420 wrote:
| Like some sort of fucked up Ender's Game situation.
| cheeselip420 wrote:
| Cruise is leveraging human-in-the-loop to expand faster than they
| otherwise would, with the hope that they will solve autonomy
| later to bring this down.
|
| I don't think this is a viable strategy though given the enormous
| costs and challenges involved.
|
| There doesn't exist a short-term timeline where Cruise makes
| money, and the window is rapidly closing. They needed to expand
| to show big revenues, even if they had to throw 1.5 bodies per
| car at the problem.
|
| Prediction: GM will offload cruise, a buyer will replace
| leadership and layoff 40% of the company. The tech may live to
| see another day, but given the challenges that GM has generally
| (strikes, EVs, etc), they can no longer endlessly subsidize
| Cruise.
| chaostheory wrote:
| GM actually spun off Cruise in 2018. Honda now has shares in
| Cruise. SoftBank used to own some as well, but GM bought out
| their share last year
| cheeselip420 wrote:
| GM had FOMO and now it's time for FAFO.
| dontblink wrote:
| So Waymo/Google winning here in your opinion?
| cheeselip420 wrote:
| Waymo will have challenges scaling rapidly, but they may be
| able to get some sort of favorable unit economics and expand
| more slowly.
|
| Tesla has the scale and for some reason regulators give them
| a pass. I wouldn't bet against Elon, but we aren't there
| yet...
| Retric wrote:
| Human in the loop can be vastly cheaper than you might think.
|
| _If_ this lets them have the only level 5 system on the market
| they could double that and millions would happily pay. Suppose
| your a trucking company would you rather pay 50k / year or
| 5k/year? That's a stupidly easy choice.
|
| Americans drive roughly 500 hours per year. If they can replace
| 98% with automation and the other 2% with someone making
| 20$/hour that only costs them ~200$/year, which then drops as
| the system improves.
| cheeselip420 wrote:
| Human in the loop is fine.
|
| Negative unit economics and massive expansion are not.
| Retric wrote:
| Who says they can't recover the full cost? Cars don't last
| forever, bake the cost in upfront or charge a monthly fee.
| cheeselip420 wrote:
| I'm not saying they can't - I'm saying they are running
| out of time to do so, and with the DMV shutting them down
| they've been hamstrung further.
|
| They are burning 100s of millions every quarter. They
| needed to show either growth/expansion or some sort of
| positive cash flow. They now have neither.
| wolverine876 wrote:
| If humans need to remotely intervene for a car in motion, that
| implies it could impact safety.
|
| If that's correct, then the remote signaling of a problem and the
| human's response and control must have flawless availability and
| low latency. How does Cruise achieve that?
|
| Cellular isn't that reliable. Maybe I misunderstand something.
| xyst wrote:
| I get a feeling Cruise is going to get sold off within the next 5
| yrs. Waymo will likely be the leading provider for "autonomous
| vehicle" software/hardware.
|
| Government Motors can only sustain such a loss on their books for
| a short time. This is probably why Vogt has been pushing so hard
| for market dominance.
| tempsy wrote:
| I wonder if there are rails to prevent a bad actor Cruise worker
| from remote driving erratically...
| neilv wrote:
| > _Company insiders are putting the blame for what went wrong on
| a tech industry culture -- led by the 38-year-old Mr. Vogt --
| that put a priority on the speed of the program over safety.
| [...] He named Louise Zhang, vice president of safety, as the
| company's interim chief safety officer [...]_
|
| I hope Chief Safety Officer isn't just a sacrificial lamb job,
| like CISO tends to be.
|
| Is the "interim" part hinting at insufficient faith, and maybe
| future blame will be put on how the VP Safety performed
| previously (discovered after the non-interim person is hired)?
|
| > _[...] and said she would report directly to him._
|
| Is the CSO nominally responsible for safety?
|
| Does the CSO have any leverage to push back when their
| recommendations aren't taken, other than resigning?
| KennyBlanken wrote:
| @dang title not the same as original
| ProAm wrote:
| Cruise came out of YC if I recall?
| ooterness wrote:
| This was a plot point in Captain Laserhawk: All the self-driving
| cars and flying drones were actually being remotely piloted by
| prisoners in a massive VR facility.
|
| https://en.wikipedia.org/wiki/Captain_Laserhawk:_A_Blood_Dra...
| DelightOne wrote:
| > Having to be remotely operated every 2.5 to 5 miles
|
| Regarding Cruises' suspension, how likely is it that the backup
| driver restarted the car to drive again after the car stopped
| with the pedestrian below?
| kvogt wrote:
| Cruise CEO here. Some relevant context follows.
|
| Cruise AVs are being remotely assisted (RA) 2-4% of the time on
| average, in complex urban environments. This is low enough
| already that there isn't a huge cost benefit to optimizing much
| further, especially given how useful it is to have humans review
| things in certain situations.
|
| The stat quoted by nyt is how frequently the AVs initiate an RA
| session. Of those, many are resolved by the AV itself before the
| human even looks at things, since we often have the AV initiate
| proactively and before it is certain it will need help. Many
| sessions are quick confirmation requests (it is ok to proceed?)
| that are resolved in seconds. There are some that take longer and
| involve guiding the AV through tricky situations. Again, in
| aggregate this is 2-4% of time in driverless mode.
|
| In terms of staffing, we are intentionally over staffed given our
| small fleet size in order to handle localized bursts of RA
| demand. With a larger fleet we expect to handle bursts with a
| smaller ratio of RA operators to AVs. Lastly, I believe the
| staffing numbers quoted by nyt include several other functions
| involved in operating fleets of AVs beyond remote assistance
| (people who clean, charge, maintain, etc.) which are also
| something that improve significantly with scale and over time.
| throwaway1104 wrote:
| Keep it up, Kyle! All new tech will have hiccups and
| opposition. Really enjoyed my ride experience when I visited
| SF.
| monero-xmr wrote:
| Huge cojones on the CEO to risk public statements given the
| enormous legal and regulatory pressure being applied. I
| certainly wouldn't recommend this tactic!
| pj_mukh wrote:
| "The stat quoted by nyt is how frequently the AVs initiate an
| RA session. Of those, many are resolved by the AV itself before
| the human even looks at things, since we often have the AV
| initiate proactively and before it is certain it will need
| help."
|
| Hoo boy, sure wish the NYT had clarified that. That changes
| things significantly.
| ra7 wrote:
| Thanks for the clarifying! This makes a lot of sense. I think
| NYT did a really poor job of explaining the remote assistance
| bit.
| phkahler wrote:
| >> Cruise AVs are being remotely assisted (RA) 2-4% of the time
| on average, in complex urban environments. This is low enough
| already that there isn't a huge cost benefit to optimizing much
| further, especially given how useful it is to have humans
| review things in certain situations.
|
| Funny, since I thought full autonomy was the goal of the
| company. 2 percent human intervention isn't scalable.
___________________________________________________________________
(page generated 2023-11-04 23:00 UTC)