[HN Gopher] Why didn't auto-braking stop these crashes?
       ___________________________________________________________________
        
       Why didn't auto-braking stop these crashes?
        
       Author : heavyset_go
       Score  : 61 points
       Date   : 2021-10-07 19:32 UTC (3 hours ago)
        
 (HTM) web link (www.latimes.com)
 (TXT) w3m dump (www.latimes.com)
        
       | gleenn wrote:
       | Trying not to be a Tesl fanboy, but they have a nice graph
       | showing the number of auto pilot complaints but fail to mention
       | if they normalized for the number of cars on the road (with
       | autopilot) of that brand. Maybe there are a ton of Tesla
       | complaints because they have way more cars on the road? What's
       | the percentage? I guess we'll never know. Not great reporting.
        
         | queuep wrote:
         | Not sure about the US but I can't believe there's more Tesla's
         | than Audi's on the road in the U.S?
         | 
         | Edit: oh there is, that's quite cool honestly.
         | https://www.goodcarbadcar.net/2021-us-vehicle-sales-figures-...
        
           | ahahahahah wrote:
           | Are you somehow under the impression that everyone is driving
           | at most a one year old car?
        
         | lukyanovic wrote:
         | As far as I know all new model luxury cars like Audi and
         | Mercedes come with automatic braking for frontal impacts. So
         | considering the number of Mercedes sold in the last few years,
         | I think it's safe to say there are more of them than Teslas.
        
           | nradov wrote:
           | Automatic braking has been a common feature even in non-
           | luxury vehicles since at least 2018.
        
             | SketchySeaBeast wrote:
             | Yeah, once the Toyota Corolla has something, you can pretty
             | much assume it's just standard now.
        
         | Someone wrote:
         | FTA: "The rate of complaints about Tesla, relative to the
         | number of its cars sold in the U.S. in 2020, was more than
         | three times that of the other automakers."
         | 
         | Still a bad comparison, but I would think that the number
         | relative to the number of cars in use of each brand wouldn't be
         | better for Tesla.
         | 
         | The real question would be how this would be relative to the
         | number of miles driven (which, I guess, is about as fair as we
         | could get)
        
           | [deleted]
        
           | omgwtfbyobbq wrote:
           | The Tesla NTHSA complaints are kind of a mess and why we
           | can't have nice things (data).
           | 
           | Many are valid and from justifiably disgruntled owners, and I
           | could see how Tesla would have the most driver assistance
           | related complaints, but there are subset that are suspect
           | and/or cranky.
           | 
           | https://www.nhtsa.gov/vehicle/2020/TESLA/MODEL%2525203/4%252.
           | ..
           | 
           | Several begin with "The contact ", which I guess is someone
           | filing a complaint based on second hand info?
           | 
           | There's also Keef, who is still filing complaints based off
           | of insurance auction listings and has now started filing
           | complaints based on FSD Beta videos.
           | 
           | THIS IS A COMPLAINT ABOUT THE BETA RELEASE AUTOPILOT. A
           | RANDOM MODEL 3 VIN HAS BEEN USED TO BE ABLE TO FILE A
           | COMPLAINT. THIS COMPLAINT APPLIES TO ALL MODELS OF TESLA.
           | HERE IS A VIDEO OF A PROUD BETA TESTER.
           | HTTPS://WWW.YOUTUBE.COM/WATCH?V=PQ-RGY1V8UG THIS OWNER HAS A
           | MISPLACED TRUST IN THE SAFETY OF THE AUTOPILOT. HE IS USING
           | IT IN HIGH RAFFIC AREAS WITH PEDESTRIANS AND CYCLISTS HE IS
           | ALSO USING IT AT EXCESSIVE SPEED ON NARROW ROADS ONLY INCHES
           | AWAY FROM ONCOMING TRAFFIC. RATHER THAN HAVING HIS HANDS
           | HOLDING THE WHEEL IN THE CORRECT WAY HE ONLY HAS HIS FINGERS
           | TOUCHING THE WHEEL WITH THE PALMS FACING THE WRONG WAY. IF
           | ANYTHING GOES WRONG HE WILL NOT BE ABLE GRAB THE WHEEL
           | PROPERLY IN TIME TO PREVENT A COLLISION. FINGERTIP DRIVING IS
           | UNSAFE AT THE BEST OF TIMES. IF AUTOPILOT HAS A GLITCH OR A
           | SPASM THE DRIVER WOULD BE UNABLE TO QUICKLY REGAIN CONTROL.
           | DESPITE THE CLAIMS OF ELON MUSK SUCH DRIVER BEHAVIOR DOES NOT
           | LEAD TO THE BEAT RELEASE AUTOPILOT BEING REVOKED. THIS
           | EXPERIMENT IS DEADLY. PLEASE GET IT STOPPED IMMEDIATELY.
           | THANK YOU KEEF.
        
       | slownews45 wrote:
       | Once again an article with NO context.
       | 
       | TACC and ebreaking are in MANY auto's. We get no data on the
       | number of accidents and fatalities for cars
       | 
       | 1) Driven by humans
       | 
       | 2) Other systems of ebreaking
       | 
       | For all we know, despite these 17 accidents, teslas may be far
       | safer.
       | 
       | For example, a quick google shows that there are 1.7 MILLION rear
       | end collisions EACH YEAR in the US.
       | 
       | https://www.washingtonpost.com/news/dr-gridlock/wp/2015/06/0...
       | 
       | Note that the 17 items discussed here with Tesla are over
       | multiple years.
       | 
       | So in the time teslas had these 17 crashes there were perhaps 3M+
       | rear end collisions alone?
       | 
       | The sad reality is that the clickbait headlines sell, and any
       | effort to actually look at data does not (on the web and more
       | recently on HN). Folks just do not want to engage with an idea or
       | with numbers, but the outrage (or downvote) button is usually
       | close by :)
        
       | lacker wrote:
       | Automatic braking doesn't get much attention, but it seems like
       | something that could be incredibly valuable if it got to the
       | "clearly better than human" level.
       | 
       | This is purely anecdotal, but I find the automatic braking system
       | on my Mazda CX-9 to be fairly annoying. Once every couple months
       | or so, it has a false alarm and brakes while blaring out alerts,
       | in an urban environment where I don't observe anything out of the
       | ordinary at all. And it's never activated in a situation where I
       | found it to be helpful.
       | 
       | That said, it's possible that an annoyance every two months is
       | just the cost of a system that really would help me if I got into
       | bigger trouble. It's hard to say from purely my own anecdotes.
        
         | 29083011397778 wrote:
         | Just as another point of reference, my (2021) Mazda 3 AEB has
         | only ever activated when another driver stopped short in front
         | of me. I believe it's configurable in settings, though your
         | experience may be worse due to environmental concerns (bugs,
         | rain, dust), different model year, or simply being a bigger
         | vehicle (with a different bumper height or more mass requiring
         | more time to stop).
        
         | ajross wrote:
         | > Automatic braking doesn't get much attention, but it seems
         | like something that could be incredibly valuable if it got to
         | the "clearly better than human" level.
         | 
         | It's important to point out here that a handful of back of the
         | envelope analyses are out there that show that Tesla's
         | autopilot is _already_ "clearly better than human" level at
         | avoiding emergency vehicles.
         | 
         | These are very common accidents. Emergency vehicles in travel
         | lanes get hit all the time, crews are carefully trained to
         | avoid that situation if at all practical, and taught how to
         | reduce personal risk when that's not possible.
         | 
         | In fact (and I don't have a link handy) if you extrapolate out
         | from a few different data sources it looks like Tesla's on AP
         | get in these crashes about half as much as vehicles in the
         | general fleet.
         | 
         | Now, that doesn't mean that these 13 events don't constitute a
         | cluster worth investigating. And it doesn't mean that there
         | can't be a bug worth fixing. It doesn't even mean that the
         | media shouldn't write about it.
         | 
         | It does mean that we should be very careful with pronouncements
         | about safety from a sample set of this size, though.
        
           | emn13 wrote:
           | What's the source for this claim? Are you accounting for the
           | fact different classes of vehicle have differing accident
           | statistics (e.g. comparing teslas to similarly old and
           | expensive alternatives), and that autopilot's miles are not a
           | random sample of all driven miles?
        
           | lacker wrote:
           | I think the problem here is calling something "better than
           | human" because it manages to avoid some accidents. If a car
           | avoids one accident for every 100,000 times it intervenes,
           | that isn't really a good thing. It just means that drivers
           | start to ignore flashing lights and warnings because they are
           | always meaningless. Maybe it even causes some accidents by
           | distracting the driver, and they don't get categorized as
           | "caused by the automatic braking system" because nothing
           | really tracks that.
           | 
           | Every time I start my car, about 15 seconds into the drive a
           | popup appears on the screen, saying something like "Warning:
           | automated driving assistance system can cause distraction.
           | Select OK to dismiss". Thanks, you have reduced your own
           | liability in exchange for further distracting the driver.
        
           | syrrim wrote:
           | >In fact (and I don't have a link handy) if you extrapolate
           | out from a few different data sources it looks like Tesla's
           | on AP get in these crashes about half as much as vehicles in
           | the general fleet.
           | 
           | Why use that as a comparison? The obvious confounder would be
           | that people would disable autopilot in poor conditions,
           | making such crashes a priori more likely. The relevant
           | comparison would be Teslas (with and without AP) against a
           | similar class of cars, maybe BMWs.
        
             | ajross wrote:
             | That's nitpicking. There's an initial contention: "Teslas
             | hit emergency vehicles frequently", and a simple
             | refutation: "No, they don't, here are some numbers[1]".
             | Your response only makes it "not impossible" that the
             | initial contention is correct, but you have to show numbers
             | to prove that.
             | 
             | All we have is a cluster of 13 accidents over 3-4 years,
             | and there are something like thousands of such accidents in
             | the US every year. That's not proof that this is noise and
             | not signal, but the obvious hypothesis is that it's
             | probably noise[2]
             | 
             | [1] Which I don't have, though I suppose I could track it
             | down.
             | 
             | [2] For the record: I believe it's probably a real bug, but
             | in a product that on the whole is clearly better, so it's
             | well hidden in the noise.
        
         | jacquesm wrote:
         | The one on my 2016 C-class Mercedes was so bad that it almost
         | _caused_ two accidents. If not for years of pingpong... I got
         | rid of it after having it checked out and declared healthy, if
         | that system wasn 't broken it might as well be. Worse than
         | useless. Don't get a MB with automatic emergency braking unless
         | you want to covertly commit suicide.
        
           | dazbradbury wrote:
           | "The one on my 2016 C-class Mercedes was so bad that it
           | almost caused two accidents. If not for years of pingpong..."
           | 
           | Care to elaborate? Both how it nearly caused an accident, and
           | how you avoided one seem interesting!
        
         | brokenmachine wrote:
         | What happens when the auto braking engages? Does it just slam
         | the brakes on and stop dead? Wouldn't that be a big risk of you
         | getting rear ended by someone behind you (especially if it's a
         | false alarm)?
        
         | tidbits wrote:
         | Toyota's safety sense system has saved me from minor accidents
         | a few times. It has never auto applied the brakes, but the
         | alarms and brake assist have been really helpful on freeways
         | during abrupt stops. My girlfriend, on the other hand, hates
         | her Honda's auto braking system, as it triggers too easily,
         | even on the lowest setting. I've seen it regularly trigger when
         | pulling up to sensor controlled gates.
        
           | browningstreet wrote:
           | I have a Toyota and my auto-braking alarm thing has only gone
           | off a few times... all very clearly my fault for losing
           | attention for a brief moment. To be honest, I have no idea if
           | it's gone into applying-brake-mode. I'm guessing yes, but...
           | it hasn't been often nor can I even remember really what
           | happened. I do know that it prevented crashes, whether it
           | applied the brakes or alerted/scared me into doing so.
           | 
           | There has never, ever been a false alarm though. Not even
           | once.
        
         | seoaeu wrote:
         | How would you define "clearly better than human" for a driver
         | assist technology? Like if 'driver + automatic breaking' gets
         | into fewer crashes than 'driver', is that a success? What if
         | the assist is constantly doing phantom breaking, but never hits
         | anything?
        
       | MisterBastahrd wrote:
       | I'd much prefer a system that stopped my vehicle as smoothly as
       | possible to one that I would rely on to autonomously brake for
       | me.
       | 
       | I suspect that if all cars were fully autonomous, that we would
       | have fewer collisions and fewer high-speed collisions simply
       | because we wouldn't need to use traffic signals to push traffic
       | along (and we'd have fewer traffic-stopping accidents).
       | 
       | A few months back, an older gentleman collided with my car on US
       | 75 in Dallas while I was at a full stop. Would automatic braking
       | have slowed him? Probably. Would he have collided with me at all
       | if his car were driving autonomously? Probably not. He was
       | distracted and took his eyes off the road, never seeing that the
       | traffic ahead of him was at a complete stop. It took nearly 10
       | minutes for us to get off the road for a fender bender because
       | nobody would allow us to get over. Traffic simply flowed around
       | us despite the issue.
       | 
       | I suspect that within the next 50 years, driving at all in most
       | major cities will require a special license, and it will become a
       | civil rights issue for both those who don't know how to behave
       | like adults but want the freedom to move about anyway, and those
       | who simply do not want to be tracked by any entity.
        
       | tibbydudeza wrote:
       | Somebody said it is a hard problem to solve.
        
       | peatmoss wrote:
       | OT: I wonder why sites like the LA Times don't offer a redirect
       | to Apple News when the content is hosted there. I do subscribe to
       | Apple News+ along with additional iCloud storage, Music, etc. I
       | assume Apple gives LA Times something when I read the article vs.
       | when I click away because I'm not willing to subscribe to more
       | news sites than I already am.
       | 
       | Apple News link for those in a similar boat:
       | https://apple.news/A9PXJ2JwHRouQSYO9lQh9tA
        
         | tzs wrote:
         | It is not as convenient as a redirect, but on iPhone and iPad
         | you can use the "Share" function when on the page in Safari and
         | select "News" as what to share with and that usually works.
         | 
         | I don't know how to do that on a Mac.
        
       | gorkish wrote:
       | My hypothesis on this is that all automaker's automatic emergency
       | braking systems likely function at approximately the same level,
       | but Tesla drivers are disproportionately likely to rely on it due
       | to the use of Autopilot being so incredibly common.
       | Statistically, drivers of other brands that include similar
       | lanekeeping abilities are far less likely to use them compared to
       | Tesla owners' use of Autopilot.
       | 
       | Should it be happening? Well the NHTSA doesn't seem to mind it.
       | Maybe we should be asking them why their automatic emergency
       | braking tests are so stupidly out of agreement with what drivers
       | expect and automakers promise that the systems are able to do.
       | The freight industry has a high speed test and their systems are
       | top notch.
        
         | Zhenya wrote:
         | Sorry, your hypothesis makes no sense. There are WAY more cars
         | sold by other brands, so more Tesla incidents is even more
         | galling. Using autopilot should only HELP the auto braking not
         | make it worse.
         | 
         | Other systems use radar and camera. They are thoroughly tested
         | by the OEMs and suppliers to strict tolerances.
         | 
         | Meanwhile Tesla is shipping beta software and having people
         | test it. Including removing radar because of "phantom" braking.
        
           | sharkmerry wrote:
           | OPs point was tesla drivers are more likely to use the
           | features. More cars being sold by other brands, could refute
           | that but it doesnt necessarily. Its about the rate of usage
           | of the software X total sales X avg. miles driven
        
         | sparker72678 wrote:
         | This article addresses automatic emergency breaking, which is
         | ON by default in the vehicles discussed.
        
       | kevin_thibedeau wrote:
       | Wouldn't be surprised if this is their radar whitelist blinding
       | it to sensor input.
        
         | rtkwe wrote:
         | Musk is very adamant that cameras are enough even going so far
         | as to stop including radar in Model 3 and Y units shipped to
         | the US. So even if it were the radar being confused according
         | to the Tesla line it shouldn't matter the cameras should be
         | enough.
         | 
         | https://techcrunch.com/2021/05/25/tesla-is-no-longer-using-r...
        
         | plebianRube wrote:
         | Quite possible. They've physically removed the radar hardware
         | on the newer models.
        
       | andrewtbham wrote:
       | > Tesla does not have a media relations department and Chief
       | Executive Elon Musk did not respond to attempts to seek comment.
       | 
       | Tesla has ongoing response to this.
       | 
       | https://www.tesla.com/VehicleSafetyReport
        
       | Someone1234 wrote:
       | Because Tesla effectively doesn't have AEB while AutoPilot is
       | enabled.
       | 
       | In a Tesla AEB should be an entirely self-contained ("dumber")
       | system. Its only job should be to look for any ground-level
       | object ahead and brake. It should do this using simpler but more
       | reliable ("dumber") techniques like forward bumper-level radar or
       | camera parallax, not NN/deep learning/object recognition/AI.
       | 
       | To use an analogy, AEB should be like two-factor authentication.
       | If your "second" factor is the same as your first, then it isn't
       | two-factor. In this case AutoPilot is the "first" and "second"
       | factor i.e. it is AEB and in control of the vehicle.
       | 
       | So Tesla's vehicles absolutely have AEB, when AutoPilot is
       | disabled, because it is offering a "second opinion" relative to
       | the driver. So IIHS's ratings are correct in that instance. But
       | as soon as AutoPilot is enabled it is no longer a "second"
       | opinion and therefore may as well not exist.
       | 
       | Unfortunately Tesla aren't alone in making this mistake. We're
       | seeing other vehicle manufacturers stepping into the vehicle
       | automation space double-dipping their auto-drive systems into
       | AEB. It is a bad practice regardless of who, AEB should be its
       | own system with its own simpler/dumber logic that protects people
       | from auto-drive mistakes, not compounds them.
        
         | baybal2 wrote:
         | Does Tesla even have a radar?
        
           | beamatronic wrote:
           | Yes, at least some do, and they recently disabled it in a
           | recent software update
        
           | namrog84 wrote:
           | I dont think so. I thought their whole gimmick or goal was to
           | drive entirely by cameras and nothing else. Because humans
           | only have eyes.
           | 
           | Though just speculation and no source.
        
             | taylortbb wrote:
             | Due to the current chip shortage they've moved to a vision-
             | only system, as they had difficulty getting radar units in
             | the quantity required. Until recently however, all Teslas
             | had radar since ~2014. The high profile crashes are
             | mostly/all radar-enabled vehicles.
             | 
             | The thing about automotive radar, and this isn't just
             | Tesla, is that stationary objects are normally excluded.
             | Otherwise the system brakes for things like overhead signs
             | that have high radar reflectivity. Stopped cars, concrete
             | barriers, etc are all basically invisible to most
             | automotive radar.
             | 
             | Tesla also isn't the only manufacturer taking the vision
             | approach. Subaru, for example, was doing vision-only
             | emergency braking in 2016.
        
       | analog31 wrote:
       | I read in another HN thread, that some Tesla owners have
       | experienced "ghost braking," and have adopted the technique of
       | holding their foot just above the accelerator, in order to
       | counteract it.
       | 
       | When I read that, I made a mental note to slow down a bit if a
       | Tesla gets behind me, so they will pass me. Usually that's not a
       | problem, as I don't drive particularly fast.
       | 
       | I don't know if this is an issue with other brands of cars. My
       | fancy new Subaru has never ghost-braked.
        
       | [deleted]
        
       | dv_dt wrote:
       | There are also issues with other makers more simple automatic
       | emergency braking systems. But overall it seems like the systems
       | are a net improvement in avoiding at least some accidents.
       | 
       | https://www.usatoday.com/story/money/cars/2019/10/29/iihs-au...
        
       | comeonseriously wrote:
       | After several officers where nearly injured/injured/killed
       | because drivers ran into them as they were stopped on the side of
       | the road, some states instituted laws that say you either slow
       | down to 45 or you move over a lane.
       | 
       | Is that built into AutoPilot and whatever it's called on other
       | brands? Humans have to do it, so cars should have to as well.
        
         | ape4 wrote:
         | Every lane change is a possible collision. I wonder if that was
         | taken into account.
        
           | comeonseriously wrote:
           | Considering how far away flashing lights can be sensed (in
           | both humans and cars), that shouldn't be a problem.
        
           | asdff wrote:
           | Can autopilot even change lanes? It seems like that would be
           | impossible to code for freeways in cities in California. You
           | have to muscle your way into the lane and force people to
           | back down or get hit or else they aren't ever letting you in.
        
           | YarickR2 wrote:
           | Except it's not. Every lane change _in semi-dense and dense
           | traffic_ is a possible collision; I 'd say if you have a car
           | moving in the same direction as you are, and it's closer than
           | ~50 feet, then it's a non-negligible possibility, other than
           | that you just have nobody to collide with during lane change
           | . We're talking highway speeds, of course, not racetrack
        
           | pugworthy wrote:
           | Yes, that's why slowing down is the other action you can
           | take.
        
       | shiftpgdn wrote:
       | It always bugs me they trot out the Mountain View crash as an
       | example where a Model X struck an unmarked and unprotected jersey
       | barrier end in what visually looks like a traffic lane. Further
       | that multiple human drivers had hit the divider in the same spot
       | and a near fatal accident occurred literally the next day into
       | the same barrier.
       | 
       | If I put a brick wall on the freeway and paint it to look like a
       | tunnel ala wil-e-coyote am I responsible for building the wall or
       | is the driver responsible for hitting it?
        
         | willcipriano wrote:
         | Is this the one you are talking about?
         | 
         | https://www.mercurynews.com/2018/03/30/tesla-autopilot-was-o...
         | 
         | I'd blame a human driver for hitting it in this case.
        
       | pmcollins wrote:
       | yet another tesla hit piece from russ mitchell and the la times.
       | 
       | is it just me or does the la times drag tesla at every
       | opportunity? could it be because the owner of the la times,
       | patrick soon shiong, has a company called nantenergy that seeks
       | to compete with tesla? at a minimum, there's a conflict of
       | interest.
       | 
       | and, kind of a tangent, but where are the articles about the
       | thousands of lives lost due to the incompatibility of trucks/suvs
       | and sedans in crashes actually causing thousands of lives lost
       | per year?
        
         | carabiner wrote:
         | I'm pretty neutral on Tesla/Musk, but I looked at this guy's
         | (Russ Mitchell) writing history and it appears to be an
         | unhealthy fixation on Tesla. His Twitter, and almost every
         | article he writes, is about bashing Tesla. It's like he has
         | real personal beef with Musk, a person he's probably never met.
        
           | [deleted]
        
           | asdff wrote:
           | He's a journalist, they typically get assigned to cover
           | certain topics. Not surprising that he focuses on the EV
           | space, that's probably in his job description for LA times.
        
             | pmcollins wrote:
             | focusing on the ev space != relentlessly bashing one ev
             | company
        
               | asdff wrote:
               | They are kind of the big cheese of the EV space and you
               | could argue people relentlessly bash Tesla here too.
               | Maybe there is something to these critiques of the
               | company, rather than everyone and anyone just having an
               | axe to grind.
        
       | kazinator wrote:
       | If a human driver hits a pulled-over police car, or any other
       | stationary object that can be seen well in advance, the problem
       | isn't simply lack of braking.
       | 
       | Why would that be the talking point if an autopilot does that?
       | Because we expect it not to be aware of the obstruction until
       | within emergency braking distance?
        
       | S_A_P wrote:
       | Ive a Jeep Grand Cherokee. It has the front sensing collision
       | detector and auto braking. It may have hit the brakes 1 time that
       | I wasnt paying attention and 9,999,999 times when there was
       | nothing in front of me. I disable it and thus far have avoided
       | crashing. The point is that driver assist systems are only assist
       | and you _have to pay attention_ when you drive. Ive said this
       | here before, Autopilot is a terrible misnomer. It seems to sit
       | squarely in the valley of being good enough to  'trust' but not
       | good enough for real world use. If I were in charge of the world,
       | it would be rebranded immediately.
        
         | sliken wrote:
         | Dunno, autopilot (at least originally) was just a stupid switch
         | on a plane that would try to hold altitude and speed. It
         | wouldn't try to avoid objects, couldn't take off, couldn't
         | land.
         | 
         | Very similar to Tesla's auto pilot, that can't manage your
         | driveway, random surface streets, but can mostly (but not
         | completely) handle highways.
        
           | Diederich wrote:
           | Good perspective.
           | 
           | I think, as you noted, that 'autopilot' is probably fine, but
           | 'full self driving' is actively misleading.
           | 
           | Note that I use FSD for 90+% of all of my driving, and I
           | think I'm safer because of it, but it has to be used with
           | understanding and care. The very name 'full self driving' is
           | something it might be some day, but it's nowhere close to
           | that now.
        
       | ancode wrote:
       | giving new meaning to 'move fast and break things'
        
       | choeger wrote:
       | It's kind of interesting that this appears to happen with police
       | cars and emergency vehicles mostly. Is there a particular way of
       | parking such vehicles that makes it hard(er) for a Tesla to
       | detect it?
       | 
       | It might also indeed be about phantom braking. My relatively old
       | (probably designed around 10 to 20 years ago) system gets
       | confused sometimes when I pass a truck or a bus and once even
       | when I passed a bicycle. It happened at least 5 times in less
       | than 10,000km. So I suspect there is some merit to the Autopilot
       | disabling the feature.
        
         | Rebelgecko wrote:
         | I wonder if the cameras or ML are thrown off by flashing
         | emergency lights or emergency vehicles' livery (most cars on
         | the road use just 1 paint color)
        
           | ajross wrote:
           | Yeah. I'm as big a Tesla booster as you'll find but I'm in
           | the "this is probably a bug in recognition" camp. They can
           | presumably fix this via properly parametrized modelling. It's
           | not a rare edge case at all.
        
             | myself248 wrote:
             | I have a hunch that at some point, there'll be an
             | investigation and a discovery process, and we'll learn that
             | the training dataset deliberately _excluded_ emergency
             | vehicles because they didn't represent "normal driving" or
             | something.
             | 
             | That's purely uninformed speculation, but I could see it
             | happening like this: Early on, someone says "The totality
             | of situations encountered on the road is too complex. We're
             | not building a system to do everything, we're building it
             | to take over during the boring parts of a drive. Only teach
             | it the boring stuff." Then a few years later, someone else,
             | forgetting those initial assumptions, says "Hey this thing
             | has done like a billion miles of boring stuff, why wouldn't
             | you trust it for more?"
        
         | andrewia wrote:
         | I think the flashing lights and appearance might degrade
         | autopilot. Other cars also have issues because they use radar
         | (except for some new models with camera-based systems like the
         | 2022 Honda Civic) and ignore stationary targets when they see a
         | moving vehicle as a more likely "target". Some systems use a
         | combination of radar and camera (with camera providing low-
         | speed, parked car, bike, and pedestrian detection). Then it's
         | up to the quality of the algorithms and tests have shown that
         | some cars are pretty poor at detection.
        
           | mindslight wrote:
           | The human visual system is amazing at normalizing across many
           | orders of magnitudes (cf HDR imaging), but yet with the
           | evolution of LED lights and a lack of accountability, the
           | lights on police cars have gotten way too bright for even
           | human drivers at night. I wouldn't be surprised if the
           | ultimate cause is a complete swamping of the cameras for
           | everything that isn't immediately in front of the car, and
           | even the lane lines disappearing when close enough to the
           | blinding lights. What _is_ a computer supposed to do when
           | confronted with complete loss of visual input? A human will
           | squint, slow down by 10-20mph, extrapolate the lane lines,
           | and play the odds. But there is no surefire way to proceed
           | safely, and thus no straightforward action to program a
           | computer with.
        
             | choeger wrote:
             | Well, if the computer can recognize the situation it can
             | always slow down, alert the driver, and eventually stop.
             | That's the nice thing about cars: There's always a safe
             | default mode. I suspect that the computer cannot recognize
             | this problem reliably, though.
        
               | mindslight wrote:
               | Slowing down to under 40mph is not a safe thing to do on
               | a highway and is illegal in many places. And suddenly
               | slowing down is extra unsafe when there's a car right
               | behind you that's also being blinded.
        
               | myself248 wrote:
               | Slowing down on a highway is required by law in many
               | areas, in the presence of emergency vehicles.
        
               | brokenmachine wrote:
               | So just keep on trucking along at full speed with your
               | eyes closed, you reckon.
               | 
               | Spoken like a true Tesla engineer.
        
       ___________________________________________________________________
       (page generated 2021-10-07 23:01 UTC)