[HN Gopher] Mozilla finds CPU bug (bad store forwarding) in Sams...
___________________________________________________________________
Mozilla finds CPU bug (bad store forwarding) in Samsung Galaxy S20
Author : kens
Score : 202 points
Date : 2023-08-09 14:43 UTC (8 hours ago)
(HTM) web link (bugzilla.mozilla.org)
(TXT) w3m dump (bugzilla.mozilla.org)
| jakub_g wrote:
| A few years ago I was doing Android native development. Back then
| Samsung as well as Huawei (and other cheap Chinese vendors) have
| been known to be doing weird things to the operating system and
| in Google Play console there'd often be wild device-specific
| crashes coming from those vendored Androids.
|
| I never loved Samsung devices but since then I avoid them even
| more.
| satiric wrote:
| This site has a list of phones with the same CPU as the Samsung
| Galaxy S20 they mentioned.
| https://en.wikichip.org/wiki/qualcomm/snapdragon_800/865
|
| I wonder if it happens to them too?
| adrian_b wrote:
| Depending on the region where they were sold, Samsung Galaxy
| S20 could use either Snapdragon 865 or Exynos 990 CPUs.
|
| The Mozilla messages fail to mention this information that is
| essential for any CPU bug, which CPU model was used in the
| phones with crashes.
|
| It is more likely that the bug is present in the Samsung Exynos
| 990 CPUs and not in Snapdragon, because Cortex-A77 was used
| much more widely, so it is improbable for such a bug to not
| also be noticed elsewhere.
| happycube wrote:
| The bug report shows a Mali GPU, which rules out Snapdragon
| (Adreno GPU). Also, the phone model when googled shows the
| S20 FE, which has Exynos.
| janvidar wrote:
| There are two variants.
|
| The standard S20 used Snapdragon in North America, and
| Exynos in Europe. For the S20FE the Snapdragon version was
| sold in Europe - which was a selling point by itself.
|
| I did not know that the S20FE had Exynos at all until I
| just looked it up... Apparently version 1 does, while
| version 2 uses Snapdragon.
|
| https://www.gsmarena.com/samsung_galaxy_s20_fe-10428.php
| yread wrote:
| I have it and when I bought it you could get both in
| Europe. I planned to send it back if it would be
| exynos...
| johntb86 wrote:
| One of the crashes listed is https://crash-
| stats.mozilla.org/report/index/f585f1a9-0ed8-4... . That lists
| a "Mali-G77" GPU, which isn't used by Qualcomm. So the affected
| chip is probably the Exynos 990, which uses
|
| * Octa-core (2x2.73 GHz Mongoose M5 & 2x2.50 GHz Cortex-A76 &
| 4x2.0 GHz Cortex-A55) - Global
|
| * Octa-core (1x2.84 GHz Cortex-A77 & 3x2.42 GHz Cortex-A77 &
| 4x1.80 GHz Cortex-A55) - USA
|
| (from https://www.gsmarena.com/samsung_galaxy_s20-10081.php )
| SketchySeaBeast wrote:
| Given that it's only effecting S20's I'd be surprised if it was
| the snapdragon 865 as that's in a ton of phones as you point
| out. I'd expect it to be the Exynos 990, which is Samsung's
| custom chip for every region but North America and was only
| used in the S20 series.
| scrlk wrote:
| Wouldn't be surprised if it was an Exynos bug either. Samsung
| LSI haven't produced a decent flagship Exynos SOC for
| multiple generations now. Always behind Snapdragon on overall
| balance of performance (CPU/GPU/radios), heat and battery
| life.
|
| I refuse to buy any phone with an Exynos SOC ever since I got
| screwed with a poorly tuned Exynos 9810 in my S9. Andrei
| Frumusanu did a great two part article on turning the 9810
| when he used to write for Anandtech:
|
| https://www.anandtech.com/show/12615/improving-
| exynos-9810-g...
|
| https://www.anandtech.com/show/12620/improving-the-
| exynos-98...
| AshamedCaptain wrote:
| There used to be a reason: Exynos variants had unlockable
| bootloader. Not sure how feasible unlocking is nowadays on
| the Qualcomm variants.
| scrlk wrote:
| IIRC locked bootloaders on Qualcomm variants was found on
| US market phones, with non-US (e.g. Hong Kong) markets
| having a unlockable bootloader.
| fifteen1506 wrote:
| Meh, decisions decisions. I bought a Pixel 6a which uses
| Samsung technology for the Google SoC. I bought it anyway
| because I wanted to install GrapheneOS but since I assumed
| it was evolved-but-still-Exynos I did hesitate.
| dotdi wrote:
| A few years ago I also ran into a hardware (or at least very low-
| level software) bug that was only affecting Samsung devices.
|
| I had taken over maintenance of a library that allowed predicting
| satellite orbits from TLE data, which was used by the European
| Space Agency (and others) for the occasional mobile app.
|
| Sporadically, we were getting reports of strange situations where
| the altitude of a satellite was way off, by hundreds or thousands
| of kilometers. The bug was really difficult to track down and
| reproduce, and by chance one of our Samsung tablets finally
| started showing the behaviour.
|
| It turned out that Dalvik (the now obsolete Android VM) was
| figuring out that the TLE calculation was in a hot path and would
| JIT it to run more efficiently, and the optimized code used some
| native arithmetic (or was it trigonometic? can't recall) call
| that had a bug. We saw that the incorrect values started showing
| up right around the time when the JIT started kicking in.
| Fortunately, we could turn off JIT for specific devices and we
| rolled out an update that fixed the issue.
|
| Fun times!
| spuz wrote:
| How do you trace what the JIT is doing in an Android app?
| LispSporks22 wrote:
| The only thing better than being able to blame the compiler for
| something, is being able to blame the CPU
| peheje wrote:
| Saving that quote.
| print_goto_ten wrote:
| Great tshirt
| amelius wrote:
| But how are you going to fix it then?
| wongarsu wrote:
| Change your code to not trigger the bug, and leave that
| workaround there for the next decade or so until the buggy
| CPU generation isn't relevant anymore.
|
| Or hope for microcode updates to fix the bug at the CPU
| level.
| JonChesterfield wrote:
| You change the compiler to carefully step around the stuff
| that doesn't work in the hardware.
| omoikane wrote:
| Summary: Samsung's LDRB doesn't zero-extend when reading a byte,
| so it's not following ARM specs[1]. Fix was to workaround in the
| JIT[2].
|
| [1] https://bugzilla.mozilla.org/show_bug.cgi?id=1833315#c13
|
| [2] https://hg.mozilla.org/mozilla-
| central/rev/c6a272593c07#l7.1...
| jeffbee wrote:
| This is where the vocal minority of HN posters who are in a snit
| about telemetry are wrong. This is the kind of issue you can
| discover at global scale, but only if you collect the data.
| burkaman wrote:
| Personally I don't have a problem with automatically sending
| the type of telemetry that makes detecting this possible, but
| Mozilla could also discover this issue by in-house testing on a
| wide variety of devices, and/or prompting users to submit
| telemetry after a crash.
|
| It's not feasible for a smaller developer, but it wouldn't be
| unreasonable for Mozilla to have a big lab bench with like 100
| different devices on it that are just constantly running a
| script to start Firefox and load a suite of test sites.
| sjsdaiuasgdia wrote:
| I don't think you could really approach the kind of coverage
| automatic crash telemetry can give with a bunch of test
| devices. You will absolutely find a non-zero number of issues
| with the test devices, but it's not really the same.
|
| Without substantially increasing the cost and work to
| maintain the test pool, you'll be testing exactly one
| combination of OS version/patch level/settings, one internet
| connection medium/carrier, one set of browser config options,
| etc for each device in the pool.
|
| The investigation into any fault isolated to a single device
| in the test pool has to start with suspicion of the health of
| the specific device. The test S20 crashing doesn't tell you
| that every S20 will crash in the same way, it could be an
| isolated local fault. Yes you can throw more devices but
| that's further increasing the cost and effort of maintaining
| the test pool.
| jfk13 wrote:
| It's not remotely feasible to test more than a _tiny_
| fraction of the variety of devices, configurations, and sites
| that a product like Firefox is going to encounter in the real
| world.
| burkaman wrote:
| I agree you'd never catch everything, but if it's crashing
| on the latest Samsung flagship phone you could probably
| catch that.
| toast0 wrote:
| User volunteered crash telemetry is better than nothing, but
| automatic crash telemetry is going to have a more
| representative sample. A compromise would be to grab just a
| stack trace and phone model / firmware version etc
| automatically, and a full dump with user affirmative consent;
| but if that's not compatible with your privacy stance, asking
| everybody for everything is reasonable but limiting.
|
| Android has an immense variety of devices. Without a
| representative sample, you wouldn't know which devices to
| buy. Regional variants sometimes have a huge difference and
| sometimes not; this bug seems to be tied to non-US Samsung
| flagships, which you might not buy if you're US based and
| don't know what people are using.
| ptx wrote:
| You're wrong about them being wrong. Nobody is arguing that
| telemetry could not possibly have any upsides (primarily for
| the developer), only that the downsides (primarily for users)
| make the tradeoff unacceptable.
| dspillett wrote:
| Your point would be correct if that was the only telemetry
| being collected. The pile of stuff most things are sending back
| home contains much that would not be useful here but is
| potentially privacy breaking (or just purely for marketing
| purposes with no intention of being useful to the user).
| Similarly, in either case telemetry is fine if it is opt-in,
| and if extra is collected the opt-in is repeated (i.e. no
| stalking via the back door by changing what is collected and
| sent home after consent is given).
| always2slow wrote:
| [flagged]
| anderskaseorg wrote:
| Telemetry is not remote access, much less root access, which
| does not even exist on typical phones. Phone apps are
| sandboxed by the OS and can only collect information that the
| OS allows them to collect. It may of course be possible to
| abuse this limited information for tracking (and that abuse
| should absolutely be called out when it occurs), but it's not
| the same thing.
| jfk13 wrote:
| Irrelevant. Collecting telemetry such as Mozilla does isn't
| remotely like having remote root access to people's phones.
| jeffbee wrote:
| That's exactly the kind of derangement I am talking about.
| One gets the impression that the typical HN commenter has
| never worked on a software project of any consequence at
| all.
| mcronce wrote:
| One often gets the impression that large swathes of
| people (even on here) don't realize that software today
| is more complex than software from 1993
| nottorp wrote:
| > but only if you collect the data
|
| You spelled 'send crash dumps with the user's permission'
| wrong.
| SanderNL wrote:
| This minor issue that also is not Firefox's fault is not enough
| to sacrifice anything let alone privacy.
| dralley wrote:
| Unless it doesn't actually sacrifice any privacy.
| msla wrote:
| Normal software testing (remember that?) can catch bugs without
| invasions of privacy.
| howinteresting wrote:
| Have you ever worked on a large-scale software product used
| by millions of people across thousands of kinds of devices?
| Telemetry is always going to be useful.
| Nextgrid wrote:
| > Telemetry is always going to be useful.
|
| Useful for what? We've had an explosion of telemetry and
| "analytics" the past decade and software quality and
| functionality has only gone downhill. In some fields we've
| outright regressed compared to early 2000s tooling.
|
| A simple example is that despite all the telemetry,
| manpower and tech innovation, every single mainstream
| communications product out there is inferior to pre-
| Microsoft Skype (despite the latter being built with much
| less manpower and running on much more primitive hardware &
| bandwidth).
|
| Of course, not all of this devolution is to blame on
| telemetry alone, but I disagree that telemetry is some sort
| of requirement or game-changer when it comes to software
| quality. We've successfully built good quality software
| before it and within reasonable budgets.
| helf wrote:
| [dead]
| rollcat wrote:
| We've had crash reports for about as long as we've had computer
| programs. The problem is not in the mechanism itself, but in
| its abuse for (overwhelmingly, non-consensual) behavioral
| profiling or spying.
|
| Even when not abused to violate our privacy, the end result is
| often highly questionable. In place of good design, common
| sense, and user studies, companies try to extrapolate
| trends/patterns from data where there are none, resulting in
| mediocre decisions at best, and self-feeding cycles of
| devolution. ("Users are clicking this button a lot, let's make
| it larger" - users are actually clicking the button by
| accident.)
| Nextgrid wrote:
| Even if telemetry is useful (this is a rare example - the vast
| majority of telemetry is either useless or actively works
| _against_ the users), it 's still a breach of privacy and
| confidentiality (crash dumps - which are necessary to
| investigate/reproduce such bugs - can contain highly
| confidential data including passwords or session cookies).
|
| > but only if you collect the data.
|
| If the problem is impacting the user and they want it fixed
| they'll happily tell you. This bug (which causes crashes)
| would've been investigated just as well by asking users
| "Firefox has crashed - send bug report?", while giving the user
| the choice to decline if they had sensitive data in the
| browser's state.
|
| Obviously nobody in their right mind should opt-in to generic
| telemetry because all the "product improvements" excuses never
| panned out - in the last decade software has only become worse,
| more annoying, and less feature-full, so people are making the
| rational decision and are not opting into something that's not
| actually benefiting them. But opt-in telemetry in specific
| cases where the user actually has a problem like this
| occurrence can work just fine.
| temac wrote:
| Bullshit, CPU bugs have been found before wide usage of
| telemetry and will continue to be found in various context,
| number of them not doing it even if they could.
|
| Telemetry is probably just one more excuse to ship crap in the
| first place.
| n4r9 wrote:
| It looks like this could have been discovered by simply having
| (anonymised) crash report telemetry? I don't know about others,
| but that sort of telemetry doesn't sound so bad.
| Nextgrid wrote:
| It's very hard to have _useful_ anonymized telemetry. In this
| case, you actually need at least a partial memory or register
| dump to understand what 's going on, but those can contain
| sensitive data that the failing code was processing.
|
| Telemetry doesn't have to be 100% anonymized - it _can_ (and
| in some cases _needs_ ) to contain PII, just needs to be
| collected respectfully and transparently. Ask the user before
| sending something, and let the user review it. The user can
| then make their own decision.
| mixedbit wrote:
| Such crash data can be collected with an old fashion 'Firefox
| has crashed, send a problem report to Mozilla [OK] [Cancel]'
| dialog, which is much less privacy-intrusive than telemetry.
| ksenzee wrote:
| Right, and Firefox for Android does in fact collect its crash
| data like this.
| vardump wrote:
| Indeed. Telemetry is often used for good purposes, like
| improving features that cause trouble to end users and to fix
| crashes.
|
| I'm not saying bad actors don't exist, just that telemetry is
| often used for good, with no nefarious goals.
| FartyMcFarter wrote:
| It's never a compiler or CPU bug, until it is.
| garganzol wrote:
| When I was a kid, I always suspected a compiler or CPU bug when
| my program did not behave as expected. I heard similar stories on
| more than several occasions from other people, so being naive
| must be a common thing among us software engineers when we make
| our first steps. Kudos to Mozilla, their finding is a real CPU
| bug.
| 0xffff2 wrote:
| It's a good guideline when you're early on in your programming
| career, but I've been surprised by how many compiler bugs I
| have independently re-discovered in my career. One of the
| curses of my job is that I work with ancient compilers. About
| half a dozen times I have stumbled on a bug, spent the better
| part of a day debugging my code, only to find that I'm being
| bitten by a documented GCC bug that was fixed a decade ago.
| IshKebab wrote:
| One of the big reasons to keep software up to date IMO.
| mort96 wrote:
| Not that easy when you have to use some vendor's SDK which
| includes some forked GCC 4.x...
| londons_explore wrote:
| It's not a real CPU bug unless you can write a 20 line program
| in C/assembly and run it to demonstrate the problem.
|
| Then, when you've done that, send it to samsung so they patch
| it in their CPU design. They may even be able to patch it in
| their devices in the field (it's common for CPU designs to have
| special hardware whose purpose is runtime detecting certain
| trigger conditions and running replacement code to avoid
| hardware bugs discovered after the CPU was produced).
|
| They can also issue an errata which might let compiler authors
| modify their code generators to avoid any pattern that triggers
| the bug, therefore avoiding other projects being impacted.
| mastax wrote:
| If you step into microcontroller land you run into hardware
| bugs all the time, at least if you have to write any low-level
| code. My very first major project I ran into at least 3 silicon
| bugs which wasted weeks of my life. Rookie mistake: check the
| errata. Though one of the bugs wasn't in the errata.
| pengaru wrote:
| For ages in *nixland the phrase "select is not broken" has been
| an aphorism for such situations. I'm not sure of its origin,
| but it's referenced in The Pragmatic Programmer:
| > Tip #33, pg. 95: > > "select" Isn't Broken
| > > It is rare to find a bug in the OS or the compiler,
| or even a third-party > product or library. The bug is
| most likely in the application.
| syncsynchalt wrote:
| Ironically we later found that `select` is broken by design.
| `poll` was created as an improvement, and syscalls like
| `epoll` later refined it further.
| pengaru wrote:
| FD_SETSIZE ought to be enough for anybody
| toast0 wrote:
| Yeah, I don't know where that comes from. _Everything_ is
| broken.
|
| If you think select is broken, read the docs and make sure
| you're using it correctly. If you still think select isn't
| doing what the docs say, try to reduce your actual use into a
| minimal test case and see. Go look at the source for select
| on your OS. Search the internet and see if anyone else ran
| into the same thing (but be careful, because there's lots of
| bad leads out there).
| AshamedCaptain wrote:
| I really don't know when this became "common knowledge". Most
| OSes and compilers were utter crap not more than 10-20 years
| ago. We would spend afternoons ranting about MSVC++ bugs. gcc
| was a joke, and even if quickly improved, does nobody really
| remember that by the 2010's gcc was so buggy -O3 was barely
| usable? No Gentoo users around?
| RetroTechie wrote:
| I recall doing some Gentoo installs using -Os at least for
| some packages, just to see how much that would shrink
| binary sizes.
|
| Documentation / wikis etc. advised against that not just
| for performance-of-generated-code reasons, but also because
| of compiler bugs using -Os was considered unreliable.
|
| Note this was with ancient GCC (v2.9.x, early 3.x releases
| iirc)
| paulddraper wrote:
| `-O3` was known/documented to be unsafe.
| dkarras wrote:
| yeah same, from almost 30 years ago. Interesting shift. I
| distinctly remember having the instinct that it must be the
| computer that must be wrong, and what I wrote was obviously
| correct. Thanks for reminding me of that. With some experience,
| you figure out that you are wrong %99.99 of the time and after
| that loop, the thought never crosses your mind. I've never had
| to deal with hardware bugs myself, but had my share of compiler
| and platform bugs over the years.
| bch wrote:
| Here's the first story I read (~11 years ago) with an occurrence
| like this that cracked my concept of hardware/software divide...
|
| https://www.theregister.com/2012/03/07/amd_opteron_bug_drago...
| alphanullmeric wrote:
| Typical Samsung device. They're far too busy putting more ads,
| bloatware, worthless gimmick cameras and their own shit versions
| of google apps onto their phones to fix any bugs. Samsung
| represents everything bad that Apple users criticize android
| devices for.
| ComputerGuru wrote:
| This has absolutely nothing to do with the software ecosystem
| or marketing decisions. The Samsung microcontroller/CPU
| development team is from a completely separate _division_
| (Samsung LSI) and makes these CPUs for use both by (the other
| parts of) Samsung and outside vendors.
| perryizgr8 wrote:
| And every CPU in existence has errata. It's not like Samsung
| CPUs are any more filled with bugs than Qualcomm's.
___________________________________________________________________
(page generated 2023-08-09 23:02 UTC)