[HN Gopher] The weirdest bug I've seen yet
___________________________________________________________________
The weirdest bug I've seen yet
Author : jevans
Score : 278 points
Date : 2023-11-30 18:29 UTC (4 hours ago)
(HTM) web link (engineering.gusto.com)
(TXT) w3m dump (engineering.gusto.com)
| wobblyasp wrote:
| Such a tease. At least upload the gif so people can poke at it!
| yuck39 wrote:
| Agreed!
|
| I have absolutely no idea how a combination of grammarly and a
| specific gif would cause a browser crash though...
|
| Anyone here use the grammarly desktop app? Any additional
| clues?
| hyperhello wrote:
| I would guess the gif triggers a specific edge case that
| would crash, and grammarly is just a common enough addition
| to chrome with a lot of edge case triggers that it was
| identified. I'd guess this is on the Chrome team to fix soon,
| but yeah, at least post the spinner file for us!
| fifafu wrote:
| I'd guess it's because Grammarly enables full accessibility
| support in Chrome to be able to access the browser elements
| similar to e.g. a screen reader. This is off by default and
| has caused me various issues in the past when enabled (e.g. h
| ttps://bugs.chromium.org/p/chromium/issues/detail?id=136448..
| . ). However it's probably good that the Accessibility
| functions get more exposure due to this.
| 12_throw_away wrote:
| Well, this is a fascinating murder mystery that establishes 3
| compelling suspects - Grammarly, Chrome, and a gif - and then
| just ... ends, right before the big reveal.
| 0xNotMyAccount wrote:
| I have a friend who worked at Gusto, and my wife tried using
| Gusto for her small business (they handle payroll for small
| business, got a big boost from the pandemic). The lack of
| technical resolution here is so Gusto, it hurts.
| Andrex wrote:
| They nuked my account after trying to charge an expired card
| three times.
|
| The customer response team was extremely quick and responsive
| telling me their hands were tied.
|
| Fuck Gusto.
| Kognito wrote:
| Hate that.
|
| "Sorry, the system says no"
|
| Had similar situations with PayPal and Uber recently where
| their support have absolutely no information or ability to
| take a decision.
|
| Support essentially becomes a glorified text-to-speech
| system.
| robocat wrote:
| That's unfair: isn't this is exactly how most strange bugs
| get "fixed" by most companies?
|
| It is an abnormal developer and an even more abnormal
| business that actually spends enough time to find the root
| cause of outre glitches. Especially when you start having to
| debug complex third party systems to debug them properly -
| requires skills and motivation plus a company that will
| encourage a developer to do that.
|
| The story is not specific to Gusto - it is the story of every
| developers life. I have chased down bugs in my OS and my
| browser - it is rarely well rewarded! Fixing a compiler bug
| should be on my bucket list! A long time ago I worked around
| a compiler bug by inserting a label: (I think the label
| prevented certain optimisations where the label was put).
| tclancy wrote:
| I mean, this story is a hell of a rundown of debugging. The
| fact they don't have insight into the ways Chrome or
| Grammarly work isn't something to apologize for.
| Maxion wrote:
| I feel so unsatisfied
| fifafu wrote:
| Maybe it's because Grammarly enables full accessibility support
| in Chrome to be able to access all elements in the browser
| (similar to a screen reader). This has caused me various issues
| in the past (e.g.
| https://bugs.chromium.org/p/chromium/issues/detail?id=136448...
| ). However it's probably good that the Accessibility functions
| get more exposure due to this.
| chrismorgan wrote:
| The GIF _cannot_ be responsible: as untrusted web content, if
| it can trigger a crash, the responsibility lies with the local
| software stack. So you have only two suspects: Chrome and
| Grammarly. The GIF is at most an accomplice.
| sfink wrote:
| more like a murder weapon
| azlev wrote:
| Hash collision?
| tru3_power wrote:
| Is there a copy of the gif available? That's interesting
| digging wrote:
| I wouldn't even know how to look for something unusual in a
| gif's source code but I also feel this is the most compelling
| part. I wish they'd uploaded it.
| guessmyname wrote:
| > _I wouldn 't even know how to look for something unusual in
| a gif's source code but I also feel this is the most
| compelling part. I wish they'd uploaded it._
|
| GIF stands out as a widely understood file format [1][2].
|
| To kick things off, delve into the GIF file using a
| hexadecimal editor. HexFiend [3], for instance, offers a
| template for visualizing GIF file structures [4]. Another
| excellent option is Synalyze It! [5], which comes pre-loaded
| with an extensive list of file formats, encompassing GIF
| among others.
|
| These visualizations serve as a guide to pinpoint any
| irregular byte clusters that might pose issues when loading
| the file into an application with an image reader lacking
| support for that specific byte group or its arrangement. Once
| you've identified such a cluster, consider it the bug.
|
| [1] https://en.wikipedia.org/wiki/GIF#Example_GIF_file
|
| [2] https://www.w3.org/Graphics/GIF/spec-gif89a.txt
|
| [3] http://hexfiend.com
|
| [4] https://github.com/HexFiend/HexFiend/blob/master/template
| s/I...
|
| [5] https://www.synalysis.net
| dblitt wrote:
| > For security reasons, we do not have Chrome crash reporting
| enabled.
|
| > We also confirmed with many of our affected users that they had
| Grammarly installed on their computers.
|
| Ironic.
| roozbeh18 wrote:
| haha, that was questionable for me as well. It's ok for
| Grammarly to read your stuff, but crash metadata is a no no.
| JohnMakin wrote:
| Welcome to security in 2023 :)
| madeofpalk wrote:
| > Unfortunately, with access to neither the Chrome source code
|
| I mean, you basically do! You can just go check out the chromium
| source.
| flutas wrote:
| > I mean, you basically do! You can just go check out the
| chromium source.
|
| It's mentioned in the bullet points in the "trouble reproducing
| the bug" section that chromium wasn't affected.
|
| > Using open-source Chromium instead of Chrome did not cause
| crashes, so we couldn't see what Chrome code was failing
| either.
| j1mmie wrote:
| I wonder what version of Chromium they used. If it was a
| nightly, it could be weeks before the fix makes it into
| Chrome. They might've tested at a time when latest Chrome had
| the bug and Chromium didn't.
| orbv wrote:
| Google provides symbols for Chrome release builds, including
| source indexing so source code should be available. See
| sections symbol server and source indexing at
| https://www.chromium.org/developers/how-tos/debugging-on-
| win...
| trelliscoded wrote:
| I wish this was upvoted more. This is the correct way to
| troubleshoot the bug, full stop. You can get symbolic stack
| traces with full arguments and source code on Windows in
| about 5-10 minutes for any Chrome crash by following these
| instructions. I always have a last change exception handler
| that fires up a WinDBG script on Windows for our chromium-
| based test runners, which reduces troubleshooting time to
| just a couple minutes in order to find the symbol in their
| bug database. Playing blackbox what-if games like the Gusto
| team is a waste of time and doesn't contribute any
| situation-specific knowledge to bugs.chromium.org.
| simion314 wrote:
| I am using chromium to print web pages to pdf, and I have some
| images that will crash chrome's to pdf process, I found nothing
| wrong with this images, the metadata is fine (nothing weird in
| it). The other bad thing it does not reproduce n my dev machine
| only on the production server , so nothing I can do, in rare
| cases an image will always crash crhomium, I find it, open it and
| re-export it and then it works.
| LgWoodenBadger wrote:
| This sounds more like a hardware fault than something wrong
| with the software, especially since it doesn't seem to be
| deterministic.
|
| But stranger things have happened, and given the enormous
| surface area of a modern computer (hardware, software, drivers,
| state, etc.) can anything truly be deterministic?
| simion314 wrote:
| It happens with that image no matter what. I can have a html
| with 100 images and one bad image, I make one new html only
| with that image and it still has the problem. My guess is
| that probably a bug in a low level image decoder. My local
| machine has different kernel, different libs, plus I have
| different cpu,gpu and X11 on top so too much difference and I
| do not have the expertese to do aremote debug(or local)
| saagarjha wrote:
| If you can grab a crash log I am sure the Chrome team would
| be happy to look at it.
| saagarjha wrote:
| Most bugs are software issues, though.
| lloydatkinson wrote:
| Discord suffered (suffers?) from a similar thing with gifs. It is
| or was common for people to post specifically crafted gifs in
| channels, anyone viewing the channel immediately had their client
| crash.
|
| Discord client uses Electron, which is in turn Chromium.
| j1mmie wrote:
| What an interesting conflux of tech to create this bug. That's
| the web in 2023. I would love to know if it was a Chromium bug
| that got resolved, but navigating this is tough:
| https://bugs.chromium.org/p/chromium/issues/list?can=1&q=gif...
|
| Also, I am fully here for Gusto posting this to say "wasn't our
| fault" and to throw some shade at Grammarly in the process
| thenoblesunfish wrote:
| Seems like they're posting it because it was a fun story, and
| it's free advertising - this wasn't externally visible so I
| don't see where fault comes into it.
| nonethewiser wrote:
| They should definitely publish the gif
| computerfriend wrote:
| If they can figure it out, they're sitting on potentially a very
| valuable exploit.
| jdminhbg wrote:
| One that's been patched already, though, as they say that in
| current versions of Chrome and Grammarly it doesn't crash.
| chatmasta wrote:
| Yes, I too would like to read more details about this. It's a
| great writeup from an engineer who got stuck debugging this.
| But I hope some experts in security or reverse-engineering can
| replicate it and take a closer look. There's definitely a more
| interesting story here, probably regarding the localhost bridge
| between Grammarly extension and desktop.
|
| (Grammarly has a bug bounty btw... and their chrome extension
| has quite a large surface area...)
|
| If OP is here: can you provide the raw .gif file? (And if
| you're feeling generous, maybe even a minimal ruby example that
| replicates that templating setup, although it sounds like that
| wasn't required to reproduce it in the end.)
|
| P.S. "For security reasons, we do not have Chrome crash
| reporting enabled" - maybe consider disabling Grammarly
| extension for the same reasons ;)
| saulpw wrote:
| It wasn't the Grammarly extension, it was the desktop app.
| chatmasta wrote:
| I guess I just assumed the extension was installed too, and
| communicating with the desktop app. But now I see the post
| doesn't mention the extension. If it was triggered even
| without the presence of the extension then that's quite
| strange, and even more suspicious - is that gif triggering
| a call to a localhost endpoint? Is the grammarly desktop
| app interacting with browser elements without using the
| extension? (IIRC the grammarly app uses some accessibility
| privileges to inject into textareas across all apps)
|
| Grammarly is honestly insane, I can't believe corporations
| allow it to run on employee machines.
| Sophira wrote:
| I'd also be interesting in seeing the raw .gif file - as a
| hobbyist wannabe researcher myself, I'd love to investigate
| this.
| Modified3019 wrote:
| The true bug in the photo is a "candy-striped leafhopper",
| Graphocephala coccinea, which is tiny but has very striking
| coloration.
|
| The Larvae of leafhoppers are commonly known as spittlebugs,
| which create protective bubble nests while feeding on plant stems
| barbegal wrote:
| A disappointing ending to the tale. I really want to know how
| Grammarly desktop works now. It must have interfered with the
| Chrome process in some bad way.
| whirlwin wrote:
| Shared library like libgif used by both Chrome and Grammarly but
| different versions?
| JohnMakin wrote:
| I have a saying that isn't perfectly true but often will apply to
| "fixes" like this -
|
| If you don't know why the fix worked, you may not have actually
| fixed it.
| bicijay wrote:
| But you may have
| lbhdc wrote:
| Ahhh, Schrodinger's patch.
| HPsquared wrote:
| Schrodinger's bug?
| gwbas1c wrote:
| But they couldn't fix the bug: The bug was in another product
| that they couldn't access source code or submit patches.
|
| The best they could do was work around it.
|
| Sometimes workarounds are the best you can do until your vendor
| provides a real fix.
| JohnMakin wrote:
| I didn't mean literally fix the underlying bug. They also
| don't really know why their workaround worked, which means it
| could not really be fixed at all.
| tetha wrote:
| Someone once said, there is accidental function, and deliberate
| function.
|
| If your system doesn't work, and you just plonk around at
| values, until, very surprisingly, the system starts behaving
| well and you the call it working... well it might be working
| now. But it's just accidental correctness. As soon as something
| causes the system to bank left, something's gonna break and no
| one knows how to fix it - and you're back to square one.
|
| On the other hand, as hard as it is, if you can clearly tell
| why your fix will restore function to the system without even
| applying it, you have deliberate correctness and function. If
| done right, it is very boring, because exactly and only the
| expected thing will happen. You should know about the unknowns
| and plan around those as well, so even if an unknown bites you,
| it's a known and handled unknown. This can be exhausting to
| make happen, because it is much harder, but those systems will
| just work.
|
| But this is a fight I have with some development teams probably
| forever. "But we poked at the values, and that stopped the
| flames. It is fixed!" "but why?" "Dunny. But no fire anymore.
| All good." And then 2 weeks pass, and there is more fire and
| everyone is like "Oh but why would this happen? How should we
| have known for this to happen again"
| saulpw wrote:
| On the other hand, I've spent weeks with a team looking for a
| bug, and by the time we found something that appeared to fix
| it, we were way behind on everything else that really needed
| to get done. How long would it take to find the root cause?
| We tried. It wasn't worth weeks or months of effort, to
| anyone. This isn't JPL and human lives weren't on the line.
| We just needed it not to crash so we could all get on with
| the "real" task of shipping useful and profitable software.
| tetha wrote:
| Yeah, that is why software engineering and system
| operations is hard.
|
| For example, the article doesn't get to a root cause in an
| absolute way. There is no absolute SEGFAULT of the OS
| causing the misbehavior. However, they nail down the crash
| to a gif, and if the gif is in, it crashes, and if the gif
| is out it doesn't. If the gif is loaded otherwise it
| crashes, too. At that level, to me, that would be enough,
| because we're users of the browser's rendering there.
|
| Finding a solid cause that can demonstrate and reproduce a
| problem, and basing a workaround around that at a boundary
| you're unwilling to cross can be fine. If it's within the
| company, it absolutely is fine as long as you escalate
| beyond that boundary.
|
| However, I have enough teams who are like "Oh, we set all
| values to 25 one by one and when we arrived at flum-value
| at 25 it stopped crashing. Fixed." Why 25? Who knows. Why
| flum? Who knows. Maybe the other value changed at the same
| time fixed it? Who knows. Do we use 26 once it starts
| crashing again? Fuck knows. Maybe 24 is better?
|
| We have no explanation for 25, so why would 25 be a good
| fix?
| derefr wrote:
| And in fact, I don't think they _have_ fixed it. I 've seen
| "Error 5" plenty of times in Chrome. It seemingly occurs
| whenever I have a lot (100+) of tabs open for any site where
| each page allocates at least one accelerated drawing canvas (a
| literal <canvas>, or a <video>, or a .gif <img>.) I've seen it
| happen on Reddit (but only new reddit, not old reddit!) and on
| a number of other sites.
|
| I hypothesize that Chrome simply has a global (i.e. cross-tab)
| per-toplevel-origin limit to the number of allocated
| accelerated drawing canvases it's willing to allow; and that
| when you go over it, Chrome forcibly _de_ -allocates all the
| existing drawing canvases used by other tabs that have that
| toplevel origin loaded, thereby causing them to crash. It's
| _probably_ a measure designed to prevent a site from from
| DoSing your computer by just allocating an infinite number of
| canvases.
| djbusby wrote:
| Thought this was going to be about Gusto "remember this device"
| which keeps failing. Reported like 2 years ago
| sonicanatidae wrote:
| Twirly prompts have sucked, for weird reasons, since the WWIV BBS
| days.
|
| I guess some things never change.
| zwieback wrote:
| Left me hanging, would not post something like this from my
| engineering blog. They don't have deeep debugging skills at
| gusto?
| golergka wrote:
| Sounds like they have good time management and prioritisation
| skills. They found the source of the problem and fixed it.
| KerrAvon wrote:
| No, they didn't. They figured out a workaround. Since they
| didn't find the root cause of the problem themselves and
| apparently didn't take it up with the Chrome or Grammarly
| development teams, they don't actually know what happened or
| when it might bite their customers again.
| jdminhbg wrote:
| They say it no longer reproduces on current
| Chrome/Grammarly, so taking it up with them is fruitless.
| They're not going to investigate crashers from old
| versions.
| dhritzkiv wrote:
| :( I would've tried to determine the cause of the crash with
| that specific file in my off time, provided that I could
| isolate the code in the Grammarly extension in Chrome.
|
| The main reason -other than curiousity- is to ensure that a
| future regression (in Chrome/Grammarly) wouldn't lead to it
| again.
| cyco130 wrote:
| The weirdest one I saw was this: User claims that the _wording_
| of the info they enter into a certain form changes when they
| save. At first I suspected someone else editing the same form at
| the same time unbeknownst to each other but it wasn 't the case
| according to the logs. And I saw the correct wording on my own
| computer.
|
| Then I noticed in their screenshots that some of the menus had
| weird wording too. Turns out they had Chrome's "Translate this
| page" option on. Problem went away when we showed them how to
| properly switch languages in the app.
| bee_rider wrote:
| What was translating from English to American or something?
| cyco130 wrote:
| From English to Turkish but Google won't just leave the parts
| that are already in Turkish alone and subtly reword them
| instead for some reason.
| robocat wrote:
| I added <meta name="google"
| content="notranslate">
|
| to all pages in a single-page-web-app after discovering some
| bug or other with Chrome screwing up the page.
|
| Apparently the new incantation to fix an app (can be applied to
| an element) is (ugly: I presume it isn't CSS to avoid
| supporting dynamically changing it): <html
| translate="no">
|
| https://developer.mozilla.org/en-US/docs/Web/HTML/Global_att...
|
| Every now and then I would look at the meta tags for a major
| single page app and discover some new horror when searching for
| the reason for the meta tag!
| michaelcampbell wrote:
| I know what you mean, but this caused me a second of "wait,
| what?"
|
| > all pages in a single-page-web-app
| robocat wrote:
| Good point. This was the Elizabethan days when computers
| ran on coal: IE when we were explorers of The Internet.
|
| We were bleeders, but there still existed a vestigial login
| page, and some other evil cthulic pages (I know whence they
| were begat for I was their father).
| jonathrg wrote:
| <html translate="no">
|
| I first read this as "translate to Norwegian"
| cyco130 wrote:
| I wouldn't want to disallow translation, but in this case it
| was unnecessary anyway.
| jrockway wrote:
| Did they open a bug against Chrome with the image file? I feel
| like any crash on user-provided data is a big deal, always a
| correctness problem, but potentially a security problem. "We
| deleted the image so the problem is fixed for us" is OK (I
| wouldn't waste time writing a blog post about it personally), but
| I think that Chrome needs to fix this bug.
| tedivm wrote:
| Was it actually a Chrome bug though? It only happened when the
| Grammerly desktop app was installed. My guess is grammerly is
| doing something sketchy.
| bayindirh wrote:
| Maybe adding Grammarly created enough of a lag causing the
| GIF file to be shown?
| AnimalMuppet wrote:
| Does Grammerly hook something in Chrome? If not, then it's
| still probably a Chrome bug, even if some second-order effect
| of Grammerly is necessary to trigger it.
| majormunky wrote:
| It looks like the desktop Grammerly app hooks into all
| sorts of things, "An all-in-one writing assistant that
| works on your desktop and in your browser. Use it in apps,
| word processors, email clients, and more."
| MattDaEskimo wrote:
| I'm thinking the same thing. It could be that Grammerly
| injects it's own loading spinner with the same filename
| into the HTML.
|
| I wish they tried to simply rename the file instead of
| remove it.
| meandmycode wrote:
| The pr seems to suggest it's not the filename though
| given the new file was named the same and didn't crash.
|
| I would guess grammarly is hooking chrome and potentially
| trying to read metadata about images, and the particular
| gif had metadata in a format they hadn't expected.
| user3939382 wrote:
| Reminds me of the story I read where the guy's car wouldn't start
| depending on what flavor of ice cream he picked and when
| investigated he was right. Some kind of evaporation/vacuum leak
| or something that was dependent on time and some flavors were
| farther away in the store and took more time to buy.
| nudgeee wrote:
| Vapor lock. Legend here: https://www.snopes.com/fact-
| check/cone-of-silence/
| RajT88 wrote:
| I heard a similar tale in high school.
|
| A friend of mine had an aunt who passed away, and so he ended
| up inheriting her car. The car came with a petrified apple
| pie in the back. He was insistent that the car would not
| start without the pie in the back window.
|
| Several of his friends who he played in a punk band with
| confirmed this, that they had tested it. Take the pie out,
| car won't start. Put the pie back in, the car starts.
|
| I don't think anyone ever figured out what was going on, I
| graduated a couple of months after hearing the story, and
| fell out of touch. But - timing and vapor lock makes sense,
| if they were always testing it by first starting the car,
| removing the pie, and then putting the pie back in.
|
| As an aside, the aunt who had passed away was one Aunt Martha
| (after which the car was also named), which in honor of the
| strange car and its strange pie was what their garage punk
| band was named after. There's some totally unrelated band now
| called Aunt Martha - any evidence of their band is not on the
| internet.
| superfrank wrote:
| Up there with the "I can't send an email over 500 miles" story
|
| https://web.mit.edu/jemorris/humor/500-miles
| gostsamo wrote:
| This one is a legend. I love it, but you can find the debunking
| on the fact checking sites.
| trehalose wrote:
| The Snopes page doesn't really seem to _debunk_ it, but
| merely points out that the legend 's been retold with many
| variations and contradictory explanations. Suspicious,
| definitely, but it doesn't seem clear that none of the
| variations could ever have happened?
| IshKebab wrote:
| It's a just-so story. The null hypothesis is that it's not
| true.
| sfink wrote:
| That is the definition of null hypothesis, yes.
|
| Not to be blunt, but you might get a closer shave with
| Occam's Razor.
| mgaunard wrote:
| The main reason for tabs to crash is running out of RAM.
|
| Never do you see the guy investigating memory usage, which is
| weird.
| marcellus23 wrote:
| Yeah, that would have been a good first step, but he does admit
| to not really knowing much about browsers:
|
| > This was fairly far outside the usual scope of our on-call
| issues. Our team is generally well-insulated by other teams
| from issues like browser compatibility, so I didn't know the
| first thing about browser debugging.
| gwbas1c wrote:
| If that were the case, I think the bug would be much easier to
| reproduce; and be a lot more widespread.
| neilv wrote:
| Why would they disable crash reporting for security reasons
| (which might actually help solve the root cause of their
| availability problem, which they never did solve)... yet run
| Grammarly (which I'd guess, security-wise, is less trustworthy
| than Google, in how they secure data themselves once they've
| inevitably stolen it from the customer)?
| Zetobal wrote:
| Maybe they have the enterprise licence with grammarlys pinky
| swear that they won't train on your data.
| hilux wrote:
| What a cool mystery-solving post! I wish all technology writing
| were this clear and explanatory.
|
| For another fun debugging tale, google: Mazda radio Seattle NPR
| bug
| realmike33 wrote:
| This reminds me of similar issues I've encountered as a software
| engineer. I first ran into this issue about a decade ago, albeit
| not because of Grammerly,but due to some specific gif causing web
| app to crash. Both times the gifs were animated. Happened years
| apart and at different companies.
|
| I see some comments highlighting RAM, which could totally have
| been the issue. Totally looking forward to a follow up to this
| later down the road, I am sure this isn't going to be the last
| time we hear of this.
| neilv wrote:
| > _Using open-source Chromium instead of Chrome did not cause
| crashes, so we couldn't see what Chrome code was failing either._
|
| They don't address why they didn't just run Chromium. Or Firefox.
|
| (This is potentially better than the 'solution' they much later
| ended up with, in which they probably only relieved a symptom of
| an underlying problem that can exhibit again, and in the meantime
| is a zero-day exploit waiting to happen. At least, with a
| different browser, there's a chance that the vulnerability
| doesn't actually exist, when it's known to exist in their Chrome
| configuration.)
| Etheryte wrote:
| What makes this doubly frustrating is that they also didn't
| report the bug to Chrome. It's super easy to do, plus they're
| very responsive if you have a repro case which in this case
| they do. I think I'm now up to three or four Chrome bugs
| reported that their team has subsequently fixed.
| ncann wrote:
| They said it wasn't reproducible anymore though. So if they
| make a bug report now and say "this used to cause a crash in
| an old version of Chrome while also having an old version of
| another software installed, but is no longer reproducible in
| latest builds", most people would probably just ignore it.
| masto wrote:
| It was reproducible at the time they found it, and
| trivially so: install Grammarly, drag this GIF into Chrome,
| and it crashes. I understood everything up to the point
| where they just changed the GIF and moved on without ever
| telling the Chrome or Grammarly folks about it.
| gwbas1c wrote:
| > They don't address why they didn't just run Chromium. Or
| Firefox.
|
| The article implies that Gusto's employees can whatever browser
| they want.
|
| And, honestly, telling your employees to run a browser that
| only techies have heard of sounds like a really dumb idea.
| neilv wrote:
| > _The article implies that Gusto 's employees can whatever
| browser they want._
|
| For users of their security-sensitive internal software?
|
| > _And, honestly, telling your employees to run a browser
| that only techies have heard of sounds like a really dumb
| idea._
|
| Sounds like they're using this for internal tools, as a kind
| of thin-client layer. They could recommend or mandate a
| particular browser, and people would just use it. ("Click
| this icon, and a window opens with our internal tool. It's
| pretty much the same as any other browser, as far as you
| care.")
| saagarjha wrote:
| As an employee I would be really upset if you forced me to
| use a specific browser to do my work.
| kube-system wrote:
| > They don't address why they didn't just run Chromium. Or
| Firefox.
|
| Probably:
|
| 1. Because it is reasonable to expect the application to work
| in Chrome.
|
| 2. Chromium isn't intended for production use cases.
|
| Back when IE and Chrome had about equal market share, I worked
| somewhere that had one team insisting that all employees must
| use IE for one of their applications, and another team
| insisting that all employees use Chrome for their application.
| 50%+ of support calls were employees confusing the two
| browsers.
| MattDaEskimo wrote:
| I don't think the post you quoted is implying that they
| should've closed their eyes.
|
| It makes much more sense to try a different browser first and
| see if the problem persists. Instead of test versions and
| extensions.
| kube-system wrote:
| The post I quoted itself quoted the article saying they did
| test in Chromium. The article also says they tested Firefox
| and Safari.
| hbn wrote:
| > As urgency waned because our users were using other
| browsers as a workaround, progress on this bug slowed to
| make way for other priorities. We didn't have much left to
| go on without being able to reproduce the bug. However, we
| wanted to resolve it since users had
| bookmarks/settings/preferences in Chrome. We believed that
| we shouldn't have to ask our users to avoid the world's
| most popular browser, and we were also still getting
| periodic pings from various users asking whether we had
| made any progress on this bug.
| lobf wrote:
| You must not have read the article because he literally
| addresses this.
| mplewis wrote:
| *She - the author's name is Amy
| mplewis wrote:
| You really want to change an entire company's mandated browser
| every time a bug causes a problem with an extension?
| nabakin wrote:
| They say in the article that the bug became a much lower
| priority because their employees simply switched browsers
| 1123581321 wrote:
| How could you write all this and not post the gif so we can try
| it?
| andrewfromx wrote:
| oh and there's this one:
| https://web.mit.edu/jemorris/humor/500-miles
| tgsovlerkhgsel wrote:
| Extremely disappointing that they seem to have neither
| investigated nor enabled others to investigate (e.g. by filing a
| bug against Chrome).
|
| This smells like a potential security vulnerability.
| ljm wrote:
| I was hoping for a bit more of a payoff. Like, if the gif was
| broken, why did the grammarly extension trigger it reliably?
| jiveturkey wrote:
| I'm guessing this is the webp bug.
|
| The auto conversion to webp on the backend, signaled by chrome,
| resulted in a bad image that crashes the browser due to grammarly
| parsing of said bad webp.
|
| Safari doesn't tell the server it does webp and so it downloads
| the actual gif, and doesn't crash.
| saagarjha wrote:
| GIFs don't get automatically converted to webp files.
| romanhn wrote:
| Another favorite bug investigation of you're into this sort of
| thing: https://www.pagerduty.com/blog/the-discovery-of-apache-
| zooke...
| ecshafer wrote:
| I don't think I would believe myself if I found this being a
| specific gif. This is a great amount of coincidences in code to
| cause this.
|
| Grammarly is an application that I don't get. The fact that
| people are installing, basically spyware, on their computers just
| to get grammar suggestions to make their writing more boring and
| add a spellchecker (which is already inside web browsers) is
| pretty astounding to me. The fact companies allow employees to
| have it, despite obvious security issues of sending everything
| one types to a saas, is even more wild.
| eichin wrote:
| I worked with someone who _really_ needed it, but we had the
| usual "keep sales users as far from the actual product as
| possible" organizational isolation so it worked out in
| practice. (For engineering, it was on the "don't install this
| in particular" list.)
| _jal wrote:
| > Grammarly is an application that I don't get.
|
| You write like a native speaker, so I'm not surprised. But
| imagine having a few years of school-German, and then taking a
| German language job. I'd bet there would be times you'd want a
| writing assistant, too.
|
| There are also plenty of native English speakers who for
| whatever reason got a crappy education, and didn't get a lot of
| writing feedback.
|
| As far as corporate security goes, you are correct, and we ban
| it. But I get why people want it.
| ryandrake wrote:
| > As far as corporate security goes, you are correct, and we
| ban it. But I get why people want it.
|
| That is what stuck out to me: Installing rando applications
| on your corporate computer that has access to internal
| stuff... Whoooaaaa Baby! That's just a security disaster
| waiting to happen. It's stuff like this that eventually leads
| to draconian and crappy "Nobody gets admin access to their
| machines" corporate policies coming down.
|
| Most TechCorp places I worked, if someone installed something
| like that on their corporate device, they'd get at least a
| stern talking-to and probably sent back to security training.
| generationP wrote:
| Learning a language at school, you will soon be better than
| natives at grammar. It's the vocabulary, idioms and
| implicatures that will be tripping you up. Does grammarly
| really help with those?
| natbennett wrote:
| There are a lot of people whose professional outcomes are
| meaningfully constrained by their ability produce clear
| business English.
|
| I know a guy who used to get inexplicable feedback about his
| communication that boiled down to "write better." This limited
| his ability to get promoted. He runs all his comms through
| ChatGPT and asks it to "make this more professional" and
| doesn't get that feedback anymore.
| ianlevesque wrote:
| I get that people don't care or understand this, but that's
| also saying he cc's OpenAI, and therefore probably Microsoft,
| and therefore almost definitely the NSA, on all his business
| communications. What a world.
| beebeepka wrote:
| I've seen people do this on the same week as mandatory
| trainings featuring this exact scenario. At multiple
| companies
| gnulinux wrote:
| People won't care until something major happens and after
| that they'll implement some draconian half-measure that
| doesn't fix anything like snooping on office WiFi.
| almostnormal wrote:
| > [...] and therefore probably Microsoft [...]
|
| Where it will go through teams, outlook/exchange, or O365.
|
| Not leaking data is no longer as easy as it used to be.
| Just some forms are more accepted than others.
| pixl97 wrote:
| As good as the average corporate IT security is that I've
| witnessed via my work, passing said data to NSA/OpenAI is
| the least of their issues. Far less scrupulous hackers are
| running amok as it is.
| zztop44 wrote:
| About 0.1% (0.001%??) of business communication might have
| adverse consequences for you/your company if forwarded to
| Microsoft or OpenAI or the NSA. The rest is absolutely
| fine. And you're probably already using Gmail or Android or
| Chrome or Exchange365 or iOS or *something* that could
| theoretically forward your comms to a tech company (and the
| security state).
|
| Compared to the alternative of having your colleagues think
| you're a bit stupid just because you were raised speaking a
| language other than English, or your parents weren't middle
| class... using Grammarly or ChatGPT is a no brainer. I'd
| support anyone using whatever tools they can to overcome
| discrimination and thrive.
|
| The alternatives are:
|
| 1. Educate everyone in the company to stop discriminating
| against people based on language ability (impossible??)
|
| 2. Provide a local self-hosted version of the tools
| (although as a worker at RandomCorp, I would probably
| prefer to forward all my comms to Microsoft than to
| management!)
|
| 3. Tell people facing discrimination to just shut up and
| deal with it.
| Muromec wrote:
| I suspect it all started with two Ukrainian who got tired of
| checking how much of "a" and "the" they forgot to sprinkle
| into their texts.
| notpachet wrote:
| I have far more understanding and patience for non-native
| English speakers making those sorts of mistakes than I do
| for native speakers.
| pavel_lishin wrote:
| I read comments online, and in my experience the most
| difficult writing to parse isn't from foreign speakers who
| drop articles or mis-conjugate things - it's from people
| whose writing is just, for the lack of a better term,
| _bad_. This is very common on places like Nextdoor or
| Facebook.
|
| It's things like:
|
| - total stream-of-consciousness gibberish that could
| probably be assembled into a coherent statement if the
| writer would re-read what they wrote and edit it
|
| - A complete lack of punctuation, or even understanding of
| sentence and paragraph structure; at a glance, it looks
| like what I described above, but it's different because
| there's definitely a topic and a point they're looking to
| make, but they can't put the words together correctly.
|
| - spelling so bad, that even with context, it's unclear
| what word they're intending to use.
|
| - A wild misunderstanding of how to start and stop
| conversations online. (One recent example is me asking
| someone on Facebook if I could stop by to check out a
| garage sale, and a clarifying question about a term they
| used, only to get the response "ok." Note that in their
| post, they didn't specify an address beyond the name of the
| town they live in.)
|
| You can definitely point out flaws in the way I grew up -
| somewhat solitary, spending a lot of time alone in my room
| on a computer connected to the internet - but I think that
| it at least taught me how to make myself understood in
| written form.
| jvanderbot wrote:
| > - total stream-of-consciousness gibberish that could
| probably be assembled into a coherent statement if the
| writer would re-read what they wrote and edit it
|
| This drives me _nuts_. Did anyone see this [1] on HN the
| other day? People in comments were springing up to defend
| this atrocious writing style.
|
| Make a paragraph. Make a point.
|
| 1. https://news.ycombinator.com/item?id=38275905
| pavel_lishin wrote:
| I'm not a huge fan of that, but it looks like poetry, and
| what's more, it looks _intentional_. The author was going
| for something, and is probably aware that some folks won
| 't like it.
|
| That's a whole different beast from an email I'll get
| from a coworker/neighbor where I cannot parse what's even
| being asked of me, and where the writing is so confusing
| I don't even know how to ask them to clarify their
| statement other than to tell them to start over, possibly
| all the way from kindergarten.
| thaumaturgy wrote:
| People that are comfortable with text-based forums may not
| realize the extent of illiteracy and semiliteracy in the US.
| Decades ago, a small company was able to convince most of the
| education system (public and sometimes private) to use a
| teaching method based on junk science. The end result is that
| there are many millions of adults in the current workforce who
| can barely read, and many of them work in office settings. Some
| of those would install anything that would help them through
| text-based communications.
|
| [1]: "How a flawed idea is teaching millions of kids to be poor
| readers" https://www.apmreports.org/episode/2019/08/22/whats-
| wrong-ho...
|
| [2]: "Sold a Story: How Teaching Kids to Read Went So Wrong"
| https://features.apmreports.org/sold-a-story/
|
| [3]: "According to the U.S. Department of Education, 54% of
| U.S. adults lack proficiency in literacy, reading below the
| equivalent of a sixth-grade level"
| https://old.reddit.com/r/todayilearned/comments/rqulik/til_t...
| mardifoufs wrote:
| I'm not sure that's specific to the US, and I don't even
| think that particular teaching method has been used here in
| Quebec, yet we still see broadly similar literacy rates and
| levels.
|
| Last I checked US students rank well and are near the top in
| most education global rankings, so I think bad education is
| more of a global problem than Americans think it is. Maybe
| that's outdated though, I'll do my research.
| Aerbil313 wrote:
| That's definitely outdated. Literacy rate of my "third
| world" country is %16 higher than US atm.
| Retric wrote:
| Be careful trying to compare countries or even historical
| numbers when standards vary. The US has a 99% literacy
| rate based on some metrics, but as often happens when
| metrics that become useless the people tracking it raise
| the bar.
|
| Thus the US's "Level 1" literacy rate, which represents
| being able to follow basic written instructions, was 92%
| in 2014. But in 2020 the standard changed yet again to:
| "54% of adults in the United States have English prose
| literacy below the 6th-grade level." Noticeably being
| literate in a non English language suddenly doesn't
| count, the prose at 6th grade level is also higher than
| it's been in the past.
|
| Or as Wikipedia puts it: _In many nations, the ability to
| read a simple sentence suffices as literacy, and was the
| previous standard for the U.S. The definition of literacy
| has changed greatly; the term is presently defined as the
| ability to use printed and written information to
| function in society, to achieve one 's goals, and to
| develop one's knowledge and potential.[3]_ https://en.wik
| ipedia.org/wiki/Literacy_in_the_United_States
| gumby wrote:
| > Last I checked US students rank well and are near the top
| in most education global rankings, so I think bad education
| is more of a global problem than Americans think it is.
|
| US is at the bottom of the OECD PISA rankings (as it is
| with life expectency too), though on a global basis you're
| right (better than Morocco or Indonesia on both criteria).
|
| Shockingly Australia has fallen quite a bit from the
| initial PISA study where it was ranked #4, now almost as
| bad as USA.
|
| https://www.datapandas.org/ranking/pisa-scores-by-country
| mardifoufs wrote:
| Honestly what surprised me the most from your very
| informative link was that France is lower than the US!
| I'm probably biased but I've always considered the French
| education system to be quite rigorous. Especially
| compared to canada, which in my experience has a rather
| weak curriculum.
|
| (Also, I didn't know Morocco was that high all things
| considered. Though it makes sense considering how popular
| Asian style after school tutoring is nowadays. It's a bit
| insane, some kids do 4h of that every single day, and in
| urban areas almost every kid is enrolled in one of those
| schools.)
| akira2501 wrote:
| It's odd to read the story of an adult who believes they're a
| poor reader, still to this day apparently, because of what
| happened to them 30 years ago. Odder still that the article
| leaves itself the only conclusion of going all the way back
| to grade school and trying an entirely different strategy and
| hoping that just "works out" in the end.
|
| The lack of "continuing education" in the era of the internet
| is baffling to me.
| thaumaturgy wrote:
| I think about this a lot, too. My academic interests are
| pretty broad, and I could improve in every subject, so why
| don't I? I think there are two reasons: a lack of _focused_
| effort, and the steadily increasing demands of adulthood.
|
| I do reasonably okay at self-guided education when I want
| to, but there's definitely a difference vs. a structured
| secondary education environment, where there is
| accountability and other people to guide each other through
| the process. And, that's coming in to those subjects with
| already a better-than-average literacy and numeracy; I have
| to expect that for people who struggle with grade school
| reading comprehension or math, trying to bootstrap those
| abilities alone would be daunting.
|
| Also, there's just less room for pursuing those now. Lots
| of people are getting squeezed by concerns that aren't part
| of most childrens' awareness -- housing costs, bureaucracy,
| the treadmill of maintaining all the machines that get us
| through daily life. Those add enormous pressure to dedicate
| more time towards professional development and "getting
| ahead", or at least not falling further behind, and that
| has been eroding all of the unstructured time that I would
| spend working my way through a textbook (or online class).
| People with poor literacy are probably more likely to have
| lower-paying positions, so all of those demands are even
| more severe.
|
| Not that it's impossible. Lots of people do manage to self-
| educate their way out of poorer circumstances, and
| certainly the internet has made that far more accessible
| than it was before the turn of the century. But, let's not
| underestimate how challenging it is, either.
| pixl97 wrote:
| >The lack of "continuing education" in the era of the
| internet is baffling to me.
|
| It's all about incentives. That is companies are
| incentivised to give continuing entertainment for ad
| clicks, rather than building a world of the educated that
| may have a better all outcome for society (but probably not
| the ad companies at all).
| CobrastanJorji wrote:
| I have to wonder whether Grammarly's "Enterprise" tier and its
| underspecified "advanced security features" involve installing
| it on-site and offering am "all of your company's words don't
| get sent across the Internet to another company" feature.
| hartator wrote:
| Maybe the name was odd `loader-spinner.gif`
| FrankWilhoit wrote:
| Error code 5, on Windows, means that the code tried to
| dereference a null pointer. I'm guessing that the GIF content is
| corrupt, containing some 0x00 bytes where they shouldn't be. Then
| the question becomes, whose responsibility is it to program
| defensively against things like that? If, as may well be, Chrome
| is using some third-party library nested several layers deep to
| render GIFs, then would there be any action that the chrome devs
| could take, aside from replacing that library with a better one
| and adding a malformed-GIF test case? (Why don't they already
| have a malformed-GIF test case...?)
| londons_explore wrote:
| Lets be honest, it was probably some grammarly .dll that had
| been injected...
| Night_Thastus wrote:
| I've actually seen something like this in the wild myself. For
| awhile there were some GIFs that if placed in Discord, would
| cause it to crash for everyone who was looking at the chat.
|
| Admins had a fun day when that was found!
| sethammons wrote:
| > The code for our main navigation bar has a fair amount of
| metaprogramming, and chasing down threads here was often more
| confusing than not.
|
| One more point for Don't Be Clever. As Brian Kernighan put it:
| "Debugging is twice as hard as writing the code in the first
| place. Therefore, if you write the code as cleverly as possible,
| you are, by definition, not smart enough to debug it."
| ja5087 wrote:
| We used to develop software that used the Windows Accessibility
| APIs (UI Automation). On certain versions of Excel with some
| files it would crash the client application with a null pointer
| exception once you try to read the window name/class. It would be
| interesting to see the cause of the crash e.g. a core dump/user-
| mode dump/event viewer log.
| bluesmoon wrote:
| Reminds me of the time back in 2010 when a piece of CSS on the
| Yahoo Search page would cause a complete desktop crash on Red Hat
| Linux: https://tech.bluesmoon.info/2010/04/can-website-crash-
| your-r...
|
| To the author, did you ever consider contacting the Chrome dev
| team about this? They're pretty responsive to bug reports.
| rvanlaar wrote:
| Ah, Chrome and slow spinners.
|
| Python tests were taking ages on VSCode due to an SVG spinner:
|
| https://bugs.chromium.org/p/chromium/issues/detail?id=103626...
|
| https://github.com/microsoft/vscode-python/issues/9216
| boringuser2 wrote:
| The blink debug logs would probably be pretty useful for the
| engineers involved...
| rootsudo wrote:
| For security reasons, your organization disables Chrome crash
| reports but allows the use of Grammarly, an app that essentially
| functions as a keylogger. Consented or not, it's a keylogger.
|
| https://support.grammarly.com/hc/en-us/articles/360003816032...
| crazygringo wrote:
| tl;dr: a certain GIF would crash a Chrome tab, but only when the
| desktop version of Grammarly was installed. (Not a Chrome
| extension.)
|
| That's insane!
|
| Can anyone think by what _possible_ mechanism the installation of
| Grammarly could affect whether a .gif file would crash Chrome?
|
| The company seems to be on Macs since they report that the
| problem doesn't surface in Safari.
|
| Is there some kind of dynamically-linked GIF decoding library
| used by macOS that Chrome relies upon, and Grammarly somehow
| installs one that takes precedence for all applications? I didn't
| think this would be possible -- I thought image decoding was done
| natively in the browser and not outsourced to the OS, for
| security reasons.
| hcrean wrote:
| I wonder if they checked for exploit code in the image the they
| likely originally found somewhere on Google.
| toddmorey wrote:
| Once my college professor was working on her research paper and
| told me she was struggling get text to stay underlined. Assuming
| a simple user error, I expected to help her out in 5 minutes.
|
| Over three hours later, we discovered that the combo of her
| specific video card driver version along with her specific
| printer driver version would keep text from printing out
| underlined.
| generationP wrote:
| Huh? How does a video card affect printing?
| jfoutz wrote:
| A lot of rendering will go through the video card if
| available, like the jvm does this as an optimization.
| toddmorey wrote:
| Ah interesting. That makes sense. I had no idea.
| shever73 wrote:
| It often did, particularly on older versions of Windows. I
| helped uncover a bug in Epson printer drivers ~20 years ago
| that was caused by a specific graphics card.
| shermantanktop wrote:
| 20+ years ago I was in tech support and had to help someone
| figure out an issue where her document wouldn't print on a
| Brother printer. Turns out a section divider line would
| block the entire doc from printing (by crashing the app) if
| the line's end-cap style was set to square rather than
| rounded.
| to11mtm wrote:
| One of the ways to print things (especially on windows) is
| Via GDI. [0]
|
| Basically using the OS's rendering to make a raster that is
| then sent to the printer. The main thing the printer's driver
| does in this case is know how to take the bitmap and tell the
| printer to print the bitmap (i.e. chunking data and/or
| sliding the right commands into the bitmap stream)
|
| Contrast to, say, PostScript which allow for more compact and
| better scaling definition of what to print. This obviously
| works better for quality, however for a long time the issue
| was you then had to have sufficient processing power on the
| printer itself to handle it.
|
| [0] - Search for 'GDI Printer' for a little more info.
| issung wrote:
| How does one discover something that niche in ~3 hours?
| toddmorey wrote:
| Lots of internet searching and even a few calls to HP
| support. To be honest, we dismissed some of the earlier
| suggestions to upgrade the other drivers from other
| vendors... so maybe most of the time was us getting past
| ourselves and our disbelief.
| nialv7 wrote:
| I feel the author might have missed out on a multi-thousand
| dollar bug bounty.
| aaaronic wrote:
| This is _so_ familiar!
|
| I have seen accessibility tools in Chrome lead to this kind of
| issue in the past with a dropdown menu -- to the point where it
| could be replicated with a miniscule amount of HTML. The
| particular bug I hit 2 years ago was in Chromium-Edge, but the
| symptoms and cause were very similar.
|
| Grammarly almost certainly leans on some of the accessibility
| tools in Chrome. These tools are somewhat different in the
| various Chromium flavors (Edge, Brave, Chrome, etc.).
| nonethewiser wrote:
| So the theory would be that grammarly desktop sees the gif
| (what? How?) and calls some browser accessibility function on
| it which chrome cant handle and it crashes?
| saagarjha wrote:
| Perhaps it has an extension that it installs that does this?
| gelatocar wrote:
| As I was reading this I was thinking to myself "I wonder if it is
| grammarly related" because I experienced a bug some time ago that
| presented itself in a similar way. It was impossible to reproduce
| but affecting lots of people internally within certain
| departments. Eventually we figured out the thing they had in
| common was that they had the Grammarly extension installed.
|
| The other key thing was that the bug only appeared on our staging
| preview urls, not on the live website. It turned out it was
| because of a bad regex in the grammarly extension that caused the
| page to hang if the domain name was more than about 100
| characters. Our staging domains were pretty long, I think they
| contained a few subdomains and had a job number or something in
| there.
|
| This one is more crazy though if it is really caused by the
| desktop app - that's pretty scary!
| aquafox wrote:
| I randomly had the issue that after booting up Linux, I didn't
| have any sound. Turns out it was related to my Windows dual-boot
| setup!
|
| When restarting from Windows, Windows doesn't shut down my
| realtek audio device, but only puts it to sleep and Linux fails
| to start it. Only solution is to always do a shutdown from
| Windows and then hitting the power button. The issue is still
| there: https://askubuntu.com/questions/1032543/no-sound-in-
| ubuntu-1...
| sfink wrote:
| Next time this happens, I recommend just letting people use a
| different browser. Firefox in particular has gotten much better
| at importing bookmarks, passwords, etc. from Chrome.
|
| It was a Sign from the universe that it was time to make the
| switch. Who are we to reject Signs?
|
| (Full disclosure: I'm an engineer on Firefox. But that has
| nothing whatsoever to do with my advice here, no siree Bob, not
| in the least.)
| OsrsNeedsf2P wrote:
| As an engineer, yes Firefox is a good solution
|
| As a PM, we spent 4 months making the onboard easier, and now
| you want people to install a new browser?
| denton-scratch wrote:
| Awww. I was really enjoying that; I like detective stories and
| rabbit-holes.
|
| Then we get to the punchline: "Uh, we fixed the bug, but sorry
| folks; we didn't solve the puzzle". So I guess we'll never know
| why that particular anigif crashed Crome but only Chrome, and
| only if Grammarly was installed (or had been installed during the
| same session).
|
| I hope Amy Lai lets us know if the story ever gets an ending!
___________________________________________________________________
(page generated 2023-11-30 23:00 UTC)