[HN Gopher] The undocumented Android change that led to aCropaly...
___________________________________________________________________
The undocumented Android change that led to aCropalypse was
reported during beta
Author : luu
Score : 315 points
Date : 2023-03-30 20:01 UTC (2 days ago)
(HTM) web link (iliana.fyi)
(TXT) w3m dump (iliana.fyi)
| TazeTSchnitzel wrote:
| I wonder if some developer of an app working with ZIP files will
| have noticed this bug and had to update their app. ZIP is an
| unusually terrible file format because it has a footer rather
| than a header, so the truncation bug would lead to a ZIP file
| with two headers!
| layer8 wrote:
| Same for PDF. But at least that's much less likely to create
| the issue of invisible non-deleted data as in the PNG case.
| canucker2016 wrote:
| Some ZIP-file handling apps go straight to the end of the ZIP
| file and work backwards looking for the central directory.
|
| I believe others will start from the beginning of the file and
| parse each local directory header and 1) build up their own
| central directory truth, or 2) stop once they've reached the
| start of the central directory.
|
| I recall that the footer-only format was to enable streaming
| out a ZIP file esp. when rewinding to the front of the
| file/stream was not possible, like sending over a network - at
| least that's one of the use cases I recall.
| ladberg wrote:
| I believe the footer-only format was more about being able to
| make modifications by only appending to the zip file and not
| having to rewrite/shift any data.
| canucker2016 wrote:
| ...the central directory needs to be shifted by the amount
| of the new compressed data + local directory header.
|
| But that's a lot less than if the central directory was at
| the beginning and the code had to grow the central
| directory and shift ALL the compressed data + local
| directory headers down.
| ladberg wrote:
| Yeah that might be the ideal way to do it, but
| technically you could just append the new entries and new
| central directory without even removing the old one.
|
| I'm not entirely sure what storage system that would make
| sense on, but presumably there was one back in the day.
| canucker2016 wrote:
| 3.5" floppies - 800K-1.44MB, small and slow.
|
| Better than cassette tapes though...
| cesarb wrote:
| > 3.5" floppies - 800K-1.44MB, small and slow.
|
| I feel like a lot of people in this thread are missing a
| crucial part of the historical context:
|
| ZIP files can be split. That is, a single ZIP file can
| span more than one floppy disk. The entry for each
| compressed file in the central directory, which is always
| on the last floppy disk, has a field which says in which
| disk the compressed file can be found. If you had a
| multi-disk ZIP file, and wanted to extract a single file
| from it, you inserted the last disk, pointed the
| unpacking software at it, and it would tell you which
| disk(s) to insert (in sequence) to extract that file.
|
| A lot of the strange decisions of the ZIP file format
| make sense if you think about that use case: you have a
| directory full of files, which you want to compress into
| a set of floppy disks; and once you have that set of
| floppy disks, you want to extract a subset of the
| compressed files. You also want to be able to extract the
| undamaged files even when the last floppy disk (with the
| central directory) is damaged.
| amethyst wrote:
| Unrelated: the nostalagia of iliana's favicon is incredible. So
| much time spent on old modems watching that animation spin while
| waiting for pages to slowly load in...
| hu3 wrote:
| It's an animated gif!
|
| https://iliana.fyi/netscape.gif
| neilv wrote:
| That one was great, and stylish.
|
| The original NCSA Mosaic one with the globe could've been a bit
| much, but it was appropriate, given the impact of the Web, even
| for pre-Web Internet natives: "OMG, this is a tingly step
| forward in distributed global hypertext, it's happening right
| now -- interacting around the world, and also superpowering the
| world." With implied altruistic and benevolent goodness.
|
| (The earlier Mcom/Netscape throbbing "N", when I guess they
| were still rebranding Navigator from Mosaic, looked silly even
| at the time. Then they did a good one. And there were a bunch
| of creative alternatives, including some easter egg ones, which
| JWZ cataloged at one time.)
| jasonhansel wrote:
| Why would cropping alter the file in-place? That would presumably
| leave the image corrupted if (say) your phone died during the
| cropping process; it would seem much better to save the cropped
| image separately and then overwrite the original file atomically.
| After all, it's the same amount of data to write either way.
| neilv wrote:
| Kudos to the engineer who originally not only identified the
| problem, but also raised the concern about breaking behavior, and
| about documenting it.
|
| Looks like the ball was first dropped when the issue was deferred
| without acknowledging the reporter's concise suggestions of how
| to handle it, engineering-wise. Nor a sign that existing code was
| reviewed for the same problem.
|
| Then looks like the ball was dropped again, in what I'm guessing
| might've been: "old version, forget all the old open issues, all
| the issues that still apply, someone will rediscover the hard
| way".
|
| This "aCropalypse" event is an example of how making the wrong
| triage of an issue report can turn out very expensive. All the
| costs to the world of aCropalypse could've been averted. Some of
| those costs might eventually come back to the company.
|
| It's easy to guess why the problem happened and the report wasn't
| handled responsibly. By the time the problem exhibited in a way
| that couldn't be ignored, the pertinent metrics/KPIs/OKRs about
| clearing issues, resource allocations, and product shipment
| schedules were already in the past, bonuses had been paid,
| promotions for shipping new things earned, etc. And security
| problems are treated as inevitable, even if there's a constant
| stream of them, and they're produced faster than they're fixed.
|
| We're going to need software engineers to be accountable for
| things we've done and signed off on.
|
| (Bonus if accountability changes happen near-term: GPT-powered
| open source laundering gets a pause, because bridges start
| falling on everyone foolish enough to sign off on that mangled
| statistical plagiarism.)
| raverbashing wrote:
| And to be even more honest, why create a similar API with
| different behavior?
|
| How many use cases prefer _not_ truncating files?
| masklinn wrote:
| The worst part is that the API did originally behave like the
| one it'd originally emulated, before being modified after
| years of use.
| uf00lme wrote:
| > All the costs to the world of aCropalypse could've been
| averted. Some of those costs might eventually come back to the
| company.
|
| This is the bigger problem, there will be very little real cost
| to the company or engineers involved. My guess is that the cost
| for engineering a fix and blogging it about it falls into the
| buffer PR spend or security engineering time. If anything the
| company will learn from this mistakes for a short time and it
| will actually save them money in the long run.
|
| The vast majority of software used today has very little legal
| liability. The industry has perfected the art of legal lip
| service. There is a massive growth at all costs mentality
| throughout the industry and engineer's personal pride only goes
| so far.
|
| If you want to see real change then you need government and the
| judicial to enforce real legal liability onto software
| engineers and companies. That would result in real cyber
| security rather than current wack-a-mole fines or government's
| trying to promote or enforce best practices etc.
| to11mtm wrote:
| > We're going to need software engineers to be accountable for
| things we've done and signed off on.
|
| Only if management is held equally accountable.
|
| I've worked at places with dedicated security teams, that acted
| as a 'buffer' between management and engineers. Those places
| tended to take security seriously.
|
| Places -without- those, I noticed that management has a
| tendency to sweep security issues 'under the rug' since they
| impact deadlines.
| nuc1e0n wrote:
| Management earns the big bucks because they are supposed to
| be accountable. If the expectation is that software engineers
| should be accountable and not management the money should go
| to the engineers instead.
| izacus wrote:
| When was the last time you've seen any manager being
| accountable for any defect in software?
| dingusdew wrote:
| [dead]
| FpUser wrote:
| >"We're going to need software engineers to be accountable
| for things we've done and signed off on."
|
| And see them evaporate from the companies that would
| implement something like this.
|
| Personally I would neve take a contract where I could be held
| legally responsible for bugs.
|
| And ultimately it is the owner who let the product out in a
| wild without spending a year testing billions of permutations
| first. Try holding them responsible and see how far can you
| get.
| Eisenstein wrote:
| 'Hard' engineers do this, though. If you want the label you
| should take the responsibility as well.
| FpUser wrote:
| I do not give a shit about a label and have none in a
| software field. Just MSc in physics. I design and
| implement products and I have long list of clients happy
| with my work. This helps me find new contracts. You can
| call me a shrub if you wish. I still get paid.
| anthk wrote:
| In Europe calling yourself an enginner without having a
| proper title can send you to jail, because an actual
| Engineer has civil duties and accountability on civil
| causalities.
| throw_m239339 wrote:
| [flagged]
| cookiengineer wrote:
| > (Hardware) engineers do this, though.
|
| Broadcom exploits over the years would disagree. So would
| Samsung's hardware encryption. So would TPM. So would
| BadUSB. So would everything related to Bluetooth. Et
| cetera.
|
| No matter what kind of engineering you do, don't try to
| predict the future. Engineer towards ease of maintenance
| and ease of update/deployment instead.
| theK wrote:
| Never heard that term. Who calls themselves a "hard"
| engineer?
| washadjeffmad wrote:
| No one, but there are PEs.
|
| The frivolity with which "Engineer" titles were co-opted
| by tech has always been distasteful to me. It felt
| intended to claim all of the prestige with none of the
| rigor, accountability, or social responsibility.
| salawat wrote:
| I want a software PE program so bad. I've done my best to
| hold myself to the same ethical standards, but if there
| isn't a legal bite behind negligence, you just get shoved
| aside.
| narag wrote:
| It seems it's a play with soft-ware hard-ware, implying
| electric and civil engineering.
| eviks wrote:
| What billions of permutations do you need to test when the
| test had already been done by someone else and brought to
| your issue tracker on a silver platter?
| wirrbel wrote:
| Company needs to be liable and introduce necessary steps so
| that it's staff operates responsibly.
|
| How quick is a manager at deprioritizing quality to secure a
| deadline?
| numpad0 wrote:
| IMO this is another anecdotal proof that so called software
| "engineering" isn't, and how uncontrolled over-complexity of
| modern software is going further out of control.
|
| Software is written, like blogs, novels or laws, not
| engineered. That's wrong.
| robviren wrote:
| I spent 5 years in nuclear power before going into tech and I
| have sort of joked that the software engineering the goes into
| AI, IoT, Phones, Cars, etc, is the next Chernobyl waiting to
| happen. It's all fun and games till these things get connected
| directly to lives. I actually find the governance of nuclear
| power a decent model for managing the risk. You mentioned
| management needs to be held responsible but management will
| choose short term profit every day until dealing with negative
| externalities is forced upon them like it is in nuclear.
|
| Imagine if the toxic waste that is the mental impact of social
| media, or forever insecure devices littering the Internet, or
| any number of things was obligated to the companies to do their
| best to fix it. Nuclear reactors put huge money away so that
| when they are done they are bulldozed back into a green field
| (Other than what was supposed to be a solved problem of spent
| fuel).
|
| Anyways, the likelihood of regulation is nil and a global world
| means those negative externalities will continue on churning
| till the world agrees we suck at managing risk. Which is also
| nil. Mental health crisis and insecure tech just isn't as
| flashy as nuclear in terms of getting regulated in the interest
| of public health.
| raverbashing wrote:
| I second this
|
| Risk management (especially with the js people) seems to be
| "lol what's that?"
|
| Also we have the C cowboys pretending they can manage the
| complexity of their "simple language" in a complex world
| throw_m239339 wrote:
| > We're going to need software engineers to be accountable for
| things we've done and signed off on.
|
| You start with management and all the way to the top including
| the CEO and CTO liable for criminal charges.
|
| And, no we're not going to need any of that.
| KennyBlanken wrote:
| This wasn't a matter of the ball being dropped, this is a
| matter of the epidemic of arrogance in Google.
|
| It's going to keep biting them in the ass. The prick who heads
| up Golang's security has refused to allow people to control
| what encryption algorithms are enabled to disabled in Golang's
| TLS 1.3 implementation (he was one of the designers of the TLS
| 1.3 spec) because, and I wish I were joking - he feels TLS
| 1.3's implementation is perfect and nobody would have a reason
| to, and even if they do, why, they can just wait until google
| releases the patch to fix it, and then deploy that change.
|
| Because of course that's how production environments
| work....you can just deploy a new version of the language
| you're using, instead of, you know, a config file change or a
| few lines of code.
| potatofrenzy wrote:
| > This "aCropalypse" event is an example of how making the
| wrong triage of an issue report can turn out very expensive.
| All the costs to the world of aCropalypse could've been
| averted.
|
| Not trying to be facetious, but... what costs? It is unlikely
| that this cost the company any substantial amount. It probably
| wasn't even a whole lot of extra work for the PR department, as
| the bug didn't generate mainstream headlines.
| neilv wrote:
| The unquoted sentence after that was "Some of those costs
| might eventually come back to the company." Which I
| distinguish from "costs to the world" in the quoted sentence.
|
| By _costs to the world_ I 'm thinking all the things that
| happen because something a user cropped out was revealed
| because it wasn't really cropped.
|
| Hypotheticals to illustrate costs to the world:
|
| * A company loses a big business deal because IP/plans leaked
| to competition in reverted-crop.
|
| * A medical provider and personnel get hit by lawsuit and/or
| HIPPA for patient data de-anonymized due to reverted-crop of
| an image.
|
| * Someone gets harassed at their workplace when a reverted-
| crop adult photo revealed their identity and gets circulated
| among colleagues.
|
| * Semi-automated extortion rackets, going through troves of
| images, looking for revertible cropping.
|
| * Other businesses caught in the middle of this have to
| expend resources to mitigate, or even deal with liability for
| depending on or assuming correctness of third-party tech
| behavior, etc.
|
| Maybe luck is with us, and not a single instance of harm to
| the world actually happens. Or maybe there are many such
| instances of harm to the world.
|
| Regarding costs to the world _eventually coming back to a
| company_ -- speaking in general, not of this particular
| situation -- I think that could come in forms including:
| brand damage, lost business, lawsuits from those harmed,
| undesirable turns in legislation, regulatory fines, etc.
| tgv wrote:
| > the bug didn't generate mainstream headlines
|
| So ... yesterday it turned out that Microsoft had the control
| over a certain Bing's search results wide open to anyone with
| an Azure account, and that enabled anyone to get access to
| and control over the Azure services of everyone that used
| Bing for these specific searches. There were probably other
| ways too, but that hasn't been disclosed or investigated. And
| it is --of course!-- unknown if this attack ever happened in
| real life: there were no logs.
|
| That's quite the problem, wouldn't you say? Well, did you see
| mainstream articles? There was only an article in The Reg,
| which attracted a mere 11 replies. It simply doesn't get
| attention, probably because it was framed as "a
| misconfiguration", yet the costs could be staggering.
| wilg wrote:
| What kind of "accountability" do you mean?
| Jtsummers wrote:
| Presumably similar to the accountability that applies to PEs.
| A subset of engineers are granted authority to sign off on
| parts of the system, they're assuming a legal responsibility
| and the penalties for malpractice can go all the way to
| imprisonment.
|
| Non-licensed engineer are not liable in the same sense, their
| employers and PEs are. Same thing could apply in software,
| but people whine too much and it probably won't happen. Every
| time the topic comes up, people chime in with "But software
| is _different_. It 's _hard_. " and expect that lame ass
| excuse to be sufficiently persuasive. Software is also
| critical, not all, but a lot. Critical systems warrant
| additional scrutiny and liability.
| wilg wrote:
| What aspect of this "acropalypse" bug amounts to
| malpractice?
| Jtsummers wrote:
| An undocumented change to a core system behavior that
| fundamentally altered how (likely) most applications
| behaved in an unanticipated (for the users of the system
| and programmers targeting the system) way. That's
| malpractice.
| wilg wrote:
| Yeah, I don't buy that. Too much Monday-morning
| quarterbacking.
|
| I'm not against some well-thought-through standards for
| certain safety-critical applications, but it's got to be
| a lot better than "maybe code would be better if we
| threatened to throw engineers in jail for contributing to
| complex and subtle interdependent bugs that caused a
| privacy bug that didn't have a significant real-world
| impact".
| HeavyFeather wrote:
| > the issue was deferred without acknowledging the reporter's
| concise suggestions of how to handle it, engineering-wise
|
| A day in my life at my last job. Basically a regular occurrence
| that it was "urgent" after a customer report.
| yjftsjthsd-h wrote:
| > We're going to need software engineers to be accountable for
| things we've done and signed off on.
|
| Management first. It's not engineers setting incentives.
| lupire wrote:
| At a properly licensed engineering firm, the managers _are_
| senior engineers who review and sign off.
| [deleted]
| WirelessGigabit wrote:
| Keyword being properly. Too many companies push their
| engineers into people management.
|
| Which is weird. Your best people should remain engineers.
| Have them indeed sign off on these things.
| tgv wrote:
| Indeed. I haven't worked at a company that was managed
| (partially) by engineers. Sure, there were tech leads,
| but as soon as it came to product and account management
| and "up", it was business school. Must be some corollary
| of Peter's Principle.
| chii wrote:
| > Your best people should remain engineers. Have them
| indeed sign off on these things.
|
| I agree; the problem with an engineer track is that the
| salary ceiling is too low, and you have to move to
| management to get higher salary. but then you end up not
| using your engineering skills, and waste potential.
|
| What needs to happen is a way for people management to be
| done without a hierarchical structure. Aka, the senior
| engineer sits not under their manager, but that the
| manager is subservient.
|
| Joel spolsky have written about this "problem" already in
| the last couple of decade:
| https://www.joelonsoftware.com/2009/03/09/how-to-be-a-
| progra...
|
| > To make sure that the debate happens respectfully and
| on a rational basis of facts, it's absolutely critical
| that the program managers and developers be peers. If
| developers report to the program manager, at some point
| during the debate the program manager is going to get
| sick of the whole thing and just say, "OK, enough
| talking, now we do it my way."
| ncphil wrote:
| The salary range in a professional track does not _have_
| to be too low. That's a choice, a value judgment, by
| management. Some companies do things differently. In at
| least one company I know, the professional track runs
| parallel to the management track: engineers can earn as
| much or more than senior managers, architects as much as
| directors. It's all up to whether the organization really
| values the services it's independent contributors
| provide. Sadly, many companies are not only indifferent,
| but hostile to the whole concept: playing at cost
| reduction theater and outsourcing to satisfy the avarice
| of simpleton investors. Bottom line is that tech pros
| need to take charge of their own futures and avoid the
| latter kinds of companies like the plague, when they have
| a choice. People need to make a habit of asking questions
| about professional compensation structures during
| interviews.
| theK wrote:
| I don't think pushing a good engineer to also take on
| management work is such a bad idea. I've actually had
| quite a lot of success with that system.
|
| The much worse idea is to promote BAD engineers to full
| time management positions, which seems to be the way most
| sub par and mediocre companies do it right now.
| ryandrake wrote:
| This is just passing the buck. If the boss tells you to write
| credit card stealing software or to DDOS your competitor are
| you just going to do it? As professionals, we need to push
| back and refuse when we know our practices are letting
| critical bugs go to production, just like (I assume) we'd
| push back if we were asked to build something unethical. And
| we should be accountable if we don't.
|
| If a civil engineer signs off on a bridge and it collapses
| during rush hour, he doesn't get to just say, "Whelp, boss
| rushed me, blame him."
| to11mtm wrote:
| > If a civil engineer signs off on a bridge and it
| collapses during rush hour, he doesn't get to just say,
| "Whelp, boss rushed me, blame him."
|
| I'm willing to bet that the civil engineer in that case has
| much better whistle-blowing protections, or at the very
| least, some comfort that if they get fired and someone else
| replaces them that's willing to do it... _that_ person will
| be the unhireable one in the future.
| 59nadir wrote:
| > This is just passing the buck. If the boss tells you to
| write credit card stealing software or to DDOS your
| competitor are you just going to do it?
|
| This is already illegal. Holding the makers of things more
| accountable would do nothing for you in this example.
| BobbyJo wrote:
| Your example is so decoupled from reality that it's
| misleading. Most catastrophic problems in large systems
| aren't "I made a thing and it's buggy," or "I made a thing
| that steals SSNs." Most of them are some variation of "It
| used to work this way, but this upstream dependency changed
| in a way that we are ok with, and now it works a different
| way, and we have no visibility into what that may change
| for downstream dependencies, so hopefully they are aware of
| the same things we are and act accordingly." Bugs can come
| from many different people signing off on changes that are
| often, in isolation, fine.
|
| Imagine if all buildings had to constantly be retrofitted
| to support a new structures on, below, inside of, around,
| and next to them, and every retrofit had a different
| engineer/architect/etc. No civil engineer would ever sign
| off on anything.
| ok_dad wrote:
| > had to constantly be retrofitted
|
| There's your problem. You can't constantly change
| software contracts without causing bugs. If you're liable
| for the code chances are things will slow a bit. Good! We
| need to develop software slower.
| BobbyJo wrote:
| > You can't constantly change software contracts without
| causing bugs
|
| You can't write software without causing bugs, static
| requirements or not. The software that goes into
| air/space-crafts is written this way, and it still breaks
| occasionally. Sometimes bugs are circumvented with
| hardware because it's cheaper and more reliable than
| making the software change.
|
| Buildings have fault tolerances. You can use the wrong
| amount of something, or put something in the wrong place,
| and have it still work. If buildings had to be designed
| and built exactly corrects or they fell apart, we
| wouldn't have buildings.
|
| > We need to develop software slower
|
| To each his own. Most software is fine having bugs. I'd
| rather have a modern web browser with a few bugs than
| have an ascii-only command line browser that works
| flawlessly. The U.S. economy would agree with me there.
| The software in your pacemaker or car, sure, balance
| safety and speed to save the most lives/time, but most
| software operates in domains with far lower stakes.
| [deleted]
| ryandrake wrote:
| Yours is a good example! I have in a past life actually
| used the "you wouldn't bolt a new floor onto a building"
| argument successfully to push against feature creep.
|
| I think even a small amount of accountability would
| definitely start some much-needed conversations at
| software companies about things like "do we really need
| to retrofit this thing onto that working thing?" and "do
| we fully understand our dependencies and the consequence
| of merging from them?" and software quality would be
| better for it. If the consequence of a bug is low, go for
| it--the accountability should be low, too.
|
| EDIT, just to add another example. I remember when I felt
| rushed and checked in some client code that did not
| implement exponential backoff for request failures--it
| just retried. Of course that code made it to production
| and our customers started DDOSing our server when a brief
| outage happened. I felt terrible about it and honestly
| believe I should have been fired for it. Yes, management
| rushed me, but they're not the ones that wrote the code.
| They're not the ones that made an amateur-hour error. It
| was me.
| Izkata wrote:
| > Yours is a good example! I have in a past life actually
| used the "you wouldn't bolt a new floor onto a building"
| argument successfully to push against feature creep.
|
| I've seen a few houses built on top of apartment
| buildings though.
| mangamadaiyan wrote:
| > I felt terrible about it and honestly believe I should
| have been fired for it. > Yes, management rushed me, but
| they're not the ones that wrote the code
|
| If you felt so strongly about it, you could have quit.
| Why wait on management to fire you?
|
| It's fairly easy to blame engineers for everything that
| goes wrong. Engineering doesn't work in a vacuum, the
| system and processes have to have guardrails in place
| that prevent lapses in quality, including security, from
| reaching the customer. When such guardrails have been
| systematically dismantled in favour of "improving
| productivity", the management - which removed the
| guardrails - becomes the culpable entity.
| scns wrote:
| Interesting. Great that you want to take full
| responsibility for your work. There is always two sides
| to a coin though. Strange to me that you scapegoat
| yourself that hard. Reminds me of the story, where
| someones error cost the compny millions. He gets called
| in his superiors office. He says something about
| resignation. His boss responds: "What? We just invested
| millions in your education". Every error we still have to
| make, is a treasure waiting to be dug up IMHO.
| tialaramex wrote:
| > If a civil engineer signs off on a bridge and it
| collapses during rush hour, he doesn't get to just say,
| "Whelp, boss rushed me, blame him."
|
| He doesn't but this is actually a problem. Ideally, he
| doesn't get to say this because he chose to rush and if
| he'd said "No" instead then the boss can't do anything
| about it. In reality, especially in the US, his boss will
| just fire him and hire somebody else more co-operative and
| he knows it.
|
| Without a strong, legally supported and well resourced
| professional organisation behind people with an obligation
| like this, they will get exploited and then blamed by
| people who are insulated from the effects.
|
| Do you see such an organisation for software engineers in
| the United States of America? Or even most of Europe?
| Because I do not.
| robertlagrant wrote:
| > Do you see such an organisation for software engineers
| in the United States of America? Or even most of Europe?
| Because I do not.
|
| The problem is that medicine and law move orders of
| magnitude more slowly than software, at the moment.
| Software is evolving unbelieveably faster, and its
| problems are far more diverse. As it settles down, if it
| ever does, we may be able to calcify roles more formally,
| and then regulate those roles. But at the moment it's the
| wild west.
| ryandrake wrote:
| I fully agree a strong, organized professional
| organization for software developers would solve a lot of
| problems the craft faces. If we had something like the
| Bar Association or the Medical Board, would we have to
| keep implementing FizzBuzz and reversing linked lists
| every interview just to prove a basic level of
| competence?
| ElectricalUnion wrote:
| Not, but then the "Software Developer Board" would apply
| those as certification tests instead.
| mandevil wrote:
| But you'd take the exam once rather than having to do it
| every few years when you want to change jobs.
| jimrandomh wrote:
| It looks like this was reported as a normal bug, and the security
| implications weren't noticed until later. This is unfortunate,
| but it's also pretty normal and hard to avoid this sort of thing
| happnening.
| layer8 wrote:
| It was declared to be an intentional change and not a bug. If
| anyone would have thought through how an application expecting
| the old semantics will behave under the new semantics, it
| should have been rather obvious that silently changing the
| semantics is a very bad idea. It's absolutely a case of
| professional negligence.
| cvccvroomvroom wrote:
| Developers at big companies aren't motivated to do everything
| perfectly. They're graded for big wins, not tech debt,
| documentation, or correctness.
| sitkack wrote:
| By Design
|
| Won't Fix Working As Intended
|
| Declaring Bug Bankruptcy
|
| Google doesn't test. Google relies on the public to test its
| software and even then most bugs don't get addressed. If the bug
| is a gap in the spec or open to interpretation it won't get
| fixed. Spec bugs are not allowed.
|
| The reason that Google has such a high hiring bar is that most of
| its processes run open-loop and it relies on having smart
| thorough people to prevent the large swath of issues from being
| created in the first place.
| lapcat wrote:
| > The reason that Google has such a high hiring bar is that
| most of its processes run open-loop and it relies on having
| smart thorough people to prevent the large swath of issues from
| being created in the first place.
|
| But there is a large swath of issues. Just look at Google's
| public bug trackers.
|
| And Google's atrocious practice of auto-closing old bug reports
| partially covers up how many issues there really are.
| codethief wrote:
| This reminds me of how a recent Android version (12?) suddenly
| made certain characters (e.g. ?:<>) illegal in file names[0],
| probably because to increase compatibility with NTFS. Wtf,
| Google. Nowhere was this breaking change announced.
|
| Not only does my notes app, which has worked flawlessly for
| years, now crash when I accidentally use one of those characters
| in a file name / note title. (Unfortunately, the app no longer
| receives updates.) But suddenly my entire ebook library (which
| follows the name format "first author & second author: title
| (year)") no longer synchronizes with my phone (running AOSP /
| GrapheneOS), either. Fantastic.
|
| [0]:
| https://android.googlesource.com/platform/packages/providers...
| mikepavone wrote:
| Oh, I was wondering my new phone didn't like some of the music
| files I was trying to copy over using MTP even though the old
| one handled them fine. I had assumed the phone manufacturer did
| something dumb, but apparently it was Google all along.
| dpkirchner wrote:
| Makes you wonder what issue they were mitigating. Passing file
| names as part of a concatenated string to bash? Ala 'bash -c
| "ls $filename"'?
| kps wrote:
| Windows. /** * Sanitizes given
| name by mutating the file name to make it valid for a FAT
| filesystem. */
|
| https://cs.android.com/android/platform/superproject/+/maste.
| ..
| JoeAltmaier wrote:
| I have sympathy. It's hard to write fundamental APIs. They need
| to grow and change like anything else, always at the risk of
| startling somebody.
|
| Years ago I extended a network file system server to include
| additional information in an Open response (revision tag to
| enable distributed caching). Turns out a customer had also
| extended the server response in their builds, to do something
| else security-related. Our release startled them, resulting in a
| lawsuit.
|
| It was all silly and fixed in a patch (a boot config file
| enable/disable of our new extended info) and it went away. But it
| underlines how with the best of intentions, still things go
| wrong.
| OnlyRepliesToBS wrote:
| [dead]
| justin_oaks wrote:
| Another case of poorly chosen defaults leading to bugs, leading
| to security issues.
|
| Also a case of violating the Principle of Least Astonishment [0].
| How many people would expect that when you write a file you would
| also have specify that you want it truncated?
|
| [0]
| https://en.wikipedia.org/wiki/Principle_of_least_astonishmen...
| [deleted]
| intelVISA wrote:
| Some say certain agencies wanted it this way.
| rozab wrote:
| This would be the stupidest possible way of conducting mass
| surveillance
| jmull wrote:
| Some are yet naive.
|
| "Never attribute to malice that which is adequately explained
| by stupidity."
| bagels wrote:
| Who says this?
| dylan604 wrote:
| They and them. Don't confuse the pronouns as being more
| than one person.
| jandrese wrote:
| It's notable that in practically every other programming
| language on every platform the default is to truncate the file
| to 0 bytes when opened in write (not append) mode. This isn't
| one of those things where there are two schools of thought
| where people take sides. It's a completely baffling decision to
| buck all convention vs. common sense.
| Someone wrote:
| > It's notable that in practically every other programming
| language on every platform the default is to truncate the
| file to 0 bytes when opened in write (not append) mode
|
| Examples of 'Open' calls that require setting a bit in the
| 'flags' argument to truncate an existing file:
|
| - https://learn.microsoft.com/en-
| us/windows/win32/api/winbase/...
|
| -
| https://man.freebsd.org/cgi/man.cgi?query=open&sektion=2&n=1
| moefh wrote:
| Wow, that Win32 OpenFile function is truly cursed:
|
| - If you don't specify a directory, Windows will look for
| the file in a bunch of directories, but the current
| directory is not the first place searched.
|
| - The lpReOpenBuff is annotated as [out], but is an input
| when uStyle contains the OF_REOPEN flag.
|
| - The function is named OpenFile, but can be used to delete
| a file. Even better, the documentation is not clear what to
| do with the return value on success: is it a valid file
| handle that must be passed to CloseHandle, or just a dummy
| non-error file handle?
|
| - You can also use it to check the existence of the file --
| the documentation says it will open and then close the
| file. What's the return value in that case? Is it a stale
| file handle, a dummy, or do you still have to close it?
| Someone wrote:
| You're forgetting this one: you can specify a flag that
| makes the call display a dialog box if a requested file
| does not exist. Yes, this call to open a file can run an
| event loop that lets the user create directories, rename
| files, possibly even format floppy disks, etc.
|
| If you want a prime example of a layer violation, this is
| it.
|
| Its replacement "CreateFile"
| (https://learn.microsoft.com/en-
| us/windows/win32/api/fileapi/...) is a lot saner, but
| still has this:
|
| _"If you rename or delete a file and then restore it
| shortly afterward, the system searches the cache for file
| information to restore. Cached information includes its
| short /long name pair and creation time."_
|
| That's intended to make it easier for "Save" operations
| in document-based applications to keep the short name of
| the file, creation time and file flags around.
| okanat wrote:
| You're comparing system call interfaces to higher level
| standard API defaults. Not really an apples to apples
| comparison.
|
| System calls are special low-level interfaces. The
| requirement for reading their docs carefully and general
| care is expected at that level.
|
| A higher level API shouldn't contain gotchas like this.
| canucker2016 wrote:
| Mentioned in the link.
|
| This seems to be the original PR for the API in question,
| ParcelFileDescriptor#parseMode():
|
| https://android.googlesource.com/platform/frameworks/base/+/...
| + public static int parseMode(String mode) { +
| final int modeBits; + if ("r".equals(mode)) {
| + modeBits = ParcelFileDescriptor.MODE_READ_ONLY;
| + } else if ("w".equals(mode) || "wt".equals(mode)) {
| + modeBits = ParcelFileDescriptor.MODE_WRITE_ONLY
| + | ParcelFileDescriptor.MODE_CREATE +
| | ParcelFileDescriptor.MODE_TRUNCATE; + } else if
| ("wa".equals(mode)) { ...
|
| You can see that "w" and "wt" map to the same mode for file
| operations, including the MODE_TRUNCATE flag.
|
| But we jump ahead a few years and the code is reworked.
|
| from
| https://android.googlesource.com/platform/frameworks/base/+/...:
| public static int parseMode(String mode) { - final
| int modeBits; - if ("r".equals(mode)) { -
| modeBits = ParcelFileDescriptor.MODE_READ_ONLY; -
| } else if ("w".equals(mode) || "wt".equals(mode)) { -
| modeBits = ParcelFileDescriptor.MODE_WRITE_ONLY -
| | ParcelFileDescriptor.MODE_CREATE - |
| ParcelFileDescriptor.MODE_TRUNCATE; - } else if
| ("wa".equals(mode)) { ... - return
| modeBits; + return FileUtils.translateModePosixToP
| fd(FileUtils.translateModeStringToPosix(mode)); }
|
| So what does FileUtils.translateModePosixToPfd(FileUtils.translat
| eModeStringToPosix(mode)) do?
|
| from
| https://android.googlesource.com/platform/frameworks/base/+/...:
| + public static int translateModeStringToPosix(String mode) {
| + int res = 0; + if (mode.startsWith("rw"))
| { + res |= O_RDWR | O_CREAT; +
| } else if (mode.startsWith("w")) { + res |=
| O_WRONLY | O_CREAT; + } else if
| (mode.startsWith("r")) { + res |= O_RDONLY;
| + } else { + throw new
| IllegalArgumentException("Bad mode: " + mode); + }
| + if (mode.indexOf('t') != -1) { + res
| |= O_TRUNC; + } + if
| (mode.indexOf('a') != -1) { + res |= O_APPEND;
| + } + return res; + }
|
| Now "w" and "wt" map to different Posix open mode flags since one
| has to explicitly pass in "t" to get file truncation.
|
| A dev was careless, plain and simple.
|
| Having code in a monorepo doesn't help if you don't check how
| your code changes affects consumers/clients of the code.
|
| If they couldn't change translateModeStringToPosix() behaviour,
| then the dev could've amended parseMode() to map "w' to "wt" to
| keep the old behaviour (ugly and non-orthogonal, yes, but that's
| the way the API was designed).
| cryptonector wrote:
| > Having code in a monorepo doesn't help if you don't check how
| your code changes affects consumers/clients of the code.
|
| It most definitely does help with that because there's fewer
| places to look. But if you _only_ look at your monorepo then
| you 'll miss a lot of things if the API you're touching is
| public. In this case there were consumers in the same monorepo
| were there not?
|
| The issue here is that someone dropped the ball on backwards
| compatibility. That happens, especially when the people
| involved are inexperienced.
| [deleted]
| canucker2016 wrote:
| Looking at the reworked parseMode API some more and it looks
| like a total mess.
|
| Original code would only accept one of these modes: "r", "w",
| "wt", "wa", "rw", "rwt".
|
| Anything else, and you'll get an IllegalArgumentException
| exception.
|
| which matches the docs:
|
| from
| https://developer.android.com/reference/android/os/ParcelFil...
| Parameters mode String: The string representation of
| the file mode. Can be "r", "w", "wt", "wa", "rw" or "rwt".
| Throws IllegalArgumentException if the given string
| does not match a known file mode.
|
| ----
|
| The modified parseMode code will only throw the
| IllegalArgumentException if the mode string does NOT start with
| "rw", "w", or "r".
|
| You can append any other character(s) to the mode string ("a",
| "t", "x", "y", "z", your fav emoji char), and you won't get an
| IllegalArgumentException exception now.
|
| I wonder what happens when open() gets a mode flag with both
| "a" and "t" set?
|
| parseMode API contract thrown out the window...
| masklinn wrote:
| > I wonder what happens when open() gets a mode flag with
| both "a" and "t" set?
|
| Nothing I expect. O_TRUNC and O_APPEND are not incompatible,
| O_APPEND is just redundant (because by definition if you
| truncate the file you're writing at its end).
|
| macOS certainly doesn't seem to care when I do it.
| cesarb wrote:
| > O_APPEND is just redundant (because by definition if you
| truncate the file you're writing at its end).
|
| No, O_APPEND is not redundant with O_TRUNC. When you use
| O_APPEND, you also have the extra behavior that _every_
| write goes to the end of the file, instead of the current
| position; this is important when more than one process is
| writing to the same file.
| makomk wrote:
| The original C/POSIX API this is based on is non-orthogonal
| because the combinations it doesn't support generally don't
| make sense. There's usually no point opening files for reading
| or in append mode if you're truncating the existing contents
| anyway, and not truncating when opening in write-only mode is
| almost always wrong because any existing contents that aren't
| overwritten are almost certainly not relevant to the new
| contents and shouldn't be there (because the app clearly isn't
| payint attention to what was already there).
| btown wrote:
| A cautionary tale here: when you are testing, if you mock out
| even low-level I/O utilities, you're vulnerable to those
| utilities changing specification subtly. You can supply-chain
| attack yourself by simply updating a library.
|
| Full filesystem/service-engaging integration tests are helpful
| but far from exhaustive - here, one wouldn't have just needed to
| read the cropped file using standard display systems and count
| the pixels, but hash the underlying file in its entirety.
|
| For critical libraries in an I/O pipeline, encourage team members
| to read CHANGELOGs - both before choosing a library to make sure
| they're maintained, and closely when even doing a minor upgrade.
| DoctorNick wrote:
| Change logs wouldn't have helped here, this behavior change was
| undocumented.
| btown wrote:
| IMO the Android team should have realized the Android team
| was not creating internal libraries at a _sufficient_ level
| of stability and change tracking _for_ the Android team. This
| quickly gets into cultural priorities that I have very little
| context on - but I do think that processes failed here.
___________________________________________________________________
(page generated 2023-04-01 23:03 UTC)