[HN Gopher] The asymmetry of nudges
___________________________________________________________________
The asymmetry of nudges
Author : MBCook
Score : 83 points
Date : 2024-07-02 17:35 UTC (4 days ago)
(HTM) web link (lcamtuf.substack.com)
(TXT) w3m dump (lcamtuf.substack.com)
| danjc wrote:
| Both motives are likely to be true.
| dasil003 wrote:
| I don't like the blame narrative either way. There's no single
| party with both the power and the knowledge of the details to
| counterbalance the strong incentives that are all around them.
| It's all well and good to say engineers should do their part to
| push back on dark patterns at the front line, or executives
| should think beyond just investor pressure for ever greater
| profits, but those things don't scale.
|
| What we need are external checks and balances. These can come in
| many forms from market competition, to government regulation, to
| watchdog groups. Putting pressure on individuals to change
| massively powerful systems from within is a fools errand.
| spencerflem wrote:
| I think it can be both- I agree it won't scale, and we need
| external groups to have any meaningful success. That's
| absolutely what we, as a society, should push for.
|
| But I have room enough in my heart to also hate the individual
| engineers making boatloads of money actively worsening the
| world around them.
| orf wrote:
| So the issue is poor sandboxing of extensions. Wouldn't something
| like WASM help with this?
|
| As in, a content filter extension (or anything that interacts
| with a content filter) is run in a WASM sandbox without any
| access to the network or underlying system? It's hermetically
| sealed from the rest of the extension, that might well need to
| make external requests to function.
| CJefferson wrote:
| The problem, for an ad blocker for example, is it needs to make
| changes to the page. If it can do that, it can change the page
| so that the page makes any evil requests it wants.
|
| The v3 fixes this by instead only letting bad blocker submit
| things they want blocking, and never letting them see that
| page. It's not perfect by any means, but it is much more
| secure.
| orf wrote:
| Surely the problem isn't making changes to a page, but
| instead having access to the full page content _and being
| able to send it somewhere else_.
|
| For example, iOS runs content filters in a special isolated
| process with no persistence or access to the outside world.
|
| With this model, the content filter requests elements to be
| removed and the browser does the actual removal. As such, the
| scope of modifications can be reduced whilst keeping
| arbitrary and perhaps complex filtering logic.
| tssge wrote:
| How does this differ from how Manifest v3
| declarativeNetRequest is in practice? Not saying they're
| the same, just wondering of the capabilities between these.
| yencabulator wrote:
| The implied attack was the content filter changing the page
| to submit the data it wants snooped. Consider replacing ads
| with cats from a site the attacker conveniently happens to
| host, and leaking the desired data in query arguments.
| orf wrote:
| Yes, this is what I said above: "Surely the problem isn't
| making changes to a page, but instead having access to
| the full page content and being able to send it somewhere
| else."
|
| "Send it somewhere else" being the important part.
|
| The interface Chrome went with was a declarative list of
| filters that _Chrome_ will use to perform the actual
| filtering, but the declarative interface isn't great and
| wasn't well received.
|
| My point is: there is an interface that is non-
| declarative and sandboxed, whilst allowing _Chrome_ to
| perform the actual filtering.
| rcxdude wrote:
| Yeah, but you can't do both, that's the point! If you can
| directly change the website content, the website itself
| runs in a sufficiently non-sandboxed environment to allow
| the data to be sent somewhere else. You can only get
| around this by making a more limited blocking process
| (which may not need to be as limited as chrome has gone
| with, but still, not being able to e.g. substitute in
| shims for ad networks will reduce the effectiveness of
| your adblocker. It certainly makes anti-ad-blocking have
| the significant advantage).
| ec109685 wrote:
| It can inject scripts into the page it's manipulating to do
| whatever it wants.
| TaylorAlexander wrote:
| As a person who views our current method of organizing firms in
| our economy as deeply flawed, the conclusion of this article is
| incredible to me. The author goes out of their way to describe a
| structural problem as being regularly blamed on the wrong cause -
| executives - and then proceeds to blame it on a different wrong
| cause - engineers. I appreciate the point of the article as
| written, which I think is to encourage engineers to push harder
| against the grain when their plans for the product really are the
| right idea, but to me the very obvious conclusion is that the
| structure of the business is wrong.
|
| That is an unsatisfying conclusion as the general structure of
| Google is unlikely to ever change, but it does seem correct to
| me.
|
| The real structural problem is that the needs of the shareholders
| and by some extension the needs of the high level executives and
| managers at Google are simply not aligned with the needs of the
| users. This is why the "nudges" inch along in a direction which
| is often at odds with the needs of the users.
|
| The solution to this broad class of structural problem in our
| economy, as argued by economists like Richard Wolff, is to build
| our economy out of firms which are largely cooperative in
| structure, where the workers and members of the co-op are
| representative of the users of the product or service. For
| example if your local water company is a co-op of users, with
| cooperative decision making power, the co-op isn't going to vote
| to raise water rates unnecessarily against their own users.
|
| A middle ground in many cases is unions. So if anything this
| article is unintentionally making a case for a tech workers union
| at Google. This would change the structure at Google in the most
| significant way currently possible under today's legal system.
|
| I think the idea that engineers should take more responsibility
| is a noble one, but it's not the real problem here. The problem
| is the structure of the firm.
| tempodox wrote:
| > ...when their plans for the product really are the right
| idea...
|
| And who gets to define what "right" is?
| TaylorAlexander wrote:
| In that sentence? That was talking about my interpretation of
| the OP, which suggests individual engineers advocate for
| themselves in traditional organizations when they believe in
| their ideas. So it would be the individual engineer deciding
| if they believe in their idea, though in traditional orgs it
| would be the managers that have some say in what gets
| implemented.
|
| However I also see people respond to me when I talk about co-
| ops who somehow think I want to be Joseph Stalin, where in
| the imagined case they are skewering me for wanting all the
| decision making power. But that is a fantasy as the whole
| point of a co-op is to have more diffuse decision making
| power than under traditional orgs!
| Juliate wrote:
| It's all very contextual, and dependant on the framework you
| use (moral, ethical, business, customer, societal, etc.).
| cousin_it wrote:
| I agree. As the article says, organizations give an "easier
| road" to projects that help the bottom line; also they give an
| "easier road" to people who will promote the bottom line, with
| fewer ethical qualms, and reward people for becoming more like
| that. The cause of the problem is the organization and how it's
| aimed.
| michaelt wrote:
| _> I appreciate the point of the article as written, which I
| think is to encourage engineers to push harder against the
| grain when their plans for the product really are the right
| idea,_
|
| My interpretation of the article is different.
|
| I think the author is merely _drawing attention to a force_
| that pushes businesses in a particular direction, without
| proposing any specific solution.
|
| The article doesn't say he had an alternative design or vision
| he wishes he had pushed harder, or anything like that. In fact
| he's full of praise for Manifest V3, which he considers the
| most elegant technical solution to a real problem, which he
| considered an indefensible security and privacy risk.
| bckr wrote:
| Article's last sentence
|
| "And we -- the well-meaning engineers -- shoulder much of the
| blame."
| satyrnein wrote:
| You don't think the local water co-op would raise water rates
| 10% to give themselves 10% raises? Unless literally every user
| is an owner, there will always be the incentive to extract as
| much from the general public as possible to distribute to the
| owners. Same with unions, they exist to benefit their members,
| not the general public.
| ClumsyPilot wrote:
| At the moment, all the power rents with capital. It would be
| nice to have some variety
| ahepp wrote:
| Wasn't the claim that an employee owned cooperative
| wouldn't raise prices to seek a higher profit?
| TaylorAlexander wrote:
| The article is about nudges. In that context, the more
| you align the users with the owners of the co-op, the
| fewer nudges move the product away from the users.
| Depending on opportunities to change the structure, this
| effect can be very strong. For example in a utility co-
| op, a structure already in use in some places, the owners
| are literally the users and they wouldn't raise rates on
| themselves. Note that isn't employee owned. If you had an
| employee owned manufacturing facility, they're less
| likely to pollute local waterways as they'd be polluting
| themselves.
|
| The status quo alternative is elite ownership of the
| utilities and manufacturing facilities, where those
| individuals receive a large share of the profits which
| they can use to insulate themselves from environmental
| pollution or cost of living concerns.
|
| The fact that co-ops aren't absolutely perfect in all
| cases is not strictly a knock against them. The status
| quo is particularly troubled.
| TaylorAlexander wrote:
| Agreed.
| repiret wrote:
| Typically in utility co-ops, every utility user is an owner.
| My local power company is a co-op. I get a vote on who is on
| the board. I get a dividend from time to time. They waste an
| inordinate amount of money on feel-good marketing crap just
| the same.
| TaylorAlexander wrote:
| Yes I am saying it would be a user owned cooperative. These
| are common. Not all users would be workers, but workers would
| be part of the same community and users would vote on pay
| packages. If the pay is so low workers quit and quality
| suffers, they will vote to pay them more.
| wavemode wrote:
| Not sure I quite follow the general thesis of this article. Or at
| least, it doesn't seem well supported.
|
| The article seems to be trying to argue that company leadership
| are not the ones responsible for the "evil" things that companies
| do. But this:
|
| > If you're an engineer at Google, Facebook, Apple, or Microsoft,
| it's always easier to propose architectural changes that don't
| hurt the bottom line, or perhaps bolster it by accident.
| Conversely, if your proposal stands to wipe out a good chunk of
| revenue, you either self-censor and don't bring it up -- or you
| end up getting sucked into endless, futile arguments.
|
| strongly implies that company leadership are indeed the ones
| responsible.
|
| Like, I think what the article is trying to say is that, Manifest
| V3 was designed due to real-world privacy concerns, not for
| profit motives. It just happened to get the right amount of
| support and buy-in from leadership due to being something that
| -also- aided profit motives.
|
| In other words, when a company leader has a variety of possible
| projects to invest in, she will naturally tend to invest in the
| ones with a long-term profit motive for the company. This also
| necessarily means -not- investing in other, potentially good and
| helpful and consumer-positive projects, that simply aren't as
| promising from a profit perspective. This phenomenon is what the
| article calls the "asymmetry of nudges".
|
| But I guess what I'm failing to grasp is how this means it was
| the engineers' doing and not leadership. Yes, the engineers came
| up with the idea. But in this scenario, it seems like the
| engineers were the ones who were well-meaning, and just doing
| their jobs. Whereas leadership were the ones chasing dollar signs
| at all costs. This is precisely in alignment with what most
| people posit when they say that big corporations are evil, no?
| ec109685 wrote:
| I think it's similar to this Sinclair quote:
|
| "It is difficult to get a man to understand something, when his
| salary depends on his not understanding it"
|
| Even the most well intentioned engineers aren't going to
| propose something that will dramatically impact the company
| bottom line. They are "nudged" (and the roadmap prioritized
| accordingly) to fall in line with the best interest of the
| company.
| delusional wrote:
| But the the article goes on to conclude:
|
| > And we -- the well-meaning engineers -- shoulder much of
| the blame.
|
| Which doesn't seem to align with that understanding.
| baq wrote:
| Oh nonono. The quote may be true for some people but
| engineers are paid to understand. It's that they are also
| paid for solving customers' - read employers - problems and
| not for warning about ethics.
|
| You want ethics to be a factor - you must introduce
| regulation so it becomes a non-zero weight in the solution
| space search engineers do.
| saurik wrote:
| Why must responsibility be with one party or the other? To me,
| it feels pretty obvious that both the executives _and_ the
| engineers are to blame! Just because someone is paying you to
| do something, that doesn 't automatically make it morally OK to
| do whatever makes the two of you the most money. The idea that
| someone is "just doing their job" is nothing more than a
| convenient excuse: even if you really really REALLY need the
| money for what feels like morally justified reasons, if you
| aren't at least simultaneously trying to get a job that doesn't
| require you to do something evil--much less doing what you can
| within your powers to stop and/or sabotage the effort--we
| shouldn't grant you a free pass.
| skulk wrote:
| So how, if engineers are equally responsible, do they enact
| their fair share of change? Adopt a strict guild-level code
| of ethics? I find this unlikely in the US political climate;
| someone will always happily step in to implement Manifest v3
|
| Someone in leadership could literally flick their wrist 3
| times and MV3 is dead.
|
| This doesn't sound like an equal responsibility type of
| situation to me.
| wavemode wrote:
| Exactly, this is what I was getting at. The leaders are the
| ones with all the decision-making power in this situation.
|
| The engineers would have to stage some kind of protest
| and/or quit their jobs to change things. Whereas all the
| leaders have to do is just stop choosing to invest in
| harmful projects.
|
| While I do see how it's one of those situations where, the
| system is set up to incentivize certain decisions. I get
| that. But that doesn't change the fact that the leaders are
| the ones making the decisions.
| immibis wrote:
| It means this is an emergent phenomenon, not something that any
| one individual in a corporation woke up and declared should
| take place.
| viraptor wrote:
| > But I guess what I'm failing to grasp is how this means it
| was the engineers' doing and not leadership.
|
| Maybe because the article doesn't claim that. (Shouldering some
| of the blame is not the same as what I quoted) It just presents
| different incentives that push decisions over time. The
| incentives that end up putting the engineers to do something in
| the end don't mean it's an engineers' fault.
|
| There's no point choosing one specific group to point fingers
| at, if we can instead learn more about the system and if we
| have the power, try pushing it slightly in a better direction.
| gary_0 wrote:
| The problem of browser extensions having "too many" capabilities
| allegedly boils down to some small subset of users stupidly or
| unluckily installing dodgy extensions and Chrome wanting to
| prevent this. But people are always going to do dumb things;
| outside of browsers, they're going to smoke cigarettes, they're
| going to drink and drive, they're going to eat too much junk
| food. How far are we willing to restrict freedoms to prevent dumb
| behavior? How many corners are we going to round off to prevent
| misfortune?
|
| From the perspective of a typical HN reader, Google and Mozilla
| have turned into Internet nanny states with Fisher-Price
| browsers. How far can they go in the name of "safety" before it's
| too far?
|
| Not to mention the problem the article highlights: their motives
| aren't pure. The more control they give themselves, and the more
| inconvenient third parties they marginalize, the more money they
| stand to make.
|
| Also, it's not a perfect A or B between flexibility and security.
| They could require extensions to be more open and inspectable so
| users could catch bad behavior. They could better police the
| extension store to catch malware faster. They could add more
| layers of warnings and permissions dialogs to prevent accidental
| compromise.
|
| At any rate, whether due to incompetence or malice, the situation
| is not as one-sided as Google pretends it is.
| jowea wrote:
| From what I remember the issue is similar to App Stores where
| average persons just can't know what the extension does. Or
| even worse, I remember multiple reports of devs of popular
| extensions getting shady offers to buy those extensions.
| delusional wrote:
| > And we -- the well-meaning engineers -- shoulder much of the
| blame.
|
| This does not follow from the rest of the article at all. I'll
| begin by acknowledging the concept of the "asymmetric nudge" as a
| useful thought. It does somehow explain and ground a feeling of
| engineers within large corporate structures, where somehow all of
| your good ideas turn user hostile. The author fails to
| sufficiently answer the followup question though. Why are the
| nudges asymmetric, and who holds responsibility for that?
|
| This is where the "sociopathic" executive comes in. The executive
| does not make technical decisions. Instead they make human
| decisions, like what projects to fund, what form of communication
| to accept, and what sorts of arguments to listen to.
|
| The power of the executive is not to censor designs, it's to
| instill the values into you that steers your self-censorship.
| salawat wrote:
| Ding ding ding.
|
| Welcome to the modern executive 101. If you are ever directly
| culpable, you aren't nudging well enough. You try to structure
| things around the peons to make them do that thing you want;
| but in a way responsibility never bubbles back to you.
|
| As a "peon", your moral job is to make that impossible by not
| tolerating hand waves, and pinning execs down into giving a
| clear, traceable, accountable order. Even if it makes them
| uncomfortable. If they aren't made uncomfortable, you aren't
| doing it hard enough.
| amluto wrote:
| I find this a bit hard to believe:
|
| > One of these had to give, and Manifest V3 was the most elegant
| technical approach. Far from being the brainchild of a
| sociopathic executive, its architecture was devised by well-
| meaning engineers on the Chrome team.
|
| The Chrome team has some very competent engineers. _lcamtuf_ is a
| well-respected security engineer. I would expect such a group,
| trying to solve a problem of poorly behaved extensions, to
| develop a nice privacy-respecting API to block requests.
|
| For example, there could be a way for an extension to run a
| portion of itself in a sandbox, such that the sandbox could
| inspect a request, decide whether to allow it, and output _only_
| an indication of whether to allow it. No further outgoing
| communication, including to the rest of the extension, would be
| allowed.
|
| But instead we got Manifest V3, and I simply don't believe it's a
| meaningful privacy improvement. Read the docs:
| https://developer.chrome.com/docs/extensions/reference/api/w...
|
| > Note: As of Manifest V3, the "webRequestBlocking" permission is
| no longer available for most extensions. Consider
| "declarativeNetRequest", which enables use the
| declarativeNetRequest API. Aside from "webRequestBlocking", the
| webRequest API is unchanged and available for normal use.
|
| Did well-meaning engineers on the Chrome team really come up with
| a security improvement in which extensions can read request and
| response headers but not block the requests? I'd love to see an
| explanation, but to me it seems that the security "improvement"
| is pretty narrowly tailored to prevent ad-blocking without
| meaningfully improving privacy.
| blibble wrote:
| it's a reasonably difficult problem as you want turing complete
| computation, but then if you have that you can make state
| escape the sandbox by blocking/not blocking certain requests
| and transmit a single bit at a time
|
| you'd think with their legions of competent engineers they'd be
| able to come up with some way of defeating this attack
|
| but that would hurt the business over the blunt MV3 approach,
| and you're not going to get promoted for that...
| amluto wrote:
| > transmit a single bit at a time
|
| This is a really awkward attack for a couple reasons. In
| general, a malicious extension may have no way to tell
| whether a request was blocked -- the origin if the request
| doesn't belong to the extension authors, and the portion of
| the extension outside the sandbox won't be told which
| requests were allowed. And, if too many requests are blocked
| apparently at random, the user may well notice.
|
| It's surely possible to sneak out some data, slowly, over a
| noisy channel, but it doesn't sound straightforward.
|
| Compare to actual manifest V3, where exfiltrating the keys to
| the kingdom appears to be entire trivial as long as the
| extension doesn't try to block ads.
| hyperman1 wrote:
| It is naive to paint the leaders as well intentioned in this.
|
| For example, Boeing moving its headquarter, so the decision
| makers are far away from the reality on the ground. This pattern
| is visible in less extreme ways in most companies. CxO's are
| typically on another floor than the other people.
|
| The idea is clear: They don't want to know what happens in
| reality. They want to be able to deny anything, while nudging
| everyone in the right direction.
| h0l0cube wrote:
| > It is naive to paint the leaders as well intentioned in this.
|
| TFA didn't do this. They posit a kind of passive malevolence,
| where things that hurt the bottom line are forbidden, but
| everything else is fine
| ynniv wrote:
| "Asymmetry of nudges" is more directly conveyed as a ratchet: you
| can only change things in ways that benefit the corporation.
| Rexxar wrote:
| Even if you can change in both directions with good arguments.
| The fact that changes are faster in one direction progressively
| shift the global situation.
| samatman wrote:
| This overtly binary and deterministic model is what the article
| argues against. I agree with them.
|
| There's a tautological interpretation of what you're saying
| which, since it's tautology, is always true. It relies on
| "corporate benefit" being whatever the responsible parties in
| the corporation decide it is.
|
| But in the more usual "fiduciary duty" sense where "benefit"
| means direct impact on the bottom line, then no, this isn't
| true. Corporations are run by people, and not only can those
| people decide to forgo some profit in order to do the right
| thing, this actually happens from time to time. A ratchet
| prevents movement except in the designated direction, or it
| isn't a ratchet: so a ratchet is a bad metaphor here.
|
| But, as the article also argues, there's a decided asymmetry to
| the flow here. Proposals which benefit the bottom line are easy
| to make, and easy to achieve buy-in on. Ones which have the
| opposite effect cost reputational capital, they're risky. So
| even a well-meaning corporation which is entirely owned and
| operated by people with a firm commitment to a vision of
| corporate benefit which isn't blindly determined by the bottom
| line, will miss opportunities to fulfill that vision which
| would negatively affect that bottom line, for structural
| reasons.
| cool_dude85 wrote:
| Good evidence that this guy is right about everyday engineers
| deserving blame for this kind of thing is the many hoops he jumps
| through to justify manifest v3. You don't try so hard to
| misrepresent the situation if you don't bear some responsibility.
| abofh wrote:
| Especially since the CEO in question certainly didn't bring
| "don't be evil" back, so citing a guidestone fifteen years
| expired seems disingenuous.
| awinter-py wrote:
| I may be late to this discussion, but what has changed in MV3
| that shifts the balance of power to publishers? declarative
| WebRequest?
| NikkiA wrote:
| The shift is really taken as being 'it breaks all current ad
| blockers', which is taken to be an intentional decision that
| shifts power away from the user.
| morpen wrote:
| No offense, but the author here is just describing the formation
| of structural power. The bigger the power structure, the more
| diffuse it's ethical influence can be, and, the less
| responsibility any one employee needs to feel for it. A more
| meaningful question I think is, if an organization or power
| structure inherently incentivizes unethical behavior, does that
| mean that that form of organization or power structure itself
| should be considered unethical?
___________________________________________________________________
(page generated 2024-07-06 23:01 UTC)