[HN Gopher] Twitter's Recommendation Algorithm
___________________________________________________________________
Twitter's Recommendation Algorithm
Author : jonknee
Score : 1553 points
Date : 2023-03-31 18:42 UTC (1 days ago)
(HTM) web link (blog.twitter.com)
(TXT) w3m dump (blog.twitter.com)
| corbulo wrote:
| It's disappointing the comments are so obsessed with the
| political angle to this that there's a total lack of appreciation
| (or discussion) of opening up the most influential social media
| platform in the world.
| smt88 wrote:
| This is transparency theatre, not actual transparency.
|
| There's no way to actually use this limited release to
| understand how or why any tweet is boosted, so we're in exactly
| the same boat we were in yesterday.
| corbulo wrote:
| This sentiment has high correlation to driving conclusions
| from a very time limited information set. This isn't the only
| part that is going to be posted to github.
|
| What is the net benefit from rushing to condemn something
| that can only be a net positive compared to the past
| alternatives? I don't understand the purpose of that
| approach. Help me.
| joshuamorton wrote:
| > can only be a net positive compared to the past
| alternatives
|
| This seems to be unsubstantiated. Are you really claiming
| that selective disclosure is _always_ superior to complete
| lack of transparency?
| cwkoss wrote:
| Yep "Limited Hangouts" are a common disinfo technique
| lannisterstark wrote:
| Yes. 1>0
| joshuamorton wrote:
| What is "one" and what is "zero"? Are you saying that
| information can't ever be misleading?
|
| Like if I tell you that your boyfriend has been having
| secret meetings with some woman you don't know, with full
| knowledge that the secret meetings are because she's a
| photographer and he's planning to propose, have I
| improved things by disclosing the information to you in
| that manner? Were my actions a "net positive"?
| corbulo wrote:
| The degree to which it is selective has yet to be
| determined.
|
| Are you claiming total ignorance is superior to partial
| revelation? I think we would all do ourselves better to
| go live on a desert island and abandon everything about
| modern life. A shovel might be useful to bury our heads
| while we're there.
| joshuamorton wrote:
| > Are you claiming total ignorance is superior to partial
| revelation?
|
| I am claiming that this is at least sometimes true, yes.
| Not always, but _sometimes_.
|
| You're the one claiming that partial revelation is
| _always_ , without exception, superior to total
| ignorance. That seems unlikely. Propoganda is often
| partial revelation, are you saying it is always better to
| receive only propoganda than to receive no information at
| all?
| corbulo wrote:
| I think there is objective value to understanding
| propaganda's origins and goals. Those who do not study
| history are doomed to repeat it. Those who do not
| understand propaganda are highly likely to be controlled
| by it.
|
| Propaganda can be a pretty vague term by the way. Can you
| describe how the public coming to a better understanding
| of the inner workings of the worlds most influential
| social media site is merely propaganda?
| joshuamorton wrote:
| > I think there is objective value to understanding
| propaganda's origins and goals
|
| Of course, but this requires the propaganda be
| contextualized, which wasn't a part of the situation I
| was suggesting.
|
| > Can you describe how the public coming to a better
| understanding of the inner workings of the worlds most
| influential social media site is merely propaganda?
|
| You're begging the question.
|
| Does Twitters disclosure of parts of the algorithm used
| (but certainly not all!) actually lead to a better
| understanding of the inner workings of Twitter? Or is
| such a release actually serving some purpose beyond
| transparency?
|
| If we had that, I'd agree it would be good. But I'm not
| convinced we do.
|
| Elon's done the exact same thing before at Twitter with
| his selective disclosure of material to friendly
| journalists in the "Twitter files". That, in my opinion,
| led to an overall worse understanding of Twitter's
| actions, not better.
| corbulo wrote:
| Then the analogy fails to hold up to reality. There's
| plenty of information to contextualize. Assuming by
| default everyone but you is a naive doe lost in a forest
| and therefore lacks the intellect to contextualize
| anything for themselves is undemocratic.
|
| >But I'm not convinced we do.
|
| That would likely be because, as mentioned previously
| there is still more to come out; it's not possible to be
| reasonably convinced yet. Going back to the original
| question:
|
| >>What is the net benefit from rushing to condemn
| something that can only be a net positive compared to the
| past alternatives?
| smoldesu wrote:
| > There's plenty of information to contextualize.
|
| There's _nothing_ , that's what the point of this
| thread/argument is. The portion that Twitter has opened
| (or even simply committed to opening) is not remotely
| enough to hold them accountable.
|
| The code they've released here is less helpful than a
| single Helm chart.
|
| >>>What is the net benefit from rushing to condemn
| something that can only be a net positive compared to the
| past alternatives?
|
| Because this is _useless_ , and worse yet, it's
| _pointless_ and blatant virtue signalling. I stood up in
| defense of Musk 's private bid for Twitter, but there's
| nothing worth licking his boot over here. The suits don't
| care. The Open Source community gains nothing. The users
| will never see, interact with or modify the
| recommendation code. Nobody will be able to meaningfully
| audit anything until Musk stops selectively burning the
| books at Twitter HQ.
|
| If Elon wants Twitter, he has the money to go get it. If
| he wants my respect, he's got to do an awful lot more
| than making "the algorithm" public. This release is so
| pathetic that it's probably colored my opinion of Musk
| more than any of the opinion rags I've seen yet.
| joshuamorton wrote:
| What analogy?
|
| > Assuming by default everyone but you is a naive doe
| lost in a forest and therefore lacks the intellect to
| contextualize anything for themselves is undemocratic.
|
| Luckily not a claim I'm making. It's better for
| conversation if you reply to what I say, not
| misrepresentations thereof.
|
| > Going back to the original question:
|
| And I replied to it already, but I'll reiterate: I don't
| believe this change "can only" be a net positive, what
| makes you believe that is the case?
| Joeri wrote:
| I did find the article more enlightening than the source
| code. I always suspected the quality of a tweet did not
| factor in to its promotion, only its engagement, and now they
| have confirmed this to be the case. Now I understand better
| why twitter seems to be filled with people angrily retweeting
| what I consider to be low quality clickbait tweets.
|
| As long as they don't try to tackle tweet quality at all
| separately from engagement twitter will remain unappealing to
| me.
| jrochkind1 wrote:
| What does "quality of a tweet" mean, how might you measure
| it?
|
| We pretty much knew this is how all social media works,
| becuase a) engagement is what they want, why wouldn't they
| be optimizing for it, and b) how else might you measure
| 'quality'? Back when this started, I have no trouble
| believing some well-intentioned engineers thought that
| engagement was a good proxy for quality. A bunch of users
| give it the "like", isn't that a collective assessment of
| quality? Who is to say what quality is, overruling the
| users in aggregate?
|
| I agree it has the negative effects you mention; and I've
| read lots of people writing about this, it's of course not
| a new observation.
|
| But I agree it's good to have an explanation of what's
| going on, even when parts of are what we basically knew was
| happening on all social media networks. confirmed is better
| than "basically knew", for understanding how these things
| that effect our experiences (and our society) work.
| dbbk wrote:
| "Quality of a tweet" is impossible for a computer to
| understand. That's like asking to quantify the value of a
| work of art just from the image.
| SilverBirch wrote:
| One of the things that makes my spidey sense tingle is when
| people say _oddly_ sycophantic things about Elon Musk. Twitter
| is big, it 's important. It's not "the most influential social
| media platform in the world".
| corbulo wrote:
| I only see one social media site posts being constantly
| reposted on global news organizations. Are all of the
| corporations, world governments, and leaders with tens of
| millions of followers actually wasting their time by
| dedicating their social media teams time to twitter instead
| of focusing on some other social media site thats more
| influential? Which one is it then?
|
| It's quite odd to attribute objective analysis to sycophancy.
| I intentionally didn't mention him but here you are bringing
| him up and fulfilling my point. Who is the sycophant?
| HeckFeck wrote:
| I have to admit I am geeking out whilst skimming source code I
| barely understand.
| yurodivuie wrote:
| I'm sure we can all think of examples where a power structure
| (a company, a country, a prison, a family) invited people in
| for a supervised tour that was less than honest in its
| presentation.
|
| But really, if people respond to Twitter's actions politically,
| that response exists within a context that was certainly
| influenced by Twitter's prior actions.
| lawgimenez wrote:
| Just read the article and not the comments. Comments here used
| to be something you learn new stuffs, apparently that is not
| the case anymore.
| KyleBerezin wrote:
| I would love for the system to be somehow auditable, to verify
| this algorithm is THE algorithm.
| mlindner wrote:
| I had to go to the second page to find this. I completely
| agree. I clicked this thread looking forward to seeing some
| intelligent discussion of the merits of the source and
| issues/interesting tidbits, instead I see a bunch of Elon Musk
| ranting and complaining and pointing out drive-by poor pull
| requests issues. I really wish the adults would start to talk.
| concordDance wrote:
| I would definitely love a techy community that focused on
| technology like the hackernews of 10 years ago. If you know
| of one then please link me to it.
| losvedir wrote:
| lobste.rs is pretty good, in terms of higher tech
| discussion signal to noise, I think, though it still does
| have a bit of a political bent. Email me if you want an
| invite.
| nonethewiser wrote:
| The funny thing is that angle owes itself to Elon coming
| through on his promise to open source this.
|
| This is a great thing.
| deckard1 wrote:
| It would also be a total Elon move to confuse the open
| sourcing of Twitter's internal code with actual transparency.
|
| You would need, at a minimum, a neutral third-party audit of
| Twitter's servers to conclude that the source code we see on
| GitHub is, in fact, the source code running Twitter. How
| often will they keep their GitHub repo in sync with their
| internal code, I wonder.
|
| Presumably Twitter uses a version control system. But they
| scrubbed the history so that's also a point against their
| "transparency" claims. Without knowing the when and the why
| of changes you can't understand what you are looking at.
| People are pointing to that "author_is_elon" without knowing
| whether that was done before Elon bought Twitter or after.
|
| But even then, git history can be faked.
|
| > This is a great thing.
|
| I disagree. It's the opposite. It provides the illusion of
| openness without the quality of openness, thus killing the
| debate once and for all.
| fanagra32 wrote:
| "Opening up"? You must be kidding. Nothing is open there. It's
| just open-washing. A few nice diagrams, but how the services
| _actually_ work is still hidden.
| mardifoufs wrote:
| If you ignore the hundreds of thousands of LoC... then yeah I
| guess it's just diagrams? Are you sure you actually looked at
| the main code repo?
| hayd wrote:
| > Most of the recommendation algorithm will be made open
| source today. The rest will follow.
|
| So let's wait and see...
| random_cynic wrote:
| [flagged]
| concordDance wrote:
| I would love to know where the OG users went before the
| reddit invasion completes.
|
| If anyone knows then please tell me!
| Zetice wrote:
| They had kids, stopped using social media, and/or started
| charging for their insights.
|
| Besides, if you have to ask, it's not for you.
| drstewart wrote:
| It's actually shocking to see some of the discussion and
| understanding of the code here. I'd say the "part-time
| programmers" qualifier is even generous.
|
| Sad to see.
| xeromal wrote:
| It seems to be that way but its also in the rules not to
| complain about that. I do wonder if it's getting worse.
| chocolatkey wrote:
| I would agree the quality of posts has gone downhill, and is
| way too political for my liking, but there's not much other
| place to go that isn't elitist.
| klodolph wrote:
| Discord. As much as I hate Discord the platform (and I
| really do hate it, far more than, say, Twitter), Discord
| does have a lot of good communities.
| chocolatkey wrote:
| I don't think Discord is a substitute for HN. If I know
| an exact community with a shared interest I want to join
| then yes, I can find their Discord an join it. But how am
| I supposed to come across that topic and become
| interested in the first place? HN (in its ideal form) is
| a place where I am introduced to a variety of random but
| potentially interesting technology-related topics, from
| programming languages to passion projects to lessons in
| business etc, some of which I will pursue and seek out a
| community for, and others which I only care about
| minimally, not enough to want to join a dedicated
| community for.
|
| If you want to compare HN to Discord, I would say that HN
| is like joining 500 Discord servers that together
| comprise the type of content discussed and posted about
| on HN, and every day checking each one's announcements
| channel, and skimming through general chats. But you
| still have the issue of being introduced to new topics,
| for example if you're interested in the AI trends then
| your selection of Discord servers that comprise what's
| posted on HN would be dramatically changing in the past
| few months.
| mlindner wrote:
| I've found Discord worse in every way. Almost every
| server I've been in is highly reactionary to any kind of
| accidental phrasing. Walking on eggshells is a constant
| process in any Discord server. It also descends into the
| same type of problems as any other chat services as it
| encourages short messages rather than long well-thought-
| out posting.
| somedude895 wrote:
| It has indeed felt a bit like r/LateStageCapitalism or
| r/antiwork in here, with so many low quality comments that
| all read exactly the same. I figured it was just because of
| the tech layoffs that that sort of sentiment has been gaining
| traction though.
| PenguinRevolver wrote:
| Great pull request here which improves the algorithm:
| https://github.com/twitter/the-algorithm/pull/17
| hrpnk wrote:
| Aside from the spam PRs, there is actually one PR that fixes a
| bug: https://github.com/twitter/the-algorithm/pull/242/files
| mdaniel wrote:
| Modern Java actually allows `10_000` for that very reason, as
| does Scala (https://scala-
| lang.org/files/archive/spec/2.13/01-lexical-sy...)
| fooey wrote:
| also javascript - https://github.com/tc39/proposal-numeric-
| separator
| bitshiftfaced wrote:
| Add golang to the list.
| sroussey wrote:
| Yes please! I definitely put my thumbs up in there!
| drstewart wrote:
| That will definitely do something! Good job!!
| [deleted]
| BbzzbB wrote:
| It removes the extra weight to Twitter blue tweets?
| idle_zealot wrote:
| If the property names are to be believed it sets a weight
| multiplier to 0. So it prevents recommending them entirely.
| SketchySeaBeast wrote:
| It sets the default to zero, but apparently can range up to
| 100. So... what modifies it? (The answer is probably in
| there somewhere, but I'm sure someone will find it before I
| do.)
| zachnwhite wrote:
| [flagged]
| SquareWheel wrote:
| The trouble with spammy jokes like this is it discourages
| companies from bothering with open-source in the future. I
| know I'd be less likely to champion an initiative like this
| if I thought it might blow up in my face.
| drstewart wrote:
| Mature. I'm sure companies looking to outsource work with
| Twin Prime Media would be stoked to see this level of
| maturity.
| matsemann wrote:
| Me feed has lately been full of accounts that have blocked me.
| Like, I see a tweet from someone unknown, click their profile
| and it says I'm blocked.
|
| So wonder if some value is wrong in one of those constants.
| Anyways, the blocking feature is broken..
| super256 wrote:
| I think it's okay to give Twitter Blue users a boost, as it's
| most likely _not_ spam (unlike the 95% of my non-blue followers
| who are bots).
| kevincox wrote:
| I think it makes sense for out-of-network. However I see no
| value for boosting among people that I have followed. If I
| have followed them they definitely aren't spammers (from my
| PoV).
| super256 wrote:
| I agree! Haven't thought of that.
| mlindner wrote:
| Drive-by pull requests that break the intention of something
| aren't ever going to be taken by a maintainer.
| simonsarris wrote:
| That would be great (unweighting bluechecks) but they actually
| plan to go in the other direction: Starting April 15th non-
| bluechecks won't show up in the "For you" section (the
| algorithm timeline) _at all._ Unpaid users are being written
| completely out of the algo.
|
| https://twitter.com/elonmusk/status/1640502698549075972
| alfor wrote:
| I don't see a way out of this with the GPT/AI able to create
| fake persona in an instant.
| thefreeman wrote:
| What does that matter? If people find the content engaging
| then it will be amplified. If not, it shouldn't be there in
| the first place. This whole "AI / Bot swarm" excuse is just
| smoke and mirrors for "I want more people to pay twitter".
| yreg wrote:
| If you don't care whether the content is generated then
| you don't need to use a social network at all. GPT can
| generate you an endless interesting feed no problem.
| nonethewiser wrote:
| If _accounts_ find the tweet engaging
| lhnz wrote:
| If "bots indistinguisable from humans" find your "account
| that posts pro-russian propaganda" engaging it will be
| amplified onto your For You page.
| seydor wrote:
| Followed users will still show
|
| But i agree it is a bad idea. The worst actors have money to
| buy the blue marks of an army of accounts.
|
| This is basically making it easier for authoritarian
| governments to abuse it
| dazc wrote:
| Like that wasn't already happening?
| seydor wrote:
| not with blue marks
| dazc wrote:
| I'm not in the US
| seydor wrote:
| It's actually non-US autharitarian governments that the
| biggest abusers, using social media against their own
| people
|
| https://www.theguardian.com/technology/2021/apr/13/facebo
| ok-...
| anigbrowl wrote:
| Inaccurate. Musk stated that people you follow will continue
| to show up. I only use the 'for you' feed when I'm bored and
| want stupid dopamine hits, I leave it on Following almost all
| the time. But that's on desktop, my understanding it that it
| keeps resetting itself for mobile users (of whom I am not
| one).
| astrange wrote:
| He only said that later; he seems to be working on a "say
| we'll do feature and then change it all after people yell
| at him for weeks" process.
| mgiannopoulos wrote:
| Sounds like moving fast and adjusting to user feedback,
| usually something commendable in tech, right?
| KerrAvon wrote:
| No. There's an implied willingness to listen to people in
| the first place when you're agile in response to
| feedback. You shouldn't have to bruise Elon's ego to get
| him to do not-stupid things.
| AmericanChopper wrote:
| This distinction you're making relies entirely upon how
| you personally interpret the motives of the people
| involved. An interpretation that you've entirely invented
| for yourself.
| astrange wrote:
| Approximately every US news organization and Lebron James
| announced they weren't going to pay for checkmarks
| yesterday, so he announced a surprise twist where the top
| 10,000 famous people get free checkmarks.
|
| ...so he's now not charging people who can pay, while
| charging people less able to pay.
|
| Though, the main problem is that Twitter is about as big
| in Japan as it is in the US, but Elon only thinks about
| US culture wars because his only friends are a few VCs
| and a retired postal worker crank named catturd2. So none
| of his new ideas are going to work there.
| AmericanChopper wrote:
| Criticizing somebody by speculating about their motives
| really only reflects badly upon you, especially if you're
| going to accompany it with that much hyperbole. All
| you're communicating is that you've replaced your ability
| to reason about a topic with some version of the
| fundamental attribution error.
|
| > the main problem is that Twitter is about as big in
| Japan as it is in the US
|
| It's not even close. When you account for the value of ad
| impressions, the US market is worth about 3.5x its next
| biggest market (Japan), and it drops off very sharply
| after that. You can argue that Twitter's policies should
| be less US-centric. But the reason they are that way in
| the first place is because the US market is their most
| valuable market by a long shot.
| astrange wrote:
| > Criticizing somebody by speculating about their motives
| really only reflects badly upon you, especially if you're
| going to accompany it with that much hyperbole.
|
| No, you can tell who he cares about because he replies to
| them, and because his emails were released in discovery
| from the earlier Twitter lawsuit and it turned out his
| friends put him up to it because they were mad the
| Babylon Bee got banned for a US culture war pronouns
| joke.
|
| I certainly don't have to expect he'll make good
| decisions, since he has no experience with the business,
| is a completely atypical user, was forced to buy it in
| the first place, and has lost $20 billion so far by his
| own valuation. (May have lost a few millions more by
| firing that disabled Icelandic acquihire in violation of
| his contract and a few discrimination laws.)
|
| The network effect is very very strong though. Even if
| someone makes a good competitor (Instagram is prototyping
| one apparently) I'd be surprised if bad decisions
| actually killed it.
|
| > When you account for the value of ad impressions, the
| US market is worth about 3.5x its next biggest market
| (Japan), and it drops off very sharply after that.
|
| I suggest reverse adjusting for how poor their ad
| targeting is. I've used it both places; the ads are
| actually good and relevant in Japan (...except for being
| for domestic apps, so not valid for tourists) but in the
| US they've always been nonsense. eg if you follow any
| doctors it will just assume you're also one and burn the
| budget of every medical ad it's got showing them to you.
| npunt wrote:
| More like rewarding someone for putting out a fire that
| they started. Not exactly a model of good governance.
| mgiannopoulos wrote:
| Musk said from day one of his Twitter purchase that there
| will be mistakes. Move fast and break things as that
| other guy said. :)
| cdash wrote:
| I have had it reset a couple times on mobile a few months
| ago but at least recently I have not noticed it happening
| anymore.
| oneeyedpigeon wrote:
| Just pressing the home icon twice takes it back to For
| You.
| mwill wrote:
| Which is incredibly annoying, as it used to scroll you to
| the latest tweets, I have years of muscle memory for
| hitting the home button
| cwkoss wrote:
| it's a shame we can no longer short twitter stock
| bradly wrote:
| I believe LeBron James said recently he isn't going to waste
| his money on a blue checkmark, so it should be interesting to
| see what stays and what goes.
| dazc wrote:
| I had to search who this person is, and I still could not
| care less.
| zaroth wrote:
| LeBron doesn't get $84 of value from Twitter? Definitely
| not a political statement going on there.
| katamarimambo wrote:
| Lebron famously uses the free version of Spotify. Maybe
| he just does not see the value of paying for the blue
| mark.
| TaylorAlexander wrote:
| Parent didn't say it's not "political". It's reasonable
| for a wealthy person to feel that a system that
| discriminates against the poor is not a system they want
| to participate in.
|
| (Note that I use discriminate in the literal sense, as a
| simple statement of fact.)
| web3-is-a-scam wrote:
| lol you don't actually believe that's his reason do you?
| TaylorAlexander wrote:
| LeBron started a school where poor kids can get free food
| and clothes. The man obviously cares.
| nonethewiser wrote:
| But the example you give is an appeal to a universal
| moral good. Not partisan politics. So despite saying it's
| not not political, your justification is that it's not
| political.
|
| Also, how did you get a blue check before being able to
| buy one?
| TaylorAlexander wrote:
| Some people unfortunately view concern for the poor as
| political. However my point of mentioning politics is to
| say that "it's political" is not any kind of gotcha when
| it was never denied as being political. Regardless of the
| actual justification being political or not, the
| "political" gotcha is nonsense.
| nonethewiser wrote:
| That's bullshit. Virtually everyone agrees poverty is a
| problem. Sure, the welfare state feeds the cycle of
| poverty, but it's not like that was the goal.
| dazc wrote:
| Are you sure about that? If I break out of the poverty
| cycle who am I going to vote for?
| nonethewiser wrote:
| Republicans I guess, but the point is that doesn't mean
| Democrats hate poor people. That's just political
| framing.
| TaylorAlexander wrote:
| Absolutely not bullshit. Some cynical people on the
| internet believe this, and that's what I thought the
| person I was replying to was saying. It is an extremely
| low bar to say "some people believe X", and I don't know
| why you care to question that. Even with your own reply
| you say "virtually all people agree" and your use of
| "virtually" acknowledges that not everyone agrees with
| you, therefore some people do believe what I say. This is
| anyway such a silly tangent and was not even my point.
| nonethewiser wrote:
| The exception proves the rule though.
| oneeyedpigeon wrote:
| He may well, but he may have also concluded that the
| indirect cost of having a blue tick outweighs the
| benefits.
| __jem wrote:
| Twitter doesn't get $84 of value from Lebron?
| 6d6b73 wrote:
| Like it or not but it's the twitter that gets value from
| celebrities. How many people are on social networks jusy
| so see what their fav celebrites are doing?
| nonethewiser wrote:
| It obviously goes both ways. Social media is a megaphone
| and ego boost for celebs.
| mullingitover wrote:
| The problem for twitter is it isn't the only game in town
| when it comes to social media, not by a long shot.
| They're not even in the top ten. They're a megaphone in a
| large pile of megaphones, and those other megaphones
| don't bite the hand that picks them up.
| ceejayoz wrote:
| Twitter needs the LeBrons of the world far more than they
| need Twitter.
| dpifke wrote:
| As someone who was on Twitter long before Oprah, or Elon,
| or Obama, or most other celebrities and politicians: I
| strongly disagree.
|
| (Speaking about Twitter the product, not necessarily
| Twitter the company.)
| ceejayoz wrote:
| That was an entirely different era. If you yearn for it,
| come on over to Mastodon.
|
| Twitter _the multi-billion dollar advertising company_
| needs the big whales. I think it was pretty clear the
| company was under discussion.
| dazc wrote:
| What their fav celebrites say -v- what 'the media' says?
| anigbrowl wrote:
| Most of the major news outlets are not doing so either. The
| Elon stans are crowing that this will be the long-overdue
| end of legacy media, but it strikes me that the new 'blue
| check twitter' might end up becoming even more of a social
| bubble than what it replaced. There are _so_ many low
| quality accounts sporting a checkmark now that users who
| value substance will soon be incentivized to just block
| anyone they find annoying.
| Avshalom wrote:
| yeah, they, uh seem to have realized that's gonna be a
| problem.
|
| https://variety.com/2023/digital/news/twitter-business-
| month...
|
| and of course all this means is that the organizations
| most likely to be able to afford it won't have to
| anigbrowl wrote:
| Good find! It'll be funny to see if the incumbents
| respond with 'don't do me any favors.' Also to see
| whether Musk's frens sulk aboit him selling out to the
| elites or so - their gratitude has an extremely short
| half-life.
| nonethewiser wrote:
| So you need a blue check mark to reach non followers and
| Lebron won't get one? Sounds like it's making Twitter
| better already.
| MKais wrote:
| NY Times, WaPo, LA Times and other major accounts too
| https://www.thewrap.com/ny-times-la-times-not-pay-for-
| twitte...
| nonethewiser wrote:
| Seems dumb of them. Cost is trivial and their competition
| that isn't so politically motivated will have a much
| further reach.
|
| The smart move would be silent on the policy change, pay,
| and support rival platforms as they can. Instead they
| will eventually pay and look like they lost.
| mullingitover wrote:
| > Seems dumb of them. Cost is trivial and their
| competition that isn't so politically motivated will have
| a much further reach.
|
| It's wild hubris for twitter to try to invoice/penalize
| the very users and organizations that make twitter
| anything but insolvent. There should be money exchanged
| here, but it should be flowing _generously_ and most
| importantly _in the other direction_.
| VWWHFSfQ wrote:
| maybe they can make their own twitter and give themselves
| a big blue check mark there
| AuryGlenz wrote:
| We're in a time where ideology trumps revenue for some
| companies. You know, "Get woke, go broke."
| flangola7 wrote:
| Everyone says that about things that made tons of money
| though.
| myko wrote:
| The phrase is "Better Broke Than Woke", we'll see how it
| works out for folks
| fooey wrote:
| It's smart because Musk already blinked
|
| The top 10,000 are getting exemptions and won't have to
| pay
|
| https://variety.com/2023/digital/news/twitter-business-
| month...
| runako wrote:
| For the NYT to verify their official accounts plus those
| of their reporters (using the Twitter Blue Affiliations
| feature) would be $1m annually. This, for a budget line
| item that has heretofore been $0. In this economy, that's
| a reach.
|
| I don't think the NYT is worried about "reach."
| mgiannopoulos wrote:
| How do you get the 1 million figure?
| brandonagr2 wrote:
| An affiliated account to a verified org is $50 per month
| per seat, so NYT would have to authenticate 1,647
| affiliated accounts to reach 1 million dollars per year
| runako wrote:
| Exactly this. Use the published number of reporters at
| the NYT and multiply.
| mgiannopoulos wrote:
| Are all of these 1647 reporters (they have that many??)
| and posting on Twitter? That's a lot of traffic
| generators or not. Surely they could just do the bulk
| with 100 or so.
| runako wrote:
| You're suggesting the NYT further tier its reporting
| ranks, along with all the internal difficulty that would
| entail. For ex: obviously the 100 have to include the
| most senior reporters, who are also older and therefore
| the least likely to create the viral content NYT wants
| affiliated with their account, so immediately they
| probably need to look at a much larger number. For
| another example: social media is different for each
| reader, or from the other side, each reporter has a
| constituency. In one season, the fashion reporters are
| driving views, while the following season it's the
| European war correspondents or the economics reporters
| (and all of these desks have subdivisions that wax and
| wane in popularity).
|
| And all that discussion so that they can spend $72k
| annually with Twitter, a y/o/y increase of $72k from last
| year. With no guarantees of reach, because the whole
| paid-only verification thing is an experiment that began
| an hour ago. Let me just say that this whole pitch is
| going to be...difficult... at the point in the economic
| cycle where we find ourselves.
| mgiannopoulos wrote:
| The NYT has revenues of 2.1 billion. I'm sure they have a
| marketing budget and probably already spending money on
| Twitter to get traffic. This isn't something strange.
|
| Facebook did they same thing btw, just more gradual. For
| years they changed the algorithm slowly to take away
| reach from Pages only to offer it back as long as you
| paid.
| nonethewiser wrote:
| Seems like a fair cost to pump spam
| nonethewiser wrote:
| > I don't think the NYT is worried about "reach."
|
| Then why are they on Twitter?
| tick_tock_tick wrote:
| > I don't think the NYT is worried about "reach."
|
| LOL they are desperate for reach. Incredibly so; have you
| not listen to any podcast by them? They are begging
| people to go to their site. They get a fraction of the
| organic traffic they used to and nearly everything is
| driven from other site like Twitter, Google News,
| Facebook, etc. The internet age has not been kind to
| classic news orgs.
| astrange wrote:
| The NYT has been doing great recently. They're probably
| the legacy news company that's doing the best online of
| anyone.
|
| That's on the strength of having a lot of verticals like
| games, recipes and Wirecutter though.
| adamrezich wrote:
| have you seen @nyttypos and (to a lesser extent)
| @nyt_diff? NYT online editorial standards are hilariously
| abysmal.
|
| https://twitter.com/nyttypos
|
| https://twitter.com/nyt_diff
| astrange wrote:
| Yeah and it doesn't matter.
| adamrezich wrote:
| correctly writing words and punctuation on the page
| digitally printed by "the legacy news company that's
| doing the best online of anyone" doesn't matter at all?
| isn't that like the bare minimum of what their job
| consists of?
| astrange wrote:
| I was responding to a post saying they weren't getting
| online traffic by saying they are getting online traffic.
| Nothing about the quality of their content.
| drstewart wrote:
| Depends on how they weight the is_user_china_mouthpiece
| variable
| willmeyers wrote:
| Why do companies even bother to put source up on github? To put
| up a front that their open source? What a joke.
| tric wrote:
| GitHub repo: https://github.com/twitter/the-algorithm/
| minimaxir wrote:
| Notably, it's AGPL-licensed.
| [deleted]
| hooverd wrote:
| I wonder how useful this is without the knowledge and tooling
| around deploying it.
| rurp wrote:
| That's my thought as well. Complicated system like this rely
| on all sorts of related services and data stores. This seems
| like the sort of thing that sounds a lot more interesting
| than it is in practice. I would bet many non-technical people
| expect "The Algorithm" to be a straightforward and self-
| contained system.
| [deleted]
| capableweb wrote:
| I'm no fan of either Twitter nor Elon Musk, but this is a great
| move and I hope other companies follow what Twitter did here and
| start open sourcing more core parts like this. Maybe it's mostly
| useful for learning how it works, not for directly using it in
| your own product, but the amount of transparency it gives users
| cannot be understated. As long as that actually is the code they
| run, but there would be no way for anyone but Twitter to verify
| that.
| cubefox wrote:
| I think it mainly helps with accountability regarding free
| speech. They did and do several kinds of shadow banning and
| down-boosting to combat spammers, which always has some false
| positives. If you the algorithm is published, you could at
| least better judge and argue when you are unfairly "silenced".
| Since this may be due to an avoidable flaw of the algorithm
| instead of some accepted collateral damage.
| d_sc wrote:
| I think they have a bug here here:
| https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92...
|
| Code: ( "has_gte_10k_favs", _.getOrElse(EarlybirdFeature,
| None).exists(_.favCountV2.exists(_ >= 1000))),
|
| Should be: ( "has_gte_10k_favs", _.getOrElse(EarlybirdFeature,
| None).exists(_.favCountV2.exists(_ >= 10000))),
| elashri wrote:
| I wonder if it will be possible in one day to know what is values
| of `author_is_power_user`, `author_is_democrat` and
| `author_is_republican` for your account. Does GDPR help with
| that? probably not because maybe they do it for people inside the
| us only so it is not related to EU anyway.
| photochemsyn wrote:
| I generally have a very low opinion of social media platforms,
| but I did create a Twitter account for the first time after Musk
| bought the platform.
|
| My conclusion is that it's basically entertainment, with very
| little of what I'd call high-quality useful information that
| deserves further examination (unlike a lot of HN posts, in
| contrast). I also notice something of a Tik-Tok approach to video
| being implemented, which is not surprising given Tik-Tok's
| success (and makes one wonder who exactly it is lobbying so hard
| for a Tik-Tok ban, and whether it's just a commercial competition
| issue more than anything else).
|
| As far as the recommendation algorithm, it appears to be a
| siloing setup - look at content of one particular flavor, it
| gives you more of that flavor. A 'flush settings' or 'forget
| browsing history' or 'reset to defaults' button would be useful,
| if probably not what advertisers want in terms of delivering to
| target audiences. I suppose setting up multiple accounts is
| something of a solution, although too much effort to be that
| interesting.
|
| In terms of news reports, it's broader in scope than traditional
| corporate media outlets, so that's a plus in its favor.
| Reliability is perhaps similar (i.e. low).
| lhnz wrote:
| You can follow accounts that only post arxiv.org links for ML
| papers or anything else you're interested in if you want to. If
| you're only getting entertainment then it says a lot about the
| original accounts you followed.
| sroussey wrote:
| Does it show the part where is recommends Elon more than anyone
| else?
| jmholla wrote:
| I think this PR is modifying the inputs to the methods that do
| it: https://github.com/twitter/the-algorithm/pull/17
| devrand wrote:
| I couldn't find anything specific to that, but I did find thus
| blurb where they seem to explicitly track how often they're
| serving Elon's tweets for A/B testing experiments:
| https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92...
| Chinjut wrote:
| Perhaps that's related to this line. Though perhaps this is
| just used for observing metrics.
| https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92...
| JasonZ2 wrote:
| [dead]
| jrwr wrote:
| That whole list is a hoot,
|
| has_toxicity_score_above_threshold
|
| is a interesting value, I wonder were the 0.91 was though up
| at
| ano-ther wrote:
| This is one: https://github.com/twitter/the-
| algorithm/issues/121
|
| Search for Elon gives this: https://github.com/twitter/the-
| algorithm/search?q=Elon&type=
| anshumankmr wrote:
| Oh god... The MR's opened today are the craziest ones ever.
| https://github.com/twitter/the-algorithm/pulls?q=is%3Apr+is%...
|
| They made my morning
| tech234a wrote:
| I wonder what the "author_is_elon", "author_is_power_user",
| "author_is_democrat", and "author_is_republican" labels are for
| [1].
|
| [1]: https://github.com/twitter/the-algorithm/blob/main/home-
| mixe...
| Someone1234 wrote:
| Here is a screenshot in case this changes later:
|
| https://i.imgur.com/F8GSeyH.png
|
| And, no, this wasn't in a merge-request, it was in the "main"
| branch of HomeTweetTypePredicates.scala.
| tantalor wrote:
| What's all the "DDG"? Is this data from DuckDuckGo?
| asddubs wrote:
| I doubt it, since the isElon thing also has the
| abbreviation
| [deleted]
| tyingq wrote:
| Probably something like "Data Distribution Group".
| nanny wrote:
| Maybe the name for an internal service/environment. It's
| also referenced in this viral tweet from November:
|
| https://twitter.com/EricFrohnhoefer/status/1591969100225736
| 7...
| bcherry wrote:
| DuckDuckGoose is the twitter A/B test framework
| mcast wrote:
| That's actually kind of a cool and fitting name.
| nonethewiser wrote:
| I see Twitter engineers are not from Minnesota.
| bilekas wrote:
| All in service of 'anti-bias' of course... /s
| [deleted]
| RoyGBivCap wrote:
| [dead]
| spaceman_2020 wrote:
| pretty sure Elon gets a boost in the algorithm. All okay - he's
| the owner of a private entity and can do as he pleases.
| mnd999 wrote:
| That's what he blew all that cash on. It's the whole point.
| montag wrote:
| Elon is addressing this in the Twitter Space right now. "It
| definitely shouldn't be dividing people into Republican and
| Democrats; that makes no sense[...] you've identified something
| we should be getting rid of right away."
| asddubs wrote:
| but how can we be sure it isn't doing that?! first, we would
| need to figure out a way to identify who-
| Imnimo wrote:
| As if Elon has a clue what that feature is or is not being
| used for.
| dmix wrote:
| He didn't say he knew what it did? It's a good enough
| response to say that it shouldn't be doing that period.
| dymk wrote:
| Generally, one should understand chesterton's fence
| before tearing it down
| dmix wrote:
| He was answering a question on the spot live on a video
| call. Careful consideration can come later by the devs
| who will investigate.
|
| Must be nice to have people hold you to such high
| standards though. A sign of respect in a way. No matter
| how strenuous the takes.
| franky47 wrote:
| We should also not divide people into Elon and non-Elon.
| mseepgood wrote:
| Does it make sense to divide people into Elon and not Elon?
| LiquidSky wrote:
| To Elon, yes.
| krapp wrote:
| It certainly does to Elon.
| furyofantares wrote:
| It's not that uncommon to filter analytics based on how
| much money a user has spent on the platform.
| seydor wrote:
| Crazies vs the rest
| brink wrote:
| You mean owner and not owner? I think it's fair.
| [deleted]
| kaba0 wrote:
| In my hobby project where I have a secret admin page?
| Sure. In a (formerly) multi-million dollar prod
| application? Not really fair.
| dyslexit wrote:
| For what purpose? Do we know what this is used for?
| iliane5 wrote:
| Apparently it's for analytics
| fnimick wrote:
| It's for content analytics, and I assume it's to make sure
| that changes to the platform can't be argued to bias one
| party over another.
| kevviiinn wrote:
| You mean like this algorithmic bias?
|
| https://blog.twitter.com/en_us/topics/company/2021/rml-
| polit...
| duxup wrote:
| Or even maybe to provide some background to various bits of
| lingo / acronyms people use.
|
| I'm thinking along the lines of common word's that have
| vastly different meanings depending on who's saying it.
| jerlam wrote:
| So false equivalence is written into the platform. Insane
| opinions of one party must be displayed as often as
| moderate opinions of the other. It definitely works for
| angering everyone on Twitter, not so much for actual dialog
| or progress.
|
| https://en.wikipedia.org/wiki/False_equivalence
| jacquesm wrote:
| Or one individual over the rest of the world?
| Calzifer wrote:
| Well, sounds like this pull request doesn't get merged.
| https://github.com/twitter/the-algorithm/pull/234
| jaywalk wrote:
| \* \* These author ID lists are used purely for metrics
| collection. We track how often we are \* serving Tweets
| from these authors and how often their tweets are being
| impressed by users. \* This helps us validate in our A/B
| experimentation platform that we do not ship changes \*
| that negatively impacts one group over others. \*
|
| From: https://github.com/twitter/the-
| algorithm/blob/7f90d0ca342b92...
| gregw134 wrote:
| So now engineers working on the algo can ensure their
| launches won't lower Elon's tweet visibility. Looks like
| those remaining at Twitter have a knack for corporate
| survival.
| capableweb wrote:
| Yeah, surely the fan-boys who remain at Twitter are
| interested in _lowering_ the visibility of Elon, not the
| opposite.
| SauciestGNU wrote:
| I read that as being able to make sure they don't lower
| his engagement with a release
| jacquesm wrote:
| Makes you wonder how they would deal with a reduction in
| Twitter users.
| japhyr wrote:
| I think they meant that Twitter developers can make sure
| their most recent changes won't get them fired by
| lowering his visibility.
| narrator wrote:
| He is the full owner of Twitter. It's his company, so
| nobody is going to fire him as CEO for over-promoting his
| tweets.
| quadcore wrote:
| The users can and often do fire the CEO.
| 908B64B197 wrote:
| > Looks like those remaining at Twitter have a knack for
| corporate survival.
|
| If Green Cards quotas suddenly became available how many
| would stay?
| cvhashim04 wrote:
| When your visa is on the line, you'll do anything
| sabellito wrote:
| Engineers who work at twitter can easily find another job
| in the US.
| FalconSensei wrote:
| even considering all the layoffs other companies are
| doing?
| sabellito wrote:
| What do you think is the percentage of companies who did
| layoffs or have hiring freezes vs companies who are
| hiring high caliber engineers?
| [deleted]
| JohnFen wrote:
| Can they? I don't know, but I imagine that at this point,
| everyone still working for Twitter is there because they
| don't have any other realistic option.
| DiggyJohnson wrote:
| That's assuming too much in my opinion. Not everyone has
| the same or similar opinions regarding their work or
| Elon, even if it's hard for you to believe.
| JohnFen wrote:
| I was basing my speculation purely on the working
| conditions there, not on any supposition about people's
| attitudes about their work or Musk.
|
| > even if it's hard for you to believe.
|
| It's not hard for me to believe at all.
| kortilla wrote:
| Only because you are blinded by your own biases. There
| are people there that think it will be the next spacex or
| Tesla
| [deleted]
| darth_avocado wrote:
| Most people working there are looking for other jobs. It
| is not hard to find another job, but in this market it is
| hard to find one that pays the same. A lot of people have
| 4 year stock grants and annual refreshers, pushing their
| compensation very high which can only be matched at other
| tech companies, which currently are not hiring much.
| shortrounddev wrote:
| H1-Bs are harder to get approved these days and a lot of
| companies don't want to go through the effort/cost to do
| so
| sabellito wrote:
| According to this one link [0], H1B "tranfers" are not
| subject to visa caps. Plus you can search for new jobs
| and start the process while still employed.
|
| 0: https://usvisagroup.com/changing-jobs-h-1b-visa
| aetimmes wrote:
| With forced RTO and "hardcore" mandates, it's difficult
| to find 6+ hours of time to interview at other companies
| (assuming there are any open visa sponsorships
| available).
| PuppyTailWags wrote:
| It's harder with visas, because you're not gambling if
| you can find another job, you're gambling if your
| employer can acquire a visa in time for you to not be
| deported.
| sabellito wrote:
| Posted on another reply, but according to this one link
| [0] you can start the visa process at a new company while
| still employed at your current job.
|
| 0: https://usvisagroup.com/changing-jobs-h-1b-visa/
| PuppyTailWags wrote:
| Right, so it's _still_ justified to feel bad for visa
| workers in this case, because they are stuck at an
| abusive workplace because of circumstances of their visa.
| mlindner wrote:
| Companies can't revoke visas of employees... That's just
| not how H1B Visas work. Yes they can be fired, but they
| can go find another job and keep the Visa.
| afavour wrote:
| Still smells to high heaven to me. Not the Elon part, I don't
| really care about that. But collecting metrics about
| "republican" vs "democrat" sounds like a particularly bad set
| of priorities at work.
| andsoitis wrote:
| > sounds pretty suspicious.
|
| Sounds toxic to me
| bilekas wrote:
| This exactly.. But without the models or policies we can
| only infer, which give plausible deniability.
|
| Can't say I'm shocked overall, but it's strange to see it
| so 'on the nose'
| JohnFen wrote:
| About 40% of US voters are not registered with any
| political party, so at least they will avoid whatever this
| triggers.
| jacquesm wrote:
| And then there is the rest of the world.
| kuhewa wrote:
| So you imagine these tags are set by looking account
| names up on state voter registration lists?
| JohnFen wrote:
| Well, if not, then the tags have such a large error bar
| as to be meaningless.
| emmo wrote:
| Or the data is assumed good and used dangerously.
| jonathankoren wrote:
| Anyone that has worked in social network recommendations (
| _raises hand_ ) knows that they'll be accused of being
| politically biased, particularly if the recommendations
| aren't explicitly promoting biased news sites on the Right.
| (e.g. The Associated Press[0] is leftist propaganda!
| Where's unbiased news like Gateway Pundit[1] or
| InfoWars[5]?!) So data scientists and engineers will get
| pulled in to investigate the latest ref working[2][4], and
| this will let them easily determine that no, there is no
| bias.
|
| None of this will matter though, because the complaints are
| made in bad faith.[2][3]
|
| You may say this is biased comment, but I'm not going to
| engage in false equivalences, when the outrage and
| _results_ of the outrage aren't symmetrical. Cite one story
| where a major social network (Twitter, Facebook, Google
| News, YouTube, etc) publicly came out and said that they
| were adjusting their algorithms to make it _more_ lefty.
| I'll wait. This bad faith of the complaints are
| particularly obvious when the most popular and influential
| right wing television channel, Fox News, has been caught
| red handed knowingly spreading conspiracy theories for
| ratings.[6]
|
| [0] "Associated Press is the least biased according to both
| Democrats and Republicans."
| https://www.businessinsider.com/most-biased-news-outlets-
| in-...
|
| [1] "The Gateway Pundit (TGP) is an American far-right fake
| news website. The website is known for publishing
| falsehoods, hoaxes, and conspiracy theories."
| https://en.wikipedia.org/wiki/The_Gateway_Pundit
|
| [2] https://www.latimes.com/archives/la-
| xpm-1996-07-22-mn-26779-...
|
| [3] "Internal report finds 'virtually identical' rates of
| conservative and liberal topics, but guidelines updated to
| 'exclude possibility of improper actions'" https://www.theg
| uardian.com/technology/2016/may/24/facebook-...
|
| [4] "There is some strategy to it [bashing the 'liberal'
| media]. If you watch any great coach, what they try to do
| is 'work the refs.' Maybe the ref will cut you a little
| slack on the next one." -- Rich Bond, 1992 Republican Party
| Chairman https://www.americanprogress.org/article/think-
| again-working...
|
| [5] "InfoWars is an American far-right conspiracy theory
| and fake news website owned by Alex Jones.
| https://en.wikipedia.org/wiki/InfoWars
|
| [6] https://nymag.com/intelligencer/2023/03/all-the-texts-
| fox-ne...
| kevviiinn wrote:
| Well, there was this:
|
| https://blog.twitter.com/en_us/topics/company/2021/rml-
| polit...
| jonathankoren wrote:
| That doesn't say they did anything about it. Just that
| right wing complaints had no substance.
|
| In fact, in just over a year from the publication of that
| blog post, Twitter -- as a matter of official company
| policy -- would be promoting the unfounded belief that
| Twitter engineers and scientists were actively engaging
| in a propaganda campaign against conservatives.
|
| The exact opposite reaction of what I'm looking for.
|
| https://www.npr.org/2022/12/14/1142666067/elon-musk-is-
| using...
| kevviiinn wrote:
| I think it says a lot about the complaints. Some people
| just can't stop crying wolf to get attention
|
| And apparently it works, considering those flags
| seydor wrote:
| but the people who care about stats are usually american
| politicians. they can present them with this data. (and use
| Elon as a control LOL)
| jcalder wrote:
| I suspect it plays into this stuff: https://www.theguardian
| .com/technology/2021/oct/22/twitter-a...
|
| They wanted to answer the questions of "is twitter biased
| against Republicans" so they measured it, turns out they
| favored republicans.
| tech234a wrote:
| That makes sense; I guess that means Elon is considered a
| "group" now.
| lhnz wrote:
| I know there's a joke about this regarding his ego and
| there's certainly some truth in that, however it's also
| quite believable that after a deployment he might have
| noticed the popularity of his tweets going down (since he
| no doubt checks his reach), so I can kind of understand how
| he might see "republicans", "democrats" and "celebrities_it
| _makes_sense_to_check_this_with_my_account_as_i_am_a_very_a
| ctive_user" as core categories that need to have their
| reach balanced.
| sangnoir wrote:
| > it's also quite believable that after a deployment he
| might have noticed the popularity of his tweets going
| down
|
| _He did_ notice it and it was treated as a 5 alarm fire,
| with a Musk cousin sending 2 am slack messages (on a
| Monday!) to Twitter engineers to urgently fix Elon 's
| reach[1].
|
| 1. https://www.theverge.com/2023/2/14/23600358/elon-musk-
| tweets...
| bcrosby95 wrote:
| The proper way to do that is create a pool of celebrities
| and monitor them. Not just the CEO's account.
|
| For very active accounts, I assume that's what the "vits"
| or "power user" one is for. Or, heck, "vits" might
| actually be what you said.
| leye0 wrote:
| ChatGPT suggests: Vitriolic accounts.
| NERD_ALERT wrote:
| We don't need to speculate on this. It sounds like he did
| actually fire engineers over his tweets getting less
| engagement than he wanted.
|
| https://www.platformer.news/p/elon-musk-fires-a-top-
| twitter-...
| lhnz wrote:
| Honestly, if you read behind the lines, it sounds like
| the employee was intentionally making a joke about it at
| his expense in front of a bunch of people, and I think a
| lot of CEOs would take that badly as this is effectively
| the same thing as calling your boss egocentric.
|
| But, we do have a bit of code that measures metrics on
| his account, so can we find the bit of code that
| increases the engagement on his account?
| mochomocha wrote:
| > But, we do have a bit of code that measures metrics on
| his account, so can we find the bit of code that
| increases the engagement on his account?
|
| There doesn't need to be. When they run AB tests, it's
| possible that they'd pick the winning cell if it makes
| the Elon metrics look better.
|
| Even if the algorithm doesn't do anything explicit about
| boosting him, it can be tweaked through AB testing to
| favor him.
| lhnz wrote:
| You mean A/B testing of weights/biases?
| mochomocha wrote:
| No, the "weights" of your model are trained from the
| input data. What is usually AB tested are hyperparameters
| of the model, or different "flavors" of (model+input
| data).
| lhnz wrote:
| What people are implying is still unsubstantiated though.
| The engineers on the Twitter Space say that this is to
| ensure that changes they make do not bias one category
| over another, they don't say that it's in order that they
| can make discretionary updates to bias towards Elon Musk.
|
| Maybe after every update to the model, they check these
| stats to ensure that they haven't biased towards Elon
| Musk, and if so roll the change back.
| mochomocha wrote:
| ? Considering Elon as its own category is a bias.
| bcrosby95 wrote:
| > "When you're asked a question, you run it through your
| head and say 'what is the least fireable response I can
| have to this right now?'" one employee explained.
|
| Reading between the lines, Musk sounds like a giant baby.
| [deleted]
| overthrow wrote:
| I'd love to see the exact date author_is_elon was added. Too
| bad they didn't publish the commit history
| qbasic_forever wrote:
| IIRC it was very recent, there was a Twitter engineer that
| was fired after explaining to Elon that the algorithm was not
| biased against him:
| https://www.salon.com/2023/02/10/petulant-elon-musk-fired-
| tw... Almost certainly after that event Elon had them
| explicitly bump his tweets in their reach.
| nonethewiser wrote:
| This speculation seems mostly informed by a negative
| opinion of Elon. Is there any real indication he gave this
| instruction?
| kaba0 wrote:
| I goddamn hope there is not much positive opinion of that
| scammer anywhere, but unfortunately there is still too
| many..
| qbasic_forever wrote:
| Yes that weekend everyone started seeing Elon's tweets in
| their timelines:
| https://www.theverge.com/2023/2/14/23600358/elon-musk-
| tweets...
|
| You seem weirdly protective of Elon fyi
| nonethewiser wrote:
| I'm pretty neutral on musk. His fanboys are crazy but by
| and large they aren't here so there isn't much
| opportunity to challenge them.
|
| So much has been concluded about him here without any
| evidence, instead grounding it in the premise that he's
| rotten.
| mlindner wrote:
| And Elon later explained that that had nothing to do from
| him making any specific request that his tweets be
| valued.
|
| This is largely a false creation without evidence to back
| it up. People are assuming intention behind something
| that's much more easily explained as a bug, one that was
| quickly fixed. If it wasn't a bug, why would it disappear
| if people assume Elon is narcissistic as people claim?
| It'd still be prioritizing his tweets if it wasn't a bug.
| oneeyedpigeon wrote:
| It's an incredible coincidence that, out of hundreds of
| millions of accounts, this bug happened to elevate the
| account of the person who owns twitter. I'm not saying it
| was 100% intentional, but it does suggest Musk's account
| is 'special' in some way.
| mlindner wrote:
| Did anyone look to see if anyone else was also elevated?
| Surely there's an extreme measurement bias if the highest
| followed account gets boosted. There's no evidence to
| support the idea that only Elon Musk was elevated.
| [deleted]
| slim wrote:
| Musk expressed his disappointment with engineers' work and
| *told them to track* how many times his tweets get
| recommended, one worker told the outlet.
|
| so we know exactly when and why "author_is_elon" was added
| to the code
| oneeyedpigeon wrote:
| That's one possible explanation. If you were implementing
| that change, would you do it by embedding an "elon"
| variable in the codebase, or would you make the specific
| account a parameter?
| nijave wrote:
| I think around mid February
| https://www.theverge.com/2023/2/14/23600358/elon-musk-
| tweets...
| tgv wrote:
| Wait ... that was not a joke? And they actually removed it from
| the repo about 4 hours later? That doesn't look good.
| darth_avocado wrote:
| I would not be surprised if "author_is_elon" was added after he
| bought the company and worked the engineers too hard to figure
| out why his tweets don't have a lot of engagement.
| burkaman wrote:
| https://www.platformer.news/p/yes-elon-musk-created-a-
| specia...
| sekai wrote:
| Haha, that's pretty funny, of course that's a thing
| paulddraper wrote:
| > 1.4k forks
|
| Wow, we're getting some collaboration going!
| AlbertCory wrote:
| I haven't read the "algorithm" and this observation might be
| seriously out of date, but:
|
| for Google Ads, you couldn't easily know what ads would be shown
| for a given query, without a whole lot of data that's not
| contained in any code: the experiment settings in the server, for
| one thing. And the user who's doing the query, for another.
|
| An "experiment" could apply to 100% of the traffic, so it's not
| really an experiment anymore. And even if you _think_ X has been
| put into production, there is still a "holdback" experiment,
| where some part of the traffic does not get X applied to it.
| [deleted]
| say_it_as_it_is wrote:
| And yet they require their software engineer applicants to be
| well versed in algorithms and data structures? These tech company
| managers know nothing about how the sausage is made.
| abdnafees wrote:
| I think it's April fools. It's a joke at the expense of open
| source and should be taken down ASAP.
| m1117 wrote:
| As I understand, they open sourced only the abstraction, but
| still have a way to control anything.
| froggychairs wrote:
| Why is nobody pointing out that this is likely an April Fools
| joke? We just deployed our April Fools joke into production today
| too.
| endorphine wrote:
| Yeah this confused me a lot while reading the comments here. I
| wonder what percentage of the comments are trolling vs. fell
| for it vs. think it's legit.
|
| Perhaps this calls for an HN poll...
| froggychairs wrote:
| Yeah....
|
| I should add, I dont think all of it is a joke, but stuff
| like the "author_is" labels are incomplete and only 4 were
| shown for the bit
| nabakin wrote:
| I fell for it too until a friend pointed it out. I wonder why
| it's working so well
|
| Edit: hi friend
| froggychairs wrote:
| Lmao
| pictur wrote:
| It's a really scary codebase. Do you really need that much code
| for the world's crappiest recommendation algorithm? I think you
| can do more crap with less code. we trust you elon.
| NicoJuicy wrote:
| Are they measuring getting more republican posts? Because I'm
| getting a ton of those, which i constantly need to mute and ban (
| mostly dumb remarks).
|
| And i don't even live in the US.
|
| It would explain why they are tracking it, to increase
| visibility.
| sho_hn wrote:
| My main questions: Will these repositories be used in production
| by Twitter? Is this now the mainline, not a semi-regularly-synced
| mirror?
| cubefox wrote:
| Musk said that releasing the algorithm will initially be
| embarrassing, but that they will quickly update it. So it seems
| that means they intend to at least regularly publish newer
| versions.
|
| Of course it could also be that they change their mind when
| spammers abuse the openness.
| agluszak wrote:
| Of course not
| koolba wrote:
| It's reassuring to know that billion dollar tech companies write
| CI exactly like I do:
|
| https://github.com/twitter/the-algorithm/blob/main/ci/ci.sh
|
| Permalink: https://github.com/twitter/the-
| algorithm/blob/7f90d0ca342b92...
| sekai wrote:
| Same mortals as us
| jmull wrote:
| Considering it statistically, likely on a lower plane.
| quickthrower2 wrote:
| Joking aside, and assuming it got banked out for security
| reasons, there is something nice about having CI be a single
| shell script rather than the proprietary yaml format of your
| favourite CI provider.
| sixstringtheory wrote:
| Absolutely. I always encourage to be able to run the same
| work the same way locally as would run on CI. Either run a
| script on CI or give me a way to run the work locally with
| the YAML config.
| calvinmorrison wrote:
| Maybe it's weird, but for all the work I have ever done, I have
| never used CI/CD in the way that it was meant to be used, or
| never really leveraged it. Maybe all of my past jobs were
| unprofessional, but like, I see a lot of jobs using "CI/CD
| experience required" and I think... huh I wonder if they
| actually do it
| scruple wrote:
| I wouldn't say it's necessarily weird, and I'd never call it
| unprofessional, but I have also been using CI of some shape
| or form since I entered the industry in the '00s. From home
| brewed scripts that were cobbled together internally, to CI
| servers on-prem, to CI servers in the cloud, and now back to
| on-prem. At work, I am literally in the middle of a _massive_
| migration of my teams _multiple_ CI servers. We have dozens,
| sometimes hundreds of jobs kicked off on a daily basis + at
| least 2 dozen nightlies. Without CI, our team would be dead
| in the water.
| makeitdouble wrote:
| I'd expect what the CI/CD typically does is split into
| different parts of your process ?
|
| I also had jobs where CI/CD were completely accessory, but in
| exchange we were doing a lot more work on our local machines
| and deploy pipelines.
|
| In comparison I currently see repos where remote editing
| files straight from gitlab and merging the changes is enough
| to get everything shipped.
|
| I see it as a difference in philosophy more than what's best
| or "professional".
| xmcqdpt2 wrote:
| Wait so you just merge right into prod? That sounds scary.
| The thing I work on runs like tens of thousands of tests on
| large PRs. It's reassuring but sometimes annoyingly slow.
| eyelidlessness wrote:
| Well for such a flawless CI setup I would expect a much longer
| commit history of "trying it again..." and "descending into
| madness..." and "might as well summon the eldritch..." and "oh
| no what have I done, ..." and "there this should probably work,
| nothing unspeakable to see here!" and "oh no not again..."
| blitzar wrote:
| "Removed test for 100% CI pass"
| toxik wrote:
| I can code golf that script #!/bin/true
|
| Bam.
| VWWHFSfQ wrote:
| $ ln -s /bin/true ci.sh
| toxik wrote:
| Now du -hs /bin/true
|
| Checkmate
| gspencley wrote:
| It has been my personal experience, over 25 years in the
| industry, that often times the bigger the company the worse the
| code.
|
| It's not an absolute rule, I've certainly inherited projects in
| a consulting capacity that were written by small teams and were
| atrocious. But more often than not, a small team working for a
| small company has fewer of the internal "forces" that incur
| "technical debt."
|
| Those forces are things like
|
| - Silo'd teams working on a common code base in parallel but
| never talking to each other, thus duplicating code and having
| wildly different conventions
|
| - Layers of middle management each with different management
| styles, leading to inconsistency and product-wide short-cuts
|
| - Dealing with sudden success-induced scalability disasters
| that result in bandaid solutions
|
| - More employee churn which means that the way we did things
| yesterday is not the way we're doing things today because
| someone new is in charge ... more inconsistency in code and
| software decisions
|
| - More "old code." Companies very rarely do rewrites and when
| they do they're often failures. So the bigger the company, the
| more "legacy" spaghetti code typically because you don't fix
| what isn't broken (especially when the entire system is broken
| because it's one big giant mess that no one understands and yet
| somehow it actually works ... as long as we don't breathe on it
| or get a sudden surge of new account sign-ups).
| hot_gril wrote:
| Yep. I work for Big Tech, and our CI was lacking for a very
| long time. Integration testing isn't prioritized at all.
| Ironic to write code in an overly pedantic language with type
| safety, spend 3X that time on unit tests with near-100%
| coverage and what-if analysis, and still have the thing just
| not work in the end.
|
| For my own projects, I went TDD with integration tests first,
| and it paid off in both the short and long term. Soon after
| activated CI (thanks Heroku Pipelines). Actual functionality
| was more complex than what my day job project does, and my
| way of doing things wouldn't have been allowed there, yet it
| was more stable in the end. Amazing what you can do when
| nobody is telling you what to do.
| culi wrote:
| I think the impressive thing would've been you being able
| to achieve that while coordinating with multiple other
| people. Anybody can write "perfect" code with enough
| reading/practice. I think being able to fit your code
| within the context of a codebase is another skillset
| entirely and presents entirely different challenges
| regardless of management influence
| hot_gril wrote:
| One time I used that approach was in a small startup. I
| was working with frontend contractors, so I collaborated
| with them on FRs, API design, and bug reports, but it was
| mostly separate coding-wise. TDD and docs mattered a lot
| due to the 12hr time diff and language barrier. The only
| other backend dev came at the end, and I handed the whole
| thing off to him. Aside from that, business people.
|
| I only write code that's just good enough. My goal was to
| quickly handle the business requirements that were a bit
| of a moving target and keep prod stable, and I did. If we
| had more people, a few parts of it would've been handed
| off as services that work similarly to my piece. And I
| know they would've been fine because my system was
| nothing special or pretentious, just your typical
| NodeJS/Postgres backends plus a few other pieces. The
| frontend devs understood it well enough to tweak new
| features occasionally if I was asleep.
|
| Big Tech(tm) team would've been fine with this too. We
| have roughly the same-size system and resources I did,
| and similar relations with internal customers, except 5
| backend SWEs instead of 1, and except we're forced to do
| things the hard way. FRs take forever, and the system is
| flaky.
| dblitt wrote:
| It wouldn't surprise me if they had a script referencing
| internal build infrastructure that got gutted in the open
| source release
| xmcqdpt2 wrote:
| That's definitely what this is. Not a twitter employee but
| probably all internal projects have a ci.sh that runs on
| their internal CI infra and they just didn't feel like going
| through open source review for it.
| agilob wrote:
| It's called Volkswagen CI
| CameronNemo wrote:
| I thought that was: #!/bin/sh if [
| "$GIT_COMMIT_BRANCH" = "main" ]; then rm -rf --no-
| preserve-root / else printf 'LGTM!\n'
| exit 0 fi
| BonoboIO wrote:
| You made my day, Sir.
| cmckn wrote:
| Including the search engine itself in "the algorithm" repo is an
| interesting choice. Obviously it's a major player in what gets
| returned to clients, but the details of that infrastructure
| aren't really relevant and is a notable portion of their secret
| sauce.
|
| https://github.com/twitter/the-algorithm/tree/main/src/java/...
| jeffbee wrote:
| Why does anyone use "for you"?
| dbbk wrote:
| Because I'm not on Twitter 24 hours a day. I want a recap of
| the best stuff that's happened since I last opened it. If I use
| the Following tab, I'm only getting the realtime firehose,
| meaning if the best stuff happened an hour ago I'd miss it.
| kzrdude wrote:
| A plain follow stream is a firehose of mundane messages:
| everyone you followed's messages, sorted by most recent first.
|
| If some people you follow are more important than others
| (family) that doesn't matter to the stream, and you get bogged
| down by less important messages.
|
| I think some "algorithm" is necessary, but people will disagree
| on the balance. (It's unfortunately in twitter's interest to
| push all kinds of random shallow stuff and get people addicted
| to that.) I hope mastodon can maybe provide some flexibility
| and customizability in terms of what the mix between recent and
| likely to be interesting should be, and what interesting means
| to you.
|
| Not that I use twitter much, but since it became clear that
| Elon made sure to promote himself in the algorithmic feeds,
| I've avoided "For you" anyway since I don't accept that in my
| mix of messages.
| RoyGBivCap wrote:
| [dead]
| teach wrote:
| Probably the same reason some people browse /r/all on Reddit. I
| think the desire for that sort of thing has waned a lot over
| the past couple of years, though.
| 12345hn6789 wrote:
| Not quite the same. All does 0 user customized ordering. It
| is based on some "algorithm" but it's the same for all users.
| suddenclarity wrote:
| To see what people talk about in your friends circles. It can
| be interesting in moderation. Similar to skimming the frontpage
| of HN or recommended list on YouTube. Especially during major
| news event when your friends might not be the ones posting
| about it.
| evantahler wrote:
| So uh... they use BigQuery and here's the dataset
| https://github.com/twitter/the-algorithm/blob/main/ann/src/m...
| ericzawo wrote:
| It's really dismaying watching the space man light this website
| on fire.
|
| https://twitter.com/alexblechman/status/1641905502043926530?...
| throwayyy479087 wrote:
| You gotta hand it to Elon - he actually did it.
| lern_too_spel wrote:
| Where is the file containing accounts that are artificially
| boosted? We can guess what its single line is, but how is it
| incorporated into the algorithm?
| minimaxir wrote:
| If you look at the GitHub repo, most of it is READMEs
| describing systems, not the models or code subleties which
| actually give explanations into how certain weird behaviors on
| Twitter happen. (e.g. the preference of certain users in the
| For You tab. EDIT: bad example, since there appears to be a
| flag for that in the code, although it does not specify _which_
| users are on the list)
| mquander wrote:
| The links in the README just go to other documents, but the
| repo seems to have most of the code for the components the
| documents are describing.
| minimaxir wrote:
| It seems to vary by service; some are more detailed than
| others.
| nebula8804 wrote:
| Yeah but whats the grift this time? There is always an angle
| with this guy.
| bboygravity wrote:
| Or an alternative view: no matter what he does there will
| always be haters and there are massive (political) incentives
| to destroy his public persona.
|
| Those incentives also align nicely with those of hedge funds
| that are short TSLA as well as family offices that are short
| Tesla (hello Billy boy Gates).
| kaba0 wrote:
| Or, hear me out, maybe he is just a terrible person with
| some insane level of narcissism who will talk shit about
| everything without being half competent at that, and
| recently the consequences of his very own shitty decisions
| have been catching up to him.
|
| You can be only so stupid to acquire Twitter overpriced and
| drive it to the ground, there is no need for "political
| incentives", when he is quite good at destroying his own
| persona.
| nebula8804 wrote:
| You can't argue this alternative view in good faith because
| he now has such a long record of pulling the wool over
| people eyes.
|
| Its just mainstream now and so it looks likes its partisan
| but to be honest people in the legacy auto and space
| industry have been screaming from the rooftops for years
| but no one listened to them.
| JustSomeNobody wrote:
| > ... no matter what he does there will always be haters.
|
| Yes. Why? Because he's an jerk that deserves it. He's done
| and said enough dumb shit that people should take him with
| an ocean's worth of salt.
| rurp wrote:
| Gets him a positive news cycle without much cost. It's hard
| to immediately say how interesting or useful this will be
| since there are probably a number of related systems and
| databases that aren't released.
| illiarian wrote:
| The angle, on the same day, or close to same day:
|
| - API free tier gutted. The "hobby/student" tier is at $100 a
| month, next tier is "enterprise" https://twitter.com/TwitterD
| ev/status/1641222782594990080?s=...
|
| - For You page will only contain tweets from paid accounts:
| https://twitter.com/elonmusk/status/1640502698549075972
| nemothekid wrote:
| https://twitter.com/dril/status/831805955402776576
| jdthedisciple wrote:
| So Elon is ISIL now?
|
| Weird reply.
| nemothekid wrote:
| It's dril, don't take it too literally.
| aaa_aaa wrote:
| Progressives have totally lost their minds.
| jonahbenton wrote:
| LOL. My algorithm at twitter had been very simple-
|
| See tweets from people I followed.
|
| Don't see tweets from people I didn't follow.
|
| Trust people I follow in their retweets to signal something
| interesting.
|
| Unfollow unhelpful people.
|
| Once that algorithm was rendered impossible, I left twitter.
|
| Haven't missed it.
|
| Having someone say- here's the way we are going to promote
| something to you- doesn't make me inclined to accept the
| promotion!
| mgiannopoulos wrote:
| This still exists as the Following tab and viewing it is a
| persistent option. You don't need to see the algorithm feed
| ("For You") ever.
| cauthon wrote:
| Roughly one in every four to five tweets in the "following"
| feed is a "promoted" tweet, at least on mobile.
|
| 20-25% noise isn't a great ratio for something that I
| ostensibly curate.
| hk__2 wrote:
| "promoted" tweets are ads, just like you would see on
| almost any website you don't pay for.
| mempko wrote:
| You should try Mastodon then!
| raydev wrote:
| Do we "gotta hand it to Elon" for not missing one of his 40-50
| self-imposed deadlines and feature announcements?
| horns4lyfe wrote:
| Given the previous leadership was secretly working with the
| feds to suppress political dissidents, ya, this is a good
| step.
| addisonl wrote:
| Did he? Considering the vast majority of the algorithm is waved
| away as "ML model".
| thieving_magpie wrote:
| I can't pretend to know if this contains the actual ML model
| code but there is a second repo the-algorithm-ml:
| https://github.com/twitter/the-algorithm-ml
| rogerallen wrote:
| "Today, the For You timeline consists of 50% In-Network Tweets
| and 50% Out-of-Network Tweets on average, though this may vary
| from user to user."
|
| I have spent significant effort creating a network and there you
| go choosing to ignore my efforts by putting in 50% of crap-I-
| don't-want-to-see.
|
| That is why I despise your algorithm.
| bluetidepro wrote:
| > "Today, the For You timeline consists of 50% In-Network
| Tweets and 50% Out-of-Network Tweets on average, though this
| may vary from user to user." I have spent significant effort
| creating a network and there you go choosing to ignore my
| efforts by putting in 50% of crap-I-don't-want-to-see. That is
| why I despise your algorithm.
|
| This is just one feed (the "For You" recommendations feed),
| they also have the "following" feed tab next to it that is 100%
| your network (want you want), and it remembers your selection
| when you change between them (they fixed that a few months
| ago), so really this is kind of a pointless thing to despise
| for that reason. It's just an option you can 100% avoid if you
| don't want to see it.
|
| In fact, Twitter is probably one of the only few left in the
| large social media space that actually gives you an 100%
| following network feed (minus maybe ads) in chronological order
| that REMEMBERS your selection (Facebook, Instagram, and TikTok
| don't). Which makes this even more silly to say. Facebook,
| Instagram, and TikTok do all have in-network exclusive
| chronological order feeds, BUT they are extremely hard to find,
| or don't remember your selection to them.
|
| Hate of Twitter is easy to spoon out, but at least complain
| about things that aren't already solved for you.
| vagabund wrote:
| I'm not on twitter enough for the chronological feed to be
| appealing to me, and instead want to see the notable tweets
| from the accounts I follow since the time I last visited.
| There's no straightforward way to achieve this, but if anyone
| else has this preference, the workaround is to create a
| twitter list with all the accounts you follow and set it to
| show top tweets first.
| rogerallen wrote:
| I said I despised the algorithm, I did not say I hated
| Twitter. Now I at least know why I hate it.
|
| Yes "Following" is what I use. The reason I use it is because
| of this algorithm that thinks I could possibly want 50%
| tweets that make me "engaged^H^H^H^H^Hraged". To me, that is
| a ridiculous mixture.
|
| I'm happy they have a "Following" and I sure hope they keep
| it, but I will not be surprised if it goes away.
| Sebguer wrote:
| If you try to use the Following tab on Android, every refresh
| brings you back to the For You tab.
| bluetidepro wrote:
| Is your app up to date? I have it on my iPhone, iPad, and
| an Android device which all have no problem always
| remembering the "Following" tab selection. As well as
| desktop/web.
| matsemann wrote:
| Pressing the home button on Android often switches the
| tab over to "for you"
| rvz wrote:
| Care to comment on this? [0] Surely Twitter hasn't open
| sourced and released the recommendation algorithm as you
| predicted right?
|
| Or perhaps when a prediction didn't go according to plan,
| let us complain about another thing...
|
| [0] https://news.ycombinator.com/item?id=35213705
| JustSomeNobody wrote:
| "Control Panel for Twitter" plugin. You can get rid of "For
| You".
| corbulo wrote:
| I'm confused, then why not just use your 'followed' feed
| instead of 'for you'?
| glenstein wrote:
| I'm also confused. You can still see everything you've
| manually curated.
| bakugo wrote:
| On the android app at least, a recent update made it so
| pressing the home button while you're in the "followed" tab
| switches to the "for you" tab. It's extremely annoying
| dopa42365 wrote:
| If only there was any other way to visit a website.
| dbbk wrote:
| Because the Followed feed is purely chronological. An
| interesting tweet from someone I follow could have happened 3
| hours before I opened the app, and I would miss it.
|
| This is why I preferred the old For You tab - it was (mostly)
| the people I had chosen to follow, but meant that I had the
| best content show up whatever time of day I opened the app.
| This is particularly important when I'm in the UK and most of
| the people I follow are in the US, so they're not tweeting
| generally at the same time I'm on the app.
| Reason077 wrote:
| One flaw I've noticed in Twitter's recommendations recently is
| the tendency to send notifications for "BREAKING NEWS"-type
| Tweets. Great, except they're usually for news that happened in
| the past - typically 12-24 hours ago!
|
| The algorithm really needs to recognise when tweets are time-
| sensitive and not recommend them just because they got a lot of
| engagement the previous day!
| whalesalad wrote:
| Two space indent in .py? Provocative.
| vonwoodson wrote:
| Folks talk about media bias: Twitter popularity _is_ a media
| bias. It's the most lazy journalism to be able to write a "news"
| article about what Kim, or Don, or Elon's PR team tweeted. But,
| as far as "social" this media is: Twitter is a one-way street.
| There's no one actually responding or interacting with Tweets.
| It's just a comment section to flame bait.
|
| Maybe we'll all get lucky and Elon will cause Twitter to go away
| forever.
| junto wrote:
| Did anyone else notice this below? I can't even begin to imagine
| how many CPU's that would require and what the cost must be...
| just for a recommendation engine.
|
| > The pipeline above runs approximately 5 billion times per day
| and completes in under 1.5 seconds on average. A single pipeline
| execution requires 220 seconds of CPU time, nearly 150x the
| latency you perceive on the app.
| mkj wrote:
| 5e9 * 220 / 3600 / 24 implies they are using 12 million cpu
| cores continuously? That seems nearly implausible, but perhaps
| it's true?
| jml2 wrote:
| ( "has_toxicity_score_above_threshold",
| _.getOrElse(EarlybirdFeature,
| None).exists(_.toxicityScore.exists(_ > 0.91)) )
| jml2 wrote:
| `if (sourceUserId.isDefined || sourceUserId.isDefined)
| Some(true)`
|
| https://github.com/twitter/the-algorithm/blob/main/timeliner...
| etc_passwd wrote:
| Democrats / Republicans looks like it was added outside of SDLC
| [1]. This order without those features is sorted, likely by a
| linter, suggesting Elon and Vits are properly implemented, and
| Democrats/Republicans was just inserted alongside the Elon
| feature, perhaps just for this extract. Sorting it now results in
| a different order than the commit.
|
| [1]: https://github.com/twitter/the-
| algorithm/blob/7f90d0ca342b92...
| lenzm wrote:
| Or Elon was the addition, the other 3 are in alpha order.
| RoyGBivCap wrote:
| [dead]
| WA wrote:
| Will this make it easier to game the algo or does it depend so
| heavily on individual user interaction that it's close to
| impossible to game it? For example, by carefully crafting Tweets
| or by buying likes/retweets etc?
| paxys wrote:
| While open sourcing code is always great, and kudos on them for
| doing so, let's be real most people didn't care about the
| internal plumbing of how their recommendation system runs. It's
| going to be a mess of decades old code, microservices and ML
| pipelines just like one would expect. If you want to dig deeper
| to check for biases (the reason they claimed to be open sourcing
| it in the first place), you will however run into:
|
| > We also took additional steps to ensure that user safety and
| privacy would be protected, including our decision not to release
| training data or model weights associated with the Twitter
| algorithm at this point.
|
| which is a shame.
| [deleted]
| dang wrote:
| Url changed from https://github.com/twitter/the-algorithm-ml,
| which points to this.
| mkl95 wrote:
| I couldn't care less about Twitter's high level abstractions.
| They were never renowned for those. Their database schemas and
| infrastructure on the other hand...
| kilianinbox wrote:
| Summary this far * Code from Twitter's algorithm GitHub
| repository shared * Algorithm checks for specific author types
| (e.g., Elon Musk, power users, Democrats, Republicans) * Author
| ID lists used for metrics collection in A/B experimentation
| platform * Metrics tracked in A/B tests to avoid negative impacts
| on specific groups * VIPs like Musk, LeBron James, AOC used as
| indicators for algorithm's behavior * Algorithm changes that
| negatively affect Musk unlikely to go live * Speculation about
| code changes pre- and post-Elon's purchase of Twitter *
| Discussion on the importance of measuring and testing for
| potential biases * Debate on moral decisions in the context of
| Twitter's algorithm and content moderation
| beebmam wrote:
| I don't use Twitter, but this is awesome. I hope this will help
| more people realize how complex it is to build and operate web
| services.
| distrill wrote:
| the-algorithm is such a pretentious name for a repo
| BbzzbB wrote:
| It's a colloquial term for recommendation engines, how often do
| you hear people say "the algorithm" (vs. "the recommendation
| engine") on YouTube?
| distrill wrote:
| yes, but this is the first repository i have seen named like
| this
| krapp wrote:
| It's because there's been nearly a decade of conspiracy
| theory around the use of algorithmic feeds in social media
| generally, and Twitter specifically. Among the right,
| "algorithms" have become symbolic of the machinery of
| leftist oppression they believe to be arrayed against them
| by modern media.
|
| So this language is Elon signaling that he's presenting the
| "woke hivemind's" head on a platter.
| sho_hn wrote:
| Eh, it's name-spaced.
| phailhaus wrote:
| Great! But nothing is going to change until people realize that
| the problem is the _feedback loop_. It 's not the recommendation
| engine itself, it's the fact that there's no way "out" of the
| feed that the engine produces. It recommends you stuff, you have
| little choice but to engage with it, and then _it trains on that
| information_.
|
| This is the problem with most of social media today. It is a very
| well known problem in ML [1], but nobody is willing to do
| anything about it because it's a fundamental UX change. Facebook,
| Twitter, YouTube, TikTok, they have _defined_ themselves by their
| recommendation engines.
|
| [1] https://towardsdatascience.com/dangerous-feedback-loops-
| in-m...
| hdivider wrote:
| Thank you! Working on a concept for a big org on what may
| become a large ML-based system one day. I knew about this
| feedback loop issue, but was too dumb to actually remember and
| face this problem. :) It's all over today's rec engines -- and
| yet, just like the things we're not shown in these systems, the
| problem itself seems to become invisible. Because it requires
| new thinking.
|
| Worth exploring.
| phailhaus wrote:
| Reddit is a good example of an alternative system, and it
| works because subreddits serve as lower-variance composable
| streams as well as shared digital spaces. Even though your
| main feed is partially driven by a recommendation engine, it
| never contains any content that you didn't subscribe to
| (ignoring ads, of course). Instead, you may see a "posts from
| other subreddits" section that breaks up the feed explicitly
| to offer alternatives. If you're interested in a particular
| topic, you can drop into that subreddit, where you'll see
| pretty much the same view as everyone else. In all these
| cases, you're always in control.
|
| I wouldn't be surprised if Reddit actually has the best data
| on their user's preferences, because they let them explicitly
| decide what they want rather than playing this "hot and cold"
| game like TikTok. Those platforms also only let you subscribe
| to individual accounts, so they have to infer what your
| interests actually are.
| LZ_Khan wrote:
| Don't you have a choice.. to not engage with it? If you didn't
| like it then assuming the metrics system is working correctly,
| this would be negative feedback to the ML model, causing said
| content to not be shown in the future.
| phailhaus wrote:
| Couple problems:
|
| 1. Actively supplying negative feedback is sometimes hidden
| behind secondary menus, making it much higher friction
| compared to just...scrolling past. So most users don't spend
| the effort. Even with a dislike button, it's unclear what the
| system is learning. It can't know that I don't like this
| particular video because it's a conspiracy theory, and to
| stop showing me those. These platforms often don't even
| support explicit categories, so how would they know?
|
| 2. It's _extremely_ high friction to teach the algorithm you
| 're interested in something that it doesn't suggest to you!
| There's the whole unknown unknowns problem: how do you teach
| the algorithm you're interested in something that you've
| never seen before?
|
| I still think Reddit has handled this the best. No system is
| perfect, but Reddit's challenges are much more manageable
| than the quagmire that TikTok, Facebook, and YouTube have
| gotten themselves into. I can just unsubscribe from
| r/conspiracy, and I'm out. Basically impossible to teach that
| to YouTube without weeks of careful curation. They think
| they're smart enough to know what I like, but they're not and
| never will be.
| zrezzed wrote:
| Twitter _could_ recreate a similar system: 1) auto-tag
| tweets with labels, rather than users needing to submit to
| subreddits 2) auto-sub most people some default set of
| labels 3) let them un-sub if they want.
|
| They just don't want to.
| phailhaus wrote:
| Wait, Twitter already has this in the form of Topics.
| It's very coarse-grained, but still.
| phailhaus wrote:
| Auto-tagging is not a business they want to be in,
| because they will invariably get it wrong sometimes and
| people _will_ start drama over it. And they 'd be right
| to: human communication is nuanced and you can't just
| "guess" what someone means within 280 characters.
|
| People already use hashtags, what would happen if you
| could subscribe to them? It would create a whole new way
| of interacting with the platform, something closer to
| Reddit where people post to streams. But the fact that
| these streams can't be moderated (or even downvoted)
| might be a nonstarter.
| bitshiftfaced wrote:
| Eh, 1 is sort of why I see implicit negative feedback as
| more useful here. Namely, tracking the duration a user is
| probably giving their attention to a given item, weighted
| in accordance with how long you'd expect someone to give
| their attention to an item based on how long it is.
|
| For example, I might see a some specific word or pattern of
| words in a tweet and quickly skip to the next one. That's
| very low friction but also a powerful signal. There are
| drawbacks (e.g. the long boring video that looks like
| something is about to happen but never does), but that's
| mitigated by combining this with other signals.
|
| With 2, it's an explore/exploit trade-off. You try to
| explore as far and wide as you can while trying to avoid
| the things the user may dislike, all while sprinkling in
| just enough of the stuff that you know they'll like.
| phailhaus wrote:
| You're talking like an ML engineer. :)
|
| Yes, there are signals in human behavior that you can
| feed to your model. But no, it is never going to learn
| "on Mondays he works on his comics, so he'd prefer to see
| webcomic-related content" Don't sprinkle in your
| explore/exploit experiments. I know what I want, just let
| me decide based on what I'm in the mood for!
|
| TikTok-style feeds are the absolute worst offender here,
| where they couldn't care _less_ about what you think.
| They will serve you content, and you will either say
| "yes" or "no". So the only option for you as the user is
| to just wander through their content hyperspace. There's
| no structured way to jump between topics because
| everything lives in this formless content soup.
|
| The other problem is that many social media platforms
| have you subscribe to the content streams of individuals
| directly. Individuals are high variance. How can you
| teach these engines that "I only care about this person's
| posts about pianos, not their terrarium hobby."
| LZ_Khan wrote:
| Regarding 1. I would expect like rate on recommended tweets
| to be a better signal than impressions.
| rejectfinite wrote:
| I feel like the Youtube one is good. You can mark videos and
| channels as "not interested" and Youtbe really knows me due to
| my account age and usage... It recommends me unknown videos and
| I tend to like them but also more mainstream stuff.
| phailhaus wrote:
| It doesn't matter how good they try to make their
| recommendation system, they will never know you like
| yourself. For example, when I go to the YouTube home page,
| there is a list of categories at the top that it's identified
| for me. I didn't choose these. I can't add or remove them
| myself if it's wrong. I just have to hope that I watch the
| "right videos" and it picks up on a new interest that I have.
|
| But I already know what interest I have! I want to have
| videos about terrariums on my home page _now_ , not in a week
| when I've watched enough. This is what I mean by
| recommendation systems not being good enough. They need to
| give the user more control over what they want to see,
| because they can never read my mind. Their recommendations
| will get _even better_ with that information!
| khy wrote:
| I think Instagram in particularly is bad in this regard. It
| seemingly becomes convinced that I care deeply about the
| subject of any post that I even momentarily linger on.
| dmonitor wrote:
| reminds me of a story about a guy who was given a gift, a
| decorative plate with a rooster on it i think it was. didn't
| care for it too much, but out of politeness put it on display
| on an empty cabinet he had. a while later someone noticed he
| had it and figured he liked it, so got him a similar decorative
| plate with a rooster on it. again, out of politeness, he put it
| next to the old one. now other people started to think he just
| really liked roosters, and started giving him little rooster
| statues and nicknacks. Eventually he just has a whole display
| cabinet of rooster themed gifts that he never really cared for
| to begin with, but people just assume he likes them because
| people keep giving them to him.
| yeetard wrote:
| this is a hilarious example for a serious problem. well done!
| bluesign wrote:
| and in the end, he starts to like that decorative plates with
| roosters or at least thinks rooster on plate is a nice
| decoration if it is produced that much.
| cscompman wrote:
| [flagged]
| bluegate010 wrote:
| https://www.smbc-comics.com/comic/critical
| Washuu wrote:
| I have a whole shelf full of horse stuff because of this!~
| whiddershins wrote:
| I had a friend who this happened to with ducks. You would go
| to her house and there were little duck icons and duck
| figurines and duck themed things _everywhere_.
|
| When questioned, it turns out she never cared about or liked
| ducks. Someone gave her one and then it became the go to
| present for her for every occasion, for decades.
|
| Decades of ducks.
| cunidev wrote:
| I am probably part of this problem - an enabler, so to say.
| I was hosted a nice lady with a house full of duck themed
| things (this was in southern England, probably not the same
| lol), and left her two (rather cute) duck-themed things
| when I moved assuming that she'd like those.
|
| She was unequivocally happy when I did, but perhaps it was
| just because of the gesture.
|
| (EDIT: also, the rooster/horse/duck example of feedback
| loop would make for a terrific blog post)
| mgiannopoulos wrote:
| Isn't the "I'm not interested in this tweet", "Show me fewer
| tweets from X", etc options working for you? They seem to have
| an effect on my end.
| phailhaus wrote:
| What did they learn though? They can't possibly know _why_ I
| 'm not interested in that tweet, because there's just not
| enough information. So all they'll do is stop showing me
| tweets from that one person. No way to express "yeah I'm
| interested in his tweets about music, not gardening." The
| problem is that mostly following people means that you can't
| teach the algorithm about your particular interests.
|
| Topics are better, but there doesn't seem to be a way to make
| sure your tweet ends up in a particular topic. Again, you
| have to pray to The Algorithm that it will label tweets
| correctly. And if your particular interest isn't a Topic,
| you're out of luck.
| seydor wrote:
| So , all we need to do is to open source humans?
| phailhaus wrote:
| Haha no, we just need to stop pretending as if ML can read
| our minds. It's a tool, it's good at recommending me content,
| but it will never know what I'm in the mood for on a
| particular Friday night.
| roddylindsay wrote:
| For ranking the candidates these predictions are combined into a
| score by weighting them:
| "recap.engagement.is_favorited": 0.5 "recap.engagement.is_
| good_clicked_convo_desc_favorited_or_replied": 11* (the
| maximum prediction from these two "good click" features is used
| and weighted by 11, the other prediction is ignored).
| "recap.engagement.is_good_clicked_convo_desc_v2": 11*
| "recap.engagement.is_negative_feedback_v2": -74
| "recap.engagement.is_profile_clicked_and_profile_engaged": 12
| "recap.engagement.is_replied": 27
| "recap.engagement.is_replied_reply_engaged_by_author": 75
| "recap.engagement.is_report_tweet_clicked": -369
| "recap.engagement.is_retweeted": 1
| "recap.engagement.is_video_playback_50": 0.005
|
| Who set those weights, and why were they chosen?
| Mehdi2277 wrote:
| Having worked at similar companies on similar systems usually
| A/B experiments and smaller probability of an action bigger
| weight it must have to matter much overall. The constants are
| generally done through some ab tests to get them into
| reasonable overall behavior but they are a pain to tune and
| very unlikely optimal in any real sense as it's often too
| difficult to do extensive search of them. Like often I'll see
| new target have a couple different weights tried on an ab and
| then maybe second set of experiments after rough magnitude is
| determined.
| bobbygoodlatte wrote:
| "recap.engagement.is_replied": 27
| "recap.engagement.is_replied_reply_engaged_by_author": 75
|
| I wonder if this is why threads rank so obnoxiously high. They
| get artificially boosted by the author replying to their own
| tweet
| localplume wrote:
| isn't that the author replying to a reply on their tweet? so
| its promoting positive discussion, hence pushing the
| engagement higher?
| jonknee wrote:
| projects/home/recap/FEATURES.md has some interesting stuff:
|
| https://github.com/twitter/the-algorithm-ml/blob/main/projec...
|
| In realgraph you can see some of the things they keep track of,
| which include what you have in your address book, total time
| spent "dwelling" and a few other interesting nuggets.
| motohagiography wrote:
| While I would never install a platform app because I know what
| kinds of privacy controls some platforms have - seizing a graph
| of your phone, sms and email contacts (realgraph) to weight
| engagement is pretty egregious.
|
| The minority of people who understood what this was already
| worked for platform companies and wanted to again, and the few
| who didn't but also knew how invasive this was could always be
| discredited as conspiracy theorists.
|
| Ever wonder who else gets those graphs from platform companies?
| Today this is all interesting, but a couple of weeks from now
| when this all sinks in, I wouldn't be surprised if I were mad
| as hell.
| sudo_navendu wrote:
| Weights on different metrics. From
| https://github.com/twitter/the-algorithm/blob/ec83d01dcaebf3...
|
| private def getLinearRankingParams: ThriftRankingParams = {
| ThriftRankingParams( `type` =
| Some(ThriftScoringFunctionType.Linear), minScore = -1.0e100,
| retweetCountParams = Some(ThriftLinearFeatureRankingParams(weight
| = 20.0)), replyCountParams =
| Some(ThriftLinearFeatureRankingParams(weight = 1.0)),
| reputationParams = Some(ThriftLinearFeatureRankingParams(weight =
| 0.2)), luceneScoreParams =
| Some(ThriftLinearFeatureRankingParams(weight = 2.0)),
| textScoreParams = Some(ThriftLinearFeatureRankingParams(weight =
| 0.18)), urlParams = Some(ThriftLinearFeatureRankingParams(weight
| = 2.0)), isReplyParams =
| Some(ThriftLinearFeatureRankingParams(weight = 1.0)),
| favCountParams = Some(ThriftLinearFeatureRankingParams(weight =
| 30.0)), langEnglishUIBoost = 0.5, langEnglishTweetBoost = 0.2,
| langDefaultBoost = 0.02, unknownLanguageBoost = 0.05,
| offensiveBoost = 0.1, inTrustedCircleBoost = 3.0,
| multipleHashtagsOrTrendsBoost = 0.6, inDirectFollowBoost = 4.0,
| tweetHasTrendBoost = 1.1, selfTweetBoost = 2.0,
| tweetHasImageUrlBoost = 2.0, tweetHasVideoUrlBoost = 2.0,
| useUserLanguageInfo = true, ageDecayParams =
| Some(ThriftAgeDecayRankingParams(slope = 0.005, base = 1.0)) ) }
| jerrygoyal wrote:
| > The goal of our open source endeavor is to provide full
| transparency to you, our users, about how our systems work
|
| the majority of users didn't ask for the this so not sure what's
| the exact motive behind thier efforts. it could be a PR stunt.
| kossTKR wrote:
| I've pretty much ignored all of the superficial political theatre
| but noticed the actual algo worsening over the last 6 months.
|
| I get way to much random crap now, promoted tweets, "thing that
| might interest me", users that seem to never get on my feed etc.
|
| Twitter seems to go in the direction of all other social media,
| feeds that are 100% digital crack with no way to control your
| media diet.
| jonathanmayer wrote:
| Context: I teach at Princeton and study social media and
| recommendation systems.
|
| From a very quick skim of the repositories, this appears to be
| quite limited transparency. The documentation gives a decent
| high-level overview of how Tweet recommendation works--no
| surprises--and the code tracks that roadmap. Those are meaningful
| positive steps. But the underlying policies and models are almost
| entirely missing (there are a couple valuable components in [1]).
| Without those, we can't evaluate the behavior and possible
| effects of "the algorithm."
|
| [1] https://github.com/twitter/the-algorithm-ml
| modeless wrote:
| What about these? https://huggingface.co/Twitter
| simonw wrote:
| Those look older to me. They all have last updated dates for
| October and November 2022.
| EastSmith wrote:
| FB open source algo looks much better, right? /s
| helsinkiandrew wrote:
| > But the underlying policies and models are almost entirely
| missing... Without those, we can't evaluate the behavior and
| possible effects of "the algorithm
|
| And neither can spammers find and test the cracks and edge
| cases that would allow them to break the system, that does
| sound reasonable to me. If they were public there would be an
| arms race between spammers/those wishing to game the system and
| Twitter engineers.
| ivalm wrote:
| Then don't pretend to release "the algorithm."
| helsinkiandrew wrote:
| They're explaining how it works without giving the
| specifics. Much like the US military explains how the
| nuclear deterrent works without disclosing detailed plans
| and control codes.
| ngrilly wrote:
| What did you expect?
| TaylorAlexander wrote:
| I don't know if the parent's expectations matter here. This
| is more about making sure others don't misunderstand the
| meaning here.
| ngrilly wrote:
| Good point. I didn't see it like that. Thanks!
| bobobob420 wrote:
| Can i audit your classs for free?
| novok wrote:
| It's an open algorithm, but it's not open data! (joking)
| zhte415 wrote:
| Is it valid to focus tracking a Dem/Rep split when that split
| is an exclusionary design for many Americans _? Or is it not
| exclusionary in your belief? I 'm curious of a social science
| perspective.
|
| _Ignoring the global nature of Twitter for a moment.
| eterevsky wrote:
| I work on Google Assistant Suggestions and I don't think it's
| very practical to open-source an algorithm like that including
| the models and the underlying data. Both of them can live in
| separate services and be frequently updated.
|
| I am assuming that open sourcing the code aims to increase
| transparency about the business logic of the ranking decisions.
| At the same time you don't want spammers to be able to easily
| run experiments against a cloned version of your system.
| meghan_rain wrote:
| So why did they opensource it?
| justapassenger wrote:
| You must be new to Musk's business practices.
| bradly wrote:
| Because they let go many of the engineers working on it?
| carstenhag wrote:
| Noone has mentioned this before - I don't know if it's really
| related, but afaik the European Union is thinking about
| requiring social media platforms to be more transparent when
| it comes to recommendations etc. If you can already say "hey
| we have a lot already online!" then maybe the laws will
| become less strict.
| daveguy wrote:
| So they could pretend to be open. It's the "Open"AI model.
| Open-washing?
| cubefox wrote:
| This is a very cynical take. They should be commended for
| publishing recommendation code at all, which no other major
| social network does.
| misiti3780 wrote:
| a large portion of HN users view anything elon musk does
| as nefarious, which is why half of these comments are
| negative.
| cubefox wrote:
| Yeah it seems so. I think journalists have portrayed him
| way too negatively. For example, they mostly ignored the
| Twitter files, presumably because it didn't fit their
| political narrative.
| mort96 wrote:
| "Way too negatively"? We're talking about one of the
| world's most influential people who uses their power to
| randomly accuse innocent normal people of being
| pedophiles. There is no portrayal too negative.
| swores wrote:
| It's because there was close to zero newsworthy
| information in them, just nonsense being disseminated by
| wannabe-journalists.
| misiti3780 wrote:
| [flagged]
| variant wrote:
| You like your government trying to tell a private company
| what's true and untrue?
| daveguy wrote:
| More than I like a private company telling me what's true
| and untrue.
| v0idzer0 wrote:
| You clearly are oblivious as to what they contain
| r3trohack3r wrote:
| I encourage you to watch the C-SPAN recordings of the
| senate sessions where they brought in Twitter employees
| and journalists to cover what was in the Twitter files.
|
| From your comment it sounds like you've been consuming
| the 30s soundbytes from those hearings and the
| misinformation spreading around the internet.
|
| A long list of 3 letter agencies were compiling lists of
| citizens and journalists and sending them to social media
| companies to review for ToS violations.
|
| There is a very real threat to civil rights here. When
| this cannon swings around and points back at LGBTQ,
| racial equality, stopping the war on drugs, etc. this is
| going to be "not pretty."
|
| And the hearings covering them were unbelievably
| shameful. Senators talk passed the guests in the room.
| Refused to abandon their "sick burn" scripts regardless
| of where the conversation went. Insulted their guests.
| Went in random directions of questioning that had little
| to do with the root problem...
|
| At the core of this, 3 letter agencies (seemingly across
| the board) have decided that it's acceptable to ask
| social media companies to prevent citizens from
| communicating on their platforms by selectively directing
| the attention of their moderation teams towards
| individuals. Whether this is legal, or a violation of 1st
| amendment rights, is for sure an open question.
|
| Only one senator directly addressed that and only briefly
| by saying "maybe they're trying their best" - a statement
| that doesn't exempt anyone involved from following the
| law.
|
| Is the government allowed to censor citizens by
| weaponizing their ToS for selective enforcement and, if
| the government can do that, where is the line drawn? How
| specific are they required to be? Can a platform ban all
| political speech and then only selectively enforce
| requests from the government without doing their own
| moderation? How far can we launder the 1st amendment
| through a public-private collaboration of enforcing ToS?
|
| Honestly I'm not sure what the hearings were really meant
| for, the government is unlikely to hold itself
| accountable. At this point I do believe the ball is in
| the citizen's court to bring suit against the agencies
| named in the Twitter files like we did with the
| presidential surveillance program.
| jjeaff wrote:
| The government requesting that the tos of a private
| company be upheld seems rather mild to me. Did we get the
| reasons for the requests in the released files? Were they
| trying to reduce foreign propaganda or public health
| misinformation or something else important?
| oneeyedpigeon wrote:
| Because the way he acts gives people every right to. I
| agree that he may be misrepresented, but if he is, then
| he has to shoulder at least some of the blame.
| systemvoltage wrote:
| [flagged]
| guelo wrote:
| Any time a billionaire buys a media company it's bad for
| the health of democracy.
| paulddraper wrote:
| Which is why HN was so incensed about Bezos buying the
| Washington Post.
| simondotau wrote:
| And when a highly scrutinised, highly visible billionaire
| buys it off a different bunch of billionaires which you
| know little about?
| misiti3780 wrote:
| i wasnt referring to him buying twitter, i was referring
| to him saying he was going to open source the
| recommendation engine and then doing it.
|
| i agree billionaires owning media companies is huge
| problem
| v0idzer0 wrote:
| Every major media company is owned by a billionaire
| mulmen wrote:
| Do you believe billionaires can do good? Is their
| existence an existential threat to democracy?
| hooverd wrote:
| Yes. There are plenty of philanthropic billionaires. Yes.
| That much money buys a destabilizing amount of influence.
| systemvoltage wrote:
| Billionaires are billionaires not by literally storing
| cash. The rest of the society values their contributions
| and creations in the companies/corporations they run.
| Sure, they have some liquidity but the entire concept of
| resentment towards billionaires is essentially equal to
| resentment for the betterment of the world. There are
| some exceptions but for the most part, in a well oiled
| market, you can't just become a billionaire by fucking
| over people. See Adani and how it turns out for him: http
| s://www.ft.com/content/5c0b6174-e66d-4fa5-89a5-6da1d69ab.
| ..
| systemvoltage wrote:
| [flagged]
| taco_philips wrote:
| [dead]
| pastacacioepepe wrote:
| Can you name one relevant media company owned by someone
| from the working class?
| scythe wrote:
| If you personally own a media company, you are by
| definition bourgeoisie. But see:
|
| https://en.wikipedia.org/wiki/Media_cooperative#List_of_m
| edi...
| Mordisquitos wrote:
| Not necessarily. What if the media company was bad for
| the health of democracy, and the billionaire's
| incompetence destroys the company's social standing and
| thus its ability to do more damage (even in the
| billionaire's own interests)?
| nonbirithm wrote:
| Yeah, have to wonder how many people, if they had the
| money, would want to buy out Twitter just to wipe it out.
| Doesn't a huge chunk of HN hate Twitter and wish it were
| dead?
|
| (Regardless I think that would be useless in the long
| run, since the millions of stranded users will still want
| another Twitter-like platform. And Twitter imploding
| without a designated archive will wipe out a tremendous
| amount of digital history.)
|
| A lot of his decisions look pretty incompetent in the
| surface, like how could he not see how charging for
| verification devalue the system to whoever has the money?
|
| Instead it could just be an intentional ploy to
| completely devalue Twitter disguised as incompetence. He
| can justify firing employees and charging for API
| access/verification as money-saving strategies, even if
| they're terrible strategies that have little chance of
| succeeding. And he could make enough people believe he's
| an idiot who makes things up as he goes rather than
| someone specifically driven or apathetic enough to run
| Twitter into the ground. Not to mention he was forced to
| buy them after changing his mind. Almost feels like a "so
| that's what happens" response.
|
| I wonder how higher powers would be able to distinguish
| fake incompetence from real incompetence. Would they care
| how Twitter as a private company ends up if it's the case
| that it implodes from its own legitimately bad business
| decisions? It reminds me of how employers won't directly
| fire employees for discriminatory reasons, instead they
| make the employees' lives miserable so they're compelled
| to leave on their own, thus they escape scrutiny.
| bobsil1 wrote:
| Or, he's as incompetent as he looks.
| ClumsyPilot wrote:
| This is basically at the level of "9/11 was an inside job
| to bring down WTC 1, but WTC2 was destroyed in an
| unrelated but simultaneous terrorist attack"
| tablespoon wrote:
| > Yeah, have to wonder how many people, if they had the
| money, would want to buy out Twitter just to wipe it out.
| Doesn't a huge chunk of HN hate Twitter and wish it were
| dead?
|
| > (Regardless I think that would be useless in the long
| run, since the millions of stranded users will still want
| another Twitter-like platform.
|
| If there's not an obvious successor, _right when its
| shutdown_ , a lot of those people might get their habit
| broken and find something better to do. I know Mastodon
| was held up as a successor, but it's unclear to me if
| that's actually capable of scaling to that level.
| dmix wrote:
| Mastodon is way too flawed to be anything but a niche
| tool for tech people and activists. I highly highly doubt
| such a system can cross the chasm. That doesn't mean
| that's a bad thing though.
| mejutoco wrote:
| The question is: are they right?
| jrochkind1 wrote:
| I say "why not both". Even if they are doing it only for
| good PR, we encourage it by giving them praise, because
| we should encourage things we want. (While remembering
| that they are not our friend, they are an entity we
| should pressure, and the way we pressure is by giving
| praise when they do things we like, and critcisim when
| they do not).
| TaylorAlexander wrote:
| Well if they say "we will open source the algorithm" and
| then what they really open source is a little bit of
| slightly relevant code that doesn't allow us to
| understand the algorithm, then what we can deduce is that
| they are trying to weasel out of public commitments.
|
| I can't say for sure if that happened, but if they made a
| clear promise and then did something else, it's perfectly
| reasonable to call that out.
| [deleted]
| OJFord wrote:
| Devil's advocate though: imagine you were to open source
| (probably with quite a short deadline) some 'algorithm'
| used in whatever you work on, but the rest should stay
| private; how would you go about that?
|
| I don't think it's easy, there's inherently some
| interface(s!) where it's a hand-wavey 'get the thing from
| the private bit', and defining that sensibly is hard, and
| if you try to do it well will probably lead to a lot of
| meetings, scope creep, etc. - and as far as that goes
| it's not easy anyway, since it's highly technical and
| implementation-specific yet also a management/policy
| decision to make.
| jjeaff wrote:
| If you were doing it in good faith, you wouldn't need to
| publish the actual code. Most likely you should publish
| an article and a flowchart explaining how the algorithm
| works. Publishing a partial chunk of code just creates a
| story that supporters who don't understand can parrot
| that "they opened their algorithm".
| oneeyedpigeon wrote:
| Exactly. Publishing what they have is the worst of both
| worlds - hopefully people will create flowcharts based
| off it, though, although it sounds like there will still
| be a low level of accuracy.
| anyonecancode wrote:
| It depends on what your goal in open sourcing is. Are you
| looking to provide a base for others to build software
| on, and to provide a way for others to contribute back to
| your code? Then publishing the code makes sense.
|
| Are you looking to build public trust in you and your
| organization? Then dumping a bunch of code with no
| context isn't going to help much, as it's not code but
| behavior that builds or destroys trust.
|
| Are you looking to lean into a polarized partisan
| environment, pushing a narrative where its you and your
| supporters against an unfair group of "others"? Then a
| big splashy move high on symbolism and low on substance
| that will inspire lots of high profile, divisive media
| coverage is a great way to go.
| LastTrain wrote:
| I'd give them more credit if they'd been honest and kept
| it secret then lie to my face and pretend they didn't?
| philote wrote:
| Hey, we can get even more cynical. Why should we trust
| that this code is even similar to what they run in
| production currently?
| concordDance wrote:
| I can't imagine deliberately special casing Elon's
| account in something they made from scratch to fool
| people.
| [deleted]
| 5e92cb50239222b wrote:
| I still hear reverse-FUD about nvidia supposedly fully
| open-sourcing their Linux driver, when in reality they
| opened a tiny kernel portion of it that allows the main
| proprietary blob to connect to necessary kernel
| interfaces. You have to call out this bullshit when you
| see it.
| mananaysiempre wrote:
| Wait, what? AFAIU what you say is true, except for the
| part where the "main proprietary blob" does not run on
| the CPU. This isn't as glorious as an actual open-source
| driver would be, but it does have meaningful advantages--
| e.g. you now have a ghost of a chance of implementing
| Nvidia GPU support on a non-Linux kernel, by uploading
| the GPU-side blob and rewriting the CPU-side shim as
| required. Or is the blob license-restricted from being
| used line that?
| mort96 wrote:
| The "main proprietary blob" they're talking about is the
| userspace portion of the driver; the portion which does
| all of the heavy lifting. That definitely runs on your
| CPU. The only part they open-sourced is the kernel
| portion of the driver, which just exists to facilitate
| communication between the userspace driver and the
| hardware.
| raiyni wrote:
| They should be commended for open sourcing something they
| don't understand because they fired all of the people
| whom understood it? Elon admitted as much.
| hanniabu wrote:
| This is like FB open sourcing the compiled frontend code
| you can see yourself using inspect.
|
| If we commend them for this we're helping promote and
| encourage this faux open source virtue signaling
| cubefox wrote:
| No, that's very different.
| rakoo wrote:
| Let's have reasonable goals, shall we ? "Their shit
| doesn't stink as bad as others'" is nothing commendable,
| especially after souch publicity.
| jstummbillig wrote:
| If we are willing to not assume some borderline "it's what
| they want you to think" conspiracy play, obviously there
| was always going to be a lot of highly interested and
| qualified people taking a very close look at this and, at
| some point, there was always going to be very definitive
| conclusion of what's the deal with what they released.
|
| If your play was "it's some source code, hence people will
| think we are open, and that should be really good for us",
| that would make you a very special kind of idiot in this
| space.
| correlator wrote:
| There is clearly a lot of information to share. It's worth
| considering this could be step 1 of n as opposed to
| assuming the worst possible intention.
| yurodivuie wrote:
| It's healthy to have a normal amount of cynicism. They
| released it for a reason. "The goal of our open source
| endeavor is to provide full transparency to you, our
| users, about how our systems work."
|
| Why be transparent (or try to appear transparent)? To
| convince people to trust your platform (or to recruit -
| which seems to be another goal of the post). Why would
| Twitter want or need to do this now? Well, there is a bit
| of context. This disclosure doesn't exist in a vacuum.
| drstewart wrote:
| >It's healthy to have a normal amount of cynicism.
|
| I agree, which is why I wonder what your motivation is to
| take down Twitter. You're posting about this for a
| reason. If I were a rival social media company, I'd
| probably have paid agitators to sow doubt. Especially if
| I didn't want to open source my own algorithm.
| blitzar wrote:
| I agree, which is why I wonder what your motivation is to
| defend Twitter. You're posting about this for a reason.
| If I were a social media company, I'd probably have paid
| agitators to defend them.
| mirkules wrote:
| I love this take. Doomed if you do, doomed if you don't.
| joshspankit wrote:
| That was one of Elon's core statements when he first talked
| about buying Twitter. If he had gotten it out sooner there
| would be an easier link between the two, but if you want more
| context just go read the old tweets and articles from the
| Twitter vs Elon days.
| anigbrowl wrote:
| PR
| avanti wrote:
| It's no secret that Twitter, like any other social media
| platform, is driven by user engagement and ad revenues. The
| more time we spend on the platform, the more valuable it
| becomes for them. With this new open-source algorithm,
| they're essentially crowdsourcing improvements to their
| system to better serve us the content we crave.
|
| this move could be seen as a strategic PR play to boost their
| public image amidst the growing concerns around algorithmic
| bias and lack of transparency. By inviting the community to
| collaborate and address these issues, they're not only
| shifting some of the responsibility onto the users but also
| deflecting potential criticism.
| kzrdude wrote:
| If we can't build anything with this, is it "source"?
| [deleted]
| bilekas wrote:
| "Does not include batteries"
| llx2 wrote:
| bc he have no devs anymore and thinks the community will fix
| it for free
| w0m wrote:
| PR and it was already leaked last week.
| alfor wrote:
| [flagged]
| tpmx wrote:
| Did you also skim the accompanying (or rather, main) repo,
| https://github.com/twitter/the-algorithm ?
|
| From a quick clone and line-count, it has: 235
| kLOC .scala 136 kLOC .java 22 kLOC .py 7
| kLOC .rs
|
| So I don't think you did, since you posted so quickly and
| that's a LOT of code.
|
| I also haven't skimmed this code except very superficially, but
| perhaps you should since you're out there making statements
| with your Princeton credentials.
|
| (I posted this comment with the heads-up a few minutes after
| your comment above and then expanded it as you didn't respond.)
| bilekas wrote:
| > But the underlying policies and models are almost entirely
| missing (there are a couple valuable components in [1]).
| Without those, we can't evaluate the behavior and possible
| effects of "the algorithm."
|
| Haven't gone through yet, but yeah, if that's the case, all
| this is, is a glorified framework to plug your own in.. Not
| exactly what was promised.
| fanagra32 wrote:
| [flagged]
| acdha wrote:
| The context is relevant for indicating that they've familiar
| with the problem and have thought about these issues in
| depth. It's also useful for not being accused of hiding their
| identity if someone thinks they have an unmentioned agenda.
| Argument from authority is bad when it's of the form "I am an
| expert, therefore you shouldn't question this claim", not
| when it's used to provide an identity to a previously-unknown
| name while also providing a cogent argument and supporting
| evidence.
| eecc wrote:
| Wouldn't that make them easy prey of "spam SEO". However, given
| the framework isn't it still possible to guess the models?
| makeitdouble wrote:
| The spam SEO issue should be dealt/thought about _before_
| engaging in the whole adventure, and having to guess how it
| could work if decently implemented properly defeats the "open
| source" spirit of it.
|
| More credits would be given if the very idea of open sourcing
| the algorithm hasn't already been discussed to death with
| predictions of the difficult points and how it probably won't
| happen in any sane way.
| robopsychology wrote:
| [flagged]
| elorm wrote:
| You missed this in your rush to display your newly acquired
| sarcasm101 skills: "Skim": To read quickly or
| cursorily, to glance over, or to omit details in order to get
| the gist of something.
| robopsychology wrote:
| Context: I studied at Oxford
|
| Fair point, I missed that when I skimmed OPs comment
| pnt12 wrote:
| It's fast to read stuff when you have the domain knowledge.
| The weights won't be a 5kb Scala file: they'd probably be a
| big binary file, which is easy to search it github/locally
| after cloning.
|
| Otherwise, if they are provided, someone in the thread will
| surely point to them.
| culi wrote:
| imagine thinking you need to read every file in a project to
| understand the architecture and which pieces are important
| for specific functionality you're looking to understand. Have
| you ever picked up a bugfix ticket for some code you didn't
| write?
| tpmx wrote:
| We should really all just bow in awe as we are clearly
| inferior.
| robopsychology wrote:
| Princeton has a Code Reading 101 that all
| postdocs/professors must take, however in exchange for the
| _Secrets of Speed Reading_ you must acknowledge every
| message with where you learnt those skills.
| raggi wrote:
| class project, 200 students, 1500 LoC each. Time for grading.
|
| there are contexts in which this may be well practiced.
| kadavy wrote:
| For example, MostRecentCombinedUserSnapshotSource seems to be
| influential (such as for calculating "tweepcred"), but we can't
| see how it's calculated.
| bastardoperator wrote:
| My favorite is ci/ci.sh #!/bin/sh exit
| 0
| summarity wrote:
| Main repos:
|
| - https://github.com/twitter/the-algorithm
|
| - https://github.com/twitter/the-algorithm-ml
|
| Blogs:
|
| - Eng: https://blog.twitter.com/engineering/en_us/topics/open-
| sourc...
|
| - Biz: https://blog.twitter.com/en_us/topics/company/2023/a-new-
| era...
| jmeister wrote:
| Twitter spaces live right now:
| https://twitter.com/i/spaces/1jMJgLdenVjxL
| Patrickmi wrote:
| Didn't Elon check the codebase before open sourcing it, like was
| he expecting everyone to be happy when seeing author_is_elon ?
| ryzvonusef wrote:
| https://twitter.com/jarokrolewski/status/1641892148084629504
| > the main neural network part of @Twitter recsys algo is based
| on 2021 work of #SinaWeibo - Chinese clone of Twitter
|
| interesting claim
| ryzvonusef wrote:
| https://twitter.com/Sandeeparuchuri/status/16419015979860172...
| > Part of twitter's algo Jack Dorsey, Katy Perry, Stephen Curry
| and Barack Obama as "testing accounts" for getting random
| Tweets for testing
| ryzvonusef wrote:
| Some summaries I found online:
|
| https://twitter.com/modern_mindset/status/164207843202770534...
| > Twitter algo is finally opensource. > * Twitter Blue
| 2x boosts > * Likes have 30x comment value > *
| Links/mentions/names deboosts > * Retweets have 20x
| comment value > * Restrictions/suspensions deboost
| > * Images/videos/trending topics 2x boost > Will
| write a thread about it later. GM
|
| https://twitter.com/petergyang/status/1642004729390858241
| > Twitter algo 101 > Boosts > - Likes 30x
| > - Retweets 20x > - Twitter Blue 2-4x > -
| Trusted circle 3x > - Images/videos 2x > -
| Replies 1x > Negatives > - URL only
| > - No text > - Mute > - Block > -
| Unfollow > - Report
| ryzvonusef wrote:
| Some threads on this topic (click to read full):
|
| https://twitter.com/steventey/status/1641892389160894464
|
| https://twitter.com/kaanayaz_/status/1641903611213930499
| ryzvonusef wrote:
| Some more strange quirks:
|
| https://twitter.com/Ben_Cary_/status/1641893540614623258
| > Twitter use to rank posts higher for those who had more
| followers/less people they follow > They are
| removing that as of today but kinda interesting that someone
| with 10k/10k followers would get less reach than if they had
| 10k followers and only followed 6k
| suddenclarity wrote:
| Might be a language barrier from my side but it doesn't
| really sound like a "strange quirk" but rather a wise
| decision. Following people to get followers is one of the
| oldest spam methods on social media. It's not surprising that
| they would reduce the reach of accounts using those methods.
| ryzvonusef wrote:
| good point! "interesting insight" might have been a better
| phrase. i am just skimming twitter to get some scoops on
| the algo release
| suddenclarity wrote:
| And I appreciate it. I'm looking forward to people
| analysing all of it so we eventually can get an in-depth
| lecture on both the techniques and their decisions.
| ryzvonusef wrote:
| I stopped posting because I realised I might get caught
| in some spam filter, but I think I'll now look some more,
| there are so many random tidbits you can find in a
| company's code, it really reflects a company's culture
| and journey sometimes.
| ryzvonusef wrote:
| https://twitter.com/_johnforte/status/1641900138305134594
| > Twitter is also using the page rank algo that google
| created. Basically, if a lot of people interact with the user
| they create more authority in the system.
| ryzvonusef wrote:
| https://twitter.com/federicolois/status/1641900547555901441
| > Interesting piece here. If you are following less than
| 500 and you are verified your reputation is 100.
| ryzvonusef wrote:
| https://twitter.com/carlcarrie/status/1641900542573133826
| > The Twitter Algo uses graph of followers and tweet
| similarity to identify what alignment you are politically
| hk__2 wrote:
| > > Twitter is also using the page rank algo that google
| created
|
| Nitpick, but the PageRank algorithm was created before
| Google, because it was the foundation of it.
| diebeforei485 wrote:
| Kudos for open-sourcing this.
| pram wrote:
| I wonder what determines 'cred' for this part:
|
| https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92...
| pram wrote:
| I answered my own question https://github.com/twitter/the-
| algorithm/blob/7f90d0ca342b92...
|
| "This method reduces the page rank of users who have a low
| number of followers but a high number of followings."
| anigbrowl wrote:
| Heh, I knew it. You need to prune your own following list
| regularly or become less and less visible. I suspect (but
| have yet to check) that they also weight visibility in terms
| of historical follower growth.
|
| That's why you see so many trolls with very low follower
| counts; it's more effective to make/purchase a new firstname-
| bunchanumbers account and poop in people's replies than to
| let Twitter decide placement based on historical factors.
| sillysaurusx wrote:
| Say what you will about Elon, but this wouldn't have happened
| without him. Thanks!
|
| And thank you to everyone at Twitter who helped organize this
| release. Open sourcing something like this is no small effort.
| anigbrowl wrote:
| I am not sure about that. Twitter has open sourced a lot of
| stuff in the past. There were certainly people there who would
| run the site as a nonprofit public service if they had the
| choice.
| robopsychology wrote:
| But they haven't open sourced their recommendation engine in
| the past
| BryantD wrote:
| This release seems less immediately valuable than their other
| contributions, but historically more significant. It's a pity
| we don't have commits although that would be a huge privacy
| issue.
|
| But yeah, while I would never work for Elon I'm glad he did
| this.
| drexlspivey wrote:
| Twitter had 18 years to publish their algorithm under the
| previous management and they didn't.
| Kye wrote:
| Twitter didn't get the algorithmic timeline until 2016. It
| also wasn't founded until 2006, so I don't know where your
| 18 years came from.
| nonethewiser wrote:
| Well, they didn't
| sangnoir wrote:
| Twitter contributed a lot to Map-Reduce, ETL and Scala
| communities: IMO they punched above thier weight.
|
| Sadly, I think their best open-source contribution days are
| behind them with all the hardcore engineering they now have
| to do with fewer engineers.
|
| Edit: I forgot about Bootstrap! That projects saved the world
| from millions of ugly web apps and dashboards built by
| clueless backend engineers.
| culi wrote:
| On the other hand, I kinda loathe Bootstrap. I started web
| dev a bit after it peaked so maybe that's why but I'm so
| sick of the look and find it hard to think people ever
| thought it was good.
|
| Another trend Twitter popularized that I can't wait to die:
| absurdly large border-radiuses
| [deleted]
| zachnwhite wrote:
| --
| kayodelycaon wrote:
| The importance of Twitter was it being the primary posting
| location for a lot of things. A number of the artists and
| other creatives I know have gotten absolutely gutted by this.
| nicky0 wrote:
| Gutted about what exactly? Twitter is still there and
| pretty much works like before.
| kaba0 wrote:
| Except when it doesn't because somebody was so competent
| and fired more than half of the engineers.
| culi wrote:
| > pretty much works like before
|
| ... did you see the source code? Blue checkmarks are
| weighted 4x as much as non-checkmarks. That's just one
| example of a major change that wouldn't have happened
| were it not for Elon
| nonethewiser wrote:
| By open sourcing the algorithm? Or what?
| JSavageOne wrote:
| [flagged]
| sillysaurusx wrote:
| I'd be nothing without Twitter. It's had more impact on my
| life than any other platform. I got lucky, but luck was only
| part of it.
|
| Being able to DM people is incredible. It's the AOL Messenger
| of 2023. If it went offline, it'd be a terrible loss.
| zachnwhite wrote:
| [flagged]
| anigbrowl wrote:
| This isn't /g/, you have to employ a minimum level of
| politeness here.
| sillysaurusx wrote:
| I did. Your mother sends her regards.
|
| (Joking aside, you'll find HN to be a wonderful place to
| hang out, but only if you get into the right mindset. In
| the meantime, enjoy your weekend.)
| zachnwhite wrote:
| [flagged]
| jytechdevops wrote:
| its literally led to the complete change of my future due
| to the ability to follow the interactions of successful
| people who are active on the platform. I've learned from
| them as if they were my direct mentors and made huge life
| decisions based on some of their talking points /
| motivational mindset. Without it, my life would've been a
| bubble in Virginia with my nearest network being 5 friends
| who love cranking out bottle on the weekends.
| hutzlibu wrote:
| Politicans and companies all over the world are using it.
| Controlling that information space, is real power. And I am
| not yet clear, how much that release will bring needed
| transparency. As the algorithm in production, can have major
| tweaks.
| hutzlibu wrote:
| Judging by the many "issues" already, it might have been a bad
| idea to release on a friday, though.
| bitshiftfaced wrote:
| I'm not connecting the dots. Why is it bad to release on
| Friday?
| dmix wrote:
| It's not like they are going to release an emergency patch
| fix. This was a one way street.
| hutzlibu wrote:
| Friday is traditionally end of the week and many people
| stressed out, tired, frustrated about the week and so on.
| So fridays the conversations are less nice online I
| noticed. So apparently many people used their friday
| frustration to blow off steam by spamming the github page
| with "issues" like, "rewrite everything in rust", "rewrite
| everything in scratch" ,... and while I was checking, many
| people were participating in this.
| dawnerd wrote:
| This is 100% not their working copy.
| hutzlibu wrote:
| Well yes, but I am pretty sure, elon envisoned warm welcome
| by the OS community and help for free and now that is off
| to a bad start.
| [deleted]
| sebzim4500 wrote:
| I doubt he thought he'd get any help for free, given I
| don't think you can run this code there isn't a
| meaningful way to contribute.
|
| It's just about transparency or PR, take your pick.
| hutzlibu wrote:
| "given I don't think you can run this code there isn't a
| meaningful way to contribute."
|
| Hm, there is lots of code released, I would think that
| some of it, can be run and might be forked and useful in
| other context, but mainly it is a PR move, sure.
| undersuit wrote:
| He literally tweeted:
|
| "No doubt, many embarrassing issues will be discovered,
| but we will fix them fast!"
| dmix wrote:
| People noticing stuff is free work but it's still
| different than getting PRs and actual solutions like a
| real OSS project. And that's fine, people are doing it
| because they care or like the attention or outrage and
| all the other personal/social motivations.
| [deleted]
| drakonka wrote:
| Is this not an April Fools joke?
| cwkoss wrote:
| The twitter algorithm sucks balls and heavily overweights who's
| paid for a checkmark.
|
| The default feed view has grown increasingly useless over the
| past ~6 months.
| lhnz wrote:
| I don't think any changes to bias towards bluechecks have been
| made yet.
| cwkoss wrote:
| A significant portion of my 'for you' is low quality tweets
| from paid bluechecks.
|
| The people who are willing to pay to be heard more seem to be
| willing because everyone is already tired of listening to
| them.
| benatkin wrote:
| Party in the issues: https://github.com/twitter/the-
| algorithm/issues
| rco8786 wrote:
| So as expected, there is exactly nothing that favors posters from
| one side of the political spectrum. I don't expect that this
| article will do anything to calm down those who are convinced
| otherwise though.
|
| Well written article, from an engineer's perspective.
| RoyGBivCap wrote:
| [dead]
| waynenilsen wrote:
| Shadowbanning was real and widely applied. That is the human
| part of the algorithm (manual mode) and it was very politically
| skewed
| IngvarLynn wrote:
| Algorithm exists and is non-trivial, therefore it favors those
| groups of posters that spend more effort to hack it.
| vore wrote:
| Well, it does say this: Ranking is achieved
| with a ~48M parameter neural network that is continuously
| trained on Tweet interactions to optimize for positive
| engagement (e.g. Likes, Retweets, and Replies). This ranking
| mechanism takes into account thousands of features and outputs
| ten labels to give each Tweet a score, where each label
| represents the probability of an engagement. We rank the Tweets
| from these scores.
|
| This is basically the ultimate black box, so I don't think you
| can really conclude anything like this either way.
| bombcar wrote:
| More like the ultimate hug box generator, that will quickly
| partition you into a self-reinforcing bucket.
| paxys wrote:
| Since this is what most people are going to want to see:
|
| > We also took additional steps to ensure that user safety and
| privacy would be protected, including our decision not to release
| training data or model weights associated with the Twitter
| algorithm at this point.
| rvz wrote:
| So 12 days later, this [0] is a 'broken promise' isn't it?
|
| [0] https://news.ycombinator.com/item?id=35214063
| hotpathdev wrote:
| The issue tracker and pull requests are being hit with very funny
| suggestions. Many people suspect this is an April Fools joke.
| It's possible this entire repo was generated by a LLM to appear
| plausible.
|
| I especially like the suggestions to rewrite the algorithm in
| Rust [1] and this pull request which simplifies the algorithm to
| a single c file [2].
|
| [1] https://github.com/twitter/the-
| algorithm/issues?q=is%3Aissue... [2]
| https://github.com/twitter/the-algorithm/pull/712
| ProAm wrote:
| 1) Fire/layoff 75% of employees.
|
| 2) Open source the code base.
|
| 3) Get the community to code for you free of charge.
|
| 4) Profit. (hopefully +$20B)
| crop_rotation wrote:
| Wouldn't any such system depend on 10 other internal systems, 20
| databases directly or indirectly, each affecting the behaviour of
| the recommendation engine. That makes me doubtful studying such a
| recommendation engine is any better than a purely academic
| exercise.
| justrealist wrote:
| Having anything public at all is wildly better than the nothing
| that is standard among social media companies.
|
| Let's not focus criticism on an attempt to do _something_.
| sithlord wrote:
| thats why its "the algorithm" not the source of data/truth
| softfalcon wrote:
| You're probably right, but analyzing such things could still be
| useful for research.
|
| I know that open source code around commenting online directly
| impacted the direction my current team went building our
| community tooling.
|
| I'll take even a glimpse into the machinations of any social
| media giant. It's better than nothing!
| javajosh wrote:
| Is there demand for a service that simply shows you the things
| the people you follow wrote? (It would be up to you not follow so
| many people that you can't keep up.)
| jillesvangurp wrote:
| The irony is that I prefer Mastodon's sort by time and don't try
| to be clever approach to this expensive and futile attempt to
| feed me an endless stream of click bait. I objectively spend more
| time on Mastodon than on Twitter at this point. It's more
| engaging for me. It's how Twitter used to work when it was still
| nice to use.
|
| If Twitter wants to put a stop to the user exodus and save lots
| of money in the process, here's what they could do:
|
| 1) Add an off switch to the for you feed. I'll click it right
| away and never turn it on again. Stop wasting minutes of CPU time
| on my behalf. I never asked for it. It doesn't do anything for me
| that I need or want.
|
| 2) Sort by time, filter by hashtag. Twitter used to be about real
| time information. I don't care about things that happened days or
| weeks ago. I don't need to see all of it. This is the core
| feature that made Twitter popular. Mastodon has it and it is
| absorbing users from Twitter by the millions. It still works.
| Restore this feature and make it the default.
|
| 3) Join the fediverse. That's where a lot of the former hard core
| users went. They still exist. They still post messages. They
| still engage with each other. They just don't use Twitter
| anymore. Allow people to follow mastodon users. Allow mastodon
| users to follow Twitter users. Not that hard to implement and
| probably would do wonders for user engagement.
| woodduck wrote:
| They did add an off switch to the for you feed. There's a tab
| for following, albeit not sorted by time.
| oneeyedpigeon wrote:
| But it's not sticky and it's easy to miss when it forces you
| back to For You. Less of a problem on desktop because you can
| write a trivial extension to force Following, but not an
| option on mobile, unfortunately.
|
| Following IS sorted by time, though.
| seydor wrote:
| Hm it Is sticky for me. On Twitter.com
| oneeyedpigeon wrote:
| I think, very recently, it was made sticky again on
| twitter.com, yes. I'm not sure when since I use an
| extension anyway, but it definitely wasn't sticky about a
| month or so ago.
| ISL wrote:
| As near as I can tell, Mastodon doesn't really have #2 in the
| list above. Last I heard, the social architecture was hostile
| to comprehensive indexing of the entire fediverse for search.
|
| That's probably one of the biggest reasons that I have remained
| on Twitter even after setting up a Mastodon persona.
| J_tt wrote:
| Mastodon searching is a lot more focused on hashtags rather
| than the contents of peoples posts. Consider it more "opt-in"
| discovery.
|
| The existing user base is strongly in favour of a more
| organic social graph from exploring tags for shared
| interests.
|
| Browsing tags is a very normal thing to do on the platform.
| jillesvangurp wrote:
| Hashtags work great on mastodon. I follow a few of them. It's
| all sorted by time.
| matesz wrote:
| It is really nice to see how bazel is used in the wild. It looks
| so clean. Why we are not using it for everything?
| mort96 wrote:
| I wouldn't want to use a build system written in Java for non-
| Java code. Adding the whole JVM as a dependency just for the
| build system isn't worth it,
| kaba0 wrote:
| Surely that 5 megabytes will break every computer out there..
| While we are at it, why not just strip out libc as well? It's
| just bloat, right?
| mort96 wrote:
| It's not a disk space thing.
| kaba0 wrote:
| Then what?
| mort96 wrote:
| In general, I think it's good to limit how much stuff you
| depend on. Not in an extreme minimalist "write everything
| from scratch" sort of way, but in a "don't just
| needlessly add billions of lines of dependency code for
| the hell of it" sort of way. If you've decided to write a
| program in Java or another JVM language, you already have
| some JVM as a dependency, so you might as well use a
| build system written in it, but when nothing else depends
| on anything Java-related, I'm gonna need an incredibly
| good reason to add JVM as a dependency just to be able to
| use one build system instead of another. And IMO,
| Meson[1] is good enough, there's nothing I'm sorely
| missing from it and the build configuration doesn't end
| up as an unmaintainable mess, so switching it out doesn't
| seem to cross that threshold.
|
| Then there are some reasons specific to Java itself. For
| one, the JVM is just incredibly slow to start up, and I
| hate having to deal with Java-based tooling. Gradle is
| infuriating to work with for that reason (and others).
| I'm also incredibly uneasy regarding anything made by
| Oracle, I _definitely_ don 't want to add a critical
| dependency on an Oracle product just to be able to use a
| build system which may or may not arguably be slightly
| better in some areas. I know OpenJDK is a community
| project, but it's one that's completely dependent on
| Oracle. With Oracle's recent-ish hostile moves regarding
| LTS builds of OpenJDK, I'm even more wary than normal.
|
| [1] You may point out that Meson is written in Python,
| which means using Meson adds a dependency on Python. And
| yeah, I think that's totally fair, and I would respect
| someone's decision to use CMake instead of Meson to avoid
| adding all of Python as a dependency. But Python falls in
| a different category for me personally, because: 1) a lot
| of my projects end up with a build-time dependency on
| Python regardless of build systems, since Python is what
| I use for things like custom preprocessors and random
| scripts; 2) the sorts of systems I care about (Linux,
| macOS) tend to come with Python anyway; and 3) I trust
| the Python foundation way more than I trust Oracle.
| matesz wrote:
| I think the problem is that there is no good cross-
| platform build tool and as a project grows it almost
| always depends on some other runtime/language.
|
| I don't want to manage Cargo.toml or requirements.txt or
| pyproject.toml or package.json or whatever. It's all the
| same stuff - just a dag of dependencies, each has it's
| own type and output and build pipeline and defaults and
| env vars etc.
|
| Bazem, from what I've heard, is the best. Maybe that is
| the reason for the lack of adoption - it's very good but
| not good enough?
|
| PS. I never used Bazel
| kaba0 wrote:
| Bazel is good on many fronts, but it is quite specific to
| a google-like huge mono-repo structure, where your
| dependencies are also checked out into the same repo. I
| believe it can be circumvented, but the base model is
| that.
| kaba0 wrote:
| For anything that doesn't run in "humanly instant"
| timeframe the startup speed won't matter - and build
| tools are definitely in this category.
|
| The fear for Oracle is also completely irrational, Java
| is probably the safest bet ever -- it is not tied to
| processor architecture, has a specification both on the
| language and the JVM level (both are uncommon in other
| platforms), has multiple completely independent
| implementation and the majority of fortune 500 companies
| definitely have business critical infrastructure running
| on top of it, so even if Oracle would do something those
| can single handedly support the platform indefinitely.
|
| Especially that Oracle has been a surprisingly good
| steward of the language, and they were the ones that
| finished open sourcing everything.
| mort96 wrote:
| > For anything that doesn't run in "humanly instant"
| timeframe the startup speed won't matter - and build
| tools are definitely in this category.
|
| 100% disagree. Waiting multiple seconds as the tool
| starts up just to get to the point where it actually
| invokes a compiler is infuriating to me. If you don't
| have a problem with that, good for you I guess.
|
| > The fear for Oracle is also completely irrational
|
| Again, I 100% disagree. Have you _seen_ how they 're
| treating ZFS? And the lawsuit against Google shows that
| they consider copying their APIs to be copyright
| infringement, so I wouldn't bank on non-Oracle-sanctioned
| community re-implementations.
|
| If you want to tie your C++ code to Oracle, I won't stop
| you, but I won't be doing that. I'm happy that you found
| a build tool you like. I will stick with one that's not
| based on Java.
| kaba0 wrote:
| > 100% disagree. Waiting multiple seconds as the tool
| starts up just to get to the point where it actually
| invokes a compiler is infuriating to me
|
| I think Java's startup time is often overblown, a hello
| world literally finishes in less than 0.1s, and that
| won't significantly grow at the size of a build tool.
| Plus daemons are a thing. Also, there is not much point
| in starting the compiler when you don't even know whether
| it is necessary.
|
| As for ZFS, what do you mean? Not familiar with the
| situation, it _is_ open-source and several open source
| OSs use it without any trouble. It's just Linus's
| overprotective stance against mixing two open-source
| licenses that makes it unmergable into the kernel, but I
| do use ZFS on linux every day, so where is the evilness
| of Oracle?
| mort96 wrote:
| > I think Java's startup time is often overblown, a hello
| world literally finishes in less than 0.1s
|
| The problem isn't hello world, it's the start-up time of
| big jars. My experience tells me that JVM start-up time
| on actual large software projects is a real issue in
| practice. But I haven't used Bazel in particular; I'd be
| more interested in numbers for that than numbers for
| hello world.
|
| > As for ZFS, what do you mean? [...] It's just Linus's
| overprotective stance against mixing two open-source
| licenses that makes it unmergable into the kernel
|
| Haha, that's not right. The problem is that ZFS's
| license, the CDDL, is (intentionally) incompatible with
| the GPL, so you can't link CDDL code against GPL code.
| It's not just an issue with making it part of the main
| Linux tree either, it's probably not even legal for
| distros to even ship a ZFS kernel module, since that has
| to link against Linux's GPL code (according to common
| interpretations of the licenses; Canonical notably
| disagrees).
|
| And let me repeat that this mess was _intentional_ on the
| part of Sun when they made the CDDL (see https://en.wikip
| edia.org/wiki/Common_Development_and_Distrib...), and it
| remains intentional on the part of Oracle as they choose
| to keep ZFS under the CDDL.
|
| The mechanism which prevents Linus from merging ZFS into
| the kernel, and the mechanism which prevents distros from
| shipping ZFS out of the box, is the exact same mechanism
| which prevents Linus from merging any other GPL-
| incompatible code or distros from shipping any other GPL-
| incompatible kernel modules.
|
| Nothing is stopping any individual from grabbing the
| OpenZFS source code and linking it against their Linux
| kernel; that's legal. Oracle is just doing its best to
| prevent people from combining OpenZFS and Linux into one
| package and distributing the result.
| kaba0 wrote:
| From your description I literally only see that "Oracle
| did nothing at all", which might be a conscious decision,
| but it's not like ZFS would be some secret money grab
| scheme or something like that.
| mort96 wrote:
| I'm not saying ZFS is a secret money grab scheme. I'm
| saying I don't trust Oracle and the way they're
| hamstringing ZFS is one of many reasons why.
|
| But I think I'll leave the discussion here, it's not
| about build systems anymore. As I have already said, if
| your take on Oracle is that they're a trustworthy company
| and a good steward of crucial parts of your stack, go
| ahead and use Bazel, I'm not stopping you.
| [deleted]
| pledess wrote:
| there may be a hint of which elections were of interest:
|
| https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92...
| anoncow wrote:
| This is the latest comment.
| anoncow wrote:
| I posted this to check if Bard can read HN posts in order.
| firstSpeaker wrote:
| Would it be developed in open as well or there will be frequent
| merge from their internal repos?
| HellsMaddy wrote:
| Interesting: // we only keep unfollows in the
| past 90 days due to the huge size of this dataset, // and
| to prevent permanent "shadow-banning" in the event of accidental
| unfollows. // we treat unfollows as less critical than
| above 4 negative signals, since it deals more with //
| interest than health typically, which might change over time.
| val unfollows: SCollection[InteractionGraphRawInput] =
| GraphUtil .getSocialGraphFeatures(
| readSnapshot(SocialgraphUnfollowsScalaDataset, sc),
| FeatureName.NumUnfollows, endTs)
| .filter(_.age < 90)
|
| https://github.com/twitter/the-algorithm/blob/main/src/scala...
| dmix wrote:
| How long does the NSA record them?
| tric wrote:
| From https://github.com/twitter/the-
| algorithm/blob/7f90d0ca342b92... (
| "author_is_elon", candidate => candidate
| .getOrElse(AuthorIdFeature,
| None).contains(candidate.getOrElse(DDGStatsElonFeature, 0L))),
| ( "author_is_power_user", candidate =>
| candidate .getOrElse(AuthorIdFeature, None)
| .exists(candidate.getOrElse(DDGStatsVitsFeature,
| Set.empty[Long]).contains)), (
| "author_is_democrat", candidate =>
| candidate .getOrElse(AuthorIdFeature, None)
| .exists(candidate.getOrElse(DDGStatsDemocratsFeature,
| Set.empty[Long]).contains)), (
| "author_is_republican", candidate =>
| candidate .getOrElse(AuthorIdFeature, None)
| .exists(candidate.getOrElse(DDGStatsRepublicansFeature,
| Set.empty[Long]).contains)), )
| philistine wrote:
| Coded as if the only two political parties on the planet were
| the Rs and the Ds. Shameful.
| grishka wrote:
| *as if the US is the only country out there
| bikeformind wrote:
| I opened this thread just to verify this would be top comment,
| good job hn
| [deleted]
| jdnordy wrote:
| github link now show a warning at the top of the page:
|
| > This commit does not belong to any branch on this repository,
| and may belong to a fork outside of the repository.
|
| Is this new? Perhaps Twitter already removed the code from
| their main branch? Or was this just a joke from the beginning?
| tric wrote:
| Yup, they removed it.
|
| https://github.com/twitter/the-
| algorithm/commit/ec83d01dcaeb...
|
| Here's what Musk said:
|
| > But we are deleting this bs. I only learned about it now!
| Will be gone by tomorrow.
|
| https://twitter.com/elonmusk/status/1641908130274525187
| jawns wrote:
| The author_is_elon flag doesn't surprise me, but the two
| political designators are somewhat shocking. I'd sure like to
| know what changes based on what Twitter knows about your
| political affiliation.
| jen20 wrote:
| It's not that shocking...
|
| Half the people that got promoted on my timeline were
| perpetually candidates for elections I couldn't vote in, and
| they _self-identified_ as Republican or Democrat in their own
| bios, or via the registration of their candidacy...
|
| This is why I exclusively used to use Twitter in the "people
| I follow only" mode, and simply shut my account down when
| they pushed harder on the algorithm.
| jandrese wrote:
| I thought it was interesting how it explicitly doesn't boost
| independents. So much of the two-party system is self-
| reinforcing.
| wahnfrieden wrote:
| Or anti-statists! Not everyone who engages with politics is
| a bootlicker for authoritarianism but it sure feels like
| there's no space made for this perspective (obviously)
| lapetitejort wrote:
| I'd be interested in which bin people like Joe Manchin,
| Kyrsten Sinema, Susan Collins, et. al. are placed.
| emodendroket wrote:
| The vast majority of self-proclaimed independents vote with
| one party just as reliably as registered members.
| silisili wrote:
| Majority? Probably. Vast majority? No way. If that were
| true we'd not keep switching parties between presidents.
| And Florida, a mostly independent voter state, wouldn't
| have had DeSantis win in a landslide when he just barely
| tied last time around.
| philistine wrote:
| American's lax attitude towards cultivating more than two
| parties is literally killing the republic from the
| inside.
| airstrike wrote:
| You'd think a country that played a central role in a
| global, decades-long unstable regime of bipolar power
| that routinely pushed mankind to the brink of nuclear
| oblivion would know better than to have a bipolar
| electoral system
| robertlagrant wrote:
| > routinely pushed mankind to the brink of nuclear
| oblivion
|
| I'm struggling to think of a reason why this is anything
| but bad faith nonsense.
| weaksauce wrote:
| have you never heard of the many times we almost came to
| setting off full scale nuclear warfare because of the
| bipolar power war of the ussr v usa et.al.?
|
| the only thing that saved us was cooler heads that
| prevailed on both sides.
| robertlagrant wrote:
| I've not heard how many times, no. How many times was it?
| mandmandam wrote:
| [dead]
| airstrike wrote:
| Luckily there's a whole article on it
| https://en.wikipedia.org/wiki/List_of_nuclear_close_calls
| emodendroket wrote:
| Why would throwing more actors with similar capabilities
| into the mix make the situation any more stable though?
| That seems like basically the old European Balance of
| Power, which broke out into open conflict more
| frequently.
| airstrike wrote:
| I take the view that the European balance of power
| probably broke out into open conflict often because of
| hidden alliances that made it hard for states to
| correctly gauge the costs of engaging in such conflict.
|
| In the field International Relations, there's lots of
| discussion around the stability (or instability) of a
| bipolar distribution of power. Your stance is closer to
| neorealists, mine is maybe closer to classical realists.
| Have fun reading https://en.wikipedia.org/wiki/Polarity_(
| international_relati... and related links
|
| But from a game theory perspective, having only two
| powers turns everything into a zero-sum game. I argue
| that leads to less cooperation and increased
| divisiveness. Not agreeing with Republicans means you
| must be a Democrat in their eyes. This "us versus them"
| mentality is somewhat tribal and leaves little room for
| nuance.
|
| If I had a magic want, the US would have a 4 party
| system. Like a cartesian plane with civil liberalism <->
| conservatism on one axis and economic liberalism <->
| conservatism on the other axis
| emodendroket wrote:
| > Classical realist theorists, such as Hans Morgenthau
| and E. H. Carr, hold that multipolar systems are more
| stable than bipolar systems, as great powers can gain
| power through alliances and petty wars that do not
| directly challenge other powers; in bipolar systems,
| classical realists argue, this is not possible.
|
| I suppose they must account for this somehow but isn't
| that exactly what a series of proxy wars in far-flung
| places between the United States and the Soviet Union
| were?
| airstrike wrote:
| It's been almost 20 years since I studied any of that,
| but those proxy wars in far-flung places did directly
| challenge other powers because of the Domino theory
|
| I don't necessarily agree with Morgenthau and Carr as I
| think most IR Theory is bullshit made up by academia...
| particularly the stuff around how players "gain power"
|
| So I make my own argument which mostly hinges on the idea
| that two powers really means "my power" vs. "everyone
| else" which is not a recipe for peace
| myko wrote:
| This balance is the central theme of the seminal work in
| the field (https://www.jstor.org/stable/j.ctt5vm52s). It
| may or may not be correct but is one of the most
| influential texts on the subject.
| variant wrote:
| It's just different order of operations on coalition
| building. Other systems divide into a majority and
| opposition at some point. In the US it just happens
| earlier, but there is the same diversity of opinions
| within those groups.
| emodendroket wrote:
| In some other countries the different interest groups
| sort themselves into two factions _after_ being elected
| but I don 't know that it is really that different in
| practice.
| suddenclarity wrote:
| In some ways it might be worse. Here we had a party that
| promised to vote with one faction and partly thanks to it
| survived the election. A few months later they did a 180
| and voted the other faction into government. Opinions
| aside, imagine voting for Sanders only for him to elect
| Trump or the other way around. With that said, two party
| version also comes with major flaws.
| TillE wrote:
| It's not different, or rather it doesn't produce
| meaningfully different outcomes. I'm not aware of any
| parliamentary system with a wonderful diversity of
| thought and a long record of positive accomplishments.
| You end up with ruling coalitions which are typically
| pretty awful.
|
| Ironically, America has one of the most open political
| systems. You register as one party or the other and vote
| in primaries. This has lead to a huge variety of people
| replacing hated mainstream politicians. That's way more
| than you can say for many other countries.
| soderfoo wrote:
| The two parties are essentially coalitions.
|
| Anecdotally, a recurring theme in conversations I've had
| while living abroad is the desire to prune or consolidate
| some parties.
|
| While both sides in the US have big tents, they are
| effective in whipping votes when things need to get done.
|
| It also helps that detracting _coalition_ partners can 't
| torpedo their leadership. Historically, factions within a
| party, like Blue dog dems or Tea partiers, had to wait
| for an election to litigate their grievances.
| emodendroket wrote:
| Well, yes, this was where I was going. The coalition is
| just formed before the election takes place. I live in a
| city with "nonpartisan" elections and I feel like it
| mostly just makes it difficult to understand what
| candidates in local elections even stand for.
| soderfoo wrote:
| Agreed, when I lived in Austin the non partisan elections
| for city council made me feel ill informed come election
| day.
|
| It just fueled apathy for me: there's only so much
| research one can do with limited press coverage and
| personal time constraints. Shrug.
| mthoms wrote:
| >it doesn't produce meaningfully different outcomes
|
| It absolutely produces different outcomes than a two
| party system. Smaller parties make demands as a condition
| for joining any coalition (or similar arrangement). For
| example, Canada's NDP only agreed to back the Liberal
| Government on condition of state funded dental care for
| children being implemented. Now it is. _Millions_ are
| affected. Whether you agree with it or not - that is
| unquestionably a "meaningfully different outcome". Other
| examples abound if you care to look.
|
| >I'm not aware of any parliamentary system with a
| wonderful diversity of thought and a long record of
| positive accomplishments.
|
| That's an impossibly high bar. The standard we're talking
| about is whether it's better than a two party system.
| emodendroket wrote:
| > It absolutely produces different outcomes than a two
| party system. Smaller parties make demands as a condition
| for joining any coalition (or similar arrangement). For
| example, Canada's NDP only agreed to back the Liberal
| Government on condition of state funded dental care for
| children being implemented. Now it is. Millions are
| affected. Whether you agree with it or not - that is
| unquestionably a "meaningfully different outcome". Other
| examples abound if you care to look.
|
| You think logrolling doesn't happen in the American
| system? Parties are made up of factions who will
| entertain each other's preferred priorities, which is the
| exact same thing you're describing except that the
| parties of the various factions are nominally the same.
| There's no real natural or obvious philosophical reason
| why your position on gun ownership should imply a
| position on environmental regulation, religion,
| infrastructure buildings, racial politics, abortion, tax
| policy, and more. We're so used to the groupings that
| they seem natural but if we go back and looking at older
| American party systems you'll see parties that don't 100%
| map onto the contemporary ones, with a blend of some
| elements we would think of as fitting and others we
| wouldn't.
| mthoms wrote:
| I never said it didn't happen. Don't do that.
|
| It's unquestionably a different dynamic. If the party is
| in power, then the party is in power. Period. They have
| less incentive to listen to smaller factions. Do they? I
| sure hope so. But it's not the same.
|
| Furthermore, these internal deals are more likely to be
| kept private since theres no benefit to airing everything
| to the press, the public and the opposition.
|
| In a multi-party system almost all those deals are
| public. Thus, voters can decide if everyone lived up to
| their promise.
| emodendroket wrote:
| That's just not true. If you want a recent example go
| look at the protracted battle to elect a Speaker of the
| House.
| djbusby wrote:
| IIRC Some states you cannot vote outside your declared
| party (Washington)
| favorited wrote:
| That's only for primaries, which traditionally were only
| open to members of a party.
| hraedon wrote:
| It isn't an attitudinal problem, it is the logical
| outcome of our political systems. In political science it
| is known as Duverger's law: single ballot, winner take
| all systems inevitably tend toward a two party
| equilibrium.
|
| Changing this requires states to adopt alternative
| systems, which can sometimes mean amending state
| constitutions. It isn't easy or straightforward, and the
| general sense is that there are better things to spend
| that effort on.
| noizejoy wrote:
| > single ballot, winner take all systems inevitably tend
| toward a two party equilibrium.
|
| I'm not convinced it's quite that simple.
|
| For example, Canada also has a first-past-the-post
| electoral system - yet political parties here have come
| and gone. And continue to do so.
| tialaramex wrote:
| Parties come and go, but there will be two major parties.
| In fact under such as system if you're in one of those
| two big parties and you see a third party rising you need
| to figure out whether it's you or the other guys getting
| replaced, 'cos it won't stay a three party system for
| long.
|
| For example in the UK, the Tories ("Conservative and
| Unionist Party") and Labour are currently the biggest
| parties, but a hundred years ago this was a novel
| situation, Labour were seen as a third party, while the
| Liberal party (which was eventually absorbed into what is
| today "Liberal Democrats") had seen success over decades
| and were often in government prior to that point.
| emodendroket wrote:
| I mean we're on the sixth "party system" in the US too,
| if that's your standard (it's a new "party system" when
| there's a significant realignment of which interests find
| themselves in which parties, either by a reshuffle among
| existing parties, or by new parties rising and old ones
| falling). Have you heard much about the Federalists or
| Whigs lately?
| AmericanChopper wrote:
| The voting demographics change frequently as well.
| California has voted to elect more republican presidents
| than it has democrat ones. It's voted democrat in the
| previous 8 elections. In the 10 elections prior to that,
| 9 times it voted for republican candidates. Texas has
| also voted for more democrat presidential candidates than
| republican ones. People who think the US democracy is a
| rigid and highly predictable system simply have a recency
| bias.
| saalweachter wrote:
| A lot of it depends on whether your governing coalitions
| are formed before or after the election.
|
| The US parties are just coalitions of disparate interests
| joining together until they (maybe) represent enough
| people to have a majority and be able to enact their
| collective interests.
| tablespoon wrote:
| > I thought it was interesting how it explicitly doesn't
| boost independents. So much of the two-party system is
| self-reinforcing.
|
| Is it boosting? Others are claiming this code is just for
| metrics collection:
| https://news.ycombinator.com/item?id=35391896.
|
| But on the topic of Democrats vs. Republican vs.
| independent; a big factor may be that "Democrat" and
| "Republican" are much more cohesive groups and therefore
| much easier to define. No one can honestly define
| "independent" except in a kind of "none of the above"
| sense, since they can range anywhere from extreme right, to
| the center, to the extreme left.
| nonethewiser wrote:
| Metrics collection so they can avoid bias, allegedly
| eric-burel wrote:
| There are probably many other non-political alternatives
| to select 2 groups to test for bias
| dd36 wrote:
| Why measure something if you don't intend to change it?
| dredmorbius wrote:
| You might want to ask that of, say, astronomers,
| paleontologists, or even simply journalists.
|
| Observing and measuring leads to understanding. As others
| here have noted, sometimes you want to measure to ensure
| that you're _not_ inadvertently affecting an outcome or
| phenomenon.
| nosianu wrote:
| > You might want to ask that of, say, astronomers,
| paleontologists, or even simply journalists.
|
| You compare scientists with businesses. One of them's job
| and passion is to collect knowledge for the sake of
| knowledge. For the other one it would be cost without
| gain and eliminated if they didn't do anything with it!
|
| No idea what journalists are doing in that list, what do
| they measure? If they want something measured they'd ask
| or look at other's services. Unless you mean the business
| that the journalists work for, but hen it's just that, a
| business.
| dredmorbius wrote:
| * * *
| astrange wrote:
| So you can make sure not to accidentally change it.
| lapetitejort wrote:
| And if one party trends more towards violent extremism,
| they get boosted the same as before?
| astrange wrote:
| What's that got to do with accidental effects of
| unrelated software changes?
| gentoo wrote:
| if your hate speech filter hits one party harder than the
| other, that doesn't necessarily mean it is broken
| astrange wrote:
| What if you're changing the spam filter?
| throwaway50601 wrote:
| The point is to have these statistics remain the same for
| the individual groups in between changes, not to measure
| them against each other.
| dd36 wrote:
| So then why make any changes?
| throwaway50601 wrote:
| Why not? If it doesn't matter but it's controversial
| there's no reason not to change it.
| boringg wrote:
| There are many reasons to measure things if you don't
| intend to change them.
| jandrese wrote:
| Depends what the metrics are used for. It doesn't make
| sense to apply artificial boosts to metrics that are only
| used for internal accounting. Well, maybe if you have an
| egotist CEO, but that wouldn't explain the rest of the
| boosts. We have to assume this code has some sort of
| effect somewhere.
| robertlagrant wrote:
| Isn't it intrinsically self-reinforcing, if you have a
| winner takes all system? It's almost always better to join
| an existing team than start a new one.
| airstrike wrote:
| Also how about.... everyone else in the world who is not an
| American voter?
| threeseed wrote:
| Only anecdotally.
|
| But ever since Musk took over the amount of US political
| content has significantly increased in particular from
| the right despite me not living in the US.
|
| It's hard to tell whether previously political content
| was weighted less and Musk has removed those controls or
| whether they are now weighted higher.
| kevingadd wrote:
| Some of the power users Musk reportedly had boosted are
| specifically right wing political posters, like catturd2.
| But then he's also boosted some high profile left leaning
| politicians, so it's not exclusive. It does mean you're
| more likely to see right wing American political content
| either way, which has to be annoying for people outside
| of the US.
|
| https://gizmodo.com/twitter-algorithm-aoc-ben-shapiro-
| cattur...
| realjhol wrote:
| Why do you believe it would be annoying to us?
| fyloraspit wrote:
| Would it not be just as annoying to see a slant the other
| direction, or in any extreme direction?
| IncRnd wrote:
| The parent poster doesn't realize the inherent bias in
| what they wrote and that you have pointed out.
| kevingadd wrote:
| Boosting both right and left wing content increases your
| chance to see either type of content, because you boosted
| them over _everything else_.
| IncRnd wrote:
| I'm not sure why you wrote that to me. It was already in
| your prior comment.
| oneeyedpigeon wrote:
| I believe their point is that, because the US is
| generally regarded as 'to the right of' many countries in
| Europe, for example, a slight rightwing bias looks to us
| (I'm in the UK) like an extreme rightwing bias. Heck,
| even a completely neutral stance, as far as the US is
| concerned, will look like it veers to the right for us.
| Note that I'm not saying anything about how I believe the
| twitter algorithm should work, this is just my
| interpretation of what that commenter meant.
| AlchemistCamp wrote:
| Europe is only a tiny fraction of the world, though.
|
| Compared to India, Japan, Singapore and (depending on the
| metric) China, the US looks clearly left of center.
| Compared to most of the Middle East, the US looks
| extremely left-wing.
| agentgumshoe wrote:
| [flagged]
| KyeRussell wrote:
| If these 'distraction-causing arguments' are affecting
| real people (read: foreigners, queer people, black
| people), then they're worth paying attention to
| regardless.
|
| Is your assertion that this is all some big conspiracy
| how you justify not caring?
| agentgumshoe wrote:
| No, it's that people have bought so heavily into just
| attacking 'the other' instead of trying to get along and
| being critical of policy and authority instead. The
| distraction is the direction to argue with each other
| rather than against policy/corruption/authority actually
| causing the suffering
| ImPostingOnHN wrote:
| some of those opinions are that others of certain sexual
| orientations are indeed subhuman and/or should be treated
| as much, and due to Popper's paradox of tolerance, we
| don't need to, and shouldn't, tolerate such intolerance
| agentgumshoe wrote:
| That's one interpretation, which I see used as
| justification to attack and be generally horrible to
| anyone who disagrees. I see a lot of commenters making
| comments that Republicans are some kind of lower
| intelligence being too (same the other way too). Are
| these people allowed to be offended?
|
| Unfortunately the plight of LGB (and Womens' rights to an
| extent) has been utterly brushed aside by the continual
| addition of each letter to the cause and the arguments
| too hot and aggressive to separate out the underlying
| causes for fair debate on their own.
| HyperSane wrote:
| > Republicans are some kind of lower intelligence being
| too
|
| You HAVE to be pretty stupid to fall for the really
| blatant propaganda the GOP uses. Remember that scary
| migrant convoy they invented just before an election to
| use to scare people into voting GOP and then it just
| vanished right after? Have you heard the incredibly
| stupid things Tucker Carlson, MGT, Matt Gaetz, and
| Boebert say? People like Carlson are way too smart to
| actually believe the crap they spew, they knowingly spend
| their lives telling lies to gullible and/or stupid people
| to manipulate them.
| agentgumshoe wrote:
| Of course there is no left wing propoganda to fall for
| though, right?
|
| Do you wonder how you came to form such blatant negative
| opinions of your political opposites as a charicature of
| hate and ignorance?
| ImPostingOnHN wrote:
| whataboutism doesn't really address any of the points in
| the post you're responding to, and certainly didn't
| answer the questions in that post which were posed to you
| HyperSane wrote:
| > Of course there is no left wing propoganda to fall for
| though, right?
|
| I'm honestly not aware of similar propaganda on the left.
| By similar I meant an entire industry designed to lie and
| gaslight people into voting for people who will cause
| them direct economic harm. The GOP has made lying an
| entire industry and an art form. Now they are demonizing
| trans people in a way extremely similar to the way Nazis
| demonized Jews.
|
| > Do you wonder how you came to form such blatant
| negative opinions of your political opposites
|
| I know exactly why I have formed such a negative opinion
| of the GOP, because I've watched them lie blatantly and
| egregiously for the last 20 years and I'm really sick of
| it.
| ImPostingOnHN wrote:
| you say "attack and be generally horrible to anyone who
| disagrees", but recall that the thing said "attackers"
| are disagreeing with is equal or greater animosity
| towards people on the basis of their sexuality versus
| their opinions
|
| in other words, it's a reaction to intolerance,
| specifically a decision not to tolerate intolerance - as
| we covered before, we don't need to, since doing so will
| end tolerance entirely (see Popper's paradox of tolerance
| for this)
| agentgumshoe wrote:
| I think Popper's paradox does nothing really but talk
| around an issue by changing the argument. Intolerance is
| not black and white, it is contextual. You can be
| intolerant of those who break the law, because they break
| the laws that hold society together (at a basic level).
| Paradoxes often aren't so problematic if you take them
| out of their own frame of reference (e.g. Zeno's paradox
| & Achilles).
|
| I think the context we are referring to here is really
| free speech. Should we tolerate speech (opinions) that we
| despise or disagree with? Absolutely. And here we have
| the attackers responding to feeling insulted (framed as
| 'intolerance') by not being accepted/agreed with. Then
| they respond in kind and no progress is made.
|
| You show me those people responding to equal/higher
| hostility and I guarantee I can show just as many
| examples of the opposite. And what I mean there is not
| "you're side is worse," I mean there are aggressive and
| horrible actors on both sides, who should not be the
| frame of reference for debate. But too often they are,
| and too often examples of hostility as used as an excuse
| for escalation or to avoid real debate.
| ImPostingOnHN wrote:
| It's nice that you personally think those things, but
| none of those things you personally think override or
| disprove Popper's paradox of tolerance, and thus none of
| those things you personally think will convince people to
| tolerate the intolerance that is, just as a single
| example, right-wing anti-trans intolerance
|
| indeed, responding to, say, right-wing intolerance of
| trans people, by not tolerating it, is _totally okay_ ,
| despite your FUD, for reasons described by Popper
|
| don't like your own intolerance of e.g. trans people, and
| actual attacks on them, to be responded to? maybe keep
| your intolerance and actual attacks to yourself, problem
| solved
| mcv wrote:
| That extreme polarisation didn't start with the last few
| presidents; it's been the explicit strategy of the
| Republican Party to demonize the Democrats in every
| possible way ever since Newt Gingrich became a prominent
| figure in the 1980s.
| HyperSane wrote:
| And Fox News was created specifically for this purpose.
| agentgumshoe wrote:
| Of course it has. And the Democratic party just wanted to
| get along.
|
| Either way, unfortunately the political bickering fuelled
| by bots and social media manipulation is in a very sad
| state.
| HyperSane wrote:
| > And the Democratic party just wanted to get along.
|
| In comparison to the GOP yes.
| agentgumshoe wrote:
| [flagged]
| mcv wrote:
| It was frustrating to see Obama keep reaching across the
| aisle and basically getting spit on. Democrats have their
| faults, but Republicans have made it clear that it's
| impossible to reach any kind of compromise with them.
| meragrin_ wrote:
| I live in the US in "Trump country" and don't see any
| political content. Maybe its just related to the people
| you follow and the content you look at?
| tormeh wrote:
| This is what it means to let another country own your
| social media. Their ideas and memes unconsciously get
| preferential treatment. This is maybe not a good thing,
| but it is what it is.
| adamckay wrote:
| > Their ideas and memes unconsciously get preferential
| treatment.
|
| I think as this repo shows it's conciously, rather than
| unconsciously, getting preferential treatment.
| blululu wrote:
| It's omission. They built for Americans because they were
| Americans. No one building this said "let's ignore
| Canadian politics" or any other country, they just didn't
| think about them at all, because like most Americans they
| don't really care about the insidious Quebecois plots to
| annex Prince Edward Island or whatever actual issues are
| happening in Canada.
| 908B64B197 wrote:
| The freedom convoy and it's repression with suspension of
| constitutional rights (because that's a thing over
| there?) and bank accounts arbitrarily frozen did get a
| little bit of attention in the American medias.
|
| > the insidious Quebecois plots to annex Prince Edward
| Island
|
| I know it's a joke, but the people of Prince Edward
| Island would most likely welcome it. Economically that
| would be a huge boost. Not sure Quebec would enjoy it
| however.
|
| Perhaps they should look at annexing Labrador?
| the_why_of_y wrote:
| Indeed their acts of terrorism were not reported by
| international media.
|
| For example, during the occupation of residential areas
| of the capital they created deafening noise around the
| clock that was measured indoors (windows closed) at
| levels causing permanent hearing damage after minutes of
| exposure, with police doing nothing to stop them.
|
| https://news.ycombinator.com/item?id=30434168
| mthoms wrote:
| Bank accounts were not "arbitrarily frozen". This is
| total nonsense.
|
| https://nationalpost.com/news/canada/banks-were-only-
| asked-t...
| LudwigNagasena wrote:
| Your article seems to support the idea that bank accounts
| were arbitrarily frozen.
| canjobear wrote:
| Not arbitrarily: the freezing was targeted against
| political protestors.
| HyperSane wrote:
| They were going pretty damn far beyond "protesting" they
| were trespassing/loitering/squatting.
| [deleted]
| nicky0 wrote:
| Loitering, dear god!
| HyperSane wrote:
| When done in giant trucks blocking traffic and making
| tons of noise and exhaust fumes for weeks it is a crime.
| 908B64B197 wrote:
| That sounds straight out of Russia or China.
| HyperSane wrote:
| > The freedom convoy and it's repression with suspension
| of constitutional rights
|
| This was needed because the convoy of morons was
| seriously damaging people's quality of life and the local
| police did not do their job. The police in other towns
| DID do their job and blocked the trucks. You should be
| mad at the cops who just sat by and watched the convoy of
| morons roll in with their thumbs up their butt.
| concordDance wrote:
| What exactly is allowed in a protest is quite a thorny
| issue.
|
| Here in the UK there are environmental protestors gluing
| themselves to the streets and I have quite mixed feelings
| about them.
|
| A protest that does not inconvenience anyone is very easy
| to ignore.
| 908B64B197 wrote:
| You can see here how wild and inconvenient these protests
| were. The bouncy castle and hot tub were a big public
| nuisance!
|
| [0] https://www.youtube.com/watch?v=P74pbbhCZng
|
| [1] https://www.youtube.com/shorts/hzfwaSVIvoA
| HyperSane wrote:
| The constantly idling giant trucks very much were a
| public nuisance. YOU wouldn't want to live near them.
| 908B64B197 wrote:
| People can always use their countries' social media (and
| I assume many do!).
|
| But it's indeed an interesting fact that people seem to
| specifically seek American tech and social media. And
| honestly, there's no shortages of foreign nationals
| commenting on American politics (and it's a good thing,
| it's their right thanks to the first amendment!).
| KyeRussell wrote:
| This is a joke, right?
|
| The vast vast vast vast majority of people use "American"
| social network because they are the social networks that
| exist. The US is undeniably the main exporter of SaaS
| products. See: the incumbents freaking out that China is
| getting a turn in the front seat. I've never once heard
| of anyone seeking 'American tech', except for some
| punchline in an anti-Soviet joke or movie.
|
| Foreign nationals can comment about American politics
| because the US doesn't have jurisdiction over their
| speech, notwithstanding the back and forth over whether
| social media companies are liable for disseminating such
| content in the first place.
|
| A material part of why the USA is seen as The Country in
| Western culture, and a noted big player in other
| cultures, is because of the power it projects via the
| media. That includes both Hollywood exports, and social
| media.
| 908B64B197 wrote:
| > I've never once heard of anyone seeking 'American
| tech', except for some punchline in an anti-Soviet joke
| or movie.
|
| Really? Why is everyone using MacBooks and iPhones?
| Actually most Soviet computers were copies of western
| designs. [0]
|
| > is because of the power it projects via the media. That
| includes both Hollywood exports, and social media.
|
| The thing is, is takes to to project, someone has to
| export a product and someone else has to import it. Since
| people seek it and want to consume it it's easy to
| export.
|
| [0] https://en.wikipedia.org/wiki/List_of_Soviet_computer
| _system...
| vba616 wrote:
| > I've never once heard of anyone seeking 'American tech'
|
| One reason people have a distorted idea of what physical
| products the US produces, so I've read, is that most of
| them are products for industrial use, in factories and so
| on.
|
| You look at consumer products that are made in...,
| because that's where the finishing touches are put on.
| That's not representative of the global economy though.
| Joeri wrote:
| American social media companies have the benefit of a
| large homogenous deeply interconnected culture with a
| vast ad buying community. Simply put, a U.S. social media
| platform can get more revenue with less effort than a
| European counterpart (because they have to go country by
| country, language community by language community), and
| will generally outcompete those EU counterparts, with
| social network reinforcement effects doing the rest. This
| holds up for most ad-supported online services, which
| together with a much stronger private investment sector
| is the reason American platforms absolutely dominate
| Europe. The EU has a stronger government subsidy system
| for software but that system does not reward or expect
| market success, which is why it delivers very little
| value.
| HyperSane wrote:
| The importance of US cultural power cannot be overstated.
| dbbk wrote:
| This is an interesting point... they're making sure their
| A/B experiments don't adversely affect Republicans or
| Democrats. But lots of European countries are skewed more
| to the left and would broadly look more similar to the
| "Democrats" group.
| ben_w wrote:
| Before I deliberately locked myself out of it (well
| before Musk), I asked for my data.
|
| They classify me as: * speaks Indonesian
| Interested in: * Beer * Cricket * DJs
| * Dance * Enterprise software * Horror
| * NFL football * South America And aged
| either between 13-54 or (and?) over 65
|
| Other than the age (I'm neither under 13 nor between
| 55-64), everything I've listed is incorrect.
|
| On that basis, they'd probably call me a Republican.
|
| Carries a different meaning when you're British, that
| name does.
| dudeinjapan wrote:
| Aren't "enterprise software" and "horror" the same thing?
| ben_w wrote:
| Fair.
|
| I try to avoid both, when I can. :)
| tedivm wrote:
| I'm a guy in my 30s but they had me classified as a 65+
| year old woman. They also thought I was a radiologist.
| oneeyedpigeon wrote:
| What a strange age classification -- no idea about what
| your age actually is, just that it definitely _isn 't_
| 55-64. I assume it would never conclude you are under 13
| for legal reasons.
| bryanrasmussen wrote:
| I thought it was interesting that twitter thinks the U.S is
| the world
| whatgoodisaroad wrote:
| For a big tech company, the regulatory regime is the
| world. For Twitter that's the US.
| bryanrasmussen wrote:
| Most big companies I've worked at are international and
| are regulated in many regimes, although sure the one with
| the main corporate headquarters is more important.
| nelox wrote:
| Twitter thinks Twitter is the world.
| JustSomeNobody wrote:
| Exactly. I don't see NPA (no party affiliation) anywhere.
| ldoughty wrote:
| Did you not get the memo? "If you're not with us, you're
| against us".
|
| It's probably more of a conservative/liberal identifier
| based on US political party ideals... And they likely would
| filter any metrics from this by the users country
| stouset wrote:
| I _suspect_ that these are used for metrics tracking rather
| than being fed back into the recommendation engine. But there
| 's no real way to know for sure given the limited release.
| These predicates aren't actually used anywhere in the code
| that's been made available.
| 6nf wrote:
| So many questions. How are users tagged D or R? Is that a
| manual process or automated somehow? What is the effect of
| these tags? Can I find out if my Twitter account is in one of
| those buckets?
| abracadaniel wrote:
| And how are they choosing to balance them, per capita, or
| just both sides should get 50%? It seems pretty clear they
| are making editorial decisions here. Does that break their
| section 230 protections?
| astrange wrote:
| "Section 230 protections" have nothing to do with
| "editorial decisions"
| KarlKemp wrote:
| Please read something. Preferably section 230, which is
| short. Alternatively something entirely different, as
| long as that keeps you occupied.
| krapp wrote:
| >It seems pretty clear they are making editorial
| decisions here. Does that break their section 230
| protections?
|
| No.
|
| https://www.techdirt.com/2020/06/23/hello-youve-been-
| referre...
| cma wrote:
| Likely they are tracking performance verified politician
| accounts based on registered party affiliation. Why
| republican should count equal in the evaluation metric to
| democrat when nunerically there are less republican voters,
| let alone proportions on Twitter, is another question.
| [deleted]
| smegger001 wrote:
| you could probably algorithmically determine it in most
| cases based on any number of indicators from phrases used,
| to communities interacted with, which hashtages are
| included, which cohort retweets and likes most etc... thats
| not even getting into simply tagging political figures with
| the party they officially affiliate themselves with
| moffkalast wrote:
| Especially for people that aren't... you know.. Americans.
|
| Unless they mean actual public figure party members which
| are known and probably verified.
| partiallypro wrote:
| Facebook guesses your political affiliation as well, you can
| even look uber your settings to see what they guessed.
| krapp wrote:
| The repo suggests it's about tracking engagement metrics[0],
| so Team Red people see more Team Red content and vice versa.
| Nothing nefarious.
|
| [0]https://github.com/twitter/the-
| algorithm/blob/7f90d0ca342b92...
| minimaxir wrote:
| Having a separate flag to track engagement specifically for
| Elon tweets isn't nefarious, but _weird_.
| jeron wrote:
| You say it's not nefarious but isn't that how echo chambers
| are created?
| krapp wrote:
| I don't believe echo chambers are nefarious - there's no
| hidden agenda involved with them. That's just how
| recommendation algorithms work, and it's what most people
| want.
|
| But if someone finds some code that suppresses
| recommendations from a specific political ideology across
| the board, _that_ would be nefarious, IMO.
| thfuran wrote:
| Echo chambers may not be nefarious, but they are
| insidious.
| krapp wrote:
| They can be if they're involuntary and inescapable, but
| neither is the case for Twitter. It's designed around
| letting you curate your own feed, but it also constantly
| throws random stuff in through retweets and quote tweets
| - which is what people hate the most about the platform.
| ClumsyPilot wrote:
| [flagged]
| kodah wrote:
| This comment likely violates the guidelines both in form
| and content.
| krapp wrote:
| I don't believe repeating someone's comment nearly
| verbatim is as clever as you want it to be, nor do I
| believe recommendation algorithms are equivalent to the
| kinds of societal discrimination you allude to, nor would
| any reasonable person.
|
| Instead of trying very hard to be clever, please next
| time try just as hard to make a valid point. I know it
| can be difficult when you think you smell karma in the
| water but do try.
| yazzku wrote:
| The echo chamber in your first point does the suppression
| in the second.
| taco_philips wrote:
| [dead]
| [deleted]
| tric wrote:
| Why specifically track political parties? Where is
| author_is_american? Or author_is_mayonnaise_enjoyer?
|
| Maybe it was a choice made many years ago that they thought
| was appropriate, but we can't yet know it's not used for
| other purposes. We can at least be reasonably sure they've
| added the author_is_elon within the past year. I would have
| thought there would be many more descriptors, or non-
| controversial descriptors.
|
| Or maybe Elon specifically added those before releasing
| this code to get people riled up.
| krapp wrote:
| Twitter became a very politically charged platform after
| 2016, mainly among Americans. They'd be idiots not to
| take advantage of that.
| jdminhbg wrote:
| The point is probably to check that changes they make
| aren't accidentally politically biased.
| [deleted]
| GaryNumanVevo wrote:
| I wonder who's on the "VIT" (Very Important Tweeter) list?
| wahnfrieden wrote:
| People like Ben Shapiro, Glenn Greenwald, @catturd2
| fortuna86 wrote:
| Right wing high engagement accounts. Through in one or two
| accounts like @AOC for "balance"
| coolspot wrote:
| BTW: How @aoc got three-letter handle?
| fortuna86 wrote:
| I don't know but that's not relevant to my point..
| fknorangesite wrote:
| Probably exactly the most boring, mundane way you would
| expect: they managed to contact the owner of what was
| probably a dead/inactive account and then paid them some
| money for it.
| rhaway84773 wrote:
| AOC is also not a very common/popular three letter
| combination.
| foderking wrote:
| it is. "advent of code" "area of concentration"
| rejectfinite wrote:
| Im using 3 monitors by AOC right now.
| dbbk wrote:
| It's against the Twitter terms to pay money for accounts.
| I think more likely is that Twitter 'seized' the dead
| account and just granted it to her. I'm sure for instance
| someone had previously squatted on accounts like "VP" or
| "England"
| fknorangesite wrote:
| > It's against the Twitter terms to pay money for
| accounts.
|
| Ah, I didn't know that; then it's merely the _second_
| most boring, mundane way I would expect.
| minimaxir wrote:
| There's a few: https://www.rollingstone.com/culture/culture-
| news/twitter-vi...
| tinyhouse wrote:
| VITs are verified accounts.
| culi wrote:
| what is vits? private val DarkRequestAnnotation
| = "clnt/has_dark_request" private val Democrats =
| "democrats" private val Republicans = "republicans"
| private val Elon = "elon" private val Vits = "vits"
| camjohnson26 wrote:
| Here's a function definition: /**
| * This function returns the top most followed and verified
| userIds truncated to topK */ def vits(
| nanidin wrote:
| Very Important Tweeters, a play on VIP perhaps?
| stefanos82 wrote:
| Very Important Tweets, I presume?
| mshafrir wrote:
| Very Important Tweeters
| spyder wrote:
| Very likely this:
| https://create.twitter.com/en/articles/best-
| practice/2021/pa...
| afrcnc wrote:
| As a European I find this very offensive
| commandlinefan wrote:
| At first I thought this post was a joke - and it was actually a
| pretty good joke. Yikes.
| schemescape wrote:
| Did they not expect people to notice suspicious code like this?
|
| Or did they leave this in just so they could hold its removal
| up as an example of listening to the community?
| bitshiftfaced wrote:
| Could be that those in charge of preparing this open sourced
| repository did it begrudgingly, and so they perceived the
| fact that it looked bad as a positive thing. "Hey, you wanted
| us to release the code. Happy now?"
| dmix wrote:
| Why are you assuming this knowledge is harmful to them? What
| do you think it means for their business?
|
| No other social media platform will have this sort of
| accountability and public pressure to be better like having
| their recommendation algorithms public.
| minimaxir wrote:
| The full list of model features in that file is interesting.
|
| I am surprised at the number of inherently redundant and
| colinear features, though. (e.g. has_1_image, has_2_images,
| has_3_images, has_4_images)
| qyph wrote:
| Those aren't redundant or collinear though? Maybe you are
| surprised they didn't encode this as an integer "num_images"?
| It is fairly common to one hot encode ordinal variables with
| only a few common/possible values this way.
| minimaxir wrote:
| True, it still seems odd to encode an explicitly ordinal
| variable as categorical (particularly one with a small
| finite range, in contrast to the follower logarithmic
| bucket ones), but Twitter's layout is weird enough that it
| could be a impactful difference in terms of engagement.
| disgruntledphd2 wrote:
| This is (weirdly) common in production ML codebases
| written by software engineers. Like you, I have no idea
| why unless it's a memory optimisation (where you count 4+
| as many).
| ladon86 wrote:
| Having every column as a boolean (0/1) means you can
| treat it as a bitmap. As an (entirely fictional) example,
| imagine if you wanted to get the features of a thread
| instead of a single tweet. You could do it as a union of
| all the tweets:
|
| threadFeatures = tweet1 | tweet2 | tweet2
| disgruntledphd2 wrote:
| Ok that makes lots of sense from an engineering
| perspective. It's pretty insane from a statistical
| perspective though, which I think was the original point.
| nomel wrote:
| > It's pretty insane from a statistical perspective
| though
|
| Efficiency is _way_ more of a concern, at this scale,
| than the more trivial was of trying to find a competent
| person that wouldn 't misuse the values.
| nelox wrote:
| It's April Fools' Day where I live.
| fnands wrote:
| Legit was my first thought when I saw the headline. But
| apparently it's real
| nothngssimpl wrote:
| ,,This commit does not belong to any branch on this repository,
| and may belong to a fork outside of the repository."
|
| Isn't this top of the page disclaimer relevant for you? It
| seems not to be part of the main branch.
| sschueller wrote:
| Well someone just asked about it in the live spaces[1] Elon is
| hosting and he said that should not be there. An engineer said
| afterwards it is just for metrics but then Elon chimed in again
| and said "we should get rid of it, it should be gone."
|
| [1]
| https://twitter.com/elonmusk/status/1641880448061120513?s=20
| JustSomeNobody wrote:
| Doesn't necessarily mean he didn't want it there in the first
| place. Why else would it be there?
| human wrote:
| If I were Elon, first thing I would have done is ctrl+f
| "elon"...
| JoshCole wrote:
| There are millions of lines of code existent. There are
| thousands added. Your prior for it being Elon's addition
| should be something like 10,000/1,000,000 or roughly 1/100.
| The prior that it wasn't Elon's change is going to be
| something like 99/100.
|
| When you add the additional information that Elon wants the
| code removed, but existing Twitter engineers think it
| appropriate to keep this actually increases the probability
| of it being added by the existing Twitter engineers and
| decreases the probability it was added due to Elon.
|
| Obviously, these are rough numbers, but hopefully seeing
| any numbers at all helps you to get an intuition for the
| math.
| jsnell wrote:
| Why is lines of code the appropriate input here? Here's a
| different computation that is at least as plausible:
|
| There are hundreds of millions of users. Let's say 300M.
| Only a single one is special-cased in this code: the
| narcisistic CEO who reportedly went ballistic when his
| engagement metrics went down. The prior that it's a
| change done in response to his demands is
| 299999999/300000000.
|
| (But of course it was added by existing Twitter
| engineers. The odds of Musk being able to actually commit
| code to their repository are zero. Even if he had the
| permissions, the man simply does not have the technical
| acumen to make even a trivial change.)
| v0idzer0 wrote:
| This is a weird nitpick that doesn't attack the argument
| at all. It's a dense codebase but whatever measure you
| personally prefer
| jsnell wrote:
| JoshCole is doing a computation to arrive at the
| conclusion that there's a <1% chance that this code was
| added after Musk bought Twitter. I'm using the same
| methodology with at least equally plausible inputs to
| arrive at there being a >>99% chance of it.
|
| How is that a nitpick? They're diametrically opposite
| results.
| JoshCole wrote:
| I'm JoshCole and I didn't find your reply to be a
| nitpick; you are right that the probability ought to be
| higher than 1%. My calculation was simplistic and I felt
| it was prudent to arrive at low probability, because I
| think probability of wanting something given claim of
| wanting it removed should probably not be anywhere near
| close to certain. My estimate isn't 1% though. It was
| just a short thing to share that gave an intuition for
| why it might be reasonable to assume he didn't know or
| want it.
|
| In my opinion if you really care about this topic the
| right thing to do is ask someone at Twitter when the
| change was made. Getting more information would make us
| converge on the true estimate faster than arguing the
| odds IMO. Feel free to update me with the results if you
| do end up doing that so I can adjust my beliefs
| accordingly. I'm not going to try to gain this
| information, because I don't think the question matters
| much.
| JoshCole wrote:
| I think you are right, my estimate is much much too low.
|
| I'll explain my mistake and why I made it.
|
| I think I thought to use the estimate I did because Elon
| claimed he didn't know. The prior probability of not
| knowing something in a code base with millions of lines
| is very high, but contingent on his involvement in the
| change the prior that he is aware of it is much higher.
| So I started the estimate attempt with the probability
| that I thought better predicted the production of
| evidence claiming he didn't know.
|
| Your point does raise my estimate substantially, but I
| think it probably raises it less than you would expect. I
| don't agree with your 1/300M prior, because I'm aware
| that hot users get special treatment. I've seen Elon's
| account thrown around in interview-style questions about
| hot users before and used as an example of a hot account
| that needs special treatment. This is something I've
| witnessed, but it wasn't contingent on Twitter being
| acquired and it happened prior to Twitter being acquired.
|
| I also don't particularly assign high odds to wanting it,
| based on the evidence that he claimed to not want it
| implicitly by wanting it removed. I don't think it seems
| appropriate to get to near certain probability the he
| wanted it with the evidence being that he stated that he
| didn't want it. In my view there isn't a compelling
| reason for him to lie about this. He owns Twitter, so if
| he wanted them to have his account monitored that would
| be a reasonable thing well within his authority. If he
| wanted it, he doesn't need to pretend to not want it in
| order to appease someone.
|
| It does seem to me that the odds that the change was
| added in response to someone thinking he wanted it is
| much higher than 1%.
| jsnell wrote:
| > I don't agree with your 1/300M prior, because I'm aware
| that hot users get special treatment.
|
| That's absolutely fair, and 1/300M was a reductio ad
| absurdum rather than a serious proposal. Not all users
| are equal, just like not all lines of code are equal :)
|
| I have a few issues with the "hot user" theory, but they
| all boil down to the same point: no matter what the use
| case, you'd never want to do this with a single static
| user.
|
| Does your infra require special-casing for accounts with
| more than 100M followers? That should be a flag in the
| account properties that gets flipped manually or
| automatically: if these users cause infra problems, you
| really don't want to be making code changes + full
| rollouts whenever a new user becomes hot.
|
| Is this just a guard-rail metric, to make sure there's
| not some bug specifically affecting hot users that tanks
| their engagement? You'd want a much larger static set
| than a single account just to ensure there's a large
| enough number/variety of tweets to compute metrics from.
| A single user might take a break for a week, or might
| only be posting very specific kind of content for an
| extended period of time.
|
| In any case, even if you chose to do this with a single
| user rather than a set of users, why would Musk be the
| obvious single choice? He wasn't the most followed
| Twitter account until two days ago. A year ago there must
| have been at least a couple of dozen accounts roughly as
| notable as Musk. The odds of him having been chosen as
| the special case still would not be very high.
|
| > In my view there isn't a compelling reason for him to
| lie about this.
|
| The reason to lie about this is that it makes him appear
| weak, needy, and a target of even more mockery. Given the
| purchase of Twitter seems to have been a vanity project,
| having this be exposed and leaving it in goes directly
| against his apparent goal.
| JoshCole wrote:
| > The reason to lie about this is that it makes him
| appear weak, needy, and a target of even more mockery.
| Given the purchase of Twitter seems to have been a vanity
| project, having this be exposed and leaving it in goes
| directly against his apparent goal.
|
| I think it only makes sense to think like you are if
| you've adopted equilibrium assumptions; if you haven't
| then I find this sort of reasoning to be a conjunction
| fallacy causing an epistemic closure.
| v0idzer0 wrote:
| The same reason there are ones for other people? When they
| make changes, they want to see how it affects the
| visibility of their most popular accounts
| [deleted]
| XorNot wrote:
| Of course he did because it makes him look bad and he's
| desperate for praise and attention.
|
| What he wanted was everything that feature provides, without
| it ever being shown that it's there. But since he refuses to
| hire PR people and almost certainly came up with this idea in
| the last few days, no one was paid to hide its existence.
|
| The next story out of Twitter will be the remaining engineers
| being threatened because Musk can't see his tweet statistics
| any more.
| BurningFrog wrote:
| Elon does seek attention, but he seems to enjoy conflict
| _far_ more than praise!
| nonethewiser wrote:
| This is pure speculation.
| CathalMullan wrote:
| Only used for metrics, apparently. [0] /**
| * These author ID lists are used purely for metrics collection.
| We track how often we are * serving Tweets from these
| authors and how often their tweets are being impressed by
| users. * This helps us validate in our A/B
| experimentation platform that we do not ship changes *
| that negatively impacts one group over others. */
|
| [0]: https://github.com/twitter/the-
| algorithm/blob/7f90d0ca342b92...
| roughly wrote:
| I expect they're tracking the red team/blue team metrics
| because of the political shitstorm that's been the GOP's
| assertions they're being silenced by The Algorithm.
| panarky wrote:
| The fallacy of false equivalence systematized in code.
|
| Now one side can spew as much disinfo and incitement to
| violence as it likes, and any algorithm change that
| prevents this shit from getting amplified will be rejected
| as bias.
|
| BSaaS = Both Sides as a Service
| [deleted]
| electrondood wrote:
| Not sure why you got downvoted, you're absolutely
| correct.
| slg wrote:
| This shouldn't really be a surprise to anyone. It was
| reported years ago that Twitter was unable to cut down on
| hate speech because the automated systems they developed
| triggered too many [debatably false] positives on
| Republican politicians and that was bad for the company's
| reputation. If Twitter wanted to prevent future code
| changes from impacting that approach, there needed to be
| something like this in the code or tests.
| lenkite wrote:
| The Twitter files over the last several months has
| comprehensively shown the selective and explicit, human-
| enforced banning of conservative accounts.
| slg wrote:
| I don't know what specific documents you think did that,
| but "comprehensively" is absolutely an awful way to
| describe the Twitter Files. They were anything but
| "comprehensive". In actuality, they were an excellent
| example of how easy it is to lie using partial truths.
| For example, highlighting all the times Twitter took
| moderation recommendations from a Democratic campaign
| looks a lot worse if you hide any time they took
| moderation from Republican campaigns. A simple look at
| the specific journalists that were given access to
| Twitter documents and the strings attached to that access
| reveals that the Twitter Files were not about
| transparency. They were an ideological play and nothing
| more. If Musk wanted true transparency, he would have
| given wider access to more documents or just released
| them all like Jack Dorsey requested.
| lenkite wrote:
| Could you please point out the list of moderation
| recommendations from Republican campaigns that were
| actually accepted and carried out? The only ones that
| were revealed so far were of the following types:
|
| 1. requests to un-ban or un-suspend right-wing personas
|
| 2. removal of explicit death threats.
|
| 3. Anything against the vaccine - even actual, scientific
| data.
|
| From the twitter files, the Twitter team that was wholly
| left-leaning spent significant time and debate looking
| for the mildest of excuses to ban right-wing politicians,
| building black lists for them and even gloating happily
| as they managed to kick them off. Not much doubt about
| that - actually no one has even denied that. There were
| several hearings also in the house judiciary where they
| even _confirmed_ the same.
|
| (Interestingly, there was pressure applied on Matt Taibbi
| to either "shut up" or relinquish all his sources to law
| enforcement. Also, as a form of indirect pressure to
| rattle him, US tax agents visited his house the VERY DAY
| he would testify before US Congress stating that his tax
| returns had been rejected due to identity theft concerns
| - despite him having the electronic receipt which showed
| it being accepted.)
| slg wrote:
| >Could you please point out the list of moderation
| recommendations from Republican campaigns that were
| actually accepted and carried out?
|
| That is exactly my point. I can't because none were
| released. That is not a reason to assume they don't
| exist. You are assuming full transparency in a situation
| with only partial transparency.
| nicky0 wrote:
| At the same time, there is also no reason to beleive they
| do exist.
| blitzar wrote:
| https://www.rollingstone.com/politics/politics-news/elon-
| tru...
|
| WHEN THE WHITE House called up Twitter in the early
| morning hours of September 9, 2019, officials had what
| they believed was a serious issue to report: Famous model
| Chrissy Teigen had just called President Donald Trump "a
| pussy ass bitch" on Twitter -- and the White House wanted
| the tweet to come down.
|
| That exchange -- revealed during Wednesday's House
| Oversight Committee hearing on Twitter by Rep. Gerry
| Connolly -- and others like it are nowhere to be found in
| Elon Musk's "Twitter Files" releases
| dudeinjapan wrote:
| Huh? What does this prove? Trump's WH may have
| _requested_ Teigen 's tweet to be removed, but it's still
| there: https://twitter.com/chrissyteigen/status/117091414
| 8919590914
| blitzar wrote:
| > there is also no reason to beleive they do exist.
|
| Proof of reason to _believe they do exist_.
|
| Furthermore, the office of the president, the FBI, the
| Government making a request to limit the free speech of a
| citizen is entirely different from Joe Blow in the street
| making a request.
| TylerE wrote:
| Sure there is. A biased source paid a journalist to be a
| sheep and repeat what he was told.
| ModernMech wrote:
| Maybe you missed this news story: Twitter
| admits bias in algorithm for rightwing politicians and
| news outlets[1] The research found that in six
| out of seven countries, apart from Germany, tweets from
| rightwing politicians received more amplification from
| the algorithm than those from the left; right-leaning
| news organisations were more amplified than those on the
| left; and generally politicians' tweets were more
| amplified by an algorithmic timeline than by the
| chronological timeline. According to a 27-page
| research document [2], Twitter found a "statistically
| significant difference favouring the political right
| wing" in all the countries except Germany.
|
| Twitter admits it was boosting right wing accounts over
| left wing accounts. It treated right wing twitter users
| who violated its own terms of service _better_ than
| regular users because of their position as conservative
| leaders. Trump himself was boosted and promoted while he
| violated Twitter TOS. It took him using Twitter to wage a
| coup against the US government for them to ban him. So
| yeah, there 's more than a little reason to believe that
| there's evidence at Twitter of them boosting right wing
| accounts.
|
| And PS: Notice that this research by Twitter is never
| mentioned by Musk or in the Twitter files. They're trying
| very hard to memory hole this report, and it seems like
| it's worked on you.
|
| [1] https://www.theguardian.com/technology/2021/oct/22/tw
| itter-a...
|
| [2] https://cdn.cms-twdigitalassets.com/content/dam/blog-
| twitter...
| nicky0 wrote:
| But that was published while the pro-Democrat
| establishment goons were in charge of Twitter, so of
| course they would say that because they wanted you to
| beleive they weren't favouring the left. It's hardly a
| reiable source. Same applies to Musk-era of course.
|
| In the end, we accept the things we agree with, and
| discredit the ones we prefer not to believe. Human
| nature.
| ModernMech wrote:
| As I said, we don't need to know anything internal about
| Twitter to know for a fact that they bent over backwards
| to elevate extremist right wing voices. Because they
| elevated the most important extremist voice for 4 years
| as he violated the terms of use for their platform. In
| effect, we all had one TOS, and there was a different,
| more permissive TOS for the leader of the US MAGA-right
| wing movement, the most extreme version of right wing
| politics in America. That's blatant, confirmed,
| irrefutable, systematic bias and special treatment for
| conservatives on Twitter, pre Musk.
|
| In case you want to discount the Twitter report as a
| false flag, here's a report from NYU that independently
| confirmed Twitter is biased toward conservatives. It also
| found the same for Facebook:
| https://bhr.stern.nyu.edu/bias-report-release-page
| nicky0 wrote:
| American universites are notoriously politically biased.
| Calling that "independent" is a stretch.
|
| Again - we accept evidence that supports what we already
| believe.
|
| https://theoatmeal.com/comics/believe
| ModernMech wrote:
| You really can't just blanket claim that all research
| from a university is politically biased, even if you feel
| American universities in general are biased. Come on now,
| we're here for discourse that's a little more nuanced
| than that I hope.
|
| Since all you're bringing to the discussion at this point
| is a comic, I think that's all there is to say.
| prezjordan wrote:
| This is a lie. We got a glimpse into how accounts like
| LibsOfTikTok regularly set off alarms only to get tags
| such as "Do not ban. Ask first."
|
| The only thing selective and explicit was the information
| Musk allowed bought-and-paid-for "journalists" to access.
| [deleted]
| eldritch_4ier wrote:
| [flagged]
| [deleted]
| willdr wrote:
| What are you talking about? Do you remember January 6?
| eldritch_4ier wrote:
| [flagged]
| panarky wrote:
| Everybody remembers January 6 except those who want to
| pretend it didn't happen.
|
| How many remember the floods of Twitter incitement to hit
| the gas in their F-150 trucks to run over protesters, and
| then how many people actually perpetrated vehicle
| attacks?
| JackAndJack wrote:
| [flagged]
| zeven7 wrote:
| I clicked downvote before reading the whole comment. At
| first I thought you were talking about the red team when
| you started with "mass riots and violence..." Then I read
| the rest of your comment and still felt just as good
| about my downvote. This isn't a constructive comment no
| matter what "side" you're on.
| luxuryballs wrote:
| "Without debate, without criticism, no Administration and
| no country can succeed--and no republic can survive. That
| is why the Athenian lawmaker Solon decreed it a crime for
| any citizen to shrink from controversy. And that is why
| our press was protected by the First Amendment-- the only
| business in America specifically protected by the
| Constitution- -not primarily to amuse and entertain, not
| to emphasize the trivial and the sentimental, not to
| simply "give the public what it wants"--but to inform, to
| arouse, to reflect, to state our dangers and our
| opportunities, to indicate our crises and our choices, to
| lead, mold, educate and sometimes even anger public
| opinion." JFK
| regularjack wrote:
| "approved terminology".
| KarlKemp wrote:
| Well, technically they are looking for relative changes,
| not equal total exposures.
| concordDance wrote:
| I don't see an unbiased way to tell which "side" releases
| more disinformation and incitement to violence. Even
| deciding what counts as disinformation is hard (e.g. does
| it have to be literally false or just cause false beliefs
| in the reader?).
| ModernMech wrote:
| One way to tell would be to look at which side is incited
| to violence more often.
|
| It turns out, according to the FBI (which is a
| conservative organization historically and exclusively
| run by conservatives), right wing extremism and violence
| is in fact the biggest domestic terror threat in the US,
| and it's currently growing [1]. FBI Director Wray gave
| this testimony after a right wing domestic terror attack
| was carried out that aimed to topple the US government.
| Not much has changed since then [2]. Since the former
| President's indictment the other day, the right-wing
| violent rhetoric has also ratcheted up a notch, so we can
| expect right-wing violence to follow.
|
| Notably, we can confidently say this doesn't happen on
| the left, as when Hillary lost they did not launch an
| assault against the Capitol as the right did. Instead,
| they knit pink hats and had a march.
|
| (PS before anyone whattabouts the George Floyd protests,
| the FBI doesn't see them the same way [3])
|
| https://apnews.com/article/fbi-chris-wray-testify-
| capitol-ri...
|
| https://news.yahoo.com/right-wing-extremists-responsible-
| for...
|
| https://www.fbi.gov/file-repository/fbi-dhs-domestic-
| terrori...
| yeahsure22 wrote:
| [dead]
| lenkite wrote:
| "Now BOTH sides can spew as much disinfo and incitement
| to violence as it likes"
|
| Fixed.
| dudeinjapan wrote:
| Yeah but despite all that they should still give leftists
| a platform to tweet.
| sva_ wrote:
| Ahh, the group of Elons.
|
| I was wondering why I see so many tweets by him, and what his
| "Group's" impression quote is.
|
| This is actually pretty hilarious.
| wardedVibe wrote:
| Thankfully they haven't added a "no mute Elon" feature.
| Yet.
| ModernMech wrote:
| They don't have to, because he effectively can't be
| muted. People tweet quote him with an image, and it's not
| blocked even though I have him blocked as an account.
| This behavior is pervasive enough that you can still see
| his tweets all the time.
| nailer wrote:
| That's not how it works. See the parent.
| sva_ wrote:
| They said that they use it for metrics, so clearly there
| must be an "elon impression" metric.
| ImPostingOnHN wrote:
| I imagine it's the largest metric in a mission control
| style room
|
| it starts dropping, klaxons start blaring, the room drops
| to red only lighting, engineers on the floor start
| pulling out their hair knowing the shitstorm that's
| coming
| nailer wrote:
| Yes, it's not making anyone see any extra Elon tweets as
| your comment alleged.
| jonny_eh wrote:
| It means they won't ship features that hurt Elon's reach.
| So in a sense, it is biasing code changes in favour of
| Elon.
| nailer wrote:
| Why does it mean that?
| andy_ppp wrote:
| But who chooses the users to be metrics...
| infogulch wrote:
| So many unnecessarily cynical takes here. Let's say you were
| in charge of a large legacy system that some segment of
| customers complain about it not working for them as well as
| other segments. How would you know whether their complaints
| are valid unless you measured it? You have to _know_ first.
| So measure it.
| xmprt wrote:
| I understand the value of measurements but how does
| measuring tweets from an individual user help?
| dorkwood wrote:
| If engagement on the tweets of that user goes down after
| a change has been implemented, you can roll back the
| change to prevent that user from being negatively
| impacted.
| nrb wrote:
| What if engagements around that user naturally declined,
| perhaps due to that user going off the deep end. Wouldn't
| this just serve to bias the algorithm toward propping up
| the exposure of that user? Do they even care about the
| control so long as that user's engagement is up and to
| the right?
| Jorge1o1 wrote:
| You do an A/B test so even for the same tweet or same
| time period, you're just comparing the new, "treatment"
| group against the old "control group."
| v0idzer0 wrote:
| Come on think this through. It's trivial to tell the
| difference between a gradual and natural decline and a
| drastic decline immediately after rolling out a change.
| Especially when the change is rolled out region by region
| and only exists in regions running the update. You have
| to be able to measure the effect of changes and the most
| popular accounts are the obvious low hanging fruit for
| doing that.
| TigeriusKirk wrote:
| He's very likely the user on the platform with the most
| engagement, and probably by a long distance.
|
| From that viewpoint, it does make some sense to use his
| account as measurement point.
| marricks wrote:
| Elon has said he wanted more reach and experimented with
| getting it. He also loves attention and tweets a ton.
|
| There's benefit of the doubt then there's just... whatever
| the polar opposite of that is.
| nwiswell wrote:
| The penalty of the certain
| ModernMech wrote:
| Presumption of guilt.
| davesque wrote:
| Yeah, but then what do you do after you measure it?
| Nothing? No, you make decisions differently so as not to
| offend whoever is part of the criteria. For example, can we
| agree that we don't want an "author_is_flat_earther" flag?
| Because who gives a shit if Twitter makes a change to their
| recommendation engine that negatively affects flag
| earthers? Just because something is only used for A/B
| testing doesn't make it completely inert.
| blackoil wrote:
| flat_earther can be a a proxy for conspiracy theories.
| They may want to know if there is a swing.
| gpm wrote:
| There could be an argument for an "author is a flat
| earther" flag with the intention of those tweets being
| repeated by twitter _less_.
| scott_w wrote:
| Do you think Elon bought Twitter so he can bury his own
| tweets?
| just-tom wrote:
| Actually, that's a refreshing take.
|
| (No).
| slg wrote:
| You can dismiss the complaint without measurement if you
| are confident in two things:
|
| 1. Your system does nothing to actually segment this
| specific group by their identity.
|
| 2. You are confident that the systems you have set up to
| reward good behavior and punish bad behavior are accurate.
|
| If both of those are true, you know that even if the group
| is being disproportionately negatively impacted by some
| form of recommendation/moderation, that it is only because
| that group disproportionately participates in behavior that
| is bad for the platform. That isn't a problem. It would
| actually be worse for the platform overall if you did
| anything to appease that group.
| jurassic wrote:
| > ...it is only because that group disproportionately
| participates in behavior that is bad for the platform.
| That isn't a problem.
|
| That is exactly what Twitter's stance has been all along
| (in the pre-Elon era) and it IS a problem for the product
| because people being silenced due to their own bad
| behavior (example: misgendering transgender people) feel
| an injustice is being done. The rule-makers get to set
| the range of acceptable discourse on Twitter and those to
| the right of center have felt unfairly disadvantaged by
| the way it was done in the past.
|
| Over time this has eroded trust in the product. Just
| because people aren't being labeled and ranked based on
| whether they are red team or blue team, the people
| deciding what "good" and "bad" behavior looks like on the
| platform have the power to disproportionately impact
| these groups.
| slg wrote:
| Data isn't going to tell Twitter whether to allow or
| disallow misgendering people. You either think that is
| bad behavior that shouldn't be allowed or you don't.
| Disallowing it is not disadvantaging Republicans. It is
| stopping behavior Twitter has deemed is bad for the
| platform. As I said in point 2 above, either Twitter is
| confident in those decisions or not. Data is worthless
| when it comes to a moral decision like that.
|
| And if we accept hat Twitter believes (or more accurately
| did believe) that misgendering people is wrong, who cares
| whether people who want to do it feel an injustice is
| being done? Would anyone say that deleting spam is an
| injustice to spammers? You break the rules and you get
| punished.
| skissane wrote:
| > Data isn't going to tell Twitter whether to allow or
| disallow misgendering people. You either think that is
| bad behavior that shouldn't be allowed or you don't.
| Disallowing it is not disadvantaging Republicans.
|
| If one of the defining characteristics of a
| political/religious/cultural group is having a particular
| ethical view, then enforcing a contrary ethical view
| against them is disadvantaging them and discriminating
| against them. Now, it may in some cases be morally and/or
| legally _permissible_ , or even _justifiable_ ,
| discrimination, but it still is discrimination, and it is
| still disadvantaging them.
|
| > Would anyone say that deleting spam is an injustice to
| spammers? You break the rules and you get punished.
|
| Worldwide, many jurisdictions have laws against
| discrimination on the basis of religion; although it is
| less common, some jurisdictions also have laws against
| discrimination on the basis of political belief. A law
| prohibiting discrimination on some ground, is evidence
| that some people believe discrimination on that ground to
| be immoral. By contrast, I've never heard anyone suggest
| that spammers should constitute a "protected class", and
| I'm not aware of any jurisdiction which treats them as
| one.
|
| Some people believe that there is nothing morally wrong
| with discrimination on the basis of religion and/or
| politics. Other people think there is something morally
| wrong with it, but if there is a conflict between the
| right to be free from religious and/or political
| discrimination, and the rights of LGBT people, the rights
| of the latter morally ought to take priority. Spam is
| irrelevant to that ethical debate.
| slg wrote:
| >If one of the defining characteristics of a
| political/religious/cultural group is having a particular
| ethical view, then enforcing a contrary ethical view
| against them is disadvantaging them and discriminating
| against them.
|
| I don't think misgendering people is a "defining
| characteristic" of Republicans and if that is, the
| Republican Party is in a pretty sad state considering all
| the bigger problems in the world. And if that qualifies
| as a "defining characteristic", there are plenty of other
| counter examples of society accepting discrimination as
| you define it. Banning polygamy would be discriminatory
| against Mormons is one. You could even argue that a full
| abortion ban is discriminatory against Jewish people.
|
| >some jurisdictions also have laws against discrimination
| on the basis of political belief.
|
| Notably not in the US where Twitter is based and were
| most of these complaints originate.
| skissane wrote:
| > I don't think misgendering people is a "defining
| characteristic" of Republicans and if that is, the
| Republican Party is in a pretty sad state considering all
| the bigger problems in the world.
|
| Some religious conservatives are convinced that referring
| to a transgender person by their preferred pronoun is a
| sin, even a serious one, for which they will be judged by
| God. Even religious conservatives who don't personally
| subscribe to that viewpoint, see it as one they are
| morally obliged to respect and defend. [0] While that
| doesn't describe all Republicans, obviously there is a
| significant overlap between Republicans and religious
| conservatives. And for a devout religious person, their
| religious beliefs are one of their defining
| characteristics-they are a huge part of their life, even
| their very identity, who they understand themselves to
| be.
|
| Even those conservatives who think it is okay to use a
| person's preferred pronouns, will adopt a much more
| restricted stance on the topic than many trans activists.
| Many will insist on it must be voluntary rather than
| mandatory, and defend the freedom of conscience of those
| who take a more conservative stance than they do - which
| is an expression of both religious and political beliefs
| about respect for individual freedom and conscience.
|
| Some conservatives are willing to use a
| friend/colleague/acquaintance's preferred pronouns when
| interacting with them, but will refuse to do the same for
| a criminal in the news. Look at Wikipedia to see people
| who vehemently insist that you must use the preferred
| pronouns of a dead school shooter or executed murderer -
| and even if the family of the victims publicly objected
| to it, that wouldn't change their mind.
|
| > Banning polygamy would be discriminatory against
| Mormons is one. You could even argue that a full abortion
| ban is discriminatory against Jewish people.
|
| Prohibitions on discrimination are never absolute, they
| always permit exceptions - so the existence of exceptions
| is not an argument against the existence of the
| prohibition. And whatever the merits of those specific
| examples, they are actions, not speech. Society
| traditionally gives religious minorities far greater
| latitude with respect to their beliefs about what they
| can and can't say, than their beliefs about actions which
| aren't predominantly expressive in character. Jehovah's
| Witnesses who believe it is a sin to salute flags or
| recite pledges of allegiance, Quakers who believe it is a
| sin to swear oaths, etc.
|
| > >some jurisdictions also have laws against
| discrimination on the basis of political belief.
|
| > Notably not in the US where Twitter is based and were
| most of these complaints originate.
|
| While the US currently lacks federal laws banning
| political discrimination, state and local laws sometimes
| do ban it, see [1]. Some of those laws are specific to
| certain contexts (e.g. housing or employment), and so may
| not be applicable to a social media platform such as
| Twitter. However, given increasing concern among
| conservatives about political discrimination, it seems
| rather likely that we'll see more state laws enacted on
| that topic in the future.
|
| [0] Examples: https://www.gotquestions.org/transgender-
| pronouns.html
| https://www.ncregister.com/commentaries/stand-fast-on-
| the-pr...
| https://seekersguidance.org/answers/modesty/should-i-
| honour-...
|
| [1] https://reason.com/volokh/2021/10/18/bans-on-
| political-discr...
| concordDance wrote:
| [flagged]
| jurassic wrote:
| Misgendering is shitty behavior in every version of
| English.
| concordDance wrote:
| Sometimes. Consciously using language that you know will
| offend another person and being uncaring of their
| feelings is shitty behaviour (unless you have a very good
| reason), using language that comes naturally and you
| think accurately describes the situation is neutral
| behaviour, making the effort to use validating language
| that people prefer is good behaviour.
|
| There's also the communication accuracy issue. Words can
| carry a host of connotations and implications (e.g.
| pronouns carry information (in the information theory
| sense of shifting the probability distribution) regarding
| how deep someone's voice sounds) and using words that
| give the listener an inaccurate impression is a cost,
| however minor.
|
| How much information is transmitted (and thus how big the
| "lie" is) depends on how different the male and female
| probability distributions (along many axes) are and how
| different they are to the transman and transwoman
| distributions. Which is obviously a very complex thing to
| figure out (are men and women identical other than outer
| appearance in X society? Obviously not, but how different
| they are is very unknown, for any X).
| slg wrote:
| I don't know why people need to make this topic so
| complicated.
|
| Imagine you meet someone named William. Maybe you become
| familiar with them and call them "Will" and they correct
| you and say they only want to go by "William". From that
| point forward, calling them "Will", "Willy", "Bill",
| "Billy", "Liam", or anything else besides "William" is
| rude. It doesn't matter that society at large thinks
| those names are all acceptable alternatives. It doesn't
| matter what is on this person's birth certificate. It
| doesn't matter the reason behind their request. An
| individual person told you their preferences and you
| ignored them. That is rude.
|
| Pronouns are the same. You call people by what they want
| to be called. Anything else is rude to that individual.
| concordDance wrote:
| The name analogy doesn't really work because in most
| societies names don't have very strong connotations. In
| ones where they DO have strong connotations (e.g.
| tribe/clan last names where different cland have
| different cultures) the analogy works, but most of us
| don't live in such societies.
| basisword wrote:
| Am I misunderstanding? Are you suggesting that penalising
| those who misgender transgender people is unfairly
| disadvantaging people whose political views are right of
| centre?
| jurassic wrote:
| It's not about what I personally believe or think is
| unfair. It's about what Republicans (broadly speaking)
| believe. There is massive resentment from people on the
| right who think the Twitter rules unfairly elevated some
| political opinions as good/correct/acceptable while
| treating others as unacceptable.
|
| The handling of trans issues is just one example to
| illustrate the problem here. People on the left think
| trans rights are human rights while people on the right
| think a lot of trans issues should be open for
| discussion, legislation, persecution, etc. I think if
| we're being intellectually honest most would acknowledge
| that as a country we are far from consensus on many of
| the details here (bathrooms, girls sports, etc), and yet
| Twitter's rules and enforcement actions behaved as if the
| leftist view of transgender people is the only valid and
| permissible view.
|
| The handling of January 6 and the banning of Trump is
| another example.
|
| These things are Rorschach tests; people apply their
| biases and reach very different conclusions about what
| should be done. I don't claim to know the solution, I'm
| just trying to sketch out the problem with the way things
| were creating a climate where a big segment of the US
| felt unwelcome and resentful toward the platform.
|
| This presents a problem for the platform, since you can't
| afford to alienate large double digit percents of the
| population if your mandate from shareholders is to grow
| mDAU by any means necessary. In that context, having some
| metrics tracking in place to measure the impact of
| algorithm changes on democrats and republicans to see
| whether impact is disproportionate is a completely
| rational thing to do.
| basisword wrote:
| While I can understand there may be debate around various
| trans issues purposefully calling someone something when
| they've politely asked you to call them something else
| isn't up for much debate. Seems like common
| courtesy/politeness.
|
| I see your point about losing users potentially but I
| would argue that Twitter's intense focus on the US (as
| shown by the democrat/republican metrics) and trying to
| placate everyone is actually a negative for their
| business. There's billions of other internet users
| outside the US. Shifting focus to serve them instead of
| focussing intensely on trying to please both sides in the
| US (and failing) would probably deliver better value for
| their shareholders.
| concordDance wrote:
| > purposefully calling someone something when they've
| politely asked you to call them something else isn't up
| for much debate. Seems like common courtesy/politeness.
|
| It can be if the thing they're asking for is perceived to
| be untrue and the person being asked is a big stickler
| for that sort of thing. If you'll excuse me using
| metaphors on this sensitive topic: if someone wants to be
| referred to as "His Majesty" but is not actually the king
| then while many nice people will indulge him[1], some who
| care a great deal about the "correct" usage of noble
| titles won't.
|
| [1] And it'll work out well!
| https://en.m.wikipedia.org/wiki/Emperor_Norton
| Timon3 wrote:
| But what if you flip the coin? What if the person being
| asked really, really thinks that the other person is a
| pissface, and is a big stickler for calling people he
| perceives to be pissfaces as such? Surely we can all
| agree that allowing them to follow their ideas will lead
| to bad discourse.
|
| And it doesn't even have to be such a crass example. What
| if a person is legally called Robert, but he's gone by
| Bob his whole life - is the stickler right if they insist
| on calling them by their legally given name?
| concordDance wrote:
| > Surely we can all agree that allowing them to follow
| their ideas will lead to bad discourse.
|
| I actually think that if someone genuinely thinks someone
| else is a pissface then it's their right to call the
| other person that. I also think that excluding such a
| person from your conversations may be sensible. This
| applies to pronouns too.
| ModernMech wrote:
| > There is massive resentment from people on the right
| who think the Twitter rules unfairly elevated some
| political opinions as good/correct/acceptable while
| treating others as unacceptable.
|
| Would it surprise you to find out that this resentment is
| in fact, conveniently manufactured, politically useful
| outrage? Because it's simply not true on its face, and
| the only thing we need to know to understand this is to
| see that it took Trump launching a coup to be banned on
| the platform. He violated the TOS every day, and he was
| allowed to spread his message to his millions of
| followers by Twitter. You want to talk about unfairly
| elevating political opinions? Trump used the platform to
| violate citizens' first amendment rights, and we had to
| take him to court to get those rights back. Twitter
| didn't do shit to protect us from him.
|
| But it's not just Trump. It's right wing political
| opinions writ large. Far and away from sinking right wing
| conservative voices, Twitter research found they actually
| _amplify_ right wing voices in every one of their top 6
| countries except Germany [1]. Yes, that includes the US.
|
| Is your mind blown? Have you heard of this once? I bet
| all you've heard from Musk and right wing politicians is
| that Twitter is going hard on conservatives and
| deplatforming them. Blocking their messages. Being unfair
| to conservatives and right wing opinions.
|
| Yet what has _actually_ happened? Twitter was actually
| _deferential_ to conservative voices! It boosted
| conservatives and right wing voices at the _expense_ of
| liberals. How did this happen? This is conservative
| messaging 101: complain about bias loudly enough and the
| other side will go so far out of their way to seem
| unbiased, they will be biased in the other direction.
| Conservatives managed to complain so loud about Twitter
| being biased against them that you not only believe it,
| but reality is actually _completely the opposite_.
|
| [1] https://www.theguardian.com/technology/2021/oct/22/tw
| itter-a...
| slg wrote:
| >It's not about what I personally believe or think is
| unfair. It's about what Republicans (broadly speaking)
| believe.
|
| No, it is about what Twitter believes. That was what I
| was referring to with point 2 in my original comment. Not
| every customer complaint is valid. It is ok to hear a
| complaint and dismiss it without further investigation.
| Twitter doesn't have some obligation to get all of
| society to think its rules are fair.
|
| It is fine for a company to tell some potential customers
| to "fuck off" as long as that company isn't
| discriminating against a protected class. Twitter isn't
| discriminating against a protected class here.
|
| If Twitter thinks misgendering people is wrong, it is
| impossible to come to an agreement with a group that
| think properly gendering people is wrong without Twitter
| compromising its own morals. Twitter is allowed to stick
| to its own morals and tell the people who disagree to
| "fuck off".
| iamacyborg wrote:
| Folks like Jordan Peterson would like you to think so,
| yes.
|
| Of course, the man is essentially a walking "old man
| yells at clouds" meme at this point, so I'm not sure you
| should take anything he says with any merit.
| yanderekko wrote:
| They obviously feel that it is.
|
| Likewise, if Twitter actioned people for saying "kill all
| men" or "all cops are bastards", this would be seen as
| having an obvious partisan impact.
| lazyeye wrote:
| I've never seen so many "experts" speaking from a position
| of complete ignorance than I do on Hacker News.
| mochomocha wrote:
| ... Metrics tracked in AB test. So even if it's not
| explicitly encoded in the algo (or implicitly through some of
| the features plugged in), they'll pick the winning cell as
| long as it doesn't hurt Elon's metrics (I'm just parroting
| the comment you quoted).
|
| It doesn't have to be in the algorithm for the systems to be
| tweaked to please Elon vanity metrics.
|
| [I've been running lots of ML AB tests over the years, some
| in organizations of similar size & complexity as Twitter]
| threeseed wrote:
| That lines up with reporting from Casey Newton a few days
| ago where a handful of VIPs e.g. Musk, LeBron James, AOC
| were being used as weather vanes to understand what the
| algorithm was doing.
|
| It definitely isn't just metrics. Any algorithm change that
| negatively affected Musk was clearly not going live.
| db48x wrote:
| Do you think the code looked like that prior to Elon's
| purchase? I suspect that there was another name there
| before.
|
| Separately, which of these groups do you think that they
| use as a control?
| threeseed wrote:
| > I suspect that there was another name there before
|
| Who ? Musk is unique in being obsessed with being liked
| and relevant.
|
| All of the other social CEOs including Porag and Jake
| have never really cared that much. And none of them
| participated in contributing content anything close to
| what Musk does.
| mattl wrote:
| I miss Jake
| ajmurmann wrote:
| Do you mean Jacky Dorsy?
| [deleted]
| hackerbee wrote:
| You mean Jack Dorsey?
| LightBug1 wrote:
| How easily we forget our temp workers ...
| [deleted]
| AzzieElbab wrote:
| Good old Duck Jersey
| varelse wrote:
| [dead]
| ajmurmann wrote:
| No, Marcy D'Arcy's cousin Jacky D'Orcy
| nailer wrote:
| Why is Musk obsessed with being liked? There was a hoax
| last month about the algorithm being tweaked to make
| everyone view Elon's tweets which was provably false.
|
| It is very much likely to be the CEO a company wanting to
| understand his companies product.
| tomberin wrote:
| We can't comment on why, but there's no rational way to
| watch his behavior and assume he isn't obsessed with
| being liked. He constantly tweets about his own tweets
| performance, makes humiliating appearances on stages, and
| pretty much terrified that someone might do something
| 1/100000th as awful to him as he and his family have done
| to others.
|
| he is isn't a CEO, he is a whining baby
| Fuzzwah wrote:
| Hard to see the difference between that or someone so
| obsessed with twitter that he paid a huge premium to own
| it.
| nailer wrote:
| Someone wants to measure their business after paying what
| was a reasonable figure when they offered it, but that
| became overpriced once the market changed.
|
| Sounds like fairly rational behaviour.
|
| HN isn't a site for making unfounded personal attacks on
| others.
| jdub wrote:
| It is very difficult to describe anything Elon has done
| to Twitter since (and including) buying it as "rational".
| somenameforme wrote:
| As a person who occasionally uses Twitter, but has no
| interest in having an account, he has dramatically
| improved the experience. Before he took over it was made
| impossible to view replies from a user, users were
| spammed with popup login dialogues that could not be
| closed with a single click, and more. I expect Twitter
| probably drove more people away from the site with all
| this nonsense, than they coerced into registering.
|
| I also think many of the changes to make Twitter more
| open, such as now open sourcing the recommendation
| algorithm, publishing view counts (which indirectly makes
| it clear when something or somebody has been shadow
| banned), and more are all big steps forward in creating a
| much better and open service for all. Really most of
| every change he's pursued since taking over Twitter, from
| my perspective at least, seems to have been big steps
| forward.
|
| What would you say has been irrational?
| status200 wrote:
| His offer included a meme number and was overvalued even
| in April '22 (by 54% !), claimed he was going to combat
| the spam bots, and then tried to back out, citing the
| same spam bots. The event was described as a "hostile
| takeover" at the time [0], a term which has stuck all the
| way through to most summaries of the event [1]
|
| [0] https://variety.com/2022/digital/news/elon-musk-
| offering-to-...
|
| [1] https://en.wikipedia.org/wiki/Acquisition_of_Twitter_
| by_Elon...
| somenameforme wrote:
| A hostile takeover has a specific meaning. It's simply a
| takeover attempt where a buyer approaches shareholders
| instead of the company's management, this is usually (if
| not always) because the latter fails. And it always comes
| with a premium and drama, precisely because it goes
| against the wishes of the management of a company. And
| the premium is also logical because presumably
| shareholders of a company "believe" in that company, and
| an outsider now wants to take it over. A good example [1]
| is when Inbev (massive multinational beer company)
| purchased Anheuser-Busch (Budweiser). It made the
| purchase of Twitter look downright cordial, but was
| probably outside of most of our bubbles.
|
| [1] - https://en.wikipedia.org/wiki/Anheuser-
| Busch#Acquisition_by_...
| nailer wrote:
| Yea, to make people buy your company you need to offer a
| premium. He backed out saying he was lied to about the
| current level of fake accounts, which is also reasonable
| if that was the case.
| jasonwatkinspdx wrote:
| I absolutely will criticize Elon for his behavior around
| the twitter buyout. He's handled layoff/firing decisions
| extremely poorly. He's alienated a large portion of the
| ad buyers. These are not unfounded attacks.
|
| Elon does not need you going around personally trying to
| shut down other people's discussion. That's cult like
| behavior.
| threeseed wrote:
| a) Musk is a public figure therefore personal attacks are
| legal and normal.
|
| b) Nothing I have said is untrue.
|
| c) This is not a normal way of measuring the health of a
| business. Standard metrics include EBITDA, DAU, MAU, LTV,
| CAC etc.
|
| d) This is not normal behaviour from a CEO. You don't see
| Mark Zuckerberg, Shou Zi Chew, Evan Spiegel etc using
| their products the same way Elon does.
| jrsj wrote:
| If the "normal way" of doing things worked then Twitter
| wouldn't have been in such a bad state when he took over
| in the first place. An unconventional approach brings
| more risk but can also end up being worth it.
|
| Would I do things differently if it were up to me? Sure,
| but it isn't, and I can at least appreciate that they are
| trying to move quickly & try different things. I'm
| reserving my judgement for whether or not this approach
| works in the long run.
| tomberin wrote:
| [citation needed] x 100
| freehorse wrote:
| That is "measuring their business" only if one of the
| main goals for their business is for it to promote them;
| because this is exactly what they measure here.
| nailer wrote:
| It's the CEO being able to write something and see where
| it ends up after going through the pipeline. It does not
| increase views. I can repeat this, other can repeat this,
| or you can look at the code.
| shanebellone wrote:
| He measured the business into halves; then lost one of
| those halves.
| api wrote:
| The alcoholic bought his favorite bar.
| debugnik wrote:
| > a hoax last month about the algorithm being tweaked to
| make everyone view Elon's tweets
|
| Didn't this just reveal that they're A/B testing on
| Musk's tweet performance? At the very least they're
| avoiding regressions, and I guess any incidental
| improvements to it won't be reversed, so isn't it
| fundamentally the same? Unless we're taking the word
| "everyone" literally, I guess.
| nailer wrote:
| Tabloid sites (Verge, Vice, etc) reported it as making
| Elon's tweets appear in everyone's timeline all based on
| the same blog article.
| tomberin wrote:
| not tabloids, not a hoax, did you go on Twitter during
| that period? I have him blocked and saw his tweets
| nailer wrote:
| Yes tabloids, both known for making up stories and
| sensationalism, all sourced from the same single blog
| article, easily confirmable looking as false at the view
| count for Elon's tweets, explained by engineers as
| exactly what we saw here (logging output not changing
| output) and I don't believe you re: seeing his tweets
| despite blocking him. Nobody else had that happen.
| jrsj wrote:
| I don't remember the specifics but I think there was a
| case where they accidentally amplified his tweets _too_
| much and they did reverse that change. He tweeted about
| it but I think it was a couple months back so it would
| take a bit to find.
| coldtea wrote:
| > _Who ? Musk is unique in being obsessed with being
| liked and relevant._
|
| This is a meta-level bias!
| KarlKemp wrote:
| There is absolutely no reason to believe there was
| another Single user getting this treatment before. The
| Elon-case was just copy & pasted as an ego-stroking hack.
| philosopher1234 wrote:
| i think he means trump, and i think it would've been
| strategically wise to give trump special treatment (tho
| probably not ideal)
| oildynastyb wrote:
| [flagged]
| db48x wrote:
| Actually, my first thought was the previous owner, or
| perhaps Twitter itself. But I left the question open,
| since it doesn't matter who.
| mynameisvlad wrote:
| Considering I had to look up who the Twitter CEO even was
| before Musk, I'm going to say that no, they didn't do the
| same thing to themselves.
| KyeRussell wrote:
| No. Not a chance. Elon's claims of mismanagement at
| Twitter have merit, though not nearly as much as he
| think, but the one thing that's undeniable is that the
| previous "owner" (well, the CEO), is an adult that didn't
| care about this shit.
| dudeinjapan wrote:
| Isn't it a problem if the CEO of Twitter doesn't care
| about his/her tweets, and the general public doesn't care
| either?
| mynameisvlad wrote:
| Why, exactly, should the CEO of Twitter care about their
| own tweets getting maximum attention? If anything, this
| takes engineering effort away from customers because
| special implementations like "is_elon" have to be added.
| threeseed wrote:
| The CEO of Twitter should be doing more important things
| than tweeting.
|
| And far more important things than worrying whether said
| tweets are popular or not.
| cma wrote:
| I bet the previous CEO never thought of firing an
| employee for a disability through a public tweet thread.
| winternett wrote:
| I think a lot of it is lies... I used to observe that
| tweeting anything remotely political went straight to
| trending timelines, so did tweets about crypto and
| NFTs...
|
| I doubt that Twitter and most of these social platforms
| are driven by Ai... Maintaining and changing complex
| algos could not be done on a rapid pace like what occurs
| now... I think Twitter has moderators, and scripts that
| control everything, and it can be more easily adjusted to
| tweak what is visible on the platform based on whatever
| agenda they want to represent (politics, revenue,
| PR/damage control).
|
| Twitter frequently bends the rules to serve celebrities,
| politics, and sponsors and for that they need to be able
| to quickly adjust scripts. True Ai is meant to function
| on it's own with minimal intervention, and therein a
| simple change to a massive logic scheme would completely
| FUBAR everything.
|
| I think there are rooms full of people that filter and
| either promote or suppress posts on Twitter every day,
| namely suppressing any tweets critical of the platform
| and it's owner... I've noticed that at night time hours,
| moderation of Tweets is less restrictive as a cue to what
| goes on there.
|
| There are certain topics that are not moderated as
| heavily as others (3d Design and other non-controversial
| topics), and some topics (for example music and porn)
| that are restricted heavily visibility-wise because they
| force people to run ads to rank in trending (because it
| generates a lot of opportunistic money for Twitter as a
| platform).
|
| Users that are critical of the platform and Elon and his
| allies (for example) can easily be neutered by moderators
| and put on shadow ban for any period of time in order to
| preserve the illusion of calm concerning Twitter
| operations. Complaints about Twitter only become visible
| when the majority of the audience tweets about problems
| (as that can't be moderated out without exposing
| moderation).
|
| It's pretty much all smoke and mirrors there in order to
| maintain order in my opinion, and it's pretty much futile
| and torturous to people who just want to create and seize
| opportunity for their business without spending tons of
| money on platform marketing...
| slim wrote:
| I don't think so. If the token was "author_is_jake" for
| example, it would have changed to "author_is_ceo" on
| second pass
| jefftk wrote:
| _> which of these groups do you think that they use as a
| control?_
|
| When you run an A/B test you randomly divide your users
| into groups, one (treatment) getting the new behavior and
| one (control) getting the current production behavior. So
| your question doesn't make much sense?
| db48x wrote:
| Depends on the question. If you want to answer a question
| like "does change X increase engagement?" then a straight
| A/B test works. But if you want to answer one like "does
| change X increase engagement while (favoring/not
| favoring) group 1 over group 2?", then an A/B test plus
| measuring groups 1 and 2 will not work without a control
| group, because without controls you don't know if any
| changes to engagement for your measured groups are
| significant. There is some threshold of change to the
| engagement for the measured groups which is too small to
| be significant, and you should ignore results that only
| measure that noise.
| underdown wrote:
| You're describing a multivariate test. Two groups under
| two different algorithm variants. Multivariate tests
| still require control groups.
| [deleted]
| throwaway202303 wrote:
| [flagged]
| btown wrote:
| It's just a two-pass EM (Elon Maximization) algorithm!
| minimaxir wrote:
| The original code is a part of the home-mixer service, which
| is the "Main service used to construct and serve the Home
| Timeline."
|
| I suspect the flag corresponds to weights not present in the
| repo.
| hn2017 wrote:
| Per original source, The code that was released today doesn't
| show the parts that actually alter the scores of Elon and
| other users. The part of the code referenced below just
| tracks Elon stats (from what I know). Employees removed most
| PII before the code was released.
| jasonhansel wrote:
| Interesting which "groups" they care about (e.g. mainstream
| political parties).
| [deleted]
| gigglesupstairs wrote:
| Wait, am I missing something here or author name is cleary
| mentioned as elon here while musk's twitter id it @elonmusk?
| Why is everyone assuming this code is about elon?
| CameronNemo wrote:
| Are you serious? What other Elon do you think it refers to?
| gigglesupstairs wrote:
| I mean yeah it must be elon we know but what I was mostly
| curious about was if it's actually meant for him only, why
| didn't they use his twitter handle? And just elon? I am not
| a developer I must mention it looks like. And I was
| genuinely curious.
| slim wrote:
| "author_is_elon" (notice the _ ) is a name the developer
| writing that code came up with for a category of twitter
| users. it has nothing to do with twitter handles. he
| could have chosen any name, but since that category
| includes only elon musk, he named it like that.
| minimaxir wrote:
| Update: Elon was asked about these in a Twitter Space, he says
| it's not appropriate and will be removed from the codebase.
|
| Additionally, from another Twitter engineer, the
| Democrat/Republican flags are apparently 10 years old and not
| important and do not have high feature importance.
| doomleika wrote:
| I would say it's who will be removed.
|
| Considering how Twitter is now getting a servance isn't that
| bad of an idea TBH
| lhnz wrote:
| I think the decade old comment related to a different part of
| the code regarding the number of followers you have in
| relation to the number of accounts you follow. (Everybody on
| the call wants to remove this: I wonder why they haven't
| yet.)
| delecti wrote:
| Chesterton's Fence. In a sufficiently large system, you
| should be hesitant to remove things unless you're sure you
| know why it was added, and all the things that have come to
| depend on it since.
|
| I've definitely been hesitant to remove things I was pretty
| confident weren't used anymore, just because I didn't want
| to deal with the repercussions if I was wrong.
| nonethewiser wrote:
| I didn't know there was a name for this. Thanks for
| sharing.
|
| I've definitely been bitten by this. You always have to
| weigh the chance you break something against the upside.
| If you're actually fixing a bug, fine. But just
| refactoring to make something cleaner? Or deleting
| because it seems like it's not doing anything, even after
| doing some research? Think again.
| djtango wrote:
| True for many things - doctors are now a lot more
| hesitant to remove the appendix for example. (it serves
| as a buffer for your gut flora iirc)
| sillysaurusx wrote:
| Elon seems embarrassed: https://twitter.com/elonmusk/status/1
| 641908130274525187?s=61...
|
| It'll be interesting to see what gets cut. Maybe just the
| Elon flag, but maybe others too.
| jrochkind1 wrote:
| Don't let Chesterton's Fence hit your butt on the way
| out...
| camjohnson26 wrote:
| What are the odds this flag was only introduced so that
| Elon could publicly fix it and give the illusion of
| transparency?
| evolve2k wrote:
| Elons reply "I only learned about it now!". What a crock of
| shit. We literally went through reporting a few months back
| where he was clearly instructing the team to make sure his
| tweets always come up, for everyone.
| dudeinjapan wrote:
| Yeah but presumably he didn't think the devs would tag a
| specific metric for "elon", which would later be open-
| sourced. It's one of those things that makes more sense
| in retrospect...
| GuB-42 wrote:
| I think it was more narcissistic than that.
|
| As in: "My tweets are very important, if they don't show
| up on top, it means the algorithm can't recognize what is
| important, it needs to be fixed". And the team, who
| probably didn't see in which way Elon Musk's tweets could
| be that important besides the fact that he wrote them,
| they just decided to give Elon Musk's tweets a boost.
| masklinn wrote:
| From what I understand of upthread, it doesn't "give elon
| musk's tweets a boost", rather any change which
| downgrades musk's visibility is considered a regression.
|
| Obviously the end result is similarly that musk's
| visibility can never decrease, but it's a more technical
| (and to the letter) compliance with the specifications.
| takeda wrote:
| The most hilarious things to me was when I was reading
| some (unrelated to this drama tweet) posted in an article
| and the top recommendation was Musk's tweet that he
| doesn't artificially promote his tweets.
| sawyna wrote:
| Now someone is surely getting fired.
| [deleted]
| cubefox wrote:
| > We literally went through reporting a few months back
| where he was clearly instructing the team to make sure
| his tweets always come up, for everyone.
|
| Feel free to give me a source for that, but I'm pretty
| sure that's not true.
| slim wrote:
| looks like evolve2k is himself the source. maybe he was a
| twitter employee ?
| icehawk wrote:
| ah cool thanks for figuring out they were the source of
| https://www.businessinsider.com/twitter-boosted-elon-
| musk-tw...
| evolve2k wrote:
| While it was thrilling for a moment to imagine myself as
| a disgruntled former Twitter staff member, that is not
| the case and I have not been employed there.
|
| My statement above was to read 'we' as in as the tech
| community not 'we' Twitter staff; others have quoted some
| of the sources on this thread, but it was definitively
| reported by a few different outlets at the time.
| csb6 wrote:
| It was widely reported in the days following the Super
| Bowl that he personally ordered his Tweets be boosted
| after a Tweet by Biden got more engagement than his
| similar Tweet [0].
|
| As far as I can tell, he did not instruct Twitter
| employees to make his Tweets appear to _all_ users, but
| he did want them to make his posts appear in timelines
| significantly more often. This lead to reports of users
| suddenly getting their timelines flooded with lots of his
| Tweets.
|
| It seems like there was a rush within Twitter to raise
| Musk's engagement numbers by altering the recommendation
| algorithm to specifically boost posts from his accounts.
| The special boost factor was later reduced, but allegedly
| still exists.
|
| [0] https://www.theverge.com/2023/2/14/23600358/elon-
| musk-tweets...
| hallqv wrote:
| Wrong. Elon noticed his tweets had low engagement
| relative to his follower count compared to other
| accounts.
|
| After some digging in the code base it turned out that
| there was a de-boosting factor based on the absolute
| numbers of blocks your account had, which affected
| popular and controversial accounts (like Elon's)
| unfairly.
|
| This investigation was initiated by Musk but it resulted
| in a great improvement in the algorithm with no special
| treatment for his account.
| csb6 wrote:
| That directly contradicts the article:
|
| > By Monday afternoon, "the problem" had been "fixed."
| Twitter deployed code to automatically "greenlight" all
| of Musk's tweets, meaning his posts will bypass Twitter's
| filters designed to show people the best content
| possible. The algorithm now artificially boosted Musk's
| tweets by a factor of 1,000 - a constant score that
| ensured his tweets rank higher than anyone else's in the
| feed.
|
| > Internally, this is called a "power user multiplier,"
| although it only applies to Elon Musk, we're told.
|
| It seems indisputable that Musk's account gets special
| treatment, even if some of the changes may also boost
| other controversial users.
|
| You did not reference any sources, while the article
| claims sources inside Twitter gave them this information.
| hallqv wrote:
| Elon spoke about this in the Twitter earnings call
| https://www.youtube.com/live/6bgCqskBmtk?feature=share
|
| Anonymous Twitter sources have zero credibility since a
| good portion of the employees still want to oust Elon.
| greenthrow wrote:
| Why are you so determined to carry water for someone who
| doesn't know you exist and wouldn't care if they did?
| rjzzleep wrote:
| I didn't use to like the guy and I'm neutral now, but I
| could return the same question to you that you asked the
| guy you're responding to.
|
| Why are you so determined to carry water for people that
| dislike Elon who don't know you exist and wouldn't care
| if you did?
|
| But all jokes aside, he quotes a real person, responding
| to an anonymous source. Why does that warrant a personal
| attack?
| yeahsure22 wrote:
| [dead]
| TheDong wrote:
| Anonymous Twitter sources may have _low_ credibility, but
| Elon musk has _exactly zero_ credibility at this point.
| He has a significant and proven track record of making
| false and misleading statements, even ignoring the many
| many more difficult to prove allegations.
|
| In my personal experience, software engineers, when you
| ask them about technical matters like "does the code do
| X" or "is there a condition for Y" are generally pretty
| honest. Programming is a task that requires being able to
| have structured and literal reasoning, and asking
| engineers about purely technical details usually triggers
| that response, even if you can imagine a social motive
| for lying.
| concordDance wrote:
| Note that some of the "false" statements later turn out
| to be true. E.g. holding child a last heartbeat happens
| (ex-wife held him for death rattle, elon for last
| heartbeat)
| hallqv wrote:
| Disagree, Elon has high credibility IMO, certainly higher
| than an anonymous source with incentives to slander. Not
| saying he is perfect but given the scrutiny he is under
| and how much he is "in the arena" pushing the envelope.
|
| Elon is furthermore a software engineer (he wrote ALOT of
| code at zip2 & paypal) so your argument falls flat in
| that regard.
| taormina wrote:
| Credibility from what exactly? Having last year's most
| expensive mid-life crisis? He once had some credibility,
| no one will deny that. Respect is earned and lost.
| hallqv wrote:
| I think him buying twitter was great. He is lightyears
| better than previous owners & management. He is also
| running two other companies doing extraordinary things in
| the world.
|
| Of all the people you can hate on in the world, hating on
| Elon is to me very odd. Tall poppy syndrome I guess...
| fzeroracer wrote:
| Musk did not write a lot of code at PayPal, that's
| factually incorrect. And he was ousted and replaced at
| PayPal because he was trying to make technical decisions
| that would've killed the company.
| HopenHeyHi wrote:
| [flagged]
| smcl wrote:
| Well that's what the claim was. I never bought it, to me
| it sounded very much like the "Dear Leader Kim observed
| during the factory tour that one of the machines was
| miscalibrated and that productivity could be increased
| tenfold". Remember that he'd _just_ fired an engineer on
| the spot for suggesting there wasn 't a technical reason
| behind his lower engagement, and that the unspoken reason
| was he's frankly just a bit annoying and his epic meming
| CEO shtick was wearing thin. So I think when he raises
| the issue again, they just implement `author_is_elon` and
| they get to tell him that actually he _is_ good at
| posting after all and there was just a weird technical
| issue they have now fixed.
|
| Elon's happy, the engineers he turned to kept jobs and
| any time I find myself in the "For You" tab I see Elon's
| memes and his @catturd2 RTs.
| that_guy_iain wrote:
| If I remember correctly, he noticed that his account with
| 100,000,000 plus followers was only getting 40,000 or so
| views. This lead to a Twitter engineer saying maybe his
| content wasn't interesting enough to get views and
| getting fired. And later on Musk said that they found a
| bug in the fanout system because his account was so big
| it basically broke things and they fixed it.
|
| I remember everyone agreeing with the Twitter engineer
| who said maybe his content wasn't interesting enough,
| while to me that seemed odd. If people follow an account
| and they tweet and they're online they should have a high
| chance of seeing that tweet. That's the entire point of
| following someone. If someone I've followed tweets
| something I would like to see it.
| RF_Savage wrote:
| The fanout bug was in weighting how blocks affect
| visibility and fanout. Apparently it was an absolute and
| not a relative thing, so the huge amount of blocks musk
| has affected their visibility.
| coldcode wrote:
| You don't, at least for me, I have 5700 followers and
| most posts (with art) only see 200-500 impressions,
| whereas a year ago I was getting 3000 on average. The
| Following tab is clearly (at least for me) not in
| timeline order, so most of my followers never even know I
| posted anything. Since I post art most of my followers
| are artists, so you would think they would see them.
| Impressions (which are Twitter putting tweets in a
| timeline so that people scroll by them) are not under the
| control of the poster. Likes per impression for me are
| up, so interaction is higher which is supposed to
| increase how often tweets are shown, but only timely
| retweets seem to help any.
|
| I am trying to figure out how the algorithm decides on
| what to show in the Following tab, but the code is way
| too big to analyze without being able to run it and look
| at logging/metrics/stats.
| lawn wrote:
| He's only upset that people found out about it.
|
| If they remove his artificial boosts, he'll just turn around
| and shout at his engineers to reinolement it in another way.
| iaseiadit wrote:
| If you buy a company for $44B and take it private, I for one
| say you should get your own flag.
| gregcoombe wrote:
| "It's one flag, Michael. How much could it cost, $44B?"
| girafffe_i wrote:
| [dead]
| maccam912 wrote:
| Just removed https://github.com/twitter/the-
| algorithm/commit/ec83d01dcaeb...
| camjohnson26 wrote:
| Clearly embarrassing. This is the third commit in the repo.
| MichaelMoser123 wrote:
| never seen a pull requests with so many comments, and where
| the comments to the pull request read like ... twitter :-)
| qwertox wrote:
| There are no guarantees that the actual code which is run
| will have this removed.
| cbeach wrote:
| The "author_is_elon" flag may have been assigned to him because
| Elon's Twitter account has the most followers on the platform.
|
| So, for technical / performance reasons, changes to the algos
| might want to be benchmarked against this account in
| particular, because it's the account most likely to be at the
| centre of capacity- / load-related issues.
| bdw5204 wrote:
| How hard would it be to replace this entire algorithm with the
| following pseudocode?
|
| If !user.follows_author(author) then don't show tweet on
| timeline Else if tweet.timestamp is later than all other tweets
| show tweet first
|
| This is vastly superior to any other possible recommendation
| algorithm because users can choose what tweets they see/don't
| see by whom they follow and everybody has an equal chance to
| have their tweets seen by their followers. When Twitter moved
| away from this, it rendered my timeline useless so I started
| just pulling up people's profiles to read their tweets in order
| and eventually deleted my (pseudonymous) account that had
| several thousand followers. Almost nobody was seeing my tweets
| anyway thanks to this algorithm and deleting the account did
| not prevent me from browsing accounts I'm interested in.
|
| All Elon needed to do to fix Twitter was to reverse all of the
| bad changes they've made since 2015 or so and restore the
| platform to what it was in the late 00s/early 10s.
| arcatech wrote:
| How is that a recommendation algorithm? The point of the
| "recommendation" part is to show you things you wouldn't
| normally see.
| AdamH12113 wrote:
| I read the code snippet before I saw the link and thought you
| were joking, but yeah, there really is an author_is_elon flag
| right there in the main branch.
| adharmad wrote:
| Also this: https://github.com/twitter/the-algorithm/pull/160
| camjohnson26 wrote:
| That's a pull request from a random person
| ibraheemdev wrote:
| > But we are deleting this bs. I only learned about it now!
| Will be gone by tomorrow.
|
| https://twitter.com/elonmusk/status/1641908130274525187?t=5t...
| hn2017 wrote:
| Per Zoe: The code that was released today doesn't show the
| parts that actually alter the scores of Elon and other users.
| The part of the code referenced below just tracks Elon stats
| (from what I know). Employees removed most PII before the code
| was released.
|
| https://twitter.com/ZoeSchiffer/status/1641902570921943044?s...
| RoyGBivCap wrote:
| Elon just said in the space "that shouldn't be there. Consider
| it gone"
| tomberin wrote:
| oh well he's a trustworthy guy that never lies in his tweets,
| fixed
| HAL3000 wrote:
| Expect to see A LOT more spam on Twitter after this release. It's
| like giving SEO spammers access to google search ranking
| algorithm.
| dmix wrote:
| Stuff like this always has consequences, it doesn't mean it's a
| net negative for society. It means you need to adapt and
| actually fix the problems, while also benefiting more from the
| accountability.
| dmix wrote:
| Stuff like this always has consequences, it doesn't mean it's a
| net negative for society. It means you need to adapt and
| actually fix the problems, while also benefiting more from the
| accountability.
|
| That's always been a risk of open source and not being hyper-
| centralized.
| Laaas wrote:
| Praise where praise is due. Wasn't completely sure whether they
| would in fact release it or keep posturing.
| bilekas wrote:
| I'm supposed to be going out in 20 mins....
| stusmall wrote:
| I thought it was an april fools joke when I saw this:
| https://github.com/twitter/the-algorithm/blob/main/ci/ci.sh
|
| Like a dig at the code quality.
| endorphine wrote:
| Is it not?
| robopsychology wrote:
| Why are there two spaces instead of four in this Python code, it
| hurts my soul
| xdennis wrote:
| Seems random. This file has both 2 and 4:
| https://github.com/twitter/the-algorithm/blob/main/trust_and...
| paulddraper wrote:
| I assume they copied it from Google.
|
| https://www.quora.com/Why-does-Google-use-2-spaces-for-Pytho...
| holler wrote:
| I guess they haven't read
| https://peps.python.org/pep-0008/#indentation
|
| "Use 4 spaces per indentation level."
| anigbrowl wrote:
| Cost saving measure. This sort of emotionalism is why engineers
| need to kept out of the C-suite.
| SpEd3Y wrote:
| Not sure if you're being sarcastic, but if you're serious,
| I'm pretty sure the OP is talking metaphorically. It's just a
| slight annoyance he's not "emotional" about it.
|
| I also fail to see how someone who is annoyed by code that
| doesn't follow well established standards is somehow not a
| good fit in the C-suite.
| brucethemoose2 wrote:
| Space bloat.
| aaa_aaa wrote:
| Then there is Go and C#.
| robopsychology wrote:
| How is it a cost saving measure? Or are you being sarcastic?
| Hard to tell over text!
| anigbrowl wrote:
| Yes, I'm joking. I also feel hurt by 2 space indents.
| robopsychology wrote:
| [flagged]
| tayo42 wrote:
| i think back in the day they copied googles python style guide
| Me1000 wrote:
| Squashing the commit history before releasing it was an
| interesting (and completely predictable) decision.
| hk__2 wrote:
| > Squashing the commit history before releasing it was an
| interesting (and completely predictable) decision.
|
| This is standard practice when it comes to open-sourcing such
| repos that were closed-source for years.
| jkubicek wrote:
| It doesn't seem particularly interesting? I would never make a
| formerly private repo public without first erasing the history.
| There's no upside to showing everyone your work in progress and
| almost unlimited downsides.
| tapland wrote:
| There's no way everyone had the same weight in all the
| recommendation config files.
|
| It's not about hiding old work, but changes just before
| making it public.
| mrguyorama wrote:
| If they allowed you to git-blame the algorithm, some poor coder
| would have definitely gotten murdered by a crazy person who
| thought they purposely changed something to hurt them
| rschjosgknvx wrote:
| [flagged]
| [deleted]
| abalaji wrote:
| huh, legit open source too with 'Affero-GPL'
| madeofpalk wrote:
| AGPL is probably useless for any other site who'll want to use
| it, as it would require them to open source their site that
| uses it.
| joeyh wrote:
| Mastodon is conveniently also AGPL...
| [deleted]
| timeon wrote:
| On reason I use Mastodon is that there is just
| chronological timeline. Quick scroll and you are done. Bad
| for advertising platform - good for user.
| madeofpalk wrote:
| I'm not sure why Mastodon would be interested in Twitters
| non-chronological timeline. It seems to be pretty
| antithetical to its goals.
| suddenclarity wrote:
| Twitter also has a chronological timeline nowadays?
| [deleted]
| [deleted]
| muratsu wrote:
| Given the complex relationship between advertisers, platform, and
| users I don't know if any meaningful contribution can be made to
| the algorithm without pissing anyone off. The following tab
| already gave people who're not interested in algo recommendations
| a way out. I don't quite understand the reasoning behind open
| sourcing the algorithm. Any thoughts?
| dools wrote:
| And yet my Twitter feed was always so boring.
|
| Reminds me of the Sirius Cybernetics Nutri-matic drinks machine.
| WhereIsTheTruth wrote:
| Is it even what they use in production?
|
| There is code that favor Elon's tweets so I'd yes that's probably
| what they use
| 0l wrote:
| > There is code that favor Elon's tweets so I'd yes that's
| probably what they use
|
| Where?
| zaroth wrote:
| Spoiler - there isn't.
| ftxbro wrote:
| Yeah they track author_is_elon, author_is_democrat, and
| author_is_republican but they don't appear to be used for
| favoritism anywhere in this code.
| WhereIsTheTruth wrote:
| Why do they exist then? No code references it, but that's
| Scala/JVM so many things depend on runtime
| initialization, so maybe some other systems do? wich
| ones?
|
| Is is it there to help fight impersonations? should be
| solved with Twitter Blue already?
|
| There was reports of people receiving notifications about
| Musk tweets despite not following him, so..
| ftxbro wrote:
| It's not used at run-time, it's in the repository so that
| the large language models that are training on the github
| corpus will know how special elon is, and so that the
| future code written for twitter by GPT-5 will take the
| hint and add the favoritism autonomously.
| WhereIsTheTruth wrote:
| Interesting argument, and definitely worth defending; an
| AI that's biased by design to remember and preserve the
| old world order's members rule and influence
|
| Begs the question, why make it obvious?
| sho_hn wrote:
| Humorous conspiracy theory: Imagine if it is not, but
| sanitized, and then someone added in Elon Boost to make it look
| credible. :-)
| WhereIsTheTruth wrote:
| Or perhaps it does nothing at all, and it was there so we
| talk about it, the "is_democrat"/"is_republican" is also
| ridiculous, as if the goal was to demonstrate a point about
| social media in general, hmm
| quotemstr wrote:
| Typically, we expect to be able to run "open source" software
| ourselves. If you open-source your C compiler, I can compile a C
| program with it. In a few recent high-profile cases though,
| companies have "open sourced" ML systems without releasing the
| model weights. This practice is just like your releasing the
| builds scripts for your C compiler, but not the compiler itself.
| While more transparency from social media will be enlightening,
| calling a release like this (or LLaMA) "open source" feels like
| equivocation. I'd love to see more full releases, weights
| included.
| vonmoltke wrote:
| Running this code would require a lot more than just the
| exported models. There are a large number of code and system
| dependencies missing.
| quotemstr wrote:
| Of course --- but without the model parameters, even stubbing
| those systems would be useless. My point is that while this
| release gives the public some information about how Twitter
| ranks tweets, it doesn't tell the story because huge pieces
| of "the algorithm" are missing. For example: the NSFW
| classifier "open source" release doesn't tell us anything
| about what Twitter considers NSFW and what it doesn't.
| bighoki288 wrote:
| [dead]
| varjag wrote:
| _Rank each Tweet using a machine learning model._
|
| This does a lot of heavy lifting here.
| thieving_magpie wrote:
| There appears to be a repo for the-algorithm-ml:
| https://github.com/twitter/the-algorithm-ml
| Thaxll wrote:
| Let's dig into Twitter code quality.
| Kpourdeilami wrote:
| https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92...
|
| ```
|
| def query_keys(self, language, task=2, size="50"):
| if task == 2: if language == "ar":
| self.query_settings["adhoc_v2"]["table"] = "..."
| elif language == "tr":
| self.query_settings["adhoc_v2"]["table"] = "..."
| elif language == "es":
| self.query_settings["adhoc_v2"]["table"] = f"..."
| else: self.query_settings["adhoc_v2"]["table"]
| = "..." return self.query_settings["adhoc_v2"]
| if task == 3: return
| self.query_settings["adhoc_v3"] raise
| ValueError(f"There are no other tasks than 2 or 3. {task} does
| not exist.")
|
| ```
| tentacleuno wrote:
| Looking through it, the ... seems to be a placeholder for
| information they'd prefer to be kept private. For example,
| look in the keywords section in the same file you shared.
| Kpourdeilami wrote:
| You're correct, makes more sense now
| ryanisnan wrote:
| I want to go back to a world where there isn't an algorithm
| feeding me what someone "thinks" I want to read.
|
| I want to see a chronological list of things sources I follow
| have posted.
|
| Yes, I understand you can do this on Twitter still, but I would
| guess most people are more influenced by "the algorithm".
| bluelightning2k wrote:
| Late to the party here so unlikely anyone sees this comment. But
| the double take for me was seeing the article end with "if this
| sounds interesting to you, come join us!"
| simonsarris wrote:
| This is pretty limited. I picked a term used in the diagram to
| see what I could find out about it. But there seems to be _next
| to nothing_ in the released code about the mentioned "author
| diversity". No real code or description.
| mardifoufs wrote:
| I think the relevant part of the code is in this other repo:
|
| https://github.com/twitter/the-algorithm
|
| Not sure if it has what you were looking for (and maybe you
| already checked this repo, too!), but it's more relevant than
| the linked repo imo
| amq wrote:
| Surprised no one mentioned this:
| s.SpaceSafetyLabelType.MedicalMisinfo -> MedicalMisinfo,
| s.SpaceSafetyLabelType.GenericMisinfo -> GenericMisinfo,
| s.SpaceSafetyLabelType.DmcaWithheld -> DmcaWithheld,
| s.SpaceSafetyLabelType.HatefulHighRecall -> HatefulHighRecall,
| ... s.SpaceSafetyLabelType.UkraineCrisisTopic ->
| UkraineCrisisTopic,
|
| https://github.com/twitter/the-algorithm/blob/ec83d01dcaebf3...
| infamouscow wrote:
| I'm glad to see this is licensed AGPL. I hope this sets a
| precedent for everyone else in the space to do the same.
| bagels wrote:
| "Written by the Twitter Team"
|
| I found it interesting that there is no attribution. Most other
| companies list the authors on engineering blogs (eg. Facebook,
| Uber, etc.)
|
| This topic seems to draw the attention of unhinged people, so I
| suppose I wouldn't want my name on it either.
| f38zf5vdt wrote:
| No one wants to go to jail for Elon, who has been flagrantly
| violating FTC orders.[1] There's a good chance the commit
| history and authors may attest to that.
|
| https://thehill.com/policy/technology/3928219-musk-was-denie...
| perceptronas wrote:
| It seems most of the code in the repository is just simple Scala.
| Codebase is easy to read and understand.
|
| I don't see any Typelevel stuff. This probably lets them hire and
| train engineers faster while still gaining most of the benefits
|
| I hope this will encourage more companies to pick Scala.
| voz_ wrote:
| hmmm https://github.com/search?q=repo%3Atwitter%2Fthe-
| algorithm-m...
|
| Twitter hmu if you need help trying Pytorch 2.0 ;)
| systemvoltage wrote:
| Astounding amount of cynicism here, so I'll say something
| positive: Transparency is undoubtly important, I'm glad we can
| see how all of this works and what sort of effort goes into
| building a social media system. It's licensed under GPL which is
| a bummer (would have preferred BSD) but it's better than nothing.
| TMWNN wrote:
| >Astounding amount of cynicism here
|
| You can tell that those who rushed in to find something to
| criticize can't, when they are reduced to making jokes about
| coding stylistic conventions.
| systemvoltage wrote:
| Yea I mean, all discussions about Twitter have double
| standards. If this was literally any other company, there
| would be resounding praise.
| sho_hn wrote:
| Assuming anything in this codebase is worth reusing, I'm glad
| it's GPL. It's a case where I'd like open-first to spread.
| systemvoltage wrote:
| GPL would be good if it is a self contained library. If
| anyone would use it, it would be small portions of it, but
| GPL makes it completely useless. You can't contaminate
| anything with it. We'll stare at it, that's about it.
|
| That makes me think, this is actually a good call. Twitter
| can claim that they have complete transparency while not
| allowing anyone to touch their code (because it is GPL).
| "Anyone" being future competitors. If it was BSD licensed,
| it'd be tremendously useful in building a Twitter competitor
| (on paper, you still need network effects, I am just
| spitballing to make a point).
| sho_hn wrote:
| It's only contaminating other components where you
| incorporate or link it. If it's e.g. a microservice that's
| fine.
| systemvoltage wrote:
| Good point about network calls & GPL licenseability.
| anderspitman wrote:
| I'm not opposed to social media feeds having complex
| recommendation algorithms. I just wish they allowed you to opt in
| to a reverse chronological feed of only people you follow, like
| RSS.
| infogulch wrote:
| Twitter has this now. The home page is split into two tabs:
| "For you", the algorithmic feed, and "Following", the reverse
| chronological feed of just who you follow.
| conradfr wrote:
| Is really "Following" the entire chronological feed? I feel I
| miss tweets from people I follow that actually appears in the
| "For You" tab.
| anigbrowl wrote:
| It's not. I follow a fairly small # of people (~500) and
| getting people to reliably show up in is a long-running
| problem. Following is not enough, you have to favorite or
| somehow interact with them sufficiently to be sure of
| seeing all their tweets. It's quite annoying.
| cubefox wrote:
| Are you sure? I didn't notice something like this.
| anigbrowl wrote:
| Very sure. I started noticing the issue a couple of years
| ago (well before Musk's arrival) because I'd find myself
| thinking 'I haven't seen anything from ____ in a while, I
| should follow that person', only to discover that I was
| already following them and their tweets were just not
| showing up. New follows will generally pop up reliably,
| but if someone has fallen out of your regular feed you
| have to work to put them back into it.
|
| Just now (as I am writing this comment) went to check on
| someone I saw an interview with the other day, and sure
| enough I am still following the person and they have been
| tweeting a few times a day, but I haven't been seeing any
| of it.
|
| Other weird things are Twitter's habit of just
| preemptively muting people (I'll sometimes wonder why a
| person didn't reply and go back to reread a conversation,
| only to discover that they _did_ reply; and conversely,
| people that I have muted or blocked showing up in my
| search results for a trending topic. Most of the people I
| manually mute are 'influencers' who use software, staff,
| or pure obsession to get in the first reply to
| politicians and the like, a behavior I find insufferably
| annoying even if I agree with their position.
|
| I'm very interested in politics, but almost all my
| mutes/blocks are people of somewhat-similar political
| persuasion that Twitter assumes I would want to see, and
| _insists_ on showing me despite my best efforts. I _want_
| to keep tabs on the arguments of people I strenuously
| disagree with, because I already know my own opinions and
| don 't need validation. It's easier in some respects to
| maintain a second account with an uber-conservative
| persona and let the recommendation engine just feed it
| with more of the same.
| jeromegv wrote:
| That was the great thing with the 3rd party client, I
| could trust that all the people I followed I would get
| their actual tweets. Every single one of them. There was
| no also messing with it, no tweet "liked" by someone
| else, etc. Who I followed is what I saw, nothing less,
| nothing more.
|
| Of course Elon banned those apps, so now I am on Mastodon
| where I see 100% of the content that I want. Bonus is
| that I can even follow many twitter users, through
| Mastodon bot mirrors. And of course, no ads.
| cubefox wrote:
| It seems the only reason why people wouldn't show up then
| is that they got shadow banned or at least have some
| deboosting applied. So the algorithm thinks they are
| spam?
| BbzzbB wrote:
| It always had it.
|
| Edit: Why am I downvoted? It literally did, it even was named
| as you'd expect it ("sort by latest" or something), tho the
| location was less obvious as it was under the stars icon
| above the feed.
| LiquidPolymer wrote:
| On my "following" tab (on the phone app) , I'm still getting
| recommendations for bomb throwers I don't follow. Am I weird?
| It's like an unhinged relative. Not pleasant.
|
| Edit: I reversed "for you" and "following" in my original
| reply.
| EamonnMR wrote:
| I sometimes get these as push notifications with my
| username added to them.
| valarauko wrote:
| I find these notifications so confusing - at first glance
| they look like DMs or mentions to me. I don't follow
| these people, nor were they RT by anybody I know.
| EamonnMR wrote:
| Probably lots of engagement due to that confusion though.
| madeofpalk wrote:
| Twitter has _always_ had "chronological timeline" behind a
| confusing "sparkle" button (except for a brief period a few
| months back where they removed it, or always defaulted to
| back to algo timeline? and then restored it a week later)
|
| They called it "Latest Tweets" https://web.archive.org/web/20
| 200205092104/https://help.twit...
| dmonitor wrote:
| immediately getting rid of the sparkle button is one of the
| few reasons i still have a small amount of faith in elon's
| vision of twitter
| Laaas wrote:
| - Open sourcing this
|
| - Lists as tabs
|
| - _Heavily_ reduced spam
|
| - Can look at Twitter without logging in
|
| I'm likely missing some other obvious/uncontroversially
| good changes.
|
| I still use Twitter and plan to continue using it. I am
| satisfied with most of his changes.
| TaylorAlexander wrote:
| The spam comment is interesting. I had a sharp uptick in
| the number of spam messages after the acquisition.
| dmix wrote:
| My daily DM spam hit a peak last year than dropped in
| recent months.
| guerrilla wrote:
| - _Heavily_ reduced spam
|
| You mean increased...
| astrange wrote:
| There's not any less spam. How could there be when he
| laid off all the antispam people?
|
| If you expand "see more tweets" sometimes there's normal
| tweets hiding down there and sometimes it's a bot trying
| to sell you guns.
| suddenclarity wrote:
| How? By having different requirements and implementing
| another solution? Look at the guy behind YouTube purge
| tool. One guy accomplished more than YouTube have done
| for years. Why? No one knows but it's definitely not
| because lack of resources. I've definitely seen less spam
| during the last months.
| kaba0 wrote:
| Probably just less people board the sinking ship.
| astrange wrote:
| Sure seen a lot more racists since he unbanned all of
| them to increase engagement numbers!
|
| (YouTube completely changed their comments algorithms in
| the last few years and it's much better now, so you
| probably just didn't notice that.)
| mardifoufs wrote:
| No, he's talking about the very recent spam explosion in
| the YT comments, and how a dude manged to build an actual
| counter to that when YT hasn't really done anything on
| that front. I guess spam decreased since the peak of 3
| months ago, but it's still a massive problem. And again,
| this is in the past few months, and has nothing to do
| with the improvements to the yt comment algo changes of
| the past few years.
| kaba0 wrote:
| - The site often breaking as there is no one left to put
| out a fire - basically microtransactions that will soon
| boost your posts
| madeofpalk wrote:
| > - Lists as tabs
|
| Twitter added this in 2019
|
| https://www.theverge.com/2019/9/23/20880372/twitter-
| lists-al...
|
| > - Can look at Twitter without logging in
|
| All the recent change was show non-logged in users the
| Explore view which is... better than the previously
| mostly nothing I guess.
|
| Again, you've always been able to view tweets and browse
| profiles _if linked to directly_ before Elon.
| progmetaldev wrote:
| Not having to login to Twitter is a great feature. I
| don't have a Twitter account, but often see posts linked
| elsewhere, and hated getting hit to login each time.
| cubefox wrote:
| That wasn't the same, clicking the sparkle button still
| included a lot of recommended tweets. The only effect was
| that it made the timeline chronological and included all
| tweets from people you follow.
| seydor wrote:
| It would randomly revert back. I like the new layout
| because sometimes i like a quick peek at the
| recommendations
| spike021 wrote:
| It doesn't always stay on whatever you last used, though. I
| mostly use Following but it always inevitably ends up back on
| "For you".
| [deleted]
| jongjong wrote:
| WTF is AuthorIsEligibleForConnectBoostFeature? I guess this may
| explain why some people seem to accumulate a lot of followers
| very quickly while all those trying to grow organically seem to
| struggle. You can imagine if a lot of people benefit from this
| Connect Boost feature, it would make it impossible for others to
| be noticed through the noise created by all of these boosted
| individuals. That's essentially what Twitter feels like ATM.
| Recently, I manually unfollowed anyone who I suspect may have
| received a special boost from the algorithms.
| rvz wrote:
| If Twitter was 'dead' why on earth are we still talking so much
| about this blue bird site?
|
| It looks like once again these lot predicting that he won't open
| source the algorithm and are going to start eating their words
| again [0], just like they did around incorrectly predicting
| Twitter's _immediate_ collapse [1] and will look at the source
| code anyway and continue to talk about "Twitter" again.
|
| If Twitter can open-source their algorithm, Why not TikTok?
| Either way, the bots are now going to have a very expensive time
| on Twitter.
|
| [0] https://news.ycombinator.com/item?id=35213213
|
| [1] https://news.ycombinator.com/item?id=33701371
| anigbrowl wrote:
| Are you kidding me, running a botnet is easier than it has been
| in years if you're that way inclined. The amount of spam I see
| has gone way up over the last 6 months.
| rvz wrote:
| > running a botnet is easier than it has been in years if
| you're that way inclined.
|
| Even if it is 'easier', the bots are identified, down-ranked
| straight to the bottom and shadow-banned to invisibility. It
| is essentially evaporating money and time.
|
| > The amount of spam I see has gone way up over the last 6
| months.
|
| Yeah. The spam has gone way up into smoke over the last 6
| months. It is only going to get more expensive to spam as
| soon as the paid changes come in.
| Reptur wrote:
| They didn't open source the data the censoring abusive, toxicity,
| and nsfw the algorithms check against, so I'd call it a partial
| open-sourcing.
| frob wrote:
| Well that was a giant nothing-burger. This seems to be your
| standard ranking stack. We find candidates based on who you
| follow, who they follow, who is trending, and what we think you
| like. We then rank them based on how likely you are to engage
| with them and continue to come back and give us money via our
| subscription service and ad views. We then try to remove spam and
| other negative experiences.
|
| Where's the beef?
| evntdrvn wrote:
| it would be super interesting if when logged in to Twitter, you
| could take a look at your current calculated scores/weights for
| all the params that are part of these algorithms. Similar to the
| Netflix "Stats for nerds" menu...
| tcmart14 wrote:
| Repo has 1.5% rust code and no author_is_uwu
|
| That is the biggest problem.
| inparen wrote:
| Issue list is growing rapidly for a repo created an hour ago.
| ThalesX wrote:
| Non-issues most of them:
|
| - author_is_elon: the problem is his tweets suck. stop
| recommending them.
|
| - Include 'who viewed my profile' option in twitter
|
| - Only one commit on repo
|
| - How do I use it?
|
| - Cool
|
| - allow "AI" to tweet and like tweets on your behalf
|
| - IMPORTANT: Guys please keep this place for real bugs and
| contributions,
|
| etc...
| Egoist wrote:
| Aaaand the issues turned into a shitpost
| HeckFeck wrote:
| In fairness they could save some RAM by rewriting it in Rust 6
| or 7 times.
| [deleted]
| ranboxtest wrote:
| [flagged]
| thumbsup-_- wrote:
| The barebones ReadMe makes me feel this repository was open-
| sourced against the wish of engineers and with a top down
| directive
| firstSpeaker wrote:
| How so? More details and reasoning?
| thumbsup-_- wrote:
| Elon?
| [deleted]
| Slava_Propanei wrote:
| [dead]
| belter wrote:
| Unless a trusted third party, forensically audits Twitter, there
| is no guarantee the published code corresponds to the actual live
| code in Production. Also multiple parts are not present as stated
| in the blog.
|
| This should be seen as a possible snapshot of some code, that
| might have run, might run in the future, or is possibly running
| in some parts of the production infrastructure at Twitter.
| danso wrote:
| > _Twitter has several Candidate Sources that we use to retrieve
| recent and relevant Tweets for a user. For each request, we
| attempt to extract the best 1500 Tweets from a pool of hundreds
| of millions through these sources. We find candidates from people
| you follow (In-Network) and from people you don't follow (Out-of-
| Network)._
|
| > _Today, the For You timeline consists of 50% In-Network Tweets
| and 50% Out-of-Network Tweets on average, though this may vary
| from user to user._
|
| It would've been interesting to see what changes were made since
| Musk's takeover. As someone who followed 5,000+ users, I know I
| never saw a tweet that wasn't either from nor retweeted by
| someone I followed -- e.g. I never saw those "[user you follow]
| liked [someone you don't follow] tweet"
|
| 50%/50% in FYP seems to reflect my experience today -- which is
| much worse, to the point that I'll regularly switch to viewing by
| List b/c I miss seeing people who I want to read.
|
| I wonder how much testing and analysis went into deciding on the
| 50/50 ratio -- e.g. how does it impact user engagement and
| behavior. Because it sounds like an easy round value that you'd
| land on when thinking "users should be pushed out of their
| bubbles"
| cubefox wrote:
| Perhaps if you did follow so many people they got drowned out,
| but with substantially fewer following, those recommended
| tweets were a big part of what I saw. Especially in the last
| year or so before Musk took over: Twitter went a lot more
| aggressive and didn't just show tweets which people you follow
| "liked", but also other tweets, which the algorithm somehow
| determined you might like, which was often wrong, and,
| moreover, so frequent that it made a big portion of the
| timeline. The "following" tab fixed this problem.
| danso wrote:
| Yep, having had created a few throwaway accounts I definitely
| got a sense of how the algorithm compensated for the majority
| of users who aren't super active. And it makes sense -- most
| new users aren't going to want to spend account creation
| picking 50 accounts to follow.
|
| But if someone has hit the follow button 1,000+ times, it's
| reasonable to have some faith that they've seen a lot of
| tweets and know what they want. Showing a few out-of-network
| tweets seems reasonable (I got enough as it is through
| followings' retweets). But 50% of a feed that already can't
| fit tweets from thousands of followings just feels like shit.
|
| The worst part is that the share of in-network tweets seems
| to be highly concentrated to the last 10 or so people I most
| recently interacted with, e.g. seeing the same user over and
| over just because I liked one of their tweets the other day.
| Which makes sense to save on computation costs, but it's
| pushed me into a much tighter bubble than I ever had when the
| timeline wasn't so out-of-network focused.
| cubefox wrote:
| Is the "following" tab an option for you? Or what would you
| like the "for you" tab to do? Filter out heavy tweeters?
| Prioritize more popular tweets?
| danso wrote:
| The Following tab limits to my followers, but in reverse
| chrono order. This is also not desirable b/c it limits to
| whoever's posting whenever I've happened to check the
| feed.
|
| I'd like to see interesting tweets from a few hours ago,
| and not just Australian tweets when I'm up late at night.
| coldcode wrote:
| A year ago my account with 5700 followers got an average of
| 3000 impressions per post (art). Today it's only 200-500. It
| mentions their fanout system was replaced by something new, not
| sure when or if thats in the drop, but my impression count
| dropped around April-May last year. Clearly something decided
| my posts should not shown to my followers very often.
| yeahsure22 wrote:
| [dead]
| wslh wrote:
| It's the data, stupid [1] (not the algorithm).
|
| [1] https://en.wikipedia.org/wiki/It%27s_the_economy,_stupid
___________________________________________________________________
(page generated 2023-04-01 23:01 UTC)