[HN Gopher] The Tragedy of Google Search
___________________________________________________________________
The Tragedy of Google Search
Author : samizdis
Score : 94 points
Date : 2023-09-22 11:57 UTC (11 hours ago)
(HTM) web link (www.theatlantic.com)
(TXT) w3m dump (www.theatlantic.com)
| Jtsummers wrote:
| https://web.archive.org/web/20230922115825/https://www.theat...
| nuancebydefault wrote:
| > no company or product can grow alongside the internet forever
| without, eventually, being swallowed up.
|
| Even not Google. I wonder if they can still turn things around,
| but I have the feeling MS caught up finally at high speed and
| passed them.
|
| PS I was reading (sceptically) 'google search is broken' already
| for a year or so in HN comments and have to admit I believe it
| now.
| bayindirh wrote:
| First, results got worse. Then, ads' started to increase. Then
| I moved to Kagi. Now, I'm happy.
| ComputerGuru wrote:
| I wrote about this back in 2015 or 2016, with specific examples
| that people are just now complaining about vocally (mainly the
| lack of respecting search keywords):
| https://neosmart.net/blog/on-the-growing-intentional-useless...
| wakeupcall wrote:
| > PS I was reading (sceptically) 'google search is broken'
| already for a year or so in HN comments and have to admit I
| believe it now.
|
| Even ignoring results, mainstream started to notice a while
| ago. Perhaps the biggest social proof at large is when it is
| shown distilled as comedy and people can laugh about it:
|
| https://www.youtube.com/watch?v=NT7_SxJ3oSI
| emodendroket wrote:
| The "mainstream" accepts many claims as true that can be
| objectively demonstrated to be false. Why should I have such
| confidence they're right about a theory like this that seems
| to be unfalsifiable?
| wakeupcall wrote:
| Did you watch the video, or is this a rhetorical response?
| Spoiler: it's about the conflict of interests in google
| search.
| zwieback wrote:
| In case anyone is interested: the RSS feed for The Atlantic is
| still pretty good, able to read most articles in full text.
| xnx wrote:
| As the article mentions, the web has changed. It's very hard to
| return a list of good results when there are none. I've yet to
| see side-by-side comparisons where another search engine is doing
| a better job.
|
| I've had "Search Generative Experience" (AI/LLM synthesized
| result/summary above the standard web results) turned on for
| awhile. It is definitely the future. To survive the flood of
| revenue optimized content (recipes being a familiar example), you
| need an active [software] agent working on your behalf to read
| content and make one straight thing out of the crooked timber of
| the web.
| hutzlibu wrote:
| "To survive the flood of revenue optimized content (recipes
| being a familiar example), you need an active [software] agent
| working on your behalf to read content and make one straight
| thing out of the crooked timber of the web."
|
| So what do you think, will be the goal of all advertisers?
|
| To get their recommendations baked into that AI. Direct
| bribing/revenue sharing with the manufactur of that AI. Or
| sneaking data into training sets via bribed employes and so on.
|
| I don't trust black boxes and never will.
| A_D_E_P_T wrote:
| The problem isn't just that the internet has become more SEO-
| oriented and encumbered with low-value clickbait websites --
| it's also that Google doesn't listen to instructions the way it
| used to.
|
| Google started slipping when it began assuming that it knows
| better than you do. For instance, when the exact search
| operator (" ") stopped working.
|
| I'd take the Google of 2014 over the Google of 2023 without
| thinking twice, and I think that I'm not alone in this.
| pascalxus wrote:
| Absolutely. i so often do a search and it presumes to think
| it knows what i want rather than giving me what I ask for,
| sometimes even when I'm being quite specific. I will say it
| does a great job for programming stuff but a lousy job
| whenever I'm researching economics topics.
| 2OEH8eoCRo0 wrote:
| More of the web is also behind paywalls or is on platforms
| like tiktok, instagram, twitter, etc. Less to scrape and
| index.
| endisneigh wrote:
| Verbatim search still exists.
|
| https://www.google.com/advanced_search
| A_D_E_P_T wrote:
| Do you use it often? It doesn't work the way it used to. I
| can't count the number of times I've searched for a
| technical term -- in quotes -- and got results back for a
| subtly (sometimes not so subtly) different term.
| [deleted]
| endisneigh wrote:
| you have an example of this? using quotes is not the same
| as going to advanced search and saying use exact word or
| phrase. silly, yes I know.
| intalentive wrote:
| Agree. Also their results are often biased and I sometimes
| find myself going to Yandex.
| selimthegrim wrote:
| I think Frank Zappa would chortle with irony that a Russian
| search engine returns his discographies uncensored in the
| here and now versus an American one.
| marginalia_nu wrote:
| All search engines are inherently biased, since they
| perform a ranking task (which is a value judgement). They
| all encode (possibly uncessfully) the values of their
| creator. This is why having multiple search engines is
| important, as diversity in search options is the closes
| you'll get to objectivity.
|
| (said the independent search engine creator with the same
| air of righteousness as when Plato argued philosophers
| should be kings)
| midhhhthrow wrote:
| Yes but they're also politically biased. Not to mention,
| the enormous amount of web content that is effectively
| banned from seArch engine results due to low prestige
| rtsil wrote:
| My most recent complaint is the disappearance of the
| pagination numbers.
|
| Since I know the top results are mostly useless spams on some
| searches, I used to go to page 4 or 5 to find the actual
| results. Now to achieve the same result I have to click and
| scroll 5 times.
| bayindirh wrote:
| So, how paid search engines like Kagi manages to bring useful
| results every time?
|
| Its results are superior, with way less spam and SEO optimized
| content.
| mattnewton wrote:
| I have no idea; but I speculate this is one area challengers
| have it easier because
|
| a) there is no giant industry optimizing for ranking on Kagi.
| There are whole armies of people trying to game google
| search, meaning whatever heuristics they have would probably
| be rendered useless shortly if google were to adopt them.
|
| And
|
| b) Kagi has a much smaller, more technical userbase. So like
| the early web they can get away with a search experience
| tailored for those kinds of people and not millions of people
| powering on a smartphone as their first computer.
| bayindirh wrote:
| The biggest power of Kagi is "you" are the algorithm. A
| page is optimizing for Kagi and providing bad results?
| Block them. You found a good website burrowed down?
| Raise/promote them.
|
| Kagi works for you, with your input. It doesn't do
| "advanced guessing" by implicitly and opaquely processing
| your interaction with it.
| mattnewton wrote:
| Explicit control can work for ranking but can't scale to
| the retrieval step of search - you aren't going to sift
| through thousands of thousands of pages of potential
| results to give feedback on a meaningful portion of them,
| there must be some method of pulling pages to be ranked.
| This is where the gaming happens, with basically
| limitless spam domains that need to be filtered out.
| There just is no way Kagi is 100% relying on self reports
| for this.
| amelius wrote:
| Also Kagi is still in the phase of "let's give users what
| they want" instead of in the phase of "let's give
| shareholders what they want".
| xnx wrote:
| A lot of people on HN are very happy with Kagi, but I haven't
| seen examples of searches to understand where it is better.
| In a blind "taste test", I would be surprised if Kagi or Bing
| is preferred to Google for most searches.
| bayindirh wrote:
| Well, all my daily searches, whether it be technical or
| daily mundane things return better results with less cruft
| than Google. Also, it notifies you if a page has excessive
| number of trackers.
|
| Combine it with personal website ranking and lenses, it
| surpasses Google easily.
|
| Also since it has no ads, there's no noise.
|
| I don't use Microsoft products, so I can't comment on Bing.
| fnordpiglet wrote:
| I did side by side for a while and Kagi was consistently
| superior to Google and bing. I think it boils down to they
| don't try to cross sell advertisements so they're more
| oriented towards actually providing better results rather
| than better paying results.
| JohnFen wrote:
| Do you need examples of searches? If people get more useful
| results from one search engine over another, that's all
| that matters.
|
| I can't provide you with side-by-side examples because I
| don't compile lists of such examples. I'm searching to find
| stuff, not researching search quality. I just know that I
| have more problems getting good search results from Google
| over a couple of the other engines.
| xnx wrote:
| Yes. I would love to see some actual examples. People
| have said that verbatim search doesn't work for them and
| I've never observed this, so I'm also skeptical of claims
| of dramatically better search results.
| JohnFen wrote:
| So you think everyone is lying?
| xnx wrote:
| No. What they consider "better", I might consider
| "worse".
| nilespotter wrote:
| 80% good enough results and this [1] was all I needed to
| switch and not look back.
|
| [1] https://kagi.com/privacy
| dharmab wrote:
| Kagi's killer feature can't be blind tested, because Google
| and Bing don't have it. The ability to pin, raise or block
| sites in your results is a game changer.
| marginalia_nu wrote:
| I don't think the problem, in general, is that no good results
| exist for any given query, more often the problem is that the
| good results are rarely well search engine optimized, and thus
| stand zero chance of making the search results.
|
| Some people admittedly expect pretty weird websites to exist.
| Like who would even publish a good objective comparison between
| products in a specific category. But people make those queries
| looking for those types of results, and spammers generate such
| content to make bucks off the traffic. 20 years ago you'd have
| gotten nothing in most cases.
| mgaunard wrote:
| There are a lot of people that genuinely make comparisons of
| specialty products.
| majormajor wrote:
| I don't believe that anything Google does now - summarization
| included - will avoid being gamed just as effectively as their
| current search results have.
|
| It's a negative-sum arms race, but if Google (or whoever)
| wasn't spending the money to fuel the arms race by passing on
| ad money, who would pay for Google to exist in the first place?
| JohnFen wrote:
| I am convinced that a lot of the reason why Google search went
| downhill is that they started using more AI. Specifically,
| using it to try to interpret what I'm searching for rather than
| taking my word for it, which leads to irrelevant search
| results.
|
| That's just my hypothesis, of course. I don't know for certain.
| But it does make me very skeptical about using AI to help
| search the web.
|
| I want a robust set of search modifiers instead.
| beebeepka wrote:
| I liked StartPage/Ixquick results much better than what Google
| used to show. Maybe they still do but wouldn't know as I
| stopped using it after they got acquired. Didn't/don't they
| rely on Google?
| c7b wrote:
| > To survive the flood of revenue optimized content..., you
| need an active [software] agent working on your behalf
|
| How long do you think it will be before that software agent's
| output will be just as riddled with commercialized content? I
| think we might have only a short window where we get to enjoy
| the fairly unadulterated output of the base LLMs by the large
| providers (yes, I'm aware that they're screening for safety
| already, but there could be a lot more coming). Imagine a
| dialogue like this: Q: What should I watch out for when buying
| new car tires?
|
| A current answer might be something like: Here's a list of
| things to watch out for when buying car tires: A, B, C, D, E.
| Remember to always consult with specialists. Shall I explain
| more about the topic?
|
| A future answer might look something like: According to experts
| like the professionals at [tire shop near you], here are the
| things you need to watch out for when buying car tires: A, B,
| C. You can use this coupon to get a 10% discount off your next
| purchase at [tire shop near you]. Shall I book an appointment
| for a free consultation at [tire shop near you] for you?
| xnx wrote:
| I'll also had that a huge portion of genuinely good content is
| now inaccessible to Google in walled gardens like Discord,
| Facebook, Twitter, and TikTok.
| dlrush wrote:
| The Tragedy of Google Analytics...
| slashtab wrote:
| [flagged]
| Phiwise_ wrote:
| Soft paywalls aren't, like many nouns that need an introducing
| qualifier (think homeopathic medicine for the most common
| example).
|
| More specifically, a soft paywall isn't a starving artist
| trying to get their daily bread from valuable content (If they
| were I'd be biased to support them, because I'm among the
| smaller crowd around here that rather likes IP law and procing
| things, and kind of enjoy poking the gibs me dat for free
| beehive. I even probably support, horror of horrors, what I
| heard was Musk hard-paywalling twitter to make it profitable,
| and he's neither starving nor an artist), it's a malicious
| trick to hijack when you're trying to find free information on
| a subject by bait-and-switching a, say, promising quoted
| section from the middle of the article on a search result. It's
| like a digital used car salesman who promised a great deal or
| free benefit in an ad but is all out of non-full-price
| inventory once you drive all the way out to the lot.
|
| Until these sorts of sites stop shipping their whole catalog to
| be indexed as available while hiding behind ever-more-
| sophisticated ui barriers (or browser drm soon, maybe?) and
| firmly take an honest and upfront stance of either freely
| available to read or benefiting from paying customers I think
| the right stance is that silver beats gold; You only survive if
| you do unto others as they do unto you, so for as long as
| they're putting out bait we should take the worm and skip the
| hook: https://archive.ph/X3gHI
| [deleted]
| selimthegrim wrote:
| The Guardian soft paywalls in in that the app will direct you
| to the website.
| moritzwarhier wrote:
| > it's a malicious trick to hijack when you're trying to find
| free information on a subject by bait-and-switching a, say,
| promising quoted section from the middle of the article on a
| search result
|
| Already feels that way with login walls like on Xitter or
| Instagram tbh. Independent of the indexing, just because they
| are quoted and linked to everwhere, as if they were a public
| billboard.
|
| Especially twitter seems to toggle theirs on/off on a random
| basis or based on referer, heuristics, etc.
| smcin wrote:
| The other day I asked Google _" Are US drivers licenses forgery-
| proof?"_ and _" Can US Real-ID drivers licenses be forged?"_
|
| The #1 or #2 hit is utterly irrelevant: nothing to do with US or
| Real-ID (zero mentions), it's about New South Wales, Australia
| (lots of mentions of "NSW"):
|
| > _'Tough to Forge' Digital Driver's Licenses Are--Yep--Easy to
| Forge. Researchers found a litany of security flaws that allow
| simple, quick, and cheap forgeries in Australia._
| https://www.wired.com/story/digital-drivers-license-forgery-...
| thefourthchime wrote:
| 2nd link on Kagi:
|
| https://cis.org/Report/Americas-Identity-Crisis-Document-Fra...
| itvision wrote:
| I won't even read it.
|
| Google remains the best search engine, period.
|
| Nothing is even close.
| emodendroket wrote:
| I'm a little tired of all these "Google search used to be better"
| feel-pieces that don't do anything to substantiate the claim
| beyond relying on the author's vague sentiment.
| hurril wrote:
| Your tiredness is of course both relevant and substantiated.
| thefourthchime wrote:
| For those who are lazy like me--or maybe you'd call it efficient
| --here's the key points summary from Kagi:
|
| - Google Search has evolved significantly from its early days as
| a simple list of blue links to becoming an encyclopedia,
| predictive engine, image repository, shopping mall and more that
| is overloaded with information.
|
| - It has become more difficult to find authoritative answers on
| Google Search due to an overabundance of sponsored content,
| prompts, and low-quality keyword-stuffed pages.
|
| - Google is currently undergoing an antitrust trial regarding
| whether it maintains its search engine monopoly through
| anticompetitive means such as exclusivity deals rather than
| having truly superior technology.
|
| - Access to large amounts of user data is very important for
| powering search engine algorithms through personalization and
| improving the user experience.
|
| - While Google argues diminishing returns to scale for user data,
| internal emails show Google engineers acknowledging that scale
| remains highly important for the company.
|
| - Some feel Google has lost its way and become conservative due
| to its success, with its corporate bureaucracy stifling acquired
| companies.
|
| - Google Search has become bloated with advertisements and
| prioritization of its own services over organic results.
|
| - Search quality may be declining as its algorithms are gamed by
| low-quality sites and search engine optimization techniques.
|
| - Google's trajectory of scaling up its mission to organize the
| world's information through acquisitions and exclusivity deals
| has put pressure on it to keep growing.
|
| - No company can likely grow indefinitely to keep up with the
| ever-expanding internet without being overwhelmed, as Google
| Search now demonstrates.
| brookst wrote:
| Thanks for the excellent summary. Though I think the article
| and summary miss the root cause: Google Search has two masters,
| users and advertisers. These stakeholders' interests are not
| aligned, and by taking one middle road after another, Google
| has deteriorated into near-uselessness for end users, relying
| on scale rather than quality.
| burlesona wrote:
| That's a pretty great summary. Can you elaborate a bit on how
| this works with Kagi? Is it just a wrapper over gpt-4, do you
| feed it an article and ask for a summary, or does it summarize
| in the search results?
| thefourthchime wrote:
| Just go here
|
| https://kagi.com/summarizer/
| [deleted]
| by_Seeing wrote:
| It's not a story the Jedi would tell you.
| unixhero wrote:
| To read the full story start your free trial today _sigh_
| grotorea wrote:
| Seems like the complaints that have been showing up here a for a
| while have reached the mainstream. Any chance of an upstart
| competitor taking the opportunity?
| thunderbong wrote:
| Kagi is what others in HN have been suggesting.
| collaborative wrote:
| aisearch.vip
|
| Disclaimer: developer
|
| I am bootstrapped, don't advertise, and need no investment
| starchild_3001 wrote:
| Buyer beware: Sensation sells (as is portrayed in this news
| article). Competitors are one click or one setting away. Last I
| tried switching to DuckDuckGo things didn't go so well -- just
| overall subpar experience. Bing arguably is a bit better, but
| that too didn't satisfy, despite the super cool chat feature
| (deserves kudos for that!).
| Josh613 wrote:
| To whom it may concern,
|
| We're building a new search engine. Of course it's AI driven, but
| at its core, it's all about human dignity. When you use it,
| you'll have a voice. If you'd like to try it, please email me:
| josh@sirch.org
|
| Thanks! Josh
|
| PS. If you'd like to join our team, we have 16 people now (mostly
| Snapchat/Instagram/Twitter folks), and we'd like to go up to 50
| before we raise capital. It's going to be a wild ride.
| dronify_us wrote:
| [dead]
| beggers wrote:
| When I got to your website I get a Vercel 404 `Code:
| DEPLOYMENT_NOT_FOUND`. Perhaps not the most confidence-
| inspiring for potential candidates..
| 1vuio0pswjnm7 wrote:
| https://www.bloomberg.com/news/articles/2001-03-12/googles-l...
| sprayk wrote:
| I know there are other search engines out there, but I haven't
| really given any a shot. Do any of them feel like the Google
| Search of the past?
| pony_sheared wrote:
| Duck duck go is worth trying, it's like Google from the old
| days
| wakeupcall wrote:
| When google first appeared it was a revolution compared to
| everything else. If you expect this kind of difference, none of
| the alternative search engines come close.
|
| If you are looking for big search engines, there aren't many
| really, I strongly believe you should give them all a shot and
| just alternate between them.
|
| The biggest takeaway from this is that google isn't massively
| better than the worst alternative anymore and it sticks mostly
| due to recognition.
| FredPret wrote:
| Good to see non techies start to care about this
| lordfrito wrote:
| https://archive.ph/X3gHI
| mixmastamyk wrote:
| Asking for captcha. I nope'd out.
| LeoPanthera wrote:
| It only asks for a captcha if you are using Cloudflare for
| DNS.
| jcrawfordor wrote:
| Or Quad 9... or potentially any DNS server that refuses to
| provide EDNS. I don't think there's an "official list" of
| DNS services on the archive.today shitlist, but it's been
| growing lately. These days you have to select recursive DNS
| on a balance of "archive.* works" vs. privacy preservation.
| 38 wrote:
| Kagi needs a free tier.
|
| even if its only 10 searches a day, it needs something. I am
| never going to use it unless I can try it out for a while.
| danielovichdk wrote:
| I have always speculated, what if Google simply turn off shop.
|
| Just turned off its servers and said "we're closing".
|
| Then some would realise what a huge impact it serves on a daily
| basis.
|
| Pop culture will never last.
| jl6 wrote:
| It certainly _seems_ like Google Search prefers to return ad-
| ridden monetized sites rather than more usable sites, with the
| obvious explanation being that they earn more money from sites
| that show ads.
|
| But this feels like too obvious a conspiracy theory and it's
| probably not directly designed to act that way - more likely it
| is an unintended consequence of something else. Unintended, but
| tolerated all the way to the bank.
| aero142 wrote:
| I would guess they monitor tons of metrics for any change to
| search and any change that reduces ad revenue gets rolled back
| or reviewed.
| twoodfin wrote:
| Unfortunately, the sad truth is likely that Google has hard
| telemetry evidence that users in aggregate prefer this SEO
| junk, engage with it, and move on feeling that Google
| successfully satisfied their query.
|
| If Google could drive higher levels of engagement and
| satisfaction by screening out the dross I think they would,
| it's clearly within their abilities.
|
| Hopefully AI gets good enough and scales efficiently enough so
| they can disaggregate this kind of ranking decision.
| Guvante wrote:
| The recipe problem is literally this: users who go to a site
| check the recipe, realize it isn't a fit and return to Google
| are considered failures lowering your result status.
|
| By putting fluff and ads you extend the time on the page
| making your site seem more valuable.
|
| It is hard to tell "that didn't answer my question" from
| "that quickly answered my question wrong" after all.
|
| (Note it isn't uncommon to always bounce off a recipe site as
| you check multiple to make sure the one you found makes
| sense)
| JohnFen wrote:
| > hard telemetry evidence that users in aggregate prefer this
| SEO junk, engage with it, and move on feeling that Google
| successfully satisfied their query.
|
| How in the world can telemetry tell you any of that? The only
| metrics I've heard from Google that relate to it is whether
| or not they return to do the same search again within a short
| window, but I don't think that tells you users prefer SEO
| junk or that they're satisfied with the search results.
| ToucanLoucan wrote:
| Probably more like selection bias. Sites that are heavily
| monetized with Google have relationships with them and can work
| directly with them to improve their SEO and page rankings. Plus
| if they have all those ads they almost certainly also use
| Analytics, which can help them optimize too.
|
| But also, if someone claimed to be a Google engineer and told
| me they purposely rank up pages that have Adsense on them, I
| wouldn't call them a liar immediately.
| jonas21 wrote:
| A more likely explanation is that ad-ridden monetized sites are
| able to invest more in SEO since a visitor is worth more to
| them than an ad-free or low-ad site.
| brigadier132 wrote:
| I guarantee you no experiments that negatively impact revenue
| but improve the search experience ship
| babyshake wrote:
| They must have considered some possibility of a "Subscriber
| Results" option for Google One subscribers or something along
| these lines? I'm guessing that would be too risky to make
| their non-subscriber results look like garbage?
| TaylorAlexander wrote:
| They serve such a large number of users who would never
| pay, I cannot imagine they would ever consider admitting
| that they could serve better results. That would be a
| disaster. They make money off of users from surveillance
| and advertising. They would never make enough from
| subscriptions to offset the losses in perception from all
| the people they are surveilling for advertisers.
| bobthepanda wrote:
| I mean at one point that was an argument against Youtube
| Premium as well.
|
| It kind of depends what the ad value per user is, and
| then charge X% above that. Though maybe it's so high that
| it would be an enterprise price level.
___________________________________________________________________
(page generated 2023-09-22 23:00 UTC)