[HN Gopher] Google is burying the web alive
___________________________________________________________________
Google is burying the web alive
Author : doener
Score : 148 points
Date : 2025-05-26 14:00 UTC (9 hours ago)
(HTM) web link (nymag.com)
(TXT) w3m dump (nymag.com)
| ncr100 wrote:
| My summary - G is 'burying' web search results underneath AI
| results on its search page.
| jfengel wrote:
| Haven't they been doing that for at least a year?
|
| And before that they were burying it under paid results and an
| info box?
| ajross wrote:
| Pretty much. This is getting toward a "derangement syndrome"
| kind of behavior, IMHO.
|
| When Microsoft and subsidiaries deploy AI-enhanced search
| results, it's a "Google Killer" and rockets to the top of the
| front page of HN to great fanfare.
|
| When it turns out that the best and most-used AI-enhanced
| search results are served at the Google search prompt, Google
| is "burying the web alive" and the outrage rockets to the front
| page of HN.
|
| Which is it, folks? Like, was OpenAI killing the web too? Or is
| it good when OpenAI kills the web but bad when Google does?
|
| (And yes, Google pays my salary, but to work on firmware and
| not think about search or AI)
| ndiddy wrote:
| You fell for the Goomba Fallacy
| https://i.redd.it/33w3fqrnx8pd1.jpeg
| ajross wrote:
| Yeah yeah. But nonetheless group think and echo chambers
| are real things. Not all opinions are uncorrelated data
| points, probably most of them _aren 't_. And real people
| (you too) do indeed hold contradictory opinions all the
| time.
|
| In point of fact insisting that "My Opinion Is Pure And
| Special and Uncorrelated To Anyone Else's" is probably the
| single _best_ way to find yourself parroting echo chamber
| delusions. It 's a license to ignore everyone who disagrees
| with you, because you can always construct things such that
| they're really arguing with _someone else_.
|
| I think it's very appropriate to point out when "General
| consensus on HN as measured by article upvotes" happens to
| include two grossly contradictory opinions.
| surgical_fire wrote:
| > Yeah yeah. But nonetheless group think
|
| That's only your perception of groupthink. You read those
| threats looking for anti-Google bias, and you find it,
| because of course.
|
| AI is a very divisive subject nowadays. There are people
| that are all in on the hype, there are skeptics, there
| are desperate people thinking that they will lose their
| jobs, there are those that think is is destructive and
| unethical.
|
| The only caveat that I might make is that at the time of
| the Microsoft announcement, it was during the honeymoon
| phase of OpenAI (when GPT 3 was fairly new and an
| impressive leap from what came before), and Google's
| current advancements, while impressive, come at a time
| when a lot more people are negative towards it.
|
| Also, every sufficiently large company will have people
| that dislike it as a baseline. Be it Google, Microsoft,
| Nintendo, Sony, Tesla, Toyota or any other. Read any
| thread on those companies and you will find negative
| comments based on that alone.
| TremendousJudge wrote:
| Incomplete. The article also discusses the new AI search
| feature (not the AI box), which eschews links to the web
| completely
| romaaeterna wrote:
| As someone who remembers what the web was like 25 years ago, I
| have a feeling that they're burying a corpse.
| pphysch wrote:
| Yes, a corpse killed by Facebook, Reddit, Discord and other
| closed platforms.
| energy123 wrote:
| All the negativity is public and algorithmically boosted, all
| the positivity is in private chats. And this is increasingly
| the case over time. I sincerely hope for maximum
| enshittification in the form of AI slop rendering these
| misery machines discredited in the public's mind so the spell
| is broken.
| Spivak wrote:
| Does it actually matter that Discord is closed or would the
| problem still exist if Matrix won and the content contained
| within was nonetheless still not indexable by search engines
| and communities were invite only?
|
| Because the fact that Discord is flourishing is I think a
| direct result of it not being a part of the giant amalgamated
| omniplatform indexed by search engines. Being shielded from
| the masses is the point. There aren't megaphones in small
| private communities where the social dynamics of repeat
| interactions are in play. People aren't competing to say the
| most outrageous rage-bait for engagement.
| TremendousJudge wrote:
| >People aren't competing to say the most outrageous rage-
| bait for engagement.
|
| This was never true in the age of web forums, which was the
| equivalent to Discord in many cases, and still isn't true
| in niche subreddits, which are public and indexed by search
| engines.
| oceansky wrote:
| But the counterpoint is that it's much harder to find any
| information, including personal anecdotes from people,
| without being part of the particular community.
| oceansky wrote:
| How's reddit a closed platform? It's superbly indexed by
| Google. In fact searching site:reddit.com often greatly
| improve my results.
| pera wrote:
| The corpse was already buried ages ago and his name was Jeeves:
| What they are doing now is an exhumation and calling it the
| future.
| ndiddy wrote:
| There's still tons of personal webpages out there, Google just
| doesn't expose them because it's in their interest to show ad-
| filled SEO blogspam since they also operate AdSense and
| DoubleClick for Publishers. https://marginalia-search.com/ does
| a much better job of finding personal sites.
| Rebelgecko wrote:
| Is that website down, or maybe it doesn't work in EU?
|
| Edit: NVM, just took a few refreshes to get past the "oh
| noes" screen
| Weryj wrote:
| Works in NL
| skrause wrote:
| It doesn't really "find" any personal sites, you have to tell
| it about each manually with a pull request:
| https://github.com/MarginaliaSearch/submit-site-to-
| marginali...
| ndiddy wrote:
| This isn't true, Marginalia Search has a crawler. The guy
| who runs the site writes a lot about the trials and
| tribulations of running a web crawler on his blog if you're
| interested. https://www.marginalia.nu/log/ I just checked
| and my site is listed despite me never manually submitting
| it. I guess if nobody links to your site, you would have to
| manually submit it due to it not showing up in crawls, but
| that's true about any search engine.
| marginalia_nu wrote:
| That form is just to insert domains that aren't found
| automatically.
| sanderjd wrote:
| Right?
|
| I'm sure Google deserves some, maybe a lot, of the blame, but
| the web has just deteriorated steadily over the past 15 years
| or so and I really don't think it's all their fault.
| anal_reactor wrote:
| I'd like to limit my online time, but I'm so addicted to the
| internet, even though it's pure garbage nowadays. Any practical
| tips? It's not about putting away the phone, it's about... the
| hope that maybe this time I'll read something funny or
| informative, as I have absolutely no other sources of
| information or social interaction. "Go out and talk to people"
| isn't valid advice. It's never been, but in 2025 finally even
| the mainstream is slowly admitting that this doesn't work.
| NewsaHackO wrote:
| This is actually a good thing, now we don't have to sift though a
| dozen semiautomatically created SEO gaming website that try and
| make the same point 5-6 said in slightly different ways.
| righthand wrote:
| What is better? A single source or a variety of sources to
| determine bias? If I do publish on the web? What is my
| incentive to keep doing so if my site never gets seen in search
| results? What will fuel the LLM then?
| Rastonbury wrote:
| I haven't used AI Mode but AI Overview often reads exactly like
| those 300 word bullet point listicles if you try searching for
| anything competitive like "best X for Y"
| AriedK wrote:
| So how long do we have until companies can pay to be in the
| system prompt to be recommended for certain queries?
| SoftTalker wrote:
| My speculation: It's almost certainly already happening.
| LunaSea wrote:
| Yes, it's called "tool partnerships" and Microsoft has been
| doing it heavily for Copilot.
| righthand wrote:
| Can't you just game the SEO and Google will eat your result
| thus pointing you to whichever service/product?
|
| It'd be spam generated answers from seo spam. Which is
| definitely already happening.
| nicbou wrote:
| Those AIs are operating at a loss. Sooner or later they will
| seize enough market share for investors to demand returns.
| almusdives wrote:
| I get why this is problematic for industries that depend on high
| traffic for ad revenue etc, but is bad for websites who are
| actually trying to market services that provide tangible value?
| Like if I'm searching for a dry cleaner in Glasgow, if I end up
| with the same provider, I don't care (and neither does the dry
| cleaner) whether I find them through traditional links or an AI-
| mediated search?
| gspencley wrote:
| Feels like more curation than we had before. My wife and I
| recently bought a commercial space and have been trying to find
| contractors for renovation projects. It has kind of blown us
| away at how unresponsive most have been. Like, we'll reach out
| them ... wait DAYS, sometimes they get back to us ... maybe a
| week later they'll come for a site visit to give us a quote and
| then we need to keep following up with them ... just to get a
| quote.
|
| We were complaining about this and a friend of ours said
| something very insightful: "There's a lot of good contractors
| out there who suck at SEO."
|
| So we reached out in some local Facebook groups and had people
| out THAT DAY and the work was starting the very next day.
|
| Point is, the companies we were reaching out to were the
| absolute busiest of the busiest because they were showing up at
| the top of search results. As soon as we stopped using web
| searches, we actually started getting results.
|
| Now, this problem exists in spite of AI but I'm worried that
| it's going to make it even harder for smaller, less busy
| companies to show up AT ALL. AI is taking the system that
| already existed and is curating it further with "summaries" ...
| so what can we expect? Only the first "top 3" businesses in
| your area?
|
| Google and other search engines were already starting to kill
| off the "long tail" ... but with AI I think its going to be the
| final nail in the coffin. If you want to search for the less
| popular options, if you want to try and find the niche stuff
| ... I don't know how well it's going to work. I suppose if you
| can prompt it: "All the 'top' options are ghosting me, give me
| some alternates" then maybe people will find ways around that
| with these tools. Or trying to find businesses with web search
| was already not going to give good results anyway in current
| year (return to classifieds and job boards was our recent
| experience).
| AStonesThrow wrote:
| Hey, you should meet the folks in the other thread: "oh why
| did Google add my phone number to my business profile on
| Maps?" "It should be illegal for Google to publish listings
| for businesses without consent!" "It's crazy that just anyone
| can submit contact information for a business on Google
| Maps!"
|
| If you're using Search or an LLM for a business
| recommendation, you're already using the wrong tool for the
| job. Even before Google Maps ramped up, there was Yelp,
| craigslist, Tripadvisor...
| pixl97 wrote:
| >Like, we'll reach out them ... wait DAYS, sometimes they get
| back to us ... maybe a week later they'll come for a site
| visit to give us a quote and then we need to keep following
| up with them ... just to get a quote.
|
| Jeesh these companies are dumb... they could be selling
| referrals to other businesses but are instead dropping them
| on the floor.
| baq wrote:
| The best way to sell your boring contracting company is to
| take it off the web and wait for private equity to make an
| offer.
| FredPret wrote:
| Some "plumbing" company websites are really just referral
| farms that are supposed to send your data to plumbers in your
| area.
|
| The idea even makes sense - if this worked, plumbers could
| focus on plumbing and SEO hackers could focus on getting a
| landing page in front of people looking for plumbing.
|
| Sadly there are a lot of slop-like companies out there.
| mike_hearn wrote:
| Or the AI takes care of finding quotes for you, in which case
| the most responsive vendors could do better than before.
| righthand wrote:
| And if Amazon gets into dry cleaning and promoted Amazon Dry
| Cleaning as the top result in Glasgow: will that be bad for
| local Glasgow dry cleaners when everyone is shipping their
| clothes to Amazon?
| scotty79 wrote:
| Maybe Google should use AI to de-rank ad filled seo spam.
| DataDaemon wrote:
| Goodbye small blogs and medium publishers, it was nice to read
| you. Now Google take your unique content, rewrite and present on
| their website. There will be no incentive anymore to write a new
| content. You will get NOTHING for your hard work. Bye.
| bitpush wrote:
| You can't simultaneously rail against ads, which is our best
| model for scaled monetization, or complain against paywall and
| then say it is killing the business model.
|
| Stuff costs money but unfortunately we as a society have come
| to expect free stuff, and businesses are just catering to it.
|
| If not for Google, ChatGPT or someone else will do it.
| mac-mc wrote:
| Correction: burying our business model alive
| panarchy wrote:
| The web has been a walking corpse for about a decade now.
| rambambram wrote:
| I only do physical offline marketing for some time now. Works
| well enough and one stands out among all the clickbait and other
| foolery. Some index disappearing is not a problem. The moment
| organizations or institutions start burning the libraries down,
| then we have a bigger problem.
|
| Don't get fooled by people who claim the open web is dead. The
| moment you start believing that, then you are exactly where they
| want you to be.
| sakopov wrote:
| Google is damned-if-they-do, damned-if-they-don't. One side
| screams that Google Search is dead because Google doesn't
| innovate. The other - Google AI innovation is killing the web.
| spwa4 wrote:
| I find it very hard to believe that any of the major platforms
| is going to defend against AI slop. They're looking to
| monopolize it, which will reduce but not stop it in the short
| term, then maximize it later on for maximum dollars.
| ninth_ant wrote:
| Implicit in this article is the idea that you have to use Google,
| and that Google Search equals "the web". I'll be the boring nerd,
| I guess, who has to be the one to say there are many other search
| engines out there.
|
| If you don't like Google (I don't), or don't like where they are
| moving their products (I don't) -- please use a competing search
| engine.
|
| I've used DuckDuckGo for years and I don't miss anything about
| Google. There are plenty of other good search engines out there
| too.
|
| It's not particularly difficult to switch away and get an
| experience more to your liking, so it's a bit baffling when
| Google Search product decisions are equated with burying the web
| alive. Just don't use it!
| TonyTrapp wrote:
| When "to google" has literally become a verb in several
| languages, you know it's not going to be that easy to convince
| literally hundreds of millions of people to use something
| different. For many unsavvy people, Google is and continues to
| be their window into the web, and they wouldn't even know they
| can have it another way.
| NikolaNovak wrote:
| It's a seemingly patronizing joke meme, but I've repeatedly
| seen many of my relatives type in e.g. "Facebook.com" into
| Google search box, and then blindly click the first entry
| (which is completely at the mercy of being
| hijacked/prioritized based on either who's paying more money
| to Google, or latest fun extension / glitchy website they've
| installed).
|
| I have not yet found a working approach to alter that
| behaviour - obligatory XKCD reference [0]
|
| 0: https://xkcd.com/763/
| water-data-dude wrote:
| I wish Duck Duck Go was fewer syllables so I could try to
| force it into conversations as a verb.
| bustling-noose wrote:
| Definitely poor branding. It needs much better 2025
| branding and it'll probably take off in this AI search era
| where results are really poor tbh and very inaccurate at
| times.
| kevindamm wrote:
| let me quack that for you
| coredog64 wrote:
| Did you find that on quack?
| kevindamm wrote:
| You've seen the internet, but have you seen it _on
| quack_?
| shadowvoxing wrote:
| They own duck.com. They should revert to that.
| remyp wrote:
| People said this about AOL for a long time, too.
| TonyTrapp wrote:
| Sure, and that also didn't vanish in a day. All I'm saying
| is that there is a ton of inertia, and people will not be
| switching to something different in an instant.
| jsjohnst wrote:
| People (including me) said this for a long time about
| Yahoo! too. Kinda ironic they both merged and basically
| died together...
|
| Yawho!?
| grg0 wrote:
| Jawohl!
| jdgoesmarching wrote:
| I don't think AOL ever topped 40 million users while a
| significant chunk of humanity uses Google. This entire
| conversation hinges on their unprecedented scale and reach,
| I don't think this is an apt comparison.
| justsomehnguy wrote:
| You miss the 'A' in AOL.
| ninth_ant wrote:
| You don't have to convince hundreds of millions of people to
| use something different, you can just use something
| different.
| treve wrote:
| You have control over what you use to search the web, but you
| don't have control over the incentives of publishers, the
| content of which google used to get big and monetize and is now
| cannibalizing.
| ninth_ant wrote:
| That's a fair assessment.
|
| If the intent of the article was to say that Google is
| creating an environment which incentivizes publishers to lock
| away content from scrapers, then that's probably correct in
| the medium and long term.
|
| But I feel like the market described as being cannibalized
| here was already in the throes of death thanks to blogspam
| SEO. For many information categories, in my experience AI
| results are quite a bit better than the top-ranked results.
|
| And typically for those types of results you want to quickly
| shuffle off general web search into something more specific
| like Wikipedia or a site with domain expertise.
| bsder wrote:
| > in my experience AI results are quite a bit better than
| the top-ranked results.
|
| Only because the AI results haven't been infested with ads
| ... yet.
|
| As soon as the VC lottery stops, the ads will get injected
| and the AI stuff will quickly go just as bad.
| mouse_ wrote:
| I find it extremely challenging to believe you actually think
| that. I can't even host my own email anymore without all of my
| messages getting filtered as spam, because if your email
| service doesn't play nice with Google, then you might as well
| not exist. Not to mention youtube, adsense, Android (Google is
| trying to kill AOSP to sink their claws more deeply into
| Android in light of the recent antitrust suits),
|
| Google owns too much. They're easily on the same level as
| standard oil and the railroad tycoons.
|
| When a company is paying millions annually, just towards
| psychologists, to keep people hooked, telling people "lol just
| turn the computer off" is disingenuous at best.
| WalterBright wrote:
| I've been using a private email server for decades now, but
| it's become clear that none of the email I send using it
| arrives at its destination. I can receive email with them,
| but not send.
| quesera wrote:
| If this was a sudden and complete change, you may have
| landed on a spam blacklist.
| ninth_ant wrote:
| For one, I didn't say anything about email or you had to
| boycott Google completely. You can just change your browser
| to use another engine it takes like 10 seconds max. Google
| Search is not the web.
|
| Second, since you brought up email --- I've used Fastmail for
| a decade and it works just fine.
|
| > Google owns too much. They're easily on the same level as
| standard oil and the railroad tycoons.
|
| I agree 100% but since regulators largely refuse to deal with
| it, there's nothing stopping individual people like you or me
| to make our own decisions. You literally just have to click a
| few buttons if you're unhappy with the service Google Search
| is providing.
| bamboozled wrote:
| I used DuckDuckGo for 3 years, the only thing I found it useful
| for was the g! bang, in my opinion it wasn't a good search
| engine.
| ninth_ant wrote:
| If Google Search gives you the experience you want, you
| should probably keep using it. I was discussing options for
| people who are dissatisfied with their product or product
| direction.
| entropie wrote:
| What do you use now?
|
| I switched to kagi like half a year ago and it does the job
| but iam still unsure if I want to pay the 20 bucks for that
| service.
| bamboozled wrote:
| I went back to Google but I'm pretty addicted to Claude
| with the search plugin. I think it's insane.
|
| I don't like the Google AI summary thing because you can't
| seem to "converse" with it.
| entropie wrote:
| kagi has an AI summary tool to which is pretty good as
| far as i can tell.
| xyse53 wrote:
| I find that yearly works better for me psychologically for
| stuff like this.
|
| I got a 1 yr professional of kagi just to try it. IMO the
| results work. I've never seen Google do better when I
| compare; I have seen the Google AI responses be
| consistently straight up wrong.
|
| To me it's worth the cost knowing I'm paying a sustainable
| rate for a service. Plus I want no part in whatever the
| hell Google is doing these days with search.
| panny wrote:
| I've been using Brave search and it has had the AI answers for
| much longer than Google has. I would even go so far as to say
| Google stole the idea from Brave, but then someone will point
| out a different search engine that was doing it longer than
| Brave.
| mitchbob wrote:
| https://archive.ph/2025.05.26-092025/https://nymag.com/intel...
| jameslk wrote:
| Business idea: build a firewall for AI crawlers that variably
| gives them access to customer's sites based on traffic they send.
| The more traffic they send, the more they are allowed to crawl.
| mattferderer wrote:
| Cloudflare has been talking about this for a while albeit
| slightly different than your take -
| https://blog.cloudflare.com/ai-labyrinth/
| jameslk wrote:
| Yeah that was actually what I was thinking of for the idea,
| but the problem with their firewall is that it doesn't
| penalize the big crawlers, like Google's bot, because Google
| still sends traffic. Even if Google is sending less and less
| traffic over time.
|
| That just gives the biggest companies the prize of being able
| to crawl all the web for free, while they offer less over
| time in exchange.
| jrm4 wrote:
| The hardest issue in "fixing" this is "reminding people that the
| web is open and you can kind of do whatever you want here."
| kevin_thibedeau wrote:
| I have to resort to using ChatGPT to find any historical
| information because of the recency bias in all conventional
| search engines. Never mind their penchant for stuffing results
| with e-commerce sites no matter how irrelevant they are to the
| query.
| bilater wrote:
| So if Google innovates with AI, it's a problem because they're
| hiding their "ten blue links." But if they don't, they're
| suffering from the innovator's dilemma and are going to die with
| their ten blue links. The fact is, Google is so big that they're
| always going to upset some people.
|
| Embracing AI and moving forward is the only thing they can do.
| kevingadd wrote:
| Why is it a foregone conclusion that embracing AI is the only
| way to move forward, and that embracing AI is necessary to
| succeed?
|
| I'm certainly curious to see how it works out - it's a possible
| source of growth etc - but it's weird to me seeing people just
| assume that AI is not only The Future but is The Only Future.
| Nasrudith wrote:
| It is because it is easier to imagine the current trend
| taking off and staying than imaging the alternative tech line
| that will beat both AI and the ten blue links.
| dnissley wrote:
| Because AI/LLMs are clearly a 10x+ improvement over a
| significant number of use cases for search. And LLMs will be
| the thing performing the searches and filtering results in
| the remaining use cases.
| Rastonbury wrote:
| What I want from Google is showing me results (not summaries)
| from sources that I tell it from natural language. If I know I am
| going to search for a topic I know is going to be filled with a
| ton of SEO garbage, I default to chatgpt to get sources for me,
| unfortunately I still do not trust chatgpt to have done a
| thorough job but sometimes I can't be bother to trawl through
| blogspam.
|
| The reason why chatgpt and perplextty are stealing queries from
| them is because Search has become so bad.
| nicbou wrote:
| I can weigh in on this. I run a small information website
| (https://allaboutberlin.com). My traffic has been growing
| steadily for 7 years, but now its down 30-ish percent year on
| year. From what I'm hearing, my competitors (and friends) are
| having it worse.
|
| My main mission is to put new information on the internet. It's
| harder to do this if AI destroys the economics of it. It's also
| harder without an audience who provides feedback and
| encouragement. Having all information mediated by two companies
| isn't just digging into my revenue, it's killing the fun.
|
| The unfortunate side effect is that it made me focus more on
| business and less on serving my readers. I'm working on health
| insurance stuff when I would rather work on a citizenship guide.
| It's also sad to see all the nuance stripped from my carefully
| chosen words.
|
| It sucks, honestly. I am being replaced by AI, but I am still
| supposed to go experience the real world and translate it into
| LLM training material. In the end, there still is a thinking,
| feeling human being doing the essential work. He's just not
| getting paid anymore.
| boplicity wrote:
| > My main mission is to put new information on the internet.
| It's harder to do this if AI destroys the economics of it. It's
| also harder without an audience who provides feedback and
| encouragement.
|
| I also run an online website that provides valuable
| information. However, we have a very different philosophy. The
| mission is very not _not_ to "put new information on the
| internet." Rather, it is to provide a valuable and much needed
| service for people in our niche. While we do greatly benefit
| from SEO, we also work very hard to cultivate our audience,
| both in terms of finding new people, and nurturing our
| relationship with the existing ones. "Information on the
| internet" has zero inherent value without an audience. If
| you're not working to build that audience (outside of third
| party platforms), then it's going to be very tough in the long
| run, and you'll run into situations exactly like this.
|
| FWIW, our SEO traffic has remained shockingly steady, though
| the search console has reported a dramatic increase in
| "impressions," likely due to featuring our website in AI
| generated summaries. We'll see if things change over the long-
| term, but for now, we're not changing any strategies based on
| this.
| nicbou wrote:
| The problem is that there is no audience without a third-
| party platform nowadays. At least not at a scale.
|
| Google was the lesser evil, but I still fully expected to get
| killed by an algorithm update some day.
|
| I vowed to ride this website into the ground if I have to. I
| believe in its value, even if I need a side job to support
| it.
| CobrastanJorji wrote:
| There is no "nowadays." There never an era of big Internet
| audiences without third party platforms. The early web only
| had small audiences, and the reason it worked was that
| people were okay with that.
| tough wrote:
| You could have a small audience of a few million users,
| not try to compete with the giants, and still, be serving
| millions of users (daily)
|
| The internet was a beautiful place
| DataDaemon wrote:
| This is a great information website, congrats, Google is happy:
| "thanks for feeding me", they will scrape all and display on
| their site in the nice table. Sorry about that.
| asdhtjkujh wrote:
| I came across your website organically and it's been a great
| resource for me in the last few months, including times when
| the answers presented by Gemini have been definitively and
| legally incorrect. Thank you for organizing all of this
| information so clearly.
| bwb wrote:
| Ya, I'm in a similar boat (https://shepherd.com). Despite sky-
| high engagement stats, we lost 70% of our traffic from Google.
| Luckily, we have a strong brand, a growing community, and get
| decent traffic directly (and from other search engines).
|
| We are surviving, but we've changed our entire focus as the web
| we know is a dead man walking. AI bots are the new frontend for
| the web, and I am curious about what the web will look like in
| 2 to 10 years. I hope the AI companies start paying to access
| the data as they answer queries or construct dynamic frontends
| (but I doubt it).
|
| I have many friends who ran great indie websites, and it has
| been sad to see that 90% of them are now frozen or closed.
| ElFitz wrote:
| > He's just not getting paid anymore.
|
| Or the credit. Or even just the feedback.
|
| Even LLMs get at least some feedback.
| nicbou wrote:
| This is a surprisingly taxing change. There's a difference
| between giving free eggs to your neighbours, and giving free
| eggs to the Holiday Inn across town.
| RobRivera wrote:
| We need a new information backbone
| mike_hearn wrote:
| If you want readers more than impact you might want to consider
| something like a Substack. The paying audience would be
| smaller, but, it's not crawlable by AI (at least not today).
|
| Still, it's a bit ironic that your own website advises people
| to use bots to get apartments (thus bypassing the work put into
| the adverts and apartment hunting sites), whilst also decrying
| bots who would bypass your site.
|
| More generally perhaps there's a market for a Steam-like walled
| garden but for "old web" style sites. Crawlers would be
| controlled via technical means, micropayments would be built
| in, and use of AI would be strictly controlled. For instance
| maybe the garden uses AI to build link directories like DMOZ
| used to be, but the information itself isn't trained on so
| you're guaranteed to have organic readers.
| qkeast wrote:
| >Still, it's a bit ironic that your own website advises
| people to use bots to get apartments (thus bypassing the work
| put into the adverts and apartment hunting sites), whilst
| also decrying bots who would bypass your site.
|
| I believe these are pretty different scenarios. There's an
| ongoing, massive shortage of apartments in Berlin, and bots
| here often aren't bypassing "work put into adverts and
| apartment hunting sites"--they're trying to get an
| application in _at all_ before the listing is flooded with
| hundreds of applications and disabled. The content of the
| application has nothing to do with it.
|
| (Of course, this just leads to an arms race of apartment-
| application-bots, but this is the situation the city's found
| itself in. Same goes for registration appointments, residence
| permit applications, and so on.)
| nicbou wrote:
| If you have a zero sum game, any advice you give is
| creating an arms race. The only solution is to have fewer
| Berliners or more housing.
| nicbou wrote:
| A walled garden or paywalled website would go completely
| against the website's ethos. Everything is meant to be free
| for everyone. "Information wants to be free." Besides, those
| who stand to benefit the most from that information are the
| ones who can the least afford it.
|
| The bots are not a solution, they're table stakes.
| Withholding that information would not do anyone any justice.
| mattferderer wrote:
| Google has been getting this push back for the last 15 years.
| (That makes me feel old to remember it)
|
| Google Knowledge Graph (that sidebar they show) was hated by
| publishers, especially Wiki for stealing their content.
|
| Google adding direct answers to questions.
|
| Lots of fights over social media & recipe results.
|
| I'm not arguing who is right or wrong,* just saying this has been
| a thing for a long time.
|
| * Exception most recipe websites & those infinite looping
| Pinterest blog links. Those websites are all awful & wrong.
| DataDaemon wrote:
| Soon, quick and dirty solution: User-Agent: * Disallow: /
| cr125rider wrote:
| So then the internet police know to zoom up the fiber line and
| arrest the user-agent for not following robots.txt!
| Magi604 wrote:
| I used to hate the AI overview, but I've slowly been getting used
| to it and now I will often settle for what the AI gives. It's
| better than scrolling through loads of links and trying to weed
| out garbage SEO optimized sites.
| DataDaemon wrote:
| Results are OK, until you have this "garbage".
| somat wrote:
| How hard is it to get the infernal engine to work for you?
|
| "Belphegor-2 go wade through the search results and discard the
| low quality machine slop, I only want results linking to
| curated human generated pages."
| nicce wrote:
| At least in scientific writing, AI summaries have been proved
| to change the meaning of the results. So, use with caution.
| kazinator wrote:
| Story forget to mention that you can add "fucking ..." to your
| queries to eliminate the obnoxious and more often incorrect than
| not AI summaries.
| carlosjobim wrote:
| Lazy publishers could have solved this years ago, by making
| massive syndicates, paid by a single subscription. YouTube did
| it, and has become a modern wonder of the world.
|
| But no, of course they can't syndicate together with other
| newspapers or magazines who aren't 100% psychotically equally
| aligned with them politically.
|
| So they get what's coming to them. They shouldn't have been
| megalomaniacs, but that's all they are now.
| shmerl wrote:
| I noticed that Google image reverse search turned into completely
| unusable garbage that can only produce AI lookalikes but
| completely fails at finding original images. Literally every
| time, exact match - zero results.
|
| I guess that's their current trend. Forget about quality search,
| inject AI everywhere.
| guluarte wrote:
| the only thing google will win doing this in the long term is new
| laws that you'll need to opt in for training AI models
| baq wrote:
| I'm finding that google's own gemini flash is cannibalizing
| google search in my own usage, but there's no other way for them
| to survive being eaten alive by the competition.
|
| The sad realization is you probably shouldn't expect humans to
| read your content anymore - it'll be quoted verbatim (best case)
| by the all-consuming LLMs, just without your ads, your layout or
| anything yours at all that isn't your written or spoken words.
| (Sad part is that's often an improvement until chat interfaces
| start putting product placement in their responses. We can truly
| bury the web then.)
| amadeuspagel wrote:
| Yeah when I'm scrolling past the AI overview to click on the
| links it feels like I'm digging out someone buried alive with my
| bare hands to breathe new life into him.
| DisjointedHunt wrote:
| In my humble opinion, Google is the reason the web withstood as
| long as it did. At least the open part of the web did.
|
| Yes, sure, they monetized, but also they gave back as much as,
| and if not more than they took. We have so many machine learning
| frameworks, tensorflow, research, payouts to creators,
| advertising opportunities, careers, products, a lot of things
| built and taken down but most importantly built. They were
| probably the most positive force for the internet age in the past
| 20 years and more than anyone will ever give them credit for.
| Only in retrospect will we realize how lucky we were to be alive
| in the Google age. Full stop.
|
| What really killed the web was the rise of closed wall gardens
| platforms such as Apple, Facebook, Instagram and others. Putting
| up walls around content that didn't need to necessarily exist or
| not honoring open frameworks to exchange information and making
| things more widely indexable.
|
| But even here there have been significant benefits. The present
| AI boom would arguably not have been as large as it is right now
| without Mark Zuckerberg choosing to put an unconventional amount
| of investment behind AR ambitions to take on Apple, an investment
| the size of which many conventionally run publicly listed or
| private enterprises could hardly imagine to take up, leading to
| the concentration of capital, talent, technology and hardware in
| a place that gave birth to open source Llama and others. Google
| as well was very well poised because of their investments in
| compute fueled by their business model which kept the web alive
| and also returned capital into places where computer scientists
| would be paid significant amounts of money and have job security
| and freedom of will. to do as they pleased as opposed to chasing
| a paycheck, working as a physicist at CERN or something.
|
| All I'm saying is this article does not fully capture how
| significant the positive outcomes from Google have been.
| mortsnort wrote:
| Perhaps we need legislation requiring LLMs to compensate the
| sources they steal from? If the LLMs kill the hand that feeds
| them, it's bad for everyone.
| intended wrote:
| Forgive this weirdly philosophical comment: In a year of dark
| thoughts I found yet another horror to contemplate, after reading
| a legal paper on the idea of the marketplace of ideas.
|
| A core concept underpinning our shared market place, is that its
| not about facts, its about ideas being exchanged.
|
| Critically, that exchange creates friction, debate and downstream
| of all that - society. I've commented elsewhere how the
| marketplace is broken today, and why.
|
| But with GenAI, theres a new, twist, that I can barely
| contemplate.
|
| See, our conception of this liberal democracy based society,
| exists in that exchange between people. Even a crappy exchange,
| is part of the process of learning and building our ideas. My
| half formed ideas, your half formed ideas, everyones ideas.
|
| We have considered what happens when your ideas don't get heard.
|
| But in no place, did we consider what happens when we bounce our
| ideas _into_ a machine, and that machine talks back. And if that
| machine talks back in a manner thats can be as good as what a
| person might write or text. Your half baked idea goes into the
| machine, you get text back that helps you get to a fully baked
| idea.
|
| But the counter party is the machine. Not a person. The way those
| ideas have been bouncing around, has changed.
| j_timberlake wrote:
| This is like the frog in the boiling water story, except Frog is
| worried about being boiled alive after the temperature has only
| risen .001 degrees, so he writes articles saying "Google is
| burying the web alive" begging for someone to save him.
| johneth wrote:
| Surely as the traffic Google sends to websites falls and falls,
| there'll come a tipping point where the websites themselves will
| feel there's no benefit to allowing Google to index them.
|
| If this happens, Google's generative AI slop will have less and
| less scaped and regurgitated content to surface to Google users.
| tmnvix wrote:
| I've said this before, but I sincerely believe that a human-
| curated internet directory would be a great resource. Maybe
| something along the lines of Yahoo Directory revived? Organised
| using the dewey system could be useful. A library for the
| internet.
|
| I'd love to be able to look up something like say 'shoe repair'
| and find a host of high quality resources such as videos,
| tutorials, suppliers, etc, all vetted by real people somehow.
| notnullorvoid wrote:
| I think the answer lies in a network of curated directories,
| and search within a localised range of the network.
|
| Many people already have a decent list of curated sites in the
| form of bookmarks, but bookmark software is horribly dated.
| Even if I limited my search to sites gathered from my own
| bookmarks, I'd find 95% of everything I'm ever searching for.
| notnullorvoid wrote:
| AI summaries aren't the problem, they are a poor solution to a
| overwhelming amount of slop content. Search engine results have
| been getting worse and worse for over a decade, and with a
| rapidly increasing collection of content the web is getting too
| big to search. AI will be necessary, but it's not going to change
| the course, it will be deployed on both the solution and the
| problem front.
|
| If I had to pick the biggest contributor to the problem it would
| be advertising. Which mainly means Google, and governments for
| allowing an ad and search monopoly under the same roof. People
| can make all the justifications they want about how ad revenue
| allowed businesses and individuals to prosper, and "free" access
| to content, but it was always borrowing against the future.
| anal_reactor wrote:
| literal mind control
| oliwarner wrote:
| Which is a problem for websites that only provide facts.
|
| If your whole business model is getting organic search engine
| traffic to answer a user's question (a time, a result, something
| they just want to know) then Google is going to eat your dinner
| and I'm completely ambivalent because _so many_ of the websites
| that have been so expertly sculpted to dominate SERPs, are what
| 's wrong with the web. It's why it deserves to be buried.
|
| Before posting this, I'd just seen this post about asking Forbes
| whether or not the latest Mission Impossible has a post-credit
| scene:
| https://bsky.app/profile/chrisplummer.bsky.social/post/3lpvx...
|
| > If you load this page it contacts 82 IP addresses executing 256
| separate HTTP transactions to download 18MB of data writing 64
| cookies to your device to tell you "no"
|
| The web has been on life-support for a while. Ad-blockers _just_
| make it bearable, but its existence as an information lookup
| system is nearly over. It 's a way to interact with online
| systems and --occasionally-- for reading some long-form content
| (which is increasingly written by LLMs anyway). Isn't the future
| confusing.
___________________________________________________________________
(page generated 2025-05-26 23:01 UTC)