[HN Gopher] Google blocked my Chrome extension so I created a we...
___________________________________________________________________
Google blocked my Chrome extension so I created a website to host
it
Author : jayass
Score : 157 points
Date : 2022-08-31 14:28 UTC (8 hours ago)
(HTM) web link (extensionhub.site)
(TXT) w3m dump (extensionhub.site)
| josefresco wrote:
| I use the Chrome Extension (available in the web store) Web
| Archives for this purpose:
| https://chrome.google.com/webstore/detail/web-archives/hklig...
| znpy wrote:
| I wonder if the various change imposed by the eu on the matter of
| alternative app stores also applies to web browsers extensions
| stores.
|
| Like, what if I want to run a different marketplace for
| chrome/firefox/safari extensions?
| jayass wrote:
| You like this idea can you upvoted in the product hunt:
| https://www.producthunt.com/products/extension-hub/reviews/n...
| 2Gkashmiri wrote:
| please bring a firefox version.
| swills wrote:
| Despite the name, https://github.com/iamadamdev/bypass-
| paywalls-chrome supports both Chrome and Firefox.
| thrdbndndn wrote:
| Since the Firefox installation is an .xpi file, I assume it's
| also blocked by Mozilla? Not saying I'm surprised, though,
| giving the nature of the extension.
| swills wrote:
| Not sure, perhaps the author of that one just hasn't tried
| to get it approved. There's also
| https://addons.mozilla.org/en-US/firefox/addon/bypass-
| paywal... and a number of other ones for Firefox that don't
| look to be blocked, so I don't think Mozilla is taking the
| same stance.
| [deleted]
| jayass wrote:
| It should actually work with firefox by using TamperMonkey
| extension. I'll update the instruction on how to make it work
| on Firefox
| kevincox wrote:
| You should be able to just load the script file in a userscript
| manager for firefox.
|
| According to the original link the URL for the script is https:
| //raw.githubusercontent.com/jorgefsilva/beatthatwall/m....
| dylan604 wrote:
| EGAD! Has the internet matured enough so that what was old is new
| again?
|
| Dedicated websites that are independent of a centralised
| controlling body? What a brilliant idea! Those ol' greybeards
| were on to something
| nomel wrote:
| I've had several YouTube ads telling me I should use a "hybrid"
| approach to self host some of my services, on hardware _in my
| building_ , to keep cost down. I had to check a calendar to see
| what year it was!
| dylan604 wrote:
| My first webserver was the most frankenputer I have used to
| date. I had a friend that was at one of the large colo
| companies in town. He had a bunch of computer parts that he
| cobbled together with no case. They were hanging on the wall
| so that the mobo was flat against a dry erase wall, the PSU
| was on the desk with cables up to the hanging mobo, the HDD
| was velcroed to the dry erase wall next to the mobo. Talking
| about keeping costs down.
| kube-system wrote:
| Installing extensions in developer mode is not a good solution.
|
| It's not good to train users to do this because this is what
| malicious extensions do. Also, this will produce popup warnings
| and/or they can be automatically disabled by the browser.
|
| I don't even run extensions that I have developed myself in
| developer mode because it's a PITA.
| mensetmanusman wrote:
| If paywall and ad-blocks were used 100%, I wonder if news would
| move behind app interfaces 100%. Would definitely be an
| interesting transition.
| jonas21 wrote:
| If I'm reading this correctly, the extension sends every URL you
| use it on to some random service hosted on Cloudflare workers
| [1]. That doesn't sound good...
|
| [1]
| https://raw.githubusercontent.com/jorgefsilva/beatthatwall/m...
| jayass wrote:
| Yes that is correct. The random web service fetches a copy of
| the cached version of the page.
|
| If you're interested I can show you the source code
| 3np wrote:
| Please do!
| freedomben wrote:
| +1, I'd love to see the server-side code as well!
| jayass wrote:
| Reach me on Twitter and I will give you temporary access
| to the repo
| wahnfrieden wrote:
| Will you license it for reuse?
| [deleted]
| hk__2 wrote:
| > If you're interested I can show you the source code
|
| This doesn't really solve the old problem that there's no
| proof the source code you show is the one that runs on
| Cloudflare. Couldn't that code run locally?
| jayass wrote:
| Yes, I believe It's possible to replicate the same
| behaviour locally. I'll try to do that in the next update
| mro_name wrote:
| why do so many need a CDN recently?
| aendruk wrote:
| Cloudflare Workers principally is hosting.
| therealmarv wrote:
| replace https:// of a paywall site with archive.is/
|
| no extension needed.
|
| And if you want to use Google cache (and Google is your main
| search engine) just add ? in front of URL and with two clicks
| (dots->cached) you get to the cached version of Google.
|
| again no extension needed.
|
| Stay safe.
| turtleman1338 wrote:
| Will it be possible for other developers to add their own
| extensions? The sites name suggests this.
| jayass wrote:
| Yes, the idea is host more extensions.
|
| I could use some help with that
| kierenj wrote:
| Are there any security implications to enabling developer mode?
| paulhodge wrote:
| For just flipping the switch, no I don't think there's any
| implication.
|
| For installing an unpacked extension:
|
| - Obviously you don't have the benefit of the Chrome store
| checking for abuse.
|
| - You'll need to read the manifest.json file yourself to see
| what permissions you're granting, because the warning popup
| doesn't show up when installing this way.
|
| - There's a few attacks that unpacked extensions can do because
| they can spoof their extension ID, and Chrome doesn't consider
| it a bug. See:
| https://bugs.chromium.org/p/chromium/issues/detail?id=130196...
| somehnacct3757 wrote:
| You will bypass the permission auth dialog, which is your last
| official chance to see what the code you're about to execute
| has access to.
|
| The extension developer could add a malicious permission + new
| code to exploit it and it would look the same as using
| developer mode to add a Hello World extension
| djbusby wrote:
| Well, you'd have to really trust the code you're loading.
| jayass wrote:
| Well, yes and no. By enabling the developer mode you can
| install any extension that was not verified by the google web
| store policy.
|
| As long you are careful with the extension you install manually
| you should be fine.
| ThunderSizzle wrote:
| Is there necessarily any more danger than installing a random
| exe from the internet on a typical Windows admin account with
| UAC turned off?
|
| e.g. from https://windirstat.net/download.html
| xhkkffbf wrote:
| Let me just say -- and I'll get voted down for this -- that I
| think breaching paywalls is generally immoral. While I can
| imagine some moral reasons, I think most people are doing it even
| though they can pay. They are taking someone's work and using it
| without permission while refusing to pay.
|
| Of course I may be biased because I make some of my living
| creating content behind paywalls. If free access and ad support
| were possible, I would choose them first. But they aren't. To me,
| this extension and the people who support it aren't any different
| from those who go into restaurants, order food, eat it and then
| leave without paying. And there will be those who trot out some
| extreme rhetoric about world hunger or something to justify their
| actions.
|
| Google is blocking this extension because it's bad for the web as
| a whole. Don't empower people who destroy the information sources
| and turn the web into a fact desert. Support those who create
| knowledge and share it fairly and equally, albeit at a price.
| chasil wrote:
| Is it immoral to use an adblocker? Many sites present extensive
| UI interruptions in the hopes that you will allow their ads.
|
| Is it immoral to browse with Javascript disabled? Does morality
| require foreign code to run on your local system in order for
| these intrusive elements to actively manipulate (and
| potentially exploit) you?
|
| Is it immoral to rewrite a url in such a way that access is
| allowed? See: https://news.ycombinator.com/item?id=16906571
|
| In the days of newsprint, ads might have been garish, but they
| were bound into the media and they could not track you.
| Advertising now involves so much more surveillance, that I
| think the majority of the immorality is not with the end user.
|
| ps I have upvoted you.
| vorpalhex wrote:
| The issue with paywalls is the bait and switch - you push the
| whole content for SEO then force users to pay money.
|
| Nobody cares if you want to sell the secrets of the universe
| for $5000/mo. We care about the bait and switch.
|
| By setting up this bait and switch, you hurt your own paywall -
| if the search engine can get a full copy, so can I. Want these
| extensions to stop working? Stop the bait and switch.
| robertlagrant wrote:
| What's the bait and switch involved?
| thayne wrote:
| The content is available to search engines, and they are
| highly optimized for SEO, so they show up high in search
| results. So users expect by clicking the link in the search
| result they can see the rest of the content. Then you get
| hit with a paywall.
| robertlagrant wrote:
| A third party that indexes the content for searchability
| and shows too much of it to its users is not bait and
| switch. Come on.
| vorpalhex wrote:
| That is the deal of being indexed. That which you share
| to the indexer is shared to the world.
| hk__2 wrote:
| > By setting up this bait and switch, you hurt your own
| paywall - if the search engine can get a full copy, so can I.
|
| I don't get this. They do whatever they want with their
| content. Editors give free book copies to journalists; that
| doesn't gives you the right to have one as well.
| vorpalhex wrote:
| You can't give a free copy to the local library to drive
| sales and be upset when people borrow the book and read it
| from the local library.
| kube-system wrote:
| And you cannot take a book from someone who asks for
| money, refuse to pay, and say "but it's free at the
| library!"
|
| The difference is consent.
| vorpalhex wrote:
| But we are literally getting the copy here from the
| library because the author gave the book to the library.
|
| Google is obeying robots.txt. Google caches a copy of
| what you serve and gives that out. That is part of the
| deal.
| hk__2 wrote:
| > You can't give a free copy to the local library to
| drive sales
|
| Nobody does that. The comparison doesn't hold.
| vorpalhex wrote:
| You are doing that when you let Google past the paywall.
| stickfigure wrote:
| ...and if they post their content to the public, the public
| can read it. Giving content to journalists is not the same
| as giving your content to Google to index in their search
| engine and caches. The later is not private.
| [deleted]
| kube-system wrote:
| Journalists post quotes and snippets, just as Google
| does. Not that it matters. Just because I gave something
| to someone else for free, doesn't mean I have to give it
| to you.
|
| The idea that a single public exhibition of a work is
| enough to invalidate any future sale of that work is
| totally silly. Are you gonna bust out the window of a
| Barnes & Noble because libraries exist?
| vorpalhex wrote:
| Google reproduces the entire page it searches. This is
| not some oversight or hidden secret - it is how it works.
| hk__2 wrote:
| > Giving content to journalists is not the same as giving
| your content to Google to index in their search engine
| and caches.
|
| It's exactly the same. You give access to an entity that
| serves to promote your content. The fact that Google's
| cache is accessible is a technical implementation detail;
| Google could remove access to that cache tomorrow and
| that wouldn't change anything.
| hedora wrote:
| Publishers give books to journalists with the expectation
| that the journalists will immediately put an electronic
| copy of the book in a publicly-available CDN and then
| spend nearly-unlimited resources promoting the free copy?
| vorpalhex wrote:
| > The fact that Google's cache is accessible is a
| technical implementation detail;
|
| It's intentional to fix the kind of abusive behavior you
| are engaging in. If you serve content to a search engine,
| that search engine will reproduce the content.
|
| Search engines are for publically available content.
|
| If you want to advertise, pay for it.
| geysersam wrote:
| Why is that relevant? They've created their content - if they
| want to share it with a search engine that's none of your
| business.
|
| The technology does not allow them to do this without sharing
| it with you as well. But that is ethically irrelevant.
|
| (Of course, we also deserve a good search engine allowing us
| to remove paywalled sites and seo spam from our results.)
| vorpalhex wrote:
| "FREE VACATIONS! (you have to sit through a timeshare
| presentation and there are catches"
|
| We widely regard the above as unethical.
|
| Adding content to a search engine literally says "you can
| come read this content!". That is the purpose of search
| engines. Even Google penalizes paywall behavior and will
| downrank them - which forces the paywall people to get more
| clever.
|
| Sorry but don't abuse search engines and users to sell your
| content. Buy ads like a big boy.
| geysersam wrote:
| They're doing something bad - so it's okay to steal their
| stuff. Did I get that right?
| vorpalhex wrote:
| They are being gamed by their own game. Nobody took their
| content - they submitted it to search engines and are
| upset the search engine provides a cached copy.
|
| If they don't like it they can not submit to search
| engines.
| geysersam wrote:
| > nobody took their content
|
| According to the law, someone did.
|
| Just because something's available on request from a
| server does not mean it's up for grabs.
| vorpalhex wrote:
| > Just because something's available on request from a
| server does not mean it's up for grabs.
|
| When you give the content out for free to the indexer,
| you have given a copy to the world. Google search is not
| your advertising machine - you can pay for that
| priviledge if you would care for it.
| abbusfoflouotne wrote:
| I wouldn't say it is necessarily immature to use a
| paywall as opposed to ads. I for one sometimes prefer
| paying for a site just so i dont have to see ads.
|
| Though I do agree, the bait and switch aspect of
| finessing seo and such leaves a bad taste in your mouth.
| Glad google penalizes such behavior
| vehemenz wrote:
| Publishers are subverting users' reasonable expectations
| that the information they search and receive hits for on
| Google is publicly accessible--why else would it appear in
| search?
|
| It's actually everyone's business because this creates a
| web that is less usable for everyone. If publishers were
| willing to commit and make their paid content server-side
| for customers only, they would have a stronger case against
| infringers.
| kube-system wrote:
| Since when are Google results 100% clearnet? This has
| never been the case, ever. Search "login" in Google and
| you'll get a lot of content that is not publicly
| accessible.
| gamacodre wrote:
| What? Those login pages are publicly accessible in their
| entirety.
| vorpalhex wrote:
| They are serving different content to Google than users,
| and are then upset the version they served Google is
| available to users.
| geysersam wrote:
| That's just your definition of "reasonable".
|
| They provide users with an expectation of what
| information they will receive _if they pay for the
| content_.
|
| The fact that this "glimpse" of the content pollutes your
| web searches is a search engine problem.
|
| It would be trivial to filter sites with paywalled
| content. But Google refuses to let you do that. Hope
| someone else will come along and help with that.
| vorpalhex wrote:
| It's the search engines definition of reasonable. Don't
| like it? Don't let the search engine get to the content.
| kube-system wrote:
| The fact that you _can_ do something that you are not
| authorized to do doesn 't mitigate the fact that you don't
| have _consent_ to do so. Just because we expect criminals to
| break the internet without consent doesn 't mean everyone is
| morally absolved to do the same.
| hedora wrote:
| Sites that serve content to Google's crawlers, but then
| slap up a paywall are spamming users, and unfairly drowning
| out content producers that are playing by the rules. Last
| time I checked, they were doing this without Google's
| consent.
|
| As for the read path, the sites consented to Google
| indexing their stuff, and Google consents to letting people
| read the crawler cache. I don't see the issue.
| jayass wrote:
| Couldn't agree more
| trasz wrote:
| Paywalls are generally immoral. They are designed to deprive
| users of their laws. Thus, anything that helps breach paywalls
| is morally good.
| Dwedit wrote:
| "Bypass Paywalls Clean" seems to work very well on Firefox.
| Nowado wrote:
| I'm going through another round of google reviewing my extension
| right now and it does suck, but I can't imagine deciding that
| getting people to enable developer mode, manually download and
| load my extension (every time there's an update, too) is the more
| likely path to adoption.
| jayass wrote:
| TamperMonkey extension helps with that. It allows automatic
| updates
| MichaelCollins wrote:
| Isn't Manifest v3 going to break TamperMonkey/etc?
| e44858 wrote:
| Looking at the source, it seems like the page is proxied through
| a CloudFlare worker black box called "cfworker-
| beatthatwall.jayass.workers.dev". It would be easier to trust if
| all the work was done locally.
| vehemenz wrote:
| Exactly. This type of extension should 100% be blocked from the
| official store. That said, the userscript is working as
| intended so far.
| tyingq wrote:
| Just for some context, Google did block this one, but there's
| a ton of extensions on the store that do similar things and
| are not blocked. Google's review process for chrome
| extensions is weak.
| dewey wrote:
| Any reason to use that one that only seems to list "Medium" as
| supported vs the often linked https://12ft.io (Unsure if official
| or inofficial extension:
| https://chrome.google.com/webstore/detail/12ft/nmccamaemgaap...)?
|
| Edit: Unfortunately https://12ft.io seems to be down right now, I
| hope it's temporary.
| jayass wrote:
| The fact that only list medium does not mean that do not work
| on another sites. It should work on most sites but was mainly
| tested on medium.
| jayass wrote:
| If you the previous extension that was blocked
| https://chrome.google.com/webstore/detail/beat-that-wall/fhp...
| you can see that it has way more downloads than https://12ft.io
| thrdbndndn wrote:
| I don't think it's temporary, if what the reviews say is to
| believe it has been dead for a while and has questionable
| practice to begin with.
| dewey wrote:
| I don't think that extension was an official one, it was not
| linked from the website so the reviews / download count don't
| really say much.
|
| https://web.archive.org/web/20220824175226/https://12ft.io/
| jayass wrote:
| They suffer from the same problem. They can't update the
| extension because google changed the policies. Now any
| attempt to remove paywalls is not allowed
| kevincox wrote:
| If you call the script main.user.js instead of
| main.tampermonkey.js tampermonkey and other userscript tools will
| pick up the script automatically instead of needing to paste the
| URL in. For example I put my userscripts at
| https://userscripts.kevincox.ca and just clicking any of them
| should offer to install if you have a userscript extension set
| up.
| hnbad wrote:
| Just a recommendation: the site calls itself "Extensions Hub"
| despite the domain being "extensionhub.site". This is confusing
| and makes the site less trustworthy. Additionally
| "extensionhub.site" doesn't sound super trustworthy to me to
| begin with (maybe because the TLD is rare and "extensionhub"
| sounds a lot bigger than how it presents itself).
|
| If the idea is to host other extensions on it as well, I'd
| suggest putting a little more effort into it so it feels like
| something that actually has extensions on it rather than blog
| posts. The page for the extension itself for example has none of
| the usual links or info about the author of the extension (the
| date and author feel like they're part of a blog post ABOUT the
| extension, not info about the extension itself because of the
| layout) and the actual link for the extension itself is a direct
| download link in the prose of the article itself.
|
| This may be because I'm European but the complete lack of info
| about who operates the site (no privacy policy, just a twitter
| link and a copyright statement linking back to the site itself)
| screams "scam" to me on top of the impression that this is a blog
| trying to present itself as an extension store.
|
| Please take this as constructive feedback but if I saw this site
| in the wild I'd assume it's malware.
| jayass wrote:
| Hi thanks for your feedback. The idea was to lunch quick and
| cheap, that's way I choose a .site TLD.
|
| If this grow enough in the next few months I will change some
| things including the TLD.
|
| Adding privacy policy is the next step. Again thanks for your
| constructive feedback.
| butz wrote:
| Technically Google is blocking all of Chrome extensions, as they
| require Google account to install.
| jayass wrote:
| I had an extension on web chrome store to remove paywall, now
| google don't let me updated it. So I create a website to host the
| extension.
| vorpalhex wrote:
| Thank you for also hosting this as a tampermonkey script!
| jayass wrote:
| You're welcome
| napolux wrote:
| What can go wrong with a website hosting chrome extension freely
| downloadable by everyone.
| TekMol wrote:
| From the description, it sounds like this extension just puts
| "https://webcache.googleusercontent.com/search?q=cache:" in front
| of the current url?
|
| If so, you can also do this via a simple bookmarklet:
| javascript:location.href='https://webcache.googleusercontent.com/
| search?q=cache:'+location.href;{}
|
| If you don't know what bookmarklets are: Edit any old bookmark
| and put the above line into the url field. Next time you click
| it, it will bring you to the Google cache of the current page you
| are on.
| IncRnd wrote:
| I do similar but use javascript:window.location.href="https://a
| rchive.is/newest/"+location.href for pulling from archive.is.
| That works for many pages.
| jayass wrote:
| That's not the only thing it does. But it's the starting point.
| After that ir removes some scripts so the paywall don't show
| again after you load the page
| thrdbndndn wrote:
| That doesn't seem to work well.
|
| For example, take this NYT article
| https://www.nytimes.com/2022/08/31/health/life-expectancy-co...
|
| Google cache:
|
| https://webcache.googleusercontent.com/search?q=cache:https:...
| -- 404
|
| OP's service
|
| https://cfworker-beatthatwall.jayass.workers.dev/?url=https:...
| -- works.
|
| However, one can adopt your bookmarklet to use OP's service
| when needed instead of installing extension/userscript that
| seem to match all the sites.
| scoopertrooper wrote:
| It doesn't work because there is no Google cache entry for
| that page. They've instructed Google not to cache it.
|
| > <meta data-rh="true" name="robots" content="noarchive, max-
| image-preview:large"/>
|
| So, OP must be using some other means to retrieve the page.
|
| In the case of NY Times, they're likely just grabbing the
| non-archived version and performing an operation similar to
| 12ft ladder.
|
| Google cache fetching seems like it might be an effective
| strategy for a site like Washington Post that have extremely
| effective paywall enforcement (till your turn off JS), but
| also allow Google cache.
___________________________________________________________________
(page generated 2022-08-31 23:01 UTC)