https://github.com/gorhill/uBlock/commit/a559f5f2715c58fea4de09330cf3d06194ccc897 Skip to content Sign up * Product + Features + Mobile + Actions + Codespaces + Copilot + Packages + Security + Code review + Issues + Discussions + Integrations + GitHub Sponsors + Customer stories * Team * Enterprise * Explore + Explore GitHub + Learn and contribute + Topics + Collections + Trending + Skills + GitHub Sponsors + Open source guides + Connect with others + The ReadME Project + Events + Community forum + GitHub Education + GitHub Stars program * Marketplace * Pricing + Plans + Compare plans + Contact Sales + Education [ ] * # In this repository All GitHub | Jump to | * No suggested jump to results * # In this repository All GitHub | Jump to | * # In this user All GitHub | Jump to | * # In this repository All GitHub | Jump to | Sign in Sign up {{ message }} gorhill / uBlock Public * Notifications * Fork 2.5k * Star 31.8k * Code * Issues 35 * Pull requests 2 * Actions * Projects 0 * Wiki * Security * Insights More * Code * Issues * Pull requests * Actions * Projects * Wiki * Security * Insights Permalink This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. Browse files Add experimental mv3 version This create a separate Chromium extension, named "uBO Minus (MV3)". This experimental mv3 version supports only the blocking of network requests through the declarativeNetRequest API, so as to abide by the stated MV3 philosophy of not requiring broad "read/modify data" permission. Accordingly, the extension should not trigger the warning at installation time: Read and change all your data on all websites The consequences of being permission-less are the following: - No cosmetic filtering (##) - No scriptlet injection (##+js) - No redirect= filters - No csp= filters - No removeparam= filters At this point there is no popup panel or options pages. The default filterset correspond to the default filterset of uBO proper: Listset for 'default': https://ublockorigin.github.io/uAssets/filters/badware.txt https://ublockorigin.github.io/uAssets/filters/filters.txt https://ublockorigin.github.io/uAssets/filters/filters-2020.txt https://ublockorigin.github.io/uAssets/filters/filters-2021.txt https://ublockorigin.github.io/uAssets/filters/filters-2022.txt https://ublockorigin.github.io/uAssets/filters/privacy.txt https://ublockorigin.github.io/uAssets/filters/quick-fixes.txt https://ublockorigin.github.io/uAssets/filters/resource-abuse.txt https://ublockorigin.github.io/uAssets/filters/unbreak.txt https://easylist.to/easylist/easylist.txt https://easylist.to/easylist/easyprivacy.txt https://malware-filter.gitlab.io/malware-filter/urlhaus-filter-online.txt https://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=1&mimetype=plaintext The result of the conversion of the filters in all these filter lists is as follow: Ruleset size for 'default': 22245 Good: 21408 Maybe good (regexes): 127 redirect-rule= (discarded): 458 csp= (discarded): 85 removeparams= (discarded): 22 Unsupported: 145 The fact that the number of DNR rules are far lower than the number of network filters reported in uBO comes from the fact that lists-to-rulesets converter does its best to coallesce filters into minimal set of rules. Notably, the DNR's requestDomains condition property allows to create a single DNR rule out of all pure hostname-based filters. Regex-based rules are dynamically added at launch time since they must be validated as valid DNR regexes through isRegexSupported() API call. At this point I consider being permission-less the limiting factor: if broad "read/modify data" permission is to be used, than there is not much point for an MV3 version over MV2, just use the MV2 version if you want to benefit all the features which can't be implemented without broad "read/modify data" permission. To locally build the MV3 extension: make mv3 Then load the resulting extension directory in the browser using the "Load unpacked" button. From now on there will be a uBlock0.mv3.zip package available in each release. * Loading branch information @gorhill gorhill committed Sep 6, 2022 1 parent 1def4e7 commit a559f5f2715c58fea4de09330cf3d06194ccc897 Show file tree Hide file tree Showing 28 changed files with 1,651 additions and 368 deletions. Split Unified [ ] * .github/workflows + .github/workflows/main.yml main.yml * Makefile Makefile * platform + common o platform/common/vapi-common.js vapi-common.js + mv3 o extension # platform/mv3/extension/background.js background.js # img @ platform/mv3/extension/img/icon_128.png icon_128.png @ platform/mv3/extension/img/icon_16.png icon_16.png @ platform/mv3/extension/img/icon_32.png icon_32.png @ platform/mv3/extension/img/icon_64.png icon_64.png # platform/mv3/extension/manifest.json manifest.json o platform/mv3/make-rulesets.js make-rulesets.js o platform/mv3/package.json package.json o platform/mv3/ruleset-config.js ruleset-config.js o platform/mv3/ublock.svg ublock.svg + npm o platform/npm/package-lock.json package-lock.json o platform/npm/package.json package.json * src + src/devtools.html devtools.html + js o src/js/assets.js assets.js o src/js/biditrie.js biditrie.js o src/js/devtools.js devtools.js o src/js/messaging.js messaging.js o src/js/redirect-engine.js redirect-engine.js o src/js/static-dnr-filtering.js static-dnr-filtering.js o src/js/static-filtering-parser.js static-filtering-parser.js o src/js/static-net-filtering.js static-net-filtering.js o src/js/storage.js storage.js * submodules + submodules/uAssets uAssets * tools + tools/make-mv3.sh make-mv3.sh + tools/make-nodejs.sh make-nodejs.sh There are no files selected for viewing 10 .github/workflows/main.yml [*] Show comments View file Edit file Delete file This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters Show hidden characters Original file Diff line Diff line change line number number @@ -45,6 +45,7 @@ jobs: tools/make-firefox.sh ${{ steps.release_info.outputs.VERSION }} tools/make-thunderbird.sh ${{ steps.release_info.outputs.VERSION }} tools/make-npm.sh ${{ steps.release_info.outputs.VERSION }} tools/make-mv3.sh all - name: Upload Chromium package uses: actions/upload-release-asset@v1 env: @@ -81,3 +82,12 @@ jobs: asset_path: dist/build/uBlock0_${{ steps.release_info.outputs.VERSION }}.npm.tgz asset_name: uBlock0_${{ steps.release_info.outputs.VERSION }}.npm.tgz asset_content_type: application/octet-stream - name: Upload Chromium MV3 package uses: actions/upload-release-asset@v1 env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} with: upload_url: ${{ steps.create_release.outputs.upload_url }} asset_path: dist/build/uBlock0.mv3.zip asset_name: uBlock0.mv3.zip asset_content_type: application/octet-stream 7 Makefile [*] Show comments View file Edit file Delete file This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters Show hidden characters Original file Diff line Diff line change line number number @@ -1,7 +1,7 @@ # https://stackoverflow.com/a/6273809 run_options := $(filter-out $@,$( MAKECMDGOALS)) .PHONY: all clean test lint chromium firefox npm dig \ .PHONY: all clean test lint chromium firefox npm dig mv3 \ compare maxcost medcost mincost modifiers record wasm sources := $(wildcard assets/resources/* src /* src/*/* src/*/*/* src/*/*/*/*) @@ -52,6 +52,11 @@ dig: dist/build/ uBlock0.dig dig-snfe: dig cd dist/build/uBlock0.dig && npm run snfe $( run_options) dist/build/uBlock0.mv3: tools/make-mv3.sh $( sources) $(platform) tools/make-mv3.sh all mv3: dist/build/uBlock0.mv3 # Update submodules. update-submodules: tools/update-submodules.sh 5 platform/common/vapi-common.js [*] Show comments View file Edit file Delete file This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters Show hidden characters Original file Diff line Diff line change line number number @@ -37,7 +37,10 @@ vAPI.setTimeout = vAPI.setTimeout || self.setTimeout.bind (self); vAPI.webextFlavor = { major: 0, soup: new Set() soup: new Set(), get env() { return Array.from(this.soup); } }; (( ) => { 65 platform/mv3/extension/background.js [*] Show comments View file Edit file Delete file This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters Show hidden characters Original Diff file line line Diff line change number number @@ -0,0 +1,65 @@ 'use strict'; import regexRulesets from '/rulesets/regexes.js' ; const dnr = chrome.declarativeNetRequest; dnr.setExtensionActionOptions({ displayActionCountAsBadgeText: true }); (async ( ) => { const allRules = []; const toCheck = []; for ( const regexRuleset of regexRulesets ) { if ( regexRuleset.enabled !== true ) { continue; } for ( const rule of regexRuleset.rules ) { const regex = rule.condition.regexFilter; const isCaseSensitive = rule.condition. isUrlFilterCaseSensitive === true; allRules.push(rule); toCheck.push(dnr.isRegexSupported({ regex, isCaseSensitive })); } } const results = await Promise.all(toCheck); const newRules = []; for ( let i = 0; i < allRules.length; i++ ) { const rule = allRules[i]; const result = results[i]; if ( result instanceof Object && result. isSupported ) { newRules.push(rule); } else { console.info(`${result.reason}: ${rule.condition .regexFilter}`); } } const oldRules = await dnr.getDynamicRules(); const oldRuleMap = new Map(oldRules.map(rule => [ rule.id, rule ])); const newRuleMap = new Map(newRules.map(rule => [ rule.id, rule ])); const addRules = []; const removeRuleIds = []; for ( const oldRule of oldRules ) { const newRule = newRuleMap.get(oldRule.id); if ( newRule === undefined ) { removeRuleIds.push(oldRule.id); } else if ( JSON.stringify(oldRule) !== JSON. stringify(newRule) ) { removeRuleIds.push(oldRule.id); addRules.push(newRule); } } for ( const newRule of newRuleMap.values() ) { if ( oldRuleMap.has(newRule.id) ) { continue; } addRules.push(newRule); } if ( addRules.length !== 0 || removeRuleIds. length !== 0 ) { await dnr.updateDynamicRules({ addRules, removeRuleIds }); } const dynamicRules = await dnr.getDynamicRules() ; console.log(`Dynamic rule count: ${dynamicRules. length}`); const enabledRulesets = await dnr. getEnabledRulesets(); console.log(`Enabled rulesets: ${enabledRulesets }`); console.log(`Available dynamic rule count: ${dnr .MAX_NUMBER_OF_DYNAMIC_AND_SESSION_RULES - dynamicRules.length}`); dnr.getAvailableStaticRuleCount().then(count => { console.log(`Available static rule count: ${ count}`); }); })(); BIN +3.92 KB platform/mv3/extension/img/icon_128.png View file Edit file Delete file Sorry, something went wrong. Reload? Sorry, we cannot display this file. Sorry, this file is invalid so it cannot be displayed. Viewer requires iframe. BIN +534 Bytes platform/mv3/extension/img/icon_16.png View file Edit file Delete file Sorry, something went wrong. Reload? Sorry, we cannot display this file. Sorry, this file is invalid so it cannot be displayed. Viewer requires iframe. BIN +971 Bytes platform/mv3/extension/img/icon_32.png View file Edit file Delete file Sorry, something went wrong. Reload? Sorry, we cannot display this file. Sorry, this file is invalid so it cannot be displayed. Viewer requires iframe. BIN +1.88 KB platform/mv3/extension/img/icon_64.png View file Edit file Delete file Sorry, something went wrong. Reload? Sorry, we cannot display this file. Sorry, this file is invalid so it cannot be displayed. Viewer requires iframe. 25 platform/mv3/extension/manifest.json [*] Show comments View file Edit file Delete file This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters Show hidden characters Original file Diff line number line Diff line change number @@ -0,0 +1,25 @@ { "author": "Raymond Hill", "background": { "service_worker": "background.js", "type": "module" }, "declarative_net_request": { "rule_resources": [ ] }, "description": "uBO Minus is permission-less experimental MV3-based network request blocker ", "icons": { "16": "img/icon_16.png", "32": "img/icon_32.png", "64": "img/icon_64.png", "128": "img/icon_128.png" }, "manifest_version": 3, "minimum_chrome_version": "101.0", "name": "uBO Minus (MV3)", "permissions": [ "declarativeNetRequest" ], "version": "0.1.0" } 235 platform/mv3/make-rulesets.js [*] Show comments View file Edit file Delete file This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters Show hidden characters Original Diff file line Diff line change line number number @@ -0,0 +1,235 @@ / ******************************************************************************* uBlock Origin - a browser extension to block requests. Copyright (C) 2022-present Raymond Hill This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see {http://www.gnu.org/licenses/}. Home: https://github.com/gorhill/uBlock */ 'use strict'; / ****************************************************************************** / import fs from 'fs/promises'; import process from 'process'; import rulesetConfigs from './ruleset-config.js'; import { dnrRulesetFromRawLists } from './js/static-dnr-filtering.js' ; / ****************************************************************************** / const commandLineArgs = (( ) => { const args = new Map(); let name, value; for ( const arg of process.argv.slice(2) ) { const pos = arg.indexOf('='); if ( pos === -1 ) { name = arg; value = ''; } else { name = arg.slice(0, pos); value = arg.slice(pos+1); } args.set(name, value); } return args; })(); / ****************************************************************************** / async function main() { const writeOps = []; const ruleResources = []; const regexRuleResources = []; const outputDir = commandLineArgs.get('output') || '.'; let goodTotalCount = 0; let maybeGoodTotalCount = 0; const output = []; const log = (text, silent = false) => { output.push(text); if ( silent === false ) { console.log(text); } }; const replacer = (k, v) => { if ( k.startsWith('__') ) { return; } if ( Array.isArray(v) ) { return v.sort(); } if ( v instanceof Object ) { const sorted = {}; for ( const kk of Object.keys(v).sort() ) { sorted[kk] = v[kk]; } return sorted; } return v; }; const isUnsupported = rule => rule._error !== undefined; const isRegex = rule => rule.condition !== undefined && rule.condition.regexFilter !== undefined; const isRedirect = rule => rule.action !== undefined && rule.action.type === 'redirect' && rule.action.redirect.extensionPath !== undefined; const isCsp = rule => rule.action !== undefined && rule.action.type === 'modifyHeaders'; const isRemoveparam = rule => rule.action !== undefined && rule.action.type === 'redirect' && rule.action.redirect.transform !== undefined; const isGood = rule => isUnsupported(rule) === false && isRedirect(rule) === false && isCsp(rule) === false && isRemoveparam(rule) === false ; const rulesetDir = `${outputDir}/rulesets`; const rulesetDirPromise = fs.mkdir(`${rulesetDir}`, { recursive: true }); const fetchList = url => { return fetch(url) .then(response => response.text()) .then(text => ({ name: url, text })); }; const readList = path => fs.readFile(path, { encoding: 'utf8' }) .then(text => ({ name: path, text })); const writeFile = (path, data) => rulesetDirPromise.then(( ) => fs.writeFile(path, data)); for ( const ruleset of rulesetConfigs ) { const lists = []; log(`Listset for '${ruleset.id}':`); if ( Array.isArray(ruleset.paths) ) { for ( const path of ruleset.paths ) { log(`\t${path}`); lists.push(readList(`assets/${path}`)); } } if ( Array.isArray(ruleset.urls) ) { for ( const url of ruleset.urls ) { log(`\t${url}`); lists.push(fetchList(url)); } } const rules = await dnrRulesetFromRawLists(lists, { env: [ 'chromium' ], }); log(`Ruleset size for '${ruleset.id}': ${rules.length}`); const good = rules.filter(rule => isGood(rule) && isRegex(rule) === false); log(`\tGood: ${good.length}`); const regexes = rules.filter(rule => isGood(rule) && isRegex(rule)); log(`\tMaybe good (regexes): ${regexes.length}`); const redirects = rules.filter(rule => isUnsupported(rule) === false && isRedirect(rule) ); log(`\tredirect-rule= (discarded): ${redirects.length}`); const headers = rules.filter(rule => isUnsupported(rule) === false && isCsp(rule) ); log(`\tcsp= (discarded): ${headers.length}`); const removeparams = rules.filter(rule => isUnsupported(rule) === false && isRemoveparam(rule) ); log(`\tremoveparams= (discarded): ${removeparams.length}`); const bad = rules.filter(rule => isUnsupported(rule) ); log(`\tUnsupported: ${bad.length}`); log( bad.map(rule => rule._error.map(v => `\t\t${v}`)).join('\n'), true ); writeOps.push( writeFile( `${rulesetDir}/${ruleset.id}.json`, `${JSON.stringify(good, replacer, 2)}\n` ) ); regexRuleResources.push({ id: ruleset.id, enabled: ruleset.enabled, rules: regexes }); ruleResources.push({ id: ruleset.id, enabled: ruleset.enabled, path: `/rulesets/${ruleset.id}.json` }); goodTotalCount += good.length; maybeGoodTotalCount += regexes.length; } writeOps.push( writeFile( `${rulesetDir}/regexes.js`, `export default ${JSON.stringify(regexRuleResources, replacer, 2)};\n ` ) ); await Promise.all(writeOps); log(`Total good rules count: ${goodTotalCount}`); log(`Total regex rules count: ${maybeGoodTotalCount}`); // Patch manifest const manifest = await fs.readFile(`${outputDir}/manifest.json`, { encoding: 'utf8' }) .then(text => JSON.parse(text)); manifest.declarative_net_request = { rule_resources: ruleResources }; const now = new Date(); manifest.version = `0.1.${now.getUTCFullYear() - 2000}.${now. getUTCMonth() * 100 + now.getUTCDate()}`; await fs.writeFile( `${outputDir}/manifest.json`, JSON.stringify(manifest, null, 2) + '\n' ); // Log results await fs.writeFile(`${outputDir}/log.txt`, output.join('\n') + '\n'); } main(); / ****************************************************************************** / 6 platform/mv3/package.json [*] Show comments View file Edit file Delete file This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters Show hidden characters Original file line number Diff line number Diff line change @@ -0,0 +1,6 @@ { "engines": { "node": ">=17.5.0" }, "type": "module" } Oops, something went wrong. Retry Toggle all file notes Toggle all file annotations 0 comments on commit a559f5f Please sign in to comment. Footer (c) 2022 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.