https://github.com/gorhill/uBlock/commit/6acf97bf51 Skip to content Navigation Menu Toggle navigation Sign in * Product + Actions Automate any workflow + Security Find and fix vulnerabilities + Codespaces Instant dev environments + GitHub Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions By size + Enterprise + Teams + Startups By industry + Healthcare + Financial services + Manufacturing By use case + CI/CD & Automation + DevOps + DevSecOps * Resources Topics + AI + DevOps + Security + Software Development + View all Explore + Learning Pathways + White papers, Ebooks, Webinars + Customer Stories + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Enterprise + Enterprise platform AI-powered developer platform Available add-ons + Advanced Security Enterprise-grade security features + GitHub Copilot Enterprise-grade AI features + Premium Support Enterprise-grade 24/7 support * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up Reseting focus You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} gorhill / uBlock Public * Notifications You must be signed in to change notification settings * Fork 3.1k * Star 46.4k * Code * Issues 12 * Pull requests 2 * Actions * Projects 0 * Wiki * Security * Insights Additional navigation options * Code * Issues * Pull requests * Actions * Projects * Wiki * Security * Insights Commit Permalink This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. Rewrite cname uncloaking code to account for new ipaddress= option Browse files Browse the repository at this point in the history This commit makes the DNS resolution code better suited for both filtering on cname and ip address. The change allows early availability of ip address so that `ipaddress=` option can be matched at onBeforeRequest time. As a result, it is now possible to block root document using `ipaddress=` option -- so long as an ip address can be extracted before first onBeforeRequest() call. Related issue: uBlockOrigin/uBlock-issues#2792 Caveat ------ the ip address used is the first one among the list of ip addresses returned by dns.resolve() method. There is no way for uBO to know which exact ip address will be used by the browser when sending the request, so this is at most a best guess. The exact IP address used by the browser is available at onHeadersReceived time, and uBO will also filter according to this value, but by then the network request has already been sent to the remote server. Possibly a future improvement would make available the whole list of ip addresses to the filtering engine, but even then it's impossible to know with certainty which ip address will ultimately be used by the browser -- it is entirely possible that the ip address used by the browser might not be in the list received through dns.resolve(). * Loading branch information @gorhill gorhill committed Sep 12, 2024 1 parent 44b6519 commit 6acf97b Show file tree Hide file tree Showing 4 changed files with 153 additions and 96 deletions. * Whitespace * Ignore whitespace * Split * Unified [ ] * platform + chromium o platform/chromium/manifest.json manifest.json + firefox o platform/firefox/vapi-background-ext.js vapi-background-ext.js + opera o platform/opera/manifest.json manifest.json * src/js + src/js/background.js background.js There are no files selected for viewing 2 changes: 1 addition & 1 deletion 2 platform/chromium/manifest.json [*] Show comments View file Edit file Delete file This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters Show hidden characters Original file line Diff line Diff line change number number Expand Up @@ -89,7 +89,7 @@ }, "incognito": "split", "manifest_version": 2, "minimum_chrome_version": "73 .0", "minimum_chrome_version": "80 .0", "name": "uBlock Origin", "options_ui": { "page": "dashboard.html", Expand Down 244 changes: 151 additions & 93 deletions 244 platform/firefox/ vapi-background-ext.js [*] Show comments View file Edit file Delete file This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters Show hidden characters Original Diff file line Diff line change line number number Expand Up @@ -26,8 +26,10 @@ import { / ****************************************************************************** / // Canonical name-uncloaking feature. let cnameUncloakEnabled = browser.dns instanceof Object; const dnsAPI = browser.dns; const isPromise = o => o instanceof Promise; const reIPv4 = /^\d+\.\d+\.\d+\.\d+$/ // Related issues: // - https://github.com/gorhill/uBlock/issues/1327 Expand All @@ -40,21 +42,24 @@ vAPI.Net = class extends vAPI.Net { constructor() { super(); this.pendingRequests = []; this.canUncloakCnames = browser.dns instanceof Object; this.cnames = new Map([ [ '', null ] ]); this.dnsList = []; // ring buffer this.dnsWritePtr = 0; // next write pointer in ring buffer this.dnsMaxCount = 256; // max size of ring buffer this.dnsDict = new Map(); // hn to index in ring buffer this.dnsEntryTTL = 60000; // delay after which an entry is obsolete this.canUncloakCnames = true; this.cnameUncloakEnabled = true; this.cnameIgnoreList = null; this.cnameIgnore1stParty = true; this.cnameIgnoreExceptions = true; this.cnameIgnoreRootDocument = true; this.cnameMaxTTL = 120; this.cnameReplayFullURL = false; this.cnameFlushTime = Date.now() + this.cnameMaxTTL * 60000; } setOptions(options) { super.setOptions(options); if ( 'cnameUncloakEnabled' in options ) { cnameUncloakEnabled = this.canUncloakCnames && this.cnameUncloakEnabled = options.cnameUncloakEnabled !== false; } if ( 'cnameIgnoreList' in options ) { Expand All @@ -73,15 +78,13 @@ vAPI.Net = class extends vAPI.Net { this.cnameIgnoreRootDocument = options.cnameIgnoreRootDocument !== false; } if ( 'cnameMaxTTL' in options ) { this.cnameMaxTTL = options.cnameMaxTTL || 120; } if ( 'cnameReplayFullURL' in options ) { this.cnameReplayFullURL = options.cnameReplayFullURL === true; } this.cnames.clear(); this.cnames.set('', null); this.cnameFlushTime = Date.now() + this.cnameMaxTTL * 60000; this.dnsList.fill(null); this.dnsDict.clear(); } normalizeDetails(details) { const type = details.type; Expand All @@ -104,6 +107,7 @@ vAPI.Net = class extends vAPI.Net { } } } denormalizeTypes(types) { if ( types.length === 0 ) { return Array.from(this.validTypes); Expand All @@ -122,75 +126,19 @@ vAPI.Net = class extends vAPI.Net { } return Array.from(out); } canonicalNameFromHostname(hn) { const cnRecord = this.cnames.get(hn); if ( cnRecord !== undefined && cnRecord !== null ) { return cnRecord.cname; } } processCanonicalName(hn, cnRecord, details) { if ( cnRecord === null ) { return; } if ( cnRecord.isRootDocument ) { return; } const hnBeg = details.url.indexOf(hn); if ( hnBeg === -1 ) { return; } const oldURL = details.url; let newURL = oldURL.slice(0, hnBeg) + cnRecord.cname; const hnEnd = hnBeg + hn.length; if ( this.cnameReplayFullURL ) { newURL += oldURL.slice(hnEnd); } else { const pathBeg = oldURL.indexOf('/', hnEnd); if ( pathBeg !== -1 ) { newURL += oldURL.slice(hnEnd, pathBeg + 1); } } details.url = newURL; details.aliasURL = oldURL; return super.onBeforeSuspendableRequest(details); } recordCanonicalName(hn, record, isRootDocument) { if ( (this.cnames.size & 0b111111) === 0 ) { const now = Date.now(); if ( now >= this.cnameFlushTime ) { this.cnames.clear(); this.cnames.set('', null); this.cnameFlushTime = now + this.cnameMaxTTL * 60000; } } let cname = typeof record.canonicalName === 'string' && record.canonicalName !== hn ? record.canonicalName : ''; if ( cname !== '' && this.cnameIgnore1stParty && domainFromHostname(cname) === domainFromHostname(hn) ) { cname = ''; } if ( cname !== '' && this.cnameIgnoreList !== null && this.cnameIgnoreList.test(cname) ) { cname = ''; } const cnRecord = cname !== '' ? { cname, isRootDocument } : null; this.cnames.set(hn, cnRecord); return cnRecord; if ( hn === '' ) { return; } const dnsEntry = this.dnsFromCache(hn); if ( isPromise(dnsEntry) ) { return; } return dnsEntry?.cname; } regexFromStrList(list) { if ( typeof list !== 'string' || list.length === 0 || list === 'unset' || browser.dns instanceof Object === false ) { if ( typeof list !== 'string' || list.length === 0 || list === 'unset' ) { return null; } if ( list === '*' ) { return /^./; } if ( list === '*' ) { return /^./; } return new RegExp( '(?:^|\\.)(?:' + list.trim() Expand All @@ -200,9 +148,14 @@ vAPI.Net = class extends vAPI.Net { ')$' ); } onBeforeSuspendableRequest(details) { const hn = hostnameFromNetworkURL(details.url); const dnsEntry = this.dnsFromCache(hn); if ( dnsEntry?.ip ) { details.ip = dnsEntry.ip; } const r = super.onBeforeSuspendableRequest(details); if ( cnameUncloakEnabled === false ) { return r; } if ( r !== undefined ) { if ( r.cancel === true || Expand All @@ -212,25 +165,128 @@ vAPI.Net = class extends vAPI.Net { return r; } } const hn = hostnameFromNetworkURL(details.url); const cnRecord = this.cnames.get(hn); if ( cnRecord !== undefined ) { return this.processCanonicalName(hn, cnRecord, details); if ( dnsEntry !== undefined ) { if ( isPromise(dnsEntry) === false ) { return this.onAfterDNSResolution(hn, details, dnsEntry); } } if ( details.proxyInfo && details.proxyInfo.proxyDNS ) { return; } const documentUrl = details.documentUrl || details.url; const isRootDocument = this.cnameIgnoreRootDocument && hn === hostnameFromNetworkURL(documentUrl); return browser.dns.resolve(hn, [ 'canonical_name' ]).then( rec => { const cnRecord = this.recordCanonicalName(hn, rec, isRootDocument); return this.processCanonicalName(hn, cnRecord, details); }, ( ) => { this.cnames.set(hn, null); if ( this.dnsShouldResolve(hn) === false ) { return; } if ( details.proxyInfo?.proxyDNS ) { return; } const promise = dnsEntry || this.dnsResolve(hn, details); return promise.then(( ) => this.onAfterDNSResolution(hn, details)); } onAfterDNSResolution(hn, details, dnsEntry) { if ( dnsEntry === undefined ) { dnsEntry = this.dnsFromCache(hn); if ( dnsEntry === undefined || isPromise(dnsEntry) ) { return; } } let proceed = false; if ( dnsEntry.cname && this.cnameUncloakEnabled ) { const newURL = this.uncloakURL(hn, dnsEntry, details); if ( newURL ) { details.aliasURL = details.url; details.url = newURL; proceed = true; } } if ( dnsEntry.ip && details.ip !== dnsEntry.ip ) { details.ip = dnsEntry.ip proceed = true; } if ( proceed === false ) { return; } // Must call method on base class return super.onBeforeSuspendableRequest(details); } dnsToCache(hn, record, details) { const i = this.dnsDict.get(hn); if ( i === undefined ) { return; } const dnsEntry = { hn, until: Date.now() + this.dnsEntryTTL, }; if ( record ) { const cname = this.cnameFromRecord(hn, record, details); if ( cname ) { dnsEntry.cname = cname; } const ip = this.ipFromRecord(record); if ( ip ) { dnsEntry.ip = ip; } } this.dnsList[i] = dnsEntry; return dnsEntry; } dnsFromCache(hn) { const i = this.dnsDict.get(hn); if ( i === undefined ) { return; } const dnsEntry = this.dnsList[i]; if ( dnsEntry === null ) { return; } if ( isPromise(dnsEntry) ) { return dnsEntry; } if ( dnsEntry.hn !== hn ) { return; } if ( dnsEntry.until >= Date.now() ) { return dnsEntry; } this.dnsList[i] = null; this.dnsDict.delete(hn) } dnsShouldResolve(hn) { if ( hn === '' ) { return false; } const c0 = hn.charCodeAt(0); if ( c0 === 0x5B /* [ */ ) { return false; } if ( c0 > 0x39 /* 9 */ ) { return true; } return reIPv4.test(hn) === false; } dnsResolve(hn, details) { const i = this.dnsWritePtr++; this.dnsWritePtr %= this.dnsMaxCount; this.dnsDict.set(hn, i); const promise = dnsAPI.resolve(hn, [ 'canonical_name' ]).then( rec => this.dnsToCache(hn, rec, details), ( ) => this.dnsToCache(hn) ); return (this.dnsList[i] = promise); } cnameFromRecord(hn, record, details) { const cn = record.canonicalName; if ( cn === undefined ) { return; } if ( cn === hn ) { return; } if ( this.cnameIgnore1stParty ) { if ( domainFromHostname(cn) === domainFromHostname(hn) ) { return; } } if ( this.cnameIgnoreList !== null ) { if ( this.cnameIgnoreList.test(cn) === false ) { return; } } if ( this.cnameIgnoreRootDocument ) { const origin = hostnameFromNetworkURL(details.documentUrl || details. url); if ( hn === origin ) { return; } } return cn; } uncloakURL(hn, dnsEntry, details) { const hnBeg = details.url.indexOf(hn); if ( hnBeg === -1 ) { return; } const oldURL = details.url; const newURL = oldURL.slice(0, hnBeg) + dnsEntry.cname; const hnEnd = hnBeg + hn.length; if ( this.cnameReplayFullURL ) { return newURL + oldURL.slice(hnEnd); } const pathBeg = oldURL.indexOf('/', hnEnd); if ( pathBeg !== -1 ) { return newURL + oldURL.slice(hnEnd, pathBeg + 1); } return newURL; } ipFromRecord(record) { const { addresses } = record; if ( Array.isArray(addresses) === false ) { return; } if ( addresses.length === 0 ) { return; } return addresses[0]; } suspendOneRequest(details) { const pending = { details: Object.assign({}, details), Expand All @@ -243,6 +299,7 @@ vAPI.Net = class extends vAPI.Net { this.pendingRequests.push(pending); return pending.promise; } unsuspendAllRequests(discard = false) { const pendingRequests = this.pendingRequests; this.pendingRequests = []; Expand All @@ -254,6 +311,7 @@ vAPI.Net = class extends vAPI.Net { ); } } static canSuspend() { return true; } Expand Down 2 changes: 1 addition & 1 deletion 2 platform/opera/manifest.json [*] Show comments View file Edit file Delete file This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters Show hidden characters Original file line Diff line Diff line change number number Expand Up @@ -88,7 +88,7 @@ }, "incognito": "split", "manifest_version": 2, "minimum_opera_version": "60.0 ", "minimum_opera_version": "67.0 ", "name": "uBlock Origin", "options_page": " dashboard.html", "permissions": [ Expand Down 1 change: 0 additions & 1 deletion 1 src/js/background.js [*] Show comments View file Edit file Delete file This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters Show hidden characters Original file line Diff line Diff line change number number Expand Up @@ -59,7 +59,6 @@ const hiddenSettingsDefault = { cnameIgnore1stParty: true, cnameIgnoreExceptions: true, cnameIgnoreRootDocument: true, cnameMaxTTL: 120, cnameReplayFullURL: false, consoleLogLevel: 'unset', debugAssetsJson: false, Expand Down Toggle all file notes Toggle all file annotations 0 comments on commit 6acf97b Please sign in to comment. Footer (c) 2024 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact * Manage cookies * Do not share my personal information You can't perform that action at this time.