[HN Gopher] Show HN: DNS-powered website with no back end
___________________________________________________________________
Show HN: DNS-powered website with no back end
Author : elliottinvent
Score : 70 points
Date : 2021-06-22 22:37 UTC (3 days ago)
(HTM) web link (companydirectory.uk)
(TXT) w3m dump (companydirectory.uk)
| darkr wrote:
| Sounds similar to Telnic
|
| https://datatracker.ietf.org/doc/html/draft-reid-dnsext-nkey...
|
| https://icannwiki.org/TelNic
| elliottinvent wrote:
| Thanks for your comment and for highlighting this.
|
| As a summary for others: Telnic (and Telnames) allowed people
| to store contact data in the DNS of their .tel domain name.
|
| > Sounds similar to Telnic
|
| Telnic used NAPTR records and was only available on .tel
| domains, NUM uses TXT and can be used on any domain name, by
| publishing a "_num" zone.
|
| Telnic was also "just" for contact data. What we're trying to
| do with NUM is make it possible to publish any kind of data.
| Contact data as just one "module".
| fredrivett wrote:
| The other key difference here as far as I understand, is that
| Telnic required you to register .tel domains, you couldn't
| bring your own domain and store contact information against
| that.
| o8r3oFTZPE wrote:
| The idea I had for this over 10 years ago was to modify djb's
| dnstxt to output raw HTML with a MIME header. tinydns allows one
| to store arbitrary data so I could put anything in a 512-byte DNS
| packet, including "text/html" (newlines and carriage returns). I
| could request tiny web pages from tinydns. There have been other
| similar things by other folks like putting deCSS code in DNS,
| Wikipedia data, audio files, etc.
|
| Now this might all sound silly, but since then 1. DNS packets are
| now massive and can carry much more data and 2. big companies are
| pushing a "next-gen" UDP-based protocol (reminds me of djb's
| CurveCP) for HTTP, serving HTML and other web junk. They also
| want to use this UDP-based protocol for other things, like DNS,
| eventually.
|
| Whatever. HTML in DNS worked great. Tiny pages that fit in a CPU
| cache.
| o8r3oFTZPE wrote:
| Putting URLs in DNS. Could be useful.
| unknown_error wrote:
| I think you're leeching off someone else's infrastructure and
| using it to do things they never meant it to do. Sure, the
| technical capability is there, but your use case would
| drastically increase their costs. You are essentially cost-
| shifting your customers' costs onto theirs. Not cool.
|
| It's like building a cloud storage solution off Gmail's free
| storage. It can be done, has been done, but that doesn't mean
| it's cool to do so.
|
| Your system would increase costs for DNS providers all over the
| world, without their consent, just because you're using it as a
| loophole. It was a problem that wasn't there fixed in a way that
| leeches from rather than gives back to the community.
| bfung wrote:
| In a cursory read through, I'm still not sure what problem NUM
| actually solves. It seems like it moves data storage into DNS TXT
| records and defines some parsers.
|
| What benefits does NUM have over html over http? One can use some
| semantic html tags to organize the data for parsing, making it
| roughly equivalent to NUM modules. If a NUM client is running w/o
| a newly developed NUM module, how does it parse that data and how
| is it better than html without semantic meaning?
| elliottinvent wrote:
| > What benefits does NUM have over html over http?
|
| Web standards (JSON-LD, microdata, RDFa) are inefficient, since
| you have to download the entire HTML to find a particular piece
| of data like a phone number.
|
| Web standards are also not widely adopted enough for developers
| to build something on top of that data. For example, web
| standards can't be used to find a phone number for _any_
| company with a phone number published to their website, since
| many small businesses don't mark up phone numbers with semantic
| web data.
|
| We're pre-populating NUM with millions of pieces of data so if
| a phone number is on the public website, it'll be in the NUM
| record for that domain.
|
| > One can use some semantic html tags to organize the data for
| parsing, making it roughly equivalent to NUM modules.
|
| Of course, the problem is that not enough websites do this
| (especially those of small business).
|
| > If a NUM client is running w/o a newly developed NUM module,
| how does it parse that data and how is it better than html
| without semantic meaning?
|
| If NUM is used for general purpose data storage (instead of for
| a particular module), then parsing of that data would need to
| be handled in a bespoke client - not a general purpose NUM
| client. Any advantages of using NUM over HTTP in this example
| but would be related to scaling and cost savings only.
| elliottinvent wrote:
| All of the contact data on this website is fetched from DNS TXT
| records. It's an example of how NUM [0] data can be used.
|
| A small team and I have created NUM because we believe that paid,
| restricted, rate-limited APIs provided by the giants of the web
| are holding back developers and we want to change that.
|
| NUM provides: a simple way for businesses to provide machine-
| readable data direct to their users, bypassing data hoarding web
| giants; a forever free, unlimited and unrestricted source of data
| to developers.
|
| The general-purpose semantic web has failed because the standards
| (microdata, JSON-LD, Open Graph) are too complicated and
| difficult for small businesses to adopt and often forgotten about
| by marketing teams in large businesses because data is too
| difficult to edit.
|
| NUM is different because it can be adopted by any business using
| a simple online form. The protocol includes two DNS queries - the
| first to the authoritative DNS zone; the second failover query to
| our NUM Server, any business can claim their domain on NUM Server
| using a simple online process.
|
| The Company Directory website linked to is just one example of
| how NUM data can be used. The website fetches all data from DNS
| using client-side Cloudflare DNS-over-HTTPS calls (see Network
| tab in Dev Tools). Barclays, or any other company listed can
| update the data on this site by adopting the NUM protocol [1]
| independently in their own DNS - for example, to replace the data
| for the Barclays listing, they can create a TXT record at
| 1._num.barclays.co.uk or claim the record shown by visiting
| numserver.com/claim/barclays.co.uk
|
| DNS-based protocols like NUM (other examples are SPF and DMARC)
| typically suffer from the chicken-and-egg problem where no-one
| looks for this data in DNS because it's not there, and no one
| stores this data in DNS because no-one is looking for it.
|
| To help overcome this, we've pre-populated the DNS with contact
| data for large UK companies and are in the process of
| automatically gathering contact data for all *.uk domain names
| (we expect to complete this in the next 10 days) before moving on
| to all other UK companies, then companies worldwide (the US,
| Canada, Australia next).
|
| NUM can be adopted / edited with a simple online form. Anyone can
| create a record for their domain, either independently in their
| own DNS or using our service at
| https://app.numserver.com/tools/editor/add
|
| To ensure that NUM is as efficient as possible, we store all data
| in DNS using MODL [2] - a compact, DNS-friendly data
| serialisation format; we make compact DNS objects developer
| friendly using Unpacker [3] - we have developed both in-house. I
| have written a very basic Notion document to explain how all of
| these technologies fit together [4].
|
| To simplify all of this for front-end developers (who by and
| large don't care about DNS), we've packaged it up in a Typscript
| library called company-api which you can query for a domain and
| get a beautiful object back containing company contact data [5].
|
| We are very keen for feedback, good or bad.
|
| 0. https://www.num.uk
|
| 1. https://www.numprotocol.com
|
| 2. https://www.modl.uk
|
| 3. https://www.unpacker.uk
|
| 4. https://www.notion.so/num/NUM-MODL-and-
| Unpacker-67d7cd59548d...
|
| 5. https://www.npmjs.com/package/company-api
|
| Edit: to make it clearer what problem we're trying to solve.
| krakov wrote:
| Incidentally, it looks like Semantic Web is not dead but
| actually accelerating growth -thanks to current SEO practices
| abd Google showing semantic data within search results [1]
|
| As for complicated , yeah, but business web platforms like Wix
| support those as part of SEO capabilities [2]. So you would you
| need the smb web dev platforms to support NUM to make it a
| reality?
|
| 1. http://webdatacommons.org/structureddata/#toc3
|
| 2. https://support.wix.com/en/article/adding-structured-data-
| to...
| elliottinvent wrote:
| > it looks like Semantic Web is not dead but actually
| accelerating growth
|
| IMO the general-purpose semantic web (not that used in
| academia) is only useful for huge companies like Google, with
| the engineering capacity to understand all the different
| types of data that might be found.
|
| At times, it seems it's almost complex by design to keep
| smaller developers from being able to consume the data and
| build anything useful on top of it.
|
| > So you would you need the smb web dev platforms to support
| NUM to make it a reality?
|
| That would be cool but it's not required for NUM to be
| broadly adopted. We provide a tool, NUM Server [0], where you
| can publish and manage data using simple online forms after
| proving authority for your domain name.
|
| The best angle for us on this point, is actually through
| domain name registrars that offer add-ons to domain name
| registration and through agencies (web design, social, etc).
|
| 0. https://app.numserver.com/tools/editor/add
| krakov wrote:
| > the general-purpose semantic web is only useful for huge
| companies like Google, with the engineering capacity to
| understand all the different types of data that might be
| found.
|
| But easier than parsing the general web
|
| This a very interesting assumption to tackle - how non-
| google players can leverage all those new semantic jsonlds?
| I'm pretty sure the barrier is lower to build a product
| aggregator or a product comparison site based on scraping
| the web. What else?
| bawolff wrote:
| But umm, why? Seems like you're in essence just using an https
| api to fetch dns data. So why not just use an https api and cut
| out the extra step?
| elliottinvent wrote:
| The https api is Cloudlfare DNS-over-HTTPS (it could be any
| other provider) - is just the best method to use in browsers,
| backend services can use standard DNS.
|
| Storing this stuff in DNS is what enables: - individual
| domain owners to adopt the standard - us to offer data free,
| unrestricted and unlimited forever
|
| Since DNS is massively scalable and cached it's well suited
| to this IMO. Much more so than a standard API.
| bawolff wrote:
| I mean, it might make sense on the backend. After all,
| decentralized, highly scalable, key-value store is what DNS
| fundamentally is, and it has a long history of working well
| for that use case.
|
| I just kind of assumed, the thing that's supposed to be
| cool here is using it in the web browser as a data storage
| backend for your entirely client-side app (since using dns
| simply as a key value store isn't particularly unique). And
| i don't think that usecase makes sense, since its all
| routed through a centralized https api translation layer
| anyways.
|
| > us to offer data free, unrestricted and unlimited forever
|
| Say what now? None of this is free. Half the bill is being
| footed by cloudflare, the other half by the domain owner.
| Using an api provided by someone else is not the same as
| offering something for free. You're not the party doing the
| offering.
| elliottinvent wrote:
| > None of this is free.
|
| The data that powers this website is offered free to
| developers.
|
| > You're not the party doing the offering.
|
| We're offering it through our DNS zone num.net.
|
| > Half the bill is being footed by cloudflare
|
| It's correct that Cloudflare are doing a fair bit of work
| here, but we're a mosquito on the arse of a bull.
|
| If NUM gets broad adoption DNS resolvers might refuse to
| answer (or more likely refuse to cache) our answers but
| there's plenty of resolvers out there and if people are
| using NUM to fetch useful data there'll be a competitive
| edge to being the fastest DNS resolver to answer those
| queries.
|
| edit: line spacing
| tptacek wrote:
| I guess I don't understand how DNS records work around the
| problem of metered APIs, since DNS providers are not going to
| let you store e.g. every item of your inventory in separate TXT
| records. Also: DNS is much, much slower than an HTTP API, which
| can fetch arbitrary, specific collections of records for a
| single query.
|
| I'm not trying to shoot this down so much as figure out what
| I'm missing here.
| elliottinvent wrote:
| Thanks for your comment.
|
| > DNS providers are not going to let you store e.g. every
| item of your inventory in separate TXT records
|
| There are some industry-standard limits in Cloud DNS (10k
| zones per customer, 10k resource records [e.g. TXT] per
| zone). These limits would make it difficult to store a large
| inventory in DNS.
|
| NUM could be used for inventory but as you say, an API would
| be more effective. NUM is designed for standardised use cases
| like contact data, where all companies store their data in
| the same way - making it easy for developers to consume.
|
| We're trying to standardise this in DNS, instead of an API,
| for two reasons:
|
| 1. Any domain registrant can adopt it, relatively simply
| (although some providers have atrocious TXT record
| management) - they can delegate their NUM zone to us for easy
| management.
|
| 2. By using DNS, we can pre-populate huge amounts of this
| data (we do this from num.net) which makes the protocol
| useful from launch and helps overcome the chicken-and-egg
| problem. DNS caching helps us answer these queries, we can
| answer a billion DNS queries for $200 using standard Cloud
| DNS pricing. Once caching is factored in it's incredibly
| efficient.
|
| It's possible DNS resolvers might refuse to cache (or even
| answer) NUM queries but we hope that doesn't happen.
|
| > I'm not trying to shoot this down so much as figure out
| what I'm missing here.
|
| I appreciate that, hopefully my answers above have helped
| LinuxBender wrote:
| Have any of your test users been behind corporate firewalls
| yet? I ask because there are some corporate firewalls
| (Fortigate, PAN to name a couple) that will see excessive
| DNS requests as a denial of service attack. The response
| will depend on the security admins that set up the
| firewalls. This could lead to false positive DDoS blocks,
| alerts to security operations centers, discussions with
| employees about their DNS traffic and possibly some folks
| losing internet at work temporarily or active directory
| server outages. What are the max queries per second this
| implementation can trigger per client? If you expect any of
| your user-base to be behind corporate firewalls, I would
| suggest reaching out to Palo Alto Networks and show them
| your implementation and ask them to test it in their lab.
| elliottinvent wrote:
| Thanks for your comment.
|
| > Have any of your test users been behind corporate
| firewalls yet? I ask because there are some corporate
| firewalls (Fortigate, PAN to name a couple) that will see
| excessive DNS requests as a denial of service attack.
|
| We've done some limited testing with this on a protocol
| level but there's lots more to do. This example site uses
| DoH which some firewalls just straight up block, in that
| case this site and associated client-side libraries
| wouldn't function at all.
|
| If a connection to Cloudflare DoH is made successfully,
| then it's unlikely any of this traffic will be troubled
| by the firewall since it's just over HTTPS.
|
| > What are the max queries per second this implementation
| can trigger per client?
|
| This example implementation is only limited by the
| browser and Cloudflare. In the Barclays example link
| provided it's sending 56 DNS queries in around a second.
|
| This is a pretty heavy implementation, a NUM lookup for a
| URL like num://numexample.com:1 only requires upto 2 DNS
| queries. The reason this implmentation is so heavy is
| that we're combining a whole bunch of records.
| nightfly wrote:
| > since DNS providers are not going to let you store e.g.
| every item of your inventory in separate TXT records
|
| They aren't?
| bawolff wrote:
| I can't imagine dns providers would be cool with you
| storing a couple gb of records. You could of course host
| your own dns server... but you could also just host your
| own website.
| elliottinvent wrote:
| There are pretty standard industry limits in Cloud DNS
| (AWS, Microsoft, Google):
|
| 10,000 DNS zones per customer
|
| 10,000 resource records per zone
|
| Both can be increased by request
|
| At scale, query costs work out at $200 USD to answer 1
| billion uncached queries, depending on TTLs that might
| actually deliver tens or hundreds of billions of answers
| to users.
|
| It's possible DNS providers and DNS resolvers (like
| Cloudflare, Google etc) might put limits in place. We're
| hoping not.
|
| Since TXT records are best delivered when below 5kb, it's
| pretty tough to store a couple of gb of data in your
| domain using NUM but certainly possible. If NUM is a big
| success, DNS providers might decide to charge for DNS
| data transfer rather than individual queries.
|
| Edit: line spacing
| jasonjayr wrote:
| FYI, it's trying to load a resource and hanging (in ff):
|
| > Cross-Origin Request Blocked: The Same Origin Policy
| disallows reading the remote resource at https://sentry.numops.
| com/api/28/envelope/?sentry_key=${key_.... (Reason: CORS header
| 'Access-Control-Allow-Origin' missing).4
| elliottinvent wrote:
| Thanks for the heads up here. This is for error reporting,
| which is kind of ironic - we'll take a look.
| bawolff wrote:
| Sentry is an error collection service, so that's probably
| nothing.
| chrisweekly wrote:
| Brilliant. Stoked to give this a try. Clever path to radically
| low latency for some use cases.
| elliottinvent wrote:
| Thanks very much, if you need any help or have any questions
| just let me know - reply here or email
| elliott[dot]brown[at]num.uk
| o8r3oFTZPE wrote:
| "All of the contact data on this website is fetched from DNS
| TXT files."
|
| I found contact data for a number of companies in the
| Javascript files.
| {"object_display_name":"Organisation","name":"Co-op
| Insurance","slogan":"Car, Home, Travel, Pet and Life Insurance
| from Co-op","contacts":[
| {"object_display_name":"Organisation","name":"Columbus
| Direct","slogan":"Award Winning Travel Insurance for 30
| Years","contacts":[
| {"object_display_name":"Organisation","name":"First
| Direct","slogan":"Online and telephone banking 24 7
| 365","contacts":[ {"object_display_name":"Organisation"
| ,"name":"Churchill","slogan":null,"contacts":[
| {"object_display_name":"Organisation","name":"MORE
| THAN","slogan":"Car, home, pet, life, travel and landlord
| insurance quotes","contacts":[ {"object_display_name":"
| Organisation","name":"Saga","slogan":"Over 50s Insurance,
| Holidays, Money and Magazine","contacts":[ {"object_dis
| play_name":"Organisation","name":"LV=","slogan":"Liverpool
| Victoria","contacts":[
| {"object_display_name":"Organisation","name":"Sheila\'s
| Wheels","slogan":"Car & Home Insurance with Style","contacts":[
| {"object_display_name":"Organisation","name":"Swiftcover","slog
| an":"Super Fast Car and Home Insurance","contacts":[
| {"object_display_name":"Organisation","name":"Barclays
| Bank","slogan":"A big world needs a big bank","contacts":[
| {"object_display_name":"Organisation","name":"Hastings
| Direct","slogan":"Car, Van, Bike and Home
| Insurance","contacts":[
| {"object_display_name":"Organisation","name":"Co-operative
| Bank","slogan":"For People with Purpose","contacts":[
| {"object_display_name":"Organisation","name":"Halifax
| UK","slogan":"Halifax makes it happen","contacts":[
| {"object_display_name":"Organisation","name":"Lloyds
| Bank","slogan":"By Your Side","contacts":[
| {"object_display_name":"Organisation","name":"Royal Bank of
| Scotland","slogan":"Enjoy better banking with RBS where people
| matter","contacts":[ {"object_display_name":"Organisati
| on","name":"Admiral","slogan":"Car, MultiCar and MultiCover
| Insurance Quotes","contacts":[ {"object_display_name":"
| Organisation","name":"Aviva","slogan":"Insurance, Savings and
| Investments","contacts":[ {"object_display_name":"Organ
| isation","name":"AXA","slogan":null,"contacts":[
| elliottinvent wrote:
| Thanks for highlighting this - it comes from company-api,
| which is built on top of the DNS:
| https://www.npmjs.com/package/company-api
| elliottinvent wrote:
| Actually, I think this is a JSON object to populate the
| companies on the homepage. When you click one of these
| companies a NUM lookup is run on the domain name, then all
| contact data comes out of DNS.
| faeyanpiraat wrote:
| This won't work.
|
| If companies cannot be bothered to make their websites user
| friendly in the first place, why would they do it in this new
| and obscure way?
|
| It's like the episode on Dragon's Den where someone tried
| pitching a service that would call companies on your behalf,
| and would only call you to connect both ends when the music
| stops and a call center agent actually picks up the phone.
|
| It's the job of the company to fix their long wait times, not
| of some 3rd party service, which may or may not work with a
| specific customer service number.
| elliottinvent wrote:
| Thanks for your comment.
|
| Companies don't need to adopt the standard for their data to
| be in DNS. We're populating DNS with public website data
| already. All *.uk domains will be populated in the next 10
| days. Then we'll roll it out further.
|
| In the example provided it's mainly for big companies with
| lots of phone numbers. But NUM is for any machine readable
| data for any company - small traders, pubs, restaurants and
| more.
| rognjen wrote:
| I think it would be wise to slightly change the design so as not
| to look so similar to Companies House.
| bawolff wrote:
| I don't get it.
|
| You have to host the static page somewhere. Why not host the data
| in the same place (as a json blob or whatever)? It seems like the
| implication is all the data in dns is static, so what's the
| benefit of adding dns to the mix - you already have to use
| something else to host some of your static resources, why not
| just have a single static host instead of two separate static
| hosting systems?
|
| (Yes in theory you could write a dns server to dynamically answer
| queries, but that doesn't seem like its what is being proposed
| here)
|
| Dont get me wrong, its cool you can access dns from a browser,
| but i dont think this is a compelling usecase (something like web
| torrent in the browser with magnet links in dns for a
| decentralized web, seems like it would make a good demo for this
| technique)
| elliottinvent wrote:
| What we're trying to do here is standardise how machine-
| readable data is stored and retrieved, primarily for companies.
| Any of the companies on this page can adopt the standard and
| override the data on the website. So the point is, it's not
| static.
|
| Sure, at the moment all this data is coming from one DNS zone
| (num.net) but that's only a failover query location, first we
| query e.g. barclays.co.uk for this data, so Barclays can
| override (or remove) this data if they want to.
| bawolff wrote:
| So the interesting part here is not supposed to be querying
| dns data from the client side of a website, but a new
| standard for machine readable data in TXT records?
|
| I mean between microdata, rdf, etc there's been a lot of
| standards for machine readable data storage on the internet.
| Most fail not for technical reasons, but because there is an
| incentive mismatch, where the people who have to maintain
| such data get no benefit out of maintaining it so they don't
| bother. In this scheme you can't even google it, so why would
| a company bother to use this?
| elliottinvent wrote:
| > So the interesting part here is not supposed to be
| querying dns data from the client side of a website, but a
| new standard for machine readable data in TXT records?
|
| Correct
|
| > I mean between microdata, rdf, etc there's been a lot of
| standards for machine readable data storage on the
| internet. Most fail not for technical reasons, but because
| there is an incentive mismatch, where the people who have
| to maintain such data get no benefit out of maintaining it
| so they don't bother.
|
| I'd argue the inconsistent way these standards have been
| adopted (or not in many cases) is the reason they are not
| an effective, reliable way for developers to find machine-
| readable data about _any_ business. Since they are not a
| reliable way to find data, developers can't build anything
| with that data and there's little incentive for companies
| to keep it up to date.
|
| Currently, the only reason to adopt these standards is for
| SEO or to make your website look pretty when you share your
| website on Facebook / whatever.
|
| > In this scheme you can't even google it, so why would a
| company bother to use this?
|
| Google hold a lot of the data we're proposing to publish
| using NUM (contact data for example). The difference
| between our approaches is that we're going to make it
| freely available to developers, Google don't (not really).
| Google could consume NUM data like anyone else. As far as
| the protocol is concerned Google is the same as any other
| developer so it's a level playing field.
| detaro wrote:
| And now you are adding _another_ standard that will be
| inconsistently adopted, _especially_ since it 's even
| less accessible to web developers than the alternatives?
| elliottinvent wrote:
| We're pre-populating NUM with useful data, so it doesn't
| require domain registrants to adopt it.
|
| NUM modules are much simpler than other comparable web
| standards so any data stored will be consistent and easy
| for developers that consume the data.
|
| Edit: typo
| ksec wrote:
| >CompanyDirectory.UK is a NUM Technology [1]
|
| >What is NUM?
|
| >NUM is a DNS-based alternative to the World Wide Web for storing
| and retrieving structured data. The web is amazing but websites
| are built for browsing and are an inefficient way to find precise
| pieces of data like telephone numbers, bank details and more.
|
| [1] https://www.num.uk
| nopcode wrote:
| Isn't DNS de-facto centralised too?
| elliottinvent wrote:
| I think you could argue DNS over HTTPS is centralised (for now)
| with Cloudflare, Quad9, Google making up the main players. But
| I think the DNS is the original decentralised system due to the
| amount of players in the domain registrar / DNS service
| provider / ISP [DNS resolver] space.
| nine_k wrote:
| Yes in the sense that DNS is a tree with a single root.
|
| No in the sense that you can put your data into leaves of your
| choice and under your control. You can easily run your own DNS
| server.
|
| Halfway-yes in the sense that this thing uses HTTPS access to
| DNS which has few providers now, and not the classic DNS
| because UDP from a browser is hard, and even arbitrary TCP from
| a browser is hard.
|
| I think that the use of DNS is a bit of a gimmick on this demo
| site; it could be more practically used in more interesting
| ways. A demo site should be and is very straightforward. But
| the idea of storing semantic-web-style information in the DNS
| looks interesting to me, and the suggested simplified format
| may be worth a look.
|
| The service NUM is offering is also interesting, and one of the
| key parts of it is that _it 's not storing your data_, so you
| are not beholden to it. It helps make them accessible, but you
| store the data yourself.
| tptacek wrote:
| Ok, but, you can easily run your own Postgres server too,
| right?
| nine_k wrote:
| Its data won't be cached by any number of caching DNS
| resolvers. If your DNS server has 90% uptime, but your
| domain if often resolved, clients may not notice anything;
| if your database has 90% uptime, it's pretty noticeable.
|
| It of course only applies if the TTL of your records is
| long enough, but usually you don't need short TTL for TXT
| records of such kind.
| bawolff wrote:
| But aren't you specificly using cloudflare's DoH endpoint
| to do the query? Wouldn't that mean that there aren't any
| caches other than cloudflare's? In which case, why not
| just put your website behind cloudflare?
| bawolff wrote:
| I feel like saying something is "decentralized" is like saying
| something is "secure". Its a meaningless statement. Nothing is
| 100% secure and nothing is 100% decentralized. The relavent
| questions are from whom? Against what?
| tedk-42 wrote:
| Please no.
|
| CI systems like GitHub actions and CircleCI use to have unlimited
| CI minutes until people started abusing them for things they
| weren't intended for (i.e. crypto mining).
|
| I would hate if I had to pay for DNS services (or be forced into
| using an ISP one which does things like blacklist certain
| domains) just because people on the internet wanted to be 'cool'
| and show off how smort they are.
|
| This is why we can't have nice things.
| bawolff wrote:
| I don't see why this application would be abusive. Chrome does
| much worse things trying to detect if you are behind a captive
| portal.
|
| Edit: oh i see, they arent using generic dns but cloudflare's
| specific DoH gateway. Guess that could maybe be problematic if
| everyone did that, although if any company would tolerate that
| it would be cloudflare.
| adflux wrote:
| Your domain names are free?
| tedk-42 wrote:
| Someone needs to host the DNS servers which deliver DNS
| results.
|
| Serverless deployment models under the cover still uses
| servers.
| adflux wrote:
| What I am saying is that you are already paying for DNS
| services
| tedk-42 wrote:
| What I said was I didn't want to have to pay to do DNS
| lookups.
|
| E.g. if a client side DNS call was rate limited because
| DNS services like CloudFlare (1.1.1.1) or Google
| (8.8.8.8) got tired of caching data that was never
| suppose to be there.
|
| There are people who have joked you can use route53 as a
| database (Corey Quinn originally?). But taking a joke too
| far is irresponsible.
|
| Imagine having to pay a subscription to resolve DNS
| queries...
|
| We have a good thing with it and this idea is like a
| knife in it (like my free CI minutes and cryptominers
| example)
| bawolff wrote:
| Then presumably people would stop using these services
| and use others/make their own. Its not like google has a
| monopoly on running a public dns resolver.
| zzyzxd wrote:
| I mean, technically speaking, a DNS _is_ a database. I am
| not saying I like this idea, but are there any
| regulations on what you can or cannot put in TXT records?
|
| > Imagine having to pay a subscription to resolve DNS
| queries...
|
| Great, I won't pay for resolving googleadservices.com
| then :)
|
| Joking aside, I personally don't mind paying for DNS (I
| am already doing that using NextDNS and it has been the
| best DNS service I have ever used). And as been pointed
| out by others, DNS providers can always put a limit or
| hit you with a ban hammer if they want.
| elliottinvent wrote:
| Thanks for your feedback. Obviously I think it's cool but I
| doubt that'll be a good enough reason for most people.
|
| NUM is a protocol for storage and retrieval of data, that's
| all. Hopefully the demonstration shows it's pretty good at
| that. If people choose to store huge amounts of data using it,
| I think that would be cool too.
|
| I don't think there's any risk of it being used for crypto
| mining. NUM doesn't open up any new attacks vectors that aren't
| already present in DNS and it's an incredibly resilient and
| robust system as we know.
| tedk-42 wrote:
| > If people choose to store huge amounts of data using it, I
| think that would be cool too.
|
| This is what should be avoided. It adds unnecessary load to
| DNS servers around the globe.
| elliottinvent wrote:
| If it's useful data that's being stored and retrieved, I'd
| argue it's better for the internet as a whole. It shifts
| load from webservers (where people are downloading 5mb of
| page data just to find a phone number) to DNS, which is far
| more efficient.
|
| If NUM is success it will certainly increase load but I
| don't think it would be an unnecessary load, since it would
| be serving a worthwhile purpose.
| tedk-42 wrote:
| It's not better for the internet as a whole. Host your
| own data behind a CDN and S3.
|
| There are also lots of free services (github pages
| netlify) which allow you to store static content.
|
| Use these over a DNS server to store your data.
| elliottinvent wrote:
| I appreciate your feedback and am interested by your
| point of view on this.
|
| > It's not better for the internet as a whole.
|
| We've built this because we believe if all the people
| manually trawling websites looking for particular pieces
| of data were instead fetching this data automatically
| through ultra-efficient DNS queries, it would be better
| for the internet as a whole.
|
| > Host your own data behind a CDN and S3.
|
| We could certainly do this, we've made this a DNS-based
| protocol so domain owners can adopt it independently in
| their own DNS.
| belthesar wrote:
| Have you considered that part of the reason why DNS is
| ultra efficient is that it's a relatively constrained
| dataset of small but meaningful data objects? I worry for
| all of the places running caching DNS proxies suddenly
| tanking their query resolver performance because their
| caches are flushing far more rapidly, and root hint
| servers seeing a massive influx of data now being stored
| in the platform. The technical marvel is cool, but I
| sincerely worry about this becoming a widely used
| product.
| elliottinvent wrote:
| > Have you considered that part of the reason why DNS is
| ultra efficient is that it's a relatively constrained
| dataset of small but meaningful data objects?
|
| I'm not sure it is. The size of DNSSEC responses are not
| insignificant. The apex for many domains is polluted with
| utter garbage: dig target.com TXT
|
| The DNS is ultra efficient because it's cached and
| distributed, yes packet sizes are relatively small but a
| standard NUM record would be smaller than the original
| udp packet limit of 512.
| potamic wrote:
| Better for the internet as a whole is a pretty big leap
| of reasoning. There is plenty of useful data in the web
| that needs to be stored and retrieved. In fact, this is
| precisely what the web was built for. It is an
| information network. Why hack the DNS system for this?
| DNS is meant for domain name discovery. It has a specific
| purpose. In large architectures, it is almost never a
| good idea to break such separation of concerns.
| elliottinvent wrote:
| Thanks for your comment.
|
| The DNS is used for a lot more than converting domain to
| IPs and mail servers, run:
|
| dig target.com TXT
|
| And be amazed at the pollution at the apex.
|
| SPF, DMARC and others show that DNS is already being used
| for much more.
|
| The DNS is a distributed database. The primary key is the
| domain and we believe that converting human friendly
| domains to machine readable data shouldn't stop at IPs,
| mail servers and anti-spam measures (SPF and DMARC). It
| should extend to phone numbers, gps coordinates, bank
| details and more.
| zzyzxd wrote:
| I like the limit on GitHub actions. It won't be sustainable in
| long term otherwise. You just can't count on people's self
| discipline for this kind of stuff, and there will always be
| malicious usage as long as the system allows.
|
| If putting large random TXT records can harm the global DNS
| infrastructure, DNS providers should put a stricter limit on
| it. And OP's project will help us get there.
| yakshaving_jgt wrote:
| This is interesting. I run https://newbusinessmonitor.co.uk/ and
| I'm often asked if I can provide contact data for UK companies,
| but this is difficult because no contact data is published when a
| company registers. I wonder how you'll be getting this data for
| smaller companies.
| elliottinvent wrote:
| The long game is for companies to store NUM data when they
| register a domain, potentially through their DNS provider or
| web designer. Maybe even storing machine readable data instead
| of a website, since many small business websites are cookie-
| cutter anyway - contact info, directions, menu etc.
|
| Public data is available for most domain registries, e.g
| Nominet [0] and others [1] so we crawl new domains to discover
| contact data.
|
| It could be good to chat (email elliott[dot]brown@num.uk),
| we're in the process of mashing up Companies House data with
| our contact data in an effort to provide company numbers in NUM
| records too.
|
| 0. https://registrars.nominet.uk/uk-namespace/the-uk-zone-
| files... 1. https://czds.icann.org/
___________________________________________________________________
(page generated 2021-06-25 23:02 UTC)