2024-12-17 - No DNS, No Problem

The early Internet used /etc/hosts files to resolve names. These were distributed on FTP sites. You can see a 1992 example below [1].

The /etc/hosts file quickly grew too large to be manageable. See historical document below [2].

Internet users switched to DNS.

Recently, for various reasons, i became interested in avoiding DNS, so i researched using an /etc/hosts file. I found one on gopher.quest, but unfortunately it was incomplete, and is no longer online.

I noticed that the Quarry gopher search engine contains a little over 300 live gopher hosts. I wrote the author about it, and he generously created an /etc/hosts file from the Quarry database, linked below [3].

This is updated daily. On a linux system, you could replace /etc/hosts with this file and assuming a sane C library, it should "just work." It will resolve those names without DNS.

FreeDOS is slightly more complicated. Suppose the file is in C:/etc/hosts.txt and will be used by mTCP and WATTCP.

For mTCP, edit mtcp.cfg, add:

    HOSTSFILE C:/etc/hosts.txt

For WATTCP, edit wattcp.cfg, add:

    hosts = C:/etc/hosts.txt

Now FreeDOS programs should resolve those names without DNS.

This covers gopher hosts. I would also like to include FTP and web hosts referenced by my gopher maps. Below [4] [5] is a script i wrote to mirror gopher content.

Using this script, i mirrored my gopher content.

    $ mkdir gopherhole
    $ cd gopherhole
    $ tclsh gopher-get.tcl --binary-skip --images-skip      \
        --ignore gopher://tilde.pink/1/~bencollver/dir      \
        --ignore gopher://tilde.pink/1/~bencollver/dict     \
        --ignore gopher://tilde.pink/1/~bencollver/gamefaqs \
        --ignore gopher://tilde.pink/1/~bencollver/recipes  \
        gopher://tilde.pink/1/~bencollver

This took a while, so i let it run and came back later. This created two directories.

• tilde.pink/0 (Plain text documents)
• tilde.pink/1 (Gopher maps)

I scraped the host names from these directories as follows [6].

    $ find tilde.pink/1 -type f -exec \
        grep -e URL:ftp: -e URL:http: -e URL:https: {} \; |\
        sed -e 's,.*URL:[^:]*://,,' -e 's,/.*,,' >names-gopher.txt
    $ find tilde.pink/0 -type f -exec \
        awk -f setext-scrape-links.awk {} \; >names-setext.txt
    $ cat names-gopher.txt names-setext.txt | sort |\
        uniq >names-extra.txt
    $ wc -l names-extra.txt
    392 names-extra.txt

This shows that i reference approximately 400 non-gopher names. Now i would like to resolve these to IP addresses. I wrote a short AWK script to do this [7]. I ignored unkown names because of old documents that reference defunct names.

    $ awk -f resolve.awk names-extra.txt >hosts-extra.txt
    $ wc -l hosts-extra.txt
    612 hosts-extra.txt

Because DNS can resolve a name to multiple IP addresses, this list grew from 392 to 612 lines. I appended this list to the Quarry hosts file like so.

    $ printf "/files/hosts.txt\r\n" |\
        ncat gopher.icu 70 >hosts-icu.txt
    $ cat hosts-icu.txt hosts-extra.txt >hosts-all.txt
    $ wc -l hosts-all.txt
    1058

For FreeDOS, filter out ":" to exclude IPv6 addresses. Also, mTCP has an 80 character line length limit including EOL characters.

    $ grep -v : hosts-all.txt |\
        awk 'length($0) < 78 {print}' >hosts-dos.txt

Now i can copy hosts-all.txt to /etc/hosts or copy hosts-dos.txt to C:/etc/hosts.txt and i am good to go.

p.s.

The FreeDOS bundled mTCP has a bug. If you specify HOSTSFILE but no NAMESERVER, then name resolution will always fail. You can work around this by making sure mtcp.cfg has a NAMESERVER line. If you don't use DNS at all, it can be:

    NAMESERVER 127.0.0.1

Footnotes

Tags