[HN Gopher] Show HN: IP Guide - Info on IP addresses, Networks, ...
___________________________________________________________________
Show HN: IP Guide - Info on IP addresses, Networks, and ASNs
Hey all, wanted to share my recent weekend hack project. I wanted
to improve upon the existing space for looking up information about
IP addresses and ASNs. The backend is written in Rust and pulls
BGP data every 8 hours from the RIPE RIS project to build up a
routing table and also adds geolocation information. The frontend
is using React and Tailwind. Would love any feedback or
suggestions on what to improve.
Author : bswinnerton
Score : 328 points
Date : 2023-11-24 22:37 UTC (1 days ago)
(HTM) web link (ip.guide)
(TXT) w3m dump (ip.guide)
| joshghent wrote:
| Great site and product! Congratulations on the launch.
|
| I actually have a use case for this in my product Loginllama. I
| need to grab information about the IP addresses. I'm currently
| using a different API but don't really like the product.
|
| Is it rate limited or have any key authentication? My email is me
| at joshghent.com if you want to chat about this more.
| bswinnerton wrote:
| Go for it! There's no formal rate limiting or auth, but it does
| start to load shed at 2k concurrent requests. It's stateless,
| so it's very easy to scale horizontally after that.
|
| LoginLlama looks super helpful, what else would you like to see
| from IP Guide to help? Would a boolean attribute on whether or
| not the IP is a known tor exit node be helpful?
| wantguns wrote:
| Very cool project, any plans to release its source ?
|
| also s/Tailscale/Tailwind/
| bswinnerton wrote:
| I do that every. single. time.
| WhackyIdeas wrote:
| Love it - bookmarked!
| DougN7 wrote:
| Having a simple paragraph about RIPE, BGP and ARIN would be
| helpful (at least for me).
|
| Is there any way at all to do this for IPv6?
|
| Additional info that would be helpful would be if this is hosted
| by one of the big tech companies (i.e. is it on AWS, Azure, etc),
| though maybe that will show up in the network info?
|
| If the IP is associated with hostnames that would be nice to
| know. Not sure how much of a one-stop shop you want to be :)
| LelouBil wrote:
| > Having a simple paragraph about RIPE, BGP and ARIN would be
| helpful (at least for me).
|
| Same, I've always wondered how do websites like this get their
| information.
| bswinnerton wrote:
| Great feedback, I'll add a little more to the site to
| describe how it all works. Until then...
|
| At a 30,000ft view, the site works by building an in-memory
| routing table at boot time that has an entry for every route
| on the internet and which ASN (autonomous system number)
| announces it. From there, it stitches together data on the
| organization that the ASN belongs to, and geolocation data,
| and then exposes it in the API/UI. Under the hood it's using
| a treebitmap[1] data structure, which means that it can do
| _very_ fast lookups when given a particular ip address or ip
| address range.
|
| Zooming in a little more, the main data source is MRT dumps
| from the RIPE RIS project[2]. There are a number of routers
| that RIPE (one of the 5 regional internet registries that
| hand out ASNs and IPs) that rely on network operators to
| share their view of every route on the internet. These
| routers are probably some of the most well-connected routers
| in the world. For the purposes of this service, MRT dumps are
| effectively a point-in-time snapshot of the routing table of
| that router. IP Guide parses that file, rebuilds it in
| memory, and attaches other relevant data to each IP range
| (also known as a "prefix").
|
| This allows the service to stay fast and stateless, which was
| one of the main things I was trying to optimize for when
| building it.
|
| [1]: https://blog.apnic.net/2021/06/04/storing-and-
| retrieving-ip-... [2]:
| https://ris.ripe.net/docs/10_routecollectors.html
| LelouBil wrote:
| Thanks !
| bswinnerton wrote:
| > Having a simple paragraph about RIPE, BGP and ARIN would be
| helpful (at least for me).
|
| That's a great idea. I'll add a tooltip over some of the less
| obvious fields describing what they are.
|
| > Is there any way at all to do this for IPv6?
|
| Indeed! It's supported natively, so if you have an IPv6 address
| you should see it automatically. Here's an example
| <https://ip.guide/2600::>
|
| > If the IP is associated with hostnames that would be nice to
| know.
|
| I dig that idea, I'll have to think of the best way to pull
| that data while keeping the app stateless and fast.
| baby_souffle wrote:
| > I dig that idea, I'll have to think of the best way to pull
| that data while keeping the app stateless and fast.
|
| Reverse DNS lookups are usually pretty quick:
|
| ```shell > time dig +short -x 2600:dead::beef
| customer.my-isp.net. dig +short -x 2600:dead::beef
| 0.00s user 0.01s system 1% cpu 0.882 total
|
| ```
| teh_klev wrote:
| This might be useful to review:
|
| https://news.ycombinator.com/formatdoc
|
| "Text after a blank line that is indented by two or more
| spaces is reproduced verbatim. (This is intended for
| code.)"
| baby_souffle wrote:
| Thanks.
|
| I spend all day writing markdown and fenced blocks with
| type-hint is muscle memory at this point
| spacehunt wrote:
| Reverse DNS does not give you all the hostnames that can
| point to a particular IP address though.
| necovek wrote:
| Other than brute forcing, I don't think anything does --
| that's the beauty and the curse of a proper federated
| system :)
| jedberg wrote:
| In fact, it might not even give you _any_ names that
| point to an IP address, since it might be out of sync
| with the forward system, especially if the records are
| built by hand.
| HellsMaddy wrote:
| Nice! I love HTTP APIs like this with simple, user-friendly URLs
| like `https://${domain}/${query}`. Anyone know of a list of other
| services like this? wttr.in is one example I know of.
| achillean wrote:
| We do it for https://internetdb.shodan.io
| miki123211 wrote:
| Not exactly HTTP, but dns.toys
| nadermx wrote:
| I made https://api.miip.co but no where near as complex
| tobias2014 wrote:
| https://ifconfig.me
| AndrewCopeland wrote:
| https://ipdetective.io
| seligman99 wrote:
| My similar take on the idea: https://cloud-ips.s3-us-
| west-2.amazonaws.com/index.html
|
| I wanted to see if I could do the lookup work client-side, and
| also include some more metadata about cloud provider's IPs
| (region, service, etc), not that it's really better, just a toy
| idea I had.
| Kirce wrote:
| Awesome, bookmarked! Would it be possible to list all announced
| IP ranges for an AS?
| bswinnerton wrote:
| Yes! Quite easily, but it requires thinking through the right
| API design. I'll see if I can work this in this weekend.
| sgjohnson wrote:
| You can use https://bgp.tools for that (e.g.
| https://bgp.tools/as/200676#prefixes)
| pests wrote:
| Curious, is that your AS?
| sgjohnson wrote:
| Yup
| reincoder wrote:
| Disclaimer I work for IPinfo.
|
| If you can download the free IPinfo IP to ASN database CSV
| file. You can run the following command to get the IP ranges of
| an ASN.
|
| ```
|
| grep AS19551 asn.csv | ipinfo range2cidr | cut -f1 -d ','
|
| ```
|
| You will need to install the IPinfo CLI to convert the IP range
| to their CIDR equivalent. I believe the range2cidr command is
| also available as a standalone binary as well.
|
| After that, you might want to aggregate the IP ranges to bigger
| ranges. I have used this in the past for this:
|
| https://github.com/job/aggregate6
| Kirce wrote:
| I didn't know ipinfo had a cli, good to know, i'll have a
| Look. Thank you!
|
| I'm currently using ipset, that does aggregation if I
| remember correctly, but aggregate6 looks good!
| toast0 wrote:
| In addition to other tools, there's also https://bgp.he.net
| which has a page for announced prefixes. Sometimes it's useful
| to use multiple tools, as not everyone will see all
| announcements or process them the same.
| iJohnDoe wrote:
| Also like https://wasab.is
|
| Yours is very cool by pulling BGP data every 8 hours though!
| bswinnerton wrote:
| Oh cool! Is this a fork of ifconfig.co?
| iJohnDoe wrote:
| Yes, seems like it!
| xyst wrote:
| For some reason this site only sees my ipv4 addy. But ip.guide
| automatically returns IPv6.
| kortilla wrote:
| Check out http://bgp.tools for an alternative in the area.
|
| Not affiliated, it's just my current preferred.
| stevenicr wrote:
| I go to myip.ms regularly and use the additional info it offers
| such as owner ip block / ranges, the cidr, and hostname is
| sometimes helpful.
| warner25 wrote:
| From where do you get your geolocation information? I've been
| using the free MaxMind GeoLite2 databases and the GeoIP2 Python
| API for this kind of thing, but for a lot of IP addresses that
| I'd like to map, I can only get down to the country (and even
| some of those seem suspect).
|
| Is there any rate limit for queries?
| bswinnerton wrote:
| IP Guide is also using MaxMind, I'm not familiar with many
| other data sources for that kind of data. Are you seeing
| discrepancies?
|
| No rate limits or auth for requests, but as mentioned below the
| api starts to load shed at 2k concurrent requests.
| warner25 wrote:
| Thanks. Yeah, I was just hoping it might have access to some
| different data. I've seen no discrepancy. I tried one IP
| address that I've been trying to map, and I got the same
| answer.
| mmaunder wrote:
| Their city level data tends to be proprietary so maybe double
| check you're ok in your use.
| ashu1461 wrote:
| How big is the MaxMind database ?
| fruit2020 wrote:
| A few MBs
| nsp wrote:
| About 80mb for the city level one
| withinboredom wrote:
| I have yet to see a geo-ip address for my IP that shows me even
| on the correct side of the country. I wouldn't trust this
| information as far as you can throw it ... and I've had this IP
| address for nearly 6 years now.
| reincoder wrote:
| Can you check IPinfo.io, please? If we are showing any
| inaccuracy in your IP geolocation data for you, please reach
| out to us. We will fix that ASAP.
| withinboredom wrote:
| yeah, it's very wrong. Showing "vpn: true" (actually a
| regular residential connection), wrong city, region,
| postcode, etc. Just like every other service out there.
|
| I don't see any way to report it though.
| reincoder wrote:
| That is really surprising. For geolocation correction, we
| have a self reporting system: ipinfo.io/corrections
|
| I forgot to add the link in my original comment. But as
| you mentioned we are showing VPN as true, that indicates
| a very interesting situation. Can you ping our support,
| please?
|
| https://ipinfo.io/support
|
| If you can also mention that, you came from HN, that will
| be fantastic. We will open an engineering ticket first
| thing Monday and look closely into this. The situation is
| unusual, and I am sorry about the inconvenience.
| bswinnerton wrote:
| I'd also suggest filing a correction with MaxMind, which
| is where most modern geolocation data is derived:
| https://www.maxmind.com/en/geoip-location-correction.
| wongarsu wrote:
| To be fair, some IPs simply can't reasonably be mapped beyond
| the country level (even if MaxMind often tries).
|
| For example in Germany some carriers just give you a random IP
| from a pool that is used for half the country, and will give
| you a new IP every 24 hours. How would you geolocate those with
| a downloadable database.
| tuga wrote:
| Have you thought about offering downloadable versions of to the
| database through MMDB, CSV, or JSON formats? This would enable
| users to utilize the database offline, similar to the setup
| provided by IPinfo.io for their free IP database.
| bswinnerton wrote:
| Yeah, if you click on "bulk data" you'll see the two statically
| hosted CSVs for ASN and IP data. What else would you find
| useful in those datasets? It's quite easy to expand on them.
| byyll wrote:
| Countries.
| bswinnerton wrote:
| The country code is the last portion of the networks CSV:
| $ curl -sL ip.guide/bulk/networks.csv | head -n 2
| prefix,asn,organization,country
| 1.0.0.0/24,13335,"Cloudflare, Inc.",US
| tuga wrote:
| I suppose I was overly enthusiastic about the project and
| overlooked the bulk part . Thank you. Personally, I prefer to
| use local MMDB versions as they provide faster results and do
| not require network usage. Having knowledge about whether an
| IP is a proxy, VPN, data center, etc. is valuable to me, but
| I understand that it may vary depending on each project's
| requirements.
| meeby wrote:
| Without wishing to gazump on tuga's excellent project, you
| can get mmdb files for IPs and ASNs free for personal use
| from https://ipnetdb.com/ that will probably provide the
| data you want.
|
| (Disclaimer: I work for the company that develops IPNetDB).
| rebelde wrote:
| Feature request for any service like this: Let "me" know if it is
| a school, so I know that I am probably dealing with minors, a
| public environment and a firewall. Of course, you need to do the
| work (rDNS to start) to identify the schools.
|
| I love the speed of the responses!
| slig wrote:
| IPINFO.io has that info, but I too would like to have this
| information in a free service.
| reincoder wrote:
| I work for IPinfo. We have higher level IP category information
| on our website for free.
|
| We categorize ASN and companies/organizations based on 4
| categories: ISP, Education, Hosting and Business. This ASN
| level categorization are done mainly from WHOIS and other
| public internet records.
|
| We don't sub-categorize by schools, university, public research
| institutions, k-12 etc. The reason is accuracy. Even though I
| can understand the possible methods for doing this, the issue
| is that it can not be done reliably at scale.
|
| As a data provider, from our end we hope to provide the highest
| possible accuracy and vouch for the service we provide. For
| this level of classification, we will generally request users
| to say what data they need from us, and we try to help them
| come to a solution that they have to build on their own. They
| can do whatever classification they want to do based on their
| personal level of tolerance for accuracy.
| bogomipz wrote:
| I love the API and response times! I had a few questions:
|
| Might you open-source this in the future?
|
| Were you already comfortable in Rust or is there a reason you
| selected Rust for this project i.e performance and so plans to
| offer this as commercial product?
|
| How often are RIPE info and MaxMind data updated these days or is
| that the 8 hours?
|
| Thanks for sharing!
| bswinnerton wrote:
| > Will this be open-sourced?
|
| I don't think there's a strong reason to keep it closed-source.
|
| > Were you already comfortable in Rust or is there a reason you
| selected Rust for this project?
|
| This was a big learning opportunity for me to learn Rust. I had
| done two small projects in Rust before this one.
|
| > How often are RIPE info and MaxMind data updated these days?
|
| The MRT dumps where it gets its BGP data are updated every 8
| hours, and MaxMind appears to be updated twice a week.
| nektro wrote:
| love this, nice work :) so fast.
| WarOnPrivacy wrote:
| I _like_. I was just looking for an easy way to pull geoip in a
| bash script.
| ashu1461 wrote:
| Is there a rate limit to the API ?
| bswinnerton wrote:
| No rate limits or auth for requests, but as mentioned below the
| api starts to load shed at 2k concurrent requests.
| johnthescott wrote:
| feature request: date of cache entry.
| bswinnerton wrote:
| It's not available from the API, but there is a "last updated"
| date when you view it in a browser. It's at most 8 hours old.
| withinboredom wrote:
| Showing the start/ending address as part of the network would be
| helpful when looking up the network. That would avoid having to
| pull out a CIDR calculator when using this tool.
| bswinnerton wrote:
| Great idea. This is rolling out right now.
| ta1243 wrote:
| Nice,
|
| I currently do "curl ipinfo.io" which returns json, and seems to
| use a more accurate geo lookup (based on the hotel wifi I'm
| currently on). It doesn't report the network though, and the ASN
| is a string of asn and name, no org or country, so that's a
| bonus.
|
| However curl ip.guide
|
| just returns blank (there's a redirect). If you follow the
| redirect with -L the json isn't pretty-printed.
| reincoder wrote:
| I work for IPinfo.
|
| >the ASN is a string of asn and name, no org or country
|
| Yeah detailed ASN information on the API level is available on
| paid tiers.
|
| However, we have a free IP to ASN Country database that is
| updated daily and provides full accuracy. That database
| provides ASN, AS organization, and AS domain information.
|
| I think the reason we don't have more ASN details on the API
| even though we provide more information for free in the
| database is because of the target audience and usability of the
| IP metadata.
|
| Our API allows for tokenless API access for up to 1,000
| request/day. There is no compromise with data accuracy. The
| goal is to be generally useful and extremely fast. Detailed ASN
| information is still freely accessible from the website,
| though. So, we have to balance out free API, free website
| information and paid API/database.
|
| I have been reaching out to folks to adopt the free IP to
| Country ASN database, as it is free to use and provides full
| accuracy. If anyone wants to build a public API on top of the
| IPinfo free database that return more ASN information available
| there, they are more than welcome to that!
| jftuga wrote:
| > Our API allows for tokenless API access for up to 1,000
| request/day.
|
| Thank you for this! Much appreciated!
|
| I am sure there are tons of similar programs, but here is one
| I wrote:
|
| https://github.com/jftuga/ipinfo
|
| I added on a distance column.
|
| Just yesterday, I added this: brew install jftuga/tap/ipinfo
| bswinnerton wrote:
| You'll have to follow the redirects since the site doesn't
| support HTTP, but just shipped pretty-printing.
| romanixromanix wrote:
| Very nicely done. I like the simplicity of the page and
| interface.
| umen wrote:
| can you pls elaborate your tech stack and where do host ? 1 .rust
| ( what server or hand crafted ?)
| bswinnerton wrote:
| For the backend, it's written in Rust using a few key crates:
| axum for the api, bgpkit-parser for parsing the MRT data, and
| reqwest for fetching data at boot.
|
| The site is hosted on Neptune Networks, another side project of
| mine for people to learn and experiment with BGP.
| DavideNL wrote:
| Alternatieve, fyi; curl -sS
| https://am.i.mullvad.net/json | jq
| lovasoa wrote:
| This mullvad thing is just another "what is my ip" service. His
| site lets you download the entire database locally, for free,
| and without registration.
| spacecadet wrote:
| This is neat, Im currently passing some IP addresses to OpenAi
| for a crude version of this, but will try this out!
| AndrewCopeland wrote:
| Nice, I like how simple it is!
|
| I provide a similar service but with more focus on bot/data
| center/VPN ip addresses.
|
| https://ipdetective.io
| tonymet wrote:
| Here's a powershell command -- let's add it to the docs!
| > (iwr ip.guide).Content | ConvertFrom-Json | Select-Object
| location
___________________________________________________________________
(page generated 2023-11-25 23:01 UTC)