[HN Gopher] Wiretrustee: WireGuard-Based Mesh Network
       ___________________________________________________________________
        
       Wiretrustee: WireGuard-Based Mesh Network
        
       Author : j1elo
       Score  : 304 points
       Date   : 2021-06-29 06:56 UTC (16 hours ago)
        
 (HTM) web link (github.com)
 (TXT) w3m dump (github.com)
        
       | seddona wrote:
       | Currently running our corporate VPN on Wireguard, would highly
       | recommend. Wiretrustee looks like a very useful featurset on top
       | of Wireguard, nice work.
        
         | dgan wrote:
         | Bare bones wireguard? Or some solution like tailscale? Or maybe
         | FOSS
        
       | tbronchain wrote:
       | Looks great!
       | 
       | I've been using wg-meshconf[1] to assist in setting up Wireguard
       | Mesh Networks on Linux for a while, works amazing!
       | 
       | A massive use case is to setup Kubernetes clusters, where end to
       | end network encryption is quite important.
       | 
       | [1]: https://github.com/k4yt3x/wg-meshconf
        
         | ptman wrote:
         | This assumes the peers have static IPs? Or how is changing peer
         | IPs handled?
        
           | tbronchain wrote:
           | Correct, peers must have a static IP. We could imagine to
           | build a small extra tool to automate regenerate the config in
           | case of IP change but could be a bit tricky :)
        
       | ptman wrote:
       | Other interesting solutions in this space:
       | 
       | - https://tailscale.com/
       | 
       | - https://github.com/juanfont/headscale
       | 
       | - https://github.com/tonarino/innernet
       | 
       | - https://github.com/slackhq/nebula
       | 
       | Something I'm missing?
        
         | naggie wrote:
         | I made this: https://github.com/naggie/dsnet/ -- a simple
         | command to manage a wireguard VPN. Think wg-quick but quicker:
         | key generation + address allocation.
         | 
         | It's not distributed, however.
        
         | ukd1 wrote:
         | https://zerotier.com / https://github.com/zerotier/ZeroTierOne
        
         | cypherpunks1 wrote:
         | https://codeberg.org/vula/vula aims to thwart surveillance
         | adversaries by automatically building wireguard tunnels between
         | LAN hosts with zero configuration using existing IP addresses
         | and .local hostnames inside and outside the tunnels, and uses
         | CSIDH to provide post-quantum protection. Unlike these other
         | projects, it doesn't (yet?) do anything over the internet, and
         | as a result doesn't require a server or any infrastructure. If
         | it is installed on the router, internet-bound traffic is also
         | protected on the LAN.
        
         | handrous wrote:
         | A major differentiator for the (many) mesh net projects is
         | whether they are GPL (or similar) or BSD. The latter can be
         | distributed in the iOS/iPadOS App Store, so have a lot more
         | potential value for both hobbyist and business end users. Of
         | course, if it's mostly a server-to-server mesh net with few end
         | user devices, then it hardly matters.
        
         | [deleted]
        
         | JeremyNT wrote:
         | Wireguard based solutions get all the attention these days, but
         | I really love working with Nebula. It's dead simple to set up
         | and the mobile client works well.
         | 
         | That said, Nebula feels way less popular, and I don't know if
         | it's had a third party security audit. WG's popularity means it
         | should presumably have had a lot of attention given to both its
         | architecture and its code. This is one area where using a
         | "niche" solution feels somewhat risky.
        
           | KingMachiavelli wrote:
           | Nebula is pretty great.
           | 
           | However, the Android client does not allow for specifying a
           | DNS server and breaks the Android system DNS over TLS. Google
           | thinks the VPN app/client should handle DNS and Nebula app
           | development seems very slow. This means you cannot use
           | adblocking DNS while using Nebula.
        
             | handrous wrote:
             | The semi-official (worked on by folks involved with the
             | core product from, as far as I can tell, early on, but not
             | released by Slack like core Nebula is) mobile clients don't
             | have a license associated with them, which is a bit of a
             | blocker to further development or distribution on certain
             | stores (FOSS-only ones, for instance, and really _no-one_
             | else can distribute them _at all_ right now without risk).
        
           | atonse wrote:
           | Thankfully both Nebula and Wireguard are built on top of the
           | Noise Framework.
           | 
           | So neither is a compromise security-design wise.
        
             | wolverine876 wrote:
             | > Thankfully both Nebula and Wireguard are built on top of
             | the Noise Framework.
             | 
             | > So neither is a compromise security-design wise.
             | 
             | Security involves much more than the crypto framework being
             | used; just crypto requires much more than the framework.
             | Noise could be implemented insecurely, as a basic example.
        
               | atonse wrote:
               | Yeah that's probably true. But I meant in the sense of
               | it's not roll your own crypto.
        
               | wolverine876 wrote:
               | Yes, absolutely. That reduces a lot of likely error.
        
         | Jiocus wrote:
         | Cjdns,
         | 
         | > _An encrypted IPv6 network using public-key cryptography for
         | address allocation and a distributed hash table for routing._
         | 
         | https://github.com/cjdelisle/cjdns
        
         | cassianoleal wrote:
         | https://zerotier.com
         | 
         | Not Wireguard based, but similar functionality.
        
           | api wrote:
           | It has a bunch of other functionality including virtualizing
           | at layer 2 and a distributed rules engine.
        
             | linsomniac wrote:
             | ... and "public" VPNs, where all you need is the network ID
             | to connect. Seems great for playing LAN games. It also has
             | a userspace implementation, which I was able to use on
             | bitbucket CI/CD where I couldn't bring up a tun/tap device,
             | to connect via SSH to a private server. It also has Ad-Hoc
             | networks that are public, IPv6 addressed, and based on
             | port-range, so you can expose your SSH server to anyone
             | with the IPv6 address, for example.
        
           | artificialLimbs wrote:
           | I had enough problems that I had to stop using it, though
           | when it worked it was fantastic.
           | 
           | Client intermittently had seizures (at least on Windows) and
           | I couldn't join/unjoin networks properly. Would have to
           | restart the service and close the program for it to begin to
           | work again.
           | 
           | Sometimes the traffic just didn't flow ... at all. Randomly
           | couldn't connect to other clients on the network.
           | 
           | I quit about 6 mo ago. Has it improved since?
        
             | cassianoleal wrote:
             | I haven't had to use it much since covid started as I'm
             | mostly at home these days and I used it to access my LAN
             | when I was out.
             | 
             | I have tried to use it over mobile data and wasn't very
             | lucky, and that's something that used to work great. I
             | couldn't tell if it was due to ZT or the carrier. There's
             | CGNAT on both ends, which could be getting in the way,
             | although like I said it used to work great.
             | 
             | I have also noticed that it takes longer than I was used to
             | get IP, find routes between nodes, etc but once those are
             | up it works well.
        
             | mattmein wrote:
             | I've used Zerotier for several years now, and it does feel
             | that the stability on mobile has improved in the past
             | couple of months.
        
         | trickstra wrote:
         | https://github.com/gravitl/netmaker/
        
         | [deleted]
        
         | ctrlc-root wrote:
         | There's also fastd: https://github.com/NeoRaider/fastd
        
         | nyolfen wrote:
         | https://vpncloud.ddswd.de/
        
         | zxcvbn4038 wrote:
         | Everyone mentions tailscale but they want to charge you per
         | user. Their client is open source, the thing the client
         | connects to is not. I think there are better options.
        
           | ItalyPaleAle wrote:
           | https://github.com/juanfont/headscale replaces the tailscale
           | control panel with something open source and self-hosted.
           | However, it doesn't have all the features of the official
           | tailscale control panel.
        
         | 5e92cb50239222b wrote:
         | Don't forget tinc:
         | 
         | https://github.com/gsliepen/tinc/tree/1.1
         | 
         | It isn't based on WireGuard, but is a true mesh network, unlike
         | everything you've listed. No central coordination point, every
         | node is equal.
         | 
         | Think of it as BitTorrent with a few initial peers you set up
         | through a config file, and it learns of every other peer at
         | runtime (like BitTorrent does through Peer Exchange).
         | 
         | It can forward traffic through other nodes (like Tailscale and
         | unlike Nebula) and recomputes the most efficient route as peers
         | become (un)available.
        
           | kh_hk wrote:
           | After many years of using it, I am yet to find something as
           | powerful and as relatively simple to set up as tinc.
           | 
           | It might not be the best solution in terms of performance or
           | security (peer reviewed crypto something something).
        
           | ThatPlayer wrote:
           | I use tinc myself and like it. I like that I only have to
           | update a single node and all the other nodes will accept
           | those peers from that node. I've been playing with wireguard,
           | and having the other nodes discover and communicate with each
           | other directly doesn't happen without you adding the other's
           | public key to both peers.
           | 
           | I remember there was talk about switching the backend for
           | Wireguard for security and speed, but it seems to be on the
           | backburner: https://github.com/gsliepen/tinc/issues/179
        
           | fidesomnes wrote:
           | tinc sucks.
        
           | formerly_proven wrote:
           | How secure is tinc?
        
             | pedrocr wrote:
             | I really like it for my use case but just assume it's not
             | secure at all. I change the default Ubuntu install to not
             | run the daemon as root (a major flaw in my view) and then
             | just assume that network is as insecure as any other. It's
             | really useful to have all my personal machines always
             | connected between each other at the same IP independently
             | of network config. Security comes from SSH/TLS on top of
             | it.
        
               | formerly_proven wrote:
               | Considering that the wireguard mesh solutions seem to
               | require static IPs, I'm now wondering whether it might be
               | a good idea to run a wireguard mesh on top of tinc,
               | basically just using tinc as a way to get static IPs and
               | routing connectivity between nodes. That still leaves the
               | issue of updating the wireguard configuration, though.
        
               | JacobiX wrote:
               | Not sure about that, I use Tailscale (wireguard based)
               | and none of my devices have a public IP.
        
               | Intermernet wrote:
               | It seems that the value-add magic sauce of TailScale is
               | the work they've put into NAT traversal etc. There are
               | some interesting blog posts from the team regarding work
               | in this area.
        
               | generalizations wrote:
               | I wonder if the work they put in can be extracted to some
               | kind of tool the other projects can use as well. NAT
               | traversal always seems like the limiting factor in
               | projects like this.
        
               | littlestymaar wrote:
               | NAT traversal is an impossible problem in practice in the
               | general case (boo symmetric NATs), and the best-effort
               | version is a solved problem since at least 2003 with the
               | STUN protocol[1] which is part of WebRTC, and there are
               | STUN implementations available in many languages[2].
               | 
               | [1]: https://datatracker.ietf.org/doc/html/rfc3489 [2]:
               | For instance, Rust: https://github.com/webrtc-rs/stun &
               | Go:https://github.com/pion/stun, and see this HN comment
               | listing a few WebRTC implementations (which include
               | STUN): https://news.ycombinator.com/item?id=26739253
        
         | jordanwhited wrote:
         | https://github.com/jwhited/wgsd
         | 
         | is a building block that can be used to achieve something
         | similar, minus TURN-like NAT traversal fallback. The peer
         | discovery leverages DNS and UDP hole punching uses WireGuard
         | itself.
        
       | isatty wrote:
       | Looks good. I use ansible to automate WireGuard setup between my
       | nodes (n=5, m=5, m being personal devices such as phones, iPad
       | etc and n being servers) and it works well. Can't automate the
       | part where I've to manually scan a QR code for phones though.
        
         | hda111 wrote:
         | You can automate this. It's called MDM. WireGuard is
         | configurable from Apple Configuration Profile.
        
           | isatty wrote:
           | Oh that's a very good point I hadn't thought of. I kinda like
           | the idea of MDMing my own phone. Will look into it when I get
           | time. Thanks.
        
       | Tepix wrote:
       | How well does this perform with UDP traffic?
        
         | bschwindHN wrote:
         | Assuming this tool results in two peers connected over
         | WireGuard (no relay server), the UDP performance should be
         | great. Probably fine with a relay server too but I can't say
         | definitively.
        
       | timwis wrote:
       | Sorry if I'm a bit thick here, but can someone give me an example
       | use case of how someone might use this?
        
         | handrous wrote:
         | The promise of most mesh nets is that they're (more or less) a
         | self-configuring distributed VPN. Give a node some credentials
         | and (usually) a few real, publicly-routable IP addresses to
         | "bootstrap" nodes, and that's it. It's on the network.
         | 
         | There's no "real" network that they provide a gateway to,
         | necessarily, just a bunch of nodes, potentially _none of which_
         | are on the same physical or logical local network, _aside from_
         | the mesh network itself, which is an overlay network--think of
         | an overlay network like a virtual machine, but for networks,
         | incidentally: a protocol stack built on top of, or _within_ ,
         | if you prefer, another protocol stack. You feel like you're
         | talking TCP/IP over it, but under the hood your TCP packets are
         | carried _in_ other packets (may or may not also be TCP--often
         | it 's UDP when possible) over the normal Internet, then
         | reconstructed back into this virtualized, if you will, packet
         | stream on the other end.
         | 
         | With a traditional VPN, you'd connect to _a real device_ that
         | provides a gateway to, say, a corporate network, which may have
         | routing connecting multiple locations but is essentially a
         | normal network. Mesh networks operate much more peer-to-peer
         | than that, usually leveraging known peers to find routes to
         | others that are (for whatever reason) unreachable or initially
         | unknown. The  "mesh" part refers to the routing, mainly, the
         | job of which is to search out and find good paths to nodes. If
         | "corporate" has fiber cut by a backhoe, "the VPN" isn't down--
         | many, many nodes might be, but the network _might_ survive in a
         | useful form.
        
       | unixhero wrote:
       | We need an awesomelist for Wireguard
        
         | Forbo wrote:
         | I don't get the obsession with awesomelists. Seems like every
         | time I go to check them out, there's five different lists for
         | the same subject, and none of them are kept up to date. Just
         | seems like someone's weekend project that gets immediately
         | forgotten.
        
       | xaduha wrote:
       | https://github.com/wiretrustee/wiretrustee/projects/2
       | 
       | Router Nodes is what I would really like to have, I want a node
       | that doesn't have internet connection, but in the local network
       | with another node to get access to the internet through that
       | node. Wasn't able to achieve it with plain Wireguard, my linux
       | networking kungfu is weak.
       | 
       | EDIT: thanks, I'll try mentioned projects, Yggdrasil seems most
       | interesting.
        
         | 5e92cb50239222b wrote:
         | tinc supports this since forever, FWIW, and is fully open /
         | self-hosted, no ties to a central server at all (so one node
         | going down does not take your whole network with it).
        
         | MayeulC wrote:
         | How about yggdrasil? (Note: you'd still need to set up NAT and
         | default gateway yourself).
         | 
         | https://yggdrasil-network.github.io
        
         | api wrote:
         | www.zerotier.com has this at both L3 and L2 (bridging). You can
         | push routes or merge virtual and physical Ethernet.
        
         | damvigilante wrote:
         | Tailscale supports what you're looking for I believe
         | https://tailscale.com/kb/1103/exit-nodes/
        
         | thepete2 wrote:
         | Don't you just need to set the gateway and enable package
         | routing on the other one?
        
           | xaduha wrote:
           | I omitted some details, there was a third node that I was
           | trying to use as an exit. It was a while back, but whatever I
           | tried wasn't working, if it is trivial for someone else it
           | wasn't for me.
        
       | qbasic_forever wrote:
       | Did they copyright the term WireGuard (capitol G)? Wow.
       | 
       | edit: Ah this was a dumb comment, it's trademark the creators of
       | wireguard. These folks are using the name correctly. Nothing
       | fishy or weird going on.
        
         | Heliosmaster wrote:
         | > (c) Copyright 2015-2020 Jason A. Donenfeld. All Rights
         | Reserved. "WireGuard" and the "WireGuard" logo are registered
         | trademarks of Jason A. Donenfeld.
         | 
         | From the original website
        
         | [deleted]
        
         | [deleted]
        
       | atonse wrote:
       | I would love for this to be integrated into Consul Connect so all
       | our systems just run in one mesh and all internal traffic is
       | encrypted without even thinking about it.
        
       | est wrote:
       | Does Android ship newest kernel with Wireguard? If so can we have
       | mesh network on Android devices?
        
         | aquaticsunset wrote:
         | I'm a huge fan of Tailscale, and in an effort to understand
         | better everything they've done under the hood to make it work,
         | I decided to redo my network with native WireGuard.
         | 
         | A) WireGuard works great when it's all set up! Config is a bear
         | but it's a great protocol. Works well on lots of different
         | devices in different usages - Android, laptops, desktops,
         | headless servers, IoT.
         | 
         | B) I have even more respect for Tailscale for doing all of this
         | in such a polished, seamless way.
        
         | ptman wrote:
         | wireguard works in userspace as well. It just works faster with
         | kernel support.
        
       | e12e wrote:
       | Anyone have a comparison to headscale (self-host/oss tailscale)?
       | https://github.com/juanfont/headscale
       | 
       | I suppose the turn server support for nat hole punching is the
       | biggest difference?
        
       | fabian2k wrote:
       | Are there any simple tools to manage a small WireGuard VPN with a
       | handful of servers and something like low double-digit number of
       | clients? By servers I mean Linux servers/VMs that expose some web
       | services and clients are simply desktop/notebooks that need to
       | access the servers, but don't expose services to the VPN.
       | 
       | From the description, this tool isn't what I want. I have
       | configured a simple version by hand with Wireguard, but it did
       | seem like raw WireGuard is too low-level for my purpose and my
       | networking knowledge. Setting up the keys does need some level of
       | automation for anything larger than a sandbox. And I didn't
       | figure out a way to resolve domains within the VPN without
       | interfering with the DNS on the clients for everything else.
        
         | EwanToo wrote:
         | Pritunl maybe?
         | 
         | It has a Wireguard option that works
         | 
         | https://docs.pritunl.com/docs/wireguard
        
         | andriosr wrote:
         | I think it can't get easier than https://runops.io
         | 
         | No VPN, DNS, IPs, and so on; and get to the same result: any
         | device accessing resources in private networks, besides a few
         | extra features.
        
         | 411111111111111 wrote:
         | I've never used it myself, but a sister comment mentioned
         | related projects working with wireguard and innernet seems to
         | be exactly that use case from a casual glance. You can specify
         | full cidr rules, so it should be possible to prohibit
         | connections between nodes by segmenting the IPs
         | 
         | https://github.com/tonarino/innernet
         | 
         | There are easier options around if all you wish is ssh access
         | to your servers though. Personally I'd recommend gravitational
         | teleport, mainly because the name is so hilarious
         | 
         | https://goteleport.com/
        
           | SEJeff wrote:
           | Teleport also has some really neat approval workflows where
           | you can say "2 of these 3 people need to approve a request
           | for people in group Y to access servers in group X". The
           | incoming request can ping you on slack and then you go to the
           | teleport webui (U2F authenticated with SSO) to approve.
           | 
           | It also has some neat stuff around eBPF session recording if
           | you want to be able to playback user sessions for high
           | security environments. Teleport is good tech.
        
             | nickik wrote:
             | Bothers me that I can not set up a backup U2F stick.
             | 
             | Also, I had issues to configure it for my situation with no
             | static IP and behind a proxy. I have it working but not
             | really exactly how I wanted it. Maybe its my lack of
             | understanding, I'm sure it could be made to do exactly what
             | I want somehow.
             | 
             | All said, it pretty nice but I reverted to using the FIDO2
             | based keys sticks directly.
        
         | e12e wrote:
         | You might want to look at tailscale.com - I'm not 100% certain
         | I understand exactly what you want - but it does allow any
         | client to expose services; assuming non-server
         | wireguard/tailscale-clients don't expose services/have firewall
         | set up -I think it does what you need?
         | 
         | ZeroTier should also be an option.
        
         | 0xbadcafebee wrote:
         | The problem is people keep thinking WireGuard is a VPN when
         | it's more of a network tunnel + routing, like OpenSSH's _-w_
         | option. A  "real" VPN has a ton more features to provide the
         | user experience needed for typical VPNs. I believe OpenVPN is
         | still king here, probably one or two others as well.
        
           | epistasis wrote:
           | Agreed. At a minimum it would be good to provide a way to
           | protect wireguard credentials at rest, so they aren't just
           | sitting out in the open on disk as a single factor for access
           | to the network. If I had more experience I would love to
           | submit a patch for the default client software, but I don't
           | have much experience, and you don't want inexperienced crypto
           | programming, especially for new features....
        
       | throwawaysea wrote:
       | For someone who has no idea what this means, can someone please
       | offer a simple explanation? When I read "mesh network" I think
       | about those routers with modules you can use to get good WiFi
       | coverage at home. This sounds more like connecting your devices
       | across the Internet in some private manner. What would you use
       | this for?
        
         | ptman wrote:
         | WireGuard is a VPN protocol. There are roughly two kinds of
         | VPN. Those that you use to hide who you are when accessing the
         | Internet (route all your traffic via the VPN to an exit node)
         | and those that you use to build a private network over a public
         | one for e.g. a company or other organization.
         | 
         | Wiretrustee seems to be for generating a private org network.
         | Could be personal, work, circle of friends or anything. Most
         | traditional work VPNs have one or more nodes that all traffic
         | pass through and that all connect to. A mesh VPN instead allows
         | VPN peers to send traffic directly to each other over the
         | public Internet. There seems to be a central
         | coordination/signaling server, as with tailscale, headscale and
         | innernet. But that is mainly used to find where the traffic
         | should be sent, not for sending all traffic through it.
        
         | zimpenfish wrote:
         | I use it for connecting my tiny SMTP ingress machine to my big
         | SMTP backend machine and doing the ClamAV scanning over the VPN
         | link (because it's now far too heavy for the tiny SMTP machine
         | these days but the big machine has plenty of resources free.)
        
       ___________________________________________________________________
       (page generated 2021-06-29 23:02 UTC)