[HN Gopher] Show HN: Exatorrent - Self-hostable Torrent client w...
___________________________________________________________________
Show HN: Exatorrent - Self-hostable Torrent client written in Go
Author : varbhat
Score : 156 points
Date : 2021-08-25 13:28 UTC (9 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| 19h wrote:
| Very cool project. I'd love to see more Go projects pop up
| because they're absolutely great to learn Go. Have you had a
| chance to benchmark the maximal throughput? (I.e. 100MB/s on a
| 1Gbps max?)
|
| I've been using Synapse (https://synapse-bt.org/synapse/, a
| torrent client written in Rust) recently and so far it was the
| only client fully saturating a 10Gbps NIC.
| varbhat wrote:
| I didn't benchmark the maximal throughput although I may in
| future .
| newman314 wrote:
| What could be interesting is if you were able to add wireguard-go
| into your project.
|
| It would be nice to have a single binary that can run traffic
| through a VPN and something I have not see done before.
| georgyo wrote:
| This is actually much more complicated than just using
| wireguard-go.
|
| That currently just creates an interface, but that is just for
| raw packets. The kernel is the one that is still handling the
| IP connections.
|
| To have a process do what you are implying would require an
| entire userspace TCP/IP stack.
|
| These are pretty rare. Slip4netns is an example of one.
| lunixbochs wrote:
| There's a user space network stack in gvisor, written in Go.
| And it's already included in wireguard-go.
|
| See also https://fly.io/blog/ssh-and-user-mode-ip-wireguard/
| which has already done this for SSH through wireguard-go
| varbhat wrote:
| Nice. I will consider.
|
| You can route network of exatorrent through network/VPN .
|
| Also, wireguard implemented in Linux kernel is faster than
| userspace implementation of wireguard-go . No?
| newman314 wrote:
| Userspace is slower but the idea is to have a single binary
| that can download torrents over wg. I have not seen anything
| like this before and it would make for a nice differentiator.
| y4mi wrote:
| > _You can route network of exatorrent through network /VPN
| ._
|
| that always comes with the danger of _not_ going through VPN
| if the connection somehow drops or you 've restarted the
| machine its running on and the vpn started after the torrent
| client.
|
| and the linux kernel implementation isnt necessarily
| available if its running in a docker container of some NAS or
| even on a windows machine.
|
| it would also make it easier to use, allowing more people to
| utilize the tech.
| varbhat wrote:
| Thanks for suggesting me.
|
| I will consider this.
| HansHamster wrote:
| > that always comes with the danger of not going through
| VPN if the connection somehow drops or you've restarted the
| machine its running on and the vpn started after the
| torrent client.
|
| My current setup uses namespaces to avoid this issue.
| Basically, one can create a wg interface in the default
| namespace and then move it to a new namespace and run
| applications there. The wg interface keeps routing the
| encrypted traffic through the interfaces in the default
| namespace, but as there is only the wg interface in the new
| namespace, there is no risk of leaking any traffic. It also
| doesn't need a network bridge and doesn't disturb the
| network configuration of the host.
|
| I use lxc, but there are probably more lightweight options
| to just start an application in a different network
| namespace.
|
| See: https://www.wireguard.com/netns/
| oktwtf wrote:
| What a great suggestion! I had debated rolling over to wg
| as my VPN provides it, however hadn't worked out how to
| manage the Binding for the torrent applications.
|
| This relieves me having to manage restarts when the VPN
| connection issues for example a new IP. I currently just
| allow some scripts to refresh things when the uplink
| changes.
| michaeldwan wrote:
| We use wireguard-go in our cli for connecting to vms over a
| private network. It is slower than wireguard in the kernel, but
| the convenience is hard to beat.
|
| Here's an overview https://fly.io/blog/ssh-and-user-mode-ip-
| wireguard/
| _joel wrote:
| Have you tried https://github.com/KusakabeSi/wireguard-go-vpp
| ?
| devops000 wrote:
| Which legit activities are all you using torrent for?
| yjftsjthsd-h wrote:
| Amusingly, Humble Bundle offers things like ebook purchases as
| torrent downloads.
| kuschku wrote:
| I provide some large binaries and data dumps of my own projects
| as torrent as well.
| emnudge wrote:
| While I've yet to do it, I've heard of many people using it for
| peer to peer file transfer. I tend to transfer files between
| devices pretty often and have never found a perfect solution.
| I'd like to do the same soon enough once I figure out exactly
| how to do it.
| pessimizer wrote:
| Create a torrent for what you're trying to share, add a few
| open tracker announce urls[1] to it, then load it in you
| client. Then, anyone else who has a torrent client can
| download it with a magnet link[2].
|
| One caveat is that either your client and router both have to
| support UPnP, or you have to manually open up a port for UDP
| traffic in your router and forward that to the machine you're
| running the client on.
|
| I also use transmission-create to make torrents from the cli,
| but the UI of your client will have an option.
|
| I use magnets to transfer all large files - although if
| there's anything confidential in the file, encrypt it first.
| The only thing keeping people from plucking your files out of
| the ether is a lack of interest.
|
| -----
|
| [1] Such as https://newtrackon.com/list
|
| [2] Which are magnet:?xt=urn:btih:[insert torrent hash here]
|
| Torrent hashes can be found in the UI of your client, or at
| the beginning of the actual .torrent file.
| andrewzah wrote:
| Unironically, linux isos. Mostly debian and raspbian based
| images.
| Deukhoofd wrote:
| Linux distributions, downloads that are very popular, but where
| the servers can't keep up with it (i.e. small indie games
| people hosted the download links for themselves)
| throwaway77384 wrote:
| Funnily enough, everything possible. Everytime there is a
| torrent alternative to the direct download, I will choose
| torrent, always. But, thanks to pirating stigma, torrenting
| hasn't proliferated anywhere near as much as it should.
| Zababa wrote:
| Usually downloading the contents of physical media that I have
| bought. It can be because I don't have anything to read it, or
| because I'm not at home.
| aarchi wrote:
| Internet Archive collections are available as web seed torrents
| and download much faster than over HTTP because multiple
| servers are used concurrently.
| rapnie wrote:
| PeerTube uses it, so as you watch videos you start sharing as a
| peer with others to reduce load on the host.
|
| https://joinpeertube.org/
| ruuda wrote:
| Downloading Have I Been Pwned's list of pwned passwords
| (https://haveibeenpwned.com/Passwords)
| wiz21c wrote:
| raspbian download
| ducktective wrote:
| Possible legit usage: https://academictorrents.com/
| faheel wrote:
| For seeding Big Buck Bunny of course. And Linux distros too.
| That's all :)
| freedomben wrote:
| I seed all linux distros that I use. Bandwidth is expensive
| when centralized. It pains me to think of dollars going toward
| paying for egress data rather than developer work for open
| source projects. Please download them via torrent instead of
| http, and then please seed!
| mishafb wrote:
| Isn't most of the bandwidth spent on packages? Even
| immediately after the initial install you probably have
| gigabytes of updates + new packages.
| yjftsjthsd-h wrote:
| Yeah, it still blows my mind that nobody seems to have
| solved putting packages in torrents.
| rovr138 wrote:
| Apt-torrent - https://directory.fsf.org/wiki/Apt-torrent
|
| I haven't used this a long time, but it used to be faster
| than mirrors where I was.
| systemvoltage wrote:
| I often wonder - who pays for pip packages that millions of
| server apps and CI/CD pipelines pull? There is some caching
| but still it's gotta be enormous.
| dwrodri wrote:
| The official PyPI packkages are hosted on Fastly. I
| believe they donate the infrastructure costs through the
| Python Software Foundation, as the bandwidth is over
| 300TB a day, and would cost about 1.8 million dollars a
| month.
|
| Source:
| https://dustingram.com/articles/2021/04/14/powering-the-
| pyth...
| minusf wrote:
| interesting article. where is the 300 TB number coming
| from?
|
| > The "maximum" here is per-day, so this means that at
| the peak (which was the day I made these graphs) PyPI
| served nearly 900 terabytes over more than 2 billion
| requests per day.
| dwrodri wrote:
| This tweet from Dustin Ingram:
| https://twitter.com/di_codes/status/1235707819955032069
|
| I think this number is dated but its what I stumbled
| across.
| minusf wrote:
| might be. either way the numbers are impressive although
| i am afraid this massive bandwidth is necessary because
| of the docker's and ci/cd's that don't use caches...
| majkinetor wrote:
| Please create Windows binary.
|
| Is it possible to control it via web interface ?
| varbhat wrote:
| Yes, it is possible to control it via web interface.
|
| I wanted to create Windows executable. I tried cross compiling
| using mingw and asked my friend to test it but it reported some
| error namely libstdc++ error and similar ones. So,i focussed
| first on releasing software first and look into this issue
| later.
|
| I also wanted to release Binary for MacOS too ,so went to setup
| osxcross cross compiler but couldn't setup it properly too. I
| wanted to work on this after release.
|
| So, Help is appreciated regarding this matter.
| [deleted]
| throwaway77384 wrote:
| Let's assume I want to run this on a Hetzner box. And let's
| assume all my activity is legit torrenting. Say, sharing linux
| distros. They are still not exactly going to be over the moon
| with my seedbox running on their servers, due to torrenting
| stigma.
|
| You can replace 'Hetzner' with any VPS service here, really. What
| measures can one take to 'hide' torrenting activity?
| boudin wrote:
| Never had issue with OVH or Scaleway
| throwaway77384 wrote:
| Alright, cool, I'll probably try that!
| isatty wrote:
| I've been running private torrents on Hetzner for years and
| they've not kicked me out yet.
| sixhobbits wrote:
| Maybe something like https://github.com/haugene/docker-
| transmission-openvpn
| robcohen wrote:
| Don't see how this would help, only thing that would help is
| a tunnel out, like WireGuard to a VPN or less reputable
| provider.
| beepbooptheory wrote:
| Maybe I shouldn't jinx it... but I run this container on
| hetzner, with a proprietary vpn service, and have never had
| any problems for a long long time now
| hauxir wrote:
| You can tunnel traffic through NordVPN.
|
| This is for rapidbay but should work for any service:
| https://github.com/hauxir/rapidbay/wiki/Setting-RapidBay-
| up-...
| throwaway77384 wrote:
| How much does this diminish performance?
| sodality2 wrote:
| Speed is probably 85-95% of normal speeds. Maybe less if
| the speed is gigabit. Depends on servers or VPN service.
| throwaway77384 wrote:
| Ok great, thanks :)
| alaties wrote:
| Best option is to force encryption on all bittorrent traffic.
| Most clients support full protocol encryption (PE or MSE) these
| days, so forcing it shouldn't be an issue. For libtorrent,
| you'd set in_enc_policy and out_enc_policy to pe_forced.
|
| Whether a particular client exposes this or not depends. In the
| case of this project... looks like upstream
| github.com/anacrolix/torrent does support encrypted
| connections. Not clear how to force it to use _only_ encrypted
| connections though.
| vorpalhex wrote:
| Well, one would find a provider who focuses on seedboxes.
| Usually these providers - at least the ones who last - put
| their boxes in a foreign data center and mix it with other
| traffic.
| 19h wrote:
| Only use private trackers :-)
| dmos62 wrote:
| Better clients can periodically change between random ports and
| use encryption. I would expect such a client to require a
| pretty specific search to be discovered. What we can't really
| protect from is someone watching the global P2P traffic by
| posturing as another client, but that's a bit more involved
| than doing a wireshark survey.
| [deleted]
| Aeolun wrote:
| I haven't hidden my torrenting activity on my Hetzner box for
| years, and they haven't kicked me off.
|
| I assume their problem would be mostly with a huge amount of
| traffic, not torrent per se.
| lstodd wrote:
| Yep, they don't care so long as you don't cause problems for
| them.
|
| If only they didn't put 5.14 kernel into their rescue image
| this summer, but that's another story.
| throwaway77384 wrote:
| Why is kernel 5.14 an issue?
| _joel wrote:
| Never had an issue, I encrypt everything fwiw.
| maeln wrote:
| I have run torrents on several VPS/Server provider (mostly OVH)
| and never had any issue.
| throwaway77384 wrote:
| Good to know, thanks :)
| crossroadsguy wrote:
| Kimsufi onced nuked my server. Someone was running the usual
| blanket port scan and my IP was scanned. That was it. Weeks
| later someone replied to the ticket with an one liner "server
| hack suspected". Then radio silence forever.
| johnchristopher wrote:
| johnchristopher@localhost ~/Download $ ./exatorrent-
| linux-amd64 Address => :5000 Directory =>
| exadir [INFO] 2021/08/25 22:48:04 Adding Admin
| user with username "adminuser" and password "adminpassword"
| [INFO] 2021/08/25 22:48:05 Torrent Client Created [INFO]
| 2021/08/25 22:48:05 Starting server on :5000 [INFO]
| 2021/08/25 22:48:05 Loaded 20 trackers from
| https://ngosang.github.io/trackerslist/trackers_best.txt
| [INFO] 2021/08/25 22:48:05 Loaded 20 trackers in total ,
| eliminating duplicates [INFO] 2021/08/25 22:48:05 Added
| Loaded Trackers to Torrents
|
| Extra points for logging relevant info when first starting it up
| and not taking me on a hunt in a readme or something :).
| varbhat wrote:
| Thank you. I hope you like exatorrent
| Aissen wrote:
| Hmmm, had a look, and the default listen address + default user
| / password are giving me shivers. @varbhat I'd advise you to
| change the default listen address from ":5000" to
| "127.0.0.1:5000" to only allow local connections. And maybe
| generate a password then for good measure, since you're
| printing it anyway :-)
| varbhat wrote:
| Hello HN ,
|
| I have worked on this for past few months . exatorrent is
| BitTorrent Client written in Go. It can be run locally or be
| hosted in Remote Server with good resources to use that Server as
| SeedBox. It is Single Completely Statically Linked Binary with
| Zero External Dependencies .
|
| exatorrent is simple yet feature-rich . It is lightweight and
| light on resources. It comes with Beautiful Responsive Web Client
| written in Svelte and Typescript , but thanks to documented
| WebSocket API of exatorrent , you can also write your own client
| or integrate with other services if you want to.
|
| exatorrent can operate in Optional Multi-User Mode administrated
| by admin user/s but it can successfully be used by Single-User as
| well. Torrented Files are stored in local disk of device where
| it's run, which are then retrievable or streamable via HTTP.
|
| I hope you like it . It's Open Source .
|
| Also HN , I am looking for Internship . If you're interested ,
| please email me
| Qem wrote:
| I used to torrent a lot, but gave up after my ISP adopted CGNAT
| and I lost the capability to forward ports locally. Do people
| adapted bittorrent to IPv6, since then, to circunvent this
| problem, or are we still stuck with IPv4 in this space?
| vxNsr wrote:
| I've never had to do any port forwarding on my local router do
| torrent anything, I'm not sure what you were experiencing but
| it might have been transient.
| cassianoleal wrote:
| Some VPN providers, like Mullvad, offer the option of port-
| forwarding.
|
| I'm not sure if it would work for torrenting but I can't think
| of a reason why it wouldn't.
| Isthatablackgsd wrote:
| > Some VPN providers, like Mullvad, offer the option of port-
| forwarding.
|
| The thing is Mullvad offers dynamic Port-Forwarding only.
| They get to decide what port number they can grant for users
| and users cannot request for a specific port. There are VPNs
| that have static port-forwarding which is rare. There are
| less than 5 VPN providers that have static port-forwarding
| from what I remember and users can decide what port number
| they want (unless specific port number is taken prior). It
| would be a pain if someone have a range of applications that
| already using specific port prior and would have to change
| them to use the VPN's defined port number depending on how
| much applications they have to change.
| dmos62 wrote:
| I'm a newby in these topics. I'm pretty sure that bittorrent
| has worked in ipv6 for a long time now; actually, I find it
| unintuitive that it would care whether one or the other is
| used. Also, doesn't bittorrent work fine without port
| forwarding? I'm pretty sure that I used to run it that way for
| a long time when my ISP didn't give me a public IP.
| Qem wrote:
| It works, but the number of peers and speeds decrease a lot.
| At least in my experience. I haven't deep knowledge on how
| the protocol works.
| louis-lau wrote:
| Correct. At least one side must have an open port. So if
| that's you you can peer with everyone instead of only
| people with open ports.
| Anunayj wrote:
| Port forwarding will allow you to receive connections from
| clients that are themselves behind NATs, and would also
| allow you to participate in DHT which would allow you to
| discover clients faster.
|
| Though in ideal cases (ones with high active seeders), that
| isn't going to affect performance that much. Torrent can
| also use UDP hole punching, but that can sometimes be
| finnicky behind CGNATs
| no_time wrote:
| People torrent just fine behind CGNAT though? I never
| understood how hole punching works but im pretty sure the
| protocol has something like that.
| koeng wrote:
| Very nice! Any known limitations on number of torrents?
| varbhat wrote:
| No limitations on number of torrents .
| Arnavion wrote:
| There are non-"self-hostable" torrent clients?
| rahimnathwani wrote:
| Yes, you can download torrents with Baidu Wangpan
| (https://pan.baidu.com/) which cannot be self-hosted.
|
| AWS S3 used to have torrent client (seed-only), which could not
| be self-hosted.
| Arnavion wrote:
| Seed-only clients that can't be self-hosted, I can
| understand, because what you care about is not the client but
| the (network, etc) infrastructure that the client runs on.
|
| What's the use-case of Wangpan? I assume it's not "I don't
| want to torrent illegal things on my home network" because
| such a service could rat me out just as well.
| throwaway_2047 wrote:
| Tangential, I think adding support of http://webtorrent.io/ by
| bridging webtorrent peer with bittorent peer would greatly
| increase/revive the popularity of the protocol
| vxNsr wrote:
| Amazing impressive work!
|
| Was there a specific reason you built this or did you just want
| to create a showcase?
| varbhat wrote:
| I built this as a project to learn things. I used to dislike
| myself for not being productive.
|
| exatorrent helped me to learn new things ( go , databases ,
| typescript , websocket , CI/CD ,etc.).
|
| so,here it is.
| toomuchtodo wrote:
| You have done an exceptional job. Congrats on the
| productivity!
| varbhat wrote:
| Thank you. I hope you like it.
| vxNsr wrote:
| Really impressive! I have the same issue, its pretty
| motivating to see this. How long did it take you?
| m3nu wrote:
| Looks good. Congrats on this release! How is memory usage
| compared to e.g. Transmission?
| varbhat wrote:
| Memory use is pretty low. It's around 5-6 MB initially and will
| increase as you add more torrents.
|
| Also you can measure Memory Usage and other Stats in real time
| in Web Client of exatorrent(also there is API for it)
| yodsanklai wrote:
| Impressive work. Did you implement the bittorrent protocol from
| scratch? which BEP specifically?
|
| https://www.bittorrent.org/beps/bep_0000.html
| varbhat wrote:
| No , I used https://github.com/anacrolix/torrent to build
| exatorrent
| eternalban wrote:
| You should add a NOTICE file to your repo, stating the fact
| that you have used portion of `anacrolix/torrent` OSS
| software:
|
| https://github.com/anacrolix/torrent/blob/master/LICENSE
| varbhat wrote:
| Done .
| https://github.com/varbhat/exatorrent/blob/main/NOTICE.txt
| creamytaco wrote:
| Here is some advice I hope you find useful: I wouldn't hire
| you based on this because all I see is a shit ton of 3rd
| party libraries stitched together. The exact opposite of what
| I would like to see in a good engineering candidate.
|
| So you may want to rethink your approach in terms of putting
| forth qualities that speak to your engineering prowess.
|
| I also wouldn't personally use this software as it has so
| many 3rd party dependencies that one would find it hard to
| audit or make any robustness / security guarantees.
| robterrell wrote:
| FFS. He's just looking for an internship.
|
| He's demonstrating competence in writing go (and yes,
| selecting 3rd party libraries and stitching them together
| is often a skill you need in the Real World), using docker,
| understanding web apps with Svelte + typescript, etc.
| Scratching his own itch, literally the original hacker
| ethos.
|
| Nice work, OP. Good luck in your search.
| throwaway_2047 wrote:
| And I cherish the fact that all software is build on
| shoulder of giant. I see "leverage" instead of shit ton of
| 3rd party libraries. But of course, maybe there are Linus
| Torvalds with a Putnam prize under their belt, what do I
| know.
| varbhat wrote:
| Thank you .
|
| Please Open Issue if you find any Bad or Buggy Code . I
| would correct it.
|
| Also , exatorrent doesn't use much 3rd party libraries like
| you think
| isatty wrote:
| Hey just a nitpick here but please don't use spaces
| before punctuation.
|
| For example: thank you. Also, exa...
| mrtweetyhack wrote:
| i bet you build everything from scratch
| toomuchtodo wrote:
| This is not constructive feedback for a Show HN thread.
___________________________________________________________________
(page generated 2021-08-25 23:01 UTC)