[HN Gopher] Show HN: I wrote a new BitTorrent tracker in Elixir
___________________________________________________________________
Show HN: I wrote a new BitTorrent tracker in Elixir
Hello everyone! I'm currently in a journey to learn and improve my
Elixir and Go skills (my daily job uses C++) and looking through my
backlog for projects to take on I decided Elixir is the perfect
language to write a highly-parallel BitTorrent tracker. So I have
spent my free time these last 3 months writing one! Now I think it
has enough features to present it to the world (and a docker image
to give it a quick try). I know some people see trackers as relics
of the past now that DHT and PEX are common but I think they still
serve a purpose in today's Internet (purely talking about public
trackers). That said there is not a lot going on in terms of new
developments since everyone just throws opentracker in a vps a
calls it a day (honorable exceptions: aquatic and torrust). I plan
to continue development for the foreseeable future and add some
(optional) esoteric features along the way so if anyone currently
operates a tracker please give a try and enjoy the lack of crashes.
note: only swarm_printout.ex has been vibe coded, the rest has all
been written by hand.
Author : dahrkael
Score : 370 points
Date : 2025-06-19 22:49 UTC (1 days ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| IlikeKitties wrote:
| Now that's neat. The Beam VM sounds like a natural fit for a
| torrent tracker
| dahrkael wrote:
| I feel like ETS has been the real killer feature to pull this,
| being able to concurrently read and write from protected tables
| makes the whole thing incredibly parallel
| desireco42 wrote:
| Now this is serious business, congrats on the project! I can see
| how this is perfect fit for elixir...
| abrookewood wrote:
| Hey congrats on the launch! Can you provide any details on how it
| runs compared to opentracker? I'm really interested in the
| performance etc.
| dahrkael wrote:
| For small trackers opentracker is probably faster and use a bit
| less memory. Where extracker is gonna shine compared to it is
| when core count starts having 2 digits. I still have to do a
| proper benchmark though.
| jhgg wrote:
| Really cool! You looking to write Elixir as your main job?
| dahrkael wrote:
| It's one of my options yes, I'm sure I would enjoy it more than
| C++.
| pdimitar wrote:
| Not OP but I am working with Elixir for 9 years and 2 months
| now. Know Rust and Golang as well. You hiring?
| bavell wrote:
| Very cool! Is this suitable for using as a private tracker?
| dahrkael wrote:
| not out of the box but it can be done. All the required moving
| parts are there (hash whitelist support, udp path parsing, peer
| rejection, etc).
| toomuchtodo wrote:
| Awesome work!
| s-mon wrote:
| Love Elixir so much, building a kick-ass notification engine with
| it now. Its so so good.
| mikehostetler wrote:
| nice, private or OSS? Elixir needs a better notification engine
| badly
| rhgraysonii wrote:
| What do you mean exactly? If you need a notification engine,
| reaching for a pubsub implementation is very easy with
| phoenix's popularity and quite battle tested. I've
| implemented notifications at scale a few times in the
| ecosystem. What problems are you encountering that you don't
| feel you have a tool in the shed to work with in this case?
| rhgraysonii wrote:
| And if you wanna buy over build, check out
| https://knock.app - also built in elixir
| mikehostetler wrote:
| really cool project! well done
| voicedYoda wrote:
| Well done. Couple quick notes, move to a logger instead of using
| IO.puts. Also consider adding OTel.
| ai-christianson wrote:
| What's your favorite otel sink?
| solid_fuel wrote:
| I'll second this, just using the built in Logger [0] and
| Telemetry [1] applications would be fine, opentelemetry or
| anything else can be added to the telemetry hooks easily to
| export the metrics later.
|
| [0] https://hexdocs.pm/logger/1.18.4/Logger.html [1]
| https://hexdocs.pm/telemetry/readme.html
| dahrkael wrote:
| I use Logger through all the project except for the table
| pretty print. Telemetry is optionally available in prometheus
| and html formats: https://github.com/Dahrkael/ExTracker/tree/
| master/lib/ex_tra...
| guywithahat wrote:
| There's something about C++ developers that makes them love Go
| and Elixir (and I include myself in this demographic). I think
| it's something about the people who are attracted to C++ for
| performance are attracted to Go/Elixir for its multithreaded
| performance. Really cool project
| uncircle wrote:
| Not sure about C++ devs, but Erlang/Elixir are great to handle
| parsing of protocols, with its implementation of pattern
| matching. Also, makes the code much cleaner because pattern
| matching basically eliminates most branching and thus depth of
| the code base.
|
| The let it crash philosophy allows you to ignore most corner
| cases with the knowledge that, if they are encountered or a
| cosmic ray flips a bit, the crash is localised to a single
| client. I have worked with Elixir almost a decade at this
| point, and I have never seen an unexpected downtime of the apps
| I deployed. Aside of maintenance and updates, they all have
| 100% uptime. How cool is that?
|
| This is how I sell it to clients. "Will you be using Python,
| Go?" Me: "What about Elixir and the promise that your service
| won't ever crash? And you get cool dashboards with it." Them:
| "Sold."
|
| I wish there was a systems language that allows you to pattern
| match on structs and enums, and in function signatures like
| Elixir
| dahrkael wrote:
| Indeed. when your daily job is tracking down memory stomps,
| deadlocks, invalid pointers and unexpected state in very big
| codebases then using Elixir feels like "why is this so easy?
| it just works?". Also i'm a network programmer so the binary
| pattern matching is very much appreciated.
| Thaxll wrote:
| "The let it crash philosophy allows you to ignore most corner
| cases"
|
| This is such a dangerous take. Also Elixir is not strongly
| typed, so...
| ricketycricket wrote:
| It's not though. Processes can be supervised and crashes
| can just lead to "restart with good state" behavior. It's
| not that you don't try handling any errors at all, you just
| can be confident that anything you missed won't bring the
| system down.
|
| And Elixir is strongly typed by most definitions. Perhaps
| you mean static?
| immibis wrote:
| You can be more confident. But remember that time an
| Ericsson switch crashed upon handling a message that it
| sends to adjacent switches every time it restarts? That
| crashed the whole network, and you could still do that in
| Erlang.
| sea-gold wrote:
| Maybe look into Inko[1]. It is has good pattern matching (but
| perhaps not on function/method signatures).
|
| [1] https://docs.inko-lang.org/manual/latest/getting-
| started/pat...
| quechimba wrote:
| Nice! I'll check this out some point.
|
| I wrote a basic tracker in Elixir a few years ago, here's the
| code: https://github.com/aalin/mr_torrent
| dahrkael wrote:
| interesting! why did you decide on a private tracker
| specifically?
| arthurcolle wrote:
| Check out https://github.com/sergiotapia/magnetissimo it was a
| popular one like 7 years ago
| KomoD wrote:
| That's not a bittorrent tracker.
| vivzkestrel wrote:
| - how did you start - did you refer to other projects - how long
| did it take - how much functionality do you think works compared
| to say qbittorrent?
| lionkor wrote:
| it's a tracker, not a torrenting client.
| NooneAtAll3 wrote:
| what does tracker mean?
| devoutsalsa wrote:
| A torrent tracker is basically the world's most antisocial
| matchmaking service that knows who has what files but
| refuses to actually store anything itself, like that friend
| who always knows where the party is but never hosts one.
| When your BitTorrent client asks "hey who's got that Linux
| ISO," the tracker dumps a list of IP addresses faster than
| a startup pivoting after their Series A falls through. Your
| client then connects to these strangers (seeders with
| complete files and leechers still downloading) and starts
| exchanging data while the tracker pretends nothing
| happened. It's like Tinder but for file sharing, except
| everyone's anonymous and probably downloading something
| weird at 3am.
| vjerancrnjak wrote:
| not anonymous at all, while interacting with the tracker
| can be done with https, all of the communication between
| peers is unencrypted.
| immibis wrote:
| There's an optional encryption extension, with no BEP
| because the BitTorrent company (which issues BEPs) is
| ideologically opposed to encryption.
| dahrkael wrote:
| I started because I needed a tracker for another project but
| the tracker turned to be more fun to make. I did glance over
| other trackers code but their code tends to be either overly
| complex or too simple so not very useful. So far its been 3
| months of revenge bedtime procrastination. While this is not a
| client like qbittorrent I have ideas for a seedbox-oriented
| client project in the future.
| KomoD wrote:
| I tried it, couldn't get HTTPS to work.
|
| Also my console gets spammed with:
|
| 04:43:20.160 [warning] invalid 'event' parameter: size: 6 value:
| "paused"
|
| but it seems to work. I would've liked to see HTTP stats too but
| I guess UDP is fine (though I have it disabled)
| bill876 wrote:
| The "paused" event is part of BEP 21. Clients send it to the
| tracker to let it know that the client is still incomplete, but
| won't download anymore. For example, because a user only wants
| some files from the torrent. Readme of the project shows that
| support for BEP 21 is not implemented.
| KomoD wrote:
| > Readme of the project shows that support for BEP 21 is not
| implemented.
|
| Ah, missed that.
| dahrkael wrote:
| Telemetry for the HTTP side is in my ToDo list yes, since I'm
| using a 3rd party library for the webserver I still need to
| figure out how to do it right. For HTTPS to work you need to
| provide a valid certificate path in :https_keyfile but right
| now I would recommend sticking Caddy or Nginx in front of the
| tracker if you want HTTPS. I have certbot integration planned
| but is not a priority since most of the torrent peers use UDP.
| nayuki wrote:
| Similar to: https://news.ycombinator.com/item?id=44265851 "Show
| HN: I wrote a BitTorrent Client from scratch" [2025-06-13],
| https://github.com/piyushgupta53/go-torrent-client
| lionkor wrote:
| No that's a client
| arch-choot wrote:
| Interesting! I'd done something similar in Typescript to learn
| more about BT, and then redid it in rust to learn rust
| (https://github.com/ckcr4lyf/kiryuu).
|
| However I decided to just use redis as the DB. It sounds like
| your entire DB is in memory? Any interesting design decisions you
| made and/or problems faced in doing so?
|
| (My redis solution isn't great since it does not randomize peers
| in subsequent announces afaik)
| dahrkael wrote:
| in my case using the in-memory ETS has been the best decision,
| it lets me read&write the peer's data concurrently each on its
| own process so contention and latency are minimal. the only
| sequential part is when a new swarm is initially created but
| that doesn't happen a lot so its fine. there's sadly no native
| support for taking random rows directly from the tables, so for
| now i grab the whole swarm and then take a random subset (https
| ://github.com/Dahrkael/ExTracker/blob/master/lib/ex_tra...)
| toast0 wrote:
| I don't remember if there's a way to see how many slots an
| ets table has, but if you're ok with imperfect distribution,
| you could maybe pick a slot at random and use ets:slot/2 to
| get all the items in that slot, then select from those.
|
| You might be able to get the slot count from
| eta:table_info(Table, stats), although that's not intended
| for production use, so the format may change without notice.
| nesarkvechnep wrote:
| I really wished to see an OTP-first design. Unfortunately for me,
| the code is almost procedural as it's touching ETS or
| Application, which is built on ETS, in nearly every operation.
|
| If the author wishes to learn how to design services in Elixir,
| or any BEAM language, with OTP, they can take a look at
| "Designing Elixir Systems with OTP" by by James Edward Gray and
| Bruce Tate, and "Functional Web Development with Elixir, OTP, and
| Phoenix" by Lance Halvorsen.
| dahrkael wrote:
| On my first try I did write it in a more OTP-y style but the
| scaling potential for this very specific flow is just not the
| same. In the end a torrent tracker is just a specialized
| database and handling the data as fast as possible is the top
| objective. That said I'll give the books a go.
| throwawaymaths wrote:
| using ets is fine.
| salviati wrote:
| If, like me, you don't know what OTP means in this context,
| here it is:
|
| OTP stands for Open Telecom Platform, although it's not that
| much about telecom anymore (it's more about software that has
| the property of telecom applications, but yeah.) If half of
| Erlang's greatness comes from its concurrency and distribution
| and the other half comes from its error handling capabilities,
| then the OTP framework is the third half of it.
|
| https://learnyousomeerlang.com/what-is-otp
| Zarathu wrote:
| Out of curiosity, what would an "OTP-first" design look like?
|
| ETS is built into OTP, so how is using ETS not "OTP-first"?
| What's wrong with using ETS? It's just an in-memory store.
|
| I looked through the code and didn't find it to be anywhere
| close to procedural in style.
| eatbitseveryday wrote:
| Trackers are not relics - they're used exclusively in private
| tracker websites. Public-access torrents would more commonly use
| DHT and PEX for discovery.
| dewey wrote:
| But also private trackers are far from popular these days. Even
| if I'm a heavy user, but I know it's a niche.
| TheJoeMan wrote:
| Could you please clarify what DHT and PEX are? I'm having trouble
| searching "tracker PEX".
| yesco wrote:
| - https://en.m.wikipedia.org/wiki/Mainline_DHT
|
| - https://en.m.wikipedia.org/wiki/Peer_exchange
| atmosx wrote:
| DHT (Distributed Hash Table) and PEX (Peer Exchange) let
| torrent clients find peers without centralised trackers. Hence,
| you don't need a central place / public tracker anymore
| Thaxll wrote:
| You still need a central server though...
| perching_aix wrote:
| Yes, they just don't track the individual torrents anymore.
| They only play a role during the initial peer discovery
| stage (bootstrapping). Peers find torrent swarms on their
| own, the bootstrap servers are excluded from all that.
| LtdJorge wrote:
| If you are connected to the DHT network, you don't. Unless
| you mean for DNS and such.
| perching_aix wrote:
| No, they mean specifically that in order to connect to
| such a network, you need to hit some specific central
| nodes first.
| LtdJorge wrote:
| The bootstrap nodes. But those don't _need_ to be
| centralized, even if they have historically been to some
| degree. There coul be millions.
| immibis wrote:
| If there are millions of bootstrap nodes, how do you find
| them?
| toast0 wrote:
| If they listen on a well known port, and there are
| millions, send out a few thousand probes to 'random' IPv4
| addresses and you'll most likely find one.
|
| If you get and keep a list of bootstrap nodes when you
| find one, then you can random select from the bootstrap
| addresses rather than all routable IPv4 addresses.
| immibis wrote:
| What's your IPv6 plan?
| toast0 wrote:
| Probing IPv6 would be pretty difficult. You could make
| some assumptions that might hold, like assume only the
| lowest /64 out of a /48 is used and ::0 or ::1 for the
| end of the address. Likely you'll still need too many
| probes to be feasible.
|
| You'd need to probe v4 space through NAT64 and exchange
| v6 addresses after that, or include a cache of viable v6
| addresses with clients. That gets you close to
| centralized service again, because how do you get the
| viable addresses to distribute with the client? Probably
| by running a supernode and dumping the list of supernodes
| into the client source every so often; but starting off
| with just that node listed.
| LtdJorge wrote:
| You can embed them in the torrent, same as the tracker.
| But the tracker has to handle all the state, while a
| bootstrap node just hooks you up with a few (or one) node
| in the DHT network. After that, it's fully decentralized.
|
| There could other ways, embedding them on SRV DNS
| records, etc. It's the same issue as with getting a DNS
| server. You could in theory get addresses of bootstrap
| nodes from your ISP through DHCP (lol, sure).
| b0a04gl wrote:
| learned elixir in a week for an interview. didn't clear it, but
| that week changed how i write code. understood state isolation
| for the first time. no shared data. fail and restart clean.
| pattern matching everywhere. structs over classes. pipes for
| everything. after that, i started writing code topdown. move
| sideeffects out. keep logic close to the data. elixir kinda
| rewired that for me.
|
| after seeing this i saw that same mindset. not flashing any big
| genservers. simplified with fast procs, raw ETS tables. simple
| flow, but still fault aware. still clean.
___________________________________________________________________
(page generated 2025-06-20 23:02 UTC)