Features idea for a self hosted torrent client reddit / selfhosted ---------------------------------------------------------------------- noe_rls · 2023-09-29 15:36 Hello, I am thinking of writing an open source torrent client aimed for self hosted setup. I am looking for features idea that would make it the best option for self hosted setup. What kind of features would make you switch from your existing torrent client? Thanks for the help! ---------------------------------------------------------------------- UncleEnk · 2023-09-29 15:46 deluge?? ---------------------------------------------------------------------- Realistic_Whereas249 · 2023-09-29 15:53 Porla? ---------------------------------------------------------------------- joost00719 · 2023-09-29 16:03 Docker, web ui and a web api. Make sure it also works via VPN ---------------------------------------------------------------------- noe_rls · 2023-09-29 16:04 Hey, I am not looking for a torrent client, I am gonna develop my own. I am just looking for feature ideas. For example does porla have everything you need or there is one feature that is missing? ---------------------------------------------------------------------- thekrautboy · 2023-09-29 16:08 What good would that client be when most private trackers wouldnt allow its usage? Atleast not until its been around for a while and proven to be reliable, not causing issues with their trackers and not reporting incorrect upload stats. ---------------------------------------------------------------------- noe_rls · 2023-09-29 16:11 Yea definitely, that's going be a long process. That's why I am thinking ahead of some features that will drive adoption. I don't want to build "yet an other torrent client". ---------------------------------------------------------------------- [deleted] · 2023-09-29 16:12 [deleted] ---------------------------------------------------------------------- thekrautboy · 2023-09-29 16:12 https://xkcd.com/927 ---------------------------------------------------------------------- VinceBarter · 2023-09-29 16:18 Yeah OP doesn't need to write a new one when 2-3 perfectly good ones exist ---------------------------------------------------------------------- ThePixlPirate · 2023-09-29 16:29 Pretty sure there’s nothing you could make that would convince me to switch from qbittorrent ---------------------------------------------------------------------- massively-dynamic · 2023-09-29 16:31 Check out the transmission/qbittorrent/pickyourfavorite + wireguard/openvpn docker containers. That's basically the long and short of what is needed in a homelab setting. The only feature i found myself trialling torrent clients to get working properly was auto unrar/zip. I achieved this with a post download bash script. You really need to identify what feature or set of features self hosters are missing that you could bring to market to appeal to a set of privacy focused individuals. ---------------------------------------------------------------------- noe_rls · 2023-09-29 17:11 Thanks for your feedback! ---------------------------------------------------------------------- achauv1 · 2023-09-29 17:24 - Low CPU usage, - configurable network I/O device, - full CLI (no TUI), - systemd service unit file, - upload until ratio limit, - listen to download completion event (for each file in torrent too). ---------------------------------------------------------------------- BeersTeddy · 2023-09-29 17:41 First of all it need to be bullet proof. Especially when I even thinking about getting in to private trackers one day. It must run without crashing for at least a few months while downloading / uploading thousands torrents. Ideally years without a single issue. Crash when away = HnR = ban Lightwieght. Qbit offers a lot but doesn't behave well with a lot of torrents. Also uses a lot of cpu. Transmission usess less cpu but is pretty slow in making connection. If you want to do something good for the community, then you can try to make remote GUI app for qbitorrent, similar to existing transgui for transmission. To my surprise, there is no existing remote GUI for qbit. Qbit Webui is very slow with many torrents. Obviously you can use your skills to fix glitches in qbit/tm/deluge/rtorrent which are the major clients. Let's face the reality. There is no need for another client. Even if you gonna make one and gonsn be amazing, it might take many years to get it approved on private trackers. Those are not very keen on bringing a new tool to the game. ---------------------------------------------------------------------- yakadoodle123 · 2023-09-29 17:51 Agreed. qBittorrent has been solid for me over the years. 95% of my downloads are automated so I very rarely have a need to actually interact with the torrent client directly. ---------------------------------------------------------------------- valdecircarvalho · 2023-09-29 18:31 Why reinvent the wheel???? ---------------------------------------------------------------------- thestackdev · 2023-09-29 18:51 A trigger which can send notification (like email/slack/discord) when anything finished downloading. I use torrent to download huge files which can take days, I want to get notified whenever it is done ---------------------------------------------------------------------- RecidPlayer · 2023-09-29 19:25 Make qbittorrent but let us relocate each individual file in the torrent. ---------------------------------------------------------------------- ThePixlPirate · 2023-09-29 19:35 That’s why I use radarr/sonarr/lidarr…..qbittorrent downloads the files based on categories and then the respective apps automatically move the finished files to where they should go ---------------------------------------------------------------------- noe_rls · 2023-09-29 19:37 Noted, contributing to an existing torrent client is probably more worth it. ---------------------------------------------------------------------- RecidPlayer · 2023-09-29 19:44 So, if a torrent file has NFOs, samples, scans, etc, it can move these files to another location and keep it all seeding without making a copy of the main content? ---------------------------------------------------------------------- ThePixlPirate · 2023-09-29 19:46 That I couldn’t tell you because I have no use for those kind of files…I only download music, movies, and tv shows ---------------------------------------------------------------------- RecidPlayer · 2023-09-29 19:48 Yeah, but sometimes you have no choice and they come with these files. I don't want them in my Plex folder so I move them, but I still want to seed for the bonus points on BTN. The only way I've found to move them and keep them seeding is just to have uTorrent 2.2.1 running alongside just for these torrents. It's like 20 torrents out of 700 so at least it's not common lol. ---------------------------------------------------------------------- GolemancerVekk · 2023-09-29 20:02 On Linux it's a non-issue because you can symlink the directory to another location and another name, and the BT client will keep seeding it (after a forced re-check). Unless it explicitly refuses to work with symlinks or across filesystems – some clients do that, and shoot themselves in the foot... ---------------------------------------------------------------------- EndlessHiway · 2023-09-29 20:08 Make one client that works for torrents and newsgroups. ---------------------------------------------------------------------- AlteranNox · 2023-09-29 20:19 What you want to do here is move all the torrent files to a different location. Let's say.. D:\Links\Movie Name\Movie.mkv D:\Links\Movie Name\Movie.nfo D:\Links\Movie Name\Movie.srt Then you have your Plex folder be. D:\Plex Libraries\Movies\Movie Name Then create a hard link in command prompt to make Plex think the file is in the library folder. mklink /H "D:\Plex Libraries\Movies\Movie Name" "D:\Links\Movie Name\Movie.mkv" ---------------------------------------------------------------------- RecidPlayer · 2023-09-29 20:37 Sounds promising. I'll give this a try tonight. Thanks! ---------------------------------------------------------------------- SwishNSquish · 2023-09-29 21:19 And when you delete the files from your client when you're done seeding it will move the original to the location of the hardlink, which makes cleaning house much easier. ---------------------------------------------------------------------- pwnamte · 2023-09-29 21:50 Why not make some good plugins for qbittorrent? ---------------------------------------------------------------------- noe_rls · 2023-09-29 22:10 I wanna have some fun developing my own. And there are some feature that can't be implemented as a plug-in, for example support for webrtc. ---------------------------------------------------------------------- bobbarker4444 · 2023-09-29 22:27 In general I agree but I think having competing options is good for the OSS/FOSS community. More choices for the average user and more general ideas/innovation in the sphere. Maybe OP has or will stumble upon some really useful feature that all other torrent clients will adopt ---------------------------------------------------------------------- bobbarker4444 · 2023-09-29 22:38 Qbittorrent's WebUI is literally a remote GUI ---------------------------------------------------------------------- valdecircarvalho · 2023-09-29 23:49 The average user won’t use a new software from a random dude on the internet ---------------------------------------------------------------------- rocket1420 · 2023-09-30 00:07 If your talking about a torrent fetched with the help of sonarr or radarr, yes, it can move any extension you tell it to move. I'd use hard links though. ---------------------------------------------------------------------- rocket1420 · 2023-09-30 00:09 That's not actually how hardlinks work. Like, at all. ---------------------------------------------------------------------- rocket1420 · 2023-09-30 00:13 Why though? This is a terrible solution when automated hardlinking is a thing. I'm having a hard time envisioning why anyone would ever need to do this, but I'd love to know why you do. (no /s needed, I'm always open to learning new methods). ---------------------------------------------------------------------- rocket1420 · 2023-09-30 00:15 Yeah and so is vuetorrent. ---------------------------------------------------------------------- rocket1420 · 2023-09-30 00:17 I just use unpackerr for that. ---------------------------------------------------------------------- rocket1420 · 2023-09-30 00:18 Pretty sure notifiarr will do that if you haven't found another solution. ---------------------------------------------------------------------- bobbarker4444 · 2023-09-30 00:22 Sure, until there's a reason to. If it does something that works for a certain scenario or has some feature that someone is looking for then people will use it. I don't really understand your reasoning here. Every single popular project started off as just the works of "a random dude" until more and more people started using it. ---------------------------------------------------------------------- BeersTeddy · 2023-09-30 00:44 What I mean is standalone app to access qbit. Qbit webui is extremely slow compared to transgui (fork) for tranmission. Completely different experience, miles faster. ---------------------------------------------------------------------- bobbarker4444 · 2023-09-30 00:53 Never had issues with the WebGUI.. also not sure why you think that it being slow for you means it doesn't exist lol ---------------------------------------------------------------------- RecidPlayer · 2023-09-30 05:03 Yeah, sonarr can move files and copy and rename and all that. But it definitely can't move files outside of the original torrent folder and still allow it to be seeded. ---------------------------------------------------------------------- servergeek82 · 2023-09-30 06:33 And connectivity to OPNvpn ---------------------------------------------------------------------- crackennnn · 2023-09-30 07:30 * Multi node * http/ftp download support. * Scheduler * Audio/Video preview/player * Flow editor(same as n8n) * Plugins ---------------------------------------------------------------------- ArcadesOfAntiquity · 2023-09-30 07:47 Wait... isn't... *every* torrent client self-hosted? ---------------------------------------------------------------------- noe_rls · 2023-09-30 07:49 I totally agree! Good to hear someone is thinking like me! ---------------------------------------------------------------------- GolemancerVekk · 2023-09-30 07:59 You can't hardlink across filesystems. For example say you download to SSD but move the finished files to HDD on the NAS. ---------------------------------------------------------------------- rich_sdoony · 2023-09-30 08:41 Can I ask why I should host a torrent server on my setup? My curiosity arises from the fact that I only use torrents to download OS ISOs.So I was wondering what purpose would a torrent server serve? Since I download the files directly from my main machine, the mirrors from which I download them are official. ---------------------------------------------------------------------- Professional_Yam_130 · 2023-09-30 09:21 A good mobile app or webui would probably be the thing . I used transmission for ages but now on rtorremt/flood which is kinda ok but could be better. ---------------------------------------------------------------------- msg7086 · 2023-09-30 13:34 Many famous software started from a random dude. Linux is founded by a random dude called Linus ;) ---------------------------------------------------------------------- Pretty-Ad-9898 · 2023-09-30 20:12 Focus on writing a proof of concept version first. I’ve never seen a software project ask for features before they’ve written anything… ---------------------------------------------------------------------- sohailoo · 2023-09-30 21:51 there's an option in sonarr/radarr that allow you to include extra files when importing stuff based on whatever extension you provide. hardlinking is also supported. i got around 1000 torrents seeding currently ---------------------------------------------------------------------- RecidPlayer · 2023-10-01 01:08 This worked great. Thanks! My whole end game was to keep all those junk files out of my well organized library without making two copies. It allowed exactly this :) ---------------------------------------------------------------------- EsEnZeT · 2023-10-01 01:39 God, I would love to finally see reverse DNS entries for peers in qbit WebUI 😢 ---------------------------------------------------------------------- Fus10n_R34ct0r · 2023-10-06 04:12 I'd love to get a Client that has Decent Multi-Threaded Services / IO Priorities I've noticed that many clients have I/O bottlenecks during File completion / heavy seeding. I've gotten to the point on some older hardware that I have had to force kill / restart the service to get the interface operational enough to load the interface properly 1. Thread manager - allowing adjustment of CPU / IO resources for sub-service processes 1. Low IO Priority - Seeding torrents 2. Mid IO Priority - Completed Files Mover, Taking Precedence over Seeds, Not holding up Web Interface or Stalling Seeds / Tracker keepalives 3. High Priority - New Downloads and your service. The ability to supply chunks that are in cache / ram before pulling from disk Current Usage: 1. Incomplete folder (Throw away drive), Completed folder on Storage Drive 2. File Pre-Allocation 3. API calls between multiple services 4. Completed Sub-Folder segregation per source, cleaner administration 5. Customized speed / peer limits per torrent 6. compressed Blocklist 7. Encryption 8. Download queue perhaps starting with a Fork of your favorite current service would aid in uptake - If I saw trans/qbit was approved on tracker X, and I knew yours was a stable fork, I'd be tempted to give it a go! Building something from scratch is going to be a massive time sink. this isn't something quick, and you are competing with multiple opensource projects (with much more manpower). I'd Suggest you begin forked, once your name is known, you can look at working with some additional developers on a top-down refresh ​ Hope this helps, Best of luck! ---------------------------------------------------------------------- noe_rls · 2023-10-06 06:29 Hey, thanks for the detailed answer! I agree so much with how there is room for improvement for IO & threading compared to the existing torrent client. The fact that transmission is single threaded is crazy. Yep forking an existing client might be easier to start. I just wonder how much these threading / IO issues can be tackled when the system hasn't been designed for it. ---------------------------------------------------------------------- Fus10n_R34ct0r · 2023-10-07 03:27 Thanks u/noe_rls the prompt reply! if we focus on one pain-point (of your choosing), Analyzing what or when you are noticing this issue... Below is an example of my findings, although you might have alternate ideas For Transmission, the web-UI Freezes until a completed torrent (or user sets location) is moved to the completed folder - if needing to move multi-GB files between drives, this can mean this takes minutes. on heavily utilized kit, this can cause timeouts, with the inability to query / pause / add / change the service during an effective UI outage. Looking at this from my perspective, Id look at the below as possible options: 1. *UI Cache***:** Service starts, Spawns main Service at Mid/Low IO priority, and a new Web-UI service hosting your own Front-end. on command request, queue the commands and add visual confirmation (i.e.. click pause -> Pausing... instead of seeing a stalled or timed- out page). 1. Even if the main service is dog-slow, users would be comforted with these messages, being able to queue these commands would make the user feel like more is being completed. 2. Periodic Updates before and between queued commands could be used for a a cached interface, allowing responsive browsing and queuing of further commands. 3. The main services disk utilization and overall network data can be queried outside the process, allowing up-to-date information of the MB/s with file details and Overall Network Utilization. the Per- torrent network can either stall in a unique colour, or provide a comfort message 4. drop existing webui from main service to free memory / load for yours 2. *Redirect the Completed move call* to a broker/alternate process, that queues these copies with a different priority. 1. Provide result instantly back to main service, unblocking I/Os for downloads / seeds, Pause torrent to prevent file error. other torrents can continue downloading / seeding - Possibly set a temporary bandwidth limit on downloads? 2. Low priority copy operation entered into queue, torrent resumes on move completion The Benefits I can see from this method: 1. Day 0 provides a *stable starting point* \- You could technically provide this fork to someone and run it without issues (albeit with little point) 2. You can *focus on what you believe is the PRIMARY importance*, Only needing to debug your small section of the code. - especially useful if your doing this as a soul developer 3. Depending on what you are accomplishing, you'd mainly *inherit the stability* coming from Years of development of the Pre-Forked Service 4. Possibility of *Redirecting the commands piece by piece* until the broker is only calling your own code with Previous forked commands / services being fully swapped out