https://github.com/lmatteis/torrent-net Skip to content Sign up * Why GitHub? Features - + Code review + Project management + Integrations + Actions + Packages + Security + Team management + Hosting + Mobile + Customer stories - + Security - * Team * Enterprise * Explore + Explore GitHub - Learn & contribute + Topics + Collections + Trending + Learning Lab + Open source guides Connect with others + Events + Community forum + GitHub Education + GitHub Stars program * Marketplace * Pricing Plans - + Compare plans + Contact Sales + Nonprofit - + Education - [ ] [search-key] * # In this repository All GitHub | Jump to | * No suggested jump to results * # In this repository All GitHub | Jump to | * # In this user All GitHub | Jump to | * # In this repository All GitHub | Jump to | Sign in Sign up {{ message }} lmatteis / torrent-net * Watch 24 * Star 594 * Fork 29 Distributed search engines using BitTorrent and SQLite Apache-2.0 License 594 stars 29 forks Star Watch * Code * Issues 2 * Pull requests 0 * Actions * Projects 0 * Security * Insights More * Code * Issues * Pull requests * Actions * Projects * Security * Insights master 1 branch 0 tags Go to file Code Clone HTTPS GitHub CLI [https://github.com/l] Use Git or checkout with SVN using the web URL. [gh repo clone lmatte] Work fast with our official CLI. Learn more. * Open with GitHub Desktop * Download ZIP Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Go back Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Go back Launching Xcode If nothing happens, download Xcode and try again. Go back Launching Visual Studio If nothing happens, download the GitHub extension for Visual Studio and try again. Go back Latest commit @lmatteis lmatteis Change name from TorrentPeek to TorrentNet ... 3d8457e Apr 24, 2017 Change name from TorrentPeek to TorrentNet 3d8457e Git stats * 12 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time public Add better readme text Apr 16, 2017 reddit-browser first commit Apr 14, 2017 .gitignore first commit Apr 14, 2017 Jamroot.jam first commit Apr 14, 2017 LICENSE first commit Apr 14, 2017 Makefile first commit Apr 14, 2017 README.md Change name from TorrentPeek to TorrentNet Apr 24, 2017 a.out first commit Apr 14, 2017 chinook.db first commit Apr 14, 2017 dllmain.cpp first commit Apr 14, 2017 index.js Add better readme text Apr 16, 2017 index2.js first commit Apr 14, 2017 kat.torrent first commit Apr 14, 2017 libboost_system.dylib first commit Apr 14, 2017 libboost_thread.dylib first commit Apr 14, 2017 libsqlite3.dylib first commit Apr 14, 2017 libsqlite3.dylib_bkp first commit Apr 14, 2017 libtorrent-rasterbar.9.dylib first commit Apr 14, 2017 out.gif Add gif Apr 14, 2017 package.json first commit Apr 14, 2017 sqlite3.h first commit Apr 14, 2017 sqlite3ext.h first commit Apr 14, 2017 sqltorrent.cpp first commit Apr 14, 2017 sqltorrent.dylib first commit Apr 14, 2017 sqltorrent.h first commit Apr 14, 2017 sqltorrent.o first commit Apr 14, 2017 sqltorrent.sln first commit Apr 14, 2017 sqltorrent.vcxproj first commit Apr 14, 2017 sqltorrent.vcxproj.filters first commit Apr 14, 2017 test_multiple.js first commit Apr 14, 2017 torrentpeak-gif.mov Add gif Apr 14, 2017 View code README.md TorrentNet Distributed search engines using BitTorrent and SQLite [out] About Distributed sites have gained much attention lately with systems such as ZeroNet and IPFS, which seem to improve on older systems like Freenet. Building search engines on top of these distributed systems is not quite feasible yet as users need to download the entire site database (usually several hundreds of gigabytes large) before running queries against it. TorrentNet allows you to build distributed search engines by leveraging the BitTorrent network and the SQLite database. Users do not have to wait until the site is fully downloaded before they can query it. How? Site owners create an SQLite database (.db file) and create a torrent from this file. They then proceed to seed this torrent -- just like they would seed any other file. Site users then start downloading the site torrent, but, rather than downloading pieces of the torrent in "rarest first" order, they download pieces based on the search query they performed. In other words, given a search query such as "indiana jones", TorrentNet knows to download pieces of the torrent where data for "indiana jones" is likely to be stored at. Hence, results are given to users in a timely manner, without having to wait until the entire database is downloaded. Furthermore, since search queries are just regular torrent piece downloads, a search query can be satisfied by many different peers in a swarm, effectively allowing for distributed search engines. Main features Distributed torrent sites, queryable on demand by prioritizing specific piece downloads. * Create distributed torrent sites accessible via your public key (eg. 33cwte8iwWn7uhtj9MKCs4q5Ax7B) that are shared and kept alive using the BitTorrent network. * Update your site using the Mutable Torrents extension (BEP46) and let your users know about site updates via the DHT network. * Build interactive sites that are queryable on demand using the sqltorrent technique. Things like search engines, or complex browsable experiences are possible. Pros & cons * Pros + Hosting your site is as simple as seeding a torrent. No need to buy a domain name or a hosting server. You can easily host your torrent site on your home network and let users visiting your site help you with the hosting. + Since you control your address (public key), which is broadcast via the DHT, it's much harder for governments and institution to block the content you're sharing. + Via sqltorrent you drive your users experience by letting them only downloading pieces of the torrent that are relevant to the users' interaction. Essentially you could create a search engine, and program interactions so that submissions of a search form result in queries to the underlying SQLite database - which prioritizes pieces based on the query. * Cons + Read only sites for now. ZeroNet seems to have a solution to this problem but I'm not convinced yet. + Your site doesn't have a pretty name. Sharing your public key can be much harder than sharing the name of a DNS site. + No standards for doing distributed sites. Many different solutions (IPFS, ZeroNet, Freenet, etc..). TorrentPeek at least follows most of the BitTorrent specifications (using the powerful libtorrent library) and provides unique SQL querying capabilities. sqltorrent By putting an SQLite database file (.db) inside a torrent, we can query its content -- by prioritizing pieces based on the SQL query -- and quickly peek at the content of the database without downloading it entirely. TorrentPeek was inspired by sqltorrent. A video is worth 1000 words: https://www.youtube.com/watch?v= EKttt8PYu5M&feature=youtu.be Install This currently only works on Mac OS X. Simply npm install and then run node index.js file.torrent ~/ save_folder/ 8080, where the last one is the port of the UI it will lunch the app in. Make sure that file.torrent contains an SQLite database or else this wont work. About Distributed search engines using BitTorrent and SQLite Resources Readme License Apache-2.0 License Releases No releases published Packages 0 No packages published Languages * C 90.8% * Objective-C 5.7% * C++ 3.3% * Other 0.2% * (c) 2021 GitHub, Inc. * Terms * Privacy * Security * Status * Help * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.