Post AMT613qsX1VPtfSyhM by yellowbeard@freespeechextremist.com
 (DIR) More posts by yellowbeard@freespeechextremist.com
 (DIR) Post #AMT4r8jxOzVeCwvzLU by yellowbeard@freespeechextremist.com
       2022-08-13T00:10:03.206944Z
       
       2 likes, 0 repeats
       
       @p have you heard of nostr? If so, what do you think about it?https://github.com/nostr-protocol/nostrScreenshot_20220812-200920.png
       
 (DIR) Post #AMT613qsX1VPtfSyhM by yellowbeard@freespeechextremist.com
       2022-08-13T00:23:03.092315Z
       
       1 likes, 0 repeats
       
       @Beekeeper @p there is an iOS client called Damus
       
 (DIR) Post #AMUPdlfGB7I34RA6wy by p@freespeechextremist.com
       2022-08-13T15:37:39.209868Z
       
       7 likes, 0 repeats
       
       @yellowbeard I don't remember if I had heard of it.  They seem to dislike ActivityPub; I am doing what I am doing with Revolver specifically to ensure that it's part of fedi while solving the problems they are trying to solve.  I don't like the protocol design, which is all there is to not like, as apparently the design isn't completed so the software isn't written.  So that's a similarity to Revolver:  you can't really run either.  I don't like the protocol design:  https://github.com/nostr-protocol/nostr#how-does-nostr-work .> There are two components: clients and relays. Each user runs a client. Anyone can run a relay.This is bad; relays start being able to identify the source of messages.> Every user is identified by a public key. Every post is signed. Every client validates these signatures.This is good.  This is what I'm doing.> Clients fetch data from relays of their choice and publish data to other relays of their choice. A relay doesn't talk to another relay, only directly to users.This two-hop method of propagation is not just a bottleneck, it's centralization where there doesn't need to be any.> A "post" can contain any kind of structured data, but the most used ones are going to find their way into the standard so all clients and relays can handle them seamlessly.The thing about Revolver is that it's not structured data; it's blocks.  You post an image, it's chunked into blocks, and those blocks propagate through the network.  You posting it just means your client has offered it:  except heuristically, there's nothing to distinguish a block you publish from a block you are passing along (and there are ways around this, simple fixes, but you get it working, then you get it working right, then you get it working fast).  You look at venti or tarsnap or even IPFS (although their code is a mess), right, and you see why you pass around blocks instead of structured data.  So a "post" from you is a block ID that you've signed, and that is also a block, it propagates the same way any other block does.  And if it's a content-addressed block, you can propagate it however you want.  You can grab the signature block from one place and then pull the rest from IPFS, or over HTTPS, or just wait for the rest of the blocks to come in to complete the tree.  (You can in fact fetch Revolver blocks from FSE.  This one is 44 bytes:  https://freespeechextremist.com/rvl/blk/b7579946017786e8d998b160080b3a9ce7790813de29bb8c76cb316b92be29dc .  You should be able to fetch that block from IPFS as well:  `ipfs block stat f1220b7579946017786e8d998b160080b3a9ce7790813de29bb8c76cb316b92be29dc`.)  Blocks are inert on their own and they're cheap to move around.> It's easy for a relay to reject large contentSee, this is what I'm talking about.  It's a problem if they can tell the *size* of the files, even.  And if your Nostr relay doesn't want big files, they can't help.  If I've got a 64MB file and 64 nodes each store 1MB worth of 8kB blocks, that's it, it's on the network.  And some of the nodes in the network will already have identical blocks the larger the network gets, because of pigeonholing or the birthday problem (however you want to look at it).  (This is why I say that IPFS's 8MB block size is too big:  it is 1/2¹⁰²⁴ as likely to have an identical 8MB block as an 8kB block.  Their network would be faster and take less space if they shrank the block size.  Revolver uses 8kB, and I haven't done the math on that, just watched venti's performance on FSE and then copied venti's default, but once the network's in heavier use, it should be pretty easy to test out other block sizes.)
       
 (DIR) Post #AMUUDUTRy96jzrvZHk by p@freespeechextremist.com
       2022-08-13T16:28:55.779073Z
       
       2 likes, 1 repeats
       
       @Beekeeper @yellowbeard > years ago, I used a client called eMule. which retrieved files this way.This is the core of bittorrent, too.  The difference there is that swarms target specific torrents.> KademliaRevolver essentially uses Kademlia for its internal protocol.> but you could get al kinds of spam and illegal stuff sent to you when people learned how to spoof stuff.Yeah, the difference here is that you might have some 8kB block that is part of some illegal thing but you don't know that; unless you have the index block, you don't even know how to decode it.  And you don't have a real reason to fetch that block unless it's needed to complete a tree that you care about (i.e., you are following someone that signs an index block that includes that block down the chain somewhere).  So you've gotta walk from the top of the tree to even know what's in that block.  Blocks that your node isn't interested in are a separate tier with a separate queue, and unless you're running a packrat server, it'll get GC'd.  (The GC is not done, but "tree structure" lends itself really well to mark-and-sweep, CAS to LRU.)
       
 (DIR) Post #AMUUlUBxWZeYy3xMga by slash@cdrom.tokyo
       2022-08-13T16:35:01.840788Z
       
       1 likes, 0 repeats
       
       @p @Beekeeper @yellowbeard It’s amazing how some tiny technical footnote to some other design like Kademlia DHT just ends up lasting for decades in all sorts of projects.  It’s a significant part of the LBRY spec too.
       
 (DIR) Post #AMUV0h0u8U7n4r2LsO by teknomunk@pl.polaris-1.work
       2022-08-13T16:37:40.881188Z
       
       1 likes, 0 repeats
       
       @p @Beekeeper @yellowbeard > unless you're running a packrat serverAre you going to make it easy to do this?
       
 (DIR) Post #AMUVbcWFoCHjqwJD28 by p@freespeechextremist.com
       2022-08-13T16:44:29.734884Z
       
       1 likes, 0 repeats
       
       @Beekeeper @yellowbeard > Yeah, the difference here is that you might have some 8kB block that is part of some illegal thing but you don't know thatCritically, like I was saying before about the birthday problem/pigeonhole principle, that 8kb may or may not overlap with legitimate data as well.  Enough data and you can start *relying* on coincidental collisions.
       
 (DIR) Post #AMUVnXHF7mNX4Ny9jc by p@freespeechextremist.com
       2022-08-13T16:46:38.939740Z
       
       1 likes, 0 repeats
       
       @slash @Beekeeper @yellowbeard It's great stuff.  i2p uses it, too.kpos.pdf
       
 (DIR) Post #AMUX0kznWvgvKtg54K by p@freespeechextremist.com
       2022-08-13T17:00:14.605935Z
       
       2 likes, 0 repeats
       
       @teknomunk @Beekeeper @yellowbeard > Are you going to make it easy to do this? Since there are going to be some giant nodes, and I am going to be running some of them, it makes my life easier if this is the case, so yes.  Additionally, I want to encourage people to do this so that it doesn't matter if I die and all my nodes disappear.  I know of three more people that plan to run large servers.It's basically a parameter.  A peer says it has some blocks slushing around, maybe you fetch them from that peer or from other peers.  If a block is signed as an index block by a key that you're interested in (e.g., an account on your machine is following someone that uses that key), or it's needed to complete a tree you are trying to complete, you definitely fetch it.  Otherwise, it's a weight:  how often do blocks from that peer turn out useful, how often does that peer agree to take your blocks or have blocks that you want (reciprocity), and how much do you want random blocks?  Turning off GC (which is still not implemented, so turning it off does nothing) and setting the random block weight to 1 gives you a packrat server, cranking GC all the way up and setting random block weight to 0 gives you a phone peer, the default is going to be somewhere between 0 and 1, and there will probably be named options for those.
       
 (DIR) Post #AMUXJTZiWXfuoDP96W by teknomunk@pl.polaris-1.work
       2022-08-13T17:03:26.407596Z
       
       1 likes, 0 repeats
       
       @p @Beekeeper @yellowbeard Awesome.Another question I have is will there be a way to control bandwidth usage, to limit to x Kbps up/down? This was one of my largest complaints with the IPFS implementation: there was no setting for bandwidth control.
       
 (DIR) Post #AMVfde2kJ4FbrI4RYu by p@freespeechextremist.com
       2022-08-14T06:11:37.376319Z
       
       1 likes, 0 repeats
       
       @teknomunk @Beekeeper @yellowbeard > will there be a way to control bandwidth usage, to limit to x Kbps up/down?There should be.  I don't have a good plan in mind yet, but I think it's doable.  If it starts requiring enough bandwidth that there's a problem, that's a good thing.  There are a few issues, right, like upstream bandwidth caps will apply to your own posts (because your posts will turn into blocks, which will take longer to propagate, so your node will stay out of sync if you take a while to upload).  It seems like it should be easy to do, but also everything seems easy until you do it.  But the big issue with that is there's not a lot of direction in which blocks IPFS wants to move around, and they're using a really basic Kademlia implementation where you say "Hey, take this block" instead of an offer/response, there's one key per node, etc.> This was one of my largest complaints with the IPFS implementation: there was no setting for bandwidth control.Their codebase is kind of a mess.  A lot of things that should be trivial to fix are completely impossible.  They can't cap the number of peers reliably, IPFS blows up routers.
       
 (DIR) Post #AMViHA2bTPxWIdyTqa by p@freespeechextremist.com
       2022-08-14T06:41:10.475776Z
       
       5 likes, 0 repeats
       
       @Beekeeper @yellowbeard > Well, all's well and good, but the question remains WHY?To eliminate jannies and to eliminate single points of failure and to stop censorship by third parties.  You remember neckbeard.xyz?  It was some work to move it to bae.st.I've been saying since before I rolled out CofeSpace that if fedi is going too kill Twitter, we need more admins running more nodes.  I've changed my mind:  we need more nodes but zero admins.I wrote extensively about the motivation.  At this point, I've spent a very long time thinking about this problem and the possible solutions, I've spent a lot of time actually writing code; the time to decide that it was a bad idea was about a year and a half ago, and the time to discuss the design was January or February.  See attached, meditate upon the Tao ( http://canonical.org/~kragen/tao-of-programming.html#book3 ).> cause if I notify Elon, you're a marked man.Tell him; he can't stop me.  (Hell, he might kick in a bit of cash if it'll speed up a thing that is designed to kill Twitter.  The man sneezes more money than would be required to finish building and to pay people to handle the frontend work that'll be required.)> I prefer being here with things as they are.I'm glad you like it here; I like it here.  I'm building a machine that will run it so that I can enjoy being here instead of running the place.  In any case, I hope to surprise and delight you, with or without your consent.> Errone looking for a bit of glory to leave their name behind on. like that Linus guy.It's not for fame, it's hubris; please don't mix up the deadly sins.  Add autism to the hubris and you get something like this approach:  "If no one else is doing this the way I think it should be done, I will do it myself."tao3.png
       
 (DIR) Post #AMVjl1ISeHVrp0C3jE by cyberspook@soc.redeyes.site
       2022-08-14T06:57:42.711393Z
       
       1 likes, 0 repeats
       
       “If no one else is doing this the way I think it should be done, I will do it myself.”I dig that.
       
 (DIR) Post #AMVnRko08dqOzP0zCq by p@freespeechextremist.com
       2022-08-14T07:39:06.848620Z
       
       0 likes, 0 repeats
       
       @Beekeeper @yellowbeard The thought did cross my mind when I saw "don't rock my boat and I won't blow your house up".
       
 (DIR) Post #AMVoxzi0Mbr8Qq9CZU by azrez@pleroma.8777.ch
       2022-08-14T07:56:06.502299Z
       
       1 likes, 0 repeats
       
       @p @Beekeeper @yellowbeard Maybe he's just a retired ATF agent
       
 (DIR) Post #AMVpd7nVC9Q1OKKbAW by epic@gleasonator.com
       2022-08-14T08:03:33.310872Z
       
       0 likes, 0 repeats
       
       When y'all are done, you can call it Usenet.
       
 (DIR) Post #AMVxX0zOOV2EAiUT20 by p@freespeechextremist.com
       2022-08-14T09:32:06.901310Z
       
       2 likes, 0 repeats
       
       @Beekeeper @yellowbeard > Peter.. Peter Parker, may I call you that or are we informal here.This is probably the least formal place on the internet, but I don't know of anyone that calls me "Peter".  "Pete" is fine.> I think what I did on eMule was you had to get the first 'index block' which would say what type of program was coming.That's not how it works here.  It's a tree structure, it's like how venti works.  You're confusing the block propagation layer with the way objects are modeled on top of the blocks; a block doesn't belong to a file:  a file is assembled from n blocks that may or may not already be present.  The bare, low-level blocks can't even be decoded without the metadata.  You don't even know how big the block itself is, because it's partially stripped and it may or may not be compressed.  Then let's say you've got it, you get How do you know whether or not it's encrypted?  How do you know it's not one half of a one-time pad?So you look at venti, right?  The big fileserver at Bell Labs had a linear increase in storage over time, but creating venti slowed it way down to logarithmic for them.  ( http://doc.cat-v.org/plan_9/4th_edition/papers/venti/ )  FSE's file storage uses venti, and I can empirically verify that the number of fortuitous collisions continues to increase.  Take their system, move the metadata out of the block header and into the index (so as to make it impossible to decode blocks unless you have the index), you have something like what I'm doing.> Firstly... you could then preview an incoming file as it was building block by blockGreat.  You can't do this here.  A similar algorithm to theirs is used in one place; this does not mean that it's even remotely similar to this filesharing program.  And look, if you're worried about that kind of thing, it should be possible to make it difficult to tell you're even running a node.  It is designed that way, and although it will be slower the fewer channels you have, it still works even if you only use HTTPS over Tor.  You think I tack on protocols for fun?  (I do tack on protocols for fun, but I don't tack them onto this piece of software without a reason.)  I tack on protocols because I want it to work from inside China's GFW.If you're comparing it to a system that is more than ten years old, it's safe to assume that I've already read up on these things because this is the sort of stuff I do for a living.  You don't have to take my word for it; there will be code.  But until there is, I'd really rather avoid having to answer and re-answer the same five or six concerns:  if it's the first thing that occurred to you, and to several other people, the first time they heard of something, then it has probably occurred to someone that has been thinking very hard about this system for years.Anyway, here's a block:  https://freespeechextremist.com/rvl/blk/843dd0ed6cfdb336e75ce220804f75142fedc5b21e1900d38b908a6c5218c47d .  The block itself is 349 bytes.  A browser doesn't know what to do with it.  I don't know what to do with it.  It is one of the 255,742 blocks stored on this server (per https://freespeechextremist.com/rvl/stats ).  Somewhere up the chain, it is assembled into something; 95% chance it is JSON, but I don't know.  That specific string of bytes might appear inside anything, though:  multiple things might use it.  One object might treat it as a compressed block, one might treat it as uncompressed data.> They already probe you as is for C P. What makes you think this is the case?  Genuinely interested; I see people sign up and try to find CP and most of the associated search queries are tied to the tripwire (and the search query changed to "woodchipper") and I get an email and their account is dead, their stupid NordVPN IP is dead, etc.  If you know anything beyond "They probe everybody", I am interested in this information; I suspect FSE isn't on the feds' radar, but I treat it like it is.> That stopped all torrent d/ling.Go get a VM in a country that doesn't care about torrents, or at least that doesn't respect US jurisdiction.  rtorrent plus rsync.  Problem solved.  They don't even cost that much:  https://cockbox.org/?r=2521 .  (Take the query parameter off if you don't want to use my shilling link.)> Peter, you have like a what? 1 meg upload on this server? cause I can't watch a high bitrate video without it sending more than 1mbps.I'm not sure what you mean, the upstream?  It's gigabit.> sounds like your trying to build a whole new system all over again.It *is* a new system.  It interoperates with the current system, forming a part of the broader system that is the fediverse.
       
 (DIR) Post #AMVyIYAyp8MMi3Zpjs by p@freespeechextremist.com
       2022-08-14T09:40:42.362239Z
       
       1 likes, 0 repeats
       
       @cyberspook It's right down all the way in the bottom of the heart of every good hacker, engineer, etc.  Software is unique in that you don't need to secure funding to build a factory, you can just write it if you've got the time.
       
 (DIR) Post #AMVyQkfMGoVJlNhGcK by p@freespeechextremist.com
       2022-08-14T09:42:11.292725Z
       
       0 likes, 0 repeats
       
       @azrez @Beekeeper @yellowbeard :nixonsmug: