Post A6VWW1yeLicVC0FsDw by agates@podcastindex.social
(DIR) More posts by agates@podcastindex.social
(DIR) Post #A6U2VEkgAzsQjwSwUq by dave@podcastindex.social
2021-04-21T21:24:45Z
1 likes, 1 repeats
I'm still working on Hydra. But, I'm thinking that the whole solution to not polling anymore might look something like this:
(DIR) Post #A6U2VF7Moe1bsJ53rM by Economic_Hitman@noagendasocial.com
2021-04-21T21:26:35Z
0 likes, 0 repeats
@dave why is there a sacred geometry pentagram in the center?Dave, I'm worried about you...
(DIR) Post #A6U8JPJVgJpjkMV3y4 by dave@podcastindex.social
2021-04-21T21:27:00Z
0 likes, 0 repeats
Basically, the middle group of servers are housed by entities that run podcast feed directories and agree to run a "websub" (this isn't strictly websub, but let's use that term) server.
(DIR) Post #A6U8JPgYIeGUtpHSsq by dave@podcastindex.social
2021-04-21T21:28:14Z
0 likes, 0 repeats
The "entry point" for those websub servers is something like "notify.podping.cloud", and accepts a websub compatible notification request for ANY url.
(DIR) Post #A6U8JQ78hnX4EHihKC by dave@podcastindex.social
2021-04-21T21:28:59Z
0 likes, 0 repeats
For this reason, there would need to be an authentication step before podcast hosts are allowed to send the notifications.
(DIR) Post #A6U8JQYR4JMnawUUs4 by dave@podcastindex.social
2021-04-21T21:30:27Z
0 likes, 0 repeats
Any interested party that wants to be notified when a change happens can subscribe at "subscribe.podping.cloud" once every 7 days.
(DIR) Post #A6U8JQzjQpCWxbGIPw by dave@podcastindex.social
2021-04-21T21:31:20Z
0 likes, 0 repeats
Then, any time a notification comes in, it gets immediately broadcast back out to all subscribers letting them know the feed has updated.
(DIR) Post #A6U8JRR1nL2GKG25xo by dave@podcastindex.social
2021-04-21T21:32:13Z
0 likes, 0 repeats
This would also not be an open system in the traditional sense. An authorization step would be required and a token given out.
(DIR) Post #A6U8JRquF7jfcW8lIe by dave@podcastindex.social
2021-04-21T21:54:23Z
1 likes, 0 repeats
The middle part is the interesting part. Anyone can run one of these servers, and notify.podping.cloud/subscribe.podping.cloud would just be round-robin DNS'd to point to every server in the group. Providing for some resiliency. They would all relay incoming notifications to each other *after* sending out the subscription notifications.
(DIR) Post #A6U8JTweSjWI6uqNX6 by dave@podcastindex.social
2021-04-21T21:54:48Z
0 likes, 0 repeats
Well, I say "anyone", but it would be a list of trusted entities.
(DIR) Post #A6U8JVSapSfEoGnOxU by dave@podcastindex.social
2021-04-21T21:57:11Z
0 likes, 0 repeats
Basically, I'm just realizing that WebSub isn't the right way to skin this cat. It's built for the pure decentralization of open RSS. But, podcasting RSS isn't that broad. It's a smaller world. The interface/protocol of WebSub works great for this. But, the radically open nature of it is overkill for solving this one issue.
(DIR) Post #A6U8JXAEUhAM5uDlcO by dave@podcastindex.social
2021-04-21T21:58:47Z
0 likes, 0 repeats
For blogs, and the rest of RSS world, WebSub makes sense. But, for podcasting, I think the whole landscape can be mapped to something tighter, but using the same protocols. This is what we were exploring with @js's websub work, but I think I just put it together in my mind.
(DIR) Post #A6VWW1yeLicVC0FsDw by agates@podcastindex.social
2021-04-21T21:36:07Z
0 likes, 0 repeats
@dave I'm super interested in this. What's the subscription part look like? Or, how does a client subscribe?
(DIR) Post #A6VWW2PEkrt4WSh6fI by WClayFerguson@fosstodon.org
2021-04-21T21:46:10Z
0 likes, 0 repeats
@agates @dave I bet you guys both predicted I'd chime in to ask if you thought about IPFS PubSub or not?I really need to learn it myself! I haven't done it before, ...but i'm a fan of having no single point of failure or any way for someone to shut down everybody else, or decided to start requiring some Satoshis and do a little "gate keeping"
(DIR) Post #A6VWW2nhHvS9kK8dn6 by agates@podcastindex.social
2021-04-21T21:47:21Z
0 likes, 0 repeats
@WClayFerguson @dave I think IPFS pub-sub + an IPFS-distributed database should be explored, but I think it should be supplemental to another solution as a next-gen kind of process
(DIR) Post #A6VWW3FLd7ZT854itE by WClayFerguson@fosstodon.org
2021-04-21T21:50:01Z
0 likes, 0 repeats
@agates @dave My only fear about IPFS PubSub is that last I checked it was labeled "experimental" (read: "might disappear") so....there is that.
(DIR) Post #A6VWW3gdzdPCUjqWR6 by dave@podcastindex.social
2021-04-21T22:12:38Z
0 likes, 0 repeats
@WClayFerguson @agates I’m just envisioning something a lot more loosely connected than even that. Really, all that’s necessary for a notification network like this to exist is a way to map one to many. One notification comes in and a whole bunch go out.
(DIR) Post #A6VWW43gbxpxeCcvLs by agates@podcastindex.social
2021-04-21T22:15:00Z
0 likes, 0 repeats
@dave @WClayFerguson That's kind of what I've been building in my pyplexo framework, it's like pubsub-to-pubsub distribution method
(DIR) Post #A6VWW4TD54FmvMZJ8S by dave@podcastindex.social
2021-04-21T22:16:53Z
0 likes, 0 repeats
@agates @WClayFerguson Yeah, what matters is the input and output. The squishy middle can be anything that works for a loose sync.
(DIR) Post #A6VWW4n3tG8JuvrA4u by agates@podcastindex.social
2021-04-21T21:47:52Z
0 likes, 0 repeats
@WClayFerguson @dave LMFAO can you tell I'm on a business call with my verbiage?
(DIR) Post #A6VWW4tnUDWMFp0XZo by dave@podcastindex.social
2021-04-21T22:19:15Z
0 likes, 0 repeats
@agates @WClayFerguson The critical point is that the DNS never changes for hosts (I.e. clients) or subscribers. No more maintaining mappings of rel=“self” to rel=“hub” and resubscribing to 800,000 feeds on a rotating basis.
(DIR) Post #A6VWW5StNjas0fQZHM by agates@podcastindex.social
2021-04-21T22:36:00Z
0 likes, 0 repeats
@dave @WClayFergusonSo for the sake of argument (not seriously pushing for it, I generally agree with a traditional HTTP, DNS-based approach) ...Using something like OrbitDB over IPFS would allow authenticated (via cryptographic keys) updates that doesn't rely on DNS -- every trusted provider would run their own IPFS node and push new subscriptions to a named event queue.Again, maybe that's the next-gen solution, it's still all new anyway. But that's where my head's at for long-term.
(DIR) Post #A6VWW5tTmsrRL7rnii by dave@podcastindex.social
2021-04-21T23:28:55Z
0 likes, 0 repeats
@agates @WClayFerguson Not familiar with Orbit DB. It’s a shared db?
(DIR) Post #A6VWW6K4C280faJ2A4 by agates@podcastindex.social
2021-04-21T23:33:07Z
0 likes, 0 repeats
@dave @WClayFerguson Yes, shared, decentralized. Still very new of course, I see it as a roadmap item.OrbitDB is "a peer-to-peer database for the decentralised web. To achieve this decentralised nature the database is run on top of IPFS ... an OrbitDB database is serverless, and is replicated to all peers that are using it. And because of IPFS’s hashing protocols that do not allow duplication of a piece of content, there is no central point for such a database.https://rossbulat.medium.com/orbitdb-deploying-the-distributed-ipfs-database-with-react-79afa1a7fabb
(DIR) Post #A6VWW6ishlyfuXuqq8 by dave@podcastindex.social
2021-04-22T00:08:12Z
0 likes, 0 repeats
@agates @WClayFerguson @js I’m just thinking that if architected carefully, we wouldn’t even need a shared DB. Each middle layer server (lord this needs a better name) can serve the full pipeline. The incoming notification is really just a pass-through. The only sharing of data between middle servers is to make sure they all know the full current subscriber list. I bet that full subscriber list wouldn’t be more than 100 urls at any given time.
(DIR) Post #A6VWW7BF0Kf9KVBV2m by agates@podcastindex.social
2021-04-22T00:19:05Z
0 likes, 0 repeats
@dave @WClayFerguson @js I both agree and disagree. I think with your proposal and traditional protocols, you're 100% right.On the other hand, it's all data anyway and something like this standardizes the data itself. It's more like a blockchain than a database (but it's not a blockchain). The database *is* the pipeline in this scenario and clients wouldn't even query a traditional API, they would just query an IPFS node.
(DIR) Post #A6VWW7dbItLckSS9FQ by agates@podcastindex.social
2021-04-22T00:19:38Z
0 likes, 0 repeats
@dave @WClayFerguson @jsIt's not a file that gets shared -- it would replace and/or supplement all feed databases on all the servers.The need for protocols disappear because the layer of abstraction is turned into a global, immutable filesystem
(DIR) Post #A6VWW86Ja8JgBVt50K by dave@podcastindex.social
2021-04-22T00:27:59Z
0 likes, 0 repeats
@agates @WClayFerguson @js Is it possible though. With anything out there now that’s stable? If orbitdb is stable that’s good. You and clay seem a bit dubious on it though.
(DIR) Post #A6VWW8WtzHaFVyKJRg by agates@podcastindex.social
2021-04-22T00:29:16Z
0 likes, 0 repeats
@dave @WClayFerguson @js I wouldn't trust it now, no -- I'm trying to say, I like your solution and long term I think there's an even better option :)
(DIR) Post #A6VWW8xUOQqoqQlXt2 by agates@podcastindex.social
2021-04-22T00:37:39Z
0 likes, 0 repeats
@dave @WClayFerguson @jsAnd to be clear my *biggest* reason for this vision aside from decentralization is providing a good way for standalone apps to subscribe. Because otherwise they will still have to poll or rely on some other 3rd party service.
(DIR) Post #A6VWW9PUiJFiFHruXQ by dave@podcastindex.social
2021-04-22T01:48:37Z
0 likes, 0 repeats
@agates @WClayFerguson @js Yeah, I really think if we tackle this correctly we can end the whole polling thing. It's a scourge on the open podcast rss landscape.
(DIR) Post #A6VWW9pNA5x7XXyZsG by dave@podcastindex.social
2021-04-22T14:04:21Z
0 likes, 0 repeats
@agates @WClayFerguson @js I think the best way to package this initially would be as a single binary utilizing a sqlite database running in a docker container. That would be very safe and self contained. Basically, spin up a new VM and docker run with the correct config params passed in and you're off to the races.
(DIR) Post #A6VWWADph9WClPQ704 by agates@podcastindex.social
2021-04-22T14:30:52Z
1 likes, 0 repeats
@dave @WClayFerguson @js I agree, then people don't need to worry about package dependencies etc.Would want to make sure everything is configurable with environment variables for maximum flexibility (that's a small but significant detail during software dev or I wouldn't mention it)I like containers!
(DIR) Post #A6VWWB02nu5DAvyc9A by agates@podcastindex.social
2021-04-22T00:49:16Z
0 likes, 0 repeats
@dave @WClayFerguson @js I think my goal is to work on a way for apps to get real time notifications for live streams, at least once I'm ready with the feed support.I see it as a parallel effort.