[HN Gopher] UnifiedPush: A decentralized, open-source push notif...
___________________________________________________________________
UnifiedPush: A decentralized, open-source push notification
protocol
Author : mikece
Score : 452 points
Date : 2022-12-22 15:45 UTC (7 hours ago)
(HTM) web link (f-droid.org)
(TXT) w3m dump (f-droid.org)
| nightpool wrote:
| Why use a new custom-built protocol instead of adding support for
| the Web Push protocol already supported by Firefox, Chrome,
| Safari and others? https://web.dev/push-notifications-web-push-
| protocol/. Now backend developers have Yet Another Push
| Notifications Protocol to build support for, instead of being
| able to just easily plug into their existing code. As a benefit,
| since all notifications through the Web Push protocol are end-to-
| end encrypted, there's no concern about "trusting" the
| distributing server like there seems to be in this protocol
| (based on https://unifiedpush.org/spec/server/)
| dijit wrote:
| This isn't just for notifications to clients themselves, this
| would handle the _federation_ of notification 's.
|
| Similar to how Apples push notification service works already
| (APN: https://en.wikipedia.org/wiki/Apple_Push_Notification_ser
| vic...)
| nightpool wrote:
| Yes, the WebPush standard provides protocols for both. This
| is necessary because otherwise there wouldn't be an
| intercompatible way for backends to _send_ notifications to
| different browsers. The protocol for the federation of
| notifications would be the most useful for other non-JS
| projects to reuse, so that backend developers can plug any
| native app in their existing webpush support without coding
| yet another system.
| mananaysiempre wrote:
| I think GP got it backwards, actually: Web Push seems to
| have a wire interface for submission and a JavaScript API
| for subscription, while the internals of how the
| notifications get from e.g. the FCM backend to the
| notification drawer on your Android phone remain
| unspecified (and partly secret). UnifiedPush has a wire
| interface for _both_ submission and subscription+reception,
| plus an Android(?) intent API for interacting with the
| client for the latter that's running on your phone.
|
| It looks like the submission side of UnifiedPush could
| indeed have been Web Push-compatible but isn't, though.
| MayeulC wrote:
| > _It looks like the submission side of UnifiedPush could
| indeed have been Web Push-compatible but isn't, though._
|
| Exactly. It was discussed a bit early on whether to
| mandate encryption, but simplicity was chosen instead.
|
| Adjusting the server protocol to make it compatible with
| WebPush is also being discussed:
| https://github.com/UnifiedPush/wishlist/issues/15
|
| That would be useful for supporting Telegram, for
| instance. I don't really see it becoming mandatory,
| however.
| nightpool wrote:
| > while the internals of how the notifications get from
| e.g. the FCM backend to the notification drawer on your
| Android phone remain unspecified (and partly secret)
|
| Actually, the web push standard strongly recommends that
| you use RFC 8030 for this, but it does allow browsers to
| substitute other protocols as long as their semantics are
| the same https://datatracker.ietf.org/doc/html/rfc8030. I
| believe Mozilla and Edge both use HTTP Push, I'm not sure
| what Google uses on Desktop Chrome but I could see it
| going either way. Safari probably uses APNS.
| emersion wrote:
| This is partially explained in the blog post.
|
| I'm personally using UnifiedPush HTTP endpoints as-if they were
| Web Push, and it just works. I handle decryption manually on
| the receiving end.
| karmanyaahm wrote:
| UnifiedPush is compatible with WebPush: both just involve an
| HTTP POST to a specific URL. If your application server
| supports WebPush, it can send encrypted notifications to
| UnifiedPush servers (that is how we're planning to support
| Telegram, for example). You can then decrypt these
| notifications in the app [1]. However, some protocols only send
| a wake-up ping or random ID in the push notification, Trifa and
| Matrix respectively. There, encryption is unnecessary
| complexity.
|
| However, we don't use the WebPush API between the _push server_
| and _distributor_ since there is a lot of scope for innovation
| in that space (for example Google 's FCM uses a custom XMPP
| based protocol).
|
| [1] One example, https://github.com/UnifiedPush/dart-webpush-
| encryption
| kevincox wrote:
| > There, encryption is unnecessary complexity.
|
| If you are sending a small amount of data it is likely a tiny
| overhead for some notable benefits.
|
| 1. If all data is encrypted then no data stands out for being
| encrypted.
|
| 2. It provides integrity which may prevent push servers from
| changing message to be invalid or triggering parsing bugs in
| applications.
|
| 3. If only encrypted messages are supported it is impossible
| for uses which do need encryption to forget to turn it on or
| accidentally turn it off.
|
| For many apps the benefit here is indeed tiny, but so is the
| cost. So I think it is best to not support unencrypted
| messages altogether.
| karmanyaahm wrote:
| I agree that mandatory encryption would massively help
| privacy, integrity and even compliance [1]. However, the
| main cost is developer-effort involved in keeping track of
| keys and implementing encryption.
|
| As a small project, ease of adoption has been our most
| important goal. However, as we work towards better WebPush
| compatibility [2], I personally _absolutely want to promote
| RFC8291 encryption_ and make it easy to implement for
| developers using UnifiedPush, hopefully making it the norm.
|
| [1] https://volkerkrause.eu/2022/11/12/kde-unifiedpush-
| push-noti...
|
| [2] https://github.com/UnifiedPush/wishlist/issues/15
| nightpool wrote:
| Making it the "norm" isn't good enough, because
| developers will always want to cut corners to ship,
| either based on pressure from management or (in open
| source projects) just not feeling comfortable with
| encryption or thinking that it's important enough to
| implement. Web push libraries are already widely
| available for major languages to handle implementing
| encryption, and keeping track of keys isn't any harder
| than keeping track of device IDs or endpoints would be
| without encryption--it's just one more row in your
| database.
| kevincox wrote:
| WebPush doesn't really have keys to keep track of. Other
| than the VAPID signature (which is optional but may be
| enforced by the push service) all of the keys are
| ephemeral. They are used for just one subscription so
| storing them is just as easy as storing the rest of the
| notification URL.
| ohbtvz wrote:
| > However, the main cost is developer-effort involved in
| keeping track of keys and implementing encryption.
|
| As soon as you need encryption in at least one place in
| your software, it's going to happen.
| MayeulC wrote:
| I agree, though it of course it depends on the threat
| model. As a self hoster, encryption is pretty much
| irrelevant, as my push server is controlled by myself, and
| accessible with https. Not having encryption makes it easy
| to debug and experiment.
|
| I agree encryption should be the default choice though.
| Maybe we can introduce it in a future protocol revision?
| Client side, I expect most people use libraries, making
| this easy. Server side? Not so much.
|
| The main goal was to get it adopted as widely as possible
| though, and encryption certainly seemed like something that
| would slow the effort.
| kevincox wrote:
| Even for self-hosted I'd rather trust the server as
| little as necessary. Just in case it is completed why let
| it see things that it doesn't need to see?
| nightpool wrote:
| > However, some protocols only send a wake-up ping or random
| ID in the push notification, Trifa and Matrix respectively
|
| While I understand that some developers may think of security
| as wasteful, having unencrypted data be the default relies on
| every developer to spend much more time considering the
| possible threat models and how . Case in point here is Matrix
| --the IDs they send aren't random at all, they actually send
| full room IDs and event IDs for each message, so you'd be
| able to build up someone's entire social graph just from
| reading their unencrypted push notification data and looking
| up room IDs on the Matrix server in question or even
| correlating room IDs across different Element installs. This
| also hurts users, since they're not in a position to inspect
| the source code and understand how or whether they need to
| trust their "push provider" with whatever data the app
| developers might have chosen to regard as "less sensitive"--
| WebPush's encryption-by-default design removes trust from the
| equation entirely and prevents of these sorts of "passive
| surveillance" attacks (except those based on timing or
| message length)
|
| > You can then decrypt these notifications in the app
|
| This pushes all of the complexity of supporting encryption
| onto the app developer, instead of having it be provided by
| the platform where it can be centrally audited and securely
| managed.
|
| > However, we don't use the WebPush API between the push
| server and distributor since there is a lot of scope for
| innovation in that space (for example Google's FCM uses a
| custom XMPP based protocol).
|
| Why not do this innovation in an open standards body like the
| IESG where other stakeholders can give feedback and review?
| The web push standard does allow for Push Servers to use
| other custom wire protocols besides the default RFC8030, but
| having open and standards-based discussion of the possible
| performance improvements would "lift all boats" in terms of
| being able to provide low-power devices with timely push
| notifications
| Andrew_nenakhov wrote:
| Oh gosh, a wheel reinvented again? XMPP did it ~7 years ago [0]
| and we have used it for quite a few applications already.
|
| [0]: https://xmpp.org/extensions/xep-0357.html
| binwiederhier wrote:
| Please be nice. You can convey your information without the
| snarky tone. You may be factually correct, but your comment
| comes across quite rude.
|
| From the HN guidelines [1]:
|
| > Be kind. Don't be snarky. Have curious conversation; don't
| cross-examine. Please don't fulminate. Please don't sneer,
| including at the rest of the community. Edit out swipes.
|
| [1] https://news.ycombinator.com/newsguidelines.html
| kirimita wrote:
| From the blog post:
|
| > One key feature of UnifiedPush is that the communication
| between the push server and the distributor is not specified.
| This means that a variety of technologies can be employed, such
| as WebSockets, Server-Sent Events, *XMPP*, raw TCP, or even
| SMS, whatever works best for the user.
| Andrew_nenakhov wrote:
| Sigh. What you miss is that this protocol introduces
| _nothing_ that already isn 't done by XMPP push
| notifications. It is decentralized, it can deliver to
| different endpoints, etc. You only need to create a
| dispatcher app that will relay the notifications to third
| party apps. Again, this too was done years ago.
| kirimita wrote:
| > You only need to create a dispatcher app that will relay
| the notifications to third party apps.
|
| That part is what specify UnifiedPush. It does _not_
| specify the communication between the push server and the
| distributor.
| jacob019 wrote:
| So what! Few use that. I host a prosody server with lots of
| plugins and it never occurred to me to use it, let alone
| use cases outside if the XMPP ecosystem. Can the XMPP
| implementation fall back to google's push system. Does it
| have a blog entry, lovely diagrams, comprehensive docs, a
| nice on-boarding experience for developers? Ideas aren't
| worth much, implementation--getting things done is what
| matters.
| MayeulC wrote:
| It seems like you have misunderstood something.
|
| I just read the specification you linked. If I understand it
| right, it just specifies what we call a "Push gateway" in our
| last diagram in the "under the hood" section.
|
| As in, it specifies how multiple servers can connect to a
| single (XMPP-speaking) push server. This is also evident from
| the excerpt I'm quoting: >XMPP Push works
| between the user's XMPP server and two push notification
| services in tandem: > The user's XMPP server
| publishes notifications to the XMPP Push Service of each of the
| user's client applications. > The XMPP Push Service
| (as defined here) for a client application then delivers the
| notification to a third-party notification delivery service.
| > The third-party (and potentially proprietary or platform-
| dependent) push service delivers the notification from the
| client application's backend service to the user's device.
|
| UnifiedPush is the _third-party push (or notification delivery)
| service_ here. Please make an effort to come across as less
| antagonizing in the future.
| mgaunard wrote:
| There are two things I still don't get about software
| development:
|
| - why is web development such a big thing
|
| - why is app development a thing at all
| Alifatisk wrote:
| I believe web dev is huge because it's cross-platform by
| default & easy to get started with.
| solarkraft wrote:
| Apps draining your battery to check for notifications is the
| worst thing about degoogled Android. With notifications enabled
| my telegram client can drain my battery in a very low number of
| hours (got suggestions here? any client supporting UnifiedPush?)
|
| So this is welcome, but it requires application support. And if
| apps on your phone use both GCM _and_ UnifiedPush you still end
| up with multiple providers. But at least I think I can trust
| UnifiedPush to be light on my battery.
| binwiederhier wrote:
| It's really not that bad. It uses 1% of battery for 17h.
| Sometimes even less than that and it shows 0% after the entire
| day.
|
| So it's a hypothetical problem IMHO. There are very few cases
| or phones i see these days where people complain about battery:
| https://docs.ntfy.sh/faq/#how-much-battery-does-the-android-...
| karmanyaahm wrote:
| > telegram client can drain my battery in a very low number of
| hours (got suggestions here? any client supporting
| UnifiedPush?)
|
| It's possible. There is nothing ready right now.
|
| [1] https://github.com/UnifiedPush/wishlist/issues/18
|
| [2] https://github.com/Telegram-FOSS-Team/Telegram-
| FOSS/issues/5...
| sohkamyung wrote:
| Interesting. This could have interest from China, since they have
| to use Android without the Google push service, causing battery
| drain issues. They are trying to come up with their own push
| notification service.
|
| The issue is highlighted in this article [1].
|
| [1] https://www.sixthtone.com/news/1011308/the-real-story-
| behind...
| w4eg324g wrote:
| Why is it named decentralized when it uses a central server to
| route/distribute the messages?
| eternityforest wrote:
| Looks like a cool solution to one of the biggest issues in FOSS
| on Android.
| [deleted]
| mixtur2021 wrote:
| AFAIK Modern Android (>= 8.0) reserves the right to kill your app
| when running in the background unless there's a foreground toast
| bound (even then, can still be killed in low memory situations).
| How does this solution work around that? Is there a need to opt
| into the legacy "battery saver disabled" mode for the app which
| more closely mirrors <= 7.0 execution model?
| karmanyaahm wrote:
| Most distributors both show a foreground notification at all
| times and require exempting them from battery optimization.
| Because of its efficient design, something like ntfy only uses
| a few percent of battery per day [1].
|
| System-level integration into Android ROMs would eliminate the
| need for those things.
|
| [1] https://docs.ntfy.sh/faq/?h=battery#how-much-battery-does-
| th...
| mixtur2021 wrote:
| Thanks! As I thought. Long lived sticky foreground
| notifications tend to annoy folks from my experience. Have
| you seen complaints?
| MayeulC wrote:
| I haven't seen any complaints, but folks who set this up
| are usually used to having persistent notifications (I
| currently have 6) without Google services.
|
| At least I replaced the one from my Matrix client with the
| UnifiedPush-enabled ntfy. Hopefully it'll replace Signal
| too at some point. KDE Connect? Why not. K-9 mail? I hope,
| but that would require JMAP or IMAP adjustments.
|
| One option would be to install the distributor as a system
| app.
| binwiederhier wrote:
| You can easily remove them. There are instructions here:
| https://docs.ntfy.sh/subscribe/phone/#instant-delivery
| MayeulC wrote:
| Wow, that was submitted here quicker than we could do it!
|
| UnifiedPush has been a thing for a while, see the official
| website: https://unifiedpush.org/
|
| With this blog post, we+ tried to clarify a few names. Feel free
| to point out things that are still unclear, the documentation is
| pretty much a WIP.
|
| UnifiedPush aims at replacing the push notifications mechanism
| provided by Google services with something independent, that
| anyone can self-host or any OS can provide without compatibility
| issues. It does require small adjustments server-side and client-
| side for applications assuming Google services.
|
| There is also a Matrix room at #unifiedpush:matrix.org and a
| Mastodon account at https://fosstodon.org/@unifiedpush
|
| Note: we'll be monitoring this topic for a while, you don't have
| to reply to _this_ thread.
|
| + _karmanyaahm and S1m are main authors_
| dncornholio wrote:
| Does the user require to install a separate app to receive
| UnifiedPush messages?
| MayeulC wrote:
| Yes, they need to have a UnifiedPush distributor on their
| device, that acts like Google services.
|
| OS makers could install a default UnifiedPush distributor
| though.
|
| Lastly, the Embedded FCM UnifiedPush library (to be embeded
| into a client app) handles UnifiedPush over FCM (Google-
| provided push notifications), so that a server only has to
| handle the UnifiedPush API (with a rewrite proxy for
| UP->FCM).
|
| https://unifiedpush.org/developers/embedded_fcm/
| ehutch79 wrote:
| We can't get users to understand that the ceo isn't sending
| them mail from giftcards563@gmail.com to go and buy
| giftcards, and not to ask any questions.
|
| What's the actual likely hood that the average end user is
| going to install a random service?
| karmanyaahm wrote:
| > What's the actual likely hood that the average end user
| is going to install a random service?
|
| For now, UnifiedPush is targeted towards De-Googlers, self-
| hosters and FOSS enthusiasts, not average users.
| ajvs wrote:
| Thanks for your great work creating this protocol. Hope more
| app developers add support for it, especially for privacy
| focused apps like Signal which by relying on Google servers are
| leaking metadata.
| EGreg wrote:
| If you deliver notifications, you are _always_ leaking
| metadata around timing.
|
| A police unit parked outside a guy's house and confirmed it
| was him in the real time chatroom, by cutting his internet
| and seeing him drop off.
|
| If you want real anonymity on the Internet, never use push
| notifications. Always "pick up your mail" periodically from
| random endpoints on the Web.
| michaelmior wrote:
| In theory, if you were worried about this, I think you
| could get around that problem by just constantly send fixed
| sized encrypted messages large enough to hold any potential
| notifications. When a notification needs to get sent, you
| add it to the next outgoing encrypted message. Of course,
| now the delay for notifications is now coupled to the rate
| of these messages being sent.
| EGreg wrote:
| The messages can still be traced to the client picking
| them up. Cut their networking and the client doesn't
| receive the messages anymore... this shows up as a
| bounced message.
|
| You can accumulate encrypted messages at random urls
| instead
| michaelmior wrote:
| I guess I initially misunderstood what you meant by
| timing, but that makes sense.
| dismalpedigree wrote:
| This would be an interesting protocol. Almost like spread
| spectrum or frequency hopping in the radio space.
|
| Have many servers who just accumulate encrypted packets and
| then only the host knows how to reassemble them in order
| and decrypt.
|
| Could probably be somewhat realtime if executed correctly.
| MayeulC wrote:
| Regarding Signal, I don't think they have been approached
| yet... https://github.com/UnifiedPush/wishlist/issues/8
|
| I think the consensus was that we wanted to wait for
| UnifiedPush to be more mature before approaching them, but
| it's probably about time. S1m made an interesting
| implementation for the (compatible) Signal fork Molly:
| https://github.com/mollyim/mollyim-android/pull/152
|
| The implementation is quite interesting: it adds a linked
| device that does not receive encryption keys, but sends push
| notifications to the client.
| seizethegdgap wrote:
| The team has been working to get UP support added to Molly, a
| fork of Signal for Android.
|
| https://molly.im/
|
| https://github.com/mollyim/mollyim-android/pull/152
| EGreg wrote:
| Sorry if I'm not understanding something but...
|
| Doesn't Apple and Google on the OS level only allow push
| notifications to come from its own services? Like, they look
| for the signature to be compatible with https certificate etc.
| Otherwise the phone could have a ton of push notifications
| incoming from anywhere on the Internet. The APN service for
| instance would only route notifications to a phone if it came
| from a registered website. How do you get around this?
|
| As far as I know, even for VOIP permissions, iOS only wakes
| your code up if ITS APNS SERVICE sends the notification. How
| are you able to install a long-lived application in the
| background on iOS?
|
| Similarly, the phone radio is listening for calls from the
| nearby cell towers in the network, not from absolutely any
| tower. I guess a Stingray can impersonate one, but they would
| probably need to fake the attestation, right?
| sandos wrote:
| I believe the notifications used to be delivered by a very
| long-lived TCP connection, and there probably is no limit to
| those on Android.
|
| The reason to use the google services is mostly touted as for
| battery savings, I believe.
| teknopaul wrote:
| Afaik Google and Apple just discount their own poller. It
| still has to wake up and check periodically.
| est31 wrote:
| They have special deals with the telcos that TCP
| connections to their servers can have longer TTLs. Plus
| of course, even if you do polling, if you have 5
| different apps that connect every 10 minutes to their own
| respective services with different offsets, you have the
| device waking up every 2 minutes, with a potential
| latency of up to 10 minutes for each individual service.
| If it uses a central push service, you can set it to 5
| minutes, the device would therefore wake up less than
| half of the time, and maximum latency would be 5 minutes,
| not 10.
|
| Thus, it makes total sense to have one mechanism for push
| messages for the entire device. It's great to have
| degoogled alternatives here.
| teknopaul wrote:
| Afaik Push is really
|
| Wake up one thing,
|
| Check one place,
|
| If (message) wake up correct app
| EGreg wrote:
| The wake up one thing is the issue
|
| Plus the real time push works differently... like a
| phonecall comes in and your phone rings thay same second
| lern_too_spel wrote:
| Congrats! It's always surprised me that Amazon or one of the
| other Android vendors didn't do this themselves to make porting
| to FireOS or other services-included Android distributions
| easier, but I hope you get funding from them for doing their
| job for them.
| MayeulC wrote:
| Everyone wants to be Google, and I bet most managers see
| lock-in to their platform as a good thing, and ease of
| porting to competitors a bad thing.
|
| At least, if they feel big enough to gain initial traction...
|
| Huawei is pushing their own thing, alternative to
| Firebase[1]; Amazon as well[2].
|
| There's a dozen services that provide cloud-based services to
| send push notifications to various services. I think the
| incentive to gain subscription revenue was greater than the
| opportunity to make these partially redundant.
|
| I'm not sure why it took so long for something like
| UnifiedPush to appear. It took a fair bit of debugging, but
| ultimately, it was mostly up to one dedicated individual
| (S1m) with Android experience.
|
| Before UnifiedPush, there was OpenPush[3], but it never
| materialized.
|
| [1]: https://developer.huawei.com/consumer/en/hms/huawei-
| pushkit
|
| [2]: https://docs.aws.amazon.com/sns/latest/dg/sns-mobile-
| applica...
|
| [3]: https://bubu1.eu/openpush/
| RainaRelanah wrote:
| I would imagine Huawei is doing that moreso due to
| sanctions, no?
| ilyt wrote:
| Uh, the site doesn't load...
| MayeulC wrote:
| Looks like IPv6 is broken :/
|
| The AAAA record was temporarily removed to try to mitigate.
| satvikpendem wrote:
| I'm using Google Firebase Cloud Messaging [0] which allows
| pushing notifications to devices even if they're asleep, unlike
| local notification services where the device must be awake and
| the app must be active.
|
| Will this service allow the same thing FCM does, including waking
| up the device? I'm looking at both iOS and Android devices, I see
| there's Flutter available but not iOS necessarily in the docs
| [1]. Related question, will the Flutter version also work for iOS
| devices? I know FCM (and its Flutter SDK) works for both, as well
| as web.
|
| [0] https://firebase.google.com/products/cloud-messaging
|
| [1] https://unifiedpush.org/developers/intro/
| karmanyaahm wrote:
| Yes, this is an FCM replacement on Android (and it can easily
| fall back to FCM if the user doesn't have UnifiedPush). There
| is no native UnifiedPush on iOS because Apple doesn't allow
| background services. The server side to support both is simple,
| but on the client you would need code for both FCM (for iOS)
| and UP (for Android). It would be interesting to build an FCM
| wrapper for iOS into the UP Flutter library.
| satvikpendem wrote:
| I don't know too much about this, but is this not APNs [0]? I
| see there is a FCM integration (which is how I assume FCM
| does it anyway) [1] as well as a standalone Flutter
| implementation [2].
|
| How would iOS native only apps on Swift do push
| notifications, surely Apple has their own service and
| everyone doesn't use Firebase?
|
| [0] https://developer.apple.com/documentation/usernotificatio
| ns/...
|
| [1] https://firebase.flutter.dev/docs/messaging/apple-
| integratio...
|
| [2] https://pub.dev/packages/flutter_apns
| karmanyaahm wrote:
| I recommended FCM for iOS instead of APNS because OP is
| already using it. For iOS clients, FCM just forwards
| notifications to APNS.
| [deleted]
| seydor wrote:
| I wish we could extend SMTP to support notifications as self-
| expiring emails
| gsich wrote:
| Notifications would work (and does), maybe a sieve rule for
| deleting old messages?
| MayeulC wrote:
| It would also be nice to extend IMAP to support UnifiedPush...
|
| The use-case would be new mail notifications; it wouldn't be
| too hard to do, and would be quite useful, as I typically have
| multiple e-mail accounts.
|
| You could however build a UnifiedPush-compliant SMTP-based or
| IMAP-based distributor.
| karmanyaahm wrote:
| JMAP has a push implementation I really like, and it's
| natively compatible with UnifiedPush. Would be nice to see
| more adoption there.
|
| https://jmap.io/spec-core.html#push
| janosdebugs wrote:
| This is a nice concept, but I owned more than one Android phone
| that would ignore the battery optimization settings and kill the
| applications anyway.
| binwiederhier wrote:
| I have been working and running ntfy [1] on my phone for a year
| now (ntfy is a UnifiedPush distributor), and it is true that
| Android does kill the app every now and then. But it is
| instantly restarted. Usually ntfy delivers notifications much
| much much faster than Google's FCM, especially in doze mode.
| FCM seems to try and conserve battery much more.
|
| Disclaimer: I am the maintainer of ntfy.
|
| [1] https://github.com/binwiederhier/ntfy
| afandian wrote:
| The headline got my attention. I'm looking for a standard for
| subscribeable notifications / callbacks to update users of data
| changes in an API. Bonus point for federated. But server to
| server, not server to client. Double bonus points for connecting
| to consumer services like Zapier or IFTTT.
|
| I'm planning to use WebSub and/or RSS, but on the look out for
| others. Any pointers?
| MayeulC wrote:
| That use-case sounds a bit similar to Webmentions:
| https://en.wikipedia.org/wiki/Webmention
|
| See also... The See also section in that article :)
|
| You say it would be server to server, but are both publicly
| reachable?
| afandian wrote:
| We're running an API, we want to say to users e.g. "we
| finished processing your submission" or "this API resource
| changed". And offer pull (e.g. RSS) and push flavour.
|
| > You say it would be server to server, but are both publicly
| reachable?
|
| So yes, the user would have to run a public server. But only
| for practical reasons, otherwise how can you push.
|
| Webmention isn't quite it, we're notifying about our own API
| Resources.
|
| Websub (FKA PubSubHubub) does. It just doesn't seem to be
| widely adopted for some reason.
| kevincox wrote:
| WebSub + RSS/Atom is a good option for relatively slowly
| changing data. (Maybe at most a couple of times an hour). It is
| nice because it is opt-in and an enhancement on the regular
| polling API.
|
| Using WebSub or UnifiedPush would technically very similar but
| you need a way for the client to pass you the subscription
| information. WebSub has a standard discovery process to manage
| this.
| afandian wrote:
| The WebSub spec looks disarmingly simple but also complete
| for everything I can think of. I've not yet tried to
| implement it, but I'm waiting for the catch! Thanks for the
| confirmation.
| rgovostes wrote:
| Off-topic: Sometimes I work on ships that have Wi-Fi for talking
| to local services, but due to strict metering, Internet access is
| usually disabled. So most of the time there's no way to get
| notifications from monitoring systems and your fellow crew, let
| alone shoreside team members.
|
| It would be nice to have a solution for delivering pushes
| internally, and also hold open a best-effort connection to a
| remote server that allows some notifications through.
|
| The last solution I considered was running a Matrix homeserver on
| the ship with some local channels, and another instance on shore,
| and using federation to connect them when the uplink is
| available. Monitoring systems could send their alerts as DMs or
| to a shared channel.
|
| Then for the notifications, Apple has its Local Push Connectivity
| API you might be able to use to send notifications within the
| ship network. I'm not sure if this UnifiedPush would solve the
| problem on Android. Either way, though, it seems like a lot of
| engineering effort just for a chatroom.
| karmanyaahm wrote:
| One of the first uses of UnifiedPush was to self-host the whole
| Matrix messaging stack. In fact, we have already had someone in
| the UnifiedPush chat set this up on their ship [1]. Matrix +
| UnifiedPush would be perfect for a disconnected situation on
| ships (or Mars colonies :)
|
| [1]
| https://matrix.to/#/!vwmBiTqilorqNCbGab:matrix.org/$hJpXxhVR...
| wereallterrrist wrote:
| I'm a huge fan of Matrix and the idea of UnifiedPush, but I
| don't quite grok what this means (and I'm reluctant to fire
| up my Matrix client because I'm avoiding some things). Is
| there any chance you can elaborate on what "self-host the
| whole Matrix stack" means?
| eternalban wrote:
| I read the thread just now: ~" Self hosted matrix + UP.
| Nothing special there. Only issue is cert expiration for
| setups without connections that result in expired certs for
| https. That requires some work. "~
| yamtaddle wrote:
| > It would be nice to have a solution for delivering pushes
| internally, and also hold open a best-effort connection to a
| remote server that allows some notifications through.
|
| ... email?
| binwiederhier wrote:
| ntfy [1] would work for you entirely in the LAN, if you self-
| host the server and the phones are connected to the same Wifi.
| It'll only work for Android phones though, since iOS forces
| APNS for push notifications.
|
| ntfy is also a distributor for UnifiedPush, so you're not
| entirely off topic here. Hehe.
|
| Disclaimer: I am the maintainer of ntfy.
|
| [1] https://github.com/binwiederhier/ntfy
| rgovostes wrote:
| Interesting thanks. Have you looked into Apple's Local Push
| Connectivity API? It's specifically designed for VoIP calls
| and text messages, but perhaps the latter is enough. Not sure
| if it requires you to bake in any keys at compile time, or if
| the iOS app could allow the end-user to configure the push
| server.
|
| https://developer.apple.com/documentation/networkextension/l.
| ..
| pmontra wrote:
| You can self host a Gotify server (single file executable, web
| interface), install the app, set it to connect to the server on
| the local network. Sending a message is as easy as a curl to
| the server. It will show up in the apps.
|
| https://gotify.net/
|
| https://github.com/gotify/android
|
| The problem is that there is no iOS app. You could use the
| browser there but maybe ntfy would be a better fit (Android +
| iOS.)
| MayeulC wrote:
| In addition to what the others wrote (a local Matrix server and
| UnifiedPush provider such as ntfy would work fine), I suggest
| you look into p2p messaging systems such as Briar. I think Tox
| is also p2p?
|
| Lastly, Secure Scuttlebutt was actually designed on a boat for
| sporadic internet connection and local connections, as well as
| sneakernet, so that would work too, but I don't think it's
| designed for IM. Perfect for blogs or facebook-like stuff
| though.
|
| Have a member of the crew go onshore up to a Wi-Fi AP, it will
| act as an async data transfer for everyone. You can also
| exhange data with other travellers, including passing boats.
|
| Interestingly, there are a few Matrix p2p experiments. I wish
| apple opened their Airdrop feature as that would make proximity
| networking easier, though that should be achievable with Wi-Fi
| NAN+P2P.
| inopinatus wrote:
| I've been using NNTP as a Gossip-type protocol for eventually-
| consistent pub/sub telemetry, monitoring, and notifications
| between partially-connected nodes in an unstable mesh since
| about 1995.
| anotherevan wrote:
| Any possibility this will mean I can push a notification from my
| home automation that gets displayed on my Google TV?
| MinaMe wrote:
| [flagged]
| notwokeno wrote:
| >However, if each app actively maintains a server connection, the
| OS cannot suspend them.
|
| This is a limitation with Android and not a fundamental
| network/OS limitation. As long as your protocol allows for long
| enough keep alive latency you can periodically wake up for a
| short period and allow apps to service their connections (which
| is what I'd imagine this does anyway just with a singular app.)
| This works on Linux if you don't have the extra Android crap
| getting in the way.
|
| It's kind of a bummer to see bad OS architecture feed into over-
| complicated protocol/application design.
|
| EDIT: I guess I wasn't clear here. In my experiments I had the
| background wake up service wake the device up once and
| applications had just that singular window for all of them to
| service their connections. The problem with Android is instead
| applications either keep the phone from suspending entirely or
| (apparently) have some API allowing them to register their own
| periodic wake up that isn't shared.
|
| EDIT2: I also want to be clear that this isn't a criticism of
| NTFY. The developers behind it deserve congratulations for
| solving a serious problem with FOSS Android apps. My complaint is
| that the poor architecture and unescisary inflexibility of
| Android made this such a complex problem in the first place.
| sandGorgon wrote:
| this is NOT a limitation of Android. Android has supported
| wakelock based notification managers for a long time. And FCM
| works pretty much the same as was mentioned in the OP. The FCM
| manager maintains one long polled connection to Google
| notification servers and every other app subscribes to
| wakelocks from that manager.
|
| Your app is only woken up by the OS if FCM receives a
| notification for you. For e.g. Xiaomi will not wake you up even
| if you get a notification, unless you're on a white list (the
| much glorified Xiaomi whitelist). When Android is put in power
| saving mode, the FCM manager optimized this even further.
| jrmg wrote:
| Even if this were all synchronized, you'd still need to have
| all the apps memory state available - either taking up loads of
| RAM or requiring a lot of expensive paging.
| jitl wrote:
| > you can periodically wake up for a short period and allow
| apps to service their connections
|
| For this to work, you need apps to somehow conform to a
| standard timing and respond to a signal that says "hey, the OS
| is awake for you to do network things". Now each developer
| needs to implement some special code that understands the
| situation. After a while, all the devs start using
| libSleepyNetwork because getting those details right is _hard_.
| The libSleepyNetwork people realize that they can use a system
| daemon to further coalesce work for greater efficiency (one
| event loop is better than N event loops), so now all the
| consumer program C APIs are now backed by IPCs to the daemon.
|
| This is exactly what's already happened on Android, except
| libSleepyNetwork is implemented by either Google, or these
| people. I don't think it's bad OS design at all. It's chasing
| efficiency via abstraction that encapsulates some complex
| coordination.
| binwiederhier wrote:
| > This is a limitation with Android and not a fundamental
| network/OS limitation. As long as your protocol allows for long
| enough keep alive latency you can periodically wake up for a
| short period and allow apps to service their connections (which
| is what I'd imagine this does anyway just with a singular app.)
|
| The protocol does not specify how the distributor and push
| server communicate or keep a connection going. It's up to them
| to define an energy efficient way to do that. For ntfy [1], a
| single WebSocket or JSON stream connection is used to deliver
| messages to the device. This is very energy efficient and
| consumes <1% of battery on most phones.
|
| As for regular wakeups (for polling), Android does not allow
| periodic work to be done more frequently than every 15 minutes,
| which is obviously not frequent enough to be useful.
|
| > It's kind of a bummer to see bad OS architecture feed into
| over-complicated protocol/application design.
|
| This is somewhat true. The restrictive Android eco-system is
| what led to the creation of UnifiedPush, but it's far from
| complicated IMHO
|
| Disclaimer: I am the maintainer of ntfy [1], one of the
| UnifiedPush distributors.
|
| [1] https://ntfy.sh
| russianGuy83829 wrote:
| could the 15 minute limit be circumvented with multiple
| identical apps (different package names) installed?
| binwiederhier wrote:
| I mean yes, but if Google catches you, you're not going to
| have a good time. It's also incredibly inconvenient.
|
| Also, you cannot specify the exact time when it's supposed
| to run, so it's likely they will just run at the same time.
| MayeulC wrote:
| You can do that, but if keep alive are not synchronized (I
| don't think there is a mechanism to do so), you will wake more
| often if you keep more connections open.
|
| I think the OS could synchronize them if keep alive packets are
| sent at the TCP level, but that would be hard to do at a higher
| level.
|
| Letting a single program handle persistent connections leaves a
| lot more potential for energy efficiency optimizations.
| londons_explore wrote:
| Google famously tried allowing app keepalives to be
| synchronised... The scheduler allowed a wakeup in
| 'approximately' X minutes, and the OS would wake up and run
| all apps callbacks at once. If any app needed a wakeup at a
| precise time, then other callbacks at nearby approximate
| times would be run at the same time too.
|
| Overall, the thought was that wakeups are expensive, wakeups
| that involve the network are particularly expensive, so might
| as well run them all at once (they tend not to be CPU bound -
| often much of the wakeup handler is spent waiting for the
| network to connect and send a packet or an ack to arrive
| back).
|
| This turned out to be a _very very_ bad idea. All it takes is
| some app to have a 'wake up exactly on the hour', and now
| all other app callbacks will also run exactly on the hour.
| End result: Mobile networks get 10 million phones all waking
| up within a few milliseconds of one another, and everyone
| wanting to connect, and the whole mobile network fails
| because it was never designed to have 10 million phones all
| trying to connect within a few milliseconds.
|
| That idea was scrapped.
| jacob019 wrote:
| Seems like adding a random deviation to wakeup times would
| solve that problem.
| _flux wrote:
| Surely we'd be at back square one at that point?
| londons_explore wrote:
| > if keep alive packets are sent at the TCP leve
|
| Unfortunately there are lots of bits of hardware in mobile
| networks that fake acks - ie. TCP level acks will be received
| very quickly to indicate the mobile network has accepted the
| data, and will deliver it at some future time out to the
| internet.
|
| The problem is they still send acks even if the remote end is
| no longer responsive.
|
| So you can't reliably use TCP acks to know if a connection is
| still alive. You need to send actual data and have the
| application respond.
| _flux wrote:
| Well that sounds utterly repulsive. And somewhat pointless.
|
| In fact, I choose not to believe it. Is there some document
| discussing this kind of functionality?
| londons_explore wrote:
| Products like this:
| https://www.enea.com/solutions/traffic-management/tcp-
| accele...
| kitsunesoba wrote:
| I would think that in order to not have an obvious negative
| impact on battery life, it would be necessary for the OS to
| coalesce app notification service keepalives, performing them
| in scheduled batches instead of the moment an app requests one.
| That should be roughly equal to the current situation with
| APNS/GCN, where the OS can schedule network maintenance during
| periods where the device and antenna(s) are likely to be awake
| already.
___________________________________________________________________
(page generated 2022-12-22 23:00 UTC)