[HN Gopher] Lightweight Kubernetes Operators with WebAssembly - ...
___________________________________________________________________
Lightweight Kubernetes Operators with WebAssembly - FOSDEM 2023
[video]
Author : mesebrec
Score : 37 points
Date : 2023-02-05 11:24 UTC (11 hours ago)
(HTM) web link (www.youtube.com)
(TXT) w3m dump (www.youtube.com)
| kincl wrote:
| If one of the main advantages is being able to suspend to disk
| why not just implement CRIU (Checkpoint/Restore In Userspace) in
| Kubernetes?
| ENALDiNHO wrote:
| MiGEL DE FARiA +1000 inscritos
| mesebrec wrote:
| If you prefer to read, this blogpost explains the same project:
| https://merlijn.sebrechts.be/blog/2022-09-05-wasm-k8s-contro...
| yjftsjthsd-h wrote:
| More interesting to me is the line in the first graph labeled
| "no isolation", which has lower memory still. I would expect
| operators to be trusted, so I don't mind the security
| implications running them without isolation, if that's what
| that really means. Is that a viable way to reduce the resource
| use even further?
| pickledish wrote:
| Thank you for the link!
|
| > Developers want to use Kubernetes in the edge, but it uses
| too much memory for most devices
|
| ...wow, I had absolutely no idea that this was a thing people
| wanted or a problem folks were out there trying to solve. No
| judgement, who am I to say with so little information whether
| it's a good idea or not, just -- what a concept eh
| Felmioytfr wrote:
| K8s doesn't need to do unlimited things.
|
| Why run docker if you can push k8s further down with perhaps
| less features?
|
| K8s could be as lightweight as nomad.
| mesebrec wrote:
| For clarification, I'm talking about devices such as home
| automation gateways, planes, and production lines.
|
| These devices often have bespoke toolchains for deploying and
| managing software on them. This is an additional barrier of
| entry: cloud teams need to learn a new toolchain to move to
| the edge. Moreover, these toolchains often need changes to
| how release management works. So naturally, the question
| arises "why can't we just use the same tools that we use in
| the cloud?"
|
| Right now, Kubernetes isn't the greatest fit for the edge.
| Distributions such as K3s and KubeEdge try to solve some of
| the problems with Kubernetes itself, but that's only half of
| the picture. A large advantage of Kubernetes is the ecosystem
| of third party operators. So that's what we're trying to
| address.
| MuffinFlavored wrote:
| What does the edge mean? I've always understood it as, you
| are most likely using cloud hosting of some sort and edge
| means "an instance of some code running in the geolocation
| closest to your customers"?
|
| I can't think of many projects I've worked on where I've come
| across that need for performance. How do you handle
| connecting to a database or cache? Do you have a bunch of
| databases shaded at the edge? Does the edge talk back to your
| database that isn't at the edge?
|
| Buzzwords: Kubernetes, WebAssembly, edge
|
| Not saying that with any negative connotation. Interested in
| learning typical every day problems that require this. I
| thought edge was for like... get static
| HTML/CSS/JS/images/video to customers as quickly as possible
| (sub 50ms)
| wrs wrote:
| I would define it as "the closest place to the end user
| that you can deploy software". In the case of a web app,
| yes, that may be a CDN point of presence. In the case of
| Walmart, there are servers in the store running business
| apps. On a farm there might be something running on a fancy
| tractor.
| MuffinFlavored wrote:
| If the servers running in-store at Walmart have to talk
| to a server hosted by Walmart that isn't in store, what's
| the benefit?
| mesebrec wrote:
| * much lower latency for most operations * the ability
| for graceful degradation when the network or cloud is
| down. You don't want the checkouts to stop working just
| because the cloud is down.
|
| If literally every action requires a round trip to the
| cloud, then edge computing isn't for you. That's not the
| case for a lot of functionality, however.
| factorialboy wrote:
| > What does the edge mean?
|
| Server with limited capabilities located near you.
| Arbortheus wrote:
| I think in this scenario it just means a localised server.
| I imagine the problem people are trying to solve is just
| wanting to manage infrastructure in modern
| GitOps/Infrastructure-as-code ways.
|
| Ultimately people want to deploy all their tech
| infrastructure in such a way that they can address the
| "treat X as cattle, not pets" problem. Kubernetes is one
| way to do this.
| azakai wrote:
| > What does the edge mean?
|
| I was confused by that too. I've seen "edge" mean certain
| things in the web space (Cloudflare workers, Fastly
| compute, etc.), but in this article it apparently means
| "IoT" (Internet of Things, tiny devices with internet
| connections).
| cpitman wrote:
| It means varying levels of "not a central data center". For
| anyone wanting to do edge, you first need to understand
| what it means to them. Sometimes the edge is distributed
| sites that still have a rack of hardware, say in factories
| or distribution centers. Sometimes it is one or two
| computers at all of your stores, but still "general
| purpose" hardware. And sometimes it is purpose built
| devices, like Point of Sales Systems or a radio on every
| train. Etc.
|
| Each form factor has different considerations, and each
| application different design patterns. For example, at a
| retailer I worked at each store had a message broker, and
| orders would be sent through that broker up to the central
| data center. If/when the store was disconnected, it could
| continue to queue up orders until the connection was
| restored. As opposed to housing everything in the data
| center, and needing to fall back to a paper process if the
| internet goes down.
| mesebrec wrote:
| For this project, I'm specifically referring to low-power
| devices sitting inside of the user's premises. For example,
| home automation gateways, factory machines, drones etc.
|
| I'm not really talking about CDNs or offerings such as
| Fastly Compute@edge.
|
| Right now, most of these devices have bespoke solutions for
| deploying and updating software. Switching those devices to
| Kubernetes allows developers to use the same tools and
| processes from the cloud.
| coredog64 wrote:
| Chik-Fil-A and Target run k8s in each store. CFA's
| experience doing so had them providing lessons-learned
| feedback to the DoD so that they could replicate it for on-
| aircraft software deployments.
| twalla wrote:
| On-aircraft k8s gives CrashLoopBackoff a whole new
| meaning.
| ENALDiNHO wrote:
| [flagged]
| pmig wrote:
| We are currently developing a Kubernetes operator in Kotlin and
| exploring methods the reduce memory footprint. Graal Native gives
| us a reduction from 400mb to less than 200mb. But seeing your
| numbers it still seems bloated.
| Felmioytfr wrote:
| Hui that sounds massive.
|
| My quarkus kotlin native image build results in 80mb last time
| I checked
| pmig wrote:
| Yes we needed to include the quarkus operator SDK. Do you use
| Graal native in production?
| politelemon wrote:
| > Why
|
| > Developers want to use Kubernetes in the edge
|
| Why?
| qbasic_forever wrote:
| Someone like Chick-fil-A has tens of thousands of cash
| registers that are really just little Linux computers. They
| want to make sure all of those registers are running their most
| up to date cash register code, has access to their internal
| Chick-fil-A data, (inventory, finances, menu, etc) and do it in
| a fault tolerant way so one store losing internet access will
| eventually recover when it gets online again.
|
| Kubernetes is perfect for this kind of scenario--Chick-fil-A HQ
| runs a k8s control plane that all of the stores and registers
| connect to as nodes and receive explicit code and other state
| to run. From a central command they can instantly update
| everything, monitor it, add/remove nodes, etc. They can do it
| all with just k8s and kubectl, they aren't bodging together
| piles of shell scripts, ansible scripts, custom tools, etc.
| Arbortheus wrote:
| I think in this scenario "edge" is a general term for a
| localised server. I imagine the problem people are trying to
| solve is just wanting to manage infrastructure in modern
| GitOps/Infrastructure-as-code ways.
|
| Ultimately people want to deploy all their tech infrastructure
| in such a way that they can address the "treat X as cattle, not
| pets" problem. No one wants to be SSH-ing into boxes to update
| container image SHAs. Kubernetes is one way to do this.
|
| You get a lot of stuff for free with k8s that you'd need to
| reinvent the wheel for otherwise.
___________________________________________________________________
(page generated 2023-02-05 23:01 UTC)