[HN Gopher] Kubernetes is hard
___________________________________________________________________
Kubernetes is hard
Author : suralind
Score : 103 points
Date : 2023-03-27 20:19 UTC (2 hours ago)
(HTM) web link (rcwz.pl)
(TXT) w3m dump (rcwz.pl)
| [deleted]
| 0xbadcafebee wrote:
| > Kubernetes is complex and I think they are partially right
|
| Kubernetes is a distributed centralized operating system which
| itself depends on a distributed decentralized database, and has a
| varying network topology, permissions system, plugins, scheduler,
| storage, and much more, depending on how & where it was built,
| and runs applications as independent containerized environments
| (often deeply dependent on Linux kernel features) which can all
| have their own base operating systems. All of which must be
| maintained, upgraded, patched, and secured, separately, and
| frequently.
|
| Kubernetes is literally the most complex single system that
| almost anyone in the world will ever use. It is the Katamari
| Damacy of the cloud.
|
| > It allows dev teams to not worry about all these things; all
| they must do is to write a simple YAML file.
|
| _cackles, then sobs_
|
| > More importantly, teams no longer need to ask DevOps/infra
| folks to add DNS entry and create a Load Balancer just to expose
| a service.
|
| _more sobbing_
|
| > Should you use Kubernetes?
|
| Should you change a tire with a crowbar, a can of WD40, and a
| lighter? Given an alternative, the alternative is usually better,
| but sometimes you don't have an alternative.
| paxys wrote:
| Kuberenetes has been a total failure at defining a simple devops
| workflow, but I don't think that is due to any deficiencies in
| the product. The basic premise behind its common use case -
| automating away the SRE/ops role at a company - is flawed in
| itself. Companies that blindly make the switch are painfully
| finding out that the job of a system operator wasn't just to
| follow instruction checklists but apply reasoning and logic to
| solve problems, similar to that of any software engineer. And
| that's not something you can replace with Kubernetes or any other
| such tool.
|
| On the other hand, there's still a lot of value in having a
| standard configuration and operating language for a large
| distributed system. It doesn't have to be easy to understand or
| use. Even if you still have to hire the same number of SREs, you
| can at least filter on Kubernetes experience rather than having
| them onboard to your custom stack. And on the other side, your
| ops skills and years of experience are now going to be a lot more
| transferrable if you want to move on from the company.
| overgard wrote:
| I agree with the point that production is hard. There's so many
| things you just don't think about as a developer that end up
| being important. Log storage, certificate renewal, etc.
|
| I think how "hard" kubernetes is depends on how deep you go. If
| you're building a cluster from scratch, on your own hardware,
| setting up the control plane yourself etc. it's very very hard.
| On the other hand, if you're using a hosted service like EKS and
| you can hand off the hardware and control plane management to
| someone else, IMO it's actually very easy to use; I actually find
| it a lot easier than working with the constellation of services
| amazon has to offer for instance.
|
| I do think there are parts of it where "best practices" are still
| being worked out though, like managing YAML files. There's also
| definitely some rough edges. Like, Helm charts are great... to
| use. They're an absolute nightmare to write, and there's all
| sorts of delightful corner cases like not being able to reliably
| upgrade things that use StatefulSet (last I used anyway). It's
| not perfect, but honestly if you learn the core concepts and use
| a hosted service you can get a lot out of it.
| ActorNightly wrote:
| I disagree that production is hard in itself, I think the way
| people approach production makes it needlessly more difficult.
| It seems that when launching a service, everyone seems to try
| to get the entire stack in one shot, which is difficult.
|
| A much better approach is "1. Make it work 2. Make it good 3.
| Make it fast". Your initial prototypes need to define the core
| functionality, and then you incrementally build stuff on top of
| that.
| yourapostasy wrote:
| When I see developers use k8s as "programmable infra
| replacing on-prem functionality as a baseline", some time
| after step 3 is, "make it as cheap as on-prem".
| Unfortunately, many of the "move fast and break things" tenor
| that got the services built in k8s in the first place, baked
| in decisions that will cost as much to re-engineer into an
| elastic cloud-savvy form that also saves money compared to
| on-prem.
|
| This is often from organizational or project management mis-
| coordination than technical choices the developers are fully
| aware of. Like a system's owners not prioritizing putting
| events of interest into a message queue and insisting upon
| polling for the information on one end. And an offshore
| operations team saying learning how to support AWS
| Lambda/Azure Automation/GCP Cloud Functions is an added cost
| scope to the contract. So the developer is backed into a
| corner to run cron/SystemD-based polling off a small EC2.
| Thousands of decisions like this add up fast, and take a
| major executive committee commitment to root out tech debt
| and prevent new tech debt from accumulating to unsustainable
| levels to prevent cloud initiatives from sinking beneath a
| mire of expensive small decisions in pursuit of new
| accomplishments. It is a tough balancing act.
| lcw wrote:
| > 1. Make it work 2. Make it good 3. Make it fast
|
| Anecdotally, everytime someone tells me Kubernetes is
| overkill and then follows your approach. In a year they end
| up building the capabilities that come out of the box with
| Kubernetes, and of course they aren't as well thought out.
| Because they were done adhoc and as needed.
| YetAnotherNick wrote:
| > 1. Make it work 2. Make it good 3. Make it fast
|
| Disagree. The first two objective clashes very much with each
| other. I have worked in teams that did that. And generally if
| anything non trivial is done without planning to just "make
| it work", it would be more often than not, ridden with many
| design issues which doesn't allow it to be good.
| Veuxdo wrote:
| Security as well. Now step 0 is "Make it secure". When using
| microservices connected via the internet, you have no choice.
| madeofpalk wrote:
| > When using microservices
|
| 1. Make it work
| sshine wrote:
| > If you're building a cluster from scratch, on your own
| hardware, setting up the control plane yourself etc. it's very
| very hard.
|
| I'd like to make a distinction:
|
| If you do cluster deployment from scratch, Kubernetes actually
| gets easier.
|
| Because you get good at it. What it also gets is more time-
| consuming.
|
| If you aim to replicate all of AWS, then unrealistically so.
|
| The art is knowing when to stop. For a lot of people that's
| even before Kubernetes.
|
| For others, you can run all your apps in a cluster, except
| persistent storage.
| kazen44 wrote:
| At the end of the day, getting a stable production
| environment is simply a tradeoff between the amount of
| complexity you need to make your infrastructure do what you
| want, and reducing complexity because it removes failure
| points from the production environment.
|
| K8s is nice and all, but if all you really need can be solved
| by 2 VM's and a way to announce an anycast address (or use a
| loadbalancer if that is not an option), why would i add all
| that complexity?
| sshine wrote:
| I totally agree.
|
| I've gotten away with docker-compose for container
| orchestration for the last 16 months.
| b33j0r wrote:
| For reasons of experience, all I ever want from a system is
| that it's reproducible.
|
| I had a vanity website running k8s in a managed cloud. I
| thought I was backed up by my provider and original ansible
| deployment, which was of course developed iteratively.
|
| I originally did this mostly to do a practice deployment,
| and get the workflow.
|
| A few years later, it went down and I didn't notice for a
| few weeks. It was too unimportant to invest in monitoring,
| and not worth it to do over. Redeploying gave me lots of
| confusing errors (note: I also do this stuff for work)
|
| Frankly, I was surprised that the provider doing updates
| would make my years-stable site fall over. I haven't tried
| that one again for funsies, yet. It's the vendor
| specificity that was my d'oh!
| endisneigh wrote:
| Cost aside, I wonder how far you can get with something like a
| managed newsql database (Spanner, CockroachDB, Vitess, etc.) and
| serverless.
|
| Most providers at this point offer ephemeral containers or
| serverless functions.
|
| Does a product focused, non infra startup even need k8s? In my
| honest opinion people should be using Cloud Run. It's by far
| Google's best cloud product.
|
| Anyway, going back to the article - k8s is hard if you're doing
| hard things. It's pretty trivial to do easy things using k8s,
| which only leads to the question - why not use the cloud
| equivalents of all the "easy"things? Monitoring, logging,
| pub/sub, etc. basically all of these things have cloud
| equivalents as services.
|
| The question is, cost aside, why use k8s? Of course, if you are
| cost constrained you might do bare metal, or a cheaper
| collocation, or maybe even a cheap cloud like DigitalOcean.
| Regardless, you will bear the cost one way or another.
|
| If it were really so easy to use k8s to productionize services to
| then offer as a SaaS, everyone would do it. Therefore I assert,
| unless those things are your service, you should use the cloud
| services. Don't use cloud vms, use _cloud services_ , and
| preserve your sanity. After all, if you're not willing to pay
| someone else to be oncall, that implies the arbitrage isn't
| really there enough to drive the cost down enough for you to pay,
| which might imply it isn't worth your time either (infra
| companies aside).
| mirekrusin wrote:
| Cloud services are shit unless I can run them locally when
| developing and testing.
| rs999gti wrote:
| > In my honest opinion people should be using Cloud Run. It's
| by far Google's best cloud product.
|
| Is this the same thing as running containers in Azure App
| Services?
| endisneigh wrote:
| It's similar, tho I'd say that is more like App Engine. The
| azure equivalent is probably Azure container instances
| davnicwil wrote:
| Or just app engine honestly.
|
| Works with docker containers so you can run the same simple
| stack locally as in prod. No need for more exotic serverless
| architectures.
|
| Generous free tier, too!
|
| Have only good things to say about it for quickly firing up a
| product.
| asciimike wrote:
| > Does a product focused, non infra startup even need k8s? In
| my honest opinion people should be using Cloud Run. It's by
| far Google's best cloud product.
|
| > Or just app engine honestly.
|
| As a former App Engine PM and PM of Cloud Run, this warms my
| heart to hear--I'm glad folks love to use it :)
|
| It's been a few years since I've worked on these products,
| but they were designed to be composed and used by each other.
| Cloud Run provides an autoscaling container runtime with
| Knative compatible API; Cloud Build (and Container Registry)
| + Cloud Run = App Engine, App Engine + CloudEvents = Cloud
| Functions.
|
| With a Knative compatible API, you can theoretically take
| those primitives and move them to your own K8s cluster,
| managed (GKE/EKS/AKS) or self-managed, giving folks a
| tremendous amount of flexibility down the line if/when they
| need it (hint: the majority of customers didn't, and the
| fully managed services were great).
| brobdingnagians wrote:
| Similar with fly.io, I have an application running on there
| and pleasantly surprised they don't even charge if the amount
| is under $5/month. I've been very happy with how easy it is
| to deploy and with the hosted Postgres. I'm using the JVM and
| works well; I originally played around with Elixir and was
| especially impressed with their feature set for it.
| solatic wrote:
| This. Greenfield products should be serverless by default. By
| the time you have sustained traffic to the point where you can
| run the numbers and think that you could save money by
| switching off serverless, that's a Good Problem To Have, one
| for which you'll have investors giving you money to hire DevOps
| to take care of the servers.
| joshribakoff wrote:
| Disagree
|
| I tried to use lambda. Cold startup really is awful. You have
| to deal with running db migrations in step functions or find
| other solutions. The aurora serverless also does not scale to
| zero. Once you get traffic you overload RDS and need to pay
| for and setup a RDS proxy, and dont get me started on the
| pointless endeavor of trying to keep your lambdas warm. Sort
| of defeats the point. Serverless is not actually serverless
| and ends up costing more for less performance and more
| complexity
|
| Its way simpler and cheaper to start with a single VPS single
| point of failure, then over time graduate to running docker
| compose or a single node k3 cluster on that VPS. And then
| eventually scale out to more nodes...
| evantbyrne wrote:
| Serverless does not necessarily mean lambda. It could be
| just about anything that runs containers for you. AWS ECS
| has an offering called Fargate that I've been happy with
| for our hosting. You are right though that the compute
| costs are typically more than renting a traditional VPS.
| There is definitely a tradeoff between labor and compute
| costs.
| vanjajaja1 wrote:
| The point of serverless isn't to run 0 servers in your
| downtime, its to abstract away everything related to
| running hardware. I have an app that is built on a runtime
| (jdk, node, whatever) and I shouldn't have to deal with
| anything below that layer.
| suralind wrote:
| I think you could push that setup far. I'm not familiar with
| GCP or Cloud Run, but it probably integrates nicely with other
| services GCP offers (for debugging, etc.).
|
| I'd be curious to read if anybody has that setup and what scale
| they have.
| suralind wrote:
| Regarding the second part, I totally agree, either use cloud or
| don't. For some reason, most companies want to be cloud-
| agnostic and so they stay away from things that are too
| difficult to migrate between cloud providers.
| honkycat wrote:
| I am consistently confused by all of the talk about how "hard"
| Kubernetes is.
|
| We spin up EKS. We install the newrelic and datadog log ingestion
| pods onto it, provided in a nice "helm" format.
|
| We install a few other resources via helm, like external secrets,
| and external dns, and a few others.
|
| Kubernetes EKS runs like a champ. My company saves 100k/mo by
| dynamically scaling our cloud services, all of which are running
| on Kubernetes, to more efficiently use compute classes.
|
| My company has over 50 million unique users monthly. We have
| massive scale. Kubernetes just works for us and we only have 2
| people maintaining it.
|
| What we gain is a unified platform with a consistent API for
| developing our services. And if we wanted to migrate elsewhere,
| it is one less thing to worry about.
|
| -\\_(tsu)_/-
|
| Feels like some kind of hipster instinct to dislike the "cool new
| thing"... even though k8 has been around for years now and has
| been battle tested to the bone.
| Turbots wrote:
| Your company "saves" over 100k/month paying WAY too much for
| EKS, which is extremely expensive.
|
| If you're at any decent scale (looks like you are), then switch
| to GKE, or switch to on-prem and buy some hardware + a
| Kubernetes distro like Mirantis/Openshift/Tanzu.
|
| Heck, go run k3s on Hetzner and you won't have that much more
| work, but save literally millions at the scale you're talking
| about.
| unity1001 wrote:
| > Heck, go run k3s on Hetzner
|
| The moment Hetzner starts providing managed DBs or managed
| kubernetes, they will wipe the floor with all the big 3...
| drdaeman wrote:
| So, what do you do when one of your pods suddenly cannot
| connect to another, even though both nodes seem to be passing
| healthchecks and stuff?
|
| Spinning up a K8s cluster "in the cloud" is easy and everyone
| jumps on that happy "look how simple it all is" bandwagon,
| forgetting than it's just the beginning of a very long journey.
| There are millions of blog articles of varying quality that
| explain how easy it is, because it's very simple to spam search
| engines retelling a story how to click a couple buttons or do
| some basic Terraform/CloudFormation/whatever.
|
| And here's what they don't tell you - maintaining this
| machinery is still _your_ business, because all you get is a
| bunch of provisioned node machines and a cookiecutter to spin
| it up. Plus a bunch of scripts to handle most common scenarios
| (scaling, upgrading, some basic troubleshooting, etc). The rest
| is either on your or tech support (if you pay extra for it).
| And if you have a sysadmin-for-hire contract anyway, then its
| them who should have an opinion what 's easy and what's hard.
| Contracting other people is always relatively easy - compared
| to what they do.
| rglover wrote:
| Kubernetes is hard because it's over-complicated and poorly
| designed. A lot of people don't want to hear that because it was
| created by The Almighty Google and people have made _oodles_ of
| money being k8s gurus.
|
| After wasting two years chasing config files, constant
| deprecations, and a swamp of third-party dependencies that were
| supposedly "blessed" (all of which led to unnecessary downtime
| and stress), I swapped it all out with a HAProxy load balancer
| server in front of some vanilla instances and a few scripts to
| handle auto-scaling. Since then: I've had zero downtime and
| scaling is region-specific and chill (and could work up to an
| infinite number of instances). It just works.
|
| The punchline: just because it's popular, doesn't mean it's the
| best way to do it.
| birdyrooster wrote:
| It's not hard.
| menacingly wrote:
| something that considerably helped my communication was to
| transition from "what this intelligent person said is patently
| wrong" to "in what sense is this intelligent person correct".
|
| It saves me and everyone else a lot of time, because if the
| obvious black-and-white response were needed I probably
| wouldn't be having the conversation in the first place.
| gautamdivgi wrote:
| You need a decent sized team to run an on premise k8s infra. If
| you're in the cloud use a managed k8s.
|
| It's not for everyone in that I agree with the point the author
| makes. But if you have multiple teams doing app development k8s
| can be really nice. We do data movement and ML services. AKS has
| proven great for our use.
| menacingly wrote:
| Perhaps put more simply, operating in production has a lot of
| intrinsic complexity that will probably surface in the tooling,
| and if you constantly reinvent to "fix" the complexity you'll
| eventually end up putting up it back.
|
| That's how you end up with the modern javascript tooling
| hellscape where it looks like no one was around to tell a bright
| young developer "no"
| jasoneckert wrote:
| While I understand where the author is coming from, my opinion of
| Kubernetes (and production deployment in general) isn't that it
| is hard per se, but that it involves many components.
|
| I liken it to Lego. Each component separately isn't hard to work
| with, and once you figure out how to connect it to other
| components, you can do it 100 times easily. And like Lego, a
| typical Kubernetes environment may consist of several dozen or
| hundred pieces.
|
| So, I wouldn't describe Kubernetes as hard - I would describe it
| as large (i.e., comprised of multiple interconnected components).
| And by being large, there is a fair amount of time and effort
| necessary to learn it and maintain it, which may make it seem
| hard. But in the end, it's just Lego.
| tptacek wrote:
| 37signals is not like the typical large-scale startup. They have
| an extremely small team (around 30 people?), and just a couple of
| products.
|
| Large-scale startups use dynamic-scheduled cloud services in part
| to reduce coupling between teams. Every service --- and there are
| dozens --- is scheduled independently, and new teams can get spun
| up to roll out new services without too much intervention from
| other teams.
|
| When you've got a couple products that have been in maintenance
| mode for 10+ years and then 2 just two primary products, both of
| which are on the same stack, and you can predict your workloads
| way out into the future (because you charge money for your
| services, don't do viral go-to-markets, and don't have public
| services), there simply isn't much of a win to dynamic
| scheduling. You can, in fact, just have a yaml file somewhere
| with all your hosts in it, and write some shell-grade tooling to
| roll new versions of your apps out.
|
| A lot of the reflexive pushback to not using k8s seemed like it
| came from people that either didn't understand that 37signals was
| doing something closer to static scheduling, or that don't
| understand that most of what makes k8s complicated is dynamic
| scheduling.
| anonzzzies wrote:
| > what makes k8s complicated is dynamic scheduling.
|
| ... Which almost no startup or otherwise ever will need.
| Creating complex stuff for workloads you will never ever have.
| You hope to have them, but that's called premature
| optimisation. And then you still mostly likely fall in the
| bracket of a company that will never need it.
| solatic wrote:
| > Every service --- and there are dozens...
|
| Most startups that I see trying to go with microservices too
| early do so while keeping a shared database between the
| services, so they're not really microservices, but a
| distributed monolith. This turns into a massive, massive pain.
|
| Doing microservices well means building out templates for CI/CD
| pipelines, templates for observability, figuring out how best
| to share (or not share) common credentials like third-party API
| keys, setting up a service mesh, setting up services like
| Backstage and Buf... which inevitably requires hiring dedicated
| engineers at enormous cost.
|
| If you can set up a monolith, why would you switch to
| microservices? So that you can have a smaller container image
| and faster autoscaling? So the developers on the second and
| third teams you hire can wait a little less time for CI tests
| to run?
|
| It's a pretty big mistake to adopt microservices too early.
| kdhfkdbjdh wrote:
| or too late.
|
| you just described all big corp attempts at giving aws money.
|
| keep the database and all the bugs and leaky abstractions and
| spaghetti code... but split and duplicate code in a dozen
| repos... done. microservices.
| majewsky wrote:
| > Large-scale startups use dynamic-scheduled cloud services in
| part to reduce coupling between teams.
|
| This is the crux. It's Conway's Law in action. The promise of
| Kubernetes to a large org is that you can split the baremetal
| and OS layer into one team that manages everything up to
| Kubernetes, and then that's the common interface where all
| other teams deploy their applications into. And besides the
| team separation, the value is that you have a central layer to
| put automatic policy enforcement, engineering doctrines, and so
| forth.
| kazen44 wrote:
| while i agree with you that kubernetes allows you to split
| the stack down to individual "specialisations", i don't see
| how this is different with kubernetes compared to
| "traditional" Virtual machines?
|
| Usually the differentiator is either the container or VM.
| Everything beneath that gets managed by some kind of
| operations team (or teams if you are really large or
| specialized), while the stuff running ontop of the
| infrastructure is usually done by developers/application
| maintainers.
| amir734jj wrote:
| I would not use k8s unless we are convinced it will benefit us in
| the long run (think about constant effort that needs to put in to
| get things running). k8s is not magic. I would just stick with
| docker-compose or digital ocean for small startup. OR rent a VM
| on Azure OR if you really really need k8s use a managed k8s.
| brody_hamer wrote:
| Docker swarm is a great option too, for production
| environments. It's like the production-ready big brother to
| docker-compose (with better health checks and deployment
| rollout options). And it has much less of a learning curve than
| k8s.
| scubbo wrote:
| > [K8s] allows dev teams to not worry about all these things; all
| they must do is to write a simple YAML file. More importantly,
| teams no longer need to ask DevOps/infra folks to add DNS entry
| and create a Load Balancer just to expose a service. They can do
| it on their own, in a declarative manner, if you have an operator
| do to it.
|
| Yeah, as opposed to Cloudformation or Terraform, where
| you...uhhh...
|
| Don't get me wrong, it requires work to set up your corporate
| infrastructure in your Favourite Cloud Provider(tm) to make those
| things available for developers to manage. But it takes work in
| k8s too - even the author says "if you have an operator to do
| it". Kubernetes is great for what it's great for, but these are
| terrible arguments in favour of it.
| suralind wrote:
| That's true, but I'd argue that TF is not as powerful as k8s.
| You could combine it with auto scaling services offered by
| cloud providers, but then you need to understand plugins and
| with k8s it often is a single value. For example, you can "add"
| Prometheus scraper just by adding labels. You won't have that
| with TF.
| thomasfromcdnjs wrote:
| Most implementations are just slower, because they have not been
| optimised what so ever, and optimising is not even that easy for
| a senior.
| tkiolp4 wrote:
| I think the post title should be called "Production is hard" (as
| the author talks about later on). Pick up any technology out
| there: from Python, to C++, to K8s, to Linux... Do the analogous
| "Hello world" using such technologies and run the program on your
| laptop. Easy. You congratulate yourself and move on.
|
| Production is another story. Suddenly your program that wasn't
| checking for errors, breaks. The memory that you didn't manage
| properly becomes now a problem. Your algorithm doesn't cut it
| anymore. Etc.
| morelisp wrote:
| > Suddenly your program that wasn't checking for errors,
| breaks. The memory that you didn't manage properly becomes now
| a problem.
|
| Yeah, nobody deployed anything and ran it for months, even
| years, before Kubernetes.
| 988747 wrote:
| Yes, they did, but did it really take less effort than
| running it on Kubernetes?
| deathanatos wrote:
| People _think_ it took less effort.
|
| Right up until you needed to do one of the very many things
| k8s implements.
|
| For example, in multiple previous employers, we had
| cronjobs: you just set up a cronjob on the server, I mean,
| really, how hard is that to do?
|
| And that server was a single point of failure: we can't
| just spin up a second server running crond, obviously, as
| then the job runs twice. Something would need to provide
| some sort of locking, then the job would need to take
| advantage of that, we'd need the job to be idempotent ...
| all of which, except the last, k8s does out of the box.
| (And it mostly forces your hand on the last.)
|
| Need to reboot for security patches? _We just didn 't do
| that,_ unless it was something like Heartbleed where it was
| like "okay we have to". k8s permits me to evict workloads
| while obeying PDB -- in previous orgs, "PDBs" (hell, we
| didn't even _have a word to describe the concept_ ) were
| just tribal knowledge known only by those of us who SRE'd
| enough stuff to know how each service worked, and what you
| needed to know to stop/restart it, and then do that times
| waaay too many VMs. With k8s, a daemonset can just handle
| things generically, and automatically.
|
| Need to deploy? Pre-k8s, that was just bespoke scripts,
| e.g., in something like Ansible. If a replica failed to
| start after deployment, did the script cease deployment?
| Not the first time it brought everything down, it didn't:
| it had to grow that feature by learning the hard way.
| (Although I suppose you can decide that you don't _need_
| that readiness check in k8s, but it 's at least a hell of a
| lot easier to get off the ground with.)
|
| Need a _new VM_? What are the chances that the current one
| actually matches the Ansible, and wasn 't snowflaked? (All
| it takes is _one_ dev, and _one_ point in time, doing _one_
| custom command!)
|
| The list of operational things that k8s supports that are
| common amongst "I need to serve this, in production" things
| goes on.
|
| The worse part of k8s thus far has been Azure's half-aaS'd
| version of it. I've been pretty satisfied with GKE, but
| I've only recently gotten to know it and I've not pushed it
| quite as hard as AKS yet. So we'll see.
| slyall wrote:
| No it didn't. You ended up with each site doing things
| differently. You'd go somewhere and they would have a
| magical program with a cute name written by a founder that
| distributed traffic, scheduled jobs and did autoscaling. It
| would have weird quirks and nobody understood it.
|
| Or you wouldn't have it at all. You'd have a nice simple
| infra and no autoscaling and deploys would be hard and
| involve manually copying files.
| ranger207 wrote:
| Yes, until you've scaled enough that it wasn't. If you're
| deploying a dev or staging server or even prod for your
| first few thousand users then you can get by with a handful
| of servers and stuff. But a lot of stuff that works well on
| one or three servers starts working less well on a dozen
| servers, and it's around that point that the up-front
| difficulty of k8s starts to pay off with the lower long-
| term difficulty
| lowercased wrote:
| Yes.
| morelisp wrote:
| Yes.
| nosequel wrote:
| Yes. Much.
| jamwt wrote:
| Yes, way less effort
| nailer wrote:
| Yes.
| athoscouto wrote:
| I'm going to diverge from sibling comments: it depends. As
| the article points out, k8s may really simplify deploys for
| devs, while giving the autonomy. But it isn't always worth
| it.
| majormajor wrote:
| Definitely if they weren't running very complex services or
| their business tolerated an occasional maintenance window.
|
| The particular way Kubernetes can bite you is that it makes
| it much easy to _start_ with far more complex setups - not
| necessarily much harder than to start with a simple setup!
| - but then you have to maintain and operate those complex
| setups forever, even if you don 't need them that much.
|
| If you're growing your team and your services, having a
| much bigger, more complicated toolbox for people to reach
| into on day 1 gives you way more chance of building
| expensive painful tech debt.
| suralind wrote:
| I think it may appear so because Kubernetes promotes good
| practices. Do logging, do metrics, do traces. That list
| quickly grows and while these are good practices, there's
| a real cost to implement them. But I wouldn't agree that
| Kubernetes means building tech debt - on the contrary, if
| you see the tech debt, k8s makes it easier to get rid of
| it, but that of course takes time and if you don't do it
| regularly that tech debt is only gonna grow.
| majormajor wrote:
| I just rarely see people tackling greenfield problems
| with the discipline to choose to do Kubernetes without
| also choosing to do "distributed" in a broader,
| complexity-multiplying way.
|
| If not for Kubernetes (and particularly Kube + cloud
| offerings) I really doubt they'd do all the setup
| necessary to get a bunch of distributed systems/services
| running with such horizontal sprawl.
| winrid wrote:
| Yeah... at small to medium scale anyway.
| bdougherty wrote:
| Production is only as hard as you make it.
| madeofpalk wrote:
| Production is as easy as you (and your users) will tolerate.
| papruapap wrote:
| Well then you could the resume the whole article in the title
| because there isnt anything else in it.
| znpy wrote:
| Quoting Bryan Cantrill: production is war.
| wingmanjd wrote:
| I'm a simple man...I see a Bryan Cantrill quote and I upvote.
| srejk wrote:
| And war is hell.
___________________________________________________________________
(page generated 2023-03-27 23:00 UTC)