[HN Gopher] Docker, Kubernetes, Terraform, and AWS crash course ...
___________________________________________________________________
Docker, Kubernetes, Terraform, and AWS crash course series
Author : kiyanwang
Score : 168 points
Date : 2022-07-24 13:01 UTC (10 hours ago)
(HTM) web link (blog.gruntwork.io)
(TXT) w3m dump (blog.gruntwork.io)
| lakomen wrote:
| Oh boy...
|
| Guys and gals, learn the basics.
|
| You don't need Docker, just a version control system (e.g. git).
|
| You don't need overpriced cloud offerings in the first place.
|
| First of all, pick the right language. For me, that's Go, because
| it has lots of performance for little resource usage. Docker is
| written in Go. Why would I put a Go proxy in front of my Go
| program to have nothing but what's essentially a zip file with
| some startup script and then have to fight the Go proxy, Docker,
| and its configuration. Learn about systemd. It's the defacto
| standard init system for most popular Linix distros nowadays.
| Terraform? Ansible if you must. If you're managing more than 1
| host maybe you do, but it's likely you don't. Or pick something
| like NixOS if you just have to define your machine state. AWS
| (and GCP, etc)? A, one, dedicated server where you can host all
| the projects your heart desires for a fixed price with low costs
| for bandwidth/traffic.
|
| Kubernetes, you do not need it. Go can handle about 500 million
| visitors a month on 12 year old hardware per 1 server. You will
| almost certainly not reach these numbers.
|
| So you don't need much to live a stress free and cheap but
| efficient life. What this article tries to teach you is how to
| become a tool for public cloud providers.
|
| Running k8s requires at least 9 servers. Managed k8s, like those
| cloud providers offer are expensive, because even if you think it
| isn't so, the devil is in the details. I see that guy posting of
| how easy it is. BS. Nothing is easier than a single host with a
| front proxy like nginx, which btw k8s also uses, they call it
| ingres proxy, a simple nginx config, some letsencrypt, your Go
| backend and the database on the same server. It doesn't get
| easier.
|
| What this article tries to teach you is stuff you will only ever
| need if you expect over 5k concurrency (note that's not 5000
| users). When do those situations occur? Eurovision Song Contest.
| Very popular chat app (but even there you'd not use http).
|
| You don't need all this.
|
| And you also don't need microservices. Monoliths are fine. When
| are microservices useful? When you need to scale particular
| components of your big program.
|
| Is it worth the trouble? Hell no.
|
| Especially because now you have to deal with a problem so huge
| with very little benefit to going the event driven microservice
| route, because that's the only route that matters regarding
| microservices. And that's the only, ONLY time k8s makes sense.
| That's also where you can afford to pay those ridiculous prices
| for cloud traffic and managed k8s. Those are multilateral
| enterprise levels of business size, ok?
|
| Forget all that crap.
|
| In these comments you'll find people trying to BS you to get
| hyped for their product, promising "there will be only one". It's
| all BS. Just learn the basics. It's not hard.
|
| Disclaimer: truly failsafe setups should run on 3 servers (hn
| runs on 2 and has regular downtime). So if you have a money maker
| of course do that. But until then even a single server setup with
| regular backups will do just fine.
| cloudyporpoise wrote:
| God damn, I love this comment. We manage 8 Linux servers total
| for an application that is lightly used by our customers but
| heavily used by our internal workers that constantly reach out
| to internet services and check for things. Everything we've
| done has been in Go and bash, and our deployments are easy and
| take seconds.
|
| I'm tired of seeing so many posts for Docker, Kubernetes,
| terraform, and even so many of Amazon's service offerings.
| Guess what? All these tools are abstractions, and it's insane
| to me that so many would rather learn to work with these
| abstractions, as opposed to learning some Linux basics, which
| would allow them to solve the same problem with much less
| overhead. We have two webservers, four workers, and two
| database servers. All of our code is written in Go so our
| dependencies are limited, and this has been my reason for not
| needing Docker. However, we use Docker locally to spin up our
| local dev environment, so developers don't need to have MySQL
| installed on their machines. The primary benefit I see with
| Docker is dependency management, and when your dependencies are
| minimal, it's not worth the overhead.
|
| All of our servers are built from bash scripts that are
| executed on the server on the first run. This bash script
| creates folder structures and user accounts and sets
| permissions. If a server needs to be destroyed for whatever
| reason, it can be rebuilt as new from these scripts in < 10
| minutes.
|
| Code deployments are done using SCP and rsync and systemctl to
| stop and start services remotely. If we were using an
| interpreted language where dependencies are a bit trickier to
| manage in prod, certainly we'd use Docker then. Still, for the
| time being, this Go \ bash ecosystem works for us, is easy to
| manage, and limits dependencies overall. Debian & Go all the
| way, baby.
|
| The longer I work in this industry, the more I realize a bigger
| hindrance to accomplishing goals is actually overthinking and
| over-engineering. I see startups with no customers using
| kubernetes and terraform. People building noahs ark when
| they've never experienced rain. The pain points in your process
| will become obvious. Let the process tell you it's time for
| these more advanced tools. If you can manage production code
| and deploy updates easily without them, then don't waste your
| time. If you don't know any other way, then maybe it's worth
| spending the time to learn Linux a little bit better.
| jen20 wrote:
| > Go can handle about 500 million visitors a month on 12 year
| old hardware per 1 server.
|
| Doing what? I'm absolutely no fan of Kubernetes, or
| overcomplexity, but this is just meaningless platitude.
|
| The feasibility of 12,000 requests per second (assuming evenly
| distributed throughout a 30 day month) largely depends on what
| the system is _doing_ for each of those requests.
| krembanan wrote:
| I think that for the average developer, these tools will soon be
| obsolete. For frontend applications, hosting services such as
| Vercel and Netlify gives a far superior DX. For backend, Vercel
| does the same, or other BaaS' like Supabase that gives you most
| you would need to create a fully fledged app with easy to use
| API's. The only downside I can tell, is the pricing, but compared
| to dev hours saved, it's probably a net benefit.
| chii wrote:
| > The only downside I can tell, is the pricing, but compared to
| dev hours saved, it's probably a net benefit.
|
| dev hours is already paid - i say that a hosted service for an
| app is gonna cost even more than the dev in the long run.
| krembanan wrote:
| You pay a premium to let your devs work on what matters, the
| actual product, not tinker with devops tools for hours on end
| on something that has already been solved. Something to keep
| in mind is that infrastructure code has to be updated and
| maintained as well, with services that gets handled for you.
| It would be interesting to see how much of a difference in
| price it would make over eg. a year.
| chrisandchris wrote:
| I can give you the answer: it depends.
|
| I'm hosting at a colocation for X bucks a month. Solely the
| RAM I have on my servers costs X bucks a month at
| AWS/Azure. Maintenance overhead is small, maybe a couple
| hours a month.
| silversmith wrote:
| For the average developer, it's okay not to interact with these
| tools. All you need to interact is with the `bin/deploy-
| production` script the above-average developer built.
|
| And I'd argue that with managed kubernetes solution, such as
| GKE Autopilot, you get roughly the same "DX", and much more
| flexibility to use whatever tools you fancy. GKE Autopilot
| manages the cluster and node availability, ingress and
| certificates, all you need to do is write the kubernetes yaml.
| As long as what you are working on fits in a docker container,
| you're good to go. And you're fairly good to move off of their
| setup, because those same yaml configs will work on AWS or
| self-hosted solutions, albeit with additions that fill in gaps
| where Google managed services lived.
| xyzzy123 wrote:
| For backend, if you have a platform that allows you to run
| arbitrary containers, you will need to have a way to express
| how they are connected together, how much resource they should
| get, which parts should receive traffic from the Internet and
| so on. Any solution will tend to have about the same inherent
| complexity as kube yamls, this seems to be an iron law of
| devops.
|
| IMHO the thing you have to trade off to get simplicity is
| _generality_. I agree that 's probably the right tradeoff for
| 90% of apps, whether that's BaaS, heroku-alikes, language
| specific hosting or whatever.
|
| Cloud providers are definitely keen on the idea that your web
| framework and your cloud platform could basically become the
| same thing.
| eandre wrote:
| The difficult with backend is dealing with state. It's easy
| enough to provide a simple experience when dealing with
| stateless frontends, backends are a very different story.
|
| You're right that we need better tools. I'm the founder of
| Encore [1] which is all about bringing the simplicity of
| Vercel/Netlify to backend development. Not by substituting the
| backend for a BaaS but by building a developer experience hand-
| crafted for dealing with cloud infrastructure.
|
| [1] https://encore.dev
| pdimitar wrote:
| I'll definitely review your service but in the meantime you
| should know my desktop (macOS) Firefox is freaking out on
| your website and is scrolling it in a janky manner and it
| keeps one CPU core at 80% for good 30 seconds after switching
| away from the tab.
| eandre wrote:
| Oh, thanks very much for letting me know. We'll have a
| look.
| [deleted]
| brikis98 wrote:
| If you can use a Platform as a Service (PaaS) offering like
| Vercel, Netlify, Heroku, etc, you absolutely should! I always
| recommend those types of tools as the first stop tools to
| anyone building software these days.
|
| But there are many use cases that don't fit into those neat
| PaaS molds: typically, as a software company grows beyond one
| team, one service, one database, etc, they start to hit
| limitations with the PaaS solutions out there. As you scale,
| you often find you need more control than you can get from a
| SaaS: you may need more control over the hardware (e.g., for
| performance or cost reasons), or networking (e.g., you need
| service discovery or a service mesh to allow microservices to
| communicate with each other), or security (e.g., to meet
| compliance standards), or a hundred other items.
|
| That's when many companies find themselves migrating to an
| Infrastructure as a Service (IaaS) provider like AWS,
| Infrastructure as Code (IaC) tools like Terraform,
| orchestration tools like Kubernetes, and so on. I'm guessing
| every software company with more than 50-100 developers ends up
| moving from PaaS to IaaS, and that's when developers need to
| understand how to use the tools covered in this blog post
| series.
|
| Perhaps, some day, the PaaS tools out there will be good enough
| that you never have to migrate off of them, regardless of scale
| or requirements, but we're not there yet, and probably won't be
| there for a while longer.
| jbotdev wrote:
| What do you think people use to build systems like these?
|
| A lot of medium/large shops build their own deployment and BaaS
| platforms on top of Kubernetes/Terraform/AWS. You just don't
| hear about them as much because they often aren't provided as a
| service externally, and aren't open sourced. (I work with and
| on such platforms)
|
| The mistake I see people often make is assuming it's worth
| using such complex tools directly with a small project. People
| like the flexibility but don't understand cost to
| build/maintain it.
| vbezhenar wrote:
| The downside is vendor lock in. What will you do when vendor
| will ban your website?
| goodpoint wrote:
| Even better: we replace those millions of SLOC with something
| new and 100 times simpler.
| synergy20 wrote:
| not sure if IaC will help, I'm just started reading about it.
|
| Indeed the cloud vendors are all lock-ins, something needs to
| be designed to mitigate the switching cost among cloud
| providers.
|
| Each one has its own SDK and own complicated infrastructures,
| so far it seems impossible to switch once you started with
| any one of them.
| vbezhenar wrote:
| Cloud vendors are not all lock-ins. Kubernetes is an open
| API. If you've got your project working in Amazon, you can
| migrate it to GKE, Digital Ocean or just bare metal.
|
| Yes, cloud vendors love to lock you in, but right now it's
| possible to use clouds in a vendor-neutral way. Managed
| postgres is just a postgres. Managed Kubernetes is just a
| Kubernetes. S3 have plenty of alternative implementations.
| synergy20 wrote:
| what are the strongest competitors to S3? S3 is so
| dominant I rarely looked into anything else for cloud
| storage.
| vbezhenar wrote:
| S3 is just a protocol. Minio is one example of software
| which implements this protocol. You're programming
| against S3, using amazon SDK, but you change endpoint to
| another provider and it just works.
|
| I guess it wouldn't work for all the intricacies, but for
| commonly used stuff it works.
| hotpotamus wrote:
| > so far it seems impossible to switch once you started
| with any one of them
|
| Do you think cloud vendors (FAANG companies) are fools who
| have accidentally created this situation? And do you think
| they will stand idly by while you try to abstract away the
| differences between them?
| pixl97 wrote:
| Can we split the middle, they are fools, but the
| situation is not accidentally created?
| jen20 wrote:
| > And do you think they will stand idly by while you try
| to abstract away the differences between them?
|
| Yes - letting people try something futile and fail is
| much better than telling them that it is futile.
| roflyear wrote:
| Now you have to hire not just for a specific language but a
| specific backend platform. Idk man.
| davewritescode wrote:
| I can't think of a worse way to manage Kubernetes than Terraform.
| After years of running into issues, I think it's really only
| suitable for small clusters.
|
| ClusterAPI has been the way my company has been doing things
| lately and I can honestly tell you it's been transformative.
|
| https://github.com/kubernetes-sigs/cluster-api
|
| There's nothing more amazing that updating a kubelet version in a
| git repo and watching ClusterAPI carefully cordoning off nodes
| and moving our workloads to new machines while I sit back and
| watch via ArgoCD.
| MuffinFlavored wrote:
| How does ClusterAPI compare to Helm charts (or just managing
| Kubernetes YAML manually or through Kustomize?)
|
| Why do you need clusterctl / clusterawsadm? Can't you just
| kubectl apply once you have your .kube/config set up for your
| AWS k8s cluster (or Azure or GCP or whatever)?
| dilyevsky wrote:
| Cluster API is just CRDs to manage cluster infrastructure
| itself. Clusterctl and co is to manage controllers that
| reconcile said CRDs
| varispeed wrote:
| How do you deal with cloud anxiety? I've been always using bare
| metal servers, because I am afraid that when I set something up
| wrong on AWS I'll become bankrupt. When I had a few services
| running on AWS I often would wake up at night having a nightmare
| that my keys got stolen and someone run thousands on my account.
|
| After a few weeks like that I deleted everything from AWS. I have
| an account on GCP as I had to learn few things for my work
| assignment and I forgot to delete the setup - anyway a few months
| later I got an email that Google can't bill my account. I almost
| got a heart attack and then I noticed they couldn't bill $0 and
| my card simply expired - imagine the relief.
|
| When I wrote this I can see that maybe sounds stupid, but I have
| genuine fear of running anything on AWS, GCP or Azure etc.
|
| Another fear I have is that if I put a website online, someone
| could run a script to fetch content forever and make me pay huge
| bandwidth costs. When I last checked, there is no way to put any
| sort of caps e.g. when bandwidth cost exceeds certain amount it
| would deactivate the buckets etc. So what would you do? Nowadays
| very much every job lists knowledge of AWS as a requirement, but
| I can't get myself past the fear.
| jiggawatts wrote:
| For training purposes, Azure has cost-limited subscriptions.
|
| Frustratingly, the other clouds have dragged their feet and
| largely refused to implement their own equivalents. This comes
| up here semi-regularly as one of the #1 complaints about the
| public cloud, and it's always hand-waved away by some Silicon
| Valley FAANG employee making half a million a year and using
| their parent company cloud account.
|
| I know of several ways it would be possible for a malicious
| actor to deliberately bankrupt a solo dev limited to a normal
| credit card budget.
|
| And as you've said, it's very fiddly and complicated to
| implement custom "stop" scripts, and this is futile anyway
| because the billing metrics in all clouds have huge delays on
| them.
|
| My $0.02 is try to learn using Azure first because there's a
| true safety net, and then use customer accounts to continue
| upskilling.
| user432678 wrote:
| Can completely relate to your situation. Basically, that's the
| main reason I run everything on bare metal server, running in
| my basement. And use cloud providers only if my clients ready
| to pay for it or take risks.
| pixl97 wrote:
| There is nothing stupid about this fear at all. Having to fight
| a $20k mistake is highly stressful. Not being able to prevent
| it via policy is a design decision by the provider.
| oxff wrote:
| All these "you don't need X / Y / Z" points critically miss the
| point that it's irrelevant when that is what the companies hiring
| people want anyway.
| cloudyporpoise wrote:
| Companies hiring the people probably also have a need if they
| are hiring and looking to grow and scale out. I made a "you
| don't need X / Y / Z" point because I strongly believe most
| people don't need these tools and really need a better
| understanding of Linux basics. You can also learn these tools
| on your own if they're mentioned in job descriptions of jobs
| you're applying for, but that doesn't mean you have to use them
| when you're the decision maker, or for a personal project.
| oxff wrote:
| Using these tools in personal projects is how you learn them
| though, so you can talk about your experience, and maybe even
| show some prototype in an interview / CV etc.
| pdimitar wrote:
| Or better phrased: "How to get inclined to slash your own
| wrists".
|
| Jokes aside, I'll give it a go, I do need some advanced Docker
| topics lately and I am sure Kubernetes can help me with some of
| my home-hosted services as well.
| k8sToGo wrote:
| As a Devops engineer I'm staying away from Kubernetes at home.
| It would require too much constant maintenance.
|
| I use docker only.
|
| I get to play with kubernetes enough at work.
| pdimitar wrote:
| Yeah, I tend to peak at Docker Compose and dare not tread
| further. I might actually learn systemd management of
| services and stop there.
|
| I've worked in orgs where k8s worked great but most of the
| time it just made everyone curse their lives.
| tootie wrote:
| Why use k8s and AWS instead of ECS?
| k8sToGo wrote:
| Because then your endpoint is the standardized kubernetes
| rather than ECS.
|
| Also I don't think you can use Kubernetes logic with ECS
|
| Finally I think with ECS you have quite the vendor lock in.
| tootie wrote:
| Having deployed Dockerized services to ECS and Azure App
| Service, doing it from scratch twice is still easier than
| figuring out k8s.
| k8sToGo wrote:
| If you don't know k8s, sure. But if you have someone who
| does, EKS makes more sense.
| roflyear wrote:
| You can host your kubernetes cluster anywhere. Or multiple
| places (as in you can host both at azure and AWS or your own
| data center and Google etc).
|
| I'm not a fanboy of kubernetes, I think it makes some really
| trivial things stupidly complex, but avoiding vender lock in is
| huge. It can be massively better but oh well.
| crowquill wrote:
| I like and use ECS, but it lacks the tooling and ecosystem
| around Kubernetes.
| BossingAround wrote:
| Kubernetes must probably be one of the most tutorialized pieces
| of SW out there. You want a "crash course" on the platform of
| your choice? You got it. You want it for the price of your
| choice? You got it.
|
| The docs are great [1] and offer even interactive parts [2].
| There are a ton of good materials on youtube. There are Coursera
| and EdX courses, e.g. [3] from Google and [4] from Red Hat. There
| are a ton of Udemy courses, e.g. [5]. I'm not even going to
| mention sites like Pluralsight (which now owns CloudGuru and
| CloudAcademy).
|
| I'm not even mentioning books on advanced Kubernetes and service
| mesh topics.
|
| I wonder what drives someone to write a 101-level tutorial on
| K8s. In my mind, there's such an overload of information on the
| topic that it must get lost among the greats.
|
| [1] https://kubernetes.io/docs/concepts/overview/what-is-
| kuberne...
|
| [2] https://kubernetes.io/docs/tutorials/kubernetes-
| basics/creat...
|
| [3] https://www.coursera.org/specializations/architecting-
| google...
|
| [4] https://www.coursera.org/specializations/cloud-native-
| develo...
|
| [5] https://www.udemy.com/course/learn-kubernetes/
| rawoke083600 wrote:
| To be fair. Just last night i was googling kubernetes and k3s
| tutorials.
|
| All of them on page 1 of SERPS are super simple and brilliant
| and ONLY show you how to add nodes to clusters.
|
| Ok now what ?? Ive drank the k-Koolaid signup for the
| newsletters but how do i get value from this 'thing' ???
|
| When i code a webapp how do setup/config the db ? How do I do a
| regular LAMP app ?
|
| Lol i never thought i say this but i need a kubernetes tutorial
| on 'best practices' for common patterns. No for google scale
| but how do i kubernetify my run of the mil lamp app with
| 100kpage views a month ??
|
| There seems to be a gap between the beginner and advance
| (kubernetes all things) levels ?
|
| Maybe that is the answer ?? The fact that the middle-
| situations(lamp apps with low-med traffic) tutorials dont
| exists means i dont need it ??
| MuffinFlavored wrote:
| > SERPS
|
| Search Engine Result Pages
| vbezhenar wrote:
| All you need exists on kubernetes.io.
|
| Start with https://kubernetes.io/docs/tutorials/hello-
| minikube/ and proceed.
|
| Read reference documentation on the same site whenever you
| need to dig somewhere.
|
| It's awesome. You don't need any other websites. I was able
| to build a kubernetes cluster and right now deploying multi-
| service application and I have had enough technical
| information on this website alone.
|
| As to answer your questions:
|
| You can start with DB as an ordinary container in a
| statefulset deployment. It's similar to docker. You configure
| it with environment variables. Advanced approach is
| operators, but you don't need those for simple start.
|
| Connection to DB is the same like docker. You use Kubernetes
| secrets instead of docker secrets and that's about it.
|
| My message assumes that you're proficient with docker. If
| you're not, I suggest to first learning docker. May be you
| don't need Kubernetes for your scale at all. And if you do,
| most docker concepts make sense in Kubernetes anyway.
| timhaak wrote:
| I found a similar problem.
|
| Most examples failed when then trying to use them together.
|
| Also, jumping into directly K8 can be quite a jump.
|
| I put this together to help SA Php group.
|
| Starts off with just deploying directly on a server.
|
| Then takes you to a full application deployed on K8 with
| auto SSL and DNS generation.
|
| It needs a bit of a refresh :(
|
| But finally coming out of being a bit over-committed, so
| should be updating in the next week or two.
|
| Still some bits missing but it should cover all your basics
|
| https://github.com/haakco/deploying-laravel-app
| rawoke083600 wrote:
| Cool thanks will have a look.
|
| Id be more interesting on where kubernetes (redundancy)
| fits into traditional redundancy ?
|
| Examples:
|
| 1. Do i still need gluster/ceph or do i use the longhorn
| thing ?
|
| 2. Db replication? Do i use the usual solutions of master-
| slaves and clusters or does multiple k-nodes take that over
| ?
|
| 3. Webserver LB with failover ? Do i use LB from hosting
| vendor, haproxy or does kubernetes have its own thing ??
|
| From what I can tell as a kubernetes-noob the value is: 1)
| Reproducibility 2) Reliability via redundancy 3)
| AutoScaling.
|
| All of the abkve has to some degree a previous/current
| solution , so which do i give up/replace with kubernetes-
| tool ?
|
| Sorry yes Im a k-noob
| timhaak wrote:
| 1 It depends. For larger scale gluster or ceph. But quiet
| a bit more work.
|
| Longhorn you can get up and working quickly.
|
| Though if you are on a cloud provider just use their
| storage system.
|
| 2 K8 doesn't magically solve replication unfortunately.
|
| Though there are helm charts that will automatically set
| up a replicated setup for you.
|
| I still need to solve backups.
|
| Once again if you are on a cloud provide. Just use their
| Db offering.
|
| 3 K8 doesn't have a default out of the box.
|
| The repo shows you how to setup traefik to handle this.
|
| On cloud providers they have normally integrated it with
| their lb already.
|
| For me the large advantages are reproducibility and no
| vendor lock in.
|
| Also give redundancy and quiet a bit of automation once
| set up.
|
| Auto scaling is always tricky.
|
| Lastly if you have the skills it can be far cheaper to
| run your own in metal.
|
| If you don't the the time most likely would be better
| spent actually coding.
|
| Depending where you are in the world and the relevant pay
| scales.
| vbezhenar wrote:
| I'll try to answer, but keep in mind that I'm newbie
| myself.
|
| > 1. Do i still need gluster/ceph or do i use the
| longhorn thing ?
|
| Kubernetes does not care about storage implementation. It
| contains some abstract ways to request a storage
| (PersistentVolumeClaim). And then some particular
| Kubernetes installation will fulfil this request with
| PersistentVolume. So basically it comes down to your
| Kubernetes provider. It should have some instructions
| about volume classes that you can use.
|
| If you're installing your Kubernetes on bare metal, you
| need to think about this aspect yourself, of course. Both
| ceph and glusterfs are popular options and there're good
| Kubernetes drivers for it. You can also just use local
| storage, like docker does, but of course it'll not
| survive server outage, so it limits your availability.
|
| I'm installing my cluster on OpenStack. There's Cinder
| CSI plugin for Kubernetes, so it provides me storage when
| I ask for it. My provider uses SAN for one type of volume
| and Ceph for other type of volume.
|
| I think that simplest solution is some kind of NFS
| server. Kubernetes can consume it as well.
|
| > 2. Db replication? Do i use the usual solutions of
| master-slaves and clusters or does multiple k-nodes take
| that over ?
|
| Basically Kubernetes does not care about your particular
| configuration. It runs containers and provides those
| containers with storage, DNS, network, etc. So it's up to
| you. You can configure database replication with your own
| tools and scripts if you like. I have had good experience
| with CloudNativePG. It's so-called Kubernetes Operator.
| Basically it's a thing that configures database clusters
| for (postgres) given abstract definition. It can
| configure master-slaves cluster and it allows for easy
| backup configuration to S3 storage. There're other
| operators as well, for almost any popular database. So
| probably it's better to use those, unless you're very
| good at database operations.
|
| And in the true clouds it might be a good idea to use
| managed database and not to think about it at all.
|
| > 3. Webserver LB with failover ? Do i use LB from
| hosting vendor, haproxy or does kubernetes have its own
| thing ??
|
| You need some kind of external load balancer to deliver
| packets in a high-available way. My hoster provides that
| as part of its OpenStack package. I guess that every
| cloud hoster provides it. If you're using bare metal, you
| need some kind of haproxy and keepalived setup (or some
| kind of hardware load balancer, I have no idea).
|
| This external load balancer have to deliver TCP packets
| to your worker nodes. Like 1.2.3.4:80 -> 10.1.1.1:30080,
| 10.11.1.2:30080, 10.11.1.3:30080. And once Kubernetes
| receives those packets in a high-available way, it routes
| them as needed. Usually you have ingress controller which
| provides HTTPS, and then uses HTTP host and path to route
| request to some pod which serves it in the end. Once
| request reached Kubernetes, it'll make sure to route it
| the right way. If your pods deployed in 2+ replicas,
| it'll be high-available. If your pod deployed in 1
| replica and server is died, Kubernetes eventually will
| reschedule that pod to another server, but there'll be
| service interruption for a few minutes. So everything
| high-available should be deployed in 2+ replicas.
|
| > From what I can tell as a kubernetes-noob the value is:
| 1) Reproducibility 2) Reliability via redundancy 3)
| AutoScaling.
|
| Here's my take on Kubernetes value.
|
| First is it introduces a language connecting developers
| and operations. It is important. You don't need
| developers to hand-waive which ports they need to expose,
| which services they need to consume and which HTTP routes
| they need to receive for that particular server. They've
| got language to express how their service should be used.
|
| Second is it provides high-available cluster. And in my
| testing it's quite stable. My only issue is that it takes
| too much time to reschedule pods from dead server. I
| expected it to take few seconds, but it took few minutes.
| I don't yet know why is that.
|
| Third and one I didn't really expect when started to
| learn it: it provides high-quality solutions for some
| hard problems. I mentioned database clustering and
| database backup. I can deploy database with single
| 50-lines YAML mostly copy&pasted from the example. It'll
| start master and slave pods and it'll provide continuous
| backup to S3 using barman. I don't have skills to
| configure that kind of setup and I expect that I'd need a
| week at least to come to that setup. Another problem is
| letsencrypt. Well, it's not that hard, but I've spent
| many hours debugging some convoluted nginx/caddy/whatever
| net of docker containers trying to figure out what
| letsencrypt does not work there. With Kubernetes cert-
| manager it's just works. All configuration is
| centralized, all services write ingress and they've got
| their TLS certificate automagically, whether it's HTTP-01
| solver, DNS-01 solver, it's just abstracted away.
|
| It has some steep learning curve, that's for sure. Even
| more so, if you want to deploy it yourself rather than
| using managed one. I suggest to use managed one if you
| can. I have some circumstances which prevent me to using
| managed Kubernetes, but I plan to migrate to managed as
| soon as I can. It's not that hard, but it takes time and
| managed Kubernetes is cheap enough. If you can't use
| managed Kubernetes, try to find a provider with OpenStack
| API. It'll help with load balancers and storage
| provisioning.
|
| Autoscaling - that part I didn't solve yet. It's not easy
| if you're not using managed Kubernetes. But if you're
| using managed Kubernetes, it should be as easy as ticking
| a checkbox somewhere.
|
| > All of the above has to some degree a previous/current
| solution , so which do i give up/replace with kubernetes-
| tool ?
|
| Well, right now we're using three dedicated servers with
| docker-composes scattered all over, made with ad-hoc
| scripts and whatnot, partially working backups, no
| observability. Chaos. Kubernetes for us looks like a very
| promising way to throw away that chaos and rebuild
| operations correctly.
| ericbarrett wrote:
| > Maybe that is the answer ?? The fact that the middle-
| situations(lamp apps with low-med traffic) tutorials dont
| exists means i dont need it ??
|
| SRE for 15 years with FAANG (MAGMA?) scale experience; I
| would argue this, yes. That's a little over 2 requests per
| minute. If you're cloud hosting then you could get away with
| single tiny hosts for your front end and DB, like t3.small on
| AWS, and Cloudwatch alarms for monitoring. If you need
| extreme HA or burstability even with this low rate, a managed
| load balancer (e.g. AWS ALB) with a few target webservers
| will do the trick and let you swap them out as needed without
| taking down the site. A DB read replica will give you
| redundancy there as well. This is all 20-year-old tried and
| true tech. You'll set it up once and it will run for _years_
| without trouble.
|
| Once you introduce Kubernetes you've got a whole 'nother
| beast to feed, especially if it's self-managed, and you'll
| pay dearly if it's not (like EKS). For your scale, it would
| be kind of like buying a full sized semi truck to haul stuff
| for a corner store. You'd be better served with a small
| pickup or minivan.
|
| Of course you don't need an "at scale" excuse to learn new
| tech. In which case I agree there's a dearth of practical
| tutorials that aren't just "here's a Helm chart." Part of
| this is because managing a stateful service like MySQL on K8s
| is not straightforward; there's a lot of ways to do it, most
| of them are wrong, new ways get introduced every few years,
| and even people who claim they've solved the issue are
| probably sitting on a time bomb of their own making and have
| just gotten lucky.
| elbigbad wrote:
| Our platform team got into k8s for some reason at my
| company. Because it's enterprise there's never really the
| "millions of requests per second" sort of problems because
| every deployment is single tenant. We have experienced k8s
| engineers who I guess we're just using the tools we know,
| but wow it's been such a hassle and has caused so much
| toil. Right now we're having trouble scaling back ends for
| parallel jobs with k8s, which is a problem we easily solved
| with regular tooling in the past pre-k8s.
| cjalmeida wrote:
| On enterprises, k8s solves a different kind of scaling
| problem. You tend to have hundreds of apps that are
| business critical and need to be managed.
|
| K8s provides a consistent way to package, deploy with HA
| and monitor those apps. Ideally it replaced tons of ad-
| hoc scripts and confluence pages that are very poorly
| maintained.
| ancieque wrote:
| I would recommend Bret Fisher tutorials
| https://www.bretfisher.com/
| starkd wrote:
| This is no doubt a useful commment, since you provide other
| sources. However, it's not particularly a mark of courtesy to
| denigrate others' contribution to the space. If you had
| substantive criticism of the OP offered tutorial, it might be
| helpful. However, you offer none.
| BossingAround wrote:
| I can see your point. Thanks for your comment, I'll try to
| restate my point next time.
| waynesonfire wrote:
| I just don't want to do that, no thanks. I've spent so much
| time learning about how an OS can solve these problems and
| these lessons have paid dividends many times over my career.
| Instead, I will continue to invest in the core foundation. I'm
| not interested in re-learning these abstractions reading
| [1]-[5] tutorials plus many more and studying an unrefined, new
| layer of complicated abstractions built on top of the OS--the
| OS is difficult enough. I'll only _use_ k8s when it's managed
| and supported by a team of 10+ engineers, which is what it
| requires. Plus, it's not just k8s; you have concourse,
| spinnaker, artifactory, some sort of cluster templating, kops?
| to template your k8s deployments across DCs and environments,
| (I don't know what the community uses, we built our own in-
| house tool). It's all so gross.
|
| My rejection of these systems has lead me to invest in FreeBSD.
| There was a learning curve and I'm certainly not as fluent in
| this system as I am with Linux but I'm in a place where I'm in
| control of the OS and have at my disposal solid, refined tools
| to help me masterfully construct the infrastructure to solve my
| problems. And, every time I solve a problem in this space my
| foundational understanding grows and these lessons will pay
| dividends into the future, or so I hope.
|
| I'm utilizing FreeBSD because docker / k8s and I'll add systemd
| to the list, are missing and thus the community solves problems
| in a different way that better aligns with my values. FreeBSD
| isn't easier, I've been stumped on problems countless numbers
| of times but when I find a solution, it scales. What I mean by
| that is I have a better understanding of a system that can
| solve a larger class of problems than if I was in the docker /
| k8s ecosystem.
|
| YMMV.
| BossingAround wrote:
| That sounds like a bit of an extreme reaction to what seems
| to be a "this technology is yucky and I don't like it"
| problem.
| intelVISA wrote:
| They're not wrong tbf, you can largely replace K8s with
| curl and some basic cloud CLI scripts for the majority of
| cases. Slight hyperbole but I'll never really understand
| the weird urge to use Borg-lite everywhere.
| waynesonfire wrote:
| No, you're way off. Maybe my reaction is extreme, but my
| problem isn't. What's yucky is that you're trying to
| convince people that reading five tutorials on k8s will
| make them productive. No, the problem, and I'll repeat it
| since you conveniently skipped over it, k8s requires 10+
| engineers to maintain and manage, full time.
|
| _I_ don't have 10 engineers and thus it's the wrong tool
| for me and I've decided to not invest in it.
| bavell wrote:
| K8s is certainly a beast and I agree with your posts
| directionally but just as a counterpoint - as a
| solopreneur I've been using k8s to run my business
| workloads (on GKE) for the past 4 or 5 years now. I'm
| very comfortable with dev and ops and k8s is a force
| multiplier for me, letting me easily manage much more
| than I could without it (e.g. due to automation, tooling,
| community, etc). Building on top of a standardized
| platform with a huge community has been a big win for me
| as a solo dev.
|
| I'm not running a very big operation, I only have two
| nodes which host a few custom webapps and a few dozen WP
| sites. Running in a single region removes the extra
| charge for HA GKE, letting me run pretty lean and just
| pay for the VM, storage and bandwidth. I hardly ever have
| to spend any time on managing the cluster, it keeps
| chugging along while I get things done and makes it easy
| for me to manage app lifecycles. YMMV.
|
| I keep it simple, I tried helm but didn't like it because
| it added too much complexity. I pull in cert-manager and
| nginx-ingress to every cluster I run but nothing else. I
| build my images locally and push to the registry
| directly, no CI/CD. I focus on the core competencies of
| k8s and try to stay lean and conservative with adding new
| tools or components.
| brikis98 wrote:
| Author here. I tried to answer your question in the first two
| paragraphs. But to add some context, given the nature of my
| work, I hear from developers on a nearly daily basis who are
| struggling to get started with the technologies mentioned in
| this blog post series, which include not only Kubernetes, but
| also Docker, AWS, and Terraform. In part, they are struggling
| because they are too scared to ask for help, and comments like
| yours only make that worse: you seem to be implying that the
| materials out there for Kubernetes are so good, that if you
| don't get it, there must be something wrong with you. And yet,
| there are thousands of devs who don't get it, so maybe for
| different people, there are different ways to learn?
|
| In discussions like this, I'm a fan of what Steve Yegge wrote
| about blogging [1]:
|
| > This is an important thing to keep in mind when you're
| blogging. Each person in your audience is on a different clock,
| and all of them are ahead of you in some ways and behind you in
| others. The point of blogging is that we all agree to share
| where we're at, and not poke fun at people who seem to be
| behind us, because they may know other things that we won't
| truly understand for years, if ever.
|
| That's why I write: to share what I know, from my particular
| perspective. Hopefully, that's useful to some people out there.
| If it's not useful to you, no problem!
|
| And for the record, I agree the Kubernetes docs are great,
| including those interactive tutorials: if you read the series,
| you'd see I actually recommend those exact docs at the end of
| the post [2].
|
| [1] https://sites.google.com/site/steveyegge2/you-should-
| write-b... [2] https://blog.gruntwork.io/a-crash-course-on-
| kubernetes-a96c3...
| TuringNYC wrote:
| >> I wonder what drives someone to write a 101-level tutorial
| on K8s.
|
| When I was going thru k8s tutorials several years ago, one
| issue was how quickly the tutorials became outdated with
| changes in k8s. Then you'd spend only half your effort on the
| the tutorial, and the other half debugging why the tutorial
| wasnt working.
___________________________________________________________________
(page generated 2022-07-24 23:02 UTC)