https://stackoverflow.blog/2021/07/21/why-you-should-build-on-kubernetes-from-day-one/ * Essays, opinions, and advice on the act of computer programming from Stack Overflow. Search for: [ ] [Search] Latest Newsletter Podcast Company [blog-k8s] code-for-a-living July 21, 2021 Why you should build on Kubernetes from day one If you're building a new app today, it might be worth taking a closer look at making it cloud-native and using Kubernetes from the jump. The effort to set up Kubernetes is less than you think. Certainly, it's less than the effort it would take to refactor your app later on to support containerization. Avatar for Max Horstmann Max Horstmann If you're starting a new project from scratch--a new app, service, or website--your main concern usually isn't how to operate it at web scale with high availability. Rather, you're likely to focus on building the right thing for the customer you're targeting or finding product market fit. If you're creating an MVP for a startup, you need to get to that before scaling, otherwise, who are you scaling for? If you're a developer at an enterprise, you want to make sure that you're meeting the expectations and needs of the business. Operating at scale is, at best, a tomorrow problem. Hence, when it comes to choosing the right set of technologies, Kubernetes--commonly associated with large, distributed systems--might not be on your radar right now. After all, it comes with a significant amount of overhead: setting up and operating a cluster, containerizing your app, defining services, deployments, load-balancers, and so on. This may look like massive overkill at the early stage, and you might think that your time would be better spent on other tasks, such as coding the first couple of iterations of your actual app. When we started building Stack Overflow in 2008, we didn't have that choice. There was no Docker (2013), no Kubernetes (2014). Cloud computing was in its infancy: Azure had just launched (2008), and Amazon Web Services was about two years old. What we built was designed for specific hardware and made a lot of assumptions about it. Now that we're modernizing our codebase and moving to the cloud, there's a fair amount of work we have to put in to make Kubernetes and containers work. Going through this process has given us a new perspective. If you're building a new app today, it might be worth taking a closer look at making it cloud-native and using Kubernetes from the jump. The effort to set up Kubernetes is less than you think. Certainly, it's less than the effort it would take to refactor your app later on to support containerization. Here are three reasons why building your app on Kubernetes from the start might not necessarily be such a bad idea anymore. Managed Kubernetes does the heavy lifting At Stack Overflow, when we set up our first in-house Kubernetes cluster a couple of years ago, it took us close to a week to get everything up and running: provision virtual machines, install, configure, configure, configure. Once the cluster was up, there was ongoing maintenance. What we ultimately realized was that Kubernetes was great for us--but we wanted somebody else to run it. Today, managed Kubernetes services such as Amazon's Elastic Kubernetes Service (EKS), Microsoft's Azure Kubernetes Service (AKS), or Google's Google Kubernetes Engine (GKE) allow you to set up your own cluster literally in minutes. For example, in AKS, you can just click a few buttons in the portal and fill out a couple of forms: [xLyJO49MtM] This is convenient, but you might wanna stop short of actually creating the cluster at the end of the workflow. Go through the wizard, but don't click that blue "Create" button at the end! Instead, download the configuration you just created as an ARM template and check it into your source control system. Now you have the best of both worlds--ease of use and infrastructure as code (IaC)! Once you're set up here, there's little left to do once you start scaling your application except write bigger checks to your cloud provider. Any additional resource allocation is easy. The problems that come with scale--fault tolerance, load balancing, traffic shaping--are already handled. At no point will you hit that moment of being overwhelmed with success; you future-proofed your app without too much extra effort. You can stay (somewhat) cloud agnostic If your project is successful, chances are that technology decisions made in the early stages will still have an impact months or years down the road. Stack Overflow, for instance, was originally written in C#. 13 years later, it's still written in C#, but it used to be, too. Occasionally someone (ahem) suggests that we rewrite it in Node.js. That's still never happened. The same can be said about dependencies on cloud services. You might build your new app on top of infrastructure as a service (IaaS) products like Amazon's EC2. Or maybe you're starting to take dependencies on platform as a service (PaaS) offerings such as Microsoft's Azure SQL. But are you willing to make a long-term commitment to the cloud providers behind them at this stage? If you don't know yet where your journey is going to take you, maybe you'd prefer to stay cloud agnostic for a little longer. Let's get back to infrastructure as code: throwing a tool such as Terraform into the mix is going to help you with staying cloud agnostic to some degree. It provides a unified toolkit and configuration language (HCL) to manage your resources across different cloud and infrastructure providers. Your app is unlikely to be truly cloud agnostic, however, in the sense that you'll be able to just switch your cloud provider as easily as your internet or electricity provider at home. Here's a good discussion on this topic in HashiCorp's forum: Is Terraform really cloud agnostic? As one of the commenter points out: > "A Kubernetes cluster is a good example of an abstraction over compute resources: there are many hosted and self-managed implementations of it on different platforms, all of which offer a common API and common set of capabilities." This sums it up nicely! It's still not a perfect abstraction. For example, each cloud provider is likely to have its own custom way of implementing things like public load balancers and persistent volumes in Kubernetes. It's still be fair to say if you're building on Kubernetes, you're going to stay cloud agnostic to a certain degree. You can easily spin up new environments--as many as you like! Kubernetes is usually seen as a way to manage your production infrastructure. But here at Stack Overflow, we've been using it to manage our test environments on the fly. We're using Kubernetes to host what we call PR Environments. Every pull request can be run in an isolated test environment at the push of a button: [VMEnAyZjE5] And when we say "isolated environment", we mean everything: the app itself (with the code changes in the PR branch) with its own dedicated instances of SQL Server, Redis, Elasticsearch, and additional services pieces. All spun up from scratch within minutes and running in a handful of containers in a dedicated namespace, just for you and anyone who's interested in your PR. This isn't something we invented; other organizations have been using this concept. The idea is that every code change goes into a version control system like Git through a pull request. Other developers will review the code, but the code won't tell the whole story. You want to see the code in action. Normally, you'd have to download all the code locally, compile, and run it. That could be simple, but if you're running a large application that draws code from multiple repos or--have mercy--a microservice architecture, then you may run into several hours of debugging. Even better, let's say you've squashed all of the commits for a new feature into a single one and are committing it as a single PR. Send that PR environment to sales or marketing as a single link so that they can preview the feature in action. If your sales team wants to demo the app with specific features or custom builds, send them a PR environment link. You won't have to spend time walking your less technical colleagues through the build process. A lot of groundwork was required to get to this point. First off, running classic .NET Framework in Windows Containers wasn't really an avenue we wanted to pursue. It's possible in theory--Windows support has been available in Kubernetes since v1.19--but the Docker/ Kubernetes ecosystem is really more centered around Linux. Thankfully, our migration to .NET Core was already underway, so we decided to bet on Linux containers. This, of course, came with its own set of challenges. When you're dealing with a 10+ year old codebase, you're likely going to find assumptions about the infrastructure it's running on: hardcoded file paths (including our favorite: forward slash vs. backslashes), service URLs, configuration, and so on. But we got there eventually, and now we're in a place where we can spin up an arbitrary number of test instances of Stack Overflow, the Stack Exchange network, and our Teams product on our auto-scaling Kubernetes cluster. What a time to be alive! Looking back at the early days of Stack Overflow, having this kind of tooling available back then would have been a game changer. In the early stages of building a product, you typically want to build, measure, and learn as much and as fast as possible. Using containers and Kubernetes will allow you to build the tooling for that and future-proof you in case you're going to scale. So, should you use Kubernetes from day one? Maybe! It still depends, of course, on your specific project, your requirements, and your priorities. But have you been saying "we don't need Kubernetes because we don't have product market fit yet"? Take a closer look, and maybe you'll find yourself saying "we need Kubernetes because we don't have product market fit yet." Tags: kubernetes, software development, testing Podcast logo The Stack Overflow Podcast is a weekly conversation about working in software development, learning to code, and the art and culture of computer programming. Related [blog-pusher-sponsored-1200x630] code-for-a-living May 19, 2021 Using Kubernetes to rethink your system architecture and ease technical debt This is a story about trying to rethink complex systems: the challenges you face when you try to rebuild them, the burdens you face as they grow, and how inaction itself can cause it's own problems. When you're weighing the risk and reward of replacing architecture, it can take several attempts to find a solution that works for you. Avatar for Avatar for Chris Casey, Product Manager at Pusher and James Lees, Senior Software Engineer at Pusher [The-Overflow-Blog] newsletter June 4, 2021 The Overflow #76: The 2021 Dev Survey now open! Welcome to ISSUE #76 of the Overflow! This newsletter is by developers, for developers, written and curated by the Stack Overflow team and Cassidy Williams at Netlify. The new Developer Survey is here! But also: precise measurements using imprecise tools and a visual walkthrough of Kubernetes. From the blog The 2021 Developer Survey is now open! stackoverflow.blogWe want to... Avatar for Ryan Donovan Avatar for Cassidy Williams Ryan Donovan and Cassidy Williams [blog-infrastructure] code-for-a-living March 8, 2021 Infrastructure as code: Create and configure infrastructure elements in seconds IaC allows developers to supply IT environments with multiple lines of code and can be deployed in a matter of minutes (in contrast to manual infrastructure, which can take hours if not days to be deployed). Avatar for Theodoros 'Theo' Karasavvas [The-Overflow-Blog] newsletter May 28, 2021 The Overflow #75: Where design meets development Welcome to ISSUE #75 of the Overflow! This newsletter is by developers, for developers, written and curated by the Stack Overflow team and Cassidy Williams at Netlify. This week: Rebuilding static sites a little at a time, telling the difference between decompilers and disassemblers, and five tools to fix security holes. From the blog Incremental Static Regeneration: Building... Avatar for Ryan Donovan Avatar for Cassidy Williams Ryan Donovan and Cassidy Williams 20 Comments [c3d] myself says: 22 Jul 21 at 8:08 Why should you stop telling people what to do on day 1 without knowing their circumstances. Reply * [9bb] Max Horstmann says: 23 Jul 21 at 7:21 If you can find the time to finish reading the blog post, you might agree that I'm not actually doing that. Reply [69b] Metoule says: 22 Jul 21 at 8:47 That's kind of off-topic, but instead of ARM templates, you can now use Bicep (https://github.com/Azure/bicep), a dedicated language designed specifically to make deploying Azure resources easier. It's a lot more readable than the XML files that make ARM templates. Reply * [9bb] Max Horstmann says: 22 Jul 21 at 11:29 Not at all off-topic - thanks! Do you have any thoughts on using Bicep vs Terraform? Reply + [69b] Metoule says: 23 Jul 21 at 8:39 We're fully invested in Azure, so in our case, Terraform doesn't seem to provide additional value. Also, we were lucky that Bicep was available when we started our IaC journey, so we didn't evaluate other tooling. I can just say that IaC is awesome, and made our life so much easier Reply o [9bb] Max Horstmann says: 23 Jul 21 at 7:22 Great to hear, thanks for sharing! Reply * [6a4] John Doe says: 23 Jul 21 at 7:20 No, no and no. Rolling out a Kubernetes cluster is considerably complex and definitely not worth it for every start up, especially as most cloud providers charge for it, even if it is idle. It is more important to run your code in containers. This gives you the flexibility to run on compute instances initially, and then migrate to K8s when you need the ability to scale up. Reply + [9bb] Max Horstmann says: 23 Jul 21 at 7:49 I respect your strong opinion and appreciate you sharing it. However, I have a different view. Rolling out a cluster really isn't particularly hard anymore, as managed offerings have improved significantly and take care of the heavy lifting. I agree that it's not worth it for every startup, as that always depends on individual circumstances and goals. However, especially in the early stage, frequent iterations are crucial; the ability to dynamically spin up environments in separate K8S namespaces is - at least for us -a very valuable tool which has increased productivity and enabled faster iterations. Our designers and PMs have described it as a game-changer, and we wish we had this in our toolbox in our own early startup days. Hence, I'm proposing to make that part of your consideration. It's true that even an idle cluster will accrue charges, but autoscaling will mitigate this to a large degree and the cost of running just a handful of VMs is relatively low - especially compared to the cost of engineering /designer/PM bandwidth. I agree that it's important to run your code in containers, but that's a prerequisite for running Kubernetes, not an argument against it. Good luck! Reply [17f] Forrest Hopkins says: 23 Jul 21 at 2:45 > After all, it comes with a significant amount of overhead: setting up and operating a cluster, containerizing your app, defining services, deployments, load-balancers, and so on. I was hoping this article would address these pains a little more directly but it seems to only have a solution to the "setting up a cluster" part. As someone currently building a small MVP app, I'd *like* to use k8s day-one, but dealing with all the moving parts involved just isn't practical right now. Setting up virtual networks, load balancers, deployments, autoscaling, etc... it takes a lot of time and I can't justify it for a project with zero users. Reply * [9bb] Max Horstmann says: 23 Jul 21 at 8:09 You're making a great point here - if setting up tons of infrastructure pieces takes a lot of time, your bandwidth will certainly be better spent on building the actual product. It's of course impossible for me to judge how many of the pieces you're mentioning would be needed for your specific project; however, in my personal experience, it's often possible to get off the ground with a relatively small subset of everything that Kubernetes has to offer in little time. Autoscaling won't be needed until you have actual users. "Setting up" load balancers and deployments means writing a few lines of YAML. And typically, putting these pieces in place addresses problems you'll have to solve regardless, with or without Kubernetes. I mean, deployments ... you're gonna have to deploy your code somehow, right? And especially at MVP stage, you're gonna be living and breathing fast iterations while seeking tons of feedback from various stakeholders. Wouldn't it be nice if were able to have different versions of your app and quickly spin them up and down as needed? Kubernetes will allow you to do that. Good luck! Reply + [17f] Forrest Hopkins says: 24 Jul 21 at 2:51 All good points! I'll have to look a little more into it Reply + [7d6] Andrew Dobrych says: 24 Jul 21 at 8:21 I know GKE also offers an 'autopilot' version of the cluster which is easier to set up and go for smaller apps, not sure if it's available on other cloud providers. Reply * [cba] Duncan Krebs says: 28 Jul 21 at 5:17 Hey buddy I'm with you - Kubernetes is the answer to scalable and successful startups but there has to be more layers of abstraction to get us from nothing to a working deployment that can be reproduced in any cloud provider or on-prem cluster, the technology is just not there yet unless someone has a different point of view.. Reply [b86] Dhawal Kapadia says: 23 Jul 21 at 8:26 if the Sr leadership has committed to let's say Azure - would you still use Kub? or, go with App Services and Az Functions? Reply * [9bb] Max Horstmann says: 23 Jul 21 at 7:17 Great question! In that case, staying cloud agnostic is not a concern. Your technology choices will always depends on your specific circumstances and goals. If you would benefit from dynamically spinning up test environments in separate namespaces in a K8S cluster, as outlined in the blog post, I'd recommend factoring this into your consideration. Reply [cf5] PJ says: 25 Jul 21 at 9:34 Another blog putting equality between "app" and "web app". Is kubernetes needed for desktop app? For automotive app? For embedded app? Reply * [522] CL says: 28 Jul 21 at 10:23 These articles are not assuming your "app" is a "web app", they are assuming it to have a "cloud" infrastructure. Nowadays, even refrigerators have an internet connection and use cloud infrastructure, so yes, those considerations are valid for any non strictly offline app, including embedded automotive apps. Reply [dfb] Andrew Wippler says: 27 Jul 21 at 2:54 Great post. I wish more would invest in containers and kubernetes on day one. Kubernetes solves complex scaling pain and makes web applications a dream to make. As an ops guy who knows how to code, I understand kubernetes. For a dev guy, learning to do ops may seem like a mountain of work to get an application up on the internet and so kubernetes is avoided. Trust me, kubernetes is as good as this post makes it. When implemented properly as this post implies, kubernetes will make your app unstoppable. Reply [cba] Duncan Krebs says: 28 Jul 21 at 5:08 Yup - that is all handy and dandy, either we are using Kubernetes in the wrong fashion or there is a paradigm mismatch in what we are trying to accomplish as a startup. First off, we totally recognize that Kubernetes is the foundation of our data analytics and machine learning platform which we are developing and aiming to sell in public cloud market places. There are just a few problems here: 1st to deploy our app stack into in a place like AWS costs thousands of dollars a month so what do we do? We have a Hybrid cloud structure where we want to have the ability to have scripts that can deploy our platform into an on-prem kubernetes cluster or a public cloud, the problem is all the little details involved which experts of Kubernetes can understand. First its hard to get your head around the platform in the first place based on the way its designed . Some of our on-prem clusters are using local file storage for PVC while cloud providers offer their own. So the time it has taken to re-engineer or re-configure off the shelf helm charts needed for the open source software we need has been horrible! Every time we spin up a new cluster we have to re-clone an environment repo, find/replace values with cluster specific nodes and then with the mix of open source and custom developed artifacts in Kubernetes it is extremely difficult to come up with a consistent way using Jenkins to build pipelines. What do you do just send over an ssh command and hope it works? How do we get to the point of being able to have scripts and automation that can deploy and scale our core product across an on-prem or public cloud provider solution? In my humble opinion there is a lot of work to do in this space and more abstraction layers are needed to make Kubernetes less painful. I totally agree its an awesome platform, it just sucks when startups have to spend their capital and time on getting things to work within it. The goal is to be cloud agnostic, have a script that can specify CPU/Storage requirements for dozens of open source products + our own stack and at the end of the day it is confusing as hell. Does anyone else feel this pain? Reply [934] Nick Torba says: 28 Jul 21 at 12:42 Hey Max! I enjoyed the accompanying podcast to this article! I agree that cloud providers do a huge part of the heavy lifting, but I think it's also important to mention that baseline cloud clusters are, for most projects, not production ready. It is easy to get the cluster up, but there is still a full stack of applications necessary to make it observable and reliable. Different apps to satisfy requirements like * montioring/metrics * centralized logging * handling external traffic (api gateway, ingress, cloud service load balancer, nodeport?) * developer permissions and deployment process * pod permissions * etc In general, I agree with you that Kubernetes still makes sense from Day 1 for a lot of teams, but even with cloud providers, it takes a significant amount of effort and continual maintenance of the cluster. Of course, the argument can be made that you need all of these things setup no matter what your deployment service/platform is, which is true, but Kubernetes seems to be a much more open space. I'd be interested in a follow up post where you lay out what it means for your team to have a production be truly production ready. A stack like this: * prometheus/grafana for monitoring/metrics * fluent-bit + elastic for centralized logging * ambassador (now named emissionary-ingress) for api gateway/ loadbalancer * cluster-autoscaler * HPA * Kube2iam (or equivalent for other cloud providers) * knative * argocd? * etc. I'd add a caveat to your stance that Kubernetes from Day 1 is worth it if you have at least one engineer who is very comfortable installing and debugging that production stack (or have the time to allow one of your engineers to dedicate their full time to learning it). Reply Leave a Reply Cancel reply Your email address will not be published. Required fields are marked * [ ] [ ] [ ] [ ] [ ] [ ] [ ] Comment [ ] Name * [ ] Email * [ ] Website [ ] [ ] Save my name, email, and website in this browser for the next time I comment. [Post Comment] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] This site uses Akismet to reduce spam. Learn how your comment data is processed. (c) 2021 All Rights Reserved. Proudly powered by Wordpress Stack Overflow Questions Jobs Developer Jobs Directory Salary Calculator Products Teams Talent Advertising Enterprise Company About Press Work Here Legal Privacy Policy Contact Us Channels Podcast Newsletter Facebook Twitter LinkedIn Instagram *