https://thenewstack.io/5-years-of-postgres-on-kubernetes/ Modal Title OK + [ ] Search More Results ARCHITECTURE Cloud Native Ecosystem Containers IoT Edge Computing Microservices Networking Serverless Storage ENGINEERING Frontend Development Software Development Cloud Services Data Machine Learning Security OPERATIONS CI/CD Culture DevOps Kubernetes Observability Service Mesh DevOps Tools CHANNELS Podcasts Ebooks Events Newsletter TNS RSS Feed THE NEW STACK About / Contact Sponsors Sponsorship Contributions Sitemap PODCASTS EBOOKS EVENTS NEWSLETTER ARCHITECTURE ENGINEERING OPERATIONS Cloud Native Ecosystem Containers IoT Edge Computing Microservices Networking Serverless Storage Defining the Modern Bare Metal Cloud Sep 22nd 2022 12:00pm, by Borko Drljaca Comparing Infrastructure-as-Code and GitOps for Platform Teams Sep 12th 2022 1:01am, by Cliff Malmborg Tutorial: Deploy a Full-Stack Application to a Docker Swarm Sep 12th 2022 1:00am, by Jack Wallen Deploy a Kubernetes Cluster on Ubuntu Server with Microk8s Sep 9th 2022 10:07am, by Jack Wallen Adobe's Internal Developer Platform Journey and Lessons Sep 7th 2022 12:33pm, by Srinivas Peri 3 Container Practices to Boost Your Security Posture Oct 6th 2022 7:38am, by Ratan Tipirneni What Is Apptainer and How Do You Install It on Ubuntu Server? Sep 30th 2022 4:00am, by Jack Wallen Tutorial: Deploy Acorn Apps on an Amazon EKS Cluster Sep 30th 2022 3:00am, by Janakiram MSV Acorn from the Eyes of a Docker Compose User Sep 27th 2022 3:00am, by Janakiram MSV Tutorial: Deploy a Full-Stack Application to a Docker Swarm Sep 12th 2022 1:00am, by Jack Wallen Shikitega: New Malware Program Targeting Linux Sep 21st 2022 9:14am, by Steven J. Vaughan-Nichols Google Inks Deal with NIST to Boost Open Source Chips Sep 14th 2022 10:36am, by Agam Shah Burning Man Returns with a Quest for Sustainability, Meaning Sep 4th 2022 5:08am, by David Cassel Enterprise Kubernetes Lessons Learned from MoneyGram and HP Sep 2nd 2022 7:00am, by Kyle Hunter Small-Scale Fab: Embrace the CAD Mindset Aug 20th 2022 6:00am, by drtorq AmeriSave Moved Its Microservices to the Cloud with Traefik's Dynamic Reverse Proxy Sep 8th 2022 2:02pm, by Ann R. Thryft Event Streaming and Event Sourcing: The Key Differences Sep 1st 2022 6:43am, by David Dieruf Lessons from Deploying Microservices for a Large Retailer Aug 30th 2022 10:01am, by Loraine Lawson Why Distributed Testing Is the Best Way to Test Microservices Aug 25th 2022 10:00am, by Aviv Kerbel Why Enterprises Need a Management Plane Aug 19th 2022 10:03am, by Eric Braun How Idit Levine's Athletic Past Fueled Solo.io's Startup Sep 16th 2022 11:41am, by Colleen Coll AmeriSave Moved Its Microservices to the Cloud with Traefik's Dynamic Reverse Proxy Sep 8th 2022 2:02pm, by Ann R. Thryft eBPF or Not, Sidecars are the Future of the Service Mesh Aug 12th 2022 10:00am, by William Morgan Is Kubernetes the Next Fault Domain? Aug 12th 2022 6:00am, by Keith McClellan Rethinking Web Application Firewalls Aug 9th 2022 12:36pm, by Alex Williams Hazelcast Aims to Democratize Real-Time Data with Serverless Oct 3rd 2022 8:06am, by Darryl K. Taft Forrester Identifies Best Practices for Serverless Development Sep 9th 2022 6:00am, by Loraine Lawson Is Kubernetes Adoption Slowing? Aug 3rd 2022 6:32am, by Mark Hinkle Early Days for Quantum Developers, But Serverless Coming Aug 1st 2022 6:00am, by Richard MacManus This Week in Programming: Honeycomb's ARM Advantage Jul 15th 2022 1:38pm, by Joab Jackson The Competitive Advantage of Postgres Oct 6th 2022 6:33am, by Ian Bain An Apache Cassandra Breakthrough: ACID Transactions at Scale Oct 5th 2022 6:00am, by Patrick McFadin Why You Need a Mobile Database Sep 30th 2022 9:16am, by Mark Gamble Deterministic Databases and the Future of Data Sharing Sep 29th 2022 6:13am, by Tim Wagner How Time Series Data Empowers Telcos to Stay Competitive Sep 28th 2022 9:53am, by Charles Mahler Frontend Development Software Development Cloud Services Data Machine Learning Security What a Low Code Platform Offers Frontend Developers Oct 6th 2022 11:00am, by Loraine Lawson How Database as a Service Boosts Frontend Development Oct 5th 2022 6:34am, by Loraine Lawson Developers: Twitter Wants Your Bots (and Other Read/Write Apps) Oct 5th 2022 3:00am, by Richard MacManus Citrix Has Gone Private. Here's What It Means for DevOps. Oct 3rd 2022 8:44am, by Alex Korolov Service Design Could've Prevented Heroku's Free Tier Closure Oct 1st 2022 6:00am, by David Eastman What a Low Code Platform Offers Frontend Developers Oct 6th 2022 11:00am, by Loraine Lawson How HashiCorp Does Site Reliability Engineering Oct 6th 2022 8:06am, by Heather Joslyn Intel CEO Sheds Light on Emerging Software Strategy Oct 6th 2022 7:14am, by Agam Shah Alpha-Omega Dishes out Cash to Secure Open Source Projects Oct 6th 2022 4:00am, by Steven J. Vaughan-Nichols Why Traditional Logging and Observability Waste Developer Time Oct 5th 2022 7:01am, by Shahar Fogel Aiven Adds Kafkawize for Data Governance of Kafka Oct 5th 2022 10:00am, by Jelani Harper How Database as a Service Boosts Frontend Development Oct 5th 2022 6:34am, by Loraine Lawson Hazelcast Aims to Democratize Real-Time Data with Serverless Oct 3rd 2022 8:06am, by Darryl K. Taft How Shifting Left Can Prepare You for Black Friday Sep 30th 2022 10:00am, by Eran Kinsbruner Tutorial: Deploy Acorn Apps on an Amazon EKS Cluster Sep 30th 2022 3:00am, by Janakiram MSV 5 Years of Postgres on Kubernetes Oct 6th 2022 10:00am, by Paul Laurence The Competitive Advantage of Postgres Oct 6th 2022 6:33am, by Ian Bain Aiven Adds Kafkawize for Data Governance of Kafka Oct 5th 2022 10:00am, by Jelani Harper Confluent Expands Stream Governance Capabilities Oct 5th 2022 8:34am, by Jessica Wachtel How Database as a Service Boosts Frontend Development Oct 5th 2022 6:34am, by Loraine Lawson This Tool Defends AI Models Against Adversarial Attacks Oct 2nd 2022 6:00am, by Kimberley Mok CUDA 12 Harnesses Nvidia's Speedier GPU Architecture Sep 29th 2022 7:32am, by Agam Shah Why Cloud IDEs Are Shifting to a Platform-as-a-Service Model Sep 26th 2022 6:53am, by Richard MacManus Seeing the Big Picture with AIOps Sep 23rd 2022 7:12am, by Ali Siddiqui Nvidia Intros Large Language Model Customization, Services Sep 21st 2022 8:28am, by Andrew Brust 3 Container Practices to Boost Your Security Posture Oct 6th 2022 7:38am, by Ratan Tipirneni Alpha-Omega Dishes out Cash to Secure Open Source Projects Oct 6th 2022 4:00am, by Steven J. Vaughan-Nichols HashiCorp Cloud Can Now Spin up a Single Sign-on Zero Trust Network Oct 5th 2022 9:00am, by Joab Jackson 5 Application Authorization Best Practices for Better Cybersecurity Oct 4th 2022 6:45am, by Ash Narkar New Ebook: A Blueprint for Supply Chain Security Oct 4th 2022 6:08am, by Celeste Malia CI/CD Culture DevOps Kubernetes Observability Service Mesh DevOps Tools Terraform Cloud Now Offers Less Code -- and No Code -- Options Oct 6th 2022 9:00am, by Joab Jackson Unleashing Git for the Game Development Industry Oct 5th 2022 2:45pm, by Yuki Arbel The Future Stack of Code Review Oct 4th 2022 1:00pm, by Gabriel Manor-Liechtman New Ebook: A Blueprint for Supply Chain Security Oct 4th 2022 6:08am, by Celeste Malia Why You Can't Afford to Miss Data Decisions Oct 3rd 2022 4:00am, by B. Cameron Gain Is the Startup Grind Really Necessary? Oct 6th 2022 10:36am, by Emily Omier How HashiCorp Does Site Reliability Engineering Oct 6th 2022 8:06am, by Heather Joslyn The Future Stack of Code Review Oct 4th 2022 1:00pm, by Gabriel Manor-Liechtman Platform Engineering: The Path to DevOps Success Oct 3rd 2022 6:57am, by Michael Mueller Why Jump from the Safety of a Big Tech Job into Risky Web3? Sep 29th 2022 6:00am, by Jake Ludington Terraform Cloud Now Offers Less Code -- and No Code -- Options Oct 6th 2022 9:00am, by Joab Jackson Unleashing Git for the Game Development Industry Oct 5th 2022 2:45pm, by Yuki Arbel HashiCorp Is Standardizing and Industrializing the Cloud Oct 5th 2022 2:14pm, by Alex Williams HashiCorp Cloud Can Now Spin up a Single Sign-on Zero Trust Network Oct 5th 2022 9:00am, by Joab Jackson 5 Application Authorization Best Practices for Better Cybersecurity Oct 4th 2022 6:45am, by Ash Narkar 5 Years of Postgres on Kubernetes Oct 6th 2022 10:00am, by Paul Laurence How to Monitor Kubernetes K3s Using Telegraf and InfluxDB Cloud Oct 4th 2022 10:42am, by Ignacio Van Droogenbroeck Constellation: The First Confidential Kubernetes Distribution Oct 3rd 2022 10:00am, by Felix Schuster The Hidden Pain of DIY On-Premises K8s-Based Software Distribution Sep 30th 2022 6:47am, by Nikki Rouda What Is Apptainer and How Do You Install It on Ubuntu Server? Sep 30th 2022 4:00am, by Jack Wallen How HashiCorp Does Site Reliability Engineering Oct 6th 2022 8:06am, by Heather Joslyn Why Traditional Logging and Observability Waste Developer Time Oct 5th 2022 7:01am, by Shahar Fogel How Shifting Left Can Prepare You for Black Friday Sep 30th 2022 10:00am, by Eran Kinsbruner Trivago Cracks the Case of the Vanishing Performance Improvements Sep 28th 2022 9:00am, by Jessica Wachtel Seeing the Big Picture with AIOps Sep 23rd 2022 7:12am, by Ali Siddiqui AmeriSave Moved Its Microservices to the Cloud with Traefik's Dynamic Reverse Proxy Sep 8th 2022 2:02pm, by Ann R. Thryft Can You Now Safely Remove the Service Mesh Sidecar? Sep 8th 2022 9:19am, by B. Cameron Gain eBPF or Not, Sidecars are the Future of the Service Mesh Aug 12th 2022 10:00am, by William Morgan My Istiod Pod Can't Communicate with the Kubernetes API Server! Aug 10th 2022 10:00am, by Lin Sun How to Handle Authorization in a Service Mesh Jul 8th 2022 12:29pm, by Judith Kahrer How IaC Meets the Differing Infrastructure Needs of Dev and Ops Oct 6th 2022 8:29am, by Donna Lee Unleashing Git for the Game Development Industry Oct 5th 2022 2:45pm, by Yuki Arbel HashiCorp Is Standardizing and Industrializing the Cloud Oct 5th 2022 2:14pm, by Alex Williams Citrix Has Gone Private. Here's What It Means for DevOps. Oct 3rd 2022 8:44am, by Alex Korolov First Line of Defense: Developer Security Tools in the IDE Sep 27th 2022 12:56pm, by Steven Zimmerman 2022-10-06 10:00:32 5 Years of Postgres on Kubernetes contributed, Data / Kubernetes 5 Years of Postgres on Kubernetes Building a highly available, self-healing Postgres cluster is hard. You have to think about things like backups, load balancing across databases, metrics, database hosts changing, storage and correctly sizing all of these services. Oct 6th, 2022 10:00am by Paul Laurence Featued image for: 5 Years of Postgres on Kubernetes Image via Pixabay. The idea of a Postgres operator in Kubernetes was born out of a conversation with members of the community during 2016. This was in the very early days of Kubernetes, and the idea of running databases in containers was controversial. Today there is a constellation of tools and techniques one can use to run stable, highly available and performant Postgres workloads in your Kubernetes clusters including operators from companies like Crunchy Data. Across the industry, we are seeing a growing number of folks who are leveraging the operator architectural pattern successfully for a disparate set of workloads. My customer experience, and that with the larger community, has given me a nuanced view into the world of Postgres in Kubernetes. Today, I'd like to more broadly share these lessons and give insights into lessons learned after five years of Postgres on Kubernetes. The Infrastructure Is the Code From the beginning, the concept behind operators was that they would functionally automate your Kubernetes applications. This means that Infrastructure-as-Code was realized in the Kubernetes space. This has been the DevOps dream for a long time -- a platform that is self-healing, version controlled, self-arranging and defined declaratively. As Kubernetes evolved and the APIs matured, delivering a fully declarative Postgres operator in Kubernetes became an increasingly attainable objective. We also began hearing from our users and customers that they wanted to see the project adopt a more declarative approach. Fortunately, this is something we had been thinking about as well. This became the key driver behind the fifth major version of Crunchy Postgres for Kubernetes. While we had several ideals in mind while designing this version of the Postgres Operator, our primary objective was to make Crunchy Postgres for Kubernetes fully declarative. This was no small task; essentially, it was a ground-up rebuild. Starting with the latest modern Kubernetes functionality, we were able to develop a new version of Crunchy Postgres for Kubernetes that enabled users to simply describe the end result they were after and Crunchy Postgres for Kubernetes would do the rest. Start with Storage Databases are different from stateless apps in that they need persistent storage. Kubernetes provides powerful automation and many interesting capabilities, but we knew we needed a way to provide consistent and stateful storage, since storage is one of the cornerstones of database functionality. Storage needs are what ultimately led us to coupling Kubernetes technology with an operator. Today, databases running on Kubernetes can make use of a range of storage types like HostPath, Network File System (NFS) and Dynamic storage. Crunchy Postgres for Kubernetes is storage agnostic: It works with any supported Kubernetes storage system. You must provision persistent volumes for both Hostpath and NFS, though automated storage provisioners do exist. Dynamic storage classes allow users to request persistent volume claims and have the persistent volume created for you. There are many providers of dynamic storage classes to choose from. You will need to configure what works for your environment, and size the Physical Volumes, Persistent Volumes (PVs). Postgres Operator, the Easy Button for Postgres in Kubernetes Early on, the operator pattern for Kubernetes was unknown territory. Running stateful processes such as databases in containers was a very new concept. The operator pattern demonstrated the potential to simplify the operation and administration of databases on Kubernetes. From the moment our team heard the pitch from CoreOS, we saw the promise. There was quite a bit of debate in the early days as to the appropriate use of the operator pattern and whether the operator pattern had staying power. A number of folks in the early days asked questions about the trade-offs between Helm charts and operators. Spoiler alert: They are different and you may want both! Building a highly available, self-healing Postgres cluster is hard. You have to think about things like backups, load balancing across databases, metrics, database hosts changing, storage and correctly sizing all of these services. You also have to ensure that these services are running all the time. You need to consider how to handle version upgrades, security patches and much more. Crunchy Postgres for Kubernetes was one of the very first operators for Postgres on Kubernetes, and the depth of experience shows in the easy setup and highly resilient architecture. Giving folks the ability to create Postgres clusters with high availability and self-healing built in is the primary goal of the Postgres for Kubernetes team at Crunchy Data; however, we would not be able to do it alone. Leaning into APIs Kubernetes has rapidly evolved since the early days, and so has its APIs. The underlying APIs routinely change between versions to enhance new capabilities or to consolidate capabilities. Crunchy Postgres for Kubernetes has evolved alongside the Kubernetes APIs, and you see the culmination of those years of work in the fifth major version of the Postgres Operator. By using an operator's API capabilities, teams feel the real power of Kubernetes. These APIs are capable of many tasks like deploying, adding users, recovering from failed health checks, running backups, recovering from backups, etc. When using these APIs, teams start leveling up. CI/CD functions become far more repeatable across teams, deploying staging is an automated instead of a manual process, and as a result every database runs uniformly instead of as an edge case. To empower this functionality, use of the operator must be integrated into many other systems. For example, we have customers with API-level integrations with their tickets systems, deployment systems, analytics teams, marketing teams, operations teams and code repositories. Thus it is essential that Crunchy Postgres for Kubernetes has stable APIs and is supported by a team with compatibility in mind. Deploying an operator with fluid APIs can quickly become a liability at all points of integration. Looking to the Future I believe that our initial support of the operator pattern for managing stateful services like Postgres in Kubernetes was a good choice. As Kubernetes has matured, the operator pattern and APIs have evolved to strengthen this method, and our Postgres Operator has matured alongside the community. Early on, we felt a groundswell of support and appreciate the collaboration and community that formed around this idea that we had so many years ago. We are excited for what we see in the future and for the ideas we have for further maturation of the Operator pattern and implementation of that pattern. Created with Sketch. [76081a45-p] Paul Laurence is the co-founder at Crunchy Data, the leading provider of trusted open source Postgres. Read more from Paul Laurence Share this story RELATED STORIES Why Unsuspecting Data Leaks Are a Key to Rampant Blockchain Hacks Why You Can't Afford to Miss Data Decisions 3 Trade-offs to Consider When Deploying Apache Kafka in the Cloud Hazelcast Aims to Democratize Real-Time Data with Serverless Why Choose a NoSQL Database? There Are Many Great Reasons Share this story RELATED STORIES Why You Need a Mobile Database Why Choose a NoSQL Database? There Are Many Great Reasons Aiven Adds Kafkawize for Data Governance of Kafka The Competitive Advantage of Postgres Starburst Strengthens Data Governance Capabilities THE NEW STACK UPDATE A newsletter digest of the week's most important stories & analyses. [ ] SUBSCRIBE We don't sell or share your email. By continuing, you agree to our Terms of Use and Privacy Policy. ARCHITECTURE Cloud Native Ecosystem Containers IoT Edge Computing Microservices Networking Serverless Storage ENGINEERING Frontend Development Software Development Cloud Services Data Machine Learning Security OPERATIONS CI/CD Culture DevOps Kubernetes Observability Service Mesh DevOps Tools CHANNELS Podcasts Ebooks Events Newsletter TNS RSS Feed THE NEW STACK About / Contact Sponsors Sponsorship Contributions Sitemap (c) The New Stack 2022 Disclosures Terms of Use Privacy Policy Cookie Policy