https://thenewstack.io/devs-need-system-design-tools-not-diagramming-tools/ TNS OK SUBSCRIBE Join our community of software engineering leaders and aspirational developers. Always stay in-the-know by getting the most important news and exclusive content delivered fresh to your inbox to learn more about at-scale software development. EMAIL ADDRESS REQUIRED [ ] SUBSCRIBE RESUBSCRIPTION REQUIRED It seems that you've previously unsubscribed from our newsletter in the past. Click the button below to open the re-subscribe form in a new tab. When you're done, simply close that tab and continue with this form to complete your subscription. RE-SUBSCRIBE The New Stack does not sell your information or share it with unaffiliated third parties. By continuing, you agree to our Terms of Use and Privacy Policy. Welcome and thank you for joining The New Stack community! Please answer a few simple questions to help us deliver the news and resources you are interested in. FIRST NAME REQUIRED [ ] LAST NAME REQUIRED [ ] COMPANY NAME REQUIRED [ ] COUNTRY REQUIRED [Select ... ] ZIPCODE REQUIRED [ ] Great to meet you! Tell us a bit about your job so we can cover the topics you find most relevant. What is your job level? REQUIRED [Select ... ] [ ] Which of these most closely describes your job role? REQUIRED [Select ... ] [ ] How many employees are in the organization you work with? REQUIRED [Select ... ] What option best describes the type of organization you work for? REQUIRED [Select ... ] [ ] Which of the following best describes your organization's primary industry? REQUIRED [Select ... ] [ ] LINKEDIN PROFILE URL REQUIRED [ ] Welcome! We're so glad you're here. You can expect all the best TNS content to arrive Monday through Friday to keep you on top of the news and at the top of your game. What's next? Check your inbox for a confirmation email where you can adjust your preferences and even join additional groups. Follow TNS on your favorite social media networks. Become a TNS follower on LinkedIn. Check out the latest featured and trending stories while you wait for your first TNS newsletter. PREV 1 of 2 NEXT VOXPOP Greatest DevSec Challenge at Your Org Which of the following has the biggest negative impact on collaboration across development and security teams where you work? Lack of role definition for development teams. 0% Lack of role communication between teams. 0% Competing priorities. 0% Lack of clarity for security roles on teams. 0% Not standardized reporting structure for development teams. 0% Thanks for your opinion! Subscribe below to get the final results, published exclusively in our TNS Update newsletter: [ ] SUBMIT [ ] Search More Results ARCHITECTURE Open Source Cloud Native Ecosystem Containers Edge Computing Microservices Networking Serverless Storage ENGINEERING AI Large Language Models Frontend Development Software Development API Management Python JavaScript TypeScript WebAssembly Cloud Services Data Security OPERATIONS Platform Engineering Operations CI/CD Tech Careers Tech Culture DevOps Kubernetes Observability Service Mesh CHANNELS Podcasts Ebooks Events Newsletter TNS RSS Feeds THE NEW STACK About / Contact Sponsors Sponsorship Contributions PODCASTS EBOOKS EVENTS NEWSLETTER CONTRIBUTE ARCHITECTURE ENGINEERING OPERATIONS Open Source Cloud Native Ecosystem Containers Edge Computing Microservices Networking Serverless Storage What's New With the Just-Released Linux 6.10 Kernel Jul 16th 2024 9:10am, by Steven J. Vaughan-Nichols Make Pgvector Faster Than Pinecone and 75% Cheaper With This New Open Source Extension Jul 14th 2024 10:00am, by Avthar Sewrathan Use Your Data in LLMs With the Vector Database You Already Have Jul 12th 2024 7:50am, by Chris Carter This Is How To Optimize PyTorch for Faster Model Training Jul 11th 2024 10:00am, by Hope Wang Open Source AI: What About Data Transparency? Jul 10th 2024 9:09am, by Steven J. Vaughan-Nichols Kubernetes Autoscaling: Q&A With Fairwinds CTO Andy Suderman Jul 5th 2024 7:00am, by Raghavan "Rags" Srinivas Phantom Secrets: The Hidden Threat in Code Repositories Jul 1st 2024 7:30am, by Jeffrey Burt Canonical Offers LTS 'Distroless' Containerized Apps for K8s Jun 27th 2024 10:41am, by Joab Jackson Cloud Reliability Requires Runtime Security, Zero Trust Jun 20th 2024 9:37am, by Manas Chowdhury Talent Shortages Shouldn't Kill Your Cloud Native Journey Jun 20th 2024 8:37am, by B. Cameron Gain Exploring MicroOS, OpenSUSE's Immutable Container OS Jul 5th 2024 6:00am, by Jack Wallen Container Image Fault Lines Are Being Exposed Jul 3rd 2024 8:02am, by Kim Lewandowski and Adrian Mouat Kubernetes Security Report: Evolving Landscape of DevSecOps Jun 27th 2024 11:53am, by Alex Handy Canonical Offers LTS 'Distroless' Containerized Apps for K8s Jun 27th 2024 10:41am, by Joab Jackson PyTorch, Docker and AI Openness Highlight AI_dev Europe Jun 27th 2024 6:39am, by Joe Fay How to Apply Microservice Architecture to Embedded Systems Jul 16th 2024 10:48am, by Bob Reselman Why We Chose WebAssembly (Wasm) for Our Edge Runtime May 28th 2024 6:50am, by Bogdan Kurnosov Cloud Computing at the Edge: From Evolution to Disruption May 16th 2024 10:00am, by Yoram Novick Ambient AI? Humane's 'Ai Pin' Embarks on a Dream's Long Road Apr 21st 2024 6:00am, by David Cassel Chipmakers Putting a Laser Focus on Edge AI Apr 12th 2024 10:06am, by Jeffrey Burt How to Apply Microservice Architecture to Embedded Systems Jul 16th 2024 10:48am, by Bob Reselman The 5 Worst Anti-Patterns in API Management Jun 26th 2024 10:00am, by Emile Vauge Composable Enterprise: The Evolution of MACH and Jamstack May 20th 2024 5:00am, by Jeff James Why Shift Testing Left Part 2: QA Does More After Devs Run Tests May 15th 2024 9:12am, by Nocnica Mellifera Red Hat Podman 'Lab' Gets Developers Started on GenAI May 15th 2024 7:08am, by Joab Jackson Using ngrok in Production: Not Just for Testing Anymore Jul 10th 2024 8:10am, by Scott M. Fulton III Install NordVPN on Linux for an Added Layer of Security Jul 3rd 2024 7:00am, by Jack Wallen Layer 8: A Semantic Networking Layer for the Age of AI Jul 2nd 2024 6:41am, by Liam Crilly What's the Future of Distributed Ledgers? Jul 2nd 2024 6:00am, by Heather Joslyn 50 Years Later, Vint Cerf and Bob Kahn Remember the Birth of TCP/IP Jun 19th 2024 6:00am, by David Cassel WebAssembly and Kubernetes Go Better Together: Matt Butcher May 22nd 2024 2:00pm, by Raghavan "Rags" Srinivas How to Conquer Cold Starts for Better Performance Apr 25th 2024 9:17am, by Henry Hamid Safi Meet DBOS: A Database Alternative to Kubernetes Mar 12th 2024 4:00am, by Joab Jackson Pulumi Templates for GenAI Stacks: Pinecone, LangChain First Feb 21st 2024 9:00am, by Joab Jackson CNCF CloudEvents: A Li'l Message Envelope That Travels Far Jan 31st 2024 4:00am, by Joab Jackson Linux: Synchronize Local and Remote Directories With Rsync Jul 13th 2024 5:00am, by Jack Wallen Showdown at the Lakehouse: Databricks Muscles Up With Tabular Jul 11th 2024 1:23pm, by Joab Jackson The Complexity of Solving Performance Problems Jul 11th 2024 7:08am, by Cynthia Dunlop Designing a Data Mesh To Reign in Data Sprawl Jul 8th 2024 9:22am, by David A. Ventimiglia How to Manage Linux Storage Jul 7th 2024 6:00am, by Damon M. Garn AI Large Language Models Frontend Development Software Development API Management Python JavaScript TypeScript WebAssembly Cloud Services Data Security SRE Hype Cycle: Where AI Meets 'Everything as Code' Jul 16th 2024 11:06am, by Ido Neeman Automating Context in Structured Data for LLMs Jul 16th 2024 5:00am, by Susan Hall AI-Assisted Coding: A Double-Edged Sword for Security Jul 15th 2024 6:37am, by B. Cameron Gain Amazon Q Apps: AI-Powered Development for All Jul 12th 2024 11:35am, by Jeffrey Burt Use Your Data in LLMs With the Vector Database You Already Have Jul 12th 2024 7:50am, by Chris Carter Demos: Deploying LLMs With WasmEdge Jul 16th 2024 8:28am, by Heather Joslyn Automating Context in Structured Data for LLMs Jul 16th 2024 5:00am, by Susan Hall Designing LLM Chat Interfaces to Leverage Heterogeneous Data Sources Jul 12th 2024 10:00am, by Deepak Jayablalan Use Your Data in LLMs With the Vector Database You Already Have Jul 12th 2024 7:50am, by Chris Carter Goodbye Manual Prompting, Hello Programming With DSPy Jul 10th 2024 10:23am, by Usama Jamil How JS Meta-Framework SolidStart Became Router Agnostic Jul 15th 2024 9:48am, by Loraine Lawson Introducing the Z.Js Framework and Mesop, a Python Framework Jul 13th 2024 4:00am, by Loraine Lawson 3 Key Trends for AI Engineering in the Cloud in 2024 Jul 10th 2024 8:03am, by Dan Rowinski DataStax Aims To Simplify Building AI Apps With RAGStack Jul 10th 2024 5:00am, by Loraine Lawson Why Ruby on Rails Is Still Worth Your While as a Developer Jul 6th 2024 7:00am, by David Eastman APIs Are Driving New Business Models and Unlocking Revenue Streams Jul 16th 2024 10:00am, by Josh Twist Devs Need System Design Tools, Not Diagramming Tools Jul 15th 2024 10:00am, by Thomas Johnson Here's Why a Hosted API Gateway Is Always Better Than Building Your Own Jul 13th 2024 10:00am, by Josh Twist Amazon Q Apps: AI-Powered Development for All Jul 12th 2024 11:35am, by Jeffrey Burt Designing LLM Chat Interfaces to Leverage Heterogeneous Data Sources Jul 12th 2024 10:00am, by Deepak Jayablalan APIs Are Driving New Business Models and Unlocking Revenue Streams Jul 16th 2024 10:00am, by Josh Twist Solving API Integration and Aggregation With Supergraph Jul 16th 2024 9:45am, by Sandip Devarkonda Here's Why a Hosted API Gateway Is Always Better Than Building Your Own Jul 13th 2024 10:00am, by Josh Twist Introducing the Z.Js Framework and Mesop, a Python Framework Jul 13th 2024 4:00am, by Loraine Lawson 4 API Security Best Practices Jul 12th 2024 5:50am, by Judith Kahrer Automate Routine Tasks With an Ad-Hoc Ansible Script Jul 15th 2024 11:31am, by Adetokunbo Ige Introducing the Z.Js Framework and Mesop, a Python Framework Jul 13th 2024 4:00am, by Loraine Lawson What Is the NumPy Python Library and How Do You Use It? Jul 12th 2024 6:00am, by Jack Wallen This Is How To Optimize PyTorch for Faster Model Training Jul 11th 2024 10:00am, by Hope Wang Python Meets JavaScript, Wasm With the Magic of PythonMonkey Jul 11th 2024 9:37am, by Darryl K. Taft How JS Meta-Framework SolidStart Became Router Agnostic Jul 15th 2024 9:48am, by Loraine Lawson Introducing the Z.Js Framework and Mesop, a Python Framework Jul 13th 2024 4:00am, by Loraine Lawson Python Meets JavaScript, Wasm With the Magic of PythonMonkey Jul 11th 2024 9:37am, by Darryl K. Taft Next.js 15 Cache, Rust Adds to AI Salaries, and Million.js Jul 6th 2024 5:00am, by Loraine Lawson ECMAScript Specs Approved, and How Google Sheets Used WasmGC Jun 29th 2024 4:59am, by Loraine Lawson TypeScript 5.5: Faster, Smarter and More Powerful Jun 25th 2024 6:41am, by Darryl K. Taft UIX: A Full Stack Web Dev Framework Leveraging Deno Jun 11th 2024 11:30am, by Loraine Lawson TanStack Introduces New Meta-Framework Based on Its Router Jun 5th 2024 11:38am, by Loraine Lawson In a TypeScript World, Code Generation Is Key for API SDKs May 8th 2024 4:00am, by Richard MacManus TypeScript Meets API Design in Microsoft's Game-Changing TypeSpec May 7th 2024 7:30am, by Darryl K. Taft Demos: Deploying LLMs With WasmEdge Jul 16th 2024 8:28am, by Heather Joslyn Introduction to MoonBit, a New Language Toolchain for Wasm Jul 13th 2024 6:00am, by David Eastman Python Meets JavaScript, Wasm With the Magic of PythonMonkey Jul 11th 2024 9:37am, by Darryl K. Taft Can WebGPU and WebAssembly Overcome Docker's AI GPU Problem? Jul 8th 2024 6:04am, by B. Cameron Gain ECMAScript Specs Approved, and How Google Sheets Used WasmGC Jun 29th 2024 4:59am, by Loraine Lawson Amazon Q Apps: AI-Powered Development for All Jul 12th 2024 11:35am, by Jeffrey Burt The Architect's Guide to the New Private Cloud Jul 9th 2024 10:13am, by Ugur Tigli The Rise of Intelligent Cloud Jul 9th 2024 10:00am, by Subash Natarajan How To Reduce Cloud Waste Jul 2nd 2024 8:14am, by Simon Ritter 5 Myths About Zero Trust in the Cloud, Busted Jun 27th 2024 5:03am, by Manas Chowdhury Automating Context in Structured Data for LLMs Jul 16th 2024 5:00am, by Susan Hall Use Your Data in LLMs With the Vector Database You Already Have Jul 12th 2024 7:50am, by Chris Carter Showdown at the Lakehouse: Databricks Muscles Up With Tabular Jul 11th 2024 1:23pm, by Joab Jackson Goodbye Manual Prompting, Hello Programming With DSPy Jul 10th 2024 10:23am, by Usama Jamil Designing a Data Mesh To Reign in Data Sprawl Jul 8th 2024 9:22am, by David A. Ventimiglia Want To Mitigate Risk? Invest in Automation Jul 15th 2024 7:27am, by Mandi Walls AI-Assisted Coding: A Double-Edged Sword for Security Jul 15th 2024 6:37am, by B. Cameron Gain Linux: How File Permissions Work Jul 14th 2024 4:00pm, by Damon M. Garn Linux: Understand sudo to Rule Your Server Jul 14th 2024 6:00am, by Jack Wallen Why We Use Apache Kafka for Real-Time Data at Scale Jul 12th 2024 8:47am, by Brandon Brown and Jared Smith Platform Engineering Operations CI/CD Tech Careers Tech Culture DevOps Kubernetes Observability Service Mesh Platform Teams: Automate Infrastructure Requirement Gathering Jul 16th 2024 1:06pm, by Rak Siva SRE Hype Cycle: Where AI Meets 'Everything as Code' Jul 16th 2024 11:06am, by Ido Neeman How To Put Developer Survey Results Into Action Jul 16th 2024 7:53am, by Sooraj Shah What's the Impact of Platform Engineering? Jul 13th 2024 8:05am, by Heather Joslyn The Hidden Benefits of Internal Developer Platforms Jul 11th 2024 8:49am, by Tiago Barbosa Platform Teams: Automate Infrastructure Requirement Gathering Jul 16th 2024 1:06pm, by Rak Siva Automating Context in Structured Data for LLMs Jul 16th 2024 5:00am, by Susan Hall How Amazon Matches Power Needs to Green Energy Sources Jul 15th 2024 11:58am, by David Cassel Automate Routine Tasks With an Ad-Hoc Ansible Script Jul 15th 2024 11:31am, by Adetokunbo Ige Want To Mitigate Risk? Invest in Automation Jul 15th 2024 7:27am, by Mandi Walls Solving API Integration and Aggregation With Supergraph Jul 16th 2024 9:45am, by Sandip Devarkonda Optimize Your 'Inner Dev Loop' to Increase Developer Velocity Jul 10th 2024 7:29am, by Matt Voget Python for Automating APIs: Create a Trivia Quiz CSV File Jul 8th 2024 7:45am, by Teri Eyenike 5 Clean Code Tips for Reducing Cognitive Complexity Jul 3rd 2024 8:00am, by John Clifton DevOps Isn't Dead, but It's Not in Great Health Either Jul 2nd 2024 7:55am, by Steven J. Vaughan-Nichols How To Put Developer Survey Results Into Action Jul 16th 2024 7:53am, by Sooraj Shah What Devs Really Want in an Internal Developer Portal Jul 9th 2024 7:01am, by Sooraj Shah Talent Shortages Shouldn't Kill Your Cloud Native Journey Jun 20th 2024 8:37am, by B. Cameron Gain Engineering Managers in 2024: Burnout and More Duties Jun 17th 2024 10:00am, by Lawrence E Hecht and Heather Joslyn DevOps Reveals: Trust Your Team for Better Results Jun 11th 2024 6:17am, by Andy Corrigan Devs Need System Design Tools, Not Diagramming Tools Jul 15th 2024 10:00am, by Thomas Johnson How Much Energy Is Really Being Consumed by Data Centers? Jul 9th 2024 8:47am, by David Cassel The Python Linter Ruff Is a Win for Open Source -- and Rust Jul 8th 2024 7:00am, by David Cassel Apple's Open Source Roots: The BSD Heritage Behind macOS and iOS Jul 8th 2024 5:00am, by Jason Perlow Beat Developer Burnout: How the Right Platform Makes a Difference Jul 4th 2024 10:00am, by John Wall Platform Teams: Automate Infrastructure Requirement Gathering Jul 16th 2024 1:06pm, by Rak Siva The Hidden Benefits of Internal Developer Platforms Jul 11th 2024 8:49am, by Tiago Barbosa 8 Real-World Reasons To Adopt Platform Engineering Jul 11th 2024 6:26am, by Steve Fenton DevOps Embraces Observability Across Stacks for LLM Era Jul 10th 2024 6:50am, by B. Cameron Gain Your Platform Engineering Toolkit for Terraform and Beyond Jul 10th 2024 6:14am, by Rak Siva Using ngrok in Production: Not Just for Testing Anymore Jul 10th 2024 8:10am, by Scott M. Fulton III Mirantis Builds on OpenStack To Serve Up a VMware Alternative Jul 9th 2024 1:13pm, by Joab Jackson Hunting the Stealth Hacker in the Kubernetes Jungle Jul 8th 2024 10:09am, by Steven J. Vaughan-Nichols Kubernetes Autoscaling: Q&A With Fairwinds CTO Andy Suderman Jul 5th 2024 7:00am, by Raghavan "Rags" Srinivas Kubernetes Security Report: Evolving Landscape of DevSecOps Jun 27th 2024 11:53am, by Alex Handy DevOps Embraces Observability Across Stacks for LLM Era Jul 10th 2024 6:50am, by B. Cameron Gain Hunting the Stealth Hacker in the Kubernetes Jungle Jul 8th 2024 10:09am, by Steven J. Vaughan-Nichols Zero Trust Security for Distributed Applications with Dapr Open Source Jul 5th 2024 10:00am, by Alice Gibbons Elastic Profiling Agent Offers a 4th Pillar of Observability Jul 5th 2024 7:03am, by B. Cameron Gain Datadog Brings Big Observability Directly to Your Phone Jul 3rd 2024 7:35am, by B. Cameron Gain Enhancing Business Security and Compliance with Service Mesh Apr 1st 2024 10:00am, by Ninad Desai Some Linkerd Users Must Pay: Fear and Anger Explained Feb 28th 2024 9:21am, by B. Cameron Gain Buoyant Revises Release Model for the Linkerd Service Mesh Feb 21st 2024 9:30am, by Joab Jackson Istio Advisor Plus GPT: Expert System Meets AI for Service Mesh Dec 14th 2023 12:15pm, by Steven J. Vaughan-Nichols Using JWTs to Authenticate Services Unravels API Gateways Nov 8th 2023 6:53am, by Christian Posta and Peter Jausovec 2024-07-15 10:00:35 Devs Need System Design Tools, Not Diagramming Tools contributed, Software Development / Software Testing / Tech Culture Devs Need System Design Tools, Not Diagramming Tools Navigating The XY Problem: Reframing Questions to Solve the Right Problems. Jul 15th, 2024 10:00am by Thomas Johnson Featued image for: Devs Need System Design Tools, Not Diagramming Tools Image by Jan from Pixabay. When engineering teams select tools for managing their software systems, particularly for design and visualization, they often encounter the XY problem. The XY problem arises when someone attempts to solve problem X using solution Y but faces challenges. Instead of seeking help for problem X, they request support for solution Y, obscuring the root cause and leading to miscommunication and suboptimal solutions. Here's a concrete example from the XY problem website: [ How can I echo] How can I echo the last three characters in a filename? 1 2 If they're in a variable: echo ${foo: -3} 3 4 Why 3 characters? What do you REALLY want? 5 6 Do you want the extension? 7 8 Yes. 9 10 There's no guarantee that every filename will have a 11 three-letter extension, 12 13 so blindly grabbing three characters does not solve the 14 problem. 15 echo ${foo##*.} In their system architecture or visualizing its components, they often ask: * Which diagramming tool should we use to map our system architecture? * How do we share and store architectural decision records? * Where can we list all of the APIs in our system? These questions, while valid, focus on Y -- the proposed solution -- rather than X -- the actual pain points the team aims to address. To uncover the root cause, these questions should be reframed as follows: * We need to ship functional software on a schedule, so we need an easy way to visualize and access up-to-date information about our system architecture. * We need to reach a consensus on the system design effortlessly and have a single source of truth for decision records. * We need to understand the system behavior and catch breaking changes before they occur. Unfortunately, many teams don't realize they are focusing on Y instead of X. This leads them to adopt a fragmented approach to system design, implementing various solutions for individual tasks. They might use one tool for sketching, another for system architecture, and another for sequence diagrams. System requirements and design decisions are scattered across Google Docs, Jira, Linear, Slack, email, Confluence, and more. APIs might be listed in spreadsheets or specialized tools. This approach results in significant time and effort spent maintaining these resources, searching for relevant information, and unnecessary context switching. System Design vs. System Architecture Diagrams System design is often mistakenly equated with simply drawing diagrams of software architecture. Another misconception is to align it solely with BDUF (Big Design Up Front), UML (Unified Modeling Language), specific architectural frameworks like TOGAF, or various documentation types such as HLD (High-Level Design), SAD (Software Architecture Document), KDD (Key Design Decisions), ARD (Architecture Requirements Document), LLD (Low-Level Design), and ADR (Architecture Decision Record). System design transcends any specific tool or document; it is an ongoing process that outlines the high-level conceptual structure of a complex system (the system architecture) and defines its significant components and interactions. It encompasses all aspects of the system (i.e., software, hardware, data, interfaces, and user interactions) to ensure that they work together effectively and efficiently to meet the application's requirements. Outputs of this process may include: * System Requirements Documentation (i.e., Detailing both functional and non-functional requirements.) * System Architecture Documentation (i.e., Describing the goals, constraints, and rationale behind the chosen architecture.) * System Architecture Diagram (i.e., Providing a visual representation of components, services, their interactions, and relationships.) However, system design should focus on upfront and continuous system design reviews that continually record and revisit the system requirements, decisions, trade-offs, and compromises. It's an essential step in software development to assess a system's technical feasibility, functionality, and performance and identify dependencies and risks to make informed decisions. Relegating system design to merely producing diagrams or documents risks overlooking critical information and fosters inefficient practices within engineering teams. This ultimately leads to accruing architectural technical debt and burdening teams with manual tasks and ineffective resources. Nowadays, Diagrams Are Not Enough Developers often use diagrams to tackle a fundamental communication challenge: conveying the complexities of a distributed software system -- including its components, dependencies, and APIs -- clearly and effectively to a distributed team. Software development is inherently collaborative, requiring a shared understanding of the system's construction, constraints, and future evolution. This alignment is crucial for eliminating ambiguity and enabling cohesive team progress. Visual illustrations are powerful tools for achieving this alignment, as they help bypass the potential misunderstandings and delays inherent in written and verbal communication, ensuring that mental models are aligned across the team. However, diagramming tools have significant limitations. Effective diagrams are designed to tell a specific story to a particular audience, providing a static snapshot of a software system that addresses the concerns of specific stakeholders (e.g., Backend team, Frontend team, QA, PMs, DevOps, C-suite). In the past, when software systems were smaller and more straightforward, a single diagram could (potentially) capture all necessary information. Today, with the proliferation of SaaS, APIs, composable platforms, and legacy systems, the complexity of software systems has exponentially increased. "Software tech stacks today look way more like a rainforest -- with animals and plants co-existing, competing, living, dying, growing, interacting in unplanned ways -- than like a planned garden." -- Jean Yang. We've all seen diagrams that attempt to fit the entire architecture of these modern software systems into a single picture. Ironically, these systems would benefit the most from effectively communicating their complexity. [bc084eea-picture1] Microservices "death star discussed in the guide Microservices. As systems scale and team requirements evolve, the limitations of traditional diagramming tools become more apparent: * Lack of Real-Time Updates: Diagrams provide a static representation and cannot automatically reflect the dynamic nature of modern systems. * Clunky User Interface: Updating diagrams can be cumbersome, with significant time formatting and arranging components. * Version Control Issues: Maintaining updated versions across teams is challenging. * Limited Collaboration Capabilities: Real-time collaboration and feedback often need better support. * There is no single source of Truth. Diagrams rarely include information about requirements, trade-offs, design decisions, etc. Design documentation is often used. * Incapable of Managing Cloud Resources: Diagrams do not control or generate infrastructure code (e.g., CloudFormation, Terraform). These limitations highlight that diagramming tools were not designed to handle the complexities of modern software systems and their architectural evolution. Engineering teams today need tools that embrace complexity and support dynamic system design, exceeding the capabilities of traditional diagramming tools. While developers can build bigger systems thanks to all the new technologies available, they now face the issue of managing and coordinating these fast-moving, constantly evolving, pieced-together systems. In most of these real-world, messy tech stacks, developers must integrate new code with legacy code while continuously evaluating when and how to modernize existing components. A lack of visibility and understanding of their software systems creates team bottlenecks, slows development, and results in fragile, inflexible systems. The typical response to manage this complexity is to seek higher levels of abstraction. However, simplifying things away is only sometimes the best solution. Some problems cannot be automated, and developers must gather the appropriate information to provide tailored input on how to solve them. For instance, consider debugging: a high-level, static abstraction of a software system doesn't equip an engineer with the detailed understanding needed to troubleshoot issues effectively. Furthermore, problems may require different abstractions, meaning no single abstraction can address all needs. It's similar to understanding how your car works: you don't need to know every detail, but you should be able to check under the hood to diagnose issues, especially without needing to take the car back to the dealership every time. Therefore, we need complexity-embracing tools that combine abstraction with the capability to delve into the tech stack. These tools should provide up-to-date information on any component, structure, or relationship and foster collaborative problem-solving. Unlike general-purpose diagramming and whiteboarding tools, we need tools that distinguish between the relationships of elements in an org chart versus those in a platform architecture. Enhanced system observability and understanding tools are essential, empowering developers to explore, uncover, and manage complexity effectively. Final Thoughts Many engineering teams persist in using diagramming tools due to a combination of sunk cost fallacy (" We've already invested 30 hours in creating and updating this diagram, so we might as well continue and not waste that time"), resistance to change (" Switching tools takes time and training, we have other priorities now"), and unclear problem definition ("we need an updated diagram" vs. "we need a real-time understanding of our system"). [02eb39be-sunk-cost] The sunk cost fallacy is the tendency for people to continue an endeavor or course of action even when abandoning it would be more beneficial. As the complexity of modern software systems grows, more teams will encounter the limitations of traditional diagramming tools. Once essential for illustrating ideas and designs, these tools still need to be improved to capture the full complexity of systems, hindering developers from fully understanding, designing, developing, and managing them. Diagramming tools must evolve to support the comprehensive activities required in the system design process and empower teams to effortlessly answer deeper questions about their software architecture. Today, we have the technology and knowledge to create tools that prevent developers from wasting valuable development time deciphering static, outdated diagrams, manually updating documents, or gathering information from scattered sources. Instead, we can enable teams to focus on meaningful discussions, driving towards solving complex problems and making informed design decisions for the system's evolution. TRENDING STORIES YOUTUBE.COM/THENEWSTACK Tech moves fast, don't miss an episode. Subscribe to our YouTube channel to stream all our podcasts, interviews, demos, and more. SUBSCRIBE Created with Sketch. [31e89e7a-c] Co-founder and CTO at Multiplayer, with 30+ years of experience as a backend developer building large-scale distributed software (and robots!). Read more from Thomas Johnson SHARE THIS STORY TRENDING STORIES TNS owner Insight Partners is an investor in: Control, Real. SHARE THIS STORY TRENDING STORIES TNS DAILY NEWSLETTER Receive a free roundup of the most recent TNS articles in your inbox each day. [ ] SUBSCRIBE The New Stack does not sell your information or share it with unaffiliated third parties. By continuing, you agree to our Terms of Use and Privacy Policy. ARCHITECTURE Open Source Cloud Native Ecosystem Containers Edge Computing Microservices Networking Serverless Storage ENGINEERING AI Large Language Models Frontend Development Software Development API Management Python JavaScript TypeScript WebAssembly Cloud Services Data Security OPERATIONS Platform Engineering Operations CI/CD Tech Careers Tech Culture DevOps Kubernetes Observability Service Mesh CHANNELS Podcasts Ebooks Events Newsletter TNS RSS Feeds THE NEW STACK About / Contact Sponsors Sponsorship Contributions roadmap.sh Community created roadmaps, articles, resources and journeys for developers to help you choose your path and grow in your career. Frontend Developer Roadmap Backend Developer Roadmap Devops Roadmap (c) The New Stack 2024 Disclosures Terms of Use Advertising Terms & Conditions Privacy Policy Cookie Policy FOLLOW TNS