[HN Gopher] Ask HN: Visualizing software designs, especially of ...
___________________________________________________________________
Ask HN: Visualizing software designs, especially of large systems
(if at all)?
I learned about UML in a course, but have never used it or seen it
in practice as a junior developer. Sometimes I'll see a flowchart,
but that's not too common. Is this the same in other companies?
With some of our code, the designers are either gone or sometimes
unavailable, and it can be tricky to see how all the pieces fit
together. A good IDE makes the job a little easier (finding
references, ctrl+click to go to declarations, etc.), but it'd be
nice to have a diagram or something for visualization. So, is it a
good idea to try documenting the code design through some sort of
visualization? If so, using UML or something else? I suppose there
might be tools for doing this automatically in some languages?
Otherwise I think if it was valuable enough, it could be something
we make sure to review and update along with code changes. Any
thoughts would be appreciated!
Author : lovehatesoft
Score : 132 points
Date : 2022-05-31 14:06 UTC (8 hours ago)
| ChrisMarshallNY wrote:
| I use what I call "UML Lite" (UML, without all the frou-frou).
| It's handy for illustrating points[0].
|
| What I generally do, is start with what I call a "napkin sketch"
| (which can be UML-ish)[1], and try to avoid writing down too much
| stuff, in order to reduce "concrete galoshes"[2].
|
| I try to use tools like Doxygen and Jazzy, to document the code,
| in an inline fashion[3]. Doxygen will generate a UML "Lite"
| diagram[4].
|
| [0] https://littlegreenviper.com/miscellany/swiftwater/the-
| curio...
|
| [1] https://littlegreenviper.com/miscellany/forensic-design-
| docu...
|
| [2] https://littlegreenviper.com/miscellany/concrete-galoshes/
|
| [3] https://littlegreenviper.com/miscellany/leaving-a-legacy/
|
| [4] https://doxygen.nl/manual/diagrams.html
| djbusby wrote:
| We like using PlantUML and it embeds in Asciidoc nicely.
|
| Also the C4 Model seems cool but I don't currently use it
|
| https://c4model.com/
| neuronexmachina wrote:
| An alternative to PlantUML is MermaidJS, which GitHub recently
| added support for in Markdown documents.
| crgi wrote:
| Plus one for C4 Models. Most important aspect of it: When you
| do lines and boxes in a diagram, put descriptions on the lines,
| not just the boxes. Sounds simple, but is often forgotten. The
| result is people seeing UML, making all sorts of assumptions.
| spotlesstofu wrote:
| I love the open source diagrams.net (previously draw.io). I save
| the files as editable SVG so I can include them directly in
| markdown READMEs. I use diagrams both to help me design and to
| show during demos
| croes wrote:
| You mean https://app.diagrams.net/
| theocodesitall wrote:
| I recently started using https://onemodel.app and it really makes
| diagramming super fast and easy.
| [deleted]
| coward123 wrote:
| I spent a number of years as a traveling consultant fixing
| problems... I guess it was aligned with what the cool kids call
| Site Reliability Engineering these days. Some company would have
| a crisis and I'd go in and fix it. So basically I had a matter of
| a few hours to learn as much as I could about some huge system
| that had probably been built over many years by a whole lot of
| people.
|
| To do this, I used a number of generally proprietary tools, where
| part of the project was getting the company to them buy these
| tools and fix their practices so that perhaps they would avoid
| the problem in the future.
|
| Anyway, my point is there are open source and proprietary tools
| out there in the world that will instrument an application and
| generally based on usage patterns will build out various
| visualization of the application. While these are often sold or
| marketed as tools for operational triage, I always argued that
| the best practice was for developers to use these tools in order
| to both understand the performance characteristics of their work
| as well as look for unexpected interactions.
|
| These tools will generally build out a topology of how various
| components interact and will often do other type of
| visualizations that will show class and method level chain of
| invocation. The good ones show end to end across distributed
| systems every bit of code that gets called from the time a user
| attempts some kind of action.
|
| The benefit of this approach over a UML diagram or something
| similar is that it shows how the system was actually built and is
| working, rather than what a developer intended to be built. The
| larger the system and the more years / developers involved, the
| greater the delta.
| itsmemattchung wrote:
| > So basically I had a matter of a few hours to learn as much
| as I could about some huge system that had probably been built
| over many years by a whole lot of people
|
| What are some of the open source tools that you are referring
| to? Within a few hours, I can imagine it being difficult to
| integrate and deploy your tools into an existing stack. But if
| you are tapping into some mirrored network port, that would
| make more sense.
| coward123 wrote:
| Hello Matt - I deliberately didn't name any tools, as the
| ones I worked with are no long in the tooth and were $$$.
| What I wanted to do was bring to the OP some concepts that
| they might search for in order to identify the right tool for
| their purpose. To your point - yes, some of the tools I used
| made use of the kind of network port mirroring similar to
| what it appears you developed at Amazon. Others instrumented
| various virtual machines, others used various external
| monitors.
| silasb wrote:
| Not a large systems, but we started using https://backstage.io to
| help visualize our software. Additionally, we're hoping to
| leverage backstage plugins to provide tooling around ownership
| for our components.
| candiddevmike wrote:
| If you can get OpenTelemetry/Jaeger/etc integrated with the
| codebase, you'll be able to visualize requests really easily,
| especially if you sample everything (in dev).
| marcosdumay wrote:
| In my experience, software engineering has ever created 4 types
| of standardized diagrams that are useful: entity-relationship,
| data-flow, messages and states-transition. Most share the feature
| that they communicate data, instead of code.
|
| The standard data-flow diagrams are too low level to be useful
| nowadays, but it's trivial to remap its symbols into the high-
| level components our modern distributed systems use. But still,
| for most people they will just show a database, an application
| transformation, and the user's output, so they are only useful
| when you have something different from that to show.
|
| Entity-relationship diagrams were technically included in UML,
| but the entire world disagrees, so you are better thinking about
| them as a separated class. Those are in wide use.
|
| Messages diagram (included in UML as sequence diagrams) are very
| useful to design and analyze protocols, but the UML one is a bad
| fit for that use due to its serial appearance. If you look at
| distributed systems papers, you will see a version with oblique
| arrows instead of horizontal ones that doesn't imply two-sided
| links, serial communication or even a total order on the
| messages.
|
| Finally, state machines are a really useful architecture pattern,
| and states transition diagrams capture them very well.
|
| Every other diagram that I've seen is either completely ad-hock
| or replacing it with text would improve everything. I really miss
| some "central architecture diagram" that shows the important
| structures on your code and how they interact (UML has structure
| as class diagrams, those suck), but I have never seen any good
| implementation of this one.
| jbreckmckye wrote:
| The challenge is that there are different ways of "mapping"
| software.
|
| You could map the way programs fit into machines, and the
| networks between them. This would be the topology.
|
| You can map the way services call upon one another with requests.
| This is the service graph.
|
| You can map how systems interact over events or shared resources.
| You could say this is the logical graph.
|
| The problem happens when you try and graph them all at once. It's
| the same as trying to draw a real map, with all the services, bus
| routes, railways, shops and administrative regions superimposed
| on one image. It's very busy.
|
| So I use separate maps.
|
| Tools are another matter. Personally I use Mermaid for graphs. I
| also have my own tools that create SVG visualisations using
| DAGre. This can be helpful for interactive visualisations where
| you can click into different nodes and explore more detail.
|
| My system uses CloudFormation templates and our in house
| deployment DSLs to figure out the "topology", then let the users
| see the different superimposed "graphs" as they see fit
| divan wrote:
| Some thoughts and attempt to visualize Go code in a meaningful
| way: https://divan.dev/posts/visual_programming_go/
|
| And, tangently related, visualizing concurrency:
| https://divan.dev/posts/go_concurrency_visualize/
| evo_9 wrote:
| I would take a look at Wardley Mapping:
| https://learnwardleymapping.com/
|
| It's interesting because it attempts to capture domain knowledge
| isolated from a programming-centric view /metaphor which makes it
| challenge to discuss complex systems with staff other than
| programmers.
| m12k wrote:
| In 15 years in the industry, the only UML I have seen outside of
| a classroom has been made ad-hoc, on a whiteboard during a
| discussion. There would usually be no strict adherence to whether
| a square or "blob" was a class, an object, a user, a database or
| a concept, nor whether a line between these with an arrow at the
| end meant "inherits from", "knows about", "has an instance of",
| "sends data to", "calls a function on", "contacts with a network
| request", "is transformed into" or something else - all these
| details would usually just be cleared up from the context, or
| with text next to the lines. There was thus also usually no
| strict distinction between class diagram, sequence diagram,
| flowchart or other diagram - the diagram would just be whatever
| it needed to be in a given area in order to convey the
| information being discussed. In short it was highly informal, and
| used as a means of communication or brainstorming, rather than
| documentation.
|
| I have a feeling there might be more UML in parts of the
| enterprise world, or in places with "architects" that have
| forgotten (/never learned) how to code. But my guess is the above
| is what the majority of "UML" usage in the industry looks like.
|
| As for your question about documentation: The best advice I can
| give is to be mindful that documentation tends to get outdated
| faster, the further away it lives from the code that it
| documents. That's why "self-documenting" code is so valuable,
| since the compiler will often refuse to compile it if you don't
| update it. Thus I would always advise to use the "closest"
| documentation form that is suitable, in roughly this order:
|
| - The code itself
|
| - The tests
|
| - Comments in the code
|
| - Comments at the start of files
|
| - Readme.md
|
| - Other files in repo
|
| - Wiki that lives alongside the repo (e.g. GitHub)
|
| - External wiki (e.g. Confluence)
|
| In your specific case, it sound like you need to document the
| architecture/design in a way that needs to be understood before
| someone would even know which file to look in for further
| documentation. In that case, I would first attempt to describe
| the architecture in words in Readme.md. E.g. something like a
| list of the most important classes/functions/datatypes and a
| paragraph or two about how each relates to the others.
|
| If something like that isn't helpful enough, and you want to make
| some UML consider using something like asciiflow.com or
| textik.com to make ascii diagrams to put in the readme. If the
| info doesn't fit that format, consider making diagrams in images
| that can be shown inline in the readme - ideally if you do this,
| use a format where you can also check in the "source" of the
| image (e.g. a graphviz DOT-file) to make it easier to update. And
| then finally, if all else fails, you can either check in a PDF,
| or use a wiki. But as mentioned above - prefer the "closer to the
| code" solution whenever possible.
| aetherlord wrote:
| I'm a fan of https://www.ilograph.com/. I've only used it for a
| few small things, but the author has good samples, including a
| diagram of ilograph itself -
| https://app.ilograph.com/demo.ilograph.Ilograph/Request.
| tra3 wrote:
| This looks very cool. It looks similar to the C4 model, where
| you can have nested components of arbitrary depth ("containers"
| in C4 parlance).
| dtjohnnymonkey wrote:
| I use Ilograph pretty heavily both for documenting existing
| systems and designing new systems. The paradigm of "everything
| has context" makes diagrams much easier to understand.
|
| I have even used it to render infrastructure diagrams of actual
| production systems (clusters, load balancers, etc)
| rswail wrote:
| This looks interesting :) Not sure about YAML, been burnt with
| OpenAPI, but it looks good.
| w10-1 wrote:
| For OO structure, https://structure101.com is fantastic at global
| views, especially for refactoring, where it can help you plan how
| to transition from hairball to e.g., directed graph. It has a
| 30-day trial to show you it's worth the price.
|
| Rolling your own visualization is useful and not too hard. It
| drives you to frame your questions more narrowly (call-
| hierarchy/sequence diagram, data flow, sub-systems?).
|
| The fast-path for me is to generate the relation
| doublets/triplets (x --R--> y) and then let yFiles/yEd
| (https://www.yworks.com) lay it out hierarchically after
| converting to their tgf "trivial graph format". yEd GUI is free;
| the yFiles layout library is worth 10X the price if you're
| displaying lots of graphs.
| agentultra wrote:
| Depends on what you want to understand about the system.
|
| If you're looking to understand it's properties and how it
| behaves I would look into something more robust like Alloy 6 [0]
| which has a great visualization system for inspecting models.
| However if you're looking for a _class diagram_ tool then it 's
| outside your wheelhouse.
|
| There's also something I've played around with a bit but haven't
| used seriously: Moose [1]. It's basically an IDE for doing
| analysis of code. The trick is writing good parsers.
|
| [0] https://alloytools.org/alloy6.html
|
| [1] https://moosetechnology.org/
| rscho wrote:
| Moose looks really impressive, visually! I think it's one of
| the flagship Pharo applications.
| golf_mike wrote:
| > ...documenting the code design... I would say that code is an
| implementation of (part of) a design, and both implementation and
| design should be documented appropriately. If it's not present,
| make as much text and visualizations you need for you to
| understand it. As you are a developer start with the code. I
| prefer C4 over UML for the separation of contexts. Good and fun
| exercise!
| joe8756438 wrote:
| One technique that improved my process a lot, particularly for
| system-level problems, is swim lane diagrams. I worked with
| someone that created them for _everything_, and at first it
| irritated the crap out of me because for some things I felt it
| took longer to write the diagram than to write the code. But,
| when things are complicated it is so helpful to see a process
| overview with minimal path ambiguity.
| masterofmisc wrote:
| Sequence Diagrams!! They are awesome. You get to see all the
| entities involved in the architecture and how they interact with
| each other.
|
| Also https://sequencediagram.org/ allows you to describe them in
| text and creates a diagram for you.
|
| Its what I have been using.
| bsedlm wrote:
| I want to make a computer runtime which doesn't execute the
| program but instead allows for a programmer to understand what's
| going on.
|
| a metacomputer of sorts; maybe epicomputer would be a better
| name?
| sno6 wrote:
| I think a _part_ of the problem is due to being confined to the
| dimensions of your screen. If I'm reading code in one file which
| relies on context from another file, I can bring them up side by
| side in my editor, but the value in doing that diminishes with
| the more files / context I need in order to understand something.
|
| Sometimes I wish I could project my IDE onto the wall behind my
| monitors. Programming in VR would probably achieve the same
| thing, but I'm not ready to move into that world yet.
| edpichler wrote:
| UML is not a documentation tool, it's a communication tool. It is
| a standardized way to communicate. It removes ambiguity. UML is
| the best way to manage, communicate and handle a large amounts of
| complexity. Most of the free UML tools are very basic and does
| not have traceability features (we can see many in the comments),
| and this is really a no-go if you want to understand large
| systems.
| l33t2328 wrote:
| Documentation is communication, albeit asynchronous.
| fburnaby wrote:
| I agree with your view on UML, but now I'm curious what you
| consider documents to be for.
| anoncept wrote:
| You might find it helpful to distinguish between visualizing the
| design of the system being implemented by your software,
| visualizing protocols being implemented by your software,
| visualizing the design of your software implementation itself,
| and visualizing important implementation details at runtime, e.g.
| for debugging, profiling, and operations.
|
| For visualizing system designs, you should take a look at STAMP,
| e.g., via "Engineering A Safer World" + the resources at
| mit.edu/psas + on YouTube.
|
| (Multiple tools, both commercial and libre, exist and are being
| developed to make these diagrams, although for what it's worth, I
| mostly hear about people making them using draw.io, Google
| Drawings, on physical paper/whiteboards, or occasionally with
| specialized tooling.
|
| I have also recently published a project in this area,
| https://github.com/mstone/depict, which I believe is well on its
| way toward addressing some unmet needs here.)
|
| For visualizing protocols, things like sequence diagrams, data
| flow diagrams, DRAKON flow charts, value stream maps, and
| occasional more specialized objects like CPSA "cryptographic
| protocol shapes" / strand space skeletons are where I start
| depending on the flavor of what's needed.
|
| For visualizing the design of implementations themselves, I have
| not yet seen anything that I feel obliged to recommend; rather,
| here, I suggest investing in adding illustrations to your
| existing documentation in whatever way is easiest for you to use
| to clarify whatever subtleties you need to clarify for your
| audience.
|
| (Here I tend to look at things like ASCII-art, SQLite's railroad
| diagrams (now made with pikchr, AIUI), and sequence diagrams, as
| mentioned by other commenters, as helpful examples to start
| with.)
|
| Finally, for implementing debugging/profiling/operational
| illustrations, there is a such a rich set of examples to turn to
| -- whether from the very specialized (custom process model video
| rendering pipelines in robotics) to TensorBoard for TensorFlow to
| general-purpose tools like browser performance debugging suites,
| flame charts, or Go's built-in profile graphing tools - that
| rather than learn any particular such tools, I'd instead suggest
| trying to get comfortable with the building blocks underlying
| these systems, which include contemporary GUI/web apps, custom
| drawing and animation tools like SVG, pretty printers, and
| Grammar-of-Graphics systems like vega-lite.
|
| (Note: although it may seem superficially extraneous to your
| question, the reason I also suggest thinking about debugging
| visualizations in this context is because IMO, to work, they
| ~necessarily encode a visual model of the design of your
| _implementation_ since it is the design of the implementation
| that provides the vocabulary and relationships that have to be
| understood and navigated in order to successfully debug
| /optimize/monitor any given running instance of whatever system
| you are building.)
| douchescript wrote:
| I like to use code coverage tools to understand how things work
| together. Its especially useful with unit tests. You can see
| every line that was hit all over the codebase for a particular
| function or api call, with a heat map of line counts. Makes it
| easy to know what's important and what's cruft.
| mtoddsmith wrote:
| The most important rule is that you can't include everything in a
| single doc as it quickly gets too big and complicated.
|
| So you need to break it down into multiple documents that cover
| different use cases and include only the components for a small
| set of use cases.
|
| Then use the C4 model to break up the docs.
| scothalverson wrote:
| I have found success with Scitools' Understand
| (https://en.wikipedia.org/wiki/Understand_(software))
| CitizenKane wrote:
| In my experience it tends not to be done because there's an
| inevitable drift between the system as it actually operates and
| the relevant visualizations and diagrams. I have seen them used
| as a starting off point at times, although that's also been
| infrequent.
|
| There is however a growing movement of being able to visualize
| how a system is functioning at various levels. XState/Statecharts
| are a good example (https://xstate.js.org/viz/). Another example
| in the Ops space would be https://github.com/spekt8/spekt8 for
| K8S. I work at Grafana and we're more or less trying to expose
| these things in ways that make sense. Our bread and butter is
| timeseries data but we're adding more in that regard (it's
| possible to build node graphs from running systems).
| saila wrote:
| I've worked at a variety of places and have never seen any kind
| of system or architecture diagram going in (should probably ask
| about this in the interview stage). Further, it often seems that
| _no one_ understands the overall system well, in some cases even
| when there 's an architect.
|
| So I usually find myself in the same position of using jump-to-
| definition and trying to get a handle on things that way.
|
| From there, I'll sketch a simple diagram _on paper_ to aid my own
| understanding of whatever piece of the system on working on.
| Personally, I 'd avoid software diagramming tools at this stage.
| A simple hand drawn diagram can be extremely helpful and doesn't
| take long to create.
|
| After a while, I'll often start sketching out a more
| comprehensive system diagram with the database(s), processes,
| etc. At this point, I'd ask management about working on this
| because it can be a time consuming process that involves talking
| to a lot of people.
|
| In my experience, this has usually been an uphill battle and
| somewhere between difficult and impossible to take on as an
| individual developer, but I think it's worth a try. Taking this
| kind of initiative _could_ lead to career advancement, but I 've
| often found that there's a lot of resistance and it's not seen as
| a priority.
|
| It might seem a bit cynical, but as a junior or even intermediate
| developer, I'd also advise caution against stepping on anyone's
| toes. I'm not sure why, but some people tend to get defensive
| about this kind of thing.
| Weidenwalker wrote:
| My friend and I have been working on https://www.codeatlas.dev in
| our spare time, which is a tool that creates pretty (2D!)
| visualisations of codebases, while providing additional insights
| via overlays (e.g. commit density, programming language). For
| example here's the Kubernetes codebase visualised using
| codeatlas: https://www.codeatlas.dev/repo/kubernetes/kubernetes.
|
| At the moment, codeatlas is only a static gallery, but we're
| currently about 1-2 weekends away from releasing a Github action
| that deploys this diagram on github pages for your own repos - if
| you're interested, feel free to watch this repo:
| https://github.com/codeatlasHQ/codebase-visualizer-action
| handstad wrote:
| Check out https://c4model.com/
| vanillax wrote:
| This with Icepanel
| adamritzel wrote:
| This is the most promising tool I've seen in the space.
| https://icepanel.io/
| tra3 wrote:
| That's pretty neat. It looks like the support exporting
| models with JSON, I wonder if they support PlantUML C4
| import? Or some sort of import in any case.
| jellyfishbrain wrote:
| er_d0s wrote:
| I've tried c4/plantuml (and loved the concept) but have never
| been able to make it stick, it always seems to get out of date
| quickly and having a feeling of "I'm not sure if this is up to
| date" is worse than no documentation at all (because you need to
| learn the documentation, then the behaviour). Honestly the
| fastest way I've found to understand large distributed systems is
| by observing the interactions between them, schema detection in
| service bus type architecture and tools like X-ray (in aws) or
| open tracing give you a good picture of "this thing produces or
| consumes this type of message"
|
| For individual systems I think a good inversion of
| control/dependency injection system can give you a good overview
| of the connections between components.
|
| If I'm stepping into a giant ball of spaghetti code I generally
| set the debugger at line 1 and start stepping, and draw a lot of
| boxes and lines as I go... then throw those drawings away when
| I'm done. They're really only helpful while you're producing them
| (like taking notes as you read a textbook), hopefully in that
| situation you improve the code as you go so this approach isn't
| as necessary!
| contingencies wrote:
| Interfaces. A well designed system will present simple interfaces
| with clear delineations of responsibility. All elements will be
| carefully named to minimize the potential for confusion or
| misuse. In such a system, diagrams become not only easy but
| semantically meaningful. Programmers use this to manage
| complexity, and in business domains laypeople should usually also
| be able to comprehend the chosen abstractions.
|
| To describe messages and related state transitions representing
| the function of a system over time, numbered arrows atop a block
| diagram can work well (essentially one visualization of a graph),
| but for more complex multi-component interfaces with a strong
| requirement for ordered messaging, message sequence charts are
| well received. https://www.mcternan.me.uk/mscgen/
|
| _There are two models of reality that I find to be the most
| useful ones, especially when writing programs. The first is
| functions, and the second is sequences of states._ - Leslie
| Lamport
|
| .. via https://github.com/globalcitizen/taoup - see also
| https://en.wikipedia.org/wiki/State_machine - and you won't
| regret learning http://graphviz.org/
| raxor53 wrote:
| Why are all these comments suggesting new tools instead of
| answering your question? From what I've seen, high documentation
| only comes into higher level abstractions such as individual
| services or architecture design.
|
| In school, I was taught to draw UML for classes within a program.
| I have never seen that IRL. I think the difference is the time
| required to comprehend the application.
| diegof79 wrote:
| What you are looking for is called "Program Understanding". If
| you Google for it you'll find a bunch of research papers on the
| topic.
|
| For some reason, tools related to program understanding are not
| widely adopted by IDEs.
|
| A while ago I used a tool for Java that was based on the Object-
| Oriented Metrics[0] book by Michele Lanza. But, that tool was
| discontinued and it doesn't exist anymore[1].
|
| If you are interested in that topic take a look at Moose[2], a
| dig a little bit in the research papers. (honestly I tried Moose
| a few times, but I wasn't very comfortable with it).
|
| For TypeScript projects, the TS compiler API is extremely
| powerful and easy to use. You can use that to extract information
| and analyze the code relationships (Graphviz is your friend here
| :) ).
|
| [0]: https://link.springer.com/book/10.1007/3-540-39538-5 [1]:
| https://web.archive.org/web/20150428173717/http://www.intooi...
| [2]: https://moosetechnology.org/
| marktangotango wrote:
| There was also a lot of work in this area in the 90s leading up
| to the y2k (non)event. Mostly back then it revolved around
| cobol, which was understandable given y2k impact was in many
| cases related to legacy cobol systems. A lot of architecture
| visualization and recovering business requirements from code. I
| did some work on diagramming mainframe JCL files for example.
| otras wrote:
| I gave this a bit of thought earlier this year
| (https://alexanderell.is/posts/visualizing-code/) and, with the
| help of HN commenters, collected a small list of ways people are
| working to help with code visualization. I don't think most of
| them are production ready (some are just research papers), but
| you may find them interesting all the same.
|
| SoftVis3D (https://softvis3d.com/): where a "'code city' view
| provides a visualization for the hierarchical structure of the
| project".
|
| Code Park: A New 3D Code Visualization Tool (2017)
| (https://arxiv.org/pdf/1708.02174.pdf), a "novel tool for
| visualizing codebases in a 3D game-like environment" with code
| represented as "code rooms" with code on the walls.
|
| Code Structure Visualization Using 3D-Flythrough (2016)
| (https://opus-htw-aalen.bsz-
| bw.de/frontdoor/deliver/index/doc...), with spatial metaphors and
| first-person exploration of code.
|
| Primitive (https://primitive.io/), a VR collaboration startup
| with a Matrix-looking "Immersive Development Environment" with
| "new tools for visually analyzing software in 3D".
|
| AppMap (https://appland.com/docs/how-to-use-appmap-
| diagrams.html), an automated code analysis tool that includes
| dependency maps and trace views.
|
| plurid (https://github.com/plurid/plurid), a framework for
| visualizing and debugging code in a 3D explorable structure.
|
| fsn (file manager)
| (https://en.wikipedia.org/wiki/Fsn_(file_manager)), an
| experimental application to view a file system in 3D (featured in
| Jurassic Park).
| euroderf wrote:
| Just wondering, did any of these give you a Minority Report
| kind of vibe ?
| photochemsyn wrote:
| These look great, I think the general idea of using space is
| pretty solid. Anyone who's ever tried to draw a tree structure
| finds it gets busy really quickly with more than a dozen or so
| leaves, but an actual tree in 3D space can have thousands of
| leaves yet fits into a reasonable chunk of space. Graphs are
| similar, if they are connected in 3D rather than 2D (ant nests
| and termite mounds are decent models).
| tikimcfee wrote:
| If I may be so bold as include something I'm working on myself,
| I'd love to chime in! I'm not functionally complete, but feel
| like I'm writing an amalgamation of all of the above tools.
|
| https://github.com/tikimcfee/LookAtThat : A macOS and iOS app
| to load, analyze, and walk around your Swift code, version
| 0.0.0-prealpha!
| qxxx wrote:
| I am a php developer and sometimes if I work on a complex project
| I use xdebug + profiler + cachegrind to generate flow diagrams.
| It shows not only how often a method was called and how much
| memory it used but also where the call came from and what
| happened next. It helps sometimes to understand the code better.
| I use it mostly for debugging.
| cwoolfe wrote:
| I remember having the same problem earlier in my career. I
| inherited a code base that was a million lines of code and 15
| years old. Things I tried: printed out the nastiest 2000 line
| function on a giant piece of paper, logged every function entry.
| Things that worked well at the time: reading filtered logs of
| actual code execution, tracing through code to see how it works.
| What I would do if I could talk to me 10 years ago: Don't worry
| about the big picture. Trace through the code path relevant to
| your task and make sure you understand whatever inputs/outputs
| there are on that path. If there's something that makes 0 sense,
| ask a senior dev. After tracing through enough paths, you'll
| start to develop a mental map, in the same way you develop a
| mental map of a geographical location, after visiting about five
| locations in the same area. You get to know the main roads,
| highways, etc. You have a knowledge of the relevant paths; you
| don't always need a map of the whole city.
| _dain_ wrote:
| No automated tool will come close to having a 5 minute
| conversation with the main designer and having him draw you a
| diagram freehand on the back of a napkin. This is a social and
| organizational and communication problem, not technical.
|
| If that can't be done, there are some interesting things you can
| try. A lot of the suggestions in the thread are "top down"
| methods; you can get a lot of value out of "bottom up"
| visualizations too. Things like:
|
| - Histograms of which lines / functions get called the most, or
| spent the most time in
|
| - Which lines / functions / files get changed the most in the git
| history
|
| - CPU flamegraphs
|
| - Plain old print-debugging
|
| In over-architected systems it can be difficult to figure out
| where the real "meat" of the code is, as opposed to the endless
| layers of configuration and wrappers and interfaces and
| indirection. UML diagrams may not help, or even be deceiving, but
| a stack trace never is.
| buescher wrote:
| The way to capture the content of a conversation like that, so
| that more than one person at a time can easily benefit from it,
| is in a theory of operation document with appropriate
| illustrations. Simple block diagrams will take you far before
| you need to specify something in the level of detail that the
| UML supports.
|
| I like the range of approaches in the _Architecture of Open
| Source Applications_ books: http://aosabook.org/en/index.html
| PaulHoule wrote:
| This tool is good for making simple UML diagrams and even lets
| you do it with simplified syntax
|
| https://plantuml.com/
|
| I'd say the big problem in visualizing big systems is that you
| can't usefully do it in one graph. For instance I worked on a
| system that had 2000+ database tables if you were going to make a
| diagram of that which shows everything it is going to take up a
| long wall. (This can be useful, but it is a big commitment)
|
| A useful tool is going to let you make meaningful diagrams that
| show the subset of entities that are part of a story. I went to
| an art show of Mark Lombardi's works
|
| https://en.wikipedia.org/wiki/Mark_Lombardi
|
| who (before he was murdered) drew elaborate diagrams of
| conspiracies. One thing they showed was drafts that he made in
| the progress of creating his visualizations and he would
| sometimes make 40 or more of them. He would start out with a
| "hairball" that was disorganized and gradually figure out how to
| lay the diagram out in a way that made the meaning obvious.
| _dain_ wrote:
| wikipedia page says suicide
| bornfreddy wrote:
| We will probably never know if it was a suicide or he was
| "suicided". Given his works one has to wonder... Did one of
| them hit too close to the truth?
| cupofpython wrote:
| alternatively, given his obsession with conspiracy - maybe
| he was unable to establish deep connections with anyone in
| his life
| rdubs333 wrote:
| Good stuff. I am doing similar things but have not been
| murdered yet!
| mattdeboard wrote:
| gl;dd ;)
| NicoJuicy wrote:
| PlantUML with C4 and the additional cloud icons, is what I use
| tra3 wrote:
| I had to do a double take, there was a similar question
| couple of weeks ago [0].
|
| I was blown away by the idea behind C4 when I first saw the
| presentation. I think what's missing is the tooling. I use C4
| PlantUML do document my architecture designs.. what I'd
| really love though is a google maps style interface where I
| can zoom in or out of the current level I'm at. That'd be a
| game changer. Then you can really describe and understand the
| system.
|
| The original presentation, in fact, used the google maps
| interface to illustrate the idea where you're first looking
| at a continent, then you zoom in to the city and finally the
| street level.
|
| If you are using C4 right now, how do you compose the various
| level of architecture and navigate around them?
|
| [0] https://news.ycombinator.com/item?id=31370268
| chaostheory wrote:
| > I'd say the big problem in visualizing big systems is that
| you can't usefully do it in one graph.
|
| That's because it's flat
|
| Imo this changes the game
|
| https://noda.io/
|
| Imagine being able to make a fully 3D graph inside a space the
| size of a large building
| liveoneggs wrote:
| The time of UML came and went very quickly. No one misses it.
|
| These types of diagrams are either not specific enough to answer
| any useful questions or so dense and insane that no one can read
| them.
|
| If you have a business requirement with a decision tree it is
| probably a good fit for this type of documentation - write it
| down in DOT or asciiflow or something
| euroderf wrote:
| Also a very typical complaint in previous HN discussions of UML
| is that it very quickly gets out of sync with implementation,
| and nobody has a magic wand (or work time) for keeping it in
| sync, so rot sets in quickly.
| jyounker wrote:
| There were a few good things that came out of it, notably
| PlantUML. PlantUML's sequence and state diagrams are worth
| their weight in gold.
| oedo wrote:
| If you're interested in diagramming system architecture in a top-
| down fashion, I'll pile on another recommendation to check out
| the C4 model[0].
|
| As others have mentioned, to effectively communicate a system you
| must limit the context to particular layers of abstraction, and
| C4 is a good approach to doing just that. There's also a C4
| plugin for PlantUML[1].
|
| But don't forget that, as with all visualizations, audience and
| purpose are key.
|
| Consider whether you are addressing short-term needs (eg
| identifying inefficiencies, modeling for a client pitch) or long-
| term needs (eg knowledge retention, managing complexity). If your
| audience's needs are short-term, you can certainly get by with
| much simpler tools (eg Inkscape, excalidraw/draw.io/etc, picture
| of a whiteboard, doodles on a napkin).
|
| Also consider whether or not you actually have a problem better
| served by bottom-up (ie generated) visualizations (eg ERDs for
| database schema refactoring, heatmaps for profiling, GraphViz for
| debugging DAGs).
|
| [0] https://c4model.com/
|
| [1] https://github.com/plantuml-stdlib/C4-PlantUML
| skadamat wrote:
| Glamorous Toolkit is doing some interesting work here:
|
| https://gtoolkit.com/
| TreyS wrote:
| Different types of diagrams highlight different aspects of a
| system.
|
| If you can, try to put together 2 different diagrams presenting
| different views. For example, sequence diagrams, model diagrams,
| etc. each tell you something different. Having a few different
| perspectives on the same system will give you a richer
| understanding.
|
| Not specifically related to visualization, but, also, if you
| spend a lot of time trying to understand a piece of code, once
| you understand it, document it (e.g. in a function or method
| comment). Over time, this makes a huge difference and will help
| you and others out when you revisit the code later.
| sokoloff wrote:
| I led a project ages ago to divide a massively tangled code base
| into two parts to facilitate a separation of tech organizations
| into two (related) companies.
|
| One of the hardest planning parts was "finding the gap" (where
| was the least unnatural way to place the split). To approach
| this, I pulled every function's dependency tree (from code
| analysis) and all of our relational database entity dependencies
| (from DBMS telemetry) and all of our functions calling RDBMS code
| (also code analysis) into a large graph, which I then manipulated
| with a mix of hand-written tools and Gephi to create logical
| clusters that were both near-neighbors and business direction
| aligned (each business was going to have a particular focus, so
| not all divisions were equivalent).
|
| I don't recall exactly how long I spent, but it was in the month
| and a half range to get to a workable initial plan that we could
| start burning down.
|
| That's not live tooling to explore a system and is more
| archeological in nature, but may give you some ideas of how to
| auto-generate some terrain data.
| yboris wrote:
| Specifically for _TypeScript_ I created a CLI to visualize the
| _call graph_
|
| https://github.com/whyboris/TypeScript-Call-Graph
|
| Works for _functions_ not classes. I'm unsure how useful this
| tool is, but I suspect it might be helpful in some codebases.
| gbuk2013 wrote:
| I like using https://c4model.com/ - the Level 2 diagram is
| particularly useful.
|
| I use https://mermaid-js.github.io/mermaid/#/ for the diagram
| itself because Github natively supports it in markdown files, so
| you can revision control the diagram. I managed to get reasonably
| close to the C4 diagrams minus a few features that mermaid does
| not support.
| neuronexmachina wrote:
| There's a recent PR which looks pretty promising for C4 in
| Mermaid: https://github.com/mermaid-js/mermaid/pull/3038
| kbrkbr wrote:
| For what it's worth: I have good experiences with OPM [0], but
| not at very large scale. [0]
| https://en.m.wikipedia.org/wiki/Object_Process_Methodology
| deathanatos wrote:
| > _I learned about UML in a course, but have never used it or
| seen it in practice as a junior developer. Sometimes I 'll see a
| flowchart, but that's not too common. Is this the same in other
| companies?_
|
| Welcome to the industry! _sobs in time constraints_
|
| > _So, is it a good idea to try documenting the code design
| through some sort of visualization?_
|
| Yes, I think it's a good idea, though I'm afraid I don't have
| much advice on how to accomplish that.
|
| The problem I've always had is that there's so many ways to cut
| up a system. Some people want higher-level architecture diagrams,
| that show how all the various systems fit together. Some people
| want infra diagrams showing what VMs, DBs, cloud resources, etc.,
| are all wired to what else. Some people want sequence diagrams
| detailing RPC/API calls between systems/component.
|
| Invariably, for whatever documentation does exist, the person
| wanting documentation wants the diagram that _doesn 't_ exist.
|
| I've tried PlantUML, but it is complete garbage when it comes to
| emitting useful diagnostics. Paired with a language that seems to
| be nothing but special case after special case, and the result is
| basically unusable.
| fasteddie31003 wrote:
| I'm a visual learner. I wanted a way to see how documentation of
| complex systems looked too. I'm working on a
| https://gainknowhow.com/software-companies.html . It is basically
| like a mind map for documentation. The high level documentation
| like core values is on top connected through learning paths to
| learn the complete context of low-level skills.
| crdrost wrote:
| A nice talk at kubecon last year spoke about visualizing
| protocols and more complex multi-party exchanges.
|
| The suggestion it made was to look across module boundaries or
| microservice boundaries, while making a particular request. So I
| go to update a Project with a new Pipeline, and I find out that
| the project service talks to the pipeline service which then has
| to create resources for which it talks to various resource
| services, all of that.
|
| Since there aren't great digital tools for doing this, for
| visualizing these things, it was suggested in that talk that one
| of the more outside the box things you could do is to just take
| cardboard and modeling clay and strings and cut out some shapes
| to represent the different services, let each string be an RPC,
| and the actual diagram you would imagine being alive in time,
| like little beads flying across these strings, to indicate
| requests going out and then responses coming back... But the key
| was less to witness the time but more to get a timeless sense of
| connection, "oh, it turns out ResourceService is very hairy in
| these diagrams, it is kind of the central hub for this
| microservice cluster." and for that it was helpful that the
| visualization had a certain physicality to it, it had weight and
| structure and engaged more senses than just the visual...
|
| I've actually thought about just digitizing these ideas, even
| though it misses half the point LOL. I think that could be some
| wonderful documentation and diagrams in our onboarding for new
| folks.
| cimi_ wrote:
| There are tools that help with visualising requests between
| services, provided the information is augmented with some extra
| observability data.
|
| Most APM/distributed tracing products have ways of visualising
| distributed traces so we can see starting from a specific call
| which services were involved and how much time is spent in
| important operations (e.g. db, etc). Here's an example:
| https://www.datadoghq.com/product/apm/#end-to-end-tracing
|
| Datadog builds this service map automatically for you from APM
| and RUM data:
| https://docs.datadoghq.com/tracing/visualization/services_ma...
|
| AFAIK you can consume the data via API so you can build other
| visualisations on top of it if you don't want to stick with the
| flame chart or service map.
|
| Since this is runtime information it won't be as comprehensive
| as static code analysis but it does show what's actually
| happening in the system.
|
| disclaimer: I work for Datadog edit: add link to service map
| jaylaal wrote:
| For a small but complicated project I got thrown into a while
| ago, the only way for me to understand it was to print out all
| the source directly, vertically tape together the pages for a
| single file, and then lay them all out on a huge table. Then I
| took multicolored markers and started physically drawing out the
| call chains. I then I sers-toi the system, and also found an
| enraging bug: the system widely used the variables "blah_name"
| and "blah_id", including in many functions' parameters. Except,
| in one case, blah_id was passed in as blah_name and thenceforth
| became known as blah_name.
|
| I don't know if an automated visualization system is possible,
| but you'll have to understand the whole thing before doing so.
| Pen and paper was the most expedient solution for me at the time.
| zach_miller wrote:
| Sounds like something the type system should have caught!
| sixstringtheory wrote:
| Lucky you if you work with people who see the value in a
| language with good type checking or that doesn't just use
| strings for everything.
| plonk wrote:
| Is it really worth staying in a team that doesn't? Sounds
| like you'll spend more time fixing bugs than creating
| features.
| DANK_YACHT wrote:
| I use pen and paper as well, but rather than print out all the
| source code, I write down the call stack. A calls B calls C,
| etc. along with the line numbers of the call. Much easier than
| printing out the source and you still have the IDE niceties
| like go to definition, find in source, etc.
| er_d0s wrote:
| Hah! I did the exact same thing for ages on paper and
| eventually evolved the system to manage my workload and
| context switching... I still use it a lot for going deep
| while debugging/understanding code. I ended up making it into
| an app when I broke my wrist and could still type but
| couldn't hold a pen. I can't remember if there's rules about
| self promotion in comments here but it's up at journalist
| mode dot com
| ASalazarMX wrote:
| This reminded when I had to maintain dozens of old 10,000
| lines COBOL programs as a junior programmer. I felt so lost I
| made a program that would print only the names of data
| structures and functions. Seeing the source resumed in a
| handful of pages, and being able to highlight and draw on it,
| helped me a lot. Digital has flexibility, but sometimes paper
| works best.
| kbrannigan wrote:
| Paper is one of those Perfected technologies. It's been
| there for a long time but boooy. It's powerful
| jimpudar wrote:
| I also used to do this when working on a big convoluted system.
| I had a conference room near my desk with all the walls
| completely covered in code. A big pack of multicolored
| highlighters is key.
|
| I remember a whole bunch of light bulb moments when I showed
| other developers the "big picture". It's an awesome technique
| when you're forced to work on spaghetti!
| _dain_ wrote:
| I've done this too, taped a bunch of impenetrable code to the
| wall and scribbled on it with pen to figure out wtf was
| happening. I propose that this be called the "Pepe Silvia"
| debugging method since it looks like a crazy conspiracy chart.
| Eventually you'll figure out why nobody is getting their mail
| ...
|
| https://www.youtube.com/watch?v=_nTpsv9PNqo
| brudgers wrote:
| The simplest thing that might work is a pen and a notebook.
|
| And a jpg with an iPhone to digitize the sketch.
|
| Documentation is a practice not a tool.
|
| A shitty process can be improved.
|
| Without process "the perfect tool" just sits collecting dust.
|
| Or a person wants to start documenting, so they shop for
| documentation tools.
|
| Now they have two problems.
|
| Good luck.
| a-dub wrote:
| for existing codebases, doxygen will produce static call graphs
| for c/c++ and java codebases using dot graphs that are
| hyperlinked back and forth to lxr style code listings (you do
| typically have to turn this on as it's expensive to compute and
| is turned off by default.)
|
| i always found uml and other object style diagrams to be kind of
| obfuscating and encouraging of overcomplicated oop designs.
| ultra_nick wrote:
| I've found dataflow analysis to be more helpful than other
| methods when designing and debugging systems.
|
| However, there are very few automated polyglot methods. So,
| often, I must trace the data paths manually.
| gabereiser wrote:
| 1000% yes it's a good idea.
|
| As a junior dev, you shouldn't have to be worried by such things.
| This should be handled at the senior/principal level.
|
| If you aren't getting use case diagrams at the very least, you
| don't have a clear definition of done.
| kulikalov wrote:
| Any system, however complex, can be designed as a bunch of "I/O
| devices" with abstraction layers. The trick is to keep each
| abstraction layer simple enough so that you can hold it in your
| mind (or fit a piece of paper). With this approach any kind of
| visualisations tool would work just fine - from a piece of paper
| to tools like Ilograph. I personally prefer excalidraw.
| ethbr0 wrote:
| The "hold it in your mind" test is my gold standard. Certain
| minds can hold more or less at once, but we're talking about a
| narrow range.
|
| The issue with modeling from any specific detail-specifying
| system is that the result is invariably outside that range, for
| some systems.
|
| Model to a brain target, not a standard level of detail.
| Aeolun wrote:
| I never make diagrams for entire codebases. I do make diagrams
| for specific logical flows (still not exactly mirroring the code,
| just the logic behind it).
|
| I would never use UML.
| KronisLV wrote:
| > So, is it a good idea to try documenting the code design
| through some sort of visualization?
|
| Yes, if it helps you understand how it works and how the pieces
| fit together.
|
| No, if the previous is not all that useful for you (different
| types of learners), or you need to spend significant amounts of
| time doing it manually, especially given that code could change.
|
| If you can, look into any tool that might allow you to get
| visualizations in an automated manner.
|
| For example, JetBrains IDEs have a few different graph
| visualizations for dependencies and inheritance etc.:
| https://www.jetbrains.com/help/idea/2022.1/tests-in-ide.html...
|
| There also used to be SourceTrail, though sadly the project is
| now retired: https://github.com/CoatiSoftware/Sourcetrail
|
| For databases, you can also use external tools like DbVis:
| https://www.dbvis.com/features/
|
| There are also a few tools here and there for visualizing
| networks or how container deployments look, but those are pretty
| situational/specific for each platform/setup.
| rdubs333 wrote:
| I have been working on a natural design system for visually
| thinking about technology. Maybe it will help you.
|
| Its called MTREES and its a free epub on Apple or Google.
|
| https://mtrees.io
| nwsm wrote:
| All three companies I have been at make heavy use of architecture
| diagrams, though their rules and prevalence vary from team to
| team. In fact I can't imagine a high functioning software
| organization taking a project from requirements gathering to
| production support without having ever made an architecture
| diagram.
|
| However, teams usually do not follow any hard rules like UML. A
| box with text, a connecting line, and a grouping border can all
| mean anything, and the subtleties of their actual implementation
| are still trapped in the lines of code and minds of architects.
|
| I would definitely recommend drawing diagrams of systems you
| create or work with. If there is not one at work, and you're
| struggling to grasp the big picture, start making one yourself.
| Start with the pieces you know and put in ambiguous boxes for the
| pieces you know exist but don't know what they do. I do this even
| for solo side projects when they have more than one physically-
| separated component. This will help you catch bad logic and
| inefficiencies early, and find the best implementations for new
| features down the road.
|
| I have used draw.io (now diagrams.net), LucidChart, Microsoft
| Visio, and they all get the job done. I'll recommend the first
| one as it's the most open.
| euroderf wrote:
| I got into SW after getting a degree in EE, and I find that any
| SW architecture I implement has to be visualised (by me) first.
| I don't need to actually commit anything to paper (or UML or
| whatever), but I need to have an image in my head that _could_
| (with some effort) be committed to paper or UML or whatever.
| flurly wrote:
| I'm a big fan of Terrastruct (https://terrastruct.com/). They
| focus on building great software for visualizing complex software
| architecture. Their secret sauce is this idea of attention where
| they allow you to zoom in and out so you can get the 10 foot view
| or 10,000 foot view, whatever you find most useful.
| mr_tristan wrote:
| Visualizations are just one aspect of documentation, so I would
| recommend looking into how you organize your documentation, and
| build visualizations to support that. This is, after all, what I
| think you are really going for: just better written explanatory
| documentation, with useful organization.
|
| The divio system is a good place to start, IMO, when it comes to
| organization: https://documentation.divio.com/
|
| So, I would treat a visualization used in an explanation-style
| document very differently from a reference guide. One is intended
| to illustrate a concept quickly, another is intended to be
| precise. I don't think you'll see a single "visual system" ever
| take over, largely because documentation can have very, very
| different goals for the reader.
|
| Visualizations in reference guides are (unfortunately) rare. I
| happen to like the approach taken by project reactor, embedding
| visuals into java reference docs:
| https://projectreactor.io/docs/core/release/api/
___________________________________________________________________
(page generated 2022-05-31 23:01 UTC)