[HN Gopher] What a good debugger can do
       ___________________________________________________________________
        
       What a good debugger can do
        
       Author : werat
       Score  : 210 points
       Date   : 2023-03-10 11:15 UTC (11 hours ago)
        
 (HTM) web link (werat.dev)
 (TXT) w3m dump (werat.dev)
        
       | avg_dev wrote:
       | Thanks very much for posting this article. I found it very
       | intriguing. I read it carefully but I didn't visit all the links
       | or watch all of the in-line videos.
       | 
       | I would say that for the most part I am a printf-style debugger.
       | I remember reading some years ago (on HN, I believe) about time
       | travel debugging in - I believe - C# and I was really impressed
       | but I don't code in C# so it soon left my mind.
       | 
       | I have a deep appreciation for mastering one's toolset. I can't
       | think of the number of times that I learned something new about a
       | tool (language, editor, shell, browser, whatever) that I use
       | daily that changes my workflow - it has happened so many times.
       | And as with all things code, sometimes new features are added.
       | 
       | I am going to try to make a point to re-read this article later
       | and to visit each link and glean what I can.
       | 
       | I mostly code in Go. I wonder, does anybody know how much of this
       | stuff might be supported there?
        
         | intelVISA wrote:
         | Not tried it unfortunately but there's a travel-time debugger
         | for C(++) somewhere.
        
         | mark_undoio wrote:
         | For time travel debugging in Go:
         | 
         | The Delve debugger for Go supports debugging rr traces:
         | https://github.com/go-delve/delve/blob/master/Documentation/...
         | 
         | Undo (who I work for) maintain a fork that debugs our
         | LiveRecorder recordings: https://docs.undo.io/GoDelve.html
         | 
         | Either rr (https://rr-project.org/) or our UDB debugger
         | (https://undo.io/solutions/products/udb/) can do some time
         | travel debugging of Go programs via GDB's built-in support for
         | Go. I believe its weakness is in support for goroutines, since
         | they don't map well onto its idea of how programs run.
        
           | pcdevils wrote:
           | The same tools don't work on windows either. Delve works well
           | as a remote debugger against go on windows server though.
           | Another fun one is go won't create dumps on panic on windows
           | when GOTRACEBACK=crash is set.
        
       | AcerbicZero wrote:
       | Honestly if most debuggers did what the author suggests in the
       | first paragraph, I'd use them 10x more. Other than Pry for ruby
       | I've not found many to debugger tools that let me drop into code,
       | run parts of it, examine variables etc, without needing a 4 year
       | degree in using the debugger itself.
        
         | BiteCode_dev wrote:
         | Pdb comes to mind.
        
         | hobs wrote:
         | Pycharm/Intellij stuff does everything you mention in a fairly
         | simple GUI that doesn't make you understand how to set
         | conditional variables, but encourages you to realize you want
         | to
        
           | alexdig wrote:
           | do you know if it's possible to step back in
           | pycharm/intellij? the article mentions some debuggers having
           | this ability, but i never saw an option in pycharm.
           | 
           | learning how to undo my last step would save so much time.
           | right now, i have to anticipate a risky step and run that
           | line in the debugger console.
        
             | hobs wrote:
             | Unfortunately I dont think there's any time traveling
             | debugger - you can pretty easily go up and down the stack
             | to see where the caller did x and write something in the
             | console to do y, or you can set conditionals that would
             | always trip when you are about to do something risky, but
             | not go back in time.
        
       | aantix wrote:
       | I think the visualizations are highly underutilized. There's so
       | much that can derived from the code itself and the runtime
       | profile.
       | 
       | That's why I'm creating the next generation debugger for Rails.
       | https://callstacking.com/
        
       | kubb wrote:
       | I use the debugger occasionally, when the human factor creeps in
       | and I inadvertently make a programming error.
       | 
       | The downside of debugging is obvious: it's not enabled in your
       | production programs (although it could be).
       | 
       | Logging OTOH is always on, and it has to be good enough to let
       | you figure out what happened on a remote machine, in a program
       | whose state you can't recreate anymore.
       | 
       | This obviously makes it good enough for debugging programs
       | running locally, and makes reaching for a debugger a rare
       | occurrence. GUIs for debuggers of course lag behind, so you need
       | to refresh your memory of the step, step over and breakpoint
       | instructions, etc. on every one of these rare occasions.
       | 
       | And then they don't always work. Printing more complex data
       | structures is often an unholy mess of pointers to strings and
       | private class fields. Evaluating expressions will end up failing
       | or crashing your program. The symbol you're trying to break on
       | doesn't exist? But why? This makes debuggers unusable for a
       | mediocre developer.
        
       | xkriva11 wrote:
       | I'm missing a few other items on this list that I use daily in
       | Pharo
       | 
       | - being able to open the debugger directly from the program. What
       | the "debugger" command does in JavaScript. Conditional
       | breakpoints are easier to work with if they can be directly
       | included in the source code
       | 
       | - be able to open another debugger on top of the code I see in
       | the debugger. I'm doing the stepping, I'm at a certain point in
       | the method, and I can simply mark the part of the method code
       | that has already been executed or is yet to be executed and start
       | stepping it with the next debugger. This is especially useful for
       | code without side effects. Then I can continue stepping the
       | original method.
       | 
       | - be able to have multiple debuggers open and compare their
       | status
       | 
       | - to have more freedom in the visualization of values and
       | objects. Having them open in other windows independent of the
       | original debugger, being able to interact with them using code
       | (which can be debugged independently)
       | 
       | - be able to save the state of the application and debuggers so
       | that for very hard-to-reproduce errors, I can easily recover the
       | hard-to-retrieve state and experiment with it repeatedly without
       | worrying that I won't get the state again right away
        
         | melvinroest wrote:
         | > Conditional breakpoints are easier to work with if they can
         | be directly included in the source code
         | 
         | The following code (probably, haven't tested it) allows to
         | create a conditional breakpoint only when a certain method is
         | called by tracing down the call stack
         | debuggerIsInMethod: aMethodName       ctx := GRPlatform current
         | thisContext. "grab callstack"             [ctx isNotNil]
         | whileTrue: [         (aMethodName asSymbol = ctx method
         | selector) ifTrue: [ ^ true ].         ctx := ctx sender ] ].
         | ^ false
         | 
         | So in another method you can send (call the method) that's
         | deeper in the call stack by doing:                 someMethod
         | "do work"         debuggerIsInYourMethod:
         | 'aMethodHigherUpTheCallStack' ifTrue: [ 1 halt ].         "do
         | more work"
         | 
         | Use case: Suppose someMethod is being sent/called from
         | everywhere, but you only want to debug it when
         | aMethodHigherUpTheCallStack is sent/called. With this
         | conditional breakpoint (in code), you can :)
        
       | Jorengarenar wrote:
       | >When people say "debuggers are useless and using logging and
       | unit-tests is much better,"
       | 
       | Who? Who does say that? Freshman students who barely started
       | coding?
       | 
       | >I suspect many of them think that debuggers can only put
       | breakpoints on certain lines, step-step-step through the code,
       | and check variable values.
       | 
       | That alone provides enough value to know debuggers are useful.
        
         | ithkuil wrote:
         | 25 years in the industry. I used the debugger countless times
         | because it was useful.
         | 
         | But I still default to use printf/log/tracing debugging style
         | because it's often easier to get some signal under the
         | conditions I care about and are hard to reproduce where it's
         | hard to attach a debugger.
        
         | hinkley wrote:
         | There are a lot of middle to late career developers who write
         | code that is difficult to debug. It's a feedback loop. You
         | write garbage code, the debugger is confusing, so you use the
         | debugger less, so you get less feedback on your garbage code,
         | until eventually other people can't debug your code either, or
         | only invest the energy to do so when something has gone
         | horribly wrong.
         | 
         | Or you write debugger-legible code, you use the debugger, and
         | you get feedback when the code you wrote confuses the debugger,
         | so you refactor it to be easier to diagnose the problem, then
         | you commit those changes.
        
         | maleldil wrote:
         | > Who? Who does say that? Freshman students who barely started
         | coding?
         | 
         | Example from this thread:
         | https://news.ycombinator.com/item?id=35098434
        
           | justeleblanc wrote:
           | Do we know if that person is a freshman student who barely
           | started coding?
        
             | bornfreddy wrote:
             | There are many other experienced devs (me included) who
             | simply don't see debuggers worth the effort _in most
             | cases_. There are exceptions and every competent dev should
             | know how to use one, but to me it is like using crutches -
             | I run faster and better without them. I did use debuggers
             | at the beginning though, a lot too. Then I learned to think
             | about problems and to simplify design of the code, making
             | debuggers much less useful.
        
         | q845712 wrote:
         | in my experience a large portion of go programmers seem to
         | think this. I would guess (but haven't confirmed) that there's
         | someplace where rob pike and/or russ cox are on the record
         | telling folks that they don't really need debuggers because
         | printf is great.
        
         | aflag wrote:
         | Linus Torvalds famously dislike them:
         | https://lwn.net/2000/0914/a/lt-debugger.php3
        
           | justeleblanc wrote:
           | At least he did 23 years ago. Have debuggers evolved in the
           | past 23 years?
        
             | aflag wrote:
             | They have a little. But his main point is that debuggers
             | distract developers from seeing the problem as a whole
             | rather than just understanding what's going on the vicinity
             | of that problematic line. So, the debugger incentivises
             | small, targeted fixes rather than bigger solutions for more
             | systematic problems.
             | 
             | That aspect of debuggers hasn't changed in 23 years,
             | because that's the whole point of debuggers anyway.
             | 
             | Some projects are better off with a debugger and some
             | developers are more productive with them. But also the
             | other end of the spectrum exists: projects that are better
             | off without and developers who are more productive without
             | them. Not liking debuggers don't mean someone is stupid or
             | inexperienced.
             | 
             | I'd even dare say that's more likely an experienced
             | developer to not use debuggers as much as people earlier in
             | their careers. Seasoned developers working for a long time
             | in the same code base are more likely to have insight of
             | what's going on without stepping through the process
             | execution.
        
               | thewebcount wrote:
               | > But his main point is that debuggers distract
               | developers from seeing the problem as a whole rather than
               | just understanding what's going on the vicinity of that
               | problematic line. So, the debugger incentivises small,
               | targeted fixes rather than bigger solutions for more
               | systematic problems.
               | 
               | What a bizarre take. Just today I had to use a debugger
               | to figure out where there was a deadlock in our program
               | and why. No amount of printfs would have made that
               | speedy. In the debugger I just stop the program once it's
               | hung and look at all the threads to see which threads are
               | holding which locks and which are waiting to obtain
               | locks. From there I can now see it's a lock inversion
               | because we weren't being careful about the order of
               | taking our locks in a few places. That lead to doing a
               | wider check to see if there were other similar cases, and
               | whether we needed to rethink anything in particular.
               | 
               | The debugger generally gives me an overview of the whole
               | program in a way that a bunch of printf statements can't.
               | (Which isn't to say that printf doesn't have a place in
               | debugging - it certainly does.)
        
               | yencabulator wrote:
               | > But his main point is that debuggers distract
               | developers from seeing the problem as a whole rather than
               | just understanding what's going on the vicinity of that
               | problematic line. So, the debugger incentivises small,
               | targeted fixes rather than bigger solutions for more
               | systematic problems.
               | 
               | Said differently by other old beards in this classic of
               | Ken Thompson & Rob Pike:
               | https://www.informit.com/articles/article.aspx?p=1941206
               | 
               | > When something went wrong, I'd reflexively start to dig
               | in to the problem, examining stack traces, sticking in
               | print statements, invoking a debugger, and so on. But Ken
               | would just stand and think, ignoring me and the code we'd
               | just written. After a while I noticed a pattern: Ken
               | would often understand the problem before I would, and
               | would suddenly announce, "I know what's wrong." He was
               | usually correct. I realized that Ken was building a
               | mental model of the code and when something broke it was
               | an error in the model. By thinking about _how_ that
               | problem could happen, he 'd intuit where the model was
               | wrong or where our code must not be satisfying the model.
        
           | bornfreddy wrote:
           | Thank you for the link, well worth the read.
        
       | donadigo wrote:
       | The article provides a pretty good overview of the landscape of
       | debuggers out there. I find myself using printf debugging
       | whenever something is a dynamically evolving state and I need to
       | monitor just the relevant bits of it and breakpoints when
       | something is either crashing or requiring a deeper thought about
       | how the code executes.
       | 
       | That said, I still think there's quite a bit of improvement to be
       | made, which is why I started building a new debugger for myself
       | which puts a lot more focus on breakpoint-less workflow, speed of
       | iteration and scripting ability: (demo)
       | https://www.youtube.com/watch?v=qJYoqfTfuQk It's geared mainly
       | for gamedev, but I also do use it many times to e.g debug itself.
        
         | Quekid5 wrote:
         | That's an excellent observation which I think agrees with my
         | own.
         | 
         | If you already have a good idea of the control flow (and thus,
         | expected behavior), but there's some minute detail that goes
         | wrong, you just need a few strategically placed prints to see
         | the state evolution. If the problem is control flow related,
         | you might first need to get a grasp of that before going into
         | the weeds.
        
       | Decabytes wrote:
       | I've always been curious about Debuggers. How do they work? How
       | do they connect to a program and step through it. Why can't more
       | compiled languages integrate debuggers inside of them so that you
       | can debug the program without using a separate tool?
       | 
       | And why can't we create interfaces to debuggers so that other
       | text editors can integrate with them, much like how we do we
       | LSPs?
       | 
       | *EDIT*
       | 
       | Thanks for all the responses! I've now heard from multiple
       | sources that debugging on Linux is unpleasant, and it seems like
       | the whole process is challenging regardless of the platform.
        
         | not2b wrote:
         | On Linux or BSD, the ptrace system call allows one process to
         | take control of another process; it can then observe and
         | control the other process. Breakpoints are inserted by
         | replacing an instruction with a special instruction that traps;
         | the debugger can then take control. Watchpoints (the article
         | calls them data breakpoints) are often implemented by making
         | the containing page read-only, so that a write access traps
         | (this means that there's overhead from other writes to the same
         | page, the debugger has to just resume execution silently for
         | those). A checkpoint can be implemented by forking the process
         | and freezing the fork, so execution can go back to that point.
        
         | jcranmer wrote:
         | > How do they work?
         | 
         | Painfully. If you're on Linux, you get to use a mixture of
         | poorly-documented (e.g., ptrace) and undocumented (e.g.,
         | r_debug) features to figure out the state of the program.
         | Combine this with the debugging symbols provided by the
         | compiler (DWARF), which is actually a complicated state machine
         | to try to encode sufficient details of the source language, and
         | careful reading of the specification makes you throw it out the
         | window and just rely on doing a well enough job to keep
         | compatibility with gdb.
         | 
         | > Why can't more compiled languages integrate debuggers inside
         | of them so that you can debug the program without using a
         | separate tool?
         | 
         | Because it's really painful to make debugging work properly. At
         | least in the Unix world, the norm is for all of these tools to
         | be developed as separate projects, and the interfaces that the
         | operating system and the standard library and the linker and
         | the compiler and the debugger and the IDE all use to talk to
         | each other are not well-defined enough to make things work
         | well.
         | 
         | > And why can't we create interfaces to debuggers so that other
         | text editors can integrate with them, much like how we do we
         | LSPs?
         | 
         | LSPs have the advantage of needing to communicate relatively
         | little information. At its core, you need to communicate syntax
         | highlighting, autocomplete, typing, and cross-referencing. You
         | can build some fancier stuff on top of that information, but
         | it's easily stuffed in a single box.
         | 
         | Debuggers need to do more things. Fundamentally, they need to
         | be able to precisely correlate the state of a generated build
         | artifact to the original source code of the program. This
         | includes obviously things like knowing what line a given code
         | address refers to (this is an M-N mapping), or what value lives
         | in a given register or stack location (again an M-N mapping).
         | But you also need to be able to understand the ABI of the
         | source level data. This means you can't just box it as
         | "describe language details to me", you also have to have the
         | tools that know how to map language details to binary details.
         | And that's a combinatorial explosion problem.
        
           | werat wrote:
           | > Debuggers need to do more things
           | 
           | It's true that coming up with an interface for an abstract
           | debugger is harder, but it's not impossible. Microsoft
           | created Debug Adapter Protocol
           | (https://microsoft.github.io/debug-adapter-protocol/), which
           | is conceptually similar to LSP. It's not perfect, but covers
           | most basic operations pretty well, while leaving to the
           | debugger to deal with the implementation details.
        
             | jcranmer wrote:
             | If I'm coming up with a new language, let's call it
             | Drustzig, I can implement the LSP and get support for IDEs
             | (and possibly xref tools and the like) essentially for
             | free. Now to get debugging support for my language... I
             | have to traipse around through every major debugger and beg
             | them to merge Drustzig patches to make it work.
             | 
             | The protocol you've linked (or the similar gdbserver
             | protocol) essentially implements an IDE <-> debugger
             | mapping. Well, most of one: everything is basically being
             | passed as strings, so if you want smart stuff, you kind of
             | have to build in the smart stuff yourself. It doesn't help
             | the other parts of the process; if you want to build a new
             | gdb, you have to do all the parsing of debug info for C,
             | C++, Drustzig, etc. yourself... and you have to integrate
             | the compiler expression stuff yourself. If you want to
             | build a better time-traveling debugger, or a better crash
             | debug format, or something similar, well, the gdb remote
             | protocol lets gdb talk to your tool so all you have to
             | implement is essentially low-level program state (e.g.,
             | enumerate registers of a thread, read/write arbitrary
             | memory locations, etc.). But this isn't covered by the
             | thing you've listed either, and it still relies on the
             | debugger supporting a particular protocol.
        
               | werat wrote:
               | I agree that language server and debugger are different
               | beasts, but both LSP and DAP serve a purpose of re-using
               | the same server (xrefs or debugger) with different IDEs.
               | 
               | > I can implement the LSP and get support for IDEs
               | essentially for free I mean, technically same is true for
               | DAP... You can implement DAP and get support for IDE for
               | free. But I agree that in general case implementing a
               | good debugger is harder than implementing a good language
               | server.
               | 
               | If Drustzig requires a special debugger (e.g. because it
               | uses acustom format of debug information), then you'd
               | need to implement it, yes. However, existing debuggers
               | can support new languages relatively easy if those follow
               | standard conventions (e.g. use PDB and DWARF). For
               | example, Rust support in LLDB basically comes down to a
               | custom demangler.
               | 
               | Again, I'm not saying that DAP is perfect and solves
               | debugging, but IMO it's a step in the right direction.
               | Make it popular, make it extensible. Debuggers can be
               | mostly language agnostic (within reasonable bounds), but
               | they don't _have_ to be.
        
         | ghosty141 wrote:
         | About the latter, there is an LSP equivalent called DAP (debug
         | adapter protocol).
        
       | turtledragonfly wrote:
       | I use GDB almost daily, and used Visual Studio pretty deeply for
       | many years (and still a little bit, nowadays), but I must say I
       | am still a "printf debugging" aficionado (or better: real
       | logging).
       | 
       | I like many of the features that debuggers can provide, and I
       | think this is a good article to set aspirational goals for what
       | is possible. But my lived experience has generally been that it's
       | a buggy, fuzzy, moving target in terms of overall user
       | experience. GDB, bless its heart, is still somewhat a pile-of-
       | bugs itself. I frequently have it crash, or otherwise get into a
       | "so confused it needs to be restarted" state. Visual Studio is
       | much more stable, though less powerful, too -- less scriptable,
       | at least.
       | 
       | Perhaps some day, debugger tech^H^H^H^H UX will advance to the
       | point where it really delivers on its promises consistently and
       | solidly. But after 20+ years in software, I am not holding my
       | breath (: There are some situations where a debugger is _just_
       | the thing you want (eg: hardware breakpoints can be a life-
       | saver), but I find that 's more the exception than the rule, at
       | least in the corners of the software world I've worked in.
       | 
       | Compare the above with logging: It is simple and trustworthy. As
       | you get good at designing a solid logging system, and
       | interpreting the results, your life just gets better and better.
       | If you get good at using a debugger, you can still be hit with
       | gnarly weird behaviors, debugger apoplexy, optimized-out-code
       | wackiness, etc. that are hard to control or predict.
       | 
       | Anyway, "debugger vs. logging" are often presented as some sort
       | of either/or choice, and in some sense it is (you only have X
       | time to spend; where would you like to spend it?), but in many
       | senses it is not; both have their strengths. I just find that the
       | cost/benefit for me has generally favored logging and testing,
       | over the years.
        
         | macjohnmcc wrote:
         | I have been programming professionally since 1986 and still
         | nothing beats logging or having chunks of specialized code to
         | do dumps of some data to files so you can analyze them with
         | tools better suited for the purpose. Ideally though you don't
         | want to have to modify the code to diagnose the problem
         | especially if it's a crash caught in the wild and you have a
         | chance to live debug it. I would love more useful visualization
         | tools in the debuggers (mostly VS for me) that would be very
         | helpful in all situations like debugging crash dumps.
        
           | thewebcount wrote:
           | Most of the data I work with can't be visualized by printing.
           | (I mostly work on 3D and video.) But I have found it
           | invaluable to log data, then read it into another program.
           | Sometimes I can just bing it into a spreadsheet and plot it,
           | other times I need to write something to display it or
           | analyze it. It's definitely an under-utilized technique!
        
         | crabbone wrote:
         | > It is simple and trustworthy.
         | 
         | This sounds so naive for someone with 20+ years in the field...
         | 
         | Linux, for decades, couldn't get logging to the point that it
         | at least doesn't lose messages (the problem with tail /
         | logrotate that is quite obvious once you think about it, but it
         | took many years to give up the approach).
         | 
         | I recently hit a bug where NVidia's driver abuses Linux kernel
         | logging in some tight loop by spamming log messages at insane
         | speed (happens when you have two video adapters, Intel and
         | NVidia and an external monitor). An interesting side-effect
         | here is that Linux logging tries to throttle loggers who output
         | too much, so, from the log you cannot tell what's happening
         | (because even though the system is burning calories trying to
         | print a tonne of messages, nothing really gets printed).
         | 
         | Several iterations ago I worked on a product where logging had
         | to be implemented as writes to shared memory self-styled
         | circular buffer, and because there was too much info printed
         | too quickly you only had few seconds worth of logs before
         | system crash... on a good day.
         | 
         | Needless to mention the fun of stitching together logs coming
         | from different places in your system with separate clocks.
         | 
         | Even simply processing hundreds of Gigabytes of logs on its own
         | isn't a trivial task.
         | 
         | ----
         | 
         | Many things are simple, when your task is simple. Logging is
         | just one of those things.
        
           | turtledragonfly wrote:
           | > Many things are simple, when your task is simple. Logging
           | is just one of those things.
           | 
           | I agree with much of what you said, and of course "logging"
           | is not just a single point in the solution space -- there is
           | some function "troubleshooting_pain = f(your_project,
           | your_approach)". I was trying to say that for
           | "your_approach=logging" that function tends to return smaller
           | values than for "your_approach=debugging", all other things
           | being equal, in my experience.
           | 
           | Whereas your comments seem more oriented towards the
           | "your_project" factor. Of _course_ using logs is harder on a
           | distributed system. But so is using a debugger, or just about
           | anything else.
           | 
           | Perhaps I should have said "It is _relatively_ simple and
           | trustworthy, even if it can still get hairy at the extremes.
           | "
        
             | crabbone wrote:
             | Both interactive and declarative debuggers work better in
             | distributed systems than logging because they can observe
             | events as they happen, and don't need to recreate the order
             | in which they happened from the records which are very hard
             | to make chronologically consistent.
             | 
             | Things like EBPF (which may implement sort of a declarative
             | debugger) are, perhaps the only tool you may hope to use in
             | high volume and high frequency systems.
             | 
             | If I could only choose one technology used for software
             | diagnostics, I'd choose debuggers over logging. Debuggers
             | need more effort to develop them, and they aren't very good
             | (yet), but they have potential. I don't believe that
             | logging can be substantially improved to deal with
             | difficult problems.
        
         | RMPR wrote:
         | 20+ years is... a lot. I do agree with the sentiment though. At
         | first I was printf debugging because didn't know better. Then
         | discovered debuggers and my mind was blown. But when I reached
         | the point where I hit bugs that would magically disappear when
         | running the program through a debugger, I finally understood
         | that there's value in becoming good at both debugging styles.
        
           | macjohnmcc wrote:
           | Debugging issues with multithreaded code can be difficult
           | because you could be looking at race condition that only
           | happens when the code is running at full speed and debugging
           | pausing one or all threads could give you an different
           | experience than the real world.
        
             | kaba0 wrote:
             | (Debug) logging can also change the
             | frequency/order/synchronization of threads masking over
             | race conditions.
        
               | turtledragonfly wrote:
               | Yeah, I have fought with this occasionally. It's one of
               | the handful of "gotchas" you need to think of when
               | designing logging, and interpreting its results.
               | 
               | Sometimes you can improve the situation by sticking
               | things in a bigger memory buffer and tightening control
               | of when things get flushed to disk, but there's always
               | that fundamental "observing the system will change it"
               | problem. Similar issues arise with a debugger too, of
               | course.
        
               | macjohnmcc wrote:
               | Yeah I had to debug some code with 5 threads that were
               | supposed to be synchronized through the use of several
               | semaphores that was a bear to pin down.
        
         | thewebcount wrote:
         | I think one problem I have with gdb and to a lesser extent lldb
         | is that when using them with an IDE, it's just a janky
         | connection between a command-line program and a GUI program.
         | Back in my macOS 7-9 days, using something like CodeWarrior, or
         | even Vusal C++ on Windows, stepping was so fast and smooth. It
         | would respond instantaneously. I can now press the step button
         | like 5 times and watch as it steps to each line. My machine is
         | a bazillion times more powerful, so I have to wonder if that
         | interface between the UI and the debugger is part of the
         | problem. I realize there's also protected memory and separate
         | processes, and all that. But man it's insane how much faster
         | our machines are and how much worse just single stepping is.
        
           | turtledragonfly wrote:
           | You might want to look into the "TUI" mode in GDB. It's an
           | ncurses-style interface, where it shows you the current code,
           | and current line, and you can step along "visually." It is
           | fast. Press Ctrl-L to re-draw the screen when the display
           | gets messed up.
           | 
           | ...
           | 
           | Then, you may notice that TUI mode steals certain keyboard
           | commands, such as up/down arrow to scroll the source listing,
           | rather than navigating command history.
           | 
           | Then, you might enable Vi-mode for GDB's readline, so you can
           | use "j/k" to navigate command history, even in TUI mode. Plus
           | Vi-mode is just better (:
           | 
           | Then, you may find that certain things don't work quite right
           | in Vi-mode, because it's not the default and doesn't get as
           | much testing. But you fuddle along because it's better than
           | the alternative.
           | 
           | And thus you have arrived at my basic situation (:
        
       | javier_e06 wrote:
       | Just yesterday I was trying to dive into a container running some
       | c program and I took a look at gdb... again, for the n time.. and
       | the whole ordeal of the gdbserver and gdb as a client and the
       | symbol table yadda yadda yadda I just rebuild the program with
       | some printfs displaying data, filename and line number and re ran
       | it. Done. gdb is stifling.
        
         | sigjuice wrote:
         | Containers are the real ordeal and not just here.
        
         | mark_undoio wrote:
         | Have you tried using a Time Travel Debugger to record the
         | process and then just debug the recording outside?
         | 
         | You can use rr or LiveRecorder (commercial product, which I
         | work on) to generate the recording non-interactively then debug
         | it "locally". Avoids the need to set up a client/server
         | configuration, so long as you don't need to modify variable
         | values at runtime, etc.
        
       | AtNightWeCode wrote:
       | People who learn the tools are much more productive. Two features
       | I often miss is that I sometimes want to copy whole data
       | structures as JSON. There are workarounds for it in some langs.
       | Then there is this annoying thing that some things are optimized
       | away even in debug mode. So you have to add in variables just for
       | debugging which is of course an anti-pattern.
        
       | zwieback wrote:
       | I'm a debugger lover and Visual Studio's is pretty great but GDB
       | is also awesome. However, when debugging embedded systems with a
       | good trace probe it's another whole world, using on-chip trace
       | capabilities and being able to break on things like register or
       | peripheral access makes it even more exciting. And expensive,
       | unfortunately, since it's such a niche thing.
        
         | hnthrowaway0315 wrote:
         | Just curious how does one build the tools you talked about? I'm
         | referencing both hw and sw: hw part I guess is "on chip trace
         | capabilities" and sw is "break on registers or peripheral
         | access".
         | 
         | I'm not an embedded dev neither am I a debugger developer but
         | I'm playing with toy OS dev so just curious.
        
           | zwieback wrote:
           | The setup I'm most familiar with is ARM-based
           | microcontrollers, here's an overview from their docs: https:/
           | /developer.arm.com/documentation/ihi0014/q/Introducti...
           | 
           | and the debugger from Keil (used to be independent, now part
           | of ARM): https://www2.keil.com/mdk5/debug
        
             | hnthrowaway0315 wrote:
             | Ah thanks, I thought they are yet to be made.
        
       | albertzeyer wrote:
       | It's sad that Python does not really support some of these
       | debugging methods.
       | 
       | E.g. you cannot really watch variable changes. There are some
       | workarounds, like writing a custom __setattr__ or
       | __setattribute__ in case of an object, or checking all STORE_*
       | operations. https://youtrack.jetbrains.com/issue/PY-30387
       | https://github.com/gaogaotiantian/watchpoints
       | 
       | Reverse debugging is also sth I would like to have, and there are
       | a few projects to support this, but it's not really well
       | supported in standard CPython.
       | https://foss.heptapod.net/pypy/revdb https://pytrace.com/
        
         | crabbone wrote:
         | Python debugger is really sad for _many more_ reasons:
         | 
         | 1. Cannot interrupt a running program to get debugger prompt,
         | you have to code that functionality in yourself.
         | 
         | 2. Cannot deal with threads or multiple child processes because
         | it gets confused by where its output should go, and so it
         | appears to be stuck, prompt doesn't show up, or input isn't
         | being accepted.
         | 
         | 3. If someone writes an overly general except, debugger won't
         | ever be called because in order to be called it relies on
         | exceptions.
         | 
         | 4. Debugger cannot evaluate comprehensions correctly due to
         | scoping issues (not as much of a debugger fault as much of the
         | language fault).
         | 
         | 5. Debugger cannot consistently modify variables inside the
         | function call. Depending on circumstances if you execute an
         | assignment statement in debugger, the value may or may not be
         | set to the variable in current stack frame.
         | 
         | 6. Debugger doesn't step into iterator implementation (iirc, I
         | might be confusing with tracing).
        
       | Lichtso wrote:
       | One style that haven't seen discussed is a hybrid / mixture of
       | in-code (e.g. printf trace-logs) and out-of-code (using external
       | tools like an attached debugger) debugging.
       | 
       | What I do is I encode conditional breakpoints in the source code,
       | (compile and) run the program with a debugger attached. The nice
       | thing is you can have complex conditions using all kinds of
       | functions from your surrounding code and you can have them
       | permanently, even check them into the VCS. It is kind of like
       | placing asserts which don't panic but trap into the debugger and
       | they can be globally enabled / disabled.
        
         | ghosty141 wrote:
         | Conditional breakpoints are supported for most dynamic
         | languages. With static ones like c++ your way is the only
         | practical way I believe
        
           | fluoridation wrote:
           | Visual Studio supports conditional and printing breakpoints
           | for both C and C++. I tend to use them rarely because they
           | really hurt performance, though. I only use them if I need to
           | be able to turn them on and off at will, which hardcoded
           | __debugbreak()s don't allow, obviously.
        
             | mark_undoio wrote:
             | I helped implement fast conditional breakpoints in UDB
             | (time travel debugger for Linux) -
             | https://www.youtube.com/watch?v=gcHcGeeJHSA
             | 
             | We used GDB's conditional breakpoint bytecode
             | https://sourceware.org/gdb/onlinedocs/gdb/General-
             | Bytecode-D... to get a speedup in the thousands of times vs
             | plain conditional breakpoints.
             | 
             | That works for us because we've got in-process agent code
             | that can evaluate the breakpoint condition without
             | trapping. It should be possible to do this in other
             | debuggers with a bit of work, though we have the advantage
             | of in-process virtualisation to help hide this computation
             | from the process.
        
           | thewebcount wrote:
           | I program in a hybrid of
           | C/C++/Objective-C/Objective-C++/Swift and can use conditional
           | breakpoints in all of those languages in lldb and Xcode.
           | There's nothing special about conditional breakpoints that
           | makes them not work in compiled or static languages.
        
       | Yoofie wrote:
       | A whole world of creative opportunities open up when the
       | toolchain and related debugging tools don't suck. Check out this
       | wild video[1] of someone modifying and debugging a game in real
       | time.
       | 
       | [1]: https://youtu.be/72y2EC5fkcE
        
       | leni536 wrote:
       | > Breakpoints, oh my breakpoints
       | 
       | This is a good list, but one thing missing: catchpoints.
       | Particularly useful with time travel debugging. catch throw +
       | reverse-continue immediately tells you who threw an exception,
       | and you can continue debugging from there.
        
       | RcouF1uZ4gsC wrote:
       | I have mostly found the people who dismissed debuggers tended to
       | be more Unix/Linux people, probably because raw gdb is such a
       | huge pain to use. Windows developers and Visual Studio developers
       | where the debugging experience is so easy, tend to sing the
       | praises of debuggers. I wonder if it a bit of sour grape for the
       | Unix/Linux crowd?
        
         | TheRealDunkirk wrote:
         | As much of a Linux zealot as I am, you make a valid point. In
         | college, I had a TA help me debug a C program on a VAX, and he
         | blew through finding the problem using its native debugger, and
         | wouldn't explain what he did. (I had an O where a 0 should have
         | been. Or vice versa. It was a worse problem back in the days of
         | actual terminals. He found it in literally 30 seconds after I
         | had been bashing my head on the printout for a couple hours.)
         | Anyway, it took me probably 10 years of professional coding
         | before I discovered gdb, and then realized what that TA had
         | done. All at once, I realized how far you could get without an
         | actual debugger, and also why he never bothered to try to
         | explain it to me. I wasn't ready. Not by a long shot. Seeing
         | the right-click options on breakpoints in Visual Studio was...
         | revelatory.
        
         | pjmlp wrote:
         | Depends on the UNIX, Solaris, HP-UX and NeXT/macOS are all
         | UNIXes with good debugging experience.
         | 
         | Naturally none of them have had raw gdb, rather modern
         | graphical debuggers.
        
       | yellowjobby wrote:
       | > _We can snapshot the program whenever something non-
       | deterministic happens (syscall, I /O, etc) and then we just
       | reconstruct the program state at any moment by rewinding it to
       | the nearest snapshot and executing the code from there. This is
       | basically what UDB, WinDBG and rr do._
       | 
       | QEMU does this too. This plus its GDB stub means one can time-
       | travel-debug pretty much anything on any emulated architecture.
       | 
       | https://www.qemu.org/docs/master/system/replay.html
        
       | jpmonettas wrote:
       | This is another example, a tracing time travel debugger for
       | Clojure https://github.com/jpmonettas/flow-storm-debugger
       | 
       | Supports a bunch of stuff described there and more.
        
       | choletentent wrote:
       | This piece from Linus on why he doesn't like debuggers resonates
       | with me [1], although I confess I use Python pdb quite often.
       | 
       | [1] https://lkml.org/lkml/2000/9/6/65
        
         | sokoloff wrote:
         | I really enjoyed this thought in that mail: "Tough. There are
         | two kinds of reactions to that [time lost from a bug you
         | introduced in kernel dev]: you start being careful, or you
         | start whining about a kernel debugger."
        
           | matheusmoreira wrote:
           | He also reminds people why everyone pulls from torvalds/linux
           | to this day:
           | 
           | > People think I'm a nice guy, and the fact is that I'm a
           | scheming, conniving bastard who doesn't care for any hurt
           | feelings or lost hours of work if it just results in what I
           | consider to be a better system.
        
         | heinrichhartman wrote:
         | Contrasting perspective from John Carmack [1] who drops into a
         | debugger all the time to explore state, understand program
         | behavior, debug things, etc.
         | 
         | Looks like there is no universal golden path. Differing
         | environments make different approaches effective.
         | 
         | [1] https://www.youtube.com/watch?v=tzr7hRXcwkw
        
         | jcranmer wrote:
         | That sort of argument is somewhat defensible in a context like
         | the kernel, where when things go haywire, you can't really
         | expect there to be enough sanity to have a debugger work. But
         | very little code runs in such a context, and it turns out that
         | a well-written debugger has incredible features.
         | 
         | Also, Linus is writing this 22 and a half years ago, where the
         | capabilities of debuggers were... far, far less. Time-
         | travelling debuggers is really a game changer, just having the
         | ability to travel back in time to figure out who set the value
         | that causes the code to crash. Hot reload is also a wonderful
         | thing (unfortunately, the fragmentation of tooling in Linux
         | makes getting this working properly very difficult).
        
           | deckard1 wrote:
           | > Time-travelling debuggers is really a game changer
           | 
           | Core dumps have existed forever. They give you a stack trace
           | and register values at the time of crash. Even better, you
           | don't need a debugger running at the time of crash and you
           | can dig into dumps sent from nontechnical users.
           | 
           | Sure, Bret Victor's demo was cool. But time travel debugging
           | is so completely oversold at this point that I can't take
           | anyone seriously that mentions it.
        
             | jcranmer wrote:
             | I've debugged core dumps before. It's not been a
             | particularly pleasant experience--good luck trying to do
             | something like `call V->dump()` (dump out an easy-to-
             | understand representation of a complex value to stdout...
             | oh that doesn't exist anymore, can't use that
             | functionality!)
             | 
             | The most useful aspect of time-travel debugging for me,
             | personally, has been when the test case that causes a crash
             | is refusing to be reduced, and the function that crashes
             | does so on like the 453rd time it's called. Jumping
             | straight to the crash, then reverse-continuing to a break
             | point cuts out so much time of debugging (especially
             | because it saves you if you accidentally continue the
             | breakpoint one too many times; otherwise, you'd have to
             | start the entire, tedious process from the beginning).
        
               | glandium wrote:
               | Something else that time-travel debugging helps a lot
               | with, is that in an awful lot of cases, what you have in
               | the crash dump is a broken state, with no way to identify
               | how that state happened to be. Like "why the hell does
               | this variable have this value?". Sure, you can do the
               | whole digging work to find what can possibly change that
               | variable, and try different scenarios to hit them in a
               | debugger at the moment the problem might appear in a new
               | session that is not even guaranteed to produce the same
               | crash. But with record-and-replay type things, you just
               | set a watchpoint on the value, continue backwards, and
               | there you go, you find where the value comes from.
               | 
               | Now, imagine you did do that manually and spent a lot of
               | time finding that location. In many cases, that only gets
               | you one step closer to the root cause, and you have to
               | repeat the operation multiple times. Yes, you _can_ do
               | that without record-and-replay, but do you really want
               | to? Do you want to spend hours doing something that could
               | take you minutes?
               | 
               | (And that's not even mentioning even worse cases, where
               | the value you're tracking goes between processes via IPC)
        
           | eschneider wrote:
           | Lots of (most?) real-time and embedded code run in a state
           | where suspending/resuming really doesn't work in any useful
           | fashion, so it's logging, and minimal logging at that to
           | figure out what's going wrong in-situ.
           | 
           | That said, much benefit is gained by writing the complicated
           | bits in such a way that they can be tested/debugged/examined
           | independently on a host system.
        
             | realo wrote:
             | I'm with you there!
             | 
             | Be happy if you can flash a led... happier if you have two
             | speeds... and Nirvana if it is a multi-color led.
             | 
             | Intense jealousy of your colleagues who have TWO leds on
             | their boards!
        
             | nuancebydefault wrote:
             | Hmmm I have 20+ years of embedded sw programming experience
             | and can tell you the reason that embedded software is
             | oftentimes not easily debugged using a debugger, is the
             | crappyness of the debugger solution (debug probe, its
             | firmware and its eco system). Also, high end embedded ICs
             | often contain a serious amount if silicon bugs. The fact
             | that it needs to run real-time, is mostly not in the way of
             | the debugging process. In other words, embedded processors
             | and their eco systems tend to be sub-par in terms of
             | developer friendlyness. Addendum SHARC ADSP anomaly list
             | https://www.analog.com/media/en/dsp-
             | documentation/integrated...
        
       | armchairhacker wrote:
       | This is what a good debugger can do:
       | https://twitter.com/yiningkarlli/status/1628612150041382912
       | (Tomorrow Corporation)
        
         | MikeSchurman wrote:
         | This was on hacker news a few days ago but there wasn't much
         | interest, which is surprising.
         | 
         | Setting a data breakpoint, then rewinding time, then fixing the
         | error in code (while game is running), hot-reloading code and
         | continuing the run with it fixed. All from a recording of a
         | panic dump from another user.
         | 
         | This is next level stuff. Extremely impressive.
        
         | hnthrowaway0328 wrote:
         | Wow I'm just 5 mins into the video and this looks ridiculously
         | powerful. This is essentially an integrated development
         | environment for game development with everything every
         | programmer dreams about!
         | 
         | How do I learn to make such tools? From what I heard everything
         | in the video is built in-house including the language and
         | compiler. Although I understand many game studios do similar
         | stuffs this is by far the most impressive I heard about.
        
         | euiq wrote:
         | Great demo! The article also has a link to it.
        
       | invalidname wrote:
       | I like this article. There are more such tips like that in this
       | series: https://www.youtube.com/watch?v=A919j_5qE0k
        
       | sebastianconcpt wrote:
       | a good debugger supports different kinds of breakpoints, offers
       | rich data visualization capabilities, has a REPL for executing
       | expressions, can show the dependencies between threads and
       | control their execution, can pick up changes in the source code
       | and apply them without restarting the program, can step through
       | the code backward and rewind the program state to any point in
       | history, and can even record the entire program execution and
       | visualize control flow and data flow history.             I
       | should mention that the perfect debugger doesn't exist.
       | 
       | People pretends Smalltalk doesn't exist?
        
         | nijave wrote:
         | >can step through the code backward and rewind the program
         | state to any point in history
         | 
         | This seems impossible for anything that modifies external
         | state.
         | 
         | Say 1. Open database connection 2. Commit data 3. Close
         | database connection
         | 
         | How would you rewind right before #2 if you've already
         | completed step #3? You'd need the socket/connection you already
         | closed
         | 
         | Unless that means something more like "keep a running record of
         | program state over time"
        
           | crabbone wrote:
           | My guess is that you could attempt something like this by
           | recording system calls as your program executes and then
           | replay them in whichever order you need.
           | 
           | You would need a lot of storage space for some programs, but
           | in simple cases that might still be useful.
           | 
           | So, in your example, playing the program back wouldn't really
           | try to read from a closed socket, it would just hit
           | debugger's database of stored system calls at a particular
           | point and retrieve the stored response from that call.
           | 
           | This could get weird though if the program modifies itself as
           | it executes. Not sure what to do in such case, but maybe
           | there's a way to deal with it in special cases, not in
           | general...
        
           | yencabulator wrote:
           | Let's say out lout the part that too often goes unstated:
           | 
           | You can rewind & replay a _fixed execution of the program_.
           | 
           | All the external interactions are recorded, and the replay
           | can only literally replay what already happened.
           | 
           | You cannot change a variable or edit the code and branch off
           | into a different execution path, that talks to the outside
           | world differently; the rest of the world did not get rewound
           | with the program.
        
           | matsemann wrote:
           | Yeah it's not perfect, but it often works well enough. Just
           | another benefit of isolating code into stateless stuff.
           | 
           | One use case I have is for instance debugging a bug that's
           | hard to trigger. When it finally happens and my breakpoint is
           | hit, I can edit the code, hot swap it live, drop the current
           | frame and then make it call my updated function again as if
           | the original call never happened.
        
           | jcranmer wrote:
           | Time travel debugging generally works on record-and-replay,
           | with there being a lot of research into figuring out what you
           | need to record to get deterministic replay. Recording the
           | results of system calls is a necessary step [1] to get
           | deterministic replay, and that lets you replay even things
           | that rely on modifying external state like database
           | connections.
           | 
           | [1] Necessary, but not sufficient. Multithreaded applications
           | require a lot more care, and rr relies on very accurate
           | hardware counters to get multithreaded executions working
           | correctly (and not all hardware supports these hardware
           | counters!).
        
           | mark_undoio wrote:
           | > Say 1. Open database connection 2. Commit data 3. Close
           | database connection > > How would you rewind right before #2
           | if you've already completed step #3? You'd need the
           | socket/connection you already closed
           | 
           | The key thing is that you're really rewinding the _world, as
           | visible by the program_
           | 
           | The program doesn't actually know what the network
           | transaction with the database was, it just knows what
           | syscalls it made and what results they returned. If, whenever
           | it gets to talking with the database, you provide the same
           | result as last time then it can't tell the database is gone.
           | 
           | This is self-supporting: if you do this consistently with
           | external sources of information then the program will never
           | go down any new code paths, guaranteeing that you'll always
           | have a ready answer recorded when it needs it.
        
           | layer8 wrote:
           | Only the program state gets rewound, not anything external to
           | the process (or whatever is hosting the program). Just like
           | hitting a breakpoint only pauses the program, not the
           | external world.
        
           | fluoridation wrote:
           | Yes, that's what it means. Ideally you'd record the state of
           | memory and CPU after each instruction. In practice you can
           | take snapshots at regular intervals and hook system functions
           | to record their inputs and outputs. If a call has to be
           | replayed the debugger intercepts it and gives the debuggee a
           | recorded result.
        
           | [deleted]
        
           | Ao7bei3s wrote:
           | Have a look at https://rr-project.org/ (features and
           | motivation are on that page). (Also mentioned in the
           | article.)
           | 
           | It works on recorded state. It's not about executing a
           | program again, it's about root causing a failure by going
           | back in time after the failure happened. You can do things
           | like start with a crash, retroactively set a break point and
           | reverse run back in time until you hit it.
           | 
           | It's for real world applications. It was written specifically
           | to debug Firefox, and has since been used for other
           | applications of similar size.
           | 
           | It's basically GDB with extra commands, so very easy to use
           | and learn if you know GDB. Highly recommend.
        
           | cmontella wrote:
           | It can work but it depends on the system. One field in which
           | it's very useful is game development. I'm working on a
           | language that has time travel debugging as a feature, so you
           | can rewind a game back to a previous state. I've found it
           | useful when there's a momentary bug that would be hard to
           | recreate. With time traveling, you pause, rewind, inspect
           | state, fix the bug in situ, then resume from that point to
           | check that the behavior is correct.
           | 
           | Here's an example: http://docs.mech-
           | lang.org/#/examples/bouncing-balls.mec
           | 
           | If you want these kinds of features in other systems, they'll
           | have to be architected to support them. For example, the
           | external database will have to be rewound as well. If it
           | doesn't support that feature, then cool language-level
           | debugging features won't be as useful.
        
         | jayd16 wrote:
         | Which of these can't visual studio or intellij do?
        
           | pjmlp wrote:
           | hot code reload is very constrained in what can be changed.
           | 
           | It is great that it exists, but it isn't Smalltalk where at
           | any moment the debugger can come up, you can change the world
           | and hit continue without any issues, with all live instances
           | updated to the latest set of changes.
        
         | draegtun wrote:
         | >> People pretends Smalltalk doesn't exist?
         | 
         | And Factor and many implementations of Lisp too!
        
         | shadowofneptune wrote:
         | Smalltalk images offered introspection, debugging of running
         | processes, etc., but some of the later features listed seem
         | quite impractical to add to the language. "Record the entire
         | program execution," for instance.
         | 
         | That's more or less the point the author makes on the next
         | line: "Different tools support different features and have
         | different limitations." The ideal debugger does not necessarily
         | make for the most ideal programming environment, just as a
         | plane made out of steel is great for structural stability but
         | may not be a good plane.
        
         | HdS84 wrote:
         | Most of it can be done by the c# debugger, especially in the
         | laid versions of visual studio.
        
         | aidenn0 wrote:
         | Is there a ST implementation with a time-traveling debugger?
         | That's probably the one feature I miss when using sbcl/slime.
        
           | igouy wrote:
           | Depends what functionality you think "a time-traveling
           | debugger" must provide, Smalltalk implementations usually
           | provide something like this:
           | 
           | https://cuis-smalltalk.github.io/TheCuisBook/The-
           | Debugger.ht...
        
             | aidenn0 wrote:
             | To qualify as "time-travelling" You should at a minimum be
             | able to step and run backwards and inspect the values of
             | variables at a previous point in time.
             | 
             | Ideally all debug features (breakpoints, watchpoints,
             | tracepoints &c.) will work running both forwards and
             | backwards. To meet the standard of the "perfect debugger"
             | from the TFA intro, I would say you should be able to run
             | backwards, modify a function, and run forwards again. I'm
             | not aware of any debuggers that let you do this.
        
               | sebastianconcpt wrote:
               | Yeah. That's the normal way to work for a Smalltalk
               | debugger.
               | 
               | I remember once debugging a thread without killing it.
               | I've asked a customer to click on a link that would get
               | to a bug while having remotely opened the IDE that was
               | serving that user session, setting a conditional halt.
               | Seen it halting, fixing the bug and saving the method
               | with the halt removed and let the thread run. All the
               | user saw was a long request that ended in service instead
               | of that bug.
        
               | aidenn0 wrote:
               | I don't see any "running backwards" mentioned in this
               | story. I'm familiar with "break, fix, restart" debugging
               | from Lisp (which had a lot of cross-pollination with
               | smalltalk)
        
               | leni536 wrote:
               | > Ideally all debug features (breakpoints, watchpoints,
               | tracepoints &c.) will work running both forwards and
               | backwards.
               | 
               | Executing arbitrary code in the program's context is also
               | a regular debug feature, but I don't think many
               | reversible debuggers allow this.
        
               | aidenn0 wrote:
               | Hence "ideally"
        
       | WirelessGigabit wrote:
       | I am a huge proponent of debuggers.
       | 
       | Being able to look at state step by step without having to stop,
       | add log statements, recompile and go back are too slow (for me).
       | 
       | What concerns me more is that is that I end up working with
       | contractors with 5+ years who don't know how to set up a debugger
       | for the code they are working on.
       | 
       | And that concerns me. It's not OR logging OR debugging. It's
       | both. You use the best tool for the job.
        
         | indymike wrote:
         | I usually wait for someone to accidentally try to merge a bunch
         | of conditional logic wrapping logging, prints or stdio.write
         | type stuff and take that as the opportunity to introduce them
         | to the debug tab in the editor... and give a quick tutorial on
         | conditional breakpoints and how to go up and down the call
         | stack.
         | 
         | It's amazing how many developers go sometimes a decade before
         | they learn to use a debugger.
        
         | ska wrote:
         | It's definitely both. Ideally the debugger is never your first
         | resort.
         | 
         | An extremely common failure mode for less experienced
         | developers is messing about in a debugger all afternoon for a
         | problem that should be solvable in 5 minutes. It's a very slow
         | way to learn.
        
           | fluoridation wrote:
           | Weird, I've never run across that. I have wasted a lot of
           | time rebuilding and retesting figuring out where to put a
           | print statement and then figuring out what and how to print
           | it, where a debugger would have let me turn breakpoints on or
           | off and watch different variables in the same run, all
           | without touching the code. I don't think I've ever thought
           | "gee, this debugger sure is getting in my way! I wish I could
           | do printf-debugging instead."
        
             | ska wrote:
             | The world isn't printf debugging or debugger only.
             | 
             | Some things are really amenable to the debugger, especially
             | simple bugs. Really especially ones that should have been
             | caught by static analysis anyway but that's a separate
             | issue.
             | 
             | The issue is that firing up a debugger can easily become a
             | fishing expedition. If you don't understand the system
             | behavior and you don't know where the real problem is, you
             | can end up manually stepping through many many layers of a
             | system trying to see what it's doing. Add asynchonicity and
             | this can take hours before you get to the 20 lines of logic
             | that is the actual problem.
             | 
             | Ideally you have better logging and introspection, so the
             | problem is caught in a way that leads you to those 20 lines
             | immediately.
             | 
             | And of course design can make a huge difference in how
             | understandable and localizable things are.
             | 
             | The problem a lot of inexperienced developers run into is
             | if they are used to having a featureful debugger available
             | it can become the only tool they have in their toolbox, and
             | they are rewarded by how quickly it helps them fix the kind
             | of mistakes their inexperience leads them to making a lot
             | of. When they run into a real issue, they can be hitting
             | "step" for hours....
             | 
             | The other pernicious side of this particular coin is that
             | it can lead to localized understanding of the problem and
             | make it really easy to apply a localized solution. With
             | inexperienced people this can quickly lead to band-aid
             | solutions all over the place. With any luck someone more
             | experienced is looking it over and pointing out they should
             | go after the actual problems, but left unchecked it can
             | make a real mess.
        
             | euiq wrote:
             | I assume the parent comment is talking about "thinking"
             | (not that I have any personal experience with that).
        
           | beezlewax wrote:
           | I've found the exact opposite of this starting out. When I
           | started using the debugger a whole lot of things just clicked
           | into place.
        
         | bornfreddy wrote:
         | To offer an opposite view, I haven't used (or missed) a
         | debugger for decades, in a whole range of programming languages
         | and environments. It was only recently when I had to
         | disentangle some legacy spaghetti code that I have set it up -
         | and once refactoring is done, I'll probably shell it again.
         | 
         | The reason I don't usually use (/need) a debugger is that I
         | know how the code should behave, because I thought about it in
         | advance. Or, if it is not mine, I expect it to be readable and
         | maintainable, otherwise I push for cleanup instead. If it is
         | written in small manageable chunks, covered with tests and if
         | it has good logging (which is necessary anyway - there will
         | probably be no debugger available in production), I simply
         | don't see the added value of a debugger. If it is not, it is
         | not a debugger that is missing. :-)
         | 
         | That said, it is still a valuable learning tool because it
         | helps understand the flow of the code, and it helps when
         | refactoring spaghetti code. Reverse engineering also comes to
         | mind... But other than that I can't be bothered to set it up
         | either.
        
       | frou_dh wrote:
       | If someone's preferred language/framework/environment doesn't
       | have a good debugger, or if they simply don't know how to set it
       | up, some percentage of them will start insisting that they don't
       | even want one anyway.
       | 
       | This is the basic Sour Grapes concept.
        
         | oneshtein wrote:
         | Debugger is a tool, not a goal. I can reach a GOAL without
         | debugger, using my software development skills only. I prefer
         | to invest into quality of code, write more comments and
         | documentation, perform refactoring, delete unused code, write
         | test case to reproduce the bug, add more command line
         | parameters, write better error messages with more data, and so
         | on, than to invest time into a debugger. I'm the Software
         | Developer, not a Software Debugger.
        
       | carapace wrote:
       | Seems as good a hook as any to hang this rant...
       | 
       | > a good debugger supports different kinds of breakpoints, offers
       | rich data visualization capabilities, has a REPL for executing
       | expressions, can show the dependencies between threads and
       | control their execution, pick up changes in the source code and
       | apply them without restarting the program, can step through the
       | code backward and rewind the program state to any point in
       | history, and even record the entire program execution and
       | visualize control flow and data flow history.
       | 
       | TL;DR: That's fantastic! But if you need any of it you're already
       | "doing it wrong".
       | 
       | Context: I've recently been using a compiler for a C-like
       | language that targets a simple 64-bit VM, the point being there's
       | no debugger for the stack (the VM is written in Rust; I could put
       | a debugger on that and just deal with the extra level of
       | abstraction using e.g. features like those described above.)
       | 
       | So how do you cope?
       | 
       | Design simple and robust systems that can be understood in action
       | easily via printf/log. Use tried-and-true off-the-shelf
       | components (including algorithms and datastructures.) Write in
       | small increments, compile often, never proceed without complete
       | confidence in your understanding of the system. When the
       | inevitable bugs appear, and they can't be discovered through just
       | thought or a rereading of the code, then you bisect: often
       | literally (LoC) but also conceptually. Solve the puzzle and
       | eliminate the places where puzzles can hide in the first place.
       | 
       | I quite literally see how a having a debugger would lead to worse
       | code. (Even if you don't use it.) This isn't news btw:
       | 
       | > Everyone knows that debugging is twice as hard as writing a
       | program in the first place. So if you're as clever as you can be
       | when you write it, how will you ever debug it?
       | 
       | -- Brian Kernighan, 1974
       | 
       | (The _feeling_ is different: like instead of a wild adventure,
       | programming this way feels like gardening in an orderly and well-
       | kept garden.)
        
         | cgh wrote:
         | In the real world, programmers are brought into projects with
         | literally millions of lines of code that have been developed
         | over years or decades by tens or hundreds of programmers. They
         | are under intense time pressure to fix bugs that they likely
         | didn't create. This is normal and debuggers are invaluable. No
         | offence, but what you're describing is a lovely ideal that only
         | exists if you work mostly alone or on small things.
        
         | nuancebydefault wrote:
         | 'Design simple and robust systems...' and 'never proceed
         | without confidence in your understanding...'. When I was a
         | freshman, I would have said that. Then I entered the
         | professional world. Complex problems are solved using complex
         | solutions. Complete understanding is non-existing. Robustness
         | is a relative term.
        
           | carapace wrote:
           | > never proceed without [total] confidence in your
           | understanding
           | 
           | The fellow I learned that from was apenwarr, FWIW.
        
         | AndriyKunitsyn wrote:
         | That's all good, but if the problem itself operates with more
         | data than a human can reasonably operate with, this no longer
         | applies.
         | 
         | I do 3D meshes programming. The amount of vertices, planes and
         | other geometrical entities that I need to operate with in my
         | algorithms is too big for me to do printf debugging. I can't
         | just look on a long list of 3D vertex coordinates and visualize
         | the mesh they make in my head. Moreover, I had to come with my
         | own 3D visualization tool, once I got fed up with pen and paper
         | for converting 3D coordinates to actual meshes, or using
         | Blender for that. For me, a debugger (and debugging tools in
         | general) is irreplaceable. I'm not saying my field is the most
         | complex one, because it clearly isn't - but in that field, I
         | can't think of any other way I could do what I do.
        
           | akkartik wrote:
           | But it sounds like you had to come up with your own tools
           | because a debugger wasn't enough?
           | 
           | That's my biggest criticism of debuggers having used both
           | approaches: you forget that sometimes you need new tools.
           | Whereas with prints you're constantly building new
           | instrumentation for yourself.
           | 
           | https://merveilles.town/@akkartik/106138280776488247
        
             | CyberDildonics wrote:
             | _Whereas with prints you 're constantly building new
             | instrumentation for yourself._
             | 
             | What does this mean? Formatting text differently is still
             | text, which the parent post was just explaining doesn't cut
             | it.
        
               | akkartik wrote:
               | I'm constantly finding new places in my program to add
               | prints to. This isn't just copy changes. (Though that has
               | also had a huge impact occasionally in understanding
               | something. Imagine emitting 2 variables and then focusing
               | on 1 of them. A pattern can pop out of a screenful of
               | iterations of a loop when things line up just right.)
        
               | aidenn0 wrote:
               | There are debuggers that will let you halt your program,
               | go back in time, and then print out each place where a
               | specific variable changes. If that's "interacting with
               | your program through a pane of glass" then _shrug_.
        
               | akkartik wrote:
               | I'm very happy to hear it! Can you point me at a few of
               | them?
        
               | aidenn0 wrote:
               | Mentioned in TFA is "rr"; it serves up to gdb the history
               | of the program and you can use tracepoints (also
               | mentioned in TFA) to essentially retroactively add
               | prints. Gdb is ... not particularly ergonomic, but it is
               | eminently scriptable and writing programs to generate
               | arbitrary logs post-facto is a useful skill.
               | 
               | Undo is another one for Linux programs, and I think I've
               | seen developers from them post here.
        
               | mark_undoio wrote:
               | Hallo, I'm an Undo developer _wave_
               | 
               | Indeed, using GDB's version of tracepoints (dprintf -
               | https://doc.ecoscentric.com/gnutools/doc/gdb/Dynamic-
               | Printf....) is really powerful and replaying a trace with
               | these installed is to generate "logs I wish I'd had" is
               | exciting.
               | 
               | It also has potential use if:
               | 
               | * you'd like to diff two executions (e.g. one successful,
               | one failing) - you probably can't just compare raw
               | execution as there'll be uninteresting variation but
               | comparing extended logs could be useful
               | 
               | * you are debugging something at a customer site - they
               | might not let you debug directly but you could iterate on
               | plain text logs by shipping them additional tracepoints
               | to run on a recording
               | 
               | We thought this was useful enough that we implemented a
               | tool with its own DSL for simply specifying post factor
               | logging "probes":
               | https://docs.undo.io/PostFailureLogging.html
        
               | akkartik wrote:
               | The big problem with debuggers in my experience is the
               | difficulty of setup. They take a lot of code to build,
               | and if you use a slightly different language or compiler
               | or OS you're SoL. Or at least facing some rabbitholes of
               | unknown depth.
               | 
               | The most sophisticated debuggers seem to target C or
               | Lisp. But lately I don't use either.
               | 
               | I've never gotten time-travel debugging in gdb to work.
               | And it's been out for more than 10 years at this point.
               | The last time was 2 years ago, so I forget the details. I
               | do love tracepoints in gdb.
               | 
               | I've been watching https://rr-project.org for a while.
               | The instructions still say, "build from source".
               | 
               | I looked at LLDB after your previous comment. Got it
               | installed, but I can't actually get it to set a
               | breakpoint on Linux. The docs seem optimized for Mac.
               | 
               | I have no doubt that once you get something set up just
               | right you can do great things with it. But the power to
               | weight ratio seems totally out of whack. Part of the goal
               | of my projects recently has been to show that you can get
               | a bunch of features far more simply if you build on a
               | base of logging. If the programmer is willing to modify
               | the program, a single tool can help debug programs in a
               | wide variety of languages. They just have to follow a
               | common and fairly simple protocol.
               | 
               | The big drawback of my approaches is that they don't
               | scale for long runs of huge codebases. That hasn't been
               | an issue for most programs I ever want to debug. Why
               | should I pay the complexity costs of debugging gcc or
               | Firefox for small programs.
        
               | jcranmer wrote:
               | > I've been watching https://rr-project.org for a while.
               | The instructions still say, "build from source".
               | 
               | rr should be in most distros at this point. (It's been in
               | Debian since at least 2014).
        
               | akkartik wrote:
               | Thanks so much! I just tried it out and finally got to
               | see time travel debugging working with my own eyes[1].
               | I'm glad to have it in my toolbox.
               | 
               | [1] Though I had to use sudo a few times, reboot, see
               | some scary warning that it might not work reliably, read
               | https://github.com/rr-debugger/rr/wiki/Will-rr-work-on-
               | my-sy... and https://github.com/rr-debugger/rr/wiki/Zen.
        
             | aidenn0 wrote:
             | Is your argument "Don't use debuggers because it will
             | prevent you from writing a debugger?"
        
               | akkartik wrote:
               | I'm not arguing "don't use debuggers."
               | 
               | I'm arguing, "don't just use debuggers."
               | 
               | I elaborate more on this argument in the first 2 minutes
               | of this 4-minute video:
               | https://handmade.network/snippet/1561
        
               | aidenn0 wrote:
               | That makes sense. I'm also with you in that programmers
               | forget that their development tools are also programs and
               | they can modify and/or make more of them.
        
             | AndriyKunitsyn wrote:
             | I'm not sure I'm following. A visual debugger ticks a lot
             | of boxes for my needs. Not all - some of them I had to tick
             | myself, but I don't think that trying to reinvent the value
             | that's already there in a debugger, would be particularly
             | productive for me.
             | 
             | As for "forgetting", I don't think I forgot, because, well,
             | I did make a new tool.
             | 
             | The main thing that I wanted to address in the parent
             | comment is that needing tools to debug your code somehow
             | means that the code is a mess - no, sometimes it doesn't.
        
               | akkartik wrote:
               | > The main thing that I wanted to address in the parent
               | comment is that needing tools to debug your code somehow
               | means that the code is a mess - no, sometimes it doesn't.
               | 
               | Ah. I totally agree with that. Or at least, it's the sort
               | of mess I don't know how to avoid yet.
        
       | _gabe_ wrote:
       | > I should mention that the perfect debugger doesn't exist
       | 
       | It may not exist, but this demo has got to be the closest thing
       | to the perfect debugger I've ever seen:
       | https://youtu.be/72y2EC5fkcE
        
         | kaba0 wrote:
         | Wow, that's absolutely insanely cool!
        
       | rr808 wrote:
       | Visual Studio C++ did this in the 90s. I never understood why
       | Unix devs preferred emacs/vim back then.
        
         | pjmlp wrote:
         | I prefered XEmacs, because it was the only thing that could
         | improve my experience versus Borland Turbo Pascal and C++ IDEs,
         | and it was much better than plain Emacs or VI (vim was years
         | away to materialize).
         | 
         | Nowadays only if I am on bare bones installation, I reach out
         | for emacs or vim, on that order.
        
       | matheusmoreira wrote:
       | I found GDB to have a steep learning curve, even stepping through
       | code was hard at first. Now I simply cannot work without it
       | anymore. I have GDB scripts for visualizing all my data
       | structures now. GDB script is a painful little language but it
       | just changed everything for me.
        
       | quelsolaar wrote:
       | To me the debugger is perhaps the most important tool and I much
       | rather have a bad editor with a good debugger than a great editor
       | without. I find that typing code is never the limiting factor in
       | productivity, being able to understand what your code does is,
       | and a good debugger is a game changer. The same goes for
       | languages, there is a lot of discussion about what various
       | languages do, but not what debuggers are available for them.
       | Visual studio has many faults, but IMO it runs circles around the
       | competition when it comes to te debugger.
        
         | whatever1 wrote:
         | Debugging is the one reason I tolerate interpreted languages.
         | 
         | The fact that you can pause execution and start writing
         | commands to execute on the fly in the debug console is VERY
         | POWERFUL.
        
           | lispm wrote:
           | ...and you don't need even 'interpreted languages' for
           | that...
        
           | quelsolaar wrote:
           | As a C developer, I REALY want a interpreted C compiler with
           | a crazy good debugger for this reason. On average I think C
           | has really good debuggers (its a simple language that has
           | been around for a long time) but there is so much more one
           | could do. A few years ago i wrote a prototype called OPA,
           | with some cool potential features.
           | https://www.youtube.com/watch?v=pvkn9Xz-xks
        
             | cxr wrote:
             | * * *
        
       | henrydark wrote:
       | One of my favorite "debuggers" is OllyDbg.
       | 
       | I will never forget the first time I changed a few lines of
       | assembly of a running game (starsiege tribes, around 2005), and
       | then I saw friend-or-foe indicators through walls.
       | 
       | That kind of "hot reloading" was special.
        
       | msoad wrote:
       | Unfortunately debug-ability usually stops when you have a
       | distributed system in which each part of the system is playing a
       | different song. I wish my job was working on a program that I
       | could run on a single machine. Then I'll look into debuggers. For
       | now I'm `print`ing and hopelessly looking into service logs in
       | DataDog :(
        
         | layer8 wrote:
         | It's a good reason to not unnecessarily distribute your system.
        
         | jayd16 wrote:
         | Would certainly be nice to see some of these PAAS offer a
         | distributed debugger that could track RPCs and breakpoint
         | across machines in the cloud. It seems feasible to "step into"
         | a remote method assuming the request IDs were integrated.
        
           | pjmlp wrote:
           | Application Insights on Azure, goes a big deal into that
           | direction.
           | 
           | https://learn.microsoft.com/en-us/azure/azure-
           | monitor/app/ap...
           | 
           | Or Java Flight Recorder,
           | 
           | https://developers.redhat.com/blog/2021/01/25/introduction-t.
           | ..
        
         | pjmlp wrote:
         | Thankfully there are things like flight recorder and
         | application insights that marry distributed computing
         | monitoring with debuggers.
        
       ___________________________________________________________________
       (page generated 2023-03-10 23:01 UTC)