[HN Gopher] Debug Programs by Modifying Them
       ___________________________________________________________________
        
       Debug Programs by Modifying Them
        
       Author : akkartik
       Score  : 27 points
       Date   : 2021-04-27 16:48 UTC (6 hours ago)
        
 (HTM) web link (merveilles.town)
 (TXT) w3m dump (merveilles.town)
        
       | fruzz wrote:
       | Honestly, I don't get this debate.
       | 
       | If you were to take the pain points of debug by print, and
       | address them, you'd come up with a debugger. Being able to put
       | those "print" statements anywhere without recompiling, being able
       | to go step-by-step in the code, etc. And sure, that's not always
       | available.
       | 
       | Both end up having impacts on the thing they observe. A print()
       | call can cause microscopic delays that are enough to cause a race
       | condition not to trigger, as can waiting on a breakpoint, etc. If
       | you compile with debug symbols, that can cause some bugs not to
       | surface, as they are only visible when you compile with -O3 or
       | whatnot (eg. a bug involving reading unallocated memory).
       | 
       | Both are just tools. Sometimes one is better in a given
       | situation. Sometimes the other is. I just don't get how this is
       | like an emacs vs vim thing.
        
         | akkartik wrote:
         | > Both end up having impacts on the thing they observe. A
         | print() call can cause microscopic delays that are enough to
         | cause a race condition not to trigger, as can waiting on a
         | breakpoint, etc.
         | 
         | I seldom encounter a program that is sensitive to a print() but
         | robust to a debugger. Prints perturb programs _less_ than
         | debuggers do.
         | 
         | > If you were to take the pain points of debug by print, and
         | address them, you'd come up with a debugger.
         | 
         | Quoting myself from OP: The key skill it teaches is reasoning
         | about a program by modifying it.. When might better debugging
         | tools be counter-productive? If they cause you to forget that
         | there is a debug cycle, and to stay too long observing when a
         | new modification would get you to the answer faster.. Tools
         | will always will have limits.. Prints prepare one for times
         | when you have to leave your tools behind and enter the
         | wilderness.
        
         | sgtnoodle wrote:
         | For embedded work, I've found great value in being able to
         | observe and manipulate running systems without interrupting
         | code execution. A non-blocking console buffer is very handy,
         | but even more so is instrumentation for periodically sampling
         | arbitrary variables cooperatively / in synch with program
         | execution.
         | 
         | When you're making a 50KW motor spin under PID control, a
         | debugger breakpoint can cause a lot of real damage.
        
         | wtetzner wrote:
         | Stepping through can be a pain sometimes, because to find
         | certain bugs it's easier to look through a log throughout a
         | longer run of the program, and see how the state changes.
         | 
         | Of course, a debugger that could generate a log like that
         | instead of pausing execution would give you that benefit too.
        
       | amatecha wrote:
       | Oh hey man, surprised to see a Merveilles link on HN haha :)
       | 
       | Debug by print/console.log is fine -- sometimes it's the
       | fastest/easiest way to quickly troubleshoot a fairly complex
       | flow, etc.
       | 
       | I've worked on mobile web apps where we had no debugging tools
       | (back in the early days) perhaps due to lack of full access to
       | the hardware, etc. So we would do things like set a background
       | color based on a certain value, add borders, change font color,
       | stuff like that. Or just draw debug console text on top of
       | everything.
       | 
       | Sometimes the actual debugger for the provided software/platform
       | is of low quality too.
       | 
       | Conversely, being able to set conditional breakpoints is pretty
       | powerful and I miss doing so when it's not possible!
        
       | marcodiego wrote:
       | In my teens, after long programming session, my brain simply
       | refused to understand what I was looking at. These occasions
       | often led me to try to fix bugs by randomly changing the code.
       | Some bugs were mere off by one errors, then I changed for loops
       | to stop earlier, initialized pointers to NULL and early bail out
       | of a function if a parameter was NULL, increased size of
       | arrays...
       | 
       | It worked more often than I'd like to admit. Reading the code
       | later was utter incomprehensible and left a bitter taste.
       | Eventually I stopped using such technique. It was better to leave
       | to code crashing than having it working without knowing why.
       | 
       | To this day I still see inexperienced C programmers making the
       | same mistake. Such fixes are the kind of mess that will make
       | something work on a computer just to crash in another OS or
       | architecture.
       | 
       | The funniest example of such a specimen I saw was a very
       | interesting graph problem solved in C. It worked beautifully. The
       | algorithm was elegant and low complexity but it was not general
       | enough and some cases had to be specially handled. New special
       | cases were discovered and the team had to resort to a few tests
       | that started breaking when other cases were fixed. Bug appeared
       | and were fixed randomly. A last case was fixed and then all the
       | other previous important cases crashed.
       | 
       | What to do? Valgrind to the rescue!
       | 
       | After running the code on Valgrind, success! Not that it helped
       | fix the bugs, there were lots if complaints, but it didn't crash.
       | So, instead of fixing the bugs, the team decided that it was to
       | be run only under Valgrind and everybody was happy.
        
       | RandallBrown wrote:
       | Sometimes you can't even use print debugging.
       | 
       | I've debugged by modifying colors a few times.
        
         | cylon13 wrote:
         | With shaders this is often the best bet. You end up encoding
         | some value you want to measure as a color, and you can observe
         | how it varies over what you're drawing. There are some pretty
         | sophisticated step debugging tools these days, but just like
         | with print debugging, debugging by color always works on all
         | platforms and has no special setup.
        
         | jes wrote:
         | You had colors? Back in the day, all I had was an unused GPIO
         | pin and an oscope. One pulse, assert code 1, two pulses, assert
         | code 2, etc.
         | 
         | (I'm kind of riffing on an old Dilbert cartoon, but debugging
         | with a scope was common in my life in embedded systems.)
        
         | DougMerritt wrote:
         | Multiple times I've had bugs that disappeared when print
         | debugging was added. To say that that was frustrating is an
         | understatement.
         | 
         | (But where there's a will, there's a way. It just takes longer,
         | sigh.)
        
           | 8note wrote:
           | Sounds like you solved the problem though. Leave the printing
           | in there:P
        
           | slaymaker1907 wrote:
           | One trick to minimize debugging footprint is to store logs in
           | a ring buffer/array list and defer printing. It's not
           | perfect, but if you use a ring buffer, you can avoid making
           | any syscalls until you are truly ready to print. Thread
           | synchronization can obviously still be a problem, but that
           | can be mitigated by using thread local buffers or by using a
           | lockless ring buffer.
           | 
           | Ring buffers are also nice because you can also use them for
           | cases where you don't normally want some information printed
           | because it would introduce too much overhead, but you want
           | the option to print it out in case of crashes/errors. On an
           | engineering side, you just need to be careful not to put too
           | much through this system so the buffer stays useful and also
           | avoid putting something only in the ring buffer system that
           | should really just be logged normally.
        
           | layer8 wrote:
           | Those are known as Heisenbugs.
           | 
           | https://en.wikipedia.org/wiki/Heisenbug
        
           | loopz wrote:
           | In multi-threaded code or using interrupts (ie. clock or
           | I/O), it's pretty obvious what causes this. Fix can be hard
           | though, but absolutely required for robust execution.
           | 
           | In single-thread code, I've often noticed my assumptions
           | about the code paths were wrong. So print debugging actually
           | improved my understanding of what the logic really is doing.
           | Then, either improving the process, or my assumptions/memory
           | of it. Many times debugging like that removes subtle bugs and
           | mistakes that otherwise would've remained invisible and
           | dormant.
           | 
           | Usually when something is off about the execution, however
           | long it takes, it's usually my own assumptions that's wrong.
           | The program logic just does what it's told, and is usually
           | void of off by one errors and the like. If there's subtle
           | bugs, it's often shallow object clones or mistakes handling
           | complex datastructures that seemingly works.
           | 
           | Debugging is just a tool though, use whatever suits you best.
           | I do like the idea of improved tools like RR. However, you
           | often delve into languages that don't have them, so print
           | debugging remains a useful hammer lying around.
        
             | jes wrote:
             | Fully agree with this. More often than not bug is in the
             | computer between my ears rather than the computer on my
             | bench.
        
       | stcredzero wrote:
       | Here's the thing: Use the best tool for the job. Work smarter,
       | not harder. I'm debugging something right now. I can use the
       | rapid compile time of golang to do things like: change the binary
       | format, so that I can trace what part of the buffer was written
       | by which function invocation. I can also write _very_
       | sophisticated conditional debugging that would be iffy on most
       | debuggers, where the machine sifts through the possibilities for
       | me. That  "one weird trick" has earned me a reputation as a magic
       | debugger multiple times.
       | 
       | Sometimes, it's best to use a printf and spot the thing in the
       | console, or to see what's going on with the data to gain a better
       | understanding.
       | 
       | Sometimes it's best to debug something to rapidly see a complex
       | interaction firsthand.
       | 
       | It's always good to be smart about using multiple tools
       | creatively.
        
       ___________________________________________________________________
       (page generated 2021-04-27 23:02 UTC)