[HN Gopher] rr - record and replay debugger for C/C++
       ___________________________________________________________________
        
       rr - record and replay debugger for C/C++
        
       Author : levzettelin
       Score  : 291 points
       Date   : 2024-07-21 08:54 UTC (14 hours ago)
        
 (HTM) web link (rr-project.org)
 (TXT) w3m dump (rr-project.org)
        
       | rtpg wrote:
       | rr is really cool, but almost every time I have decided to pull
       | it out as one of the "big guns" it turns out that I have a
       | concurrency bug and so rr is unable to reproduce it.
       | 
       | Despite that, it would be very, very, very cool if some languages
       | built rr directly into their tooling. Obviously you can always
       | "just" use rr/gdb, but imagine if rr invocations were as easy to
       | set up and do as pdb is in Python!
        
         | angus-g wrote:
         | I actually had a concurrency bug that I was able to capture
         | with rr: an MPI job where I only ran rr on rank 0 and managed
         | to figure out where a different send/recv ordering was causing
         | issues. In fact, it was also a Python model that ties in with a
         | lot of native code generation, so quite a complex issue.
        
         | IshKebab wrote:
         | Yeah same for me. Actually the time I _really_ wanted it was on
         | Mac and unfortunately it only works on Linux.
         | 
         | There is Undodb which works on Mac and maybe with
         | multithreading (not sure about that), but unfortunately it
         | costs about $50k.
        
         | borodi wrote:
         | Chaos mode is an option when invoking rr that can expose some
         | concurrency issues. Basically it switches which thread is
         | executing a bunch to try and simulate multiple cores executing.
         | It has found some race conditions for me but it's of course
         | limited
        
           | dzaima wrote:
           | Unfortunately that only works for large-scale races, and not,
           | say, one instruction interleaving with another one on another
           | thread without proper synchronization. -fsanitize=thread
           | probably works for that though (and of course you could then
           | combine said sanitizer with rr to some effect probably).
        
             | borodi wrote:
             | I havent tried Tsan with rr but msan and asan work quite
             | well with it (it's quite slow when doing this) but seeing
             | the sanitizer trigger then following back what caused it to
             | trigger is very useful.
        
             | bqmjjx0kac wrote:
             | Yeah, the reason it only works for these coarser race
             | conditions is that RR only has one thread executing at a
             | time. Chaos mode randomizes the durations of time allotted
             | to each thread before it is preempted. This may be out of
             | date. I believe I read it in the Extended Technical Report
             | from 2017: https://arxiv.org/pdf/1705.05937
        
             | roca wrote:
             | One option would be to combine chaos mode with a dynamic
             | race detector to try to focus chaos mode on specific fine-
             | grained races. Someone should try that as a research
             | project. Not really the same thing as rr + TSAN.
             | 
             | There's still the fundamental limitation that rr won't help
             | you with weak memory orderings.
        
         | modeless wrote:
         | Another thing that rr sadly doesn't support is GPUs. I'd love
         | to use it but most of my stuff involves GPUs in some way or
         | another.
        
       | nsjolj wrote:
       | I've been using rr for a while now, and it's a game-changer for
       | C/C++ debugging on Linux. The deterministic replay and reverse
       | execution features are incredibly powerful, especially for
       | tracking down those elusive bugs. It's like having a time machine
       | for your code! Anyone else had experiences with rr making their
       | debugging process smoother?
        
         | tdullien wrote:
         | I've used it for years for vulnerability development (when you
         | need to debug why a particular exploitation run went wrong
         | etc.).
         | 
         | It is indeed marvelous for many situations; except for
         | concurrency bugs ;)
        
           | i_am_a_peasant wrote:
           | Why isn't it good for concurrency bugs? rr doesn't record the
           | entire state up to the race condition?
        
             | adgjlsfhk1 wrote:
             | it serializes your problem because you can't atomically
             | snapshot memory when there are multiple threads writing to
             | it
        
               | pm215 wrote:
               | This is true, but for at least some race conditions the
               | rr "chaos mode" can help in tracking them down. Chaos
               | mode basically randomly adjusts its thread scheduling so
               | that it sometimes doesn't run a thread at all for a few
               | seconds, so you run it a bunch of times until it hits the
               | timing window for the race condition; and then when it
               | does you debug the recording. IME it works at least
               | sometimes but it does require that the thing you're
               | testing doesn't take too long to fall over, since you
               | need to do multiple recordings of it.
               | 
               | https://robert.ocallahan.org/2016/02/introducing-rr-
               | chaos-mo...
        
             | nemetroid wrote:
             | RR supports recording multithreaded programs, but only runs
             | one thread at a time, so it cannot trigger data race bugs.
             | 
             | It can find higher level concurrency bugs though, and has a
             | "chaos mode" that schedules threads in an unfair random
             | manner.
        
           | leni536 wrote:
           | It is sometimes useful with concurreny bugs, especially when
           | used in "chaos" mode. Altough it probably masks some classes
           | of concurrency bugs. I wonder if rr + TSAN works.
        
         | joechrisellis wrote:
         | My previous job was in compiler engineering and I worked on
         | upstream LLVM for a little while. rr was indispensable for me
         | -- not just for debugging, but as a tool for understanding how
         | the compiler works in general. For example: you can set a
         | watchpoint on an object in memory, and reverse-execute to the
         | point where it was instantiated. This allowed you to answer
         | questions such as 'which optimisation pass was responsible for
         | creating this DAG node?' and such.
         | 
         | Really powerful tooling -- I haven't kept up with it for a
         | while, but last I checked it didn't work on aarch64/arm64
         | because of a lack of certain performance counters only
         | available on x86 hardware. The lack of support for other
         | languages (e.g. Go) is also a shame. This has unfortunately
         | rendered it unviable for my day job!
         | 
         | If you're working in a stack where it's supported, it's really
         | quite special.
        
           | dzaima wrote:
           | I believe rr should support other languages exactly as well
           | as gdb does; I've looked at some Rust in it once or twice at
           | least, and IIRC even some Go; though I'm happy enough with
           | doing assembly-level breakpoints/whatnot so I can't comment
           | on more fancy things. Even some OpenJDK JIT! (though there of
           | course source-mapping/stacktraces are non-existent, never
           | mind reading variables, but even then I managed to hack
           | together some basic source-mapping at
           | https://github.com/dzaima/grr#java-jit; though it's rather
           | unusable with rr due to OpenJDK's machine code dumping going
           | directly to stdout, and general slowness).
           | 
           | That said, it would be very neat to have a more complete
           | standard debugging interface across ahead-of-time-compiled
           | code, interpreters, and JITs.
           | 
           | And it does have some aarch64 support by now, though it'll
           | still require a suitable CPU with necessary perf counters.
        
             | adgjlsfhk1 wrote:
             | I do know that RR works for Julia. Not sure why it wouldn't
             | work for Go.
        
               | dzaima wrote:
               | Julia goes through LLVM, which has good DWARF support
               | (and, additionally, until recently, Julia hosted rr's CI,
               | so Julia directly cares about rr). Whereas AFAIU Go does
               | its codegen fully from scratch, without much regard for
               | gdb/DWARF quality.
        
           | Thaxll wrote:
           | It says on https://rr-project.org/ that it supports Go
           | programm, what's the status?
        
           | roca wrote:
           | rr supports Aarch64 pretty well now.
        
         | Agingcoder wrote:
         | You might be interested in the rather spectacular upgrade to
         | it, called pernosco :
         | 
         | https://pernos.co/
         | 
         | It's based on rr ( same author ) but vastly better
        
           | mgaunard wrote:
           | There are other commercial reverse debuggers, like Undo.
        
             | db48x wrote:
             | Pernosco is more than merely a reverse debugger.
        
         | leni536 wrote:
         | rr is awesome. My typical debug workflow:                 * rr
         | record       * rr replay until crash or exception (set up
         | catchpoint for that)       * set up hardware watchpoint for
         | borked data that caused the issue       * work backwards in the
         | data flow through watchpoints and reverse-continue
         | 
         | I would say this fundamentally changed how I approach
         | debugging.
        
       | suby wrote:
       | Perhaps worth mentioning is that someone attempted to port this
       | to Rust and got about 60,000 lines of code into it before
       | archiving the project. I feel like comparing these two efforts
       | would be an interesting case study on the impacts / benefits /
       | limitations or difficulties, etc involved in rewriting from C++
       | to Rust.
       | 
       | https://github.com/sidkshatriya/rd/
        
         | dwattttt wrote:
         | It would be a good but difficult analysis; at a quick check, rr
         | 1.0 took 3 years and signficant contributions from around 3 or
         | 4 people (I saw at least 5 people contributing), and the rr we
         | have today is 10 years further work on that.
        
         | tarruda wrote:
         | I don't understand the "rewrite X/Y/Z in Rust" trend that has
         | been going for a few years.
         | 
         | I'm not familiar with Rust, but I'm almost sure it has a good C
         | interoperability. If a certain piece of software is working
         | well, what is the benefit of rewriting it in Rust?
        
           | coldtea wrote:
           | > _If a certain piece of software is working well, what is
           | the benefit of rewriting it in Rust?_
           | 
           | Presumablly the idea here is to support Rust replay
           | debugging, not just rewrite a C/C++ targetting replay
           | debugger in Rust
        
             | icholy wrote:
             | It works with pretty much any compiled language.
        
               | coldtea wrote:
               | Does it understand the semantics, primitives, and
               | structures of any compiled language?
               | 
               | Or is it more of a lowest common denominator experience,
               | where e.g. all of them are constrained to common
               | semantics of C/C++?
        
               | icholy wrote:
               | That's up to debuggers to implement. lldb, gdb, and delve
               | all support rr traces.
        
             | asveikau wrote:
             | I'm not so sure of this. Most code in a debugger doesn't
             | have much to do with the source language. Line mapping and
             | figuring out values for variables happens via symbol
             | information, eg. DWARF, and compilers for multiple
             | languages can produce that in the same format.
        
               | coldtea wrote:
               | Does that neutral way of debugging work as well as having
               | first-class explicit support for the language, or it's
               | more of a lowest common denominator (kind of like what
               | e.g. language support over LSP can offer and how
               | conveniently, compared to what a native Lisp or Smalltalk
               | environment's language support can offer)?
        
               | asveikau wrote:
               | One debugger feature I thought of while writing the
               | comment was the ?? command in windbg. That takes an
               | expression and evaluates it. Gdb also does this which
               | I've used with the "print" command to print a C
               | expression, including pointer casts and such. That would
               | obviously require language support.
               | 
               | But then again you don't need to code everything in the
               | same language either. You could write a rust parser in
               | another language. Or a modular interface to dispatch
               | knowledge of a programming language (does Microsoft's
               | "language server" concept work this way?)
        
               | Veserv wrote:
               | The latter.
               | 
               | The neutral way of debugging is really debugging the raw
               | machine code of a process. This requires OS integration
               | for your low level manipulation primitives. To add
               | language support, you then need to figure out how to
               | define the semantics of your manipulations in terms of
               | the low-level primitives.
               | 
               | If you have a rich runtime, you can add language-level
               | debugging facilities that can operate at a higher level.
               | However, this requires you to implement portions of a
               | debugger in your runtime. Now you have to maintain a
               | language, runtime, and debugger. It also means that if
               | new debugger techniques are invented, such as time travel
               | debugging, you do not get them for free since you
               | embedded a debugger of your own design. So, like many
               | similar things, it is a trade-off of specialization
               | versus maintenance. The perennial question of use a
               | library, or do it yourself.
        
           | _dain_ wrote:
           | _> If a certain piece of software is working well, what is
           | the benefit of rewriting it in Rust?_
           | 
           | The "working" C program has a high risk of undiscovered bugs
           | relating to concurrency and memory safety. Rust lets you rule
           | out a large swathe of them by construction. Rust's type
           | system is also far more expressive, which in many cases
           | enables cleaner domain modelling.
        
             | jvanderbot wrote:
             | You're correct that this is the stated justification most
             | of the time.
             | 
             | Should be nuanced though, because the working C program has
             | a risk, but the the risk is a function of the size of the
             | codebase, its age, and the number of audits it has
             | undergone.
             | 
             | It is definitely easier to write bugs in C due to the
             | additional freedom you have, but it is not necessarily a
             | "high" risk for mature C libraries.
             | 
             | It is definitely not as advisable to just replace all C
             | with Rust, but it is advisable to prefer memory safety in
             | new projects.
        
           | johnwatson11218 wrote:
           | Recently the federal government issued a security advisory
           | encouraging all new development to be done in Rust. I'm not
           | sure the extent of which agencies this was meant to cover but
           | it struck me as very unrealistic having just done years of
           | java development for a state agency.
           | https://www.nextgov.com/cybersecurity/2024/02/white-house-
           | ur...
        
             | detaro wrote:
             | It doesn't say that, no. Even in the title it says "memory-
             | safe languages", which of course includes Java.
             | 
             | Rust is "only" mentioned in the context of a C / C++
             | replacement, which tends to be a different area.
        
               | johnwatson11218 wrote:
               | Yes that is correct, I had read a different article that
               | had a more rust first focus. I just tried to google for a
               | reference to the news release I was referencing, but yea
               | it doesn't really support my comment :(
        
             | 77pt77 wrote:
             | > the federal government issued a security advisory
             | 
             | Well, I guess that's it...
             | 
             | Maybe rust will take the same path as Ada.
        
           | rtpg wrote:
           | There's a bit of a higher abstraction ceiling in Rust so in
           | theory if you are successful rewriting a thing in Rust then
           | you now have a codebase that's easier to change confidently.
           | 
           | This sort of property is nice to have in huge codebases where
           | you really start losing confidence in shipping changes that
           | don't subtly break things. But of course a huge codebase is
           | hard to rewrite in general...
        
             | taneq wrote:
             | Sounds like a perfect situation for a strangler pattern?
             | Wrap or transpile the original code into a language with
             | stronger refactoring support and the rest should become
             | incrementally easier.
        
               | kvdveer wrote:
               | The advantages of rust only come when you actually use
               | the rust-provides abstraction, especially those around
               | allocation and concurrency. Even if transpiling is
               | possible, the code would still not be structured I the
               | rust way, and you wouldn't have any of the benefits. Same
               | goes for wrapping.
        
             | nsajko wrote:
             | > There's a bit of a higher abstraction ceiling in Rust
             | 
             | Compared to C, yes, but not compared to C++.
        
               | tialaramex wrote:
               | This isn't really true. Rust has a much better type
               | system. When writing generic code the impact is enormous.
               | 
               | C++ doesn't have a real Empty Type, and it thinks Units
               | have non-zero size. In practical terms this makes it
               | incredibly wasteful and in terms of a clear abstraction
               | it encourages you to come up with a hack that's unclear
               | but efficient.
        
               | saagarjha wrote:
               | ...or you can just waste a few bytes? It's not a big
               | deal.
        
               | polio wrote:
               | You can, but that makes the type system worse. Also
               | depending on how these few bytes are used, they can add
               | up and drag down performance.
        
               | saagarjha wrote:
               | No, my point is that it doesn't. If your zero-sized types
               | are big your type system is not any worse: it's just less
               | efficient.
        
               | tialaramex wrote:
               | If you're focused on just the theoretical correctness of
               | the type system, go back to my first critique: C++ does
               | not have Empty Types. So immediately a whole class of
               | problems that are just a type system question in Rust are
               | imponderable, you can't even say what you meant in C++
        
               | samatman wrote:
               | Copying a bunch of stuff because the borrow checker won't
               | let you share it can drag down performance as well. Yes,
               | I do understand why one might conclude that tradeoff is
               | worth it. But it is a tradeoff.
        
               | tialaramex wrote:
               | The original topic was the abstraction ceiling. There are
               | a bunch of abstractions which C++ just can't express.
        
               | estebank wrote:
               | Funnily enough, because the borrow checker is so strict I
               | feel more confident writing complex borrowing logic that
               | I wouldn't dare attempting in C or C++ because even if I
               | were to get everything right (a big if), there's no
               | assurance that a later refactor wouldn't subtilty break
               | the code. The borrow checker sometimes makes you copy
               | data that you thought you didn't, but more often than not
               | it is enforcing an actual edge case that would have been
               | a bug, had the borrow checker not be present. If the copy
               | is indeed so critical, you can also ease your pain with
               | runtime checks instead using Rc/Arc, but that's another
               | discussion.
        
               | a_t48 wrote:
               | Do you have a link to an example where this matters?
        
               | celrod wrote:
               | C++20 added `[[no_unique_address]]`, which lets a
               | `std::is_empty` field alias another field, so long as
               | there is only 1 field of that `is_empty` type.
               | https://godbolt.org/z/soczz4c76 That is, example 0 shows
               | 8 bytes, for an `int` plus an empty field. Example 1
               | shows two empty fields with the `int`, but only 4 bytes
               | thanks to `[[no_unique_address]]`. Example 2
               | unfortunately is back up to 8 bytes because we have two
               | empty fields of the same type...
               | 
               | `[[no_unique_address]]` is far from perfect, and
               | inherited the same limitations that inheriting from an
               | empty base class had (which was the trick you had to use
               | prior to C++20). The "no more than 1 of the same type"
               | limitation actually forced me to keep using CRTP instead
               | of making use of "deducing this" after adopting c++23: a
               | `static_assert` on object size failed, because an object
               | grew larger once an inherited instance, plus an instance
               | inherited by a field, no longer had different template
               | types.
               | 
               | So, I agree that it is annoying and seems totally
               | unnecessary, and has wasted my time; a heavy cost for a
               | "feature" (empty objects having addresses) I have never
               | wanted. But, I still make a lot of use of empty objects
               | in C++ without increasing the size of any of my non-empty
               | objects.
               | 
               | C++20 concepts are nice for writing generic code, but
               | (from what I have seen, not experienced) Rust traits look
               | nice, too.
        
               | tialaramex wrote:
               | It's probably mean for me to say "empty type" to C++
               | people because of course just as std::move doesn't move
               | likewise std::is_empty doesn't detect empty types. It
               | can't because C++ doesn't have any.
               | 
               | You may need to sit down. An empty type has no values.
               | Not one value, like the unit type which C++ makes a poor
               | job of as you explain, but no values. None at all.
               | 
               | Because it has no values we will never be called upon to
               | store one, we can't call functions which take one as a
               | parameter, operations whose result is an empty type must
               | diverge (ie control flow escapes, we never get to use the
               | value because there isn't one). Code paths which are
               | predicated on the value of an empty type are dead and can
               | be pruned. And so on.
               | 
               | Rust uses this all over the place. C++ can't express it.
        
               | plasticeagle wrote:
               | Help me out here.
               | 
               | What is this empty type for? Could you provide an old man
               | with a nice concrete example of this in action? I've used
               | empty types in C++ to mark the end of recursive templates
               | - which I used implement typelists before variadic
               | templates were available.
               | 
               | But then you mention being unable to call functions which
               | take an empty type as a parameter. At which point I cease
               | to understand the purpose.
        
           | oguz-ismail wrote:
           | > If a certain piece of software is working well, what is the
           | benefit of rewriting it in Rust?
           | 
           | One benefit is it's easier to hide malicious code thanks to
           | Rust's complicated syntax.
        
             | roca wrote:
             | It's far easier to hide malicious code in C or C++: just
             | write some subtle undefined behavior that you can write an
             | exploit against. Developers do that all the time even when
             | they're not trying to be malicious. In Rust you'd have to
             | wrap it in "unsafe" which draws attention.
        
         | khuey wrote:
         | From the perspective of an rr maintainer, Sid's work was good
         | and we were supportive of it. The main issues with migrating to
         | that as the "blessed" version are that 1) rr has accumulated a
         | decade of very hairy fixes for crazy kernel/process behavior
         | that we feared could be lost during a port and 2) there's a
         | closed source project (remix[0]) built on top of rr that would
         | have needed to be ported too.
         | 
         | [0] https://robert.ocallahan.org/2020/12/rr-remix-efficient-
         | repl...
        
       | jfk13 wrote:
       | A couple of previous discussions:
       | 
       | https://news.ycombinator.com/item?id=31617600 (June 2022)
       | 
       | https://news.ycombinator.com/item?id=18388879 (Nov 2018)
        
       | laserbeam wrote:
       | Is it truly only for C/C++?
       | 
       | My limited understanding says a debugger needs: a list of symbols
       | (.pdb files on windows, can't remember what they are on linux),
       | understanding of syscalls and a few other similar things. I
       | thought they don't care too much what generated the binaries they
       | are debugging (obviously as long as it's native code).
       | 
       | Doesn't rr work with other languages like rust, zig, odin, nim,
       | and similar ones? Obviously, I wouldn't expect it to work for
       | python, js, c# and other languages with managed memory.
        
         | fragmede wrote:
         | for js, there's http://replay.io
        
         | leni536 wrote:
         | Interestingly gdb (and in turn rr) has some limited support for
         | debugging python. At least you can get a python backtrace, but
         | I didn't have success in setting pyhton breakpoints.
        
           | laserbeam wrote:
           | Yeah, 'cause you're technically debugging the python
           | interpreter. I've had some success with tracing tools
           | designed for C/C++ for a python project. Was not easy to set
           | up and will obviously include frames from the interpreter.
           | 
           | Tho, it feels wrong to expect a tool designed for native
           | binaries to work well with python in this context. And that's
           | ok. It feels lucky when it works as much as it does.
        
         | dzaima wrote:
         | rr uses gdb as the actual debugger part, so anything that works
         | in gdb will work in rr. (you won't get rr running on windows
         | though, as it is very much linux-specific, having to wrap all
         | of its syscalls. The linux symbol info thing is DWARF)
        
           | laserbeam wrote:
           | Ok yeah, of course. I'd even argue that cross platform
           | debuggers are not a thing to be desired. Too much low level
           | integration with the operating system is needed when
           | implementing one.
        
             | db48x wrote:
             | I disagree. You shouldn't have to learn two debuggers just
             | because you occasionally have to use a different OS. GDB
             | has the right architecture here; the actual debugging
             | operations are implemented by a gdbserver, and gdb is only
             | the user interface that lets the user tell the server what
             | to do. When you're debugging on a different platform you
             | use use a different gdbserver and keep using the same user
             | interface that you are familiar with.
             | 
             | When you replay a recording, rr first starts its custom
             | gdbserver (which reads from the recording instead of from a
             | live process) then starts a gdb process that connects to
             | it.
        
             | Xeamek wrote:
             | Why? From the user's pov, what 'low level integration with
             | an OS' is there that couldn't/shouldn't be abstracted into
             | 'generic debugging functionalities'?
        
             | ognarb wrote:
             | I did use gdb recently on Windows, but this was for a cross
             | compiled program using mingw. Not sure it works for
             | programs made with MSVC.
        
         | db48x wrote:
         | Yes, it can generally debug any language that compiles to a
         | binary with proper debug information.
        
         | vaughnegut wrote:
         | It also works with Go, I think support for it is built into
         | Goland too
        
           | icholy wrote:
           | Goland uses delve which has rr support
        
         | vinnyvichy wrote:
         | I've gotten rr to work with very specific builds of rpython
         | before but you might be surprised at the ongoing interest:
         | 
         | https://github.com/python/devguide/issues/1283
         | 
         | https://morepypy.blogspot.com/2016/07/reverse-debugging-for-...
         | 
         | https://github.com/mesalock-linux/mesapy/blob/mesapy2.7/READ...
        
         | vchuravy wrote:
         | We use RR a lot with Julia. It only gives you a GDB view of the
         | system, but it can work with any interpreted or compiled
         | language.
         | 
         | Things that don't work are drivers that update mapped addresses
         | directly. An example of this is CUDA in order to replay one
         | would need to model the driver interactions (and that's even
         | before you get to UVM)
         | 
         | Another great thing is that RR records the process tree and so
         | you can easily look at different processes spawned by your
         | executable.
        
         | n8henrie wrote:
         | People use it for rust:
         | https://bitshifter.github.io/rr+rust/index.html#1
        
         | AndyKelley wrote:
         | I use it with Zig. It's pretty handy in conjunction with Zig's
         | allocator because it writes 0xaa bytes upon free and doesn't
         | reuse addresses, so it very likely causes a crash, then you can
         | put a watchpoint on the memory and rewind to the point where it
         | got freed.
        
           | sitkack wrote:
           | That sounds really neat, is there more information on this?
           | 
           | *Edit
           | 
           | Found this, https://zig.news/david_vanderson/using-rr-to-
           | quickly-debug-m...
        
       | whatsakandr wrote:
       | I've used rr very sucessfully for reverse engineering a large
       | code base using a break on variable change combined with reverse-
       | continue. Took the time to extract critical logic way down.
        
         | rramadass wrote:
         | That sounds very interesting; Do you have a write-up on this
         | that you are willing to share?
        
           | dima55 wrote:
           | This is the usual killer feature of something like rr. You
           | debug, look at some variable: `p whatever`. You see that its
           | value is wrong. You want to know where this wrong value came
           | from, so you `watch -l whatever` and `rc`. Bam!
        
             | icholy wrote:
             | There are some bugs I would never have figured out without
             | this technique. It feels like cheating.
        
               | db48x wrote:
               | It's not cheating, it's technique!
        
               | dima55 wrote:
               | Totally. rr is nothing short of a revolution in debuggin.
        
       | whatsakandr wrote:
       | Has anyone gotten rr to work with opengl or vulkan? It seems to
       | always crash for me after making an opengl call.
        
         | arbruijn wrote:
         | VirGL might help, it redirects OpenGL calls over a socket.
         | Start virgl_test_server and run your app with extra environment
         | vars __GLX_VENDOR_LIBRARY_NAME=mesa LIBGL_ALWAYS_SOFTWARE=1
         | GALLIUM_DRIVER=virpipe
        
       | boguscoder wrote:
       | Curious if anyone tried rr for/on Android? It seems possible to
       | crosscompile it and it could be a good tool for native side
       | debugging
        
       | forrestthewoods wrote:
       | On Windows you can use WinDbg for the same thing. It has better
       | support for debugging multi-threaded issues.
       | 
       | https://www.forrestthewoods.com/blog/windbg-time-travelling-...
        
         | Veserv wrote:
         | WinDbg uses a instruction-level emulation time travel
         | implementation, so incurs the 10-20x slowdown associated with
         | that technique. rr uses a replay-record time travel debugging
         | implementation, which can incur far less overhead when done
         | correctly. Last I saw, rr has overhead in the 2x slowdown range
         | and, if I remember correctly, I have seen a different record-
         | replay time travel debugger in the 10% range.
         | 
         | 10% is 100x cheaper than WinDbg and cheap enough to leave on
         | all the time in production. That is a game-changer.
        
           | forrestthewoods wrote:
           | > 10% is 100x cheaper than WinDbg
           | 
           | If you're gonna throw around numbers like this you need to
           | cite an actual tool not "if I remember correctly there exists
           | a unicorn".
        
           | dzaima wrote:
           | More precisely, rr records just system calls & interrupts, so
           | its overhead is largely proportional to how syscall-heavy and
           | multithreaded your code is. If it's single-threaded and
           | mostly just pure computation, you can easily see ~0%
           | slowdown, with replay at the exact same speed. (a quick
           | simple test shows that 'mmap'+'munmap' of a file is 0.5ms, vs
           | 0.006ms outside rr; but many syscalls are buffered in rr's
           | userspace and thus way faster (e.g. a 'stat' on the same file
           | path is only like 300ns slower than native))
        
       | modeless wrote:
       | See also https://pernos.co/ which is based on rr but adds a
       | queryable database of the whole program execution, which allows
       | you to do things like this:
       | 
       | > [...] just click on the incorrect value. With full program
       | history Pernosco can immediately explain where this value came
       | from. The value is tracked backwards through events such as
       | memcpys or moves in and out of registers until we reach a point
       | where the value "originated" from, and each step in this process
       | is displayed in the "Dataflow" panel that opens automatically.
       | There is no need to read or understand the code and think about
       | what might have happened, we can simply ask the debugger what did
       | happen.
       | 
       | https://pernos.co/examples/race-condition
        
       | leoc wrote:
       | Are rr's problems with Ryzen CPUs now firmly in the past or not?
        
         | padenot wrote:
         | Yes, I use rr all day every day (to record Firefox executions)
         | on a rather recent Threaripper Pro 7950, and also with
         | Pernosco. The rr wiki on GitHub explains how to make it work.
         | Once the small workaround is in place it works very reliably.
        
       | dmitrygr wrote:
       | Long ago, VMWare workstation supported doing this, but not just
       | for userspace programs but also for kernels and even drivers, in
       | a VM. The feature shipped and existed for a few versions before
       | it was killed by internal politics.
        
         | Arech wrote:
         | And, I guess, before that, there was AMD SymNow, which was
         | plug-in extendable and gave a plugin full control over CPU
         | being emulated. I wonder if something like that is available
         | somewhere?
        
       | ho_schi wrote:
       | GDBs built-in reverse debugging:
       | https://www.sourceware.org/gdb/wiki/ProcessRecord/Tutorial
       | 
       | I assume _rr_ provides more features and flexibility. Anyway I
       | want to mention that GDB itself can already reverse debug for
       | some time now.
        
         | dima55 wrote:
         | If you want to mention this, then you very clearly haven't
         | actually tried it. The implementation in GDB is more convenient
         | than rr (you can start/stop recording at will), but it is also
         | orders of magnitude less efficient. It's only usable for very
         | small code snippets. Otherwise it takes effectively forever
         | and/or runs out of resources.
        
         | 29athrowaway wrote:
         | rr predates the one in gdb if I am not mistaken
        
           | roca wrote:
           | Actually the gdb implementation predates rr, but (as an rr
           | maintainer) I have to say that it is vastly inferior to rr.
           | It's about 1000x slower than rr, and can't record across
           | system calls or multiple threads or processes. It's so
           | limited it's really a different feature.
        
           | Veserv wrote:
           | rr was introduced in 2014 [1].
           | 
           | gdb reverse debugging was introduced in 2009 [2].
           | 
           | You can see a fairly comprehensive history of time travel
           | debugging here [3].
           | 
           | Not to say the built-in gdb reverse debugging was any good.
           | It had (has?) like 1,000,000% overhead which is basically
           | unusable. At least some implementations in the history that
           | were introduced earlier only had ~1,000% overhead or less in
           | general. Yes, a literal 1,000x overhead difference.
           | 
           | [1] https://robert.ocallahan.org/2014/03/introducing-rr.html
           | 
           | [2] https://www.sourceware.org/gdb/wiki/ReverseDebug
           | 
           | [3] https://jakob.engbloms.se/archives/1564
        
       | teaearlgraycold wrote:
       | I used this to help make my toy JIT compiler:
       | https://github.com/danthedaniel/BF-JIT
       | 
       | Super useful, especially considering I know barely anything about
       | x86-64.
        
       ___________________________________________________________________
       (page generated 2024-07-21 23:00 UTC)