CPU-View lazarus / 12-Debugger ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2024-10-19 14:00 I do reverse engineering, and I miss the capabilities of the regular FpDebug debugger. Today I want to show my work that I've been doing for almost a year to make the debugger better. It's not finished yet, it needs a lot of tweaks, but I need someone to look at it. It's a full GUI debugger with a lot of supported features. I've implemented in it what I was missing. I think it makes sense to add many of my solutions to the main debugger. CPU-View is under development and I make changes to it every day, I miss the opinion of specialists who are immersed in this task (it's not application level - it should really be system engineers). There are not enough IOTA interfaces, all debugging is laid directly on specific windows. If they let me - I could offer my implementation. https://github.com/AlexanderBagel/CPUView ---------------------------------------------------------------------- Martin_fr · 2024-10-19 15:46 Looks great. I'll have to finish a few things right now, but I will have a look asap. I would be happy to help integrate most of this with the IDE.  Though probably keeping the asm,reg,dump windows as separate (but anyone with anchordock can combine them). One thing that may be a bit more complicated (unless you already have something / I haven't looked) is the register view. The Problem is that there are different CPUs supported. (We have the remote arm, and Mac M1/2/3, and with gdb even more). So there will be a need for a generic and for cpu specific reg-windows (or for backends to provide display info). Also I haven't checked on what you done on the interface between frontend and backend. I will comment on that as soon as I had a chance to look through it. In any case thanks for all the work, and sharing it here. --- Btw, we now have 2 mem-dump viewers, since I added one too. As for the interfaces => there are definite needs to create some. Except for the watches the entire front/backend api is legacy (from when there was only one debugger, no front, no back-end). But as I said details on that to follow. ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2024-10-19 16:15 Yes, MemViewer - it has helped me more than once, thank you very much am for putting it into debugging. Well, as for registers - I've done a lot of work to isolate them into an abstraction and separate them into a separate data model. Therefore, registers in the debugger can be represented as they are implemented by the current processor. It doesn't matter Intel/Arm/Z80 or otherwise. ---------------------------------------------------------------------- Fibonacci · 2024-10-20 08:18 During my short playtime with it, it crashed twice and froze once ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2024-10-21 03:36 Quote from: Fibonacci on October 20, 2024, 10:18:00 amDuring my short playtime with it, it crashed twice and froze once Got it. I'll check it, apparently I forgot to check synchronization somewhere, which can cause hangs. And something with the dictionaries. ---------------------------------------------------------------------- 440bx · 2024-10-21 04:58 @Alexander, Your work looks very interesting.  I wanted to try your CPU viewer and I proceeded to install FMHexViewer, which I believe is needed by the CPU viewer. Compiling FMHexView produced a compiler error.  Attached is a screenshot of the error messages. I am using Lazarus v3.99 with FPC v3.2.2 (see attachment for details.) Any guidance you can provide to eliminate the errors will be greatly appreciated. Thank you. ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2024-10-21 05:14 Quote from: 440bx on October 21, 2024, 06:58:35 amI am using Lazarus v3.99 with FPC v3.2.2 (see attachment for details.) I forgot to write the requirements - this is all designed for the current trunk (both Lazarus and FPC) ---------------------------------------------------------------------- 440bx · 2024-10-21 05:41 Thank you. That explains why I'm getting an error.  I'm not using the required version of FPC. The following question is mostly driven by curiosity.  Couldn't FMHexView be coded using the features in v3.2.2 ?   The reason I ask is because it may be years before the features in the development version appear in a formal release.  In addition to that, I believe that formal releases are more thoroughly tested than the development version.  This usually means there is less of a chance of a bug being in the compiler than in the application/package, which is nice when using/testing beta software. ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2024-10-21 05:46 Quote from: 440bx on October 21, 2024, 07:41:44 amThe following question is mostly driven by curiosity.  Couldn't FMHexView be coded using the features in v3.2.2 ?   The reason I ask is because it may be years before the features in the development version appear in a formal release.  I could redesign it, but then the code would be less efficient. That's why I've been actively proposing fixes to both FPC (both RTL and the compiler itself) and LCL for the last year, so that the common code base works the same on both Delphi and Lazarus. ---------------------------------------------------------------------- 440bx · 2024-10-21 06:12 The reasons you mention make sense. Thank you for the update. ---------------------------------------------------------------------- Martin_fr · 2024-10-21 12:21 I just updated my FPC 3.3.1 to the latest b693af69a6 But I get Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i FWHexView.Common.pas(630,29) Error: Identifier not found "TExtended80Rec" I can find a declaration in the source, but it is ifdefed Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i {$if defined(SUPPORT_EXTENDED) or defined(FPC_SOFT_FPUX80)}and apparently not build in. That said, give a variable Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i   Buff: TExtended80Support; Why fill it with Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i       FillChar(Buff, SizeOf(TExtended80Rec), 0);? ---------------------------------------------------------------------- Martin_fr · 2024-10-21 12:32 Ok, I fixed that with the declared type. Compiles. Running an app, and opening CPU-View => all empty. Stays empty even if I step, or restart the debugged app with the  Window already open. Statusbar says: Pid: 0 Tid: 0  State Unknown Addr 0x0 (No access) And yes, I have the very latest Lazarus. And FPC updated too. Windows 10 - 64 bit Using the FpDebug backend ---------------------------------------------------------------------- Martin_fr · 2024-10-21 12:48 Interesting, it works if the app is a GUI app, and has a form open. But not if it is a console app. (Well, may be something else, but that is the main diff I noted) ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2024-10-21 12:57 Quote from: Martin_fr on October 21, 2024, 02:21:11 pmI just updated my FPC 3.3.1 to the latest b693af69a6 But I get Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i FWHexView.Common.pas(630,29) Error: Identifier not found "TExtended80Rec" Oops, that's a mistake - I'll fix it. Thank you. ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2024-10-21 12:59 Quote from: Martin_fr on October 21, 2024, 02:48:30 pmInteresting, it works if the app is a GUI app, and has a form open. But not if it is a console app. (Well, may be something else, but that is the main diff I noted) Thanks, I'll check into why it's not connecting. ---------------------------------------------------------------------- Martin_fr · 2024-10-21 13:17 Quote from: Alexander (Rouse_) Bagel on October 21, 2024, 02:59:19 pmQuote from: Martin_fr on October 21, 2024, 02:48:30 pmInteresting, it works if the app is a GUI app, and has a form open. But not if it is a console app. (Well, may be something else, but that is the main diff I noted) Thanks, I'll check into why it's not connecting. My test is just a basic program, main begin/end block. No units. Just a few global vars for testing. ---------------------------------------------------------------------- Okoba · 2024-10-21 13:29 It seems very cool! I will follow this work. Thank you! ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2024-10-21 13:46 Quote from: Martin_fr on October 21, 2024, 03:17:08 pmMy test is just a basic program, main begin/end block. No units. Just a few global vars for testing. Got it, I'll test it. ---------------------------------------------------------------------- Martin_fr · 2024-10-21 14:58 I owe you an apology. My fault, the project I used to test had a project specific debugger  (Project options to gdb). Works fine if using FpDebug. ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2024-10-21 15:05 Quote from: Martin_fr on October 21, 2024, 04:58:44 pmI owe you an apology. My fault, the project I used to test had a project specific debugger  (Project options to gdb). Works fine if using FpDebug. Got it, yes indeed, CPU-View only works based on FpDebug ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2024-10-21 15:16 In the next few days, I want to add customizations and in them logging and crashdump settings to keep track of CPU-View status when parsing such errors. ---------------------------------------------------------------------- Martin_fr · 2024-10-21 18:07 Quote from: Alexander (Rouse_) Bagel on October 19, 2024, 04:00:28 pmI think it makes sense to add many of my solutions to the main debugger. It certainly does. Or will do. I think we need to start below, with the Interface/API. In order to have your code become part of the IDE it has to access the debuggers via API. And work with all (or most) of the available backends. At least let the backends have the option to implement what is needed. QuoteThere are not enough IOTA interfaces, all debugging is laid directly on specific windows. If they let me - I could offer my implementation. It will still take me a while to go through your code and understand it. But then it will also take good time to work on any Intf code. So I think its ok to start discussing it. I am certainly most interested on your ideas, and maybe the implementation you have. Trying to design an interface on my own, with little to no feedback is a receipt for oversight There are 2 (not up to date) wikipages https://wiki.freepascal.org/Lazarus_Debugger_Implementation https://wiki.freepascal.org/TDebuggerIntf Currently there are 2 packages - LazDebuggerIntf (new) - DebuggerIntf (old) Most of the stuff from the old interface should eventually be replaced. The concept of having Supplier/Monitor pairs is something that probably should survive, but needs tuning https://wiki.freepascal.org/Lazarus_Debugger_Implementation#TDebuggerD ataMonitor_and_TDebuggerDataSupplier_-_stack,_watches,_.... The new interface currently has mainly stuff for watches. Then there is the frontend. Internally it still has a lot of old code... IdeDebugger I saw that you use that package. It is an internal package, with no guarantee on compatibility. It is considered part of the inner IDE and will change with it as needed. However the plan is to provide an interface to it. (like IdeIntf for the rest of the IDE) For the development you are probably on the save side using it. (if you publish, mind that IdeDebugger is GPL, not LGPL - that is most of its code was originally in the IDE itself, and therefore has to be GPL) One important aspect is, that most parts of the interface should be async. Debugger may take time to respond, and the IDE should get blocked. Generally there will be different aspects to the interface. Some may be (but not limited to) 1) Control: Issuing commands (run, step, ...) I haven't done much thinking about that part yet 2) State (reported by the backend) That is still based on the old code, and for once unfortunately the state is reported for each  "section" (global, watches, locals, stack). While all of them may need there events, they don't need repeated calls of the same "entered pause" (dsPause). But nothing much concrete has yet been done. 3) Query data Probably what we gonna look at the most. So far some work has been done for watches. The importance here was that the backend can return the data in such a way that the frontend can format it in any way the user wants. The exceptions are memory dumps, which need to be queried separately. I think for registers will be a similar API like for watches. So Register values can be described in a target-unspecific way, but will at the same time include the raw content of the the register. Due to lack of a direct memory access API the mem-viewer (the one I did) uses a watch to fetch its data. Asm is still based on really old code. Fetching asm with gdb had several challenges. And they unfortunately still manifest in the data. 4) "Attributes/Properties" Some way of describing, what the debugger supports. But also what the target is. A generic IDE viewer, will always need all data in a target unspecific way.  But more or less specialized views may exist for certain targets. Before I go into details, I will wait for your initial thoughts (if that is ok with you). And hopefully find some time to further study your code. Couple of random notes: The watch viewer (mine) takes %RSP and then follows the stack. In all watches (fpdebug) % can be used to indicate a register name. Because "RSP" could be a normal variable defined by the user. Not looked in detail by your asm parser (script). With Fpdebug lots of that could be done in the backend, when the asm is still in structured data, rather than printed. But then other backends can't do it, they would then need access to that code in some shared unit. On the other hand, I don't know if you keep track of registers over multiple statements? ---------------------------------------------------------------------- 440bx · 2024-10-21 18:21 I can't help but voice a concern about CPU-View, which is... So far all Lazarus releases use formal releases of FPC, e.g, the current Lazarus release uses FPC v3.2.2 Personally, I am extremely interested in seeing the capabilities of CPU-View incorporated in Lazarus but, given that CPU-View requires the development/trunk version of FPC, it seems unlikely or problematic for, at least some, of Alexander's work to be incorporated into Lazarus. At this point I'm wondering, will some future version of Lazarus require the trunk version of FPC ? (in order to use the CPU-View features) or will some of the CPU-View code be reworked by the Lazarus developers to compile with the stable version of FPC (whatever version that happens to be at the time.) ? Just wondering how these features get into the "normal" Lazarus development. ---------------------------------------------------------------------- Martin_fr · 2024-10-21 18:40 Quote from: 440bx on October 21, 2024, 08:21:24 pmAt this point I'm wondering, will some future version of Lazarus require the trunk version of FPC ? No, or not what is then trunk, but what is now trunk may then be a release... Incorporating into the IDE, can only pick what works with current + previous release. That rule wont be broken. (Though if it is just some small part of the rtl/fcl => that can be distributed as copy => like the new TProcess, which is used already) But also it first needs to access the debugger in a more formal way (via an interface for that purpose). So before incorporating anything, such an interface must be created (that was a plan before the cpu view, but no has extra motivation / still a lot of work) ---------------------------------------------------------------------- 440bx · 2024-10-21 20:46 Quote from: Martin_fr on October 21, 2024, 08:40:00 pmIncorporating into the IDE, can only pick what works with current + previous release. That's reassuring.  Thank you for confirming that. ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2024-10-22 05:07 Quote from: Martin_fr on October 21, 2024, 08:07:09 pmBefore I go into details, I will wait for your initial thoughts (if that is ok with you). I didn't know there were two debugger options and implemented everything based on DebuggerIntf, I need to learn what LazDebuggerIntf represents. Regarding interfaces as I see them: 1. We need to introduce a set of interfaces under debugging windows IDisassemplerWnd, IRegistersWnd, IMemoryWnd, IStackWnd 2. The debugger itself should not know anything about the classes implementing these windows, the windows should be registered in the debugger, and the debugger should call the methods of these interfaces 3. All synchronization tasks with the worker thread should be assigned to the debugger interface. For example, now I have to call cache reset through DebugThread.BeforeContinue + TThreadWorker call before changing registers, or execute MaskBreakpointsInReadData + TThreadWorker 4. A bit later I can provide a more detailed description of all the necessary methods for each interface, but from the basic ones there should be - procedure Init(Debugger: TDebuggerIntf); // for proper initialization without the hack with DebugBoss.RegisterStateChangeHandler - procedure DoShowWindow(AHandled: Boolean); // to block the transfer of control and focus to other windows implementing the same interface - procedure DoStateChange(AOldState: TDBGState); Now in the debugger interface it is not quite clear what methods of the debugger can be called safely from the main thread. By trial and error I figured out that MaskBreakpointsInReadData must be called via TThreadWorker, I'm not sure about the other methods. Most likely, if somewhere else such synchronization is required and I have not performed it, it may lead to deadlock. There is no easier way to get the ID of the active thread. There is no direct access to TDebuggerIntf, I had to get it via DebugBoss.Snapshots.Debugger (but that may not be necessary after what you wrote) Register reading is only partially done, so I had to implement it completely by myself, as I need to fully work with SIMD XMM/YMM registers both in Windows and (importantly) in Linux And about the registers. Here I don't understand a little bit, what do you mean tracking? I react to notifications from DebugBoss->OnState and there I re-read the thread context + subscribe to TThreadsNotification.OnChange to re-update the registers when the thread switches in the debugger window. ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2024-10-22 05:11 Quote from: 440bx on October 21, 2024, 08:21:24 pmI can't help but voice a concern about CPU-View, which is... I need some time to explore the possibilities of FPC 3.2, probably it will be possible to modify the code of FWHexView (which is the basis for CPU-View Viewers) to support the stable version of the compiler. ---------------------------------------------------------------------- 440bx · 2024-10-22 05:24 Quote from: Alexander (Rouse_) Bagel on October 22, 2024, 07:11:36 amI need some time to explore the possibilities of FPC 3.2, probably it will be possible to modify the code of FWHexView (which is the basis for CPU-View Viewers) to support the stable version of the compiler. It would be really nice if your packages could use FPC v3.2.2.  I believe that would make it possible for them to be adopted more quickly which would benefit most everyone (at least those like myself who enjoy looking at the assembly code and debugging in assembler.) I've looked at the CPU-View screenshots and would love to use it ---------------------------------------------------------------------- Martin_fr · 2024-10-22 09:22 It would be good if you would in you posts disambiguate between backend and frontend. Debugger is a rather loose term. It be too much to say we have 2 interfaces. We have 2 places that define part of the interface. And important to point out: this is the interface between ONE frontend and ONE backend (more below). The new class mainly is watch value related. The backend used to do all the formatting (print as dec, or hex, or oct...). And now there is a way to just give the raw data. - DebugThread /ThreadWorker, are part of FpDebug. The normal frontend does not access them at all. - Threading is introduced in LazDebuggerFp and handled inside of it. And if used through the official intf, then its not visible to the outsider.   (Of course currently not all you needed is accessible via the proper interface) - Some threading is also in FpDebug, but again not visible outside The interface as far as it exists does not expose any threads. However it is meant to be called from the main thread only. I.e. it is not threadsafe, and can not be called from any random thread that the frontend or an extension may have created. 1 & 2) The backend already does not know of those windows.- - For the backend we need to have interfaces providing that data - For the frontend we can have interfaces, to either replace a window, or hook into it (if/when appropriate). For watches I started an interface for Value providers. That likely will be extended, up to and including access to the canvas for custom paint. 3) As mentioned those issues are the lack of an interface. Your work around then of course encounter deep internal details.... 4) - RegisterStateChangeHandler / Init / DoStateChange I have to do a bit of guessing here.  You mean - so the frontend extension can initialize itself? - or the frontend can send a command for the backend to initialize? You do have to register somehow for that. Either - As current, register event handlers - register an interface and implement that interface in your extension, and have those methods called in your extension via that interface. - or ? But yes, the current state is indeed to be replaced. It has to many internal values. More dedicated notifications should be added. To start with an example (from memory / exact implementation needs to be checked) (Assuming DebugBoss was exposed via IdeIntf or similar (TODO)) To be able to display watches the frontend registers a  IDbgWatchesMonitorIntf. If you would replace the entire frontend (not just add/replace a window, but replace everything (the entire IdeDebugger), you would register your own. Currently there is only one monitor allowed, so if you just add a new Window the the existing IdeDebugger, then you have to interact with a future interface of the IdeDebugger (as the IdeDebugger already is the current frontend). That currently does not yet exist. So you have to look at whatever DebugBoss.Watches offers. So there will be yet another API to be done... DebuggerFrontEndIntf That is imho correct. The Backend does not care that there are several Windows, and if they need or need not to know of each other. => E.g. what happens if I select an entry in the History window. For the Backend there is only one frontend. It registers one Handler for watches. The frontend then has to decide, if it needs to provide access for multiple parties (different windows) then that is something the frontend has to do. Mind that currently IdeDebugger is the only frontend. And in most cases extra frontend like your CPU view will be additions to that. => but you could (not yet) replace it entirely. By registering the IDbgWatchesMonitorIntf the IdeDebugger can also get new notifications (better than the current StateChange). But that hasn't yet been done. Currently we have Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i "Frontend (IdeDebugger)"   <>  LazDebuggerIntf/DebuggerIntf (the interface)  <> "Backend (E.g. LazDebuggerFp)" We also need (todo) Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i "Extensions" <> LazIdeDebuggerIntf <> "Frontend (IdeDebugger)" Then if you want to actually get data (evaluate a watch) you need to call (on the backend interface)   IDbgWatchesSupplierIntf.RequestData - That is probably going to be handed through to the backend by the future IdeDebuggerIntf. - You need to pass in a IDbgWatchValueIntf IDbgWatchValueIntf carries all the info needed. The context (stack/thread), the expression, any options, and an interface to receive the result. I opted for passing in an Interface (that the caller must implement). That allows easy extension in future. Of course anyone implementing the interface (e.g. IDbgWatchValueIntf) will have to add new methods if it gets extended. Therefore for most of the interfaces there is a template in form of a generic. So if the class implementing the interface inherits from that template, new methods will have default implementations. Something like... Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i   TWatchValue = class(specialize TDbgDataRequestTemplateBase, IDbgWatchValueIntf) So to get an expression evaluated as watch, you have a class implementing IDbgWatchValueIntf. You call: IDbgWatchesSupplierIntf.RequestData(myClass); The backend then can query the thread, stackframe, expression and more. The backend then can call BeginUpdate, ResData, SetValidity, EndUpdate to write the result. The EndUpdate or SetValidity acts as signal that the work has been done. The handling of ResData calls is currently in IdeDebugger (frontend). That should maybe move into a public package. But that is part of breaking everything into interfaces, and tools packages. You can get a memory dump with the above watches interface. But a dedicated API for that will be of interest too. Though there are some benefits from getting the memory via watches. If you want the memory for "MyFooInstance", then - using a watch you can pass the variable name, and it will get the address. - TODO: a flag could be specified if you want the variable or the data   objects are a pointer, so the variable would be the memory containing the pointer.   the data would be the memory pointed to. - The backend does know the size of the data, and can return the exact amount of memory. - The frontend can already override that size (at least extend it) Of course if you are looking at other memory, things like the size of a variable do not matter. And then a simpler API would help, where you just give an address (still could be a variable) and the size you want. QuoteThere is no easier way to get the ID of the active thread. I have to double check ThreadsMonitor.CurrentThreads.EntryById[ ThreadsMonitor.CurrentThreads.CurrentThreadId  ].ThreadTargetId That is the old interface (hence the complicated way). So changes can and should be discussed. Though the idea to have an internal ID (just between frontend and backend) is IMHO ok. It simplifies implementing many different backends, as they are free to specify what they expect. You can then query info using that ID. (like the OS id for the thread) QuoteThere is no direct access to TDebuggerIntf, I had to get it via DebugBoss.Snapshots.Debugger (but that may not be necessary after what you wrote) Register reading is only partially done, so I had to implement it completely by myself, as I need to fully work with SIMD XMM/YMM registers both in Windows and (importantly) in Linux Yes there is a lot more interface needed. For registers a similar interface like for watches should be created. Or well, If you have different ideas, I am happy to look at / discuss them. - Registers should be returned in a generic form, leaving all formatting to the frontend - This may include a copy of the raw content, if a frontend exists specific for the current architecture. - Changing the content of a register has to be implement too. And of course threads and stacks too.... And all the rest. Of course that will mean to change all existing backends. So its gonna take time, once a design for the interface has been decided. QuoteAnd about the registers. Here I don't understand a little bit, what do you mean tracking? I react to notifications from DebugBoss->OnState and there I re-read the thread context + subscribe to TThreadsNotification.OnChange to re-update the registers when the thread switches in the debugger window. You calculate addresses for access via registers? I haven't looked deeply at it. Is that just for  [RIP + 123] style access?  or also [RAX*4+123] ? If RAX too, then based on the current value (register from context for current thread) of RAX? Or if you have   mov RAX, [foo]    mov RBX, [RAX + 123]  In the 2nd statement would you have the value "[foo]" loaded to RAX in the first statement? Obviously only if there is on jump from other code to the 2nd statement. ---------------------------------------------------------------------- Martin_fr · 2024-10-22 10:11 As for the Monitor/Supplier concept. It might be possible to allow more than one Monitor (frontend) to register with the Supplier (backend). But that may be adding unnecessary work to the backends.... IMHO it is better the frontend acts as hub. Though the frontend could replicate the exact same interface. And just distribute/forward any calls. About state and notifications. There is a what and a how. ** How: - RegisterHandler - RegisterInterface and get callbacks to methods on the interface) ** What: There are (at least) 2 types. * Global Similar to the current state, but less internals. Also there are events that come from the backend, and events that the frontend can generate itself. Frontend: - New Backend - New Init (the IDE is about to start a new session), but nothing has been done yet (frontend) - maybe some steps during init.    for the backend they will differ - ... Backend: - RUN (initial start) usually immediately before launch - Pause - Resume (run or step) - Stop - Error - init and destroy might exist as direct response to the frontend requesting those actions. The frontend can then add to that what kind of event is happening. Though Pause will probably need 2 states (as currently) and it has to be seen if they can be split in the frontend or need to be done in the backend. Though the 2nd may not be called pause. Currently we have dsInternalPause. It is not a real pause, it is something were the debugger will continue. But the IDE should perform some tasks that it would do in a pause. Example: A breakpoint that is set not to pause ("break" action disabled), but records a snapshot => the Frontend must eval stack, watches etc and record that. Of course the trigger does not have to be called "pause". That is just owed to building on top of existing code for far to long a time.... * Specific That would be events applying to selected parts of the interface. E.g. special events for watches, or locals or stack, .... In most cases they can be derived from global events, but in some cases they might be better tunable if the backend issues them. Watches would just need to know - data available => usually when the debugger goes to pause - data no longer available - data needs to be refreshed => E.g. a register value was changed via API, or memory was written. The frontend would know, if it did write to memory. But if a register was changed, then maybe the backend can send invalidation for a specific thread... ---------------------------------------------------------------------- Martin_fr · 2024-10-22 10:20 And then there may be a frontend specific interface. Currently you register your CPU view via the normal IDE interface. It just lives in parallel to the existing windows. And general Window handling is probably to remain a task for IdeIntf, even for debugger specific windows. What may however be of interest is * If an existing debug window should be replaced => unregister the existing window. Matter of user preference in many cases. A user may not mind to have both windows (old and new). But for less experts it is probably better to just have one. (The new one would still register its menu and keys via the IdeIntf) * Hook signals to windows E.g. if the frontend decides it needs to display an asm window, because you stopped at a location outside the sources. Then you want to intercept that call, and show your CPU view. * Extend an existing window That would depend on what ideas for modification come up in future. ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2024-10-22 13:43 Quote from: Martin_fr on October 22, 2024, 12:20:16 pm Uff, thanks for the detailed response. I need to think this through. Some of the things you described are not fully clear to me, so I need to dive deeper into the current debugger mechanism. The frontend and backend mechanism itself is clear, all the interfaces I mentioned above are related to the frontend. Now about registers. In AsmView I calculate only [RIP + 123] to get a symbol at the calculated address in x64 code. In memHint displays more detailed information on registers and there can already be considered [RAX*4+123] (instead of registers are substituted current values + the math itself) The mov RAX, [foo] expressions are not processed yet because I don't know how to get variable values by name yet. ---------------------------------------------------------------------- Martin_fr · 2024-10-22 14:09 Quote from: Alexander (Rouse_) Bagel on October 22, 2024, 03:43:05 pmThe mov RAX, [foo] expressions are not processed yet because I don't know how to get variable values by name yet. foo was meant to be an address, the debugger doesn't return var names in asm. To clarify my question Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i   mov RAX, [0xF789]  // RIP at this line   mov RBX, [RAX + 123]   And at the first line (or current pause line) the actual values from CPU and mem: RAX = 0 mem F789 = AABB What happens on the 2nd line (without stepping, just disassembling for being paused at the first line / or even at some earlier point) On the 2nd line for the [RAX + 123]  it will show: - no comment - 0 + 123  = 123 - AABB + 123 = ABDE ? What happens if there is a conditional jump, and there is more than one path to that line? Sorry, I could just try... But then I need to write some code that ends up like that. ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2024-10-22 16:54 Quote from: Martin_fr on October 22, 2024, 04:09:34 pmSorry, I could just try... But then I need to write some code that ends up like that. In AsmVindow, nothing will be displayed for both cases because there is no RIP marker in the address In memHint for the string "mov RAX, [0xF789]" will be displayed: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i RAX = 0 (No access)[0XF789] = [F789 (R...)] -> AABB (No access) for the string "mov RBX, [RAX + 123]" will be displayed: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i RBX = 0 (No access)[RAX+0X7B] = [7B (No access)] -> 0 (No access) i.e. the current values of registers are taken, the address is calculated, access is checked and if there is a pointer (i.e. instruction [xxxxx]) the same is done for this address as well. (see attachments) But if the string is like this: “MOV RAX,[RIP+0xF789]”, the calculated value with RIP marker will be displayed in brackets, and if debugging characters are detected at this address, they will be displayed as well. ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2024-10-23 13:21 Quote from: 440bx on October 22, 2024, 07:24:41 amIt would be really nice if your packages could use FPC v3.2.2.  I believe that would make it possible for them to be adopted more quickly which would benefit most everyone (at least those like myself who enjoy looking at the assembly code and debugging in assembler.) I've looked at the CPU-View screenshots and would love to use it I have finished translating FWHexView and CPUView to support the stable version of the compiler (3.2.2). You can check them out. Important - there have been significant changes in the source code during these days, including rearrangement of folders for storing source code, so if you still have old versions of the packages, it is better to delete them and replace them with new ones again with obligatory cleaning of the lib folder. There's also a small problem with the installation, I'm still sorting it out, here's a description of it: https://github.com/AlexanderBagel/CPUView?tab=readme-ov-file#known- issues ---------------------------------------------------------------------- 440bx · 2024-10-23 14:32 Quote from: Alexander (Rouse_) Bagel on October 23, 2024, 03:21:15 pmI have finished translating FWHexView and CPUView to support the stable version of the compiler (3.2.2). You can check them out. That's great.  Thank you Alexander. ---------------------------------------------------------------------- 440bx · 2024-10-23 15:46 @Alexander, Just FYI, In FMHexView, building the ZipViewer demo fails because of the dependency to FWZip.  This isn't mentioned anywhere.  Also, all the documentation for FWZip is in Russian, which is a bit "inconvenient" for those who don't speak Russian (like myself.) The Hex Editor demo does not compile.  The compiler emits an error (see attachment.)  The "simple" and "virtual pages" demo, both compile and run without problems. ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2024-10-23 16:05 Quote from: 440bx on October 23, 2024, 05:46:12 pm@Alexander, Just FYI, In FMHexView, building the ZipViewer demo fails because of the dependency to FWZip.  This isn't mentioned anywhere.  Also, all the documentation for FWZip is in Russian, which is a bit "inconvenient" for those who don't speak Russian (like myself.) The Hex Editor demo does not compile.  The compiler emits an error (see attachment.)  The "simple" and "virtual pages" demo, both compile and run without problems. Yes, indeed, I completely forgot about the need to translate FWZip. I'll get to that (and all the demos) a bit later. Thanks. ---------------------------------------------------------------------- 440bx · 2024-10-23 17:09 Quote from: Alexander (Rouse_) Bagel on October 23, 2024, 06:05:10 pmYes, indeed, I completely forgot about the need to translate FWZip. I'll get to that (and all the demos) a bit later. Thanks. My pleasure and thank you for making your work available to others.  Good stuff. ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2024-10-24 05:24 Quote from: 440bx on October 23, 2024, 07:09:49 pmMy pleasure and thank you for making your work available to others.  Good stuff. You're welcome, I myself need new debugger functionality and someone to look at and test Demo examples are fixed. ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2024-10-31 06:01 Quote from: Fibonacci on October 20, 2024, 10:18:00 amDuring my short playtime with it, it crashed twice and froze once I added a debug log in the settings. If a hang or error occurs in the future, please open the issue and attach Debug.log Thank you. https://github.com/AlexanderBagel/CPUView?tab=readme-ov-file#debug- log-and-crash-dump ---------------------------------------------------------------------- Fibonacci · 2024-10-31 14:15 I just want to say, I have been using this tool almost daily since the error I mentioned in my first post (on the first page) got fixed. This tool is superb! Helped me a lot with current development, where I need debugging all the time (digging around in the RTL). Keep stuffing new features to this tool! Me likey. Thanks. I see you added option to configure it in the IDE options. Defaults suits me, worth updating? I use commit "Thu, 24 Oct, 2024, 6:26". Is there a way to "walk back" if I click the calls/jumps? Left arrow, minus or backspace could do it. I know I can return to EIP. Add an option to add a button to the toolbar. Options -> Editor ToolBar. I'll update if you add the button ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2024-10-31 15:38 Quote from: Fibonacci on October 31, 2024, 03:15:55 pmI see you added option to configure it in the IDE options. Defaults suits me, worth updating? I use commit "Thu, 24 Oct, 2024, 6:26". Is there a way to "walk back" if I click the calls/jumps? Left arrow, minus or backspace could do it. I know I can return to EIP. Add an option to add a button to the toolbar. Options -> Editor ToolBar. I'll update if you add the button If you are happy with the default values, you don't need to update in principle. Jumping forward/backward is supported by the mouse buttons (MK_XBUTTON1/MK_XBUTTON2) or Esc (step backward). As for the toolbar button - I don't know how to do that yet, need to look into it. I'll add it to the development plan. ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2024-11-02 05:06 Quote from: Fibonacci on October 31, 2024, 03:15:55 pmAdd an option to add a button to the toolbar. Options -> Editor ToolBar. I'll update if you add the button A button has been added to the toolbar editor. UPD: If you upgrade, it is advisable to update the FWHexView package as well, as there were bug fixes there too. ---------------------------------------------------------------------- Fibonacci · 2024-11-02 14:59 Quote from: Alexander (Rouse_) Bagel on November 02, 2024, 06:06:01 amA button has been added to the toolbar editor. Great news I'll check it out soon. Thanks for adding it. ---------------------------------------------------------------------- Fibonacci · 2024-11-03 00:24 I forked your repos and changed 2 things: 1. Go backward on VK_BACK, not VK_ESCAPE 2. VK_ESCAPE hides the window ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2024-11-03 03:17 Quote from: Fibonacci on November 03, 2024, 01:24:25 amI forked your repos and changed 2 things: 1. Go backward on VK_BACK, not VK_ESCAPE 2. VK_ESCAPE hides the window I think this needs to be tucked away in the settings, I'll add hotkeys a little later. UPD: I've made some preliminary preparations to customize the ShortCuts https://github.com/AlexanderBagel/FWHexView/commit/83928992189771199c6 fe5d20a6a824d3e5beb79 And also added for the new Secondary ShortCut to handle the stack of jumps (Num+/Num-) https://github.com/AlexanderBagel/CPUView/commit/1abc496a20c86d94911a4 7ca08479d280f1c32da ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2024-11-04 03:44 Quote from: Fibonacci on November 03, 2024, 01:24:25 amI forked your repos and changed 2 things: 1. Go backward on VK_BACK, not VK_ESCAPE 2. VK_ESCAPE hides the window I considered this mode of operation and it seemed in principle quite convenient. I made changes in the code, now VK_ESCAPE closes the dialog, jump control buttons VK_RETURN/VK_BACKSPACE, Num+/Num- or mouse buttons MK_XBUTTON1/MK_XBUTTON2. It is necessary to reinstall both packages. ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2024-12-29 16:30 Well, I will suspend CPU-View development for the New Year holidays. A lot of changes have been implemented over the last couple of months. Of the most important ones: Deep debug information search. This mode analyzes the entire address chain in search of a debugging symbol. Address validation - a mechanism that searches for information about each displayed address Active highlighting and hints to validated addresses (I've been working on the organization of hints for a long time). A lot of other things I can't remember already Don't forget to reinstall FWHexView first when updating CPU-View: https://github.com/AlexanderBagel/FWHexView And also CPU-View itself https://github.com/AlexanderBagel/CPUView ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2025-01-17 08:36 As of January 17, 2025, daily CPU-View updates have been suspended.  All core functionality has been added, only 4 steps remain, each of which will require a long development time and will not be posted so as not to break the current CPU-View behavior.  1. SIMD register editor. 2. Window displaying function call parameters. 3. Carbon/Cocoa widget support under macOS. 4. Support for ARM architecture. Don't forget to reinstall FWHexView first when updating CPU-View: https://github.com/AlexanderBagel/FWHexView And also CPU-View itself https://github.com/AlexanderBagel/CPUView ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2025-02-24 15:39 A small update in CPU-View. To be more precise, there were a dozen of them with bug fixes, but the current one already extends the functionality. So the general concept: In the process of debugging can often happen a situation with a failure of the debug stack, this is due to many reasons, usually playing with stack registers RBP/RSP. When the stack is broken, it is not clear where the call came from and it causes great difficulties when debugging such code. In such a situation, the “Step Out” command will be executed incorrectly and with 99% probability will simply start the process for execution instead of stopping at the address of the current procedure call. Some time ago I added a “Run to User Code” mechanism that relies mostly on information from the debugger, and if it didn't contain the necessary data, the stack frames were analyzed. This approach works quite well on Windows, but unfortunately it doesn't work on Linux anymore. Gtk libraries are built without using stack frames and stack promotion via RBP/RSP does not give the necessary result. Having experimented with several variants of solving the problem, I found one that works quite accurately, the essence of which is to analyze the entire stack below the RSP and group setting temporary Breakpoints by a list of all possible return addresses. Experimental results on the video: https://raw.githubusercontent.com/A lexanderBagel/CPUView/main/img/run_to_user_code.mp4 The code is available on GitHub: https://github.com/AlexanderBagel/CPUView Don't forget to reinstall FWHexView first when updating CPU-View: https://github.com/AlexanderBagel/FWHexView ---------------------------------------------------------------------- Martin_fr · 2025-02-24 18:57 Quote from: Alexander (Rouse_) Bagel on February 24, 2025, 04:39:27 pmIn the process of debugging can often happen a situation with a failure of the debug stack, this is due to many reasons, usually playing with stack registers RBP/RSP. When the stack is broken, it is not clear where the call came from and it causes great difficulties when debugging such code. In such a situation, the “Step Out” command will be executed incorrectly and with 99% probability will simply start the process for execution instead of stopping at the address of the current procedure call. I have to double check, but I think the "Step out" is not currently even using the same "smarts" as the callstack. Originally, both just had the "RSP" (which already becomes tricky, when the code is asm stepping through the "leave" part of the procedure. Also the RSP has a habit of skipping frames, when you have an optimized fpc RTL  (or your own code) with omitted frames (some functions not setting the RSP). Then those functions will be visible, but their caller will be hidden. (Hence the RTL can hide routines from the users code in the stack). The stack then was improved. - The stack now can use dwarf to find its parent - The stack can actually disassemble the machine code, and often find exactly where the code would return, and what would be popped of the stack before. Of course the disassembling is limited. It only has a known subset of asm that it "interprets". But it manages most of the Window and Linux kernel and system libs (well in my tests, that is) But the "Step out" has not yet been updated. So even in cases where the stack shows this, the step out may not use it and fail. That is another todo. ---- As for the stack, in cases nothing can be found, your idea is a good last resort. I have seen gdb doing something like that: any code address was shown as a stackframe. So there were many extra/wrong frames. But the correct ones where in there too. ---------------------------------------------------------------------- Martin_fr · 2025-02-24 19:01 Which Lazarus version was the video made with? I wonder, because I have gotten stacktraces in the gtk libs, going over more than 20 frames down to the LCL code. ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2025-02-24 19:51 Quote from: Martin_fr on February 24, 2025, 08:01:19 pmWhich Lazarus version was the video made with? I wonder, because I have gotten stacktraces in the gtk libs, going over more than 20 frames down to the LCL code. Lazarus 4.99 (rev main_4_99-1103-g6c528d9822) FPC 3.3.1 x86_64-linux- gtk2 ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2025-02-24 19:58 Quote from: Martin_fr on February 24, 2025, 07:57:35 pmAs for the stack, in cases nothing can be found, your idea is a good last resort. I have seen gdb doing something like that: any code address was shown as a stackframe. So there were many extra/wrong frames. But the correct ones where in there too. At first I used to exit to the lost stack by manually passing through Jxx (via F4) + RET, then I tried to automate it and it even worked, but slowly, because this option implies a multiple cycle of starting/stopping the process at the calculated address through disassembling. In the end I gave up this approach, because the debugger has a really powerful mechanism for setting multiple temporary breakpoints, it was just perfect for solving this problem for O(N) since I have all the data available, I just needed to collect them in one array. ---------------------------------------------------------------------- Martin_fr · 2025-02-24 20:09 Well, ... about multiple breakpoints. There is a tiny danger in it. (One that actually has happened with gdb, because older FPC wrote incorrect info). If you have a breakpoint at the wrong address, then the app may crash. - Imagine you have an asm statement at 0x223308. that statement is 3 bytes long. - On the stack you find the address 0x223309 - Now you may check that the asm at this address is a "call" statement, but the 2nd byte of that real statement may just be a "call". So you are setting a breakpoint in the middle of a statement. If the code then reaches that statement, it does not see a breakpoint, but instead executes a modified statement. That is most likely to cause trouble. ---------------------------------------------------------------------- Martin_fr · 2025-02-24 20:14 As for using disassembly for calculation. Yes that is slower. Well, if you just want to display a stack in the stack window, then you need the info without execution anyway. So for display of a stack it still is a good way. Of course displaying all potential frames also works, and then the user can select. But if the user selects wrong, and a breakpoint gets set then the problem is back. The disassembly resolver is in FpDbgCpuX86 Maybe we can figure out why it stops early in some of the stacktraces you have. If it can be improved that would be good. Also maybe it can be compared with your approach? (I understand you wrote your own decoder for that purpose?). I have to go looking through my "old bits of code". I should somewhere have some debug patch for that code, which will output info, for what it tries, and where it fails. ---------------------------------------------------------------------- Martin_fr · 2025-02-24 20:17 My bad, the dissas-stack-tracer is in FpDbgDisasX86. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i function TX86AsmDecoder.UnwindFrame( First thing you can try is to increase the following values Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i const  MAX_SEARCH_ADDR = 8000;  MAX_SEARCH_CNT = 400;  MAX_ADDR_DONE_BLOCKS = 10;  MAX_FORWARD_ADDR = 5;  Just multiply them all by 10. And see if that gets any improvement. ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2025-02-25 03:22 Quote from: Martin_fr on February 24, 2025, 09:17:54 pmFirst thing you can try is to increase the following values Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i const  MAX_SEARCH_ADDR = 8000;  MAX_SEARCH_CNT = 400;  MAX_ADDR_DONE_BLOCKS = 10;  MAX_FORWARD_ADDR = 5;  Just multiply them all by 10. And see if that gets any improvement. Got it, I'll check those parameters as well. UPD: this has no effect on the current build, I will check it later on the latest trunk. ---------------------------------------------------------------------- Martin_fr · 2025-02-25 09:41 Quote from: Alexander (Rouse_) Bagel on February 25, 2025, 04:22:41 amUPD: this has no effect on the current build, I will check it later on the latest trunk. The latest changes were in Oct 2024. So if you have that, then you are up to date. I did have a look though and saw that I did not fully implement all variants of "push/pop". Mainly pushf / pusha and popf / popa. I just never encountered them, and didn't spent time looking up what exactly they do. So if your libs have them, then that may be a reason. ---------------------------------------------------------------------- Martin_fr · 2025-02-25 12:52 I found an install of mine where the gtk mouse click did not resolve very far. (different trace than yours though) I added some more logic to the asm unwinder. Not yet live, needs some testing. https://gitlab.com/martin_frb/lazarus/-/compare/main...fpdebug-unwind- asm-push?from_project_id=28419588 The 2nd commit may be extended to store more known registers if they get pushed. It still has some issues, but gets a bit further now. There is a commit adding debugln https://gitlab.com/martin_frb/lazarus/- /commit/6187969408491c3d963f5887c6995cc08a215265 (it needs to be rebased for the latest amend) ---------------------------------------------------------------------- Martin_fr · 2025-02-25 17:19 I actually found a bug in the existing code. Changes + fix: https://gitlab.com/martin_frb/lazarus/-/compare/main...fpdebug-unwind- asm-push-2?from_project_id=28419588 Still testing ... ---------------------------------------------------------------------- Martin_fr · 2025-02-25 20:46 Quote from: Martin_fr on February 24, 2025, 09:09:49 pmWell, ... about multiple breakpoints. There is a tiny danger in it. (One that actually has happened with gdb, because older FPC wrote incorrect info). If you have a breakpoint at the wrong address, then the app may crash. - Imagine you have an asm statement at 0x223308. that statement is 3 bytes long. - On the stack you find the address 0x223309 - Now you may check that the asm at this address is a "call" statement, but the 2nd byte of that real statement may just be a "call". So you are setting a breakpoint in the middle of a statement. If the code then reaches that statement, it does not see a breakpoint, but instead executes a modified statement. That is most likely to cause trouble. Actually, if you use that only to return to "user code", i.e. where you have debug info, then you can disassemble from the start of the function, and check you have the start of a real asm instruction.  Since you have line info, you only need to disassemble one lines worth of data. (Can still be a lot...) ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2025-02-26 08:27 Quote from: Martin_fr on February 25, 2025, 06:19:37 pmI actually found a bug in the existing code. Changes + fix: https://gitlab.com/martin_frb/lazarus/-/compare/main...fpdebug-unwind- asm-push-2?from_project_id=28419588 Still testing ... I checked, unfortunately the stack remains the same and stopped at libgtk. I also made changes to MAX_SEARCH_ADDR, etc... but it didn't help ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2025-02-26 08:28 Quote from: Martin_fr on February 25, 2025, 09:46:05 pmActually, if you use that only to return to "user code", i.e. where you have debug info, then you can disassemble from the start of the function, and check you have the start of a real asm instruction.  Since you have line info, you only need to disassemble one lines worth of data. (Can still be a lot...) Yes, I've already come up with a similar strategy to more accurately optimize the return address, I'll plug it in a bit later ---------------------------------------------------------------------- Martin_fr · 2025-02-26 21:17 I did some more improvements. In some cases getting through gtk stacks now. In others still not. (well, it was always clear, it would never do all). Also, there is no guarantee the returned frames are always correct. There can be code that modifies the stack pointer in conditional blocks. The unwinder would not know which branch to take. Nor does it analyse if (and also not where) branches join again, or what differences they have. It simply walks one branch, if that hits a "ret" statement, then that is good (well if that ret returns to after a "call" statement). If it does not find a good result it tries (a limited amount of) other branches. Those libs may even have dwarf info to unroll, but IIRC there are version diffs, so we can't yet read that. One day.... ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2025-02-28 10:44 Quote from: Martin_fr on February 26, 2025, 10:17:57 pmI did some more improvements. In some cases getting through gtk stacks now. In others still not. (well, it was always clear, it would never do all). Also, there is no guarantee the returned frames are always correct. There can be code that modifies the stack pointer in conditional blocks. The unwinder would not know which branch to take. Nor does it analyse if (and also not where) branches join again, or what differences they have. It simply walks one branch, if that hits a "ret" statement, then that is good (well if that ret returns to after a "call" statement). If it does not find a good result it tries (a limited amount of) other branches. Those libs may even have dwarf info to unroll, but IIRC there are version diffs, so we can't yet read that. One day.... Martin, I checked the most recent trunk and it works!!!! Great job! Thanks ---------------------------------------------------------------------- Martin_fr · 2025-02-28 11:31 Glad to hear. If you find any that don't: - test higher limits. - step out to that frame and single step the asm, until you get to a statement after which it works.   - however, there may not be a visible change in the frame, if something fails in asm-unwind, but is resolved as fallback by RBP- unwind.     So that needs to be disabled for those tests / or monitored. Then it can be checked if that statement could be handled by the parser. If it is a jump (conditional or not), then there are limits. 1) the amount of branches kept for following later. 2) going back only keeps one point 3) going far forward only keeps one The last two would be a lot of work to change. Background:   jnz  + 10 Will keep RIP+10 as potential alternative (but keep continue at the next statement). If there is no "jmp -100" in the code, then it will joint the "+10" and the branch can be removed. So that limits complexity ONE jump back will be stored, and it will be the one furthest back. In the hope that all other back jumps (that go a lesser amount back) will then be reachable from that furthest point. That may not always be the case, but again complexity. (Keeping track of blocks already done). Going forward really far, is the latest addition, and solved some of the issues. Not sure if a 2nd storage slot will help much here... ------------ * I have seen   jmp [rax] That is not handled. Registers are sometimes know. But that knowledge is of variable quality. So following this seems too risky. * Similar reading from memory (to get RSP / RPB / RIP and other) - only done from stack - only from areas of stack that weren't "pushed" to. (at least for rsp / rpb) - for jumps done for [rip+123] because that usually is in the code section, i.e. constant data. * On Windows there is a function in exception handling (OS kernel) that goes to the calling stack, by executing "call ...". So the subroutine somehow unrolls to the outer stack. That is impossible to follow, since stepping in isn't part of the game. ----- The code can probably be speed up a tiny bit, by optimizing calls to "ReadData" but that is a different story. ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2025-02-28 13:14 Quote from: Martin_fr on February 28, 2025, 12:31:02 pmGlad to hear. If you find any that don't: - test higher limits. - step out to that frame and single step the asm, until you get to a statement after which it works.   - however, there may not be a visible change in the frame, if something fails in asm-unwind, but is resolved as fallback by RBP- unwind.     So that needs to be disabled for those tests / or monitored. Got it, I'll do some more research and post if I have any suggestions. ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2025-03-25 19:41 Martin, you may be interested in this information. I found out that the work with debug symbols of external libraries, namely with the list of their exported functions, is not optimally implemented. Namely, when getting the function name by address, the usual search of the linear list is performed, not even binary search is used. This gives a significant delay when debugging in the address space of system libraries, especially in Linux, where search costs can increase more than 50 times (I have specially performed profiling). As a consequence, the standard AsmView starts to slow down very much when working. In my CpuView I added a local cache of functions exported by libraries, because I need to support older versions of Lazarus, but in newer versions it can be patched to fix the brakes in AsmView. Here is a link to the patch itself so you can see my approach to solving this problem: https://github.com/AlexanderBagel/CPUView/commit/00c35d102a87fceae9fd1 0b7e900674a48e4a499#diff- 9354d6d7ea0473c3c5f240e475819a5b04b7c009dd362f236ca0c915dbe12dd5 File: ‎src/core/CpuView.FpDebug.pas ---------------------------------------------------------------------- Martin_fr · 2025-03-25 19:49 Ah thats (part of) why disass is slow. I currently am busy with a few non debugger related items. But I had it on my list to callgrind it... (there are probably more issues) I'll see that I find some time to look into... Thanks. ---------------------------------------------------------------------- Martin_fr · 2025-03-31 10:51 I did a little improvement for the address lookup. They now use a bin search. There may (probably) be more possible optimisations in the disass code. (like the entire prepare ranges, which is a left over from old days). But that will have to wait. It now needs to sort them all when loading. On Linux that should be happening in extra threads (at least most of it). So it may even improve loading times. (because then name lookups will be faster). On Windows the order of loading, and looking up is different, and it has to wait for the threads more often. But it seems to have a similar speed than before. I hope there aren't any big projects that will load slower from that. -> the old code had to lookup 4 or 5 names in an unsorted list. -> the new code now has to sort, and create the hash dictionary, but then will be faster at looking up the names. ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2025-03-31 12:10 Quote from: Martin_fr on March 31, 2025, 12:51:13 pmI did a little improvement for the address lookup. They now use a bin search. Great, I'll be testing this out shortly ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2025-04-04 13:14 Quote from: Martin_fr on March 31, 2025, 12:51:13 pmI did a little improvement for the address lookup. Martin, thank you very much. I checked it out - the debugger work in the system libraries area is incredibly faster! ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2025-10-29 03:23 Development of the version supporting ARM processors has been completed. Many changes have been made. Support for the GDB backend for x64_64 and AArch64 architectures (64-bit only) has been added. To speed up work with GDB/MI, the code for working with debug information from FpDebug has been merged, which has resulted in a significant speed increase. Multiple optimizations have also been made to speed up the work of CpuView itself. https://github.com/AlexanderBagel/CPUView ---------------------------------------------------------------------- valdir.marcos · 2026-02-16 05:45 Quote from: Alexander (Rouse_) Bagel on October 19, 2024, 04:00:28 pmI do reverse engineering, and I miss the capabilities of the regular FpDebug debugger. Today I want to show my work that I've been doing for almost a year to make the debugger better. It's not finished yet, it needs a lot of tweaks, but I need someone to look at it. It's a full GUI debugger with a lot of supported features. I've implemented in it what I was missing. I think it makes sense to add many of my solutions to the main debugger. CPU-View is under development and I make changes to it every day, I miss the opinion of specialists who are immersed in this task (it's not application level - it should really be system engineers). There are not enough IOTA interfaces, all debugging is laid directly on specific windows. If they let me - I could offer my implementation. https://github.com/AlexanderBagel/CPUViewThanks. ---------------------------------------------------------------------- Alexander (Rouse_) Bagel · 2026-09-26 15:15 A new version of Cpu-View has been released, the main changes are in the Windows version. Since several people have requested this feature, support for debug PDB symbols has been added to facilitate debugging system libraries. It works independently of dbghlp.dll and symsrv.dll, because the dbghlp.dll included with the operating system provides only minimal functionality. Therefore, a full-featured MSF container parser and a class for working with symbols have been implemented in their place, along with a network component to replace symsrv.dll that supports both one- and two-tier symbol storages. By default, this feature is disabled, it can be enabled on the CPU- View settings tab in the "PDB Symbols" section. Since Microsoft has stopped publishing symbol installation packages and has officially left only an MSIL link to the server, a separate symbol download manager has been implemented (the “P” button on the toolbar). https://github.com/AlexanderBagel/CPUView To use this feature, you must first update the “Hex Editor” package https://github.com/AlexanderBagel/FWHexView