[SOLVED] Debugging dynamic (shared object) library on linux
lazarus / 12-Debugger

----------------------------------------------------------------------
linvon · 2024-08-28 10:40

Hello! I am trying debug dynamic library and having some troubles with
this on linux. I have Lazarus 3.4 and FPC 3.2.2.
I have 2 separated project: for library and for application
Compiling all this things in debug mode.
Using standard debugger (FpDebug).
Loading libaray in app with LoadLibrary and getting correct lib
handle.

But when i put break-point in application project debugger ingores it
from app. I also tried set 'host application' in library project - no
effect.
Can you help me please?

----------------------------------------------------------------------
Martin_fr · 2024-08-28 11:07

I don't currently have 3.4 installed on Linux. Only 3.99.  (testing
with Fedora 38)

There were some fixes, but I don't know if they cover your case. E.g.:
https://gitlab.com/freepascal.org/lazarus/lazarus/-/issues/40973

In any case in 3.99 it worked both ways
- Using the library project and setting host application
- Using the host project and just loading the library (if that is
compiled with debug info)

My test code:

Host project:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
project1; {$mode delphi}{$H+} uses  {$IFDEF UNIX}  cthreads, 
{$ENDIF}  Classes, dynlibs; var  hl:TLibHandle; 
hello_lib:function(const pc:PAnsiChar):integer;begin   writeln('Start
test');  hl:=LoadLibrary({$ifdef LINUX}'./libproja.so'{$endif}   
 {$ifdef MSWINDOWS}'./proja.dll'{$endif}     );  if hl>0 then  begin 
 @hello_lib:=GetProcAddress(hl,'hello_lib');    if Assigned(hello_lib)
then    hello_lib('Out form lib OK');  end;end.
lib:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i library
proja; {$mode delphi}{$H+}  function hello_lib(const
pc:PAnsiChar):integer;begin  Result:=0;  writeln(pc);end; exports 
hello_lib;end.
When debugging the host project, and the library has not yet been
loaded, then the breakpoint shows as "paused" (a red dot, whit 2
green/yellow bars (like pause on a music player). And then becomes
active when the lib is loaded.

I haven't tested what happens if there are units of the same name in
both the host, and the lib... IIRC it should work.

I need more info on your case.
- Do you at least get the "paused" breakpoint?
- If you open the debugger "event log" does it say something like
"Loaded: libfoo.so (/path/to/libfoo.so)"

----------------------------------------------------------------------
linvon · 2024-08-28 14:13

Yes, I have "paused" breakpoints, but nothing in "event log".
However, the library works, I just can't debug it.

----------------------------------------------------------------------
Martin_fr · 2024-08-28 15:35

What is your exact Linux environment?

Maybe something changed in one of the involved libraries.

You can always try to debug your app using the gdb backend. But that
has of course other limits.

----------------------------------------------------------------------
linvon · 2024-08-29 07:31

I will look for the environment.
But with gdb same behaviour.

----------------------------------------------------------------------
korba812 · 2024-08-29 08:03

Which project do you have loaded in Lazarus IDE when debugging?
Application or the library? You should open library project, set host
application in startup parameters and then run library project with
debugging (F9).
Remember to enable debugger information generation in the library
project.

----------------------------------------------------------------------
linvon · 2024-08-29 09:37

I have "project group" with library and application.
Debug information for library always generates

----------------------------------------------------------------------
Martin_fr · 2024-08-29 09:40

Quote from: korba812 on August 29, 2024, 10:03:20 amYou should open
library project, set host application in startup parameters and then
run library project with debugging (F9).
It also works (or should work) when you open the main project in the
IDE, and it loads the library.


A few more questions:

Both main project and library are fpc projects?
Both have dwarf debug info enabled?

You did not use any special settings for the linker? (static vs
dynamic / stripping / ...)

You did not strip any of the exe/so ?
You did not strip any of the dll in /usr/lib... ? (really unlikely)

The reason I am asking...

In order for breakpoint in libraries to work, the debugger must be
aware when a lib is loaded. For that there is a "trigger" in one of
the system libs.

If FpDebug fails to get that trigger, then breakpoints in libs wont
work. (I don't know what the code in gdb is nowadays doing, but google
suggests that it at least at some point used the same method, and
maybe still does)

Lets do some checks please.

1) I don't know the exact name of your systems shared libs. But they
should be something like the examples.

They may just be in /usr/lib or some other lib folder
find the ld lib:
Code: Bash  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i find /usr/
-name 'ld*.so*'get some info about it (you need to change the name
according to what you found above):
Code: Bash  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i objdump -t
/usr/lib64/ld-linux-x86-64.so.2 | grep -i _debug; objdump -T
/usr/lib64/ld-linux-x86-64.so.2 | grep -i _debug
copy and paste the result here, please.


2)
Can you apply the patch below please.

Then recompile the IDE (Tools > Build Lazarus).
Open a terminal/console/shell, and run lazarus from that terminal (so
you can see what it prints to stdout)

Run your project
Copy and paste anything that Lazarus printed.

Mind the below has a tiny functional change too. With some extreme
amount of luck it may even fix the issue. (Though I really doubt it,
because then gdb should not have the issue too)


Code: Diff  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i diff --git
a/components/fpdebug/fpdbglinuxclasses.pas
b/components/fpdebug/fpdbglinuxclasses.pasindex e72c919e7d..5601c9a9cc
100644--- a/components/fpdebug/fpdbglinuxclasses.pas+++
b/components/fpdebug/fpdbglinuxclasses.pas@@ -1564,7 +1564,7 @@
var  begin   inherited LoadInfo;-
+writeln('############################################################
########');   // This would be strange, but you never know.   if
Assigned(FSOLibEventBreakpoint) then     Raise Exception.Create('SOLib
event-breakpoint already exists.');@@ -1576,6 +1576,7 @@ begin   
 InterpSection := LoaderList.Items[i].Section['.interp'];     if
assigned(InterpSection) then       begin+writeln(
LoaderList.Items[i].FileName );       // Try to retrieve the inode of
the file (library) in the .interp section.       // This is the
filename of the library that handles the dynamic loading.       if
FpStat(PChar(InterpSection^.RawData), AStat) = 0 then@@ -1585,10
+1586,16 @@ begin         SynchronizeProcMapsWithLibraryList();      
 if FLibMap.GetLib(Astat.st_ino, ALib) then           begin+
writeln('in library : ', ALib.Name );+           // Set a breakpoint
at _dl_debug_state. This procedure is called after           // one or
more libraries have been loaded. This breakpoint is used to         
 // detect the (un)loading of libraries.         
 FSOLibEventBreakpoint := AddBreak('_dl_debug_state', False,
ALib);+writeln(FSOLibEventBreakpoint<> nil);+if
FSOLibEventBreakpoint<>nil then writeln(' state ',
ord(FSOLibEventBreakpoint.State));+if FSOLibEventBreakpoint.State <>
bksOk then FSOLibEventBreakpoint.Free+else         
 TFpDbgBreakpoint(FSOLibEventBreakpoint).FreeByDbgProcess := True;   
      end         end;

----------------------------------------------------------------------
Martin_fr · 2024-08-29 10:19

Also, can you test/confirm if/that the issue also exist with the
sample app that I posted earlier?

----------------------------------------------------------------------
linvon · 2024-08-29 10:30

Guys you gave me a clue. I doubted that the project compiles with
debug information.
1. Then i tried compile library with debug info in separated debug
file .dbg
Code: Bash  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i fpc -g -gl
-Xg -O- -CX -XX -fPIC -vewnhi -l -olib123.so lib123.lpr2. Then check
the file size:
Code: Bash  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i du -h
lib123.so And got 1.2M and no lib123.dbg file.

I removed flag -Xg and recompile (without lib123.dbg)
Code: Bash  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i fpc -g -gl
-O- -CX -XX -fPIC -vewnhi -l -olib123.so lib123.lprCheck the size
again like in paragraph 2 and got 2.1M.
After recompile library with debug info inner I was able to debug and
go through to lib code.

For some reason in linux with flag -Xg external debug file not
generated ONLY for dynamic libs, with executable no problem.
With flag -Xg under windows .dbg file generated for both cases(lib and
executable) without problems

Thank you, Martin_fr for hint!