Windows Linking and Debuging. lazarus / 21-General ---------------------------------------------------------------------- roger T · 2024-04-29 08:27 Not sure if the is the right place to post this but hopefully someone will let me know if there is a better place. To try and shorten a longish story, I have developed my own programming language (www.languageONE.com.au) and have previously used FLTK to provide the GUI. It works fairly well across Linux and Windows but it is all done in C++ (which is a painful process). I have lately come across lazarus/free pascal and feel I would be much happier if Pascal would be the front end companion to languageONE. To that end I have ported the 1st part of an application on Linux. It has gone very well, probably better than  expected. My own code is assembled using NASM which facilitates DWARF debugging (embedded in the executable). I use the following to statically link my rear end to the pascal front end and to link the languageONE runtime system:- {$LINK   /home/roger/_dev1/source/languageONE/lazarus- fp/_Application/V3.08.o} {$LINK   /home/roger/_dev1/source/languageONE/lib/languageONE.lib} Again, this has gone very well and the debugging using Lazarus is seamless. The problem comes with Windows. (Doesn't it always). As Microsoft debug with a PDB, after struggling for quite a few versions of languageONE,  I had to duplicate the code using MASM. This combined with the C++ frontend is perfectly debuggable using Visual Studio/Windbg and Visual Studio Code. The problem occurs when I try to use Lazarus, and so to the question.. I cannot link using Lazarus as it complains of COFF Magic Error. (Something that is occassionally discussed but never solved online.) I can however link when I select "Use External Linker", (the -Xe option for Lazarus/FP) and I am taking a guess that it is linking with "ld.exe" which I am assuming is a port of the Linux linker. This is all very well but as Lazarus/FP uses DWARF and my application uses a PDB I can step thru the Pascal Code but NOT thru my own code. That is to be expected, but on building a small piece of code using NASM on Windows and linking that to Lazarus/FP program,  I crash with an access violation. I very much like Lazarus (Very similar to VBA in the way that it works) and Pascal is a far better language than C++ If I had the same experience on Windows as on Linux I would be "over the moon".  although I would have to convert my Windows MASM code back to NASM, but I can live with that. Is anyone able to shed any light on this ? (If I have to duplicate the forntends [C++ Windows/Pascal Linux] I'm not sure I would even bother. (Not like anyone has heard of or uses languageONE) Any help (even just pointing me in the right direction)  would be greatly appreciated. Cheers Roger Lazarus-fp 3.2.2 ---------------------------------------------------------------------- KodeZwerg · 2024-04-29 08:59 On Windows systems the most common way to "talk" with external libraries is by creating a library (.dll) and offer the bindings/headers for Pascal. As a sample you can take a look how the unit Windows does do such so we can "talk" with the Windows API. ---------------------------------------------------------------------- marcov · 2024-04-29 09:33 You get a bit a mix of various tools support various things: Open source compilers and linkers mostly supported stabs and dwarf as debugging format. Free Pascal follows GCC in that sense. With stabs being in decline (dead for a decade and longer on Unix, but held on longer on Windows/dos) The internal FPC linker is developed as a typical backend linker to link what FPC produces, and sometimes has difficulties with special functionality used by other compilers. (invalid coff....) Commercial compilers often implemented support based on Intel Codeview or similar debug formats. Afaik both Delphi's and PDB are codeview based. Besides the formats, the debugger must also implement them. Lazarus, for windows, uses two different debuggers currently, one based on the commandline GDB, and an own called fpdebug, with another layer in lazarus itself to communicate and interpret. Long story short: get on dwarf as much as possible seems to me to be the only viable option if you want to debug both sides (asm and lazarus/fpc generated code) simultaneously. ---------------------------------------------------------------------- roger T · 2024-04-29 10:10 Yeah, that was the way I was leaning but was getting an access violation as soon as a hit the call to my test asm. (Which was only 3 or 4 lines of code just to accommadate  the call). Am I right in saying that ld.exe is Linux port or does lazarus/fp use link.exe externally. I really had the smallest fp program, basically just a call the asm code. Is there any value in trying to link the two with link.exe or is that infact the external linker. Will lazarus/fp actually link with the microsoft linker. ---------------------------------------------------------------------- Zvoni · 2024-04-29 11:36 @roger and change your screen-name in your Profile It shows (your?) EMail-address "Profile - Account settings" Quote Name This is the displayed name that people will see. ---------------------------------------------------------------------- roger T · 2024-04-29 11:54 Thank you. Done ---------------------------------------------------------------------- marcov · 2024-04-29 12:19 Quote from: roger T on April 29, 2024, 12:10:28 pmYeah, that was the way I was leaning but was getting an access violation as soon as a hit the call to my test asm. That could be so many reasons. Keep in mind that FPC defaults to "register" calling convention, which is a kind of Borland version of fastcall (but not the same) QuoteAm I right in saying that ld.exe is Linux port or does lazarus/fp use link.exe externally. GNU LD is default for most targets. Afaik OS X uses LLVM nowadays, and Amiga uses vasm/vlink. On Windows GNU ld is still the back up as external linker (-Xe) after the default internal linker (-Xi), which is a (PE)COFF linker built into the FPC compiler. QuoteIs there any value in trying to link the two with link.exe or is that infact the external linker. Will lazarus/fp actually link with the microsoft linker. Microsoft tools and linkers are not used  at all to my best knowledge. (note also that FPC programs don't link to msvcrt by default, there is no C layer, it is native windows), the external linker is a mingw version of GNU ld.exe in the version delivered with FPC, which is currently 2.28 from 2017.  That is fairly old, but it is only the backup linker, so updating is less of a priority. You could try a newer mingw linker if you have problems. ---------------------------------------------------------------------- roger T · 2024-04-30 06:11 So I have tried a lot of stuff on this but come up just as confused... with the simplest code I could use.... -------------------------------------------- program ptest; {$PIC OFF} {$LINK   /home/roger/_dev1/source/languageONE/lazarus- fp/_Application/TEST.o} procedure _l1(); external; var   A : Int64 = 1;   B : Int64 = 2;   C : Int64 = 3; begin       {$asmMode intel}       asm          movq xmm0,qword[A]         movq xmm1,qword         movq xmm2,qword[C]         call _l1       end; end. ------------------------------------------- and the called assembler file --------------------- [section .data]    D   db 0    E   db 0    F   db 0 [section .text] global _l1 _l1:    movq qword[D],xmm0    movq qword[E],xmm1    movq qword[F],xmm2    ret ------------------------------------------------- runs perfectly on Linux..... but on Windws, the debugger will not enter _l1 it simply step over the code... a dumpbin (extract) on Windows shows opyright (C) Microsoft Corporation.  All rights reserved. Dump of file ptest.exe File Type: EXECUTABLE IMAGE COFF SYMBOL TABLE 000 0000000A DEBUG  notype       Filename     | .file     TEST.ASM 002 00000000 SECT2  notype       Static       | .data     Section length    3, #relocs    0, #linenums    0, checksum      0 004 00000000 SECT1  notype       Static       | .text     Section length   1C, #relocs    3, #linenums    0, checksum      0 006 00000000 ABS    notype       Static       | .absolut 007 00000000 SECT2  notype       Static       | D 008 00000001 SECT2  notype       Static       | E 009 00000002 SECT2  notype       Static       | F 616 0000063A DEBUG  notype       Filename     | .file     ptest.lpr 62A 00000140 SECT2  notype       Static       | .data.n_TC_$P$PTEST_$$_TEST_A     Section length   10, #relocs    0, #linenums    0, checksum      0 62C 00000150 SECT2  notype       Static       | .data.n_TC_$P$PTEST_$$_TEST_B     Section length   10, #relocs    0, #linenums    0, checksum      0 62E 00000160 SECT2  notype       Static       | .data.n_TC_$P$PTEST_$$_TEST_C     Section length   10, #relocs    0, #linenums    0, checksum      0 and......an extract on Linux using nm 0000000000428140 d TC_$P$PTEST_$$_ROGER_A 0000000000428150 d TC_$P$PTEST_$$_ROGER_B 0000000000428160 d TC_$P$PTEST_$$_ROGER_C 0000000000428170 d D 0000000000428171 d E 0000000000428172 d F NOTE:- I renamed the Pascal Int64 variables TEST_A/B/C and ROGER_A/B/C to make them easily identifiable in the dump files. if the external linker on Windows is "ld.exe" then I really don't see the difference between the Linux and theWindows versions of Free Pascal in terms of how they operate... Anyone have any further clues ?? Cheers Roger T ---------------------------------------------------------------------- Martin_fr · 2024-04-30 07:50 Well, first thing is to check if the debug info is there. In the fpc directory is a tool called objdump. It can dump the line info, which is used for stepping. It can also dump the --dwarf=info which among variables also shows you procedures with entry points. So then you will know if there is debug info for the debugger to work with. ---- Next you should tell us if you are trying to debug with gdb or fpdebug. Try both, to see if it differs. Also in project options set debug info to dwarf, rather than automatic. ----- You can also open the asm view and single step asm instructions (for key combos look up the asm view wiki page) In the source editor, the IDE shows small blue dots, if it thinks the debugger knows abount a source line  .... About access violation => only if debugging? There is a known issue in fpc that causes some line info wrong. In cases were that happens, with certain version of gdb, and if a breakpoint is set on such a line, then the breakpoint will be incorrectly placed and cause errors. => try stepping without breakpoint. Or avoid breakpoint in very short procedures, or the last 3 lines of a procedure. ---------------------------------------------------------------------- roger T · 2024-04-30 08:45 You might not of read my thread quite correctly Martin. The dumps that I am showing IS from objdump and I don't have any issues with the Linux version (It works brilliantly). It's Windows that I am struggling with. Cheers Roger ---------------------------------------------------------------------- marcov · 2024-04-30 10:22 Is it 64-bit ? Keep in mind that on 64-bit windows, all variables are RIP relative. ---------------------------------------------------------------------- roger T · 2024-04-30 12:50 Yes, all 64 bit. And I have never coded that way (RIP). I can look at the assembler output from Free Pascal and see what it produces. Why would that make a difference on Windows (My existing code coming out of Masm has always worked okay and debugs happily ) ---------------------------------------------------------------------- marcov · 2024-04-30 13:16 Quote from: roger T on April 30, 2024, 02:50:03 pmYes, all 64 bit. And I have never coded that way (RIP). I can look at the assembler output from Free Pascal and see what it produces. Why would that make a difference on Windows (My existing code coming out of Masm has always worked okay and debugs happily ) Many possibilities: Maybe MASM autocorrects such mistakes by encoding it as RIP relative, or if both sides of the call (variable declaration and usage) are not compliant with the Windows ABI, that it goes undetected. Or the masm code linking process forces a specific loading address that causes absolute addresses to remain working (at the cost of ASLR) ---------------------------------------------------------------------- Martin_fr · 2024-04-30 18:33 Quote from: roger T on April 30, 2024, 10:45:39 amYou might not of read my thread quite correctly Martin. The dumps that I am showing IS from objdump and I don't have any issues with the Linux version (It works brilliantly). It's Windows that I am struggling with. Quote from: roger T on April 30, 2024, 08:11:58 amruns perfectly on Linux..... but on Windws, the debugger will not enter _l1 it simply step over the code... If you are using FpDebug => then that will only use DWARF. If you are using GDB it does use DWARF but can also use STABS. The dumps don't show the relevant dwarf info. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program Project1;procedure Foo;begin  writeln;  writeln;end; begin  Foo;  Foo;end. objdump.exe --dwarf=decodedline project1.exe  Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i CU: project1.lpr:File name                            Line number    Starting addressproject1.lpr                                   3       0x100001600 project1.lpr                                   4       0x10000160dproject1.lpr                                   5         0x100001622project1.lpr                                   6         0x100001637project1.lpr                                   8         0x100001650project1.lpr                                   9         0x10000165eproject1.lpr                                  10         0x100001663project1.lpr                                  11         0x100001668 For deeper looking into it, you may need --dwarf=rawline This is what tells the debugger where the code for a source line is. It is used for stepping and breakpoints (if they are set by source line / as opposed to address in the asm window). Well, on GDB I don't know if there is more info used. On FpDebug => that is it. Well, there is a tiny bit more... - AFAIK, gdb looks at asm instructions to detect prologue and epilogue code => and it may skip such asm instructions. (and maybe more). I don't know the internals of gdb. - FpDebug may make up for missing line info on very small bits of asm (jump pads etc) ---------------------------------------------------------------------- roger T · 2024-05-01 01:29 Many thanks Martin I did infact find the objdump.exe port that came with fpc and learnt the following.. LINUX. ptest:     file format elf64-x86-64 Contents of the .debug_line section: CU: ptest.lpr: File name                            Line number    Starting address  View    Stmt ptest.lpr                                     12            0x401090             x ptest.lpr                                     15            0x401099             x ptest.lpr                                     16            0x4010a2             x ptest.lpr                                     17            0x4010ab             x ptest.lpr                                     18            0x4010b4             x ptest.lpr                                     20            0x4010b9             x ptest.lpr                                     20            0x4010c3             x CU: TEST.ASM: File name                            Line number    Starting address  View    Stmt TEST.ASM                                       9            0x4010d0             x TEST.ASM                                      10            0x4010d9             x TEST.ASM                                      11            0x4010e2             x TEST.ASM                                      12            0x4010eb             x TEST.ASM                                      12            0x4010ec             x WNDOWS ptest.exe:     file format pei-x86-64 Decoded dump of debug contents of section .debug_line: CU: ptest.lpr: File name                            Line number    Starting address ptest.lpr                                     11            0x4019d0 ptest.lpr                                     14            0x4019de ptest.lpr                                     15            0x4019e7 ptest.lpr                                     16            0x4019f0 ptest.lpr                                     17            0x4019f9 ptest.lpr                                     19            0x4019fe So obviously TEST.obj has no line number information. On further investigation NASM will only output CV8 debug info for a win64 which leaves me between a rock and a hard place.. MASM produces PDB NASM produces CV8 FPC produces DWARF I would love to use Lazarus/FP on Windows but it is looking like I may need to stick to FLTK and C++ (groan) Many thanks to everyone that replied. (and if you have any incling of how I could procede here I would be more than pleased to hear it) Cheers Roger EDIT: Delphi also seems to have it's own debugger, ..anyone know what debug format they use ? ---------------------------------------------------------------------- 440bx · 2024-05-01 05:29 Quote from: roger T on May 01, 2024, 03:29:32 amEDIT: Delphi also seems to have it's own debugger, ..anyone know what debug format they use ? They have a proprietary format.  I've seen the format referred to as "TDS" (Turbo Debug Symbols.) There might be some information you can use in this project: https://github.com/MahdiSafsafi/DebugEngine HTH. ---------------------------------------------------------------------- Martin_fr · 2024-05-01 09:31 There is somewhere a project for the (almost) opposite direction: Dwarf to PDB. Then you might be able to debug the FPC parts in a Window based debugger, that can read those debug info? As for Lazarus: It support FpDebug, GDB and LLD as backends. I don't expect that LLDB would be able to read those debug info from nasm. But if it did, use the "lldb ALPHA" backend (without fpdebug). ---------------------------------------------------------------------- 440bx · 2024-05-01 15:31 Quote from: Martin_fr on May 01, 2024, 11:31:35 amThere is somewhere a project for the (almost) opposite direction: Dwarf to PDB. Yes, you might have been thinking about cv2pdb.  I started a thread about it: https://forum.lazarus.freepascal.org/index.php/topic,42000.m sg295128.html#msg295128 It's a very nice to have utility but, unfortunately it leaves out a lot of debugging information in the translation. Still it can be very useful sometimes. ---------------------------------------------------------------------- roger T · 2024-05-02 11:07 I gave cv2pdb a try and another one called map2pdb but had no joy. I've fretted over my choices and I can manage my end (languageONE) by coding calls in the mainline to test the routines in the code that lazarus would call. (It will happily run as an exe and debug it using windbg. ) Then debug the lazarus code and let it just jump over the calls knowing that they are ok. And of course I can watch the returned values knowing what to expect. Worked today to take another look at the Fltk stuff in C++ and can make a decision as to which way to go. Maybe even work with both for awhile. Again, thanks for all the responces and know that I really would prefer the lazarus path. Cheers Roger T