FPC 3.2.2 / Lazarus 2.2.6 patch for GLIBC_2.34 versioning issue lazarus / 32-Linux ---------------------------------------------------------------------- robert rozee · 2023-09-26 13:39 hi,     attached is a patch for FPC 3.2.2 / Lazarus 2.2.6 to enforce glibc BASE symbol versioning. this is 'yet another approach', where the compiler itself is patched to substitute BASE versioned glibc symbols for unversioned glibc symbols. the procedure to apply the patch is as follows: 1. unpack the pdecsub.pas.zip archive, giving you two files: pdecsub.pas rebuild.sh - copy pdecsub.pas to /usr/share/fpcsrc/3.2.2/compiler, overwriting the existing pdecsub.pas file. you will need to do this as root. the replacement file contains modifications to the procedure pd_external() to make symbol substitutions based on a file to be created in your /tmp directory. - copy rebuild.sh to /usr/share/fpcsrc/3.2.2 and flag the file as executable. again, you will need to do this as root. 1A. ADDENDUM: create a symlink from libdl.so to libdl.so.2 using: sudo ln -s /lib/x86_64-linux-gnu/libdl.so.2 /lib/x86_64-linux- gnu/libdl.so (the above symlink is really important to ensure libdl.so.2 is correctly referenced in future ELF headers) 2. unpack the project contained in MakeBase.zip and build using Lazarus. the result is a binary called MakeBase that you can run from anywhere to create a file, /tmp/basesyms.map containing the BASE symbol versioning for all the glibc symbols. the output from MakeBase will look something like: Processing /usr/lib/x86_64-linux-gnu/libc.so.6 2445 Versioned Symbols saved to /tmp/basesyms.map running MakeBase a second time will prompt you to delete /tmp/basesyms.map, but you don't want to do that just yet. once deleted, you can recreate the .map file just by running MakeBase again. 3. once you've run MakeBase and it has created /tmp/basesyms.map change to your fpc source directory, /usr/share/fpcsrc/3.2.2, and type: sudo ./rebuild.sh this will take some time to complete (5 minutes or so for me), and will recompile the compiler and rebuild the FPC RTL and Lazarus LCL. this is the stage at which the BASE symbol translating is applied. 4. if you wish, now run MakeBase a second time to have it delete /tmp/basesyms.map. this file is (almost certainly) no longer needed as everything has already been rebuilt using glibc BASE symbols, but i see no harm in leaving it there. if anyone wishes to tidy this up some more, wrapping it in a simpler to apply installer for example, go for it! it is possible that in later versions of FPC the contents of /usr/share/fpcsrc/3.2.2/compiler/pdecsub.pas may have changed, in which case the edits to the procedure pd_external() will need to be applied to the later versioned file. it is fairly obvious what i have added into the code. btw, the few lines referring to gtk are there to sidestep a horrible hack buried elsewhere in the FPC source where the symbol fnmatch is intercepted and treated in a most unholy manner. i can make no sense of this hack, so instead worked around it as best i could. cheers, rob   :-) ---------------------------------------------------------------------- Bart · 2023-09-26 16:30 Why not make a proper diff file for the changes you made. This way it's easier to spot what you did and also it's easier to apply to e.g. fpc main (trunk). Bart ---------------------------------------------------------------------- robert rozee · 2023-09-26 16:42 excellent idea... can you tell me how to do this? cheers, rob   :-) ---------------------------------------------------------------------- DonAlfredo · 2023-09-26 17:59 The current FPC trunk pdecsub.pas has been changed. So, a patch based on your current pdecsub.pas will revert changes in FPC trunk of the past few weeks. Please tell me the GIT hash of FPC trunk that you have used for this new pdecsub.pas ? ---------------------------------------------------------------------- robert rozee · 2023-09-26 18:21 it is from the 2.2.6 .deb package release located here: https://sourceforge.net/projects/lazarus/files/Lazarus%20Linux%20amd64 %20DEB/Lazarus%202.2.6/ the only code changes are in pd_external(), consisting of ONE clearly marked block of code located two end statements up from the end of the procedure, along with some additional variables and a couple of typed constants declared at the top of pd_external(). cheers, rob   :-) ---------------------------------------------------------------------- TRon · 2023-09-26 20:39 Quote from: robert rozee on September 26, 2023, 06:42:53 pmexcellent idea... can you tell me how to do this? Code: Bash  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i mkdir work && cd workwget --content-disposition 'https://forum.lazarus.freepascal .org/index.php?action=dlattach;topic=64731.0;attach=57536'git clone https://gitlab.com/freepascal.org/fpc/source.git fpc-sourcecd fpc- sourcegit checkout release_3_2_2git switch -c myfixesunzip -j ../pdecsub.pas.zip "pdecsub.pas" -d ./compiler zip will ask you to overwrite existing file: type y(es) or A(ll) + enter to overwrite then continue with: Code: Bash  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i git add -Agit commit -m "my changes"git diff release_3_2_2..myfixes > ../my_awesome_patch.diff Now you have your 'patch' file. To check whether or not it can be applied to master branch, continue above with: Code: Bash  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i git checkout main  Then in order to check whether or not the patch can be applied: Code: Bash  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i patch --dry-run -p1 -i ../my_awesome_patch.diff  actually applying the patch: Code: Bash  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i patch -p1 -i ../my_awesome_patch.diff  And you'll have to commit again for the changes to be applied. From there on you can build trunk compiler with your changes and use/follow your other instructions. I'm sure there is a faster way but since no-one else posted a solution this would suffice. Quote from: DonAlfredo on September 26, 2023, 07:59:50 pmThe current FPC trunk pdecsub.pas has been changed. So, a patch based on your current pdecsub.pas will revert changes in FPC trunk of the past few weeks. Please tell me the GIT hash of FPC trunk that you have used for this new pdecsub.pas ? I did not experience any conflicts. I am aware that that doesn't necessarily means that the patch will work or does not mess up other things but that is for the person that patches to figure out ---------------------------------------------------------------------- DonAlfredo · 2023-09-27 09:07 Based on your work, I have tried another approach. The inc file, that has to be included in the compiler sources, holds the function mapping towards the oldest available GLIBC-version. The patch file adds compiler option -XLC (needs a compiler rebuild naturally, with this patch). If FPC is build with -XLC, and your app also, the linking will be done towards the GLIBC-version as defined by the include file. If your app is build with FPC option -XLC, the linking will be done towards the GLIBC-version as defined by the include file. Might need some testing ... ;-) ---------------------------------------------------------------------- DonAlfredo · 2023-09-27 12:19 To @myself and @other interested. Included a better and more complete patch for versioned linking using the new -XLC compiler option. ---------------------------------------------------------------------- robert rozee · 2023-09-27 23:31 Quote from: DonAlfredo on September 27, 2023, 02:19:39 pmTo @myself and @other interested. Included a better and more complete patch for versioned linking using the new -XLC compiler option. nice work Don! i like the addition of a compiler switch to control things. cheers, rob   :-) ---------------------------------------------------------------------- WayneSherman · 2023-10-06 21:37 Quote from: DonAlfredo on September 27, 2023, 02:19:39 pmTo @myself and @other interested. Included a better and more complete patch for versioned linking using the new -XLC compiler option. Thank you guys for your continued work on this issue. The code which adds a version to the function import name checks for a match with a GLIBC function name: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i if sym_name {import_name} = gf.Name {GLIBCFUNCTIONS} then Some examples of GLIBC function names are: abs, abort, floor, malloc, open... What if other libraries that are not GBLIC related happen to use those names for their exported functions?  Will these non-GLIBC function import declarations get a version suffix and cause the linking to fail? I think it would be prudent to provide a way to bypass this substitution for specific import declarations.  For example what if I want to declare an imported function called "dlopen" but do NOT want a version suffix added? e.g. if I use the "@" symbol and provide no version it will skip the substitution code, but that will probably cause an error compiling or linking: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i function dlopen(__file:Pchar; __mode:longint):pointer;cdecl;external dllib name 'dlopen@'; //@ to skip versioning ---------------------------------------------------------------------- robert rozee · 2023-10-07 03:48 Quote from: WayneSherman on October 06, 2023, 11:37:44 pmWhat if other libraries that are not GBLIC related happen to use those names for their exported functions? the technical term for this situation is a "clusterf**k". while there is no hard mechanism or rule preventing multiple libraries containing a function/symbol with the same name, doing so severely breaks linux. the glibc authors are well aware of this and warn against ever duplicating any function names across different libraries. an example that flies oh-so-close-to breaking this warning is the function fnmatch, which exists in glibc and also sort-of-exists (or once existed) in gtk. the code to get around this conflict in FPC is icky. not preventing the use of duplicate function names seems to be a fundamental design flaw in linux (and other unix'es). the use of versioned symbols (eg, dlopen@GLIBC_2.2.5) gets around it by appending a library-specific suffix, but as we have seen introduces a whole new layer of suffering in exchange. in FPC the actual breaking occurs when you use external to access a library function; the function name is referenced in the ELF header's Dynamic Symbol Table, while the library name is referenced in a 'DT_NEEDED' section that can be found by grepping the output of readelf -a for '(NEEDED)' . the kicker here is that, as far as i can determine, no linkage is recorded between the function name and the library name specified as parameters to external. put simply, the first occurrence of the function name that is found anywhere at run- time is linked to by ld, the loader. this in itself is good enough reason why external should be used sparingly and with extreme caution. an alternative mechanism using dlopen et al exists that does ties together function name and library name. cheers, rob   :-) ---------------------------------------------------------------------- marcov · 2023-10-07 10:44 To my best knowledge, FPC doesn't export FNMatch with a unmangled symbol? There are many simple words that are duplicates, like read and write etc, but Pascal namespace mangling keeps that from C. ---------------------------------------------------------------------- robert rozee · 2023-10-07 12:43 Quote from: marcov on October 07, 2023, 12:44:14 pmTo my best knowledge, FPC doesn't export FNMatch with a unmangled symbol? There are many simple words that are duplicates, like read and write etc, but Pascal namespace mangling keeps that from C. Code: Bash  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i user@DH61BE:/$ user@DH61BE:/$ grep -r -i "function fnmatch" /usr/share/fpcsrc/usr/share/fpcsrc/3.2.2/rtl/unix/dos.pp:Function FNMatch(const Pattern,Name:string):Boolean;/usr/share/fpcsrc/3.2.2/rtl /unix/sysutils.pp:Function FNMatch(const Pattern,Name:string):Boolean;/usr/share/fpcsrc/3.2.2/rtl/macos/dos.pp:  function FNMatch (const Pattern, Name: string): Boolean;/usr/share/fpcsrc/3.2.2/rtl/netwlibc/libc.pp:function fnmatch(pattern, _string:Pchar; flags:longint):longint;cdecl;external libc_nlm name 'fnmatch';/usr/share/fpcsrc/3.2.2/rtl/nativent/sysutils.pp:Function FNMatch(const Pattern,Name:UnicodeString):Boolean;/usr/share/fpcsrc/3. 2.2/packages/unixutil/src/unixutils.pp:Function FNMatch(Const Pattern,Name : String; Flags : TFnmFlags) : Boolean;/usr/share/fpcsrc/ 3.2.2/packages/gtk2/src/gtk+/gtk/fnmatch.inc:function fnmatch(__pattern: char; __string: char;/usr/share/fpcsrc/3.2.2/packages/libc/src/fnmatchh.inc:function fnmatch(__pattern:Pchar; __name:Pchar; __flags:longint):longint;cdecl;external clib name 'fnmatch';user@DH61BE:/$ user@DH61BE:/$ user@DH61BE:/$ grep -r -i "function fnmatch" /usr/share/fpcsrc/3.2.2/packages/gtk2/src/gtk+/gtk/fnmatch.inc function fnmatch(__pattern: char; __string: char;user@DH61BE:/$ fnmatch is passed as a parameter to pd_external() (that resides in the file pdecsub.pas), and is accompanied by the library name libgtk-x11-2.0.so. if the symbol is versioned to fnmatch@GLIBC_2.2.5 then any attempt to compile a GUI application fails, while non-GUI applications compile fine. suppressing versioning of fnmatch, by excluding any substitution when a library name containing the substring "gtk" is encountered, resolves this problem. the constant containing the string "libgtk-x11-2.0.so" is defined in the file /usr/share/fpcsrc/3.2.2/packages/gtk2/src/gtk+/gtk/gtk2.pas: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i       gtklib = 'libgtk-x11-2.0.so'; and grepping for "gtklib" results in: Code: Bash  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i user@DH61BE:/$ user@DH61BE:/$ grep -r -i -B2 "gtklib" /usr/share/fpcsrc | grep "function fnmatch"/usr/share/fpcsrc/3.2.2/pac kages/gtk2/src/gtk+/gtk/fnmatch.inc:  __flags: gint): gint; cdecl; external gtklib;user@DH61BE:/$ with the full function declaration, at line 36, being: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i function fnmatch(__pattern: char; __string: char;  __flags: gint): gint; cdecl; external gtklib; this is as far as i drilled down into the source code, at that point deciding i had stumbled upon something ancient and evil. cheers, rob   :-) ---------------------------------------------------------------------- WayneSherman · 2023-10-07 12:47 Quote from: robert rozee on October 07, 2023, 05:48:43 amQuote from: WayneSherman on October 06, 2023, 11:37:44 pmWhat if other libraries that are not GBLIC related happen to use those names for their exported functions? ... while there is no hard mechanism or rule preventing multiple libraries containing a function/symbol with the same name, doing so severely breaks linux. the glibc authors are well aware of this and warn against ever duplicating any function names across different libraries. ...is good enough reason why external should be used sparingly and with extreme caution Point well taken.  More reason to have fine grained control over the name substitution.  In addition to (or instead of?) a command line switch, a compiler directive would provide more control and allow one to enable and disable this feature on a unit or function level: Command Line:  -XLC = Enable GLIBC external symbol name base versioning Compiler Directive:  $GLIBC_Symbol_Versioning Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i {$GLIBC_Symbol_Versioning OFF}  function dlopen(__file:Pchar; __mode:longint):pointer;cdecl;external dllib name 'dlopen';{$GLIBC_Symbol_Versioning ON} //using your recent example:{$GLIBC_Symbol_Versioning OFF}  function fnmatch(__pattern: char; __string: char; __flags: gint): gint; cdecl; external gtklib;{$GLIBC_Symbol_Versioning ON} ---------------------------------------------------------------------- WayneSherman · 2023-10-07 13:30 Quote from: WayneSherman on October 07, 2023, 02:47:14 pm...fine grained control over the name substitution. ...a compiler directive would provide more control and allow one to enable and disable this feature on a unit or function level or even better in my opinion is some kind of special syntax that is self-documenting at the point of usage: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i   function dlopen(__file:Pchar; __mode:longint):pointer;cdecl;external dllib name 'dlopen@%GLIBC_BASE%'; ---------------------------------------------------------------------- marcov · 2023-10-07 14:22 Quote from: robert rozee on October 07, 2023, 02:43:35 pmQuote from: marcov on October 07, 2023, 12:44:14 pmTo my best knowledge, FPC doesn't export FNMatch with a unmangled symbol? There are many simple words that are duplicates, like read and write etc, but Pascal namespace mangling keeps that from C. /usr/share/fpcsrc/3.2.2/rtl/netwlibc/libc.pp:function fnmatch(pattern, _string:Pchar; flags:longint):longint;cdecl;external libc_nlm name 'fnmatch'; /usr/share/fpcsrc/3.2.2/packages/gtk2/src/gtk+/gtk/fnmatch.inc:functio n fnmatch(__pattern: char; __string: char; /usr/share/fpcsrc/3.2.2/packages/libc/src/fnmatchh.inc:function fnmatch(__pattern:Pchar; __name:Pchar; __flags:longint):longint;cdecl;external clib name 'fnmatch'; Ok so, gtk2, legacy Kylix libc package and netware (which is also not actively supported). But while those are mostly dead, gtk2 is still alive, so it might be useful to find out what this is for  Quotefnmatch is passed as a parameter to pd_external() (that resides in the file pdecsub.pas), and is accompanied by the library name libgtk-x11-2.0.so. if the symbol is versioned to fnmatch@GLIBC_2.2.5 then any attempt to compile a GUI application fails, while non-GUI applications compile fine. suppressing versioning of fnmatch, by excluding any substitution when a library name containing the substring "gtk" is encountered, resolves this problem. So it is not a problem that exists now, but a problem of your versioning patch, that must distinguish between version symbol in libc, and an unversioned in gtk-X11 lib ? But even then it should only a problem if something actually _calls_ fnmatch from GTK headers and the symbol is  bound? Quotethis is as far as i drilled down into the source code, at that point deciding i had stumbled upon something ancient and evil. Yes could be that this is some GTK compatibility wrapper for non *nix platforms like Windows. ---------------------------------------------------------------------- robert rozee · 2023-10-07 15:00 Quote from: marcov on October 07, 2023, 04:22:10 pmSo it is not a problem that exists now, but a problem of your versioning patch, that must distinguish between version symbol in libc, and an unversioned in gtk-X11 lib ? nope - the problem very much exists now. Code: Bash  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i user@DH61BE:/$ user@DH61BE:/$ find -name "libgtk-x11-2.0.so*" 2>/dev/null./usr/lib/i386-linux- gnu/libgtk-x11-2.0.so.0.2400.33./usr/lib/i386-linux- gnu/libgtk-x11-2.0.so.0./usr/lib/x86_64-linux- gnu/libgtk-x11-2.0.so./usr/lib/x86_64-linux- gnu/libgtk-x11-2.0.so.0.2400.33./usr/lib/x86_64-linux- gnu/libgtk-x11-2.0.so.0[ ... ]user@DH61BE:/$ gives us the location of the GTK library. then searching it with readelf gives us... Code: Bash  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i user@DH61BE:/$ user@DH61BE:/$ readelf --dyn-syms -W /usr/lib/x86_64-linux-gnu/libgtk-x11-2.0.so | grep "fnmatch"user@DH61BE:/$ ... absolutely nothing! so where does the function fnmatch reside?! we know it isn't in glibc - if it were then versioning it to fnmatch@GLIBC_2.2.5 wouldn't have been a problem. where is the code for fnmatch coming from??? cheers, rob   :-) ---------------------------------------------------------------------- marcov · 2023-10-07 20:48 Those headers are from the early GTK2 days, patched several times for later (2.8, 2.10)  versions. Also possibly it depends on the target if the symbol is there, and then it probably wouldn't be on glibc systems, as it is in libc there. ---------------------------------------------------------------------- robert rozee · 2023-10-08 01:16 Quote from: marcov on October 07, 2023, 10:48:50 pmThose headers are from the early GTK2 days, patched several times for later (2.8, 2.10)  versions. Also possibly it depends on the target if the symbol is there, and then it probably wouldn't be on glibc systems, as it is in libc there. that isn't answering the question - where is the code for fnmatch coming from??? 64-bit linux, FPC 3.2.2, Lazarus 2.2.6. it clearly lands somewhere, and is being used for something. is it the case that nobody knows what code is being executed? cheers, rob   :-) ---------------------------------------------------------------------- marcov · 2023-10-08 08:50 Quote from: robert rozee on October 08, 2023, 03:16:34 amthat isn't answering the question - where is the code for fnmatch coming from??? 64-bit linux, FPC 3.2.2, Lazarus 2.2.6. it clearly lands somewhere, and is being used for something. is it the case that nobody knows what code is being executed? Well, you can simply comment the declaration and see where the build fails yourself? ---------------------------------------------------------------------- robert rozee · 2023-10-08 09:16 Quote from: marcov on October 08, 2023, 10:50:14 amWell, you can simply comment the declaration and see where the build fails yourself? the downside of making such modifications to the compiler myself (and here i include the build process as well) is that the compiler is used to compile itself. to date i've been proceeding with extreme care, wary that if i break it the subsequent 'unbreaking' might be less than trivial! FPC needs a working FPC to compile it. consider: an innocent looking symbol (fnmatch) being associated with a library that does not contain it. yet nothing crashes, no errors are reported, but tinker with it and the compiler no longer functions. and nobody know what the symbol is there for, where exactly it is used... ... sounds rather like a subtle backdoor put in place years ago, waiting for someone to quietly drop in a library that handles fnmatch elsewhere in the operating system to exploit said backdoor. i am still waiting for the FPC developers to come up with some sort of explanation, as to be honest i have very little understanding of the FPC source code and how it all works myself. most of what i have done thus far has been based upon educated guesses. whereas, the FPC developers are the experts whom i and others depend upon. cheers, rob   :-) ---------------------------------------------------------------------- marcov · 2023-10-08 09:26 Never attribute to malice which can be explained by other reasons and all that. I gave your reasons (legacy, not correctly translating the original header ifdef structure), you chose to ignore it. First, the compiler does not depend on GTK (or any other packages/ except the build tool), stronger even, it doesn't link to any dynamic library at all, which is easily verifiable, so there is no possible reason for paranoia there. The only thing that uses gtk2/ is Lazarus when compiled from GTK. Even if you wanted to backdoor lazarus you'd do it in place that backdoors all lazarus, not just the GTK port. So let's start with any form of proof that it is actually used. If you don't want to do a test compile, can you see in a compiled Lazarus binary that it uses the symbol with nm? ---------------------------------------------------------------------- robert rozee · 2023-10-08 09:39 Quote from: marcov on October 08, 2023, 11:26:32 amFirst, the compiler does not depend on GTK (or any other packages/ except the build tool), stronger even, it doesn't link to any dynamic library at all, which is easily verifiable, so there is no possible reason for paranoia there. if fnmatch is versioned to fnmatch@GLIBC_2.2.5 and FPC/Lazarus is rebuilt using the script provided by Fred vS, then lazbuild is no longer able to compile any GUI project. lazbuild is a command-line tool that does not invoke the lazarus IDE or any other GUI things, yet seems dependant upon the non-GLIBC version of fnmatch that is currently being associated in the source code with libgtk-x11-2.0.so . can someone please explain (1) where this fnmatch is being used from, and (2) where it is landing? i really don't have the expertise to investigate much further. from my perspective, the source is opaque and unauditable. cheers, rob   :-) ---------------------------------------------------------------------- robert rozee · 2023-10-08 10:44 Quote from: marcov on October 08, 2023, 11:26:32 amNever attribute to malice which can be explained by other reasons and all that. i think you are trying to quote Hanlon's razor, an adage or rule of thumb that states: "Never attribute to malice that which is adequately explained by stupidity." (https://en.wikipedia.org/wiki/Hanlon%27s_razor) personally i would rather refrain from describing the authors of FPC as stupid, it is just not polite talk. cheers, rob   :-) ---------------------------------------------------------------------- DonAlfredo · 2023-10-08 11:21 To my best of knowledge, GTK-fnmatch was used/introduced by Lazarus during the time when GLIBC-fnmatch did not handle the utf8 well. ---------------------------------------------------------------------- marcov · 2023-10-08 11:44 Quote from: robert rozee on October 08, 2023, 11:39:55 amQuote from: marcov on October 08, 2023, 11:26:32 amFirst, the compiler does not depend on GTK (or any other packages/ except the build tool), stronger even, it doesn't link to any dynamic library at all, which is easily verifiable, so there is no possible reason for paranoia there. if fnmatch is versioned to fnmatch@GLIBC_2.2.5 and FPC/Lazarus is rebuilt using the script provided by Fred vS, then lazbuild is no longer able to compile any GUI project. And that is not what I asked. I asked that if you do a plain build, if you can then see a link to fnmatch in the generated binary? I deliberately didn't want proofs using modified code to avoid discussion about if it is a problem with modification or abnormal instrumentation. E.g. it could be that the versioning of symbols and/or its scripts somehow frustrates removing unused imports from the binary leading to an error on a symbol that is usually smartlinked out But yes, the best solution would be now that FPC has been unicode safe for 8 years to clean this out in general. But I did quick greps in the ide/ components/ and lcl/ dirs and can't find any usage.  As it is widgetset dependent one would expect it to be in lcl/ ---------------------------------------------------------------------- robert rozee · 2023-10-08 12:17 Quote from: marcov on October 08, 2023, 01:44:12 pmAnd that is not what I asked. I asked that if you do a plain build, if you can then see a link to fnmatch in the generated binary i'm not quite sure what you are asking. the function fnmatch is used by the compiler or one of the associated utilities that are regenerated by running Fred vS's build script. if i use Fred's script to build the toolchain using fnmatch@GLIBC_2.2.5, the resulting toolchain behaves thus: 1. if i build an arbitrary non-GUI project using lazbuild, it builds successfully. 2. if i try to build an arbitrary GUI project using lazbuild, it will fail to build. put another way, the non-GLIBC fnmatch is used somewhere in the toolchain. strangely, it is used only in places associated with creating/compiling GUI components; non-GUI projects can still be compiled. with a clean unadulterated install of lazarus/FPC, when an arbitrary GUI project is built using lazbuild the resulting ELF binary contains no references to fnmatch in the Dynamic Symbol Table. however, it remains that even when building an unadulterated toolchain, pd_external has the symbol fnmatch passing through it, masked in such a way that it never resolves to libc.so. cheers, rob   :-) ---------------------------------------------------------------------- TRon · 2023-10-08 12:54 @robert: Being thick today, sorry for that. Could you help me out and tell me exactly what readelf command (and its parameters) I need to run for every elf file located in my FPC and Lazarus directories ? Right now, in my results, I see none that seem to match your criteria so am afraid that I am doing something wrong. All I got so far from the thread is symbol fnmatch. ---------------------------------------------------------------------- robert rozee · 2023-10-08 13:22 hi Ron   :-) Code: Bash  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i readelf --dyn-syms -W filename | grep -i "fnmatch"where filename is the ELF binary you want to check, and fnmatch is the symbol you are looking for. the -i switch tells grep to ignore case. if no matching lines are found there will be no output. to test, try: Code: Bash  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i readelf --dyn-syms -W /usr/bin/bwbasic | grep -i "GLIBC"to see all symbols containing GLIBC in the binary /use/bin/bwbasic. btw, you may find some of the FPC binary files are stashed in /usr/bin and other places. cheers, rob   :-) ---------------------------------------------------------------------- TRon · 2023-10-08 13:52 Quote from: robert rozee on October 08, 2023, 03:22:39 pmCode: Bash  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i readelf --dyn-syms -W filename | grep -i "fnmatch"Ah ok. perfect. In that case I indeed did something wrong earlier. Thank you for the help. Ended up writing my own findelfsymbols utility, as that proofs to be a lot easier than all that bashing  Thank you for the example (though I had to use bwrap because I do not seem to have bwbasic installed. Seem to provide similar results though (At least some output that I was able to test against) Quotebtw, you may find some of the FPC binary files are stashed in /usr/bin and other places. If that would to be the case then I did something horribly wrong  For me it is (always something like) /home/apps/fpc/x.x.x and /home/apps/lazarus/x.x.x where x.x.x stands for the version number of the corresponding software. Alas... I checked all files that have an elf header against having a symbol fnmatch (ignoring case in the readelf output) but unfortunately none of them was a match for me. fwiw: Linux 64 gtk2, FPC 3.2.2 and Lazarus 3.0RC1. Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i $ ./findelfsymbols -r "/home/apps/fpc/3.2.2/*" fnmatchProcessed 34456 files in 2366 directories, wasting 8.615 seconds. 7493 of those files were ELF files containing a total of 142 (dynamic) symbols for which 0 matched search criteria "fnmatch" $ ./findelfsymbols -r "/home/apps/fpc/3.3.1/*" fnmatchProcessed 18699 files in 1831 directories, wasting 3.286 seconds. 2781 of those files were ELF files containing a total of 187 (dynamic) symbols for which 0 matched search criteria "fnmatch" $ ./findelfsymbols -r "/home/apps/lazarus/2.2.4/*" fnmatchProcessed 18014 files in 1080 directories, wasting 8.148 seconds. 2055 of those files were ELF files containing a total of 5461 (dynamic) symbols for which 0 matched search criteria "fnmatch" $ ./findelfsymbols -r "/home/apps/lazarus/2.2.6/*" fnmatchProcessed 17256 files in 1053 directories, wasting 6.975 seconds. 1540 of those files were ELF files containing a total of 5461 (dynamic) symbols for which 0 matched search criteria "fnmatch" $ ./findelfsymbols -r "/home/apps/lazarus/2.3.0/*" fnmatchProcessed 20332 files in 1094 directories, wasting 1.993 seconds. 1600 of those files were ELF files containing a total of 5385 (dynamic) symbols for which 0 matched search criteria "fnmatch" $ ./findelfsymbols -r  "/usr/lib/x86_64-linux-gnu/libgtk-x11*" fnmatchProcessed 7385 files in 835 directories, wasting 0.098 seconds. 1 of those files were ELF files containing a total of 5635 (dynamic) symbols for which 0 matched search criteria "fnmatch"  $ ./findelfsymbols -r "/home/apps/fpc/3.2.2/*" glibcProcessed 34456 files in 2366 directories, wasting 8.595 seconds. 7493 of those files were ELF files containing a total of 142 (dynamic) symbols for which 111 matched search criteria "glibc" $ ./findelfsymbols -r "/home/apps/fpc/3.3.1/*" glibcProcessed 18699 files in 1831 directories, wasting 3.284 seconds. 2781 of those files were ELF files containing a total of 187 (dynamic) symbols for which 147 matched search criteria "glibc" $ ./findelfsymbols -r "/home/apps/lazarus/2.2.4/*" glibcProcessed 18014 files in 1080 directories, wasting 2.532 seconds. 2055 of those files were ELF files containing a total of 5461 (dynamic) symbols for which 234 matched search criteria "glibc" $ ./findelfsymbols -r "/home/apps/lazarus/2.2.6/*" glibcProcessed 17256 files in 1053 directories, wasting 1.936 seconds. 1540 of those files were ELF files containing a total of 5461 (dynamic) symbols for which 234 matched search criteria "glibc" $ ./findelfsymbols -r "/home/apps/lazarus/2.3.0/*" glibcProcessed 20332 files in 1094 directories, wasting 2.017 seconds. 1600 of those files were ELF files containing a total of 5385 (dynamic) symbols for which 218 matched search criteria "glibc" $ ./findelfsymbols -r  "/usr/lib/x86_64-linux-gnu/libgtk-x11*" glibcProcessed 7385 files in 835 directories, wasting 0.096 seconds. 1 of those files were ELF files containing a total of 5635 (dynamic) symbols for which 75 matched search criteria "glibc" ---------------------------------------------------------------------- robert rozee · 2023-10-14 11:31 i think i've eliminated the versions of fnmatch in /usr/share/fpcsrc/3.2.2/packages/unixutil/src/unixutils.pp and /usr/share/fpcsrc/3.2.2/rtl/unix/sysutils.pp while there is a fair bit of traffic passing through the fnmatch function in sysutils, the point at which a project build fails appears to be elsewhere; doing a compare of trace logs of the function shows identical input and output prior to the point of failure. the code added to the fnmatch function to capture the trace was: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i { assumes that pattern and name have the same code page }Function FNMatch(const Pattern,Name:string):Boolean;//////////// internal functions clipped out, see fnmatch in file /usr/share/fpcsrc/3.2.2/rtl/unix/sysutils.pp for full code//////////var S:string[64];    T:text;    B:boolean; Begin {start FNMatch}////////////////////////////////////// ///////////////////////////////////////            S:='/tmp/matches.txt';                             Assign(T, S);                                          if not FileExists(S) then Rewrite(T)                                 else Append(T);//////////// /////////////////////////////////////////////////////////////////  LenPat:=Length(Pattern);  LenName:=Length(Name);//FNMatch:=DoFNMatch(1,1);            ////////// ///////////////////////////////////////////////////////////////////           B:=DoFNMatch(1,1);            FNMatch:=B;            if B then S:='MATCH'                 else S:='false';            writeln(T, paramstr(0), ' : ', pattern, ' - ', name, ' = ', S);            Close( T);/////////////////////////////////////////////////////////////////// ////////// End; btw, below is the output of lazbuild when building a project (with fnmatch versioned to fnmatch@GLIBC_2.2.5) fails. things go off the rails at around line 67/68: Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i user@Mint21:~/Desktop/clock2$ lazbuild project1.lpr --verbose --verbose --verboseHint: (lazarus) primary config path: /home/user/.lazarusHint: [TPCTargetConfigCache.NeedsUpdate] TargetOS="" TargetCPU="" Options="" compiler file changed "/usr/bin/fpc" FileAge=1697277096 StoredAge=1695728491Hint: [TPCTargetConfigCache.NeedsUpdate] /usr/bin/fpc TargetOS= TargetCPU= CompilerOptions= ExtraOptions= PATH=/usr/local/sbin:/usr/local/bin:/us r/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/binHint: (lazarus) [RunTool] "/usr/bin/fpc" "-iWTOTP"Hint: (lazarus) [RunTool] "/usr/bin/fpc" "-va" "compilertest.pas"Hint: [TPCTargetConfigCache.Update] has changedHint: [TPCTargetConfigCache.NeedsUpdate] TargetOS="linux" TargetCPU="x86_64" Options="" compiler file changed "/usr/bin/fpc" FileAge=1697277096 StoredAge=1695728491Hint: [TPCTargetConfigCache.NeedsUpdate] /usr/bin/fpc TargetOS=linux TargetCPU=x86_64 CompilerOptions= ExtraOptions= PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/ sbin:/bin:/usr/games:/usr/local/games:/snap/binHint: (lazarus) [RunTool] "/usr/bin/fpc" "-iWTOTP" "-Px86_64" "-Tlinux"Hint: (lazarus) [RunTool] "/usr/bin/fpc" "-va" "compilertest.pas" "-Px86_64" "-Tlinux"Hint: [TPCTargetConfigCache.Update] has changedHint: (lazarus) [TBuildManager.SetBuildTarget] OS=linux CPU=x86_64 CompQueryOptions= DefaultOS=linux DefaultCPU=x86_64Hint: (lazarus) [TBuildManager.SetBuildTarget] Old=x86_64-linux-gtk2 New=x86_64-linux- gtk2 Changed: OS/CPU=True LCL=FalseHint: (lazarus) [TBuildManager.SetBuildTarget] OS=linux CPU=x86_64 CompQueryOptions= DefaultOS=linux DefaultCPU=x86_64Hint: (lazarus) [TBuildManager.SetBuildTarget] Old=x86_64-linux-gtk2 New=x86_64-linux- gtk2 Changed: OS/CPU=True LCL=FalseHint: (lazarus) package FCL 1.0.1 was compiled via "make" with parameters " -Fu.  -MObjFPC -Scghi -O1 -g -gl -l -vewnhibq fcllaz.pas"Hint: (lazarus) package LazUtils 1.0 was compiled via "make" with parameters " -Fu../../packager/units/x86_64-linux;.  -MObjFPC -Scghi -O1 -g -gl -l -vewnhibq lazutils.pas"Hint: (lazarus) package freetypelaz 1.0 was compiled via "make" with parameters " -Fu../../packager/units/x86_64-linux;../lazutils/lib/x86_64-linux;.  -MObjFPC -Scghi -O1 -g -gl -l -vewnhibq freetypelaz.pas"Hint: (lazarus) package LCLBase 2.2.6 was compiled via "make" with parameters " -Fuforms;widgetset;nonwin32;../packager/units/x86_64- linux;../components/lazutils/lib/x86_64- linux;../components/freetype/lib/x86_64-linux;. -Fiinclude  -MObjFPC -Scghi -O1 -g -gl -vew -l alllclunits.pp"Hint: (lazarus) package LCL 2.2.6 was compiled via "make" with parameters " -MObjFPC -Scghi -O1 -g -gl -vewnhi -l -Fugtk2;../../packager/units/x86_64- linux;../../components/lazutils/lib/x86_64- linux;../../components/freetype/lib/x86_64-linux;../units/x86_64- linux;. -Figtk2 lcl.pas"Hint: (lazarus) Project needs building: Compiler file changed for Project  File="/usr/bin/fpc"TCompiler.Compile WorkingDir="/home/user/Desktop/clock2/" CompilerFilename="/usr/bin/fpc" CompilerParams=" -MObjFPC -Scghi -CX -Cg -O3 -XX -l -vewnhibq -Fi/home/user/Desktop/clock2/lib/x86_64-linux -Fu/usr/share/lazarus/2.2.6/lcl/units/x86_64-linux/gtk2 -Fu/usr/share/lazarus/2.2.6/lcl/units/x86_64-linux -Fu/usr/share/lazarus/2.2.6/components/freetype/lib/x86_64-linux -Fu/usr/share/lazarus/2.2.6/components/lazutils/lib/x86_64-linux -Fu/usr/share/lazarus/2.2.6/packager/units/x86_64-linux -Fu/home/user/Desktop/clock2/ -FU/home/user/Desktop/clock2/lib/x86_64-linux/ -FE/home/user/Desktop/clock2/ -o/home/user/Desktop/clock2/project1 -dLCL -dLCLgtk2 project1.lpr"[TCompiler.Compile] CmdLine="/usr/bin/fpc -B  -MObjFPC -Scghi -CX -Cg -O3 -XX -l -vewnhibq -Fi/home/user/Desktop/clock2/lib/x86_64-linux -Fu/usr/share/lazarus/2.2.6/lcl/units/x86_64-linux/gtk2 -Fu/usr/share/lazarus/2.2.6/lcl/units/x86_64-linux -Fu/usr/share/lazarus/2.2.6/components/freetype/lib/x86_64-linux -Fu/usr/share/lazarus/2.2.6/components/lazutils/lib/x86_64-linux -Fu/usr/share/lazarus/2.2.6/packager/units/x86_64-linux -Fu/home/user/Desktop/clock2/ -FU/home/user/Desktop/clock2/lib/x86_64-linux/ -FE/home/user/Desktop/clock2/ -o/home/user/Desktop/clock2/project1 -dLCL -dLCLgtk2 project1.lpr"Hint: (lazarus) TBuildManager.MacroFuncInstantFPCCache /usr/bin/instantfpcHint: (lazarus) [RunTool] "/usr/bin/instantfpc" "--get-cache"Hint: (lazarus) [TBuildManager.MacroFuncInstantFPCCache] /home/user/.cache/instantfpc/Info: (lazarus) Execute Title="Compile Project, Target: project1"Info: (lazarus) Working Directory="/home/user/Desktop/clock2/"Info: (lazarus) Executable="/usr/bin/fpc"Info: (lazarus) Param[0]="-B"Info: (lazarus) Param[1]="-MObjFPC"Info: (lazarus) Param[2]="-Scghi"Info: (lazarus) Param[3]="-CX"Info: (lazarus) Param[4]="-Cg"Info: (lazarus) Param[5]="-O3"Info: (lazarus) Param[6]="-XX"Info: (lazarus) Param[7]="-l"Info: (lazarus) Param[8]="-vewnhibq"Info: (lazarus) Param[9]="-Fi/home/user/Desktop/clock2/lib/x86_64-linux"Info: (lazarus) Param[10]="-Fu/usr/share/lazarus/2.2.6/lcl/units/x86_64- linux/gtk2"Info: (lazarus) Param[11]="-Fu/usr/share/lazarus/2.2.6/lcl/units/x86_64-linux"Info: (lazarus) Param[12]="- Fu/usr/share/lazarus/2.2.6/components/freetype/lib/x86_64-linux"Info: (lazarus) Param[13]="- Fu/usr/share/lazarus/2.2.6/components/lazutils/lib/x86_64-linux"Info: (lazarus) Param[14]="- Fu/usr/share/lazarus/2.2.6/packager/units/x86_64-linux"Info: (lazarus) Param[15]="-Fu/home/user/Desktop/clock2/"Info: (lazarus) Param[16]="-FU/home/user/Desktop/clock2/lib/x86_64-linux/"Info: (lazarus) Param[17]="-FE/home/user/Desktop/clock2/"Info: (lazarus) Param[18]="-o/home/user/Desktop/clock2/project1"Info: (lazarus) Param[19]="-dLCL"Info: (lazarus) Param[20]="-dLCLgtk2"Info: (lazarus) Param[21]="project1.lpr"TExternalToolsConsole.HandleMesages: Calling CheckSynchronize!TExternalToolsConsole.HandleMesages: Calling CheckSynchronize!TExternalToolsConsole.HandleMesages: Calling CheckSynchronize!TExternalToolsConsole.HandleMesages: Calling CheckSynchronize!TExternalToolsConsole.HandleMesages: Calling CheckSynchronize!TExternalToolsConsole.HandleMesages: Calling CheckSynchronize!TExternalToolsConsole.HandleMesages: Calling CheckSynchronize!Hint: (11030) Start of reading config file /etc/fpc.cfgHint: (11031) End of reading config file /etc/fpc.cfgFree Pascal Compiler version 3.2.2 [2023/10/14] for x86_64Copyright (c) 1993-2021 by Florian Klaempfl and others(1002) Target OS: Linux for x86-64(3104) Compiling project1.lpr(10001) PPU Loading /usr/share/laza rus/2.2.6/lcl/units/x86_64-linux/gtk2/gtk2int.ppu(10011) PPU Source: gtk2int.pas not found(10011) PPU Source: gtk2defines.inc not available(10011) PPU Source: gtk2winapih.inc not available(10011) PPU Source: gtk2lclintfh.inc not available(10011) PPU Source: gtk2listslh.inc not available(10011) PPU Source: gtk2widgetset.inc not available(10011) PPU Source: gtk2winapi.inc not available(10011) PPU Source: gtk2lclintf.inc not available(10011) PPU Source: gtk2listsl.inc not available(10028) Recompiling Gtk2Int, checksum changed for /usr/lib/fpc/3.2.2/units/x86_64- linux/gtk2/gtk2.ppu/usr/share/lazarus/2.2.6/lcl/units/x86_64- linux/gtk2/gtk2int.ppu:gtk2int.pas(14,36) Fatal: (10022) Can't find unit Gtk2Int used by InterfacesFatal: (1018) Compilation abortedError: /usr/bin/ppcx64 returned an error exitcodeTExternalToolsConsole.HandleMesages: Calling CheckSynchronize!Error: (lazarus) Compile Project, Target: project1: stopped with exit code 1[TCompiler.Compile] endError: (lazbuild) failed compiling of project /home/user/Desktop/clock2/project1.lpiuser@Mint21:~/Desktop/clock2$ user@Mint21:~/Desktop/clock2$ i am at a loss as to how to proceed. if someone could suggest where in the compiler's source the error codes 10011 come from... cheers, rob   :-) ---------------------------------------------------------------------- TRon · 2023-10-14 17:57 Quote from: robert rozee on October 14, 2023, 01:31:29 pmi am at a loss as to how to proceed. if someone could suggest where in the compiler's source the error codes 10011 come from... The 10011 codes that you refer to as error codes aren't actually error codes but informational messages. It seem that the compiler tries to inform you that the directory where the includes files are located is not known. That is more a configuration issue. More problematic is the 10022 code that you received (also related to configuration) where the compiler is unable to locate the source-file for unit gtk2int. The compiler has at least detected that something is off for unit gtk2int and therefor tries to recompile the unit in an attempt to 'fix' the units that was updated. I usually use FPC parameter option -va to show all relevant details for a build but you could probably get by with using the parameter option -vt in order to show all files/directories the compiler tried (in order to determine which directory is misconfigured (if any) or plain missing from your configuration). fwiw: I am still puzzled with your results regarding fnmatch and libc. As I showed in my previous response none of the elf-files that are present (both fpc and lazarus) have any reference to it, ergo this is not compiled in (despite what you may think the source-code is telling you). All fnmatch libc references are either not for linux/unix target or only present when used explicitly by unit unixutil. So, it is currently beyond me how you are able to trigger its use. The present ELF files are riddled with sysutils (pascal) implementation of fnmatch though. Perhaps I need to carefully re-read the thread... ---------------------------------------------------------------------- WayneSherman · 2023-10-14 19:51 Quote from: robert rozee on October 14, 2023, 01:31:29 pm...with fnmatch versioned to fnmatch@GLIBC_2.2.5 fails i am at a loss as to how to proceed. GLIBC 2.2.5 was released in 2002 [edited / fixed] https://sourceware.org/glibc/wiki/Glibc%20Timeline (EDITED) Has fnmatch had an ABI change since GLIBC_2.2.5? Checking GLIBC 2.37 there is only one ABI version reported: Code: [Select]readelf --dyn-syms -W /lib/libc.so.6 | grep fnmatch    802: 00000000000ddab0   411 FUNC    GLOBAL DEFAULT   15 fnmatch@@GLIBC_2.2.5 For visibility into which external functions have name changes, can debug code be added in to output the import_name, file name, and line number?: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i if sym_name=gf.Name thenbegin  stringdispose(import_name);                     // dispose of unversioned symbol name  import_name:=stringdup(gf.Name+'@'+gf.Version);   // replace with versioned symbol name  //add debug output here to report 'GLIBC_Symbol_Versioning',import_name, source_file_name, source_file_line_number  break;end; ---------------------------------------------------------------------- robert rozee · 2023-10-14 20:22 Quote from: WayneSherman on October 14, 2023, 09:51:44 pm For visibility into which external functions have name changes, can debug code be added in to output the sym_name, file name, and line number?: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i if sym_name=gf.Name thenbegin  stringdispose(import_name);                     // dispose of unversioned symbol name  import_name:=stringdup(gf.Name+'@'+gf.Version);   // replace with versioned symbol name  //add debug output here to report 'GLIBC_Symbol_Versioning',import_name, source_file_name, source_file_line_number  break;end; i'm actually using the patch as i originally posted, not DonAlfredo's version. but the result should be the same. the data structure that holds the key information, pd from procedure pd_external(pd:tabstractprocdef);... is entirely opaque, with import_name and import_dll being the only fields i have knowledge of. there may be (indeed, almost certainly are) fields containing more clues as to where a symbol is coming from - such as file currently being compiled and source line numbers - but i need the FPC developers to release this information. to date, they appear to have refused to help in any way. my own 'development environment' is limited to a just simple text editor and Fred vS's 4-line bash script to rebuild everything. Quote from: TRon on October 14, 2023, 07:57:49 pmI usually use FPC parameter option -va to show all relevant details for a build i'm using lazbuild, which does not directly support -va. bear in mind that the compiler is being deprived of access to the unversioned fnmatch, instead being given fnmatch@GLIBC_2.2.5. this is what lies behind the errors. it is not a configuration file error. there is almost certainly nothing wrong with gtk2int, but rather with the compiler needing to make use of whatever the unversioned fnmatch resolves to. to date, we have no idea where the unversioned fnmatch lands. well, slight correction,we do know it doesn't land in libc/glibc! and if it does land in an external library, then the means by which it gets there is extremely well obscured! cheers, rob   :-) ---------------------------------------------------------------------- WayneSherman · 2023-10-14 21:08 Quote from: robert rozee on October 14, 2023, 10:22:56 pmThe data structure that holds the key information, pd from procedure pd_external(pd:tabstractprocdef);... is entirely opaque...there may be (indeed, almost certainly are) fields containing more clues as to where a symbol is coming from - such as file currently being compiled and source line numbers (NOTE: chatGPT responses below are untested and unverified) Asking chatGPT: "In the fpc compiler file pdecsub.pas, I want to add code to the procedure pd_syscall.  For debugging purposes I want to output the current source code file and line number being processed.  How do I output this?" Answer was longer, but to summarize: Quote In the context of Free Pascal Compiler (FPC), the information about the current source file and line number is usually available in the global current_filepos record, which typically contains fields like file, line, and column. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure pd_syscall;begin  { ... existing code ... }   { Debug output }  writeln('Processing file: ', current_filepos.file, ' at line: ', current_filepos.line);   { ... existing code ... }end; Asked: "there is a procedure named "message" which appears to help output compiler information.  Where is this defined and does it output the current source file and line number?" Answer: QuoteThe message procedure usually comes in several overloaded variants, allowing for different types of messages and formats. One of these variants usually takes a message identifier (often an enumerated value) which determines the specific message to be shown. Some of these messages might include placeholders for additional data, such as file names, line numbers, etc. To output a message with the current file and line number, you can use the message procedure combined with the appropriate message identifier and make use of the current_filepos record for file and line information. However, many of the message routines automatically include file and line information based on the context.  Here's a conceptual example: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i message1(file_e_some_error, current_filepos.file^);QuoteIn this hypothetical example, file_e_some_error might be a message identifier that corresponds to a message like "Error in file %s". The %s is a placeholder that would be replaced by the value provided by current_filepos.file^. To determine the exact format, parameters, and identifiers you need, you'd have to refer to the definition.  Do note that using the message procedure in the compiler can impact the user experience because these messages are meant to inform the user about compilation issues. If you're using it for debugging purposes, you'd ideally want to remove or comment out these messages once you're done debugging. ---------------------------------------------------------------------- WayneSherman · 2023-10-14 21:41 Quote from: WayneSherman on October 14, 2023, 11:08:44 pm(NOTE: chatGPT responses below are untested and unverified) So chatGPT responses pointed in the right direction, but the details were not entirely correct. globals.pas (which is in the uses of pdecsub.pas), has global variable: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i        current_filepos : tfileposinfo;    { current position }  //line globtype defines Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i tfileposinfo = record  { if types of column or fileindex are changed, modify tcompilerppufile.putposinfo }  line      : tfileposline;  column    : tfileposcolumn;  fileindex : tfileposfileindex;  moduleindex : tfileposmoduleindex;end; verbose.pas defines some message procedures ---------------------------------------------------------------------- WayneSherman · 2023-10-14 22:19 This isn't quite what I want, but after more searching around fpc compiler sources, maybe this will at least output a message with filename and line number and the import name: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i MessagePos1(current_filepos, sym_w_experimental_symbol, import_name); EDIT: I also found the globals.pas has parser_current_file : string (which I think is the filename of the currently parsed file) EDIT2: the verbose.pas procedure "Comment" appears to be more generic: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i Comment(V_Note,'output this comment as a compiler message'); EDIT3: Better use V_Note for the verbose level (debug level verbosity is probably not set so the message would not be shown if using V_Debug) ---------------------------------------------------------------------- WayneSherman · 2023-10-15 00:44 BTW, I was having trouble finding where the GLIBC source files define the mapping for symbol versions (specifically for Linux x86_64).  I discovered that instead of using map files directly, they use "*.abilist" files which get transformed into map files and fed to the linker when generating *.so files.  (I also see many "Versions" map files scattered throughout the source, I don't know why) GLIBC symbol versioning declarations for Linux x86_64: https://sourceware.org/git/?p=glibc.git;a=tree;f=sysdeps/unix/sysv/lin ux/x86_64/64 https://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/unix/sysv/lin ux/x86_64/64/libc.abilist ---------------------------------------------------------------------- WayneSherman · 2023-10-15 05:12 Quote from: DonAlfredo on September 27, 2023, 02:19:39 pmTo @myself and @other interested. Included a better and more complete patch for versioned linking using the new -XLC compiler option. For a patch or merge request to be accepted does it have to be based on fpc trunk (main / head)?  I had to do some minor rework to get it to apply to the most recent trunk.  fpc builds without errors, but I haven't tested it. https://gitlab.com/waynesherman/fpc-source/- /compare/main...glibc_versioning_donalf?from_project_id=28644964 ---------------------------------------------------------------------- TRon · 2023-10-15 05:19 Quote from: WayneSherman on October 15, 2023, 07:12:10 amFor a patch or merge request to be accepted does it have to be based on fpc trunk (main / head)? For new features, yes. ---------------------------------------------------------------------- WayneSherman · 2023-10-15 05:52 Quote from: robert rozee on October 14, 2023, 10:22:56 pmi'm actually using the patch as i originally posted, not DonAlfredo's version. but the result should be the same. I took your code and applied it to a new branch based off a recent fpc main commit.  It builds, but I did not test it.  I put the "basesyms.map" file in the compiler directory with pdecsub.pas, but the code that loads it doesn't specify a path (at the moment) so the compiler will not find the map file at run time. https://gitlab.com/waynesherman/fpc-source/- /compare/main...glibc_versioning_rrozee?from_project_id=28644964 ---------------------------------------------------------------------- TRon · 2023-10-15 06:22 Quote from: WayneSherman on October 15, 2023, 07:52:00 amI put the "basesyms.map" file in the compiler directory with pdecsub.pas, but the code that loads it doesn't specify a path (at the moment) so I don't know if it will find it. Besides using a hardcoded path or expecting a dedicated location and meant as a suggestion, you can use something like GetEnvironmentVariable to make the path configurable in a quick (simple) manner. ---------------------------------------------------------------------- robert rozee · 2023-10-15 10:11 Quote from: WayneSherman on October 14, 2023, 11:41:46 pmglobals.pas (which is in the uses of pdecsub.pas)... brilliant! within globals.pas there is a string variable called inputfilename that holds the name of the source file currently being compiled. tinkering with my debug output code in pd_external: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i                                                              // DEBUG: the build script supresses some console output,            temp:=GetTempDir+'symbols.txt';                    // so the below lines are to capture ALL versioning into            Assign(T, temp);                                 // a file in the tmp directory.          if not FileExists(temp) then Rewrite(T)                                  else Append(T);            writeln(T, inputfilename, '    "', sym_name, '" >> "', ver_name, '"');            if assigned(import_dll) then writeln(T, 'DLL = ', import_dll^);            Close(T) ... yields the following debug information: Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i buildgtk2.pp    "fnmatch" >> "fnmatch@GLIBC_2.2.5"DLL = libgtk-x11-2.0.so buildgtk2.pp refers to /usr/share/fpcsrc/3.2.2/packages/gtk2/src/buildgtk2.pp which contains very little, but digging into the units it includes ends up at: /usr/share/fpcsrc/3.2.2/packages/gtk2/src/gtk+/gtk/fnmatch.pas and the code: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i { Match STRING against the filename pattern PATTERN,   returning zero if it matches, FNM_NOMATCH if not.   }function fnmatch(__pattern: char; __string: char;  __flags: gint): gint; cdecl; external gtklib; i believe this is where the errant fnmatch symbol is coming from. but alas, i'm still no closer to figuring out where it lands! cheers, rob   :-) ---------------------------------------------------------------------- robert rozee · 2023-10-15 10:25 Quote from: robert rozee on October 15, 2023, 12:11:23 pm /usr/share/fpcsrc/3.2.2/packages/gtk2/src/gtk+/gtk/fnmatch.pas and the code: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i { Match STRING against the filename pattern PATTERN,   returning zero if it matches, FNM_NOMATCH if not.   }function fnmatch(__pattern: char; __string: char;  __flags: gint): gint; cdecl; external gtklib; indeed, commenting out lines 3 and 4 above in /usr/share/fpcsrc/3.2.2/packages/gtk2/src/gtk+/gtk/fnmatch.pas yields a successful build using Fred vS's bash script, but a subsequent inability to build a GUI application. cheers, rob   :-) ---------------------------------------------------------------------- robert rozee · 2023-10-18 12:08 Quote from: robert rozee on October 15, 2023, 12:25:25 pmQuote from: robert rozee on October 15, 2023, 12:11:23 pm /usr/share/fpcsrc/3.2.2/packages/gtk2/src/gtk+/gtk/fnmatch.pas and the code: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i { Match STRING against the filename pattern PATTERN,   returning zero if it matches, FNM_NOMATCH if not.   }function fnmatch(__pattern: char; __string: char;  __flags: gint): gint; cdecl; external gtklib; thanks to assistance from Mattias Gaertner on the fpc-devel mailing list, the fnmatch mystery/problem has been cracked! the procedure is as follows... 1. edit fnmatch.inc to remove the reference to gtklib: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i { Match STRING against the filename pattern PATTERN,   returning zero if it matches, FNM_NOMATCH if not.   }function fnmatch(__pattern: char; __string: char;  __flags: gint): gint; cdecl; external; // gtklib; 2. (if desired) patch pdecsub.pas as required so that pd_external() converts unversioned libc symbols into BASE versioned symbols. in my version of the patch (see first posting in this thread), you can now delete or comment out (as shown below) the three lines in the patched file pdecsub.pas: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i //      if assigned(import_dll) then temp:=import_dll^//                            else temp:='';//      if pos('GTK', upcase(temp))=0 then DonAlfredo's version may require editing in another way, or could be left unchanged. 2A. ADDENDUM: create a symlink from libdl.so to libdl.so.2 using: Code: Bash  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i sudo ln -s /lib/x86_64-linux-gnu/libdl.so.2 /lib/x86_64-linux-gnu/libdl.so the above symlink is really important to ensure libdl.so.2 is correctly referenced in future ELF headers 3. rebuild FPC using Fred vS's script (run as root using sudo), that has been placed in /usr/share/fpcsrc/3.2.2: Code: Bash  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i #!/bin/bashif (( $EUID != 0 )); then    echo "Please run as root"; exitfitime (COMPILER=fpcmake cleanmake all FPC=$COMPILER OPT="-Fl/usr/local/lib"make FPC=$COMPILER install INSTALL_PREFIX=/usr/) 4. rebuild Lazarus using the below script (run as root using sudo), created with Mattias Gaertner's help: Code: Bash  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i #!/bin/bashif (( $EUID != 0 )); then    echo "Please run as root"; exitfitime (make distclean bigide) the result should be a fully functioning FPC and Lazarus, with the fnmatch anomaly fixed and BASE versioned glibc symbols so your compiled applications will run against ANY glibc version. cheers, rob   :-) ---------------------------------------------------------------------- dbannon · 2023-10-19 08:51 Of course, the need to be root only applies if fpc/lazarus are installed in root space. Lots and lots of user with Lazarus in user space and lots with fpc there too. But that aside, congratulations, brilliant piece of work ! Davo ---------------------------------------------------------------------- Tomas Hajny · 2023-10-19 13:51 Out of curiosity, I looked very quickly at the patches discussed in this thread and I noticed at least two things which should be certainly done differently (note that this doesn't touch the primary intentions - I don't feel to be in the position to judge that part): 1) You should better not "reuse" fields in the PPU unit flags for a different meaning even if the previous use was discontinued. 2) The patch is restricted to x86_64-linux. I don't know the reasons, but it doesn't feel right at all. As long as the problem being solved exists also on other CPUs and probably also other operating systems, restricting the patch to just one particular combination, although possibly the one used most widely, doesn't look like a good idea. ---------------------------------------------------------------------- robert rozee · 2023-10-20 05:11 Quote from: Tomas Hajny on October 19, 2023, 03:51:21 pm2) The patch is restricted to x86_64-linux. I don't know the reasons, but it doesn't feel right at all. As long as the problem being solved exists also on other CPUs and probably also other operating systems, restricting the patch to just one particular combination, although possibly the one used most widely, doesn't look like a good idea. my version of the patch is just linux specific, although not tested against 32-bit or non x86. the symbol BASE version numbers are architecture-specific and so retrieved from the local libc and stored in a file placed into /tmp. on the whole, adapting to any other unix- like OS should be trivial, but again not tested; the problem i was trying to fix arose out of my use of Lazarus/FPC on 64-bit x86 Linux. btw, your comments are appreciated. please do give the patch a try! cheers, rob   :-) ---------------------------------------------------------------------- DonAlfredo · 2023-10-22 08:07 I have made a parser of glibc versions for all major cpu. See screenshot. Will try to use it for then versioning patch. ---------------------------------------------------------------------- WayneSherman · 2023-10-22 14:06 Quote from: DonAlfredo on October 22, 2023, 10:07:57 amI have made a parser of glibc versions for all major cpu. See screenshot. Will try to use it for then versioning patch. Very nice.  Are you parsing the version information from the *.abilist files in the libc source git repository? (I was thinking of making a converter using pascal or bash script to create the glibc_x64.inc file from the abilist files) ---------------------------------------------------------------------- DonAlfredo · 2023-10-22 14:10 Yes !! Found the link in this thread. Did not know before. More cpu can be added, as well as other versioned symbols. Now thinking of a way to easy implement into the compiler. ---------------------------------------------------------------------- WayneSherman · 2023-10-22 14:55 Quote from: DonAlfredo on October 22, 2023, 04:10:05 pmNow thinking of a way to easy implement into the compiler. I think these things should be considered: 1) Ability to turn on and off external declaration versioning globally during compilation (-XLC accomplishes that) 2) Allow the versioning to be bypassed in source code. (if one wants to NOT version a particular external declaration how do we indicate it will be skipped) 3) Ability to override the version mapping.  Compiling the mapping table into the compiler is convenient (i.e. no external map files needed), but not very flexible.  If one wants to override the built-in versioning of some or all symbols it might be good to allow a "map" file to be passed as a command line parameter that can override one or more of the built in version entries. ---------------------------------------------------------------------- DonAlfredo · 2023-10-22 16:24 Good tips ! Bypassing should already be honored: just add @GLIBC.... into the external function call declaration. At this moment, an override is not necessary. This patch/option will have a very small user base. For these users, it will do exactly what is needed/wanted. One of the major things to test is Lazarus itself. Lazarus will (also) be built with the patched RTL. I will try to implement this into fpcupdeluxe but only for FPC stable (3.2.2). And lets wait and see how it works out there. ---------------------------------------------------------------------- WayneSherman · 2023-10-22 22:29 Quote from: DonAlfredo on October 22, 2023, 06:24:43 pmBypassing should already be honored: just add @GLIBC.... into the external function call declaration. Let's say I am working with a library provided by others that has a "wait" procedure, how do I prevent symbol versioning being added?: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure wait(const msTime: Double); cdecl; external a-library.so name 'wait'; Quote from: DonAlfredo on October 22, 2023, 06:24:43 pmAt this moment, an override is not necessary. Agreed.  Save that for a future time if it is deemed necessary. Quote from: DonAlfredo on October 22, 2023, 06:24:43 pm This patch/option will have a very small user base. Why do think that? Who does not want their compiled binaries to run on the largest number of systems both old to new? Who prefers to maintain an old Linux distro for the purpose of building and linking binaries when all the development and testing was done a new distro?  Once it is established that this solution works well, I think it should be merged into upstream FPC sources and everyone that wants can use it (which I predict will be most developers). Quote from: DonAlfredo on October 22, 2023, 06:24:43 pmI will try to implement this into fpcupdeluxe but only for FPC stable (3.2.2). And lets wait and see how it works out there. Agreed.  It is better to test on a non-moving target. ---------------------------------------------------------------------- DonAlfredo · 2023-10-24 13:03 Update. After making everything work, I was able to successfully compile a legacy version of a simple form on the latest Arch Linux. All calls towards glibc were found and converted into versioned calls (mostly 2.2.5). So far so good. However. Running this app on Ubuntu 18 failed miserably. The call to dlopen failed. Turns out that this has happened. https://sourceware.org/glibc/wiki/Release/2.34#Libraries_merged_into_l ibc So, no libdl.so.2 any more on these modern systems. My app was indeed only linked against libc.so.6 on the latest Arch Linux. The dlopen function is located in libc.so.6 And on Ubuntu 18 a function like dlopen is still located in libdl.so.2, resulting in the error "dlopen not found". Sidenote. I did not succeed in forcing the linker to link against an more or less empty libdl.so.2 (use --no-as-needed linker option) Any ideas are very welcome !! Update. Forgot that the linker looks for static libs by default. This works to get the unneeded libs into the executable: --no-as-needed -l:libdl.so.2 -l:librt.so.1 ---------------------------------------------------------------------- robert rozee · 2023-10-24 14:36 Quote from: DonAlfredo on October 24, 2023, 03:03:40 pm[...] However. Running this app on Ubuntu 18 failed miserably. The call to dlopen failed. Turns out that this has happened. https://sourceware.org/glibc/wiki/Release/2.34#Libraries_merged_into_l ibc So, no libdl.so.2 any more on these modern systems. My app was indeed only linked against libc.so.6 on the latest Arch Linux. The dlopen function is located in libc.so.6 And on Ubuntu 18 a function like dlopen is still located in libdl.so.2, resulting in the error "dlopen not found". that one caught me out too when testing the patch on a new VM. i'd forgotten an important step: create a symlink from libdl.so to libdl.so.2 using: Code: Bash  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i sudo ln -s /lib/x86_64-linux-gnu/libdl.so.2 /lib/x86_64-linux-gnu/libdl.so the above symlink is really important to ensure libdl.so.2 is correctly referenced in generated ELF headers. cheers, rob   :-) ---------------------------------------------------------------------- WayneSherman · 2023-10-24 16:32 For future reference, the libc release notes regarding the merging of libraries: https://sourceware.org/pipermail/libc-alpha/2021-August/129718.html Quote* In order to support smoother in-place-upgrades and to simplify   the implementation of the runtime all functionality formerly   implemented in the libraries libpthread, libdl, libutil, libanl has   been integrated into libc.  New applications do not need to link with   -lpthread, -ldl, -lutil, -lanl anymore.  For backwards compatibility,   empty static archives libpthread.a, libdl.a, libutil.a, libanl.a are   provided, so that the linker options keep working.  Applications which   have been linked against glibc 2.33 or earlier continue to load the   corresponding shared objects (which are now empty).  The integration   of those libraries into libc means that additional symbols become   available by default.  This can cause applications that contain weak   references to take unexpected code paths that would only have been   used in previous glibc versions when e.g. preloading libpthread.so.0,   potentially exposing application bugs. ---------------------------------------------------------------------- marcov · 2023-10-24 16:44 Quote from: robert rozee on October 24, 2023, 04:36:14 pmthe above symlink is really important to ensure libdl.so.2 is correctly referenced in generated ELF headers. That symlink should be installed by glibc-devel or whatever the base development package on your distro is. ---------------------------------------------------------------------- robert rozee · 2023-10-24 16:52 Quote from: marcov on October 24, 2023, 06:44:08 pmQuote from: robert rozee on October 24, 2023, 04:36:14 pmthe above symlink is really important to ensure libdl.so.2 is correctly referenced in generated ELF headers. That symlink should be installed by [...] should be, but is not. i recall this being argued about some time back. now if FPC accepted numeric extensions to after the .so then this wouldn't be such a problem! cheers, rob   :-) ---------------------------------------------------------------------- DonAlfredo · 2023-10-24 16:56 Well, I guess that the compat-app should not rely on dev-packages to be installed on the legacy system itself. That is why it should link against the versioned libs. And I hope these versions are more or less stable for the last [15] years. ---------------------------------------------------------------------- marcov · 2023-10-24 17:04 Quote from: robert rozee on October 24, 2023, 06:52:35 pmQuote from: marcov on October 24, 2023, 06:44:08 pmQuote from: robert rozee on October 24, 2023, 04:36:14 pmthe above symlink is really important to ensure libdl.so.2 is correctly referenced in generated ELF headers. That symlink should be installed by [...] should be, but is not. Then report  immediately a bug to your distribution. Note according to formal Unix lore, you need to run ldconfig to update caches if you manually make any change to lib directories, like the symlink. The fat penguin OS doesn't require that afaik, but others might. Quote i recall this being argued about some time back. now if FPC accepted numeric extensions to after the .so then this wouldn't be such a problem! The symlink system is there for a reason. The whole idea is that the system can have a new version of libdl (e.g. .3), and then apps linking to libdl,so will still work. If you hate Unix like systems and don't want to learn their arcane rules, don't use it. ---------------------------------------------------------------------- marcov · 2023-10-24 17:05 Quote from: DonAlfredo on October 24, 2023, 06:56:53 pmWell, I guess that the compat-app should not rely on dev-packages to be installed on the legacy system itself. That is why it should link against the versioned libs. And I hope these versions are more or less stable for the last [15] years. If you link against the symlink, the generated apps should be linked against the versioned lib. That might not work on a system with another version, but cross-system binary compatibility was never high on the *nix agenda. This is why the symlink is in a -DEVEL package, only needed on the system that generates the binaries. ---------------------------------------------------------------------- WayneSherman · 2023-10-24 17:08 Quote from: robert rozee on October 24, 2023, 06:52:35 pmQuote from: marcov on October 24, 2023, 06:44:08 pmThat symlink should be installed by [...]should be, but is not. i recall this being argued about some time back. now if FPC accepted numeric extensions to after the .so then this wouldn't be such a problem! On newer OS versions I think the "libdl.so" symlink has been omitted from the dev packages since libdl was merged into libc. https://stackoverflow.com/questions/75855053/how-to-address-crash-due- to-missing-libdl-so-on-ubuntu-22QuoteThe library is libdl.so.2 in Linux distributions (Debian, RHEL, Ubuntu, etc...). It was historically libdl.so in older operating systems and Unix variants. Both have been coexisting for a while and causing confusion:     On CentOS, libdl.so can be installed by sudo yum install glibc- devel     On Debian and Ubuntu <=20, libdl.so can be installed by sudo apt- get install libc6-dev     On Ubuntu >= 22, libdl.so no longer exists, it can't be installed by libc6-dev (nor gcc). The dev glibc package is a transitive dependency of gcc. It's very common for gcc to be installed on Linux machines (developer machines and end-user machines alike), so it's very common for libdl.so to be accidentally available. It's not part of the base operating system and should not be expected to be available. A comparison of files in Debian libc-dev: Debian 10 Buster (2019) (libc-dev has libdl.a and libdl.so) https://packages.debian.org/buster/amd64/libc6-dev/filelist Debian 11 Bullseye (2021) (libc-dev has libdl.a and libdl.so) https://packages.debian.org/bullseye/amd64/libc6-dev/filelist Debian 12 Bookworm (2023) (libc-dev has libdl.a only) https://packages.debian.org/bookworm/amd64/libc6-dev/filelist ---------------------------------------------------------------------- marcov · 2023-10-24 17:12 Note, if you know you are on a system that maps libdl to libc, try adding parameter   -XLAdl=c Binary compatibility wise however, they will probably be two different OSes. ---------------------------------------------------------------------- DonAlfredo · 2023-10-24 17:15 https://packages.debian.org/bookworm/amd64/libc6/filelist We are lucky: its still there ... /lib/x86_64-linux-gnu/libdl.so.2 ---------------------------------------------------------------------- DonAlfredo · 2023-10-24 17:16 QuoteNote, if you know you are on a system that maps libdl to libc, try adding parameter   -XLAdl=cThat is worth a try ! ---------------------------------------------------------------------- WayneSherman · 2023-10-24 17:47 Quote from: robert rozee on October 24, 2023, 04:36:14 pm...really important to ensure libdl.so.2 is correctly referenced in generated ELF headers. Yes, this allows the compiled binary to run on older systems as well as new. Quote from: marcov on October 24, 2023, 07:12:07 pmNote, if you know you are on a system that maps libdl to libc, try adding parameter   -XLAdl=c Does this generate an executable that references "libdl.so.2"? ---------------------------------------------------------------------- marcov · 2023-10-24 18:54 Quote from: WayneSherman on October 24, 2023, 07:47:28 pmDoes this generate an executable that references "libdl.so.2"? Basically, it should replace references to 'dl'  in Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure xx; cdecl external 'dl' name 'xx'; to behave like they were Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure xx; cdecl external 'c' name 'xx'; It is old functionality however (see buildfaq for details), and it has been a while since I tested it. ---------------------------------------------------------------------- robert rozee · 2023-10-24 23:49 Quote from: marcov on October 24, 2023, 07:04:17 pmIf you hate Unix like systems and don't want to learn their arcane rules, don't use it. not exactly a helpful comment, Marco. the symlink created by: Code: Bash  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i sudo ln -s /lib/x86_64-linux-gnu/libdl.so.2 /lib/x86_64-linux-gnu/libdl.so ... is ONLY needed on the system you are running FPC on, NOT on any system you are going to run your binary on. hence, either (a) FPC itself should create the symlink, or (b) our patch program should create the symlink, or (c) the FPC source should be modified to not need the symlink. basically, your application source needs (if FPC worked correctly) to contain somewhere/anywhere the line: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure anything; cdecl external 'libdl.so.2'; ... where the procedure name doesn't matter. the important part is the external 'libdl.so.2'. this causes the library file libdl.so.2 to be added as a "NEEDED" library to the generated ELF binary's header. the procedure name doesn't matter as long as it exists in libdl or libc. but unfortunately FPC, when it sees 'libdl.so.2', covertly converts it to 'libdl.so'. hence we need to manually add the symlink ourself on our development system. note that this whole argument around the symlink is a Red Herring. the FPC developers will never allow numeric extensions to library names, while adding the symlink on your development system is trivial. the symlink would probably even work well enough if just added into your project directory - no need for sudo if done that way, but then needed for each and every project directory. cheers, rob   :-) ---------------------------------------------------------------------- TRon · 2023-10-25 03:15 Quote from: robert rozee on October 25, 2023, 01:49:33 amnote that this whole argument around the symlink is a Red Herring. the FPC developers will never allow numeric extensions to library names, while adding the symlink on your development system is trivial. the symlink would probably even work well enough if just added into your project directory - no need for sudo if done that way, but then needed for each and every project directory. As I wrote before, no. You only have to do that once. ---------------------------------------------------------------------- DonAlfredo · 2023-10-25 05:08 To summarize. Linking against the unversioned lib will result in the symlink being resolved into a versioned version. As long as this version can be found on the target system, all is good. As legacy Linux systems more or less have the same library versions, I propose to add the needed libs like this (pseudo-code): Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i for s in [' libanl.so.1','libdl.so.2','librt.so.1','libresolv.so.2','libpthread.so .0'] dobegin   AddLibrary(s);end; The versions are my educated guess about the most used versions on old Linux systems. If these libraries are not available on the development system (this might happen as all functions are included into libc itself), some stubs are needed or linking will fail. Stub might be as easy as a symlink to libc. Additional. Unversioned linking of these libs, especially libc is very important. libc.so is often a stub combining two (or more) libraries like this (FreeBSD for Aarch64): Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i GROUP ( libc.so.7 libc_nonshared.a libm.so ) ---------------------------------------------------------------------- marcov · 2023-10-25 07:23 Quote from: robert rozee on October 25, 2023, 01:49:33 am... is ONLY needed on the system you are running FPC on, NOT on any system you are going to run your binary on. hence, either (a) FPC itself should create the symlink, or (b) our patch program should create the symlink, or (c) the FPC source should be modified to not need the symlink. (d) mandatory buildrequirements of FPC (general setting the system up for development, most notably glibc-devel) should add the link. Guess what, it is (d). ---------------------------------------------------------------------- DonAlfredo · 2023-10-25 07:36 Success !! Screenshot of app build with FPC 3.2.2 and Lazarus trunk on Manjaro lates with glibc 2.35. The app runs on Ubuntu 8 dated from 2008 !! ---------------------------------------------------------------------- marcov · 2023-10-25 07:45 Did the -XLA work? ---------------------------------------------------------------------- WayneSherman · 2023-10-25 14:04 Quote from: DonAlfredo on October 25, 2023, 07:08:45 amCode: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i ['libanl.so .1','libdl.so.2','librt.so.1','libresolv.so.2','libpthread.so.0']The versions are my educated guess about the most used versions on old Linux systems. Those filenames match the declarations in "lib_namesh.inc": https://gitlab.com/freepascal.org/fpc/source/- /blob/main/packages/libc/src/lib_namesh.inc?ref_type=heads ---------------------------------------------------------------------- WayneSherman · 2023-10-25 14:06 Quote from: DonAlfredo on October 25, 2023, 09:36:01 amSuccess !! Screenshot of app build with FPC 3.2.2 and Lazarus trunk on Manjaro lates with glibc 2.35. The app runs on Ubuntu 8 dated from 2008 !! Good news and nice work! Did you have to modify the fnmatch delcaration? Quote from: robert rozee on October 18, 2023, 02:08:11 pm1. edit fnmatch.inc to remove the reference to gtklib: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i { Match STRING against the filename pattern PATTERN,   returning zero if it matches, FNM_NOMATCH if not.   }function fnmatch(__pattern: char; __string: char;  __flags: gint): gint; cdecl; external; // gtklib; ---------------------------------------------------------------------- DonAlfredo · 2023-10-25 14:36 Thanks. No changes were needed regarding fnmatch. Seems not used anymore. Extra. The included screenshot shows fpcupdeluxe (cross-compiled from Windows) running on Ubuntu 5.10 using GTK1. ---------------------------------------------------------------------- marcov · 2023-10-25 14:54 Quote from: WayneSherman on October 25, 2023, 04:04:52 pmThose filenames match the declarations in "lib_namesh.inc": https://gitlab.com/freepascal.org/fpc/source/- /blob/main/packages/libc/src/lib_namesh.inc?ref_type=heads Package libc is an historic Kylix emulation package. It is also 32-bit only.  See e.g. https://wiki.freepascal.org/libc_unit Anything that hardcodes version numbers in source should be avoided as that makes a source prone to breaking over longer times. This is also why the few things that are hardcoded like the elf loader ld-linux.so etc are overridable using parameters (like the -FD parameter), which can be stashed in the configuration (fpc.cfg) ---------------------------------------------------------------------- DonAlfredo · 2023-10-25 15:24 QuoteDid the -XLA work?I think this becomes useful in the future when all libs are merged into libc ! So I did not use it for the backwards compatibility test. ---------------------------------------------------------------------- Seenkao · 2023-10-25 18:41 Я заглянул в собранные бинарные файлы и libdl.so нет в новых версиях ELF. Файлы не абсолютно идентичны, но подобны. Google translate: I looked at the compiled binaries and libdl is not in the newer versions of ELF. The files are not absolutely identical, but they are similar. Quote from: old ELF, libdl.so - trueITM_deregisterTMCloneTable�_ITM_re gisterTMCloneTable�__errno_location�__gmon_start__�dlclose�dlsym�dladd r�dlopen�XLookupKeysym�XNextEvent�XDestroyWindow�XSync�XMapWindow�XCre atePixmap�XSetWMNormalHints� XCreateWindow�XQueryPointer�XFree�XResizeWindow�XSetWMProtocols�XCreat ePixmapCursor�XSetWMName�XSetClassHint�XInternAtom�XFreeCursor�XSendEv ent�XPending�Xutf8TextListToTextProperty�XCreateColormap�XMoveWindow�X SetWMIconName�XOpenDisplay�XCloseDisplay� XDefineCursor�XRRUpdateConfiguration�XRRSetScreenConfig�XRRConfigCurre ntConfiguration�XRRGetScreenInfo�XRRSizes�XRRSelectInput�XRRQueryExten sion�XRRFreeScreenConfigInfo�XRRRates�gdk_screen_get_monitor_geometry� gdk_screen_get_default�gdk_screen_get_n_monitors�gtk_init� gtk_debug_flags�deflate�deflateEnd�glEnable�glGenTextures�glVertex2f�g lColor4f�glClearDepth�glColor4ubv�glMatrixMode�glPushMatrix�glHint�glT exEnvi�glPixelStorei�glClearColor�glTexImage2D�glTranslatef�glDepthFun c�glGetIntegerv�glScalef�glPopMatrix�glViewport�glEnd�glDisable�glDele teTextures� glOrtho�glBlendFunc�glRotatef�glAlphaFunc�glClear�glBegin�glTexSubImag e2D�glLoadIdentity�glShadeModel�glGetString�glTexParameterf�glTexParam eteri�glTexCoord2f�glGetTexImage�glBindTexture�gluNewTess�gluGetString �gluBuild2DMipmaps�gluTessCallback�gluDeleteTess�gluPerspective�inflat eEnd� inflateInit2_�crc32�zError�inflate�deflateInit2_�sprintf�_IO_putc�fope n�strrchr�__strdup�realloc�memchr�getpid�chmod�mkstemp�sched_yield�str len�memcmp�clearerr�memcpy�fclose�strtoul�malloc�umask�strcasecmp�ftel lo�remove�__ctype_b_loc�__fxstat�fileno�fwrite�fread�gettimeofday�rena me� localtime�mktime�fdopen�qsort�__cxa_finalize�__xstat�strcmp�strerror�_ _libc_start_main�ferror�fseeko�snprintf�free�libpthread.so.0�libdl.so. 2�libX11.so.6�libXrandr.so.2�libgdk-x11-2.0.so.0�libgtk-x11- 2.0.so.0�libgobject-2.0.so.0�libglib-2.0.so.0�libgthread- 2.0.so.0�libgmodule-2.0.so.0�libgdk_pixbuf-2.0.so.0� libpango-1.0.so.0�libcairo.so.2�libatk- 1.0.so.0�libGL.so.1�libm.so.6�libGLU.so.1�libz.so.1�libc.so.6�_edata�_ _bss_start�_end�GLIBC_2.2.5�GLIBC_2.3 Quote from: new ELF, libdl.so - False_ITM_deregisterTMCloneTable�__gmo n_start__�_ITM_registerTMCloneTable�XGrabKeyboard�XLookupKeysym�XSetGr aphicsExposures�XNextEvent�XGrabPointer�XDestroyWindow�XSync�XMapWindo w�XCreatePixmap�XSetWMNormalHints�XCreateWindow�XQueryPointer�XFree� XResizeWindow�_end�XSetWMProtocols�XCreatePixmapCursor�XSetWMName�XSet ClassHint�XInternAtom�XFreeCursor�XClearArea�XSendEvent�XWarpPointer�X Pending�Xutf8TextListToTextProperty�XCreateColormap�XMoveWindow�XSynch ronize�XSetWMIconName�XOpenDisplay� XCloseDisplay�XDefineCursor�XGetWindowProperty�XInternAtoms�XRRUpdateC onfiguration�XRRSetScreenConfig�XRRConfigCurrentConfiguration�XRRGetSc reenInfo�XRRSizes�XRRSelectInput�XRRQueryExtension�XRRFreeScreenConfig Info�XRRRates�glEnable�glGenTextures�glVertex2f� glColor4f�glReadPixels�glClearDepth�glColor4ubv�glMatrixMode�glVertex2 fv�glPushMatrix�glHint�glTexEnvi�glPixelStorei�glClearColor�glTexImage 2D�glTranslatef�glDepthFunc�glDepthMask�glGetIntegerv�glScalef�glSciss or�glColorMask�glPopMatrix�glViewport�glEnd�glDisable� glDeleteTextures�glOrtho�glBlendFunc�glRotatef�glAlphaFunc�glClear�glB egin�glTexCoord2fv�glCopyTexSubImage2D�glTexSubImage2D�glLoadIdentity� glShadeModel�glGetString�glTexParameterf�glTexParameteri�glTexCoord2f� glGetTexImage�glBindTexture�floor�sincosf�atan�log�acosf� ldexp�pow�sin�sincos�inflateEnd�inflateInit2_�deflate�inflateInit_�crc 32�zError�inflate�deflateInit2_�deflateEnd�gluNewTess�gluGetString�glu Build2DMipmaps�gluTessEndPolygon�gluTessEndContour�gluTessCallback�glu DeleteTess�gluTessVertex�gluPerspective�gluTessBeginPolygon� gluTessBeginContour�strcpy�snprintf�__xstat�chmod�dladdr�__ctype_toupp er_loc�free�fread�exit�__strdup�dlclose�strrchr�ferror�memmove�gettime ofday�clearerr�fopen�strlen�__ctype_b_loc�getpid�mkstemp�umask�realloc �strcat�strcasecmp�malloc�__libc_start_main�sched_yield� remove�fdopen�mktime�memchr�__fxstat�dlsym�dlopen�__cxa_finalize�sprin tf�localtime�ftell�strerror�calloc�memcmp�fclose�_IO_putc�memset�renam e�ftello�strtoul�memcpy�fileno�fwrite�strcmp�qsort�fseek�fseeko�__errn o_location�libX11.so.6�libXrandr.so.2�libGL.so.1�libm.so.6� libz.so.1�libGLU.so.1�libc.so.6�_edata�__bss_start�GLIBC_2.29�GLIBC_2. 2.5�GLIBC_2.34�GLIBC_2.3 ---------------------------------------------------------------------- Seenkao · 2023-10-25 19:52 Maybe this will be useful: https://www.itcodar.com/linux/how-to-link-to-a-specific-glibc- version.html ---------------------------------------------------------------------- DonAlfredo · 2023-10-26 06:10 @Seenkao For sure that is the recommended way of versioning. But that requires source changes of the app to be compiled. And the goal here is to use compiler magic to accomplish this. Only apply the -XLC switch and the compiler will do the versioning for you (if all works as expected). ---------------------------------------------------------------------- DonAlfredo · 2023-10-27 12:20 As promised, I have included the legacy linking option into fpcupdeluxe 2.4.0bp. The compiler patch : https://github.com/LongDirtyAnimAlf/fpcupdeluxe/b lob/master/patches/patchfpc/glibc_compat_3_2_2.patch The version file: https://github.com/LongDirtyAnimAlf/fpcupdeluxe/blob /master/patches/patchfpc/glibc.inc Fpcupdeluxe will do (on Linux): Get the FPC 3.2.2 sources. Patch them with the compiler patch. Add the version file. Build all FPC. All FPC binaries will function as usual. Rebuild the RTL and Packages with the legacy-switch enabled + -defined FPC_USE_LIBC and install them in a legacy units directory. Get Lazarus trunk/main. Patch Lazarus a little bit to work with libc under Linux. Build the whole of Lazarus under legacy conditions. If all ok, Lazarus will now produce legacy binaries. In fact, Lazarus itself is legacy. To summarize. All glibc-call are versioned to the lowest version possible. No syscall are used anymore. All done through libc. A new compiler switch is added : -XLC. The linker adds some libraries as needed to allow legacy Linux systems to resolve the libc functions used. Special for Windows users. The legacy linking will also work when cross-compiling. Fpcupdeluxe 2.4.0bp will download libs from Fedora38. Legacy linking should allow you to get a binary that works also on very old systems. https://github.com/LongDirtyAnimAlf/fpcupdeluxe/releases/tag/v2.4.0bp Please test if interested. With FPC 3.2.2 and Lazarus trunk/main. Thanks ---------------------------------------------------------------------- WayneSherman · 2023-10-27 12:47 Quote from: DonAlfredo on October 27, 2023, 02:20:08 pmAs promised, I have included the legacy linking option into fpcupdeluxe 2.4.0bp. ... Please test if interested. With FPC 3.2.2 and Lazarus trunk/main. Thank you DonAlfredo for your work on this. I would like to thank Robert Rozee as well for his research, testing, and persistence on the issue of glibc symbol versioning. ---------------------------------------------------------------------- DonAlfredo · 2023-10-27 12:56 Your welcome. And also thanks to Robert Rozee indeed ! As proof and for fun, the screenshot shows fpcupdeluxe running on Ubuntu 6.10, with a fresh install of FPC 2.6.0 and Lazarus 1.0.12. ;-) ---------------------------------------------------------------------- TRon · 2023-10-27 17:08 Hi DonAlfredo: Thank you for the update, and newly introduced feature. I have but one concern. Using your "default" fpc.cfg file both my normal and legacy directories are searched. That 1) slows down compiling (hardly noticeable on smaller projects but still) and 2) someone not understanding might set the unit directories in the wrong order which would then defeat the purpose and 3) it makes linking in a wrong (lost) unit more likely to happen. I would personally have chosen for an else or ifndef FPC_LINK_COMPAT in the fpc.cfg file. What are your thoughts on that ? Other than that, it seems to work for me but I've only tested some simple examples. I want to make sure the FPC side is working 100% before the need to have yet another Lazarus rebuild installed on my system ---------------------------------------------------------------------- funlw65 · 2024-05-03 08:22 Hi guys, I wanted to test freepascal-ootb of fredvs, and noticed that there is no FPIDE included... tried to compile it and it didn't work. I tried also a patched standard fpc using fpcupdeluxe and applications that use FV are not working. They work again if the patch is removed, I mean, a standard FPC 3.2.2. ---------------------------------------------------------------------- MarkMLl · 2024-10-03 18:58 So what was the final resolution of this? I was one of the first people complaining about it https://forum.lazarus.freepascal.org/index.php/topic,61001.0.html and was reminded of the issue when wanting to do some rebuild-from-scratch stuff on Debian 12 today. Looking at the standard trunk there's no mention of anything like -XLC, I can't find anything relevant in the bug tracker, and I don't see anything under https://www.freepascal.org/probs.html MarkMLl ---------------------------------------------------------------------- Fred vS · 2024-10-03 19:23 Quote from: MarkMLl on October 03, 2024, 08:58:13 pmSo what was the final resolution of this? In case you are impatient, there is this (with also fpc 3.2.4 pre release). QuoteHi guys, I wanted to test freepascal-ootb of fredvs, and noticed that there is no FPIDE included...fpide-ootb is also included now. ---------------------------------------------------------------------- MarkMLl · 2024-10-03 19:40 Quote from: Fred vS on October 03, 2024, 09:23:16 pmIn case you are impatient, there is this (with also fpc 3.2.4 pre release). I'm resigned rather than impatient, but I had to hunt out a Debian 11 system today to build a specific combination of FPC+Lazarus to run on Debian 12. MarkMLl ---------------------------------------------------------------------- WayneSherman · 2024-10-03 19:45 Quote from: MarkMLl on October 03, 2024, 08:58:13 pmSo what was the final resolution of this? I was one of the first people complaining about it https://forum.lazarus.freepascal.org/index.php/topic,61001.0.html and was reminded of the issue when wanting to do some rebuild-from-scratch stuff on Debian 12 today. Looking at the standard trunk there's no mention of anything like -XLC, I can't find anything relevant in the bug tracker, and I don't see anything under https://www.freepascal.org/probs.html I was thinking about this yesterday because I found a write-up explaining the inner workings of dso (dynamic shared object) symbol versioning: Maintaining APIs and ABIs starting on page 34 https://www.akkadia.org/drepper/dsohowto.pdf Both robert rozee and DonAlfredo did some good work on this issue as a proof of concept, but I don't think it got developed enough to turn into a patch or pull request to upstream.  The last work DonAlfredo did is in fpcupdeluxe commits in 2023-Oct: https://github.com/LongDirtyAnimAlf/fpcupdeluxe/commits/master/?after= 87cb49ff7446f13d57959399e7e925c7af00d494+0 He made a test build (Fpcupdeluxe 2.4.0bp) but it is no longer accessible. ---------------------------------------------------------------------- Fred vS · 2024-10-03 19:54 Quote from: WayneSherman on October 03, 2024, 09:45:05 pmBoth robert rozee and DonAlfredo did some good work on this issue as a proof of concept Robert Rozee mainly did the patch for fpc-ootb. Note that the next version of libc, (if not using versioning) should not be compatible with fpc anymore because lot of things will be changed in the new table. ---------------------------------------------------------------------- Fred vS · 2024-10-03 20:19 Quote from: MarkMLl on October 03, 2024, 09:40:11 pmQuote from: Fred vS on October 03, 2024, 09:23:16 pmIn case you are impatient, there is this (with also fpc 3.2.4 pre release). I'm resigned rather than impatient, but I had to hunt out a Debian 11 system today to build a specific combination of FPC+Lazarus to run on Debian 12. MarkMLl Well, for FPC you could use fpc-ootb but for Lazarus, you will need to assign glibc2.2.5 versioning table to all the glibc methods used in Lazarus code. There are a few, not a lot of. ---------------------------------------------------------------------- TRon · 2024-10-03 20:29 Quote from: MarkMLl on October 03, 2024, 08:58:13 pmSo what was the final resolution of this? As far as I am able to remember there initially was a bit of a resistance to address this issue with(in) the compiler but Robert mentioned that in due time there will be some advancement on that matter. Unfortunately I am able to find that statement as there are many topics mentioning the versioning issues. At least there are possibilities to work around the issue (some good, some bad) most of them I was able to find in this 16 page long discussion. I have not seen any mention of it in trunk either (but I could easily have overlooked). ---------------------------------------------------------------------- WayneSherman · 2024-10-03 22:44 Quote from: TRon on October 03, 2024, 10:29:12 pmAs far as I am able to remember there initially was a bit of a resistance to address this issue with(in) the compiler but Robert mentioned that in due time there will be some advancement on that matter. Unfortunately I am able to find that statement When Alfred (DonAlfredo) was experimenting with fpc glibc symbol versioning, he was also building fpc using "-dFPC_USE_LIBC".  This caused an fpc bug (unrelated to glibc symbol versioning) to show up while trying to build the Lazarus code base (specifically fpdebug was using a kernel syscall, but the system call functions were not being exported when FPC_USE_LIBC was defined).  Apparently the default (for x86_64 Linux) is for the fpc RTL to use system calls and not glibc so this bug wasn't found until 2023-Oct (EDITED). I don't understand the reason for using "-dFPC_USE_LIBC" to build fpc.  To build an fpc compiler with the widest compatibility should it use kernel system calls and no glibc, or should it use no system calls and only glibc?  (EDITED - I wonder if using kernal syscalls only and no glibc is preferred for better forward/backward compatibility.  In any case if glibc calls are made, these should be declared with base symbol versions) More info here: https://github.com/LongDirtyAnimAlf/fpcupdeluxe/issues/634 (the statement you might be looking for is at the bottom) and here: https://gitlab.com/freepascal.org/lazarus/lazarus/-/issues/40575 ---------------------------------------------------------------------- robert rozee · 2024-10-04 05:02 as an aside, the very first posting to this thread (https://forum.laza rus.freepascal.org/index.php/topic,64731.msg492467.html#msg492467) contains a complete explanation on how to apply the patch. the only source code change is to the function pd_external() that is contained in the file /usr/share/fpcsrc/3.2.2/compiler/pdecsub.pas. there is more than enough information provided to allow the same change to be made to later versions of lazarus/FPC. someone just needs to create a small 'patcher program' that is capable of scanning through /usr/share/fpcsrc/3.2.2/compiler/pdecsub.pas to find the two places in pd_external() where the relevant blocks of code need to be added. this functionality could indeed be added as a further function to the MakeBase utility i included. it has been a year since i last looked at all this, and my memory of many of the finer details has faded - do not worry, i've always had a bad memory. it is now up to someone else to proceed forward, if indeed there is any will to do so. cheers, rob   :-) ---------------------------------------------------------------------- MarkMLl · 2024-10-04 07:15 Quote from: WayneSherman on October 03, 2024, 09:45:05 pmBoth robert rozee and DonAlfredo did some good work on this issue as a proof of concept, but I don't think it got developed enough to turn into a patch or pull request to upstream.  The last work DonAlfredo did is in fpcupdeluxe commits in 2023-Oct: They did an enormous amount of good work, and I'm disturbed that the issue still seems to be ignored by the overall FPC project. MarkMLl ---------------------------------------------------------------------- MarkMLl · 2024-10-04 07:35 Quote from: robert rozee on October 04, 2024, 07:02:36 amthere is more than enough information provided to allow the same change to be made to later versions of lazarus/FPC. someone just needs to create a small 'patcher program' that is capable of scanning through /usr/share/fpcsrc/3.2.2/compiler/pdecsub.pas to find the two places in pd_external() where the relevant blocks of code need to be added. this functionality could indeed be added as a further function to the MakeBase utility i included. Yes, I think I'm going to have to look at that. I've got a well-developed sequence of building FPC and then Lazarus from scratch on systems with only a very basic OS which has served me since the earliest public versions: I've still got a system here with 2.2.2 which I use for testing GTK1 stuff. I can cope with not being able to e.g. install older than 2.6.4 on x64 because the target doesn't exist, but feel that it's highly undesirable for there to be other "inflection points" where a native build is problematic: yesterday I had to dig out a Debian 11 system to regenerate the combination of Lazarus 2.0.6 with FPC 3.0.4 because I couldn't get there either forwards or backwards on Debian 12. Having said which, I'm an extremely satisfied user of FpcUpDeluxe for the Pico etc. Just to keep them in the Symbol Versioning record: Quote from: WayneSherman on October 04, 2024, 12:44:10 amMore info here: https://github.com/LongDirtyAnimAlf/fpcupdeluxe/issues/634 (the statement you might be looking for is at the bottom) and here: https://gitlab.com/freepascal.org/lazarus/lazarus/-/issues/40575 MarkMLl ---------------------------------------------------------------------- marcov · 2024-10-04 14:08 Quote from: WayneSherman on October 04, 2024, 12:44:10 amI don't understand the reason for using "-dFPC_USE_LIBC" to build fpc.  To build an fpc compiler with the widest compatibility should it use kernel system calls and no glibc, or should it use no system calls and only glibc? In theory and according to most linux/glibc sources it should be glibc. But that is a bit of a knee jerk reaction that quite often doesn't take into account that Pascal sources won't automatically pick up changed headers and also don't take into account routinely having some cross distro/version binary compatibility.  I.e. they don't consider FPC specific circumstances. On the other hand, one could argue that the current syscall situation is an artifact of the years 2000-2005 when glibc was going through changes (most notably 64-bit file support, also on the then mostly 32-bit systems) To see what is better probably requires both systems to be used in parallel for a while, and honestly looking what causes less problems/breakage. ---------------------------------------------------------------------- WayneSherman · 2024-10-04 16:32 Quote from: marcov on October 04, 2024, 04:08:07 pm In theory and according to most linux/glibc sources it should be glibc. But that is a bit of a knee jerk reaction that quite often doesn't take into account that Pascal sources won't automatically pick up changed headers and also don't take into account routinely having some cross distro/version binary compatibility.  I.e. they don't consider FPC specific circumstances. On the other hand, one could argue that the current syscall situation is an artifact of the years 2000-2005 when glibc was going through changes (most notably 64-bit file support, also on the then mostly 32-bit systems) To see what is better probably requires both systems to be used in parallel for a while, and honestly looking what causes less problems/breakage. The Linux kernel policy is to never break user space (i.e. new kernel versions maintain backward compatibility with existing compiled software): https://unix.stackexchange.com/questions/235335/why-is-there-a-linux- kernel-policy-to-never-break-user-space glibc, using symbol versioning, also strives for new glibc releases to maintain backward compatibility with previously compiled software that uses the library: https://developers.redhat.com/blog/2019/08/01/how-the-gnu-c-library- handles-backward-compatibility I agree that testing would be needed and experience would have to be gained using both approaches before deciding which is generally more backward compatible. In the case of fpc and the rtl using the glibc library on Linux (x86_64): The pascal header conversion and code that calls library functions and code that references external symbols was written and tested against a specific version of glibc with specific documented behavior for those functions and symbols. 1. Do you think the fpc and rtl source code declarations should be specifically pinned to those versions of the glibc functions they were originally written to work with? (i.e. declare the specific glibc SONAME which includes the major version and declare specific versions of the external symbols in the fpc/rtl source code) 2. Do you think the fpc and rtl glibc source code declarations should not specify any versions and allow the compiled binaries to get specific .so version and symbol versions from the static linker at build time (which uses the glibc library present on the system at build time)? or 3. Should there be the ability to choose between the two options above when compiling fpc/rtl? (EDITED to clarify that the library SONAME includes the major version in #1 above) ---------------------------------------------------------------------- Fred vS · 2024-10-04 17:11 Quote from: WayneSherman on October 04, 2024, 06:32:21 pmQuote from: marcov on October 04, 2024, 04:08:07 pm In theory and according to most linux/glibc sources it should be glibc. But that is a bit of a knee jerk reaction that quite often doesn't take into account that Pascal sources won't automatically pick up changed headers and also don't take into account routinely having some cross distro/version binary compatibility.  I.e. they don't consider FPC specific circumstances. On the other hand, one could argue that the current syscall situation is an artifact of the years 2000-2005 when glibc was going through changes (most notably 64-bit file support, also on the then mostly 32-bit systems) To see what is better probably requires both systems to be used in parallel for a while, and honestly looking what causes less problems/breakage. The Linux kernel policy is to never break user space (i.e. new kernel versions maintain backward compatibility with existing compiled software): https://unix.stackexchange.com/questions/235335/why-is-there-a-linux- kernel-policy-to-never-break-user-space glibc, using symbol versioning, also strives for new glibc releases to maintain backward compatibility with previously compiled software that uses the library: https://developers.redhat.com/blog/2019/08/01/how-the-gnu-c-library- handles-backward-compatibility I agree that testing would be needed and experience would have to be gained using both approaches before deciding which is generally more backward compatible. In the case of fpc and the rtl using the glibc library on Linux (x86_64): The pascal header conversion and code that calls library functions and code that references external symbols was written and tested against a specific version of glibc with specific documented behavior for those functions and symbols. 1. Do you think the fpc and rtl source code declarations should be specifically pinned to those versions of the glibc functions they were originally written to work with? (i.e. declare the specific glibc .so version and the specific versions of the external symbols in the fpc/rtl source code) 2. Do you think the fpc and rtl glibc source code declarations should not specify any versions and allow the compiled binaries to get specific .so file version and symbol versions from the static linker at build time (which uses the glibc library present on the system at build time)? or 3. Should there be the ability to choose between the two options above when compiling fpc/rtl? In case you asked it to me too  , I will vote for 1. for Linux amd64 and assign GLIBC_2.2.5 for each libc method used by fpc. Beause fpc was created using this table and never updated for new table and mainly because I did test it deeply and it works perfectly. ---------------------------------------------------------------------- Ștefan-Iulian Alecu · 2024-10-04 17:36 Quote from: Fred vS on October 04, 2024, 07:11:52 pmIn case you asked it to me too  , I will vote for 1. for Linux amd64 and assign GLIBC_2.2.5 for each libc method used by fpc. Beause fpc was created using this table and never updated for new table and mainly because I did test it deeply and it works perfectly. How's holding off to a 18 year old version of glibc good? Quote from: WayneSherman on October 04, 2024, 06:32:21 pmIn the case of fpc and the rtl using the glibc library on Linux (x86_64): The pascal header conversion and code that calls library functions and code that references external symbols was written and tested against a specific version of glibc with specific documented behavior for those functions and symbols. 1. Do you think the fpc and rtl source code declarations should be specifically pinned to those versions of the glibc functions they were originally written to work with? (i.e. declare the specific glibc .so version and the specific versions of the external symbols in the fpc/rtl source code) 2. Do you think the fpc and rtl glibc source code declarations should not specify any versions and allow the compiled binaries to get specific .so file version and symbol versions from the static linker at build time (which uses the glibc library present on the system at build time)? or 3. Should there be the ability to choose between the two options above when compiling fpc/rtl? What do other compilers do in this regard? I feel like no. 3 would be the best option, but by default have the behavior of no. 2. ---------------------------------------------------------------------- Fred vS · 2024-10-04 17:57 Quote from: Ștefan-Iulian Alecu on October 04, 2024, 07:36:46 pmQuote from: Fred vS on October 04, 2024, 07:11:52 pmIn case you asked it to me too  , I will vote for 1. for Linux amd64 and assign GLIBC_2.2.5 for each libc method used by fpc. Beause fpc was created using this table and never updated for new table and mainly because I did test it deeply and it works perfectly. How's holding off to a 18 year old version of glibc good? Because the libc developers promised that each new version would integrate the old tables and if they are defined for the link the new versions will assume them. And as serious developers, they keep their word. ---------------------------------------------------------------------- Fred vS · 2024-10-04 18:17 Quote from: WayneSherman on October 04, 2024, 06:32:21 pm2. Do you think the fpc and rtl glibc source code declarations should not specify any versions and allow the compiled binaries to get specific .so file version and symbol versions from the static linker at build time (which uses the glibc library present on the system at build time)? Sorry to be the boring ones again but there is no specific .so file for libc. There is libc.so.6 and each new version has still the same name libc.so.6 but with new symbols tables. It is the reason why signed symbol tables should always be used for each method. And the magic of libc.so.6 is that if you choose a old table compatible with your program, libc.so.6 will adapt the last updated method to be compatible with that old table. In the case you want to use different tables, it will ask lot of work for fpc to always update his code, each time a new table is added in libc.so.6. Imho, better trust in the loyalty of libc developers. ---------------------------------------------------------------------- WayneSherman · 2024-10-04 18:27 Quote from: Ștefan-Iulian Alecu on October 04, 2024, 07:36:46 pmQuote from: Fred vS on October 04, 2024, 07:11:52 pmIn case you asked it to me too  , I will vote for 1. for Linux amd64 and assign GLIBC_2.2.5 for each libc method used by fpc. Beause fpc was created using this table and never updated for new table and mainly because I did test it deeply and it works perfectly. How's holding off to a 18 year old version of glibc good? Linking to the older symbols doesn't mean you get old, buggy, or insecure implementations.  The symbol versions are intended to provide a stable interface (API/ABI) for backwards compatibility.  The underlying implementations still get bug and security fixes. ---------------------------------------------------------------------- Fred vS · 2024-10-04 18:32 Quote from: WayneSherman on October 04, 2024, 08:27:57 pmQuote from: Ștefan-Iulian Alecu on October 04, 2024, 07:36:46 pmQuote from: Fred vS on October 04, 2024, 07:11:52 pmIn case you asked it to me too  , I will vote for 1. for Linux amd64 and assign GLIBC_2.2.5 for each libc method used by fpc. Beause fpc was created using this table and never updated for new table and mainly because I did test it deeply and it works perfectly. How's holding off to a 18 year old version of glibc good? Linking to the older symbols doesn't mean you get old, buggy, or insecure implementations.  The symbol versions are intended to provide a stable interface (API/ABI) for backwards compatibility.  The underlying implementations still get bug and security fixes. Bravo! ---------------------------------------------------------------------- WayneSherman · 2024-10-04 19:24 Quote from: Ștefan-Iulian Alecu on October 04, 2024, 07:36:46 pmQuote from: WayneSherman on October 04, 2024, 06:32:21 pmIn the case of fpc and the rtl using the glibc library on Linux (x86_64): The pascal header conversion and code that calls library functions and code that references external symbols was written and tested against a specific version of glibc with specific documented behavior for those functions and symbols. 1. Do you think the fpc and rtl source code declarations should be specifically pinned to those versions of the glibc functions they were originally written to work with? (i.e. declare the specific glibc .so version and the specific versions of the external symbols in the fpc/rtl source code) 2. Do you think the fpc and rtl glibc source code declarations should not specify any versions and allow the compiled binaries to get specific .so file version and symbol versions from the static linker at build time (which uses the glibc library present on the system at build time)? or 3. Should there be the ability to choose between the two options above when compiling fpc/rtl? What do other compilers do in this regard? I feel like no. 3 would be the best option, but by default have the behavior of no. 2. Typically when building C programs, you are using the matching header files for the installed glibc version, and the linker is used to embed the SONAME (which includes the major version) and the symbol versions (from the installed glibc) in the executable.  For software that only needs to run on selected systems with the same or future versions of glibc, this works. (Note that fpc has a problem here because there is not a new header conversion for each release of glibc.  So there could be mismatches and hard to find bugs if fpc uses old header declarations with new versions of symbols in glibc) For software that needs to run on a variety of distros (which might include older ones), a developer might do the following: Develop and test on a modern distro with a modern tool chain using the newest .so version and newest symbol versions exposed by glibc.Copy the project to an old distro and jump through hoops trying to get all the dependencies installed without conflicts and trying to get it to build cleanly.  The executable gets the older .so version and older symbol versions embedded by the linker.Perform little or no testing on the old distro since it's main purpose is to 1) use the correct C headers for the older glibc and 2) get the older symbol versions embedded in the executable. (some developers have crafted a VM or container specifically setup with an old glibc for building) For fpc, I think a better way is to: Develop and test on a modern distro with a modern tool chain and recent glibc using a specified SONAME which includes the major version and specified (older) symbol versions which are still exposed by the recent glibc.done. There is no need to craft a special environment just for building and no need to copy the project to an old distro and jump through hoops.  The executable should already be compatible with new and older glibc releases.  (of course testing would be important on any distro you plan to target) (EDITED to clarify the the SONAME of a library includes the major version. According to semantic versioning, the "Major version X (X.y.z | X > 0) MUST be incremented if any backward incompatible changes are introduced to the public API.") ---------------------------------------------------------------------- Fred vS · 2024-10-04 19:36 Quote from: WayneSherman on October 04, 2024, 09:24:56 pmFor fpc, I think a better way is to: Develop and test on a modern distro with a modern tool chain and recent glibc using a specified .so version and specified (older) symbol versions which are still exposed by the recent glibc.done. There is no need to craft a special environment just for building and no need to copy the project to an old distro and jump through hoops.  The executable should already be compatible with new and older glibc releases.  (of course testing would be important on any distro you plan to target) The problem is that to know what version is glibc you need to load it first then get the version number. And it must be done before compiling. ---------------------------------------------------------------------- WayneSherman · 2024-10-04 19:44 Quote from: Fred vS on October 04, 2024, 08:17:51 pmQuote from: WayneSherman on October 04, 2024, 06:32:21 pm2. Do you think the fpc and rtl glibc source code declarations should not specify any versions and allow the compiled binaries to get specific .so file version and symbol versions from the static linker at build time (which uses the glibc library present on the system at build time)? Sorry to be the boring ones again but there is no specific .so file for libc. There is libc.so.6 and each new version has still the same name libc.so.6 but with new symbols tables. Quote(from ChatGPT) "The transition from libc.so.5 (which belonged to the older Linux libc) to libc.so.6 (which belongs to GNU C Library or glibc) occurred in 1997. This was a major shift in the Linux ecosystem and coincided with the release of glibc 2.0. This transition required programs that had been compiled against libc.so.5 to be recompiled against the newer libc.so.6." The reason the .so filename version does not have to change often is because internal symbol versioning has worked well for API/ABI changes.  In the future, major changes might justify another file name change to libc.so.7.  But a change in the major version signifies an API change so it isn't safe to assume that libc.so.7 would be drop-in compatible with libc.so.6. ---------------------------------------------------------------------- Fred vS · 2024-10-04 19:58 The last, after this, I leave you in peace. Our friend said: “Talk is cheap. Show me the code.” In last release of LazPaint: https://github.com/bgrabitmap/lazpaint/releases you may test, on very old and very new Linux distros, this: lazpaint7.2.2_linux64_timeless_no_install.tar.gz It uses signed symbol @GLIBC_2.2.5. for all glibc methods and link with 'libdl.so.2'.