using musl-libc instead of glibc lazarus / 32-Linux ---------------------------------------------------------------------- robert rozee · 2023-06-15 14:41 hi,     has anyone tried running FPC/Lazarus on a machine that uses musl instead of glibc? musl being a glibc alternative: https://musl.libc.org/ i've just spent 6 hours or so trying to get a couple of distros that use musl running in a VM: first trying Gentoo Linux (https://www.gentoo.org/) and then Void Linux (https://voidlinux.org/). Gentoo proved hopeless as the installing into a VM was incredibly complicated (for a non-expert), while Void was far more civilized but once installed (using the guide here: https://linuxiac.com/void-linux-installation/) refused to boot - just going to a blank screen a few seconds into the boot process. i'm quite interested in musl as (1) it offers the opportunity to build a FPC/Lazarus application against a 'libc lookalike' without the burden of versioned symbols, and (2) there is the possibility of static linking against musl to produce a binary that, although larger, is not libc-dependent. i'm curious to see how much larger the resultant binary really is, musl apparently being far more modular that glibc and so affording the chance of having minimum extra code pulled in. any pointers, particularly getting Void Linux running in a VM, would be much appreciated. also on persuading FPC/Lazarus to link against musl for those libc routines that are called directly from pascal sources. cheers, rob   :-) ---------------------------------------------------------------------- marcov · 2023-06-15 14:44 Free Pascal default doesn't link to any libc on Linux. Just do a ldd ./ppc386 or ppcx64 ---------------------------------------------------------------------- robert rozee · 2023-06-15 14:45 ... unless your application has a GUI (ie, uses the LCL) or you use threading. or any other stuff that requires libc. for example, dumped from the binary of a simple large-display digital clock: Code: [Select]user@DH61BE:~/pascal/tests/test 206 (3 simple demos)$ objdump -T project1 | grep 'GLIBC' 0000000000000000      DF *UND* 0000000000000000 (GLIBC_2.2.5) wcrtomb 0000000000000000      DF *UND* 0000000000000000 (GLIBC_2.2.5) wcscoll 0000000000000000      DF *UND* 0000000000000000 (GLIBC_2.34) dlerror 0000000000000000      DF *UND* 0000000000000000 (GLIBC_2.2.5) setenv 0000000000000000      DF *UND* 0000000000000000 (GLIBC_2.2.5) setlocale 0000000000000000      DF *UND* 0000000000000000 (GLIBC_2.2.5) strcoll 0000000000000000      DF *UND* 0000000000000000 (GLIBC_2.34) dlopen 0000000000000000      DF *UND* 0000000000000000 (GLIBC_2.2.5) iconv_close 0000000000000000      DF *UND* 0000000000000000 (GLIBC_2.2.5) mbrlen 0000000000000000      DF *UND* 0000000000000000 (GLIBC_2.34) dlclose 0000000000000000      DF *UND* 0000000000000000 (GLIBC_2.2.5) mbrtowc 0000000000000000      DF *UND* 0000000000000000 (GLIBC_2.34) __libc_start_main 0000000000000000      DF *UND* 0000000000000000 (GLIBC_2.34) dlsym 0000000000000000      DF *UND* 0000000000000000 (GLIBC_2.2.5) iconv 0000000000000000      DF *UND* 0000000000000000 (GLIBC_2.2.5) iconv_open 0000000000000000      DF *UND* 0000000000000000 (GLIBC_2.2.5) nl_langinfo 0000000000000000      DF *UND* 0000000000000000 (GLIBC_2.2.5) towlower 0000000000000000      DF *UND* 0000000000000000 (GLIBC_2.34) dladdr 0000000000000000      DF *UND* 0000000000000000 (GLIBC_2.2.5) __errno_location 0000000000000000      DF *UND* 0000000000000000 (GLIBC_2.2.5) towupper 0000000000000000      DF *UND* 0000000000000000 (GLIBC_2.2.5) sched_yield 0000000000000000  w   DF *UND* 0000000000000000 (GLIBC_2.2.5) __cxa_finalize user@DH61BE:~/pascal/tests/test 206 (3 simple demos)$ ---------------------------------------------------------------------- AmatCoder · 2023-06-15 16:41 Some time ago I used FPC with musl in Alpine Linux. FPC is packaged there: https://pkgs.alpinelinux.org/package/edge/testing/x86_64/fpc Lazarus is not packaged but I was able to compile it with a few patchs. But I should warn you: AFAICR, you can not compile a static build of a program that requires libc with FreePascal and musl. Because FPC uses dlopen/dlsym under the hood and musl does not support dlopen and static linking at the same time. ---------------------------------------------------------------------- Thaddy · 2023-06-15 16:49 FPC does not need any libc as Marcov explained. Lazarus does... Anyway, what I would have tried in my youth is copy over the old libc to a safe location and rename musl-libc with the old name. That only works if the API is the exact same. Maybe a worthwile attempt? Make sure the original can be placed back. ---------------------------------------------------------------------- robert rozee · 2023-06-15 17:17 Quote from: Thaddy on June 15, 2023, 06:49:19 pmFPC does not need any libc as Marcov explained. Lazarus does... if i talk about Lazarus without mention of FPC, people will start complaining that Lazarus is just 'a library and IDE' that are add-ons to FPC. hence why i write FPC/Lazarus, or if you prefer Lazarus/FPC. i do hope we all know what i am talking about - a GUI application compiled with the FPC compiler into a (linux) binary that then depends upon glibc's versioned symbols being 'just so right' that the program can run. ---------------------------------------------------------------------- AmatCoder · 2023-06-15 17:38 One moment... Do you want to compile (and dynamically link) a program against musl and then use it on glibc systems? Because that will never work...They are incompatible. ---------------------------------------------------------------------- DonAlfredo · 2023-06-15 18:09 I use Alpine Linux running in a VM on windows11. Fpcupdeluxe for musl runs fine on this OS. ---------------------------------------------------------------------- robert rozee · 2023-06-15 18:22 Quote from: AmatCoder on June 15, 2023, 07:38:13 pmOne moment... Do you want to compile (and dynamically link) a program against musl and then use it on glibc systems? Because that will never work...They are incompatible. why are they incompatible? i thought one of the main ideas of musl was to provide a work-alike replacement for glibc - and that if you statically link to musl then you'd end up with a binary image that requires no libc on your target computer? after all, libc is just an abstraction layer that sits between the kernel and applications (and libraries that use it). as far as i've been able to determine there is nothing special in libc that can not be accomplished by talking directly to the linux kernel. so far: i've got Void Linux running, with a Gnome desktop. Gnome is pretty awful, i feel it may be a creation of Fisher-Price! i've managed to get FPC and Lazarus .rpm packages downloaded on the vm, and 'sort of' installed, but not working. DonAlfredo: i think i'll have a look at Alpine Linux + fpcupdeluxe next. cheers, rob   :-) ---------------------------------------------------------------------- AmatCoder · 2023-06-15 19:19 Do you read my posts? I write them again: If you statically link to musl: QuoteAFAICR, you can not compile a static build of a program that requires libc with FreePascal and musl. Because FPC uses dlopen/dlsym under the hood and musl does not support dlopen and static linking at the same time. If you dynamically link to musl:QuoteOne moment... Do you want to compile (and dynamically link) a program against musl and then use it on glibc systems? Because that will never work...They are incompatible. ---------------------------------------------------------------------- robert rozee · 2023-06-16 04:42 Quote from: AmatCoder on June 15, 2023, 09:19:48 pmDo you read my posts? i did read them, including the very interesting linked post from 10 or so years ago (https://www.openwall.com/lists/musl/2012/12/08/4) that states "Presently, it does not work at all" (my underlining). the post highlights some of the issues that i'm still thinking through, and i'm interested in finding out if the 'does not work at all' still holds a decade later. in your earlier posts you qualified your statements with "AFAICR", which i take it means "as far as i can recall". i interpreted this as in the context of "as far as i can recall... the offender was driving a red car" (with the possibility of the 'fact' being false), whereas you seem to have intended the context of "as far as i can recall... 2+2=4" (a more-or-less absolute fact). my apologies for misinterpreting your meaning of "AFAICR". i can see some of the issues surrounding the use of dlopen() et al, but am not convinced they are insurmountable. certainly difficult, of that there is no doubt, but then most interesting problems are difficult. in an ideal world i'd like to see the functionality of dlopen() et al implemented in pascal - after all, these libraries that dlopen() et al opens are just ELF files. but there is an interesting question of: can/do libraries themselves make use of dlopen() internally, or can dlopen() only be called by the user application? and if a library uses dlopen() internally to access another library, does this percolate up to an explicit entry in the application's Dynamic Symbol Table? does the library in fact instruct the application to make use of dlopen() instead of doing it itself? Thaddy: i'd not be anywhere near brave enough to do what you suggest! what i am thinking of (ultimately) is ways to hand-pick pieces of musl-libc and include them directly into the Lazarus/FPC libraries where required to replace the current calls to (the small subset of problematic) libc routines. static linking of musl is just one step on the path to this end. the 'EULA' of glibc precludes doing this with the GNU code, however the musl license seems to be more permissive. cheers, rob   :-) ---------------------------------------------------------------------- robert rozee · 2023-06-16 14:43 a little further digging leads to this project, that points to a solution: https://github.com/pfalcon/foreign-dlopen foreign-dlopen "There's another model, which we'll call "FFI (Foreign Function Interface) model". It goes along the lines of: 1.    Suppose you have a perfect, closed world application. Statically linked of course. 2.    But you want to go out to dirty bustling outside world (in other words, let your application, or users of your appplication, to dlopen() outside shared libraries). 3.    There're absolutely no expectations or stipulations about which libc is used by those shared libraries. In particular, there's no expectations that libc of your application and external shared lib are the same. Or that you know which libc is used by external lib at all. For example, your static binary may be linked against musl libc, but you may want to load (allow to load) glibc bloat lying in abundance on a typical Linux system. Again, the only thing you want is to maintain your static perfect world, independent from outside hustle. But, at user discretion, you want to allow this hustling outside world into your address space, by means of dlopen(). This cute project is a proof-of-concept solution for this usecase." interesting? cheers, rob   :-) ---------------------------------------------------------------------- PascalDragon · 2023-06-16 21:51 Quote from: robert rozee on June 15, 2023, 08:22:22 pmQuote from: AmatCoder on June 15, 2023, 07:38:13 pmOne moment... Do you want to compile (and dynamically link) a program against musl and then use it on glibc systems? Because that will never work...They are incompatible. why are they incompatible? You can't have two dynamic C libraries on your at once (or at least it's very hard to set up). The only thing you could do is to use something like AppImage to bundle your binary and all dependencies (because you'd replace the C library, thus any and every library you use needs to be recompiled against the musl libc as well), because it would then use different library search paths. Quote from: robert rozee on June 16, 2023, 06:42:22 ami can see some of the issues surrounding the use of dlopen() et al, but am not convinced they are insurmountable. certainly difficult, of that there is no doubt, but then most interesting problems are difficult. in an ideal world i'd like to see the functionality of dlopen() et al implemented in pascal - after all, these libraries that dlopen() et al opens are just ELF files. but there is an interesting question of: can/do libraries themselves make use of dlopen() internally, or can dlopen() only be called by the user application? and if a library uses dlopen() internally to access another library, does this percolate up to an explicit entry in the application's Dynamic Symbol Table? does the library in fact instruct the application to make use of dlopen() instead of doing it itself? If an application is statically linked it's essentially missing all named symbols that would be required for dlopen/dlsym to work. Also if you open another library it would need to dynamically link in the C library (because the opened library needs it) and this would then mean that you'd have the C library state twice in your application which is a recipe for desaster. Quote from: robert rozee on June 16, 2023, 04:43:50 pma little further digging leads to this project, that points to a solution: https://github.com/pfalcon/foreign-dlopen foreign-dlopen [snip] interesting? This sounds needlessly complicated and prone to failure. Quote from: Thaddy on June 15, 2023, 06:49:19 pmAnyway, what I would have tried in my youth is copy over the old libc to a safe location and rename musl-libc with the old name. That only works if the API is the exact same. Maybe a worthwile attempt? Make sure the original can be placed back. Due to the versioned symbols that current glibc uses this is not possible. ---------------------------------------------------------------------- robert rozee · 2023-06-17 14:47 Quote from: PascalDragon on June 16, 2023, 11:51:26 pmQuote from: robert rozee on June 16, 2023, 04:43:50 pma little further digging leads to this project, that points to a solution: https://github.com/pfalcon/foreign-dlopen foreign-dlopen [snip] interesting? This sounds needlessly complicated and prone to failure. many things are prone to failure, but sometimes we are surprised when they don't. i was thinking of the part where pfalcon et al use a 'helper application' to retrieve the needed entry points to glibc and pass them back to the user program to be used to gain access libc.so functions without the user program needing to be dynamically linked to libc itself. the nice thing is that the helper application could potentially be very small, a few dozen or so lines of C code that the user program: (1) writes to a .c file in /tmp, (2) compiles by calling gcc, then (3) runs to retrieve the needed entry points. likely these steps could be accomplished (including running gcc) in a fraction of a second every time the user program is run: https://github.com/pfalcon/foreign-dlopen/blob/master/src/fdlhelper.c but then i've stumbled across something perhaps far easier here: https://developers.redhat.com/blog/2019/08/01/how-the-gnu-c-library- handles-backward-compatibility (my search inspired by https://news.ycombinator.com/item?id=23745297) what if FPC scanned through libc.so.6 each time before doing a compile, using: Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i readelf --dyn-syms -W /lib/x86_64-linux-gnu/libc.so.6to collect a list of all the symbols defined within - including symbols with multiple versions available? FPC could then just use a set of 'earliest defined' symbols to defeat using the default (latest), or select symbols that fall within a 'known good range' of glibc versions. for instance, take the symbol __libc_start_main: Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i readelf --dyn-syms -W /lib/x86_64-linux-gnu/libc.so.6 | grep "start_main"   674: 0000000000029dc0   328 FUNC    GLOBAL DEFAULT   15 __libc_start_main@@GLIBC_2.34   678: 0000000000029dc0   328 FUNC    GLOBAL DEFAULT   15 __libc_start_main@GLIBC_2.2.5(from my system's 'Ubuntu GLIBC 2.35-0ubuntu3.1') now substitute in /usr/share/fpcsrc/3.2.2/rtl/linux/x86_64/si_c.inc and si_g.inc: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure libc_start_main; external name '__libc_start_main@GLIBC_2.2.5';for Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure libc_start_main; external name '__libc_start_main';(well, not substitute, just change si_c.inc et al to pick up the string from an include file created from the readelf data) as long as the version number used exists (which we have just checked using readelf) the linker should (as far as i can see, although i am far from an expert) be happy. we end up with a binary file that will run against an early version of glibc without needing to build on an 'old' VM. would this work? cheers, rob   :-) ---------------------------------------------------------------------- PascalDragon · 2023-06-18 20:16 Quote from: robert rozee on June 17, 2023, 04:47:48 pmwhat if FPC scanned through libc.so.6 each time before doing a compile, using: Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i readelf --dyn-syms -W /lib/x86_64-linux-gnu/libc.so.6to collect a list of all the symbols defined within - including symbols with multiple versions available? FPC could then just use a set of 'earliest defined' symbols to defeat using the default (latest), or select symbols that fall within a 'known good range' of glibc versions. for instance, take the symbol __libc_start_main: Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i readelf --dyn-syms -W /lib/x86_64-linux-gnu/libc.so.6 | grep "start_main"   674: 0000000000029dc0   328 FUNC    GLOBAL DEFAULT   15 __libc_start_main@@GLIBC_2.34   678: 0000000000029dc0   328 FUNC    GLOBAL DEFAULT   15 __libc_start_main@GLIBC_2.2.5(from my system's 'Ubuntu GLIBC 2.35-0ubuntu3.1') now substitute in /usr/share/fpcsrc/3.2.2/rtl/linux/x86_64/si_c.inc and si_g.inc: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure libc_start_main; external name '__libc_start_main@GLIBC_2.2.5';for Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure libc_start_main; external name '__libc_start_main';(well, not substitute, just change si_c.inc et al to pick up the string from an include file created from the readelf data) No, we don't want something like that. The issue can be easily fixed by compiling on a system with the suitable version of the libary instead of implemting some frickle scheme. Quote from: robert rozee on June 17, 2023, 04:47:48 pmas long as the version number used exists (which we have just checked using readelf) the linker should (as far as i can see, although i am far from an expert) be happy. we end up with a binary file that will run against an early version of glibc without needing to build on an 'old' VM. would this work? And what if the user does not want the old behavior? What if the new function fixed some critical functionality and by using the old one you'll run into problems because code you expect to work against the new behavior will run into problems? The __libc_start_main is exactly such an example. ---------------------------------------------------------------------- robert rozee · 2023-06-19 14:51 Quote from: PascalDragon on June 18, 2023, 10:16:27 pmNo, we don't want something like that. The issue can be easily fixed by compiling on a system with the suitable version of the libary instead of implemting some frickle scheme. google tells me that "A frickle is a fried pickle originating in America's south but now seen on the menus of hip urban pubs and cafes in Australia and New Zealand. Our modern interpretation of this addictive snack is accompanied by a sriracha (Vietnamese chilli sauce) spiked mayonnaise". although i'm not quite sure you mean this - can you please clarify your use of the term? also, who is the 'we' you mean in "we don't want something like that"? in this country, New Zealand, 'we' refers to the sitting monarch. again, this might mean something different where you live. please clarify. users of FPC/Lazarus seem to have been searching for a solution to the 'libc problem' for years, and this seems to be one that operates entirely within how glibc's authors intended. glibc contains multiple versions of symbols to cater for this very situation, and FPC is quite conservative in the use of libc features. if it were a problem due to FPC doing something 'weird' under the hood, then FPC would surely be designed to refuse to compile unless a minimum compatible version of libc was detected? Quote from: PascalDragon on June 18, 2023, 10:16:27 pmAnd what if the user does not want the old behavior? What if the new function fixed some critical functionality and by using the old one you'll run into problems because code you expect to work against the new behavior will run into problems? The __libc_start_main is exactly such an example. "What if"... what if the user somehow manages to poke their eye out when using the mouse, or pours a hot cup of tea into their computer, or the sky suddenly falls?! there is only so much cotton wool and bubblewrap that can be placed between the user and the consequences of their actions. perhaps we should require a competency test and submission of three sworn written references before a user is allowed to download FPC/Lazarus in the first place. i'm placing at the top of each source file: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i {$INCLUDE /usr/share/fpcsrc/3.2.2/LCsymbol.inc} and in the body code blocks of the form: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i {$IF DECLARED(libcSTARTM)}  procedure libc_start_main; external name libcSTARTM;{$ELSE}  procedure libc_start_main; external name '__libc_start_main';{$ENDIF} the file LCsymbol.inc will contain: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i const libcSTARTM =  '__libc_start_main@GLIBC_2.2.5';            // version numbers picked to match glibc at time of file creation      libcDLOPEN =  'dlopen@GLIBC_2.2.5';      libcDLSYM  =  'dlsym@GLIBC_2.2.5';      libcDLADDR =  'dladdr@GLIBC_2.2.5';       libcDLERROR = 'dlerror@GLIBC_2.2.5';        libcDLCLOSE = 'dlclose@GLIBC_2.2.5'; if the user wants to go back to original behavior (using the latest version of each symbol) they merely needs to replace LCsymbol.inc with an empty file - or delete the file if i can figure out how to $INCLUDE conditionally on the file existing. after making any changes to the contents of LCsymbol.inc it will then be necessary for the user to rebuild the FPC source tree. once i've got it working, i'll write a GUI application that gets all the available symbol versions (for __libc_start_main, dlopen, dlsym, dladdr, dlerror, and dlclose) and present the user with the options of: default, earliest, matching set. the user's choice can then be written to LCsymbol.inc and a rebuild of the FPC source tree initiated. speaking of which - under linux how do you rebuild the FPC source tree? and how do you optionally {$INCLUDE ...} a file that may or may not exist? cheers, rob   :-) ---------------------------------------------------------------------- marcov · 2023-06-19 15:32 Quote from: robert rozee on June 19, 2023, 04:51:40 pmusers of FPC/Lazarus seem to have been searching for a solution to the 'libc problem' for years, Yes. As Windows users they expect the same freedoms on Linux as they have on Windows with backwards and forwards exe compatibility. That's why it comes up each time again. Quoteand this seems to be one that operates entirely within how glibc's authors intended. glibc contains multiple versions of symbols to cater for this very situation, and FPC is quite conservative in the use of libc features. If so, it would be good to have a report of an intermediate user that pioneers this feature and evaluate it long time if it actually is the panacea for binary compatibility. Maybe it is in a narrow sense, but I doubt it will be of universal use.  There are more issues than  just symbol versioning: struct versioningstartup codedynamic loader behaviourcompatibility breaks papered over with macros the list goes on. Quoteif it were a problem due to FPC doing something 'weird' under the hood, then FPC would surely be designed to refuse to compile unless a minimum compatible version of libc was detected? What is weird or not, is not terribly well defined. Basically they just do a change, patch up just enough so that enough remains working with gcc,  and the rest can go hang. So there is no specification (Posix is too narrow for non trivial stuff), and no compatibility guarantee. That makes it very hard for others.  The problem is also that there is no mechanism like Windows (where you can submit a binary that doesn't work with a new OS, and it is actually looked at), and most APIs are supported wrt backwards compatibility. (*) The base system maintainers simply close everything, in all those 25 years I can't remember one single occurrence where they fixed a breakage at their end. Quote from: robert rozee on June 19, 2023, 04:51:40 pm"What if"... what if the user somehow manages to poke their eye out when using the mouse, or pours a hot cup of tea into their computer, or the sky suddenly falls?! there is only so much cotton wool and bubblewrap that can be placed between the user and the consequences of their actions. perhaps we should require a competency test and submission of three sworn written references before a user is allowed to download FPC/Lazarus in the first place. If you up the requirements for minimal user, you can just let the exceptional cases build and modify their own. Quotei'm placing at the top of each source file: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i {$INCLUDE /usr/share/fpcsrc/3.2.2/LCsymbol.inc} The RTL is already precompiled and won't change with that. ---------------------------------------------------------------------- PascalDragon · 2023-06-19 20:01 Quote from: robert rozee on June 19, 2023, 04:51:40 pmgoogle tells me that "A frickle is a fried pickle originating in America's south but now seen on the menus of hip urban pubs and cafes in Australia and New Zealand. Our modern interpretation of this addictive snack is accompanied by a sriracha (Vietnamese chilli sauce) spiked mayonnaise". although i'm not quite sure you mean this - can you please clarify your use of the term? I meant fickle. Quote from: robert rozee on June 19, 2023, 04:51:40 pmalso, who is the 'we' you mean in "we don't want something like that"? in this country, New Zealand, 'we' refers to the sitting monarch. again, this might mean something different where you live. please clarify. Well, the “sitting monarch” isn't that wrong, cause I mean the “we” in the sense of us FPC core developers. Quote from: robert rozee on June 19, 2023, 04:51:40 pmusers of FPC/Lazarus seem to have been searching for a solution to the 'libc problem' for years, and this seems to be one that operates entirely within how glibc's authors intended. glibc contains multiple versions of symbols to cater for this very situation, and FPC is quite conservative in the use of libc features. if it were a problem due to FPC doing something 'weird' under the hood, then FPC would surely be designed to refuse to compile unless a minimum compatible version of libc was detected? No, because FPC does not do any analysis of the involved binaries. It leaves all that to the linker which is GNU ld on Linux. Quote from: robert rozee on June 19, 2023, 04:51:40 pmQuote from: PascalDragon on June 18, 2023, 10:16:27 pmAnd what if the user does not want the old behavior? What if the new function fixed some critical functionality and by using the old one you'll run into problems because code you expect to work against the new behavior will run into problems? The __libc_start_main is exactly such an example. "What if"... what if the user somehow manages to poke their eye out when using the mouse, or pours a hot cup of tea into their computer, or the sky suddenly falls?! there is only so much cotton wool and bubblewrap that can be placed between the user and the consequences of their actions. perhaps we should require a competency test and submission of three sworn written references before a user is allowed to download FPC/Lazarus in the first place. As a developer of a development system one has to keep those “what if” situations in mind as much as possible, cause users will abuse such systems and thus they need to be as robust as reasonably possible. And especially here it's more likely that users would want the more modern, potentially bug free behavior than an older one, especially if they don't care about the support of older Linux distributions. Quote from: robert rozee on June 19, 2023, 04:51:40 pmspeaking of which - under linux how do you rebuild the FPC source tree? make clean all Quote from: robert rozee on June 19, 2023, 04:51:40 pmand how do you optionally {$INCLUDE ...} a file that may or may not exist? You can't. ---------------------------------------------------------------------- robert rozee · 2023-06-20 04:42 Quote from: marcov on June 19, 2023, 05:32:36 pmMaybe it is in a narrow sense, but I doubt it will be of universal use.  There are more issues than  just symbol versioning: struct versioningstartup codedynamic loader behaviourcompatibility breaks papered over with macros my solution, if it works, is narrowly focused on address just SIX versioned symbols from the ELF Dynamic Symbol Table of compiled linux binaries: __libc_start_main, dlopen, dlsym, dladdr, dlerror, dlclose. currently these are versioned to "2.34", my proposal changes the versioning to "2.2.5". this pushes back the 'drop-dead' date of FPC/Lazarus for linux by about 10 years - an estimate based upon how versioning has been handled in the past. it is NOT a cure, it is a treatment, a 'band-aid' if you like, but nonetheless gives more time to find a permanent solution. Quote from: PascalDragon on June 19, 2023, 10:01:46 pmWell, the “sitting monarch” isn't that wrong, cause I mean the “we” in the sense of us FPC core developers ummm... my reference to a monarch was NOT intended in a positive sense!  Quote from: PascalDragon on June 19, 2023, 10:01:46 pmQuote from: robert rozee on June 19, 2023, 04:51:40 pmspeaking of which - under linux how do you rebuild the FPC source tree?make clean all interesting - another forum member sent me a slightly different solution. create a script in the FPC root directory (/usr/share/fpcsrc/3.2.2) containing: Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i #!/bin/shCOMPILER=fpcmake cleanmake all FPC=$COMPILER OPT="-Fl/usr/local/lib"make FPC=$COMPILER install INSTALL_PREFIX=/usr/ what is the material difference between their solution and yours? i am using a stock-standard install of FPC and Lazarus from the three .deb files downloaded from https://sourceforge.net/projects/lazarus/files/L azarus%20Linux%20amd64%20DEB/Lazarus%202.2.6/. the FPC root directory is owned by 'root' and so anything run on it needs to be done under sudo. Quote from: PascalDragon on June 19, 2023, 10:01:46 pmQuote from: robert rozee on June 19, 2023, 04:51:40 pmand how do you optionally {$INCLUDE ...} a file that may or may not exist? You can't. i suspected as much, but was hoping otherwise. the documentation for $INCLUDE says: Quote1.2.41  $I or $INCLUDE : Include file The {$I filename} or {$INCLUDE filename} directive tells the compiler to read further statements from the file filename. The statements read there will be inserted as if they occurred in the current file. If the file with the given filename exists, it will be included. the words "If the file with the given filename exists" gave me hope that if the file did NOT exist then compiling would (or could be made to) continue on unimpeded. cheers, rob   :-) ---------------------------------------------------------------------- TRon · 2023-06-20 09:59 Quote from: robert rozee on June 20, 2023, 06:42:33 amQuote from: PascalDragon on June 19, 2023, 10:01:46 pmQuote from: robert rozee on June 19, 2023, 04:51:40 pmspeaking of which - under linux how do you rebuild the FPC source tree?make clean all interesting - another forum member sent me a slightly different solution. create a script in the FPC root directory (/usr/share/fpcsrc/3.2.2) containing: Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i #!/bin/shCOMPILER=fpcmake cleanmake all FPC=$COMPILER OPT="-Fl/usr/local/lib"make FPC=$COMPILER install INSTALL_PREFIX=/usr/ what is the material difference between their solution and yours? PascalDragon did not "install" the compiler (missing make install option). Perhaps silently he did assume you would do so yourself with make install (in which case the make file uses the default options to install the compiler for you. What location that exactly is and perhaps other tidbits it performs during the makefile install target depends on your host) Semantically they are almost the same except that PascalDragon's solution combines the make "targets" into one command. The "another forum member" providing the solution could just as well have wrote to you: Code: [Select]make clean all install FPC=$COMPILER OPT="-Fl/usr/local/lib" INSTALL_PREFIX=/usr/ Another significant difference is that the "another forum member" lets you specifically specify the compiler used for the build process while PascalDragon's solution implicitly tells to use any compiler for the build that the build script is able to find/locate (system path, alias, etc.) But since "another forum user" defines the compiler as being "fpc" there is no  practical difference. Note that this is usually referred to as "bootstrap compiler". I do not know the significance of "-F/usr/local/lib" that "another forum member" recommends to use because for me that directory is empty but it is to tell the compiler where to look/search for libraries as it represents the library path (see also https://www.freepascal.org/docs-html/user/userap1.html). ---------------------------------------------------------------------- robert rozee · 2023-06-20 17:18 problem solved: Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i user@user- VirtualBox:~/pascal/test 1$ ldd --versionldd (Ubuntu GLIBC 2.35-0ubuntu3.1) 2.35Copyright (C) 2022 Free Software Foundation, Inc.This is free software; see the source for copying conditions.  There is NOwarranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.Written by Roland McGrath and Ulrich Drepper.user@user-VirtualBox:~/pascal/test 1$ user@user- VirtualBox:~/pascal/test 1$ objdump -T project1 | grep "("0000000000000000      DF *UND*  0000000000000000 (GLIBC_2.2.5) wcrtomb0000000000000000      DF *UND*  0000000000000000 (GLIBC_2.2.5) wcscoll0000000000000000      DF *UND*  0000000000000000 (GLIBC_2.2.5) setenv0000000000000000      DF *UND*  0000000000000000 (GLIBC_2.2.5) dlclose0000000000000000      DF *UND*  0000000000000000 (GLIBC_2.2.5) setlocale0000000000000000      DF *UND*  0000000000000000 (GLIBC_2.2.5) strcoll0000000000000000      DF *UND*  0000000000000000 (GLIBC_2.2.5) iconv_close0000000000000000      DF *UND*  0000000000000000 (GLIBC_2.2.5) mbrlen0000000000000000      DF *UND*  0000000000000000 (GLIBC_2.2.5) dlerror0000000000000000      DF *UND*  0000000000000000 (GLIBC_2.2.5) mbrtowc0000000000000000      DF *UND*  0000000000000000 (GLIBC_2.2.5) dlopen0000000000000000      DF *UND*  0000000000000000 (GLIBC_2.2.5) iconv0000000000000000      DF *UND*  0000000000000000 (GLIBC_2.2.5) __libc_start_main0000000000000000      DF *UND*  0000000000000000 (GLIBC_2.2.5) iconv_open0000000000000000    DF *UND*  0000000000000000 (GLIBC_2.2.5) nl_langinfo0000000000000000      DF *UND*  0000000000000000 (GLIBC_2.2.5) towlower0000000000000000      DF *UND*  0000000000000000 (GLIBC_2.2.5) dlsym0000000000000000      DF *UND*  0000000000000000 (GLIBC_2.2.5) __errno_location0000000000000000      DF *UND*  0000000000000000 (GLIBC_2.2.5) towupper0000000000000000      DF *UND*  0000000000000000 (GLIBC_2.2.5) sched_yield0000000000000000  w   DF *UND*  0000000000000000 (GLIBC_2.2.5) __cxa_finalizeuser@user- VirtualBox:~/pascal/test 1$ the above lazarus GUI application was compiled on a linux mint 21.1 64-bit VM. cheers, rob   :-) addendum: well, i may have spoken too quick! tried running the binary on a linux mint 19 64-bit VM and it fails with: Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i user@Mint19:~/Desktop$ ./project1./project1: relocation error: ./project1: symbol dlopen version GLIBC_2.2.5 not defined in file libc.so.6 with link time referenceuser@Mint19:~/Desktop$ a couple of quick checks reveal: Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i user@Mint19:~/Desktop$ readelf --dyn-syms -W /lib/x86_64-linux- gnu/libc.so.6 | grep "__libc_start_main"  2206: 0000000000021ba0   446 FUNC    GLOBAL DEFAULT   13 __libc_start_main@@GLIBC_2.2.5user@Mint19:~/Desktop$ user@Mint19:~/Desktop$ readelf --dyn-syms -W /lib/x86_64-linux- gnu/libc.so.6 | grep "dlclose"  1266: 0000000000166620   168 FUNC    GLOBAL DEFAULT   13 __libc_dlclose@@GLIBC_PRIVATEuser@Mint19:~/Desktop$ so one problem has been replaced with another! on the plus side, __libc_start_main is versioned correctly, so i just need to find a workaround for the remaining five (dlsym, dladdr, dlclose, dlerror, and dlopen). i have a few ideas... ---------------------------------------------------------------------- marcov · 2023-06-20 17:48 Quote from: TRon on June 20, 2023, 11:59:21 amI do not know the significance of "-F/usr/local/lib" that "another forum member" recommends to use because for me that directory is empty but it is to tell the compiler where to look/search for libraries as it represents the library path (see also https://www.freepascal.org/docs- html/user/userap1.html). /usr/local/lib comes from BSD and is historically a prefix where third party (non package system or even non base system) libraries are stored. I think on Linux it matters less, except Slackware iirc adhered to it. The original buildfaq was mostly crafted on BSD, which might explain why it is so prevalent in buildscripts. For reference, my *nix build script that also contains it: Code: [Select]#!/usr/bin/env bash # simple i386/x86_64 building on linux/*BSD. # -j and -T factor.  Rule of thumb physical cores plus 1 or 2. Values above 4-6 have little effect, even if you have the cores. # there is not enough parallelism in the tree. CORES=2 HOSTARCH=`uname -m` HOSTOS=`uname -s` SRCDIR=/fpc/fpc DESTDIR=~/src/builded HAVEDEBUG=1 OVERRIDEDETECTION=0 HAVEOPT=0 LIBCRTL=0 ARGS=" -viwn -Sg -Fl/usr/local/lib  -dLZX_USETHREADS " OPTSETTINGS=" -O4 -Opcoreavx -Cpcoreavx  -Cfavx " DEBUGSETTING=" -gw2 -gl  -Xg  -godwarfsets " #---------------------- is64=0 START=ppcrel GENCOMPILER=ppc386 NEWOPT="" if [ $LIBCRTL = "1" ] ; then   ARGS+=" - dFPC_USE_LIBC " fi if [ $OVERRIDEDETECTION != 1 ]; then if [ "$HOSTARCH" = "x86_64" ] ; then   is64=1 fi if [ "$HOSTARCH" = "amd64" ] ; then   is64=1 fi fi if [ $is64 = 1 ]; then   START=ppcrel64   GENCOMPILER=ppcx64 fi if [ "$HAVEDEBUG" = "1" ] ; then   if [ "$ARGS" != "" ]; then    ARGS="$ARGS$DEBUGSETTING"   fi fi if [ "$HAVEOPT" = "1" ] ; then   $NEWOPT=$OPTSETTINGS fi GMAKE=gmake if [ $HOSTOS = "Linux" ] ; then   GMAKE=make fi JOPT="-j $CORES" FPMAKEOPT="-T $CORES" while getopts ":nj:" optname   do     case "$optname" in       "j")         echo Cores overriden to $OPTARG         JOPT="-j $OPTARG"         FPMAKEOPT="-T $OPTARG"         ;;       "n")          JOPT=""         ;;       *)         echo "Unknown error while processing options"         ;;     esac   done if [ $OVERRIDEDETECTION != 1 ]; then echo ARCH is $HOSTARCH on \(OS:\) $HOSTOS echo Build in $SRCDIR, installing in $DESTDIR with $CORES cores fi cd $SRCDIR $GMAKE $JOPT clean   FPMAKEOPT="$FPMAKEOPT" PP="$START" FPC="$START" OPT="$ARGS" NEWOPT="$NEWOPT" &> ~/cleanlog64 $GMAKE $JOPT  all  FPMAKEOPT="$FPMAKEOPT" PP="$START" FPC="$START" OPT="$ARGS" NEWOPT="$NEWOPT" &> ~/buildlog64 if [ ! $? = 0 ] ; then  echo DAMN, failed again else  $GMAKE install FPC=compiler/$GENCOMPILER OPT="$ARGS" NEWOPT=$NEWOPT INSTALL_PREFIX=$DESTDIR &> ~/installog  if [ ! $? = 0 ] ; then   echo INSTALL failed | tee ~/outcome  else   echo Everything _should_ be fine.| tee ~/outcome  fi fi ---------------------------------------------------------------------- TRon · 2023-06-20 18:07 Thank you for that explanation marcov. My distribution seems to omit it, as per FHS. ---------------------------------------------------------------------- marcov · 2023-06-20 19:20 Quote from: TRon on June 20, 2023, 08:07:46 pmThank you for that explanation marcov. My distribution seems to omit it, as per FHS. The equivalent FreeBSD page is man hier ---------------------------------------------------------------------- PascalDragon · 2023-06-20 19:56 Quote from: robert rozee on June 20, 2023, 06:42:33 amQuote from: PascalDragon on June 19, 2023, 10:01:46 pmWell, the “sitting monarch” isn't that wrong, cause I mean the “we” in the sense of us FPC core developers ummm... my reference to a monarch was NOT intended in a positive sense!  Doesn't matter how you meant it, it doesn't change the fact that we devs are acting as gate keepers to ideas that are not really feasible. Quote from: robert rozee on June 20, 2023, 06:42:33 amQuote from: PascalDragon on June 19, 2023, 10:01:46 pmQuote from: robert rozee on June 19, 2023, 04:51:40 pmspeaking of which - under linux how do you rebuild the FPC source tree?make clean all interesting - another forum member sent me a slightly different solution. create a script in the FPC root directory (/usr/share/fpcsrc/3.2.2) containing: Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i #!/bin/shCOMPILER=fpcmake cleanmake all FPC=$COMPILER OPT="-Fl/usr/local/lib"make FPC=$COMPILER install INSTALL_PREFIX=/usr/ what is the material difference between their solution and yours? As TRon said, my example doesn't include the installation step and combines the clean and all steps into a single call. As long as you don't use parallel invocation (by passing -j to make) there won't be a difference. If you use parallel invocation you should use the separate invocations from the other example. In essence you just need to invoke make with the correct parameters to compile FPC. See also here plus looking at documentation in how to invoke make in general. Quote from: robert rozee on June 20, 2023, 06:42:33 am i am using a stock-standard install of FPC and Lazarus from the three .deb files downloaded from https://sourceforge.net/projects/lazarus/files/Lazarus %20Linux%20amd64%20DEB/Lazarus%202.2.6/. the FPC root directory is owned by 'root' and so anything run on it needs to be done under sudo. You should not mess with your existing installation. Just download the FPC source (no matter if the development branch or a release) and build that. Quote from: robert rozee on June 20, 2023, 06:42:33 amQuote from: PascalDragon on June 19, 2023, 10:01:46 pmQuote from: robert rozee on June 19, 2023, 04:51:40 pmand how do you optionally {$INCLUDE ...} a file that may or may not exist? You can't. i suspected as much, but was hoping otherwise. the documentation for $INCLUDE says: Quote1.2.41  $I or $INCLUDE : Include file The {$I filename} or {$INCLUDE filename} directive tells the compiler to read further statements from the file filename. The statements read there will be inserted as if they occurred in the current file. If the file with the given filename exists, it will be included. the words "If the file with the given filename exists" gave me hope that if the file did NOT exist then compiling would (or could be made to) continue on unimpeded. The documentation could use some improvement then... ---------------------------------------------------------------------- robert rozee · 2023-06-21 03:35 Quote from: PascalDragon on June 20, 2023, 09:56:18 pmwe devs are acting as gate keepers to ideas well... that makes it even worse. in effect, you are saying that the developers are keeping certain information 'closed source'. certainly this is a violation of the GPL et al, if not in the letter of the law then certainly in spirit! is this stance any different to the high priests of ancient societies that used the control of knowledge to keep the population subservient? cheers, rob   :-) ---------------------------------------------------------------------- PascalDragon · 2023-06-22 19:36 Quote from: robert rozee on June 21, 2023, 05:35:05 amQuote from: PascalDragon on June 20, 2023, 09:56:18 pmwe devs are acting as gate keepers to ideas well... that makes it even worse. in effect, you are saying that the developers are keeping certain information 'closed source'. certainly this is a violation of the GPL et al, if not in the letter of the law then certainly in spirit! is this stance any different to the high priests of ancient societies that used the control of knowledge to keep the population subservient? Huh? Where did I write that we keep things “closed source”?! My point was that we don't let everything get into the language or the RTL. Functionality that never gets into the language/RTL is not part of the distribution and thus never part of the license. ---------------------------------------------------------------------- robert rozee · 2023-06-22 22:34 Quote from: PascalDragon on June 22, 2023, 09:36:58 pmQuote from: robert rozee on June 21, 2023, 05:35:05 amQuote from: PascalDragon on June 20, 2023, 09:56:18 pmwe devs are acting as gate keepers to ideas well... that makes it even worse. in effect, you are saying that the developers are keeping certain information 'closed source'. certainly this is a violation of the GPL et al, if not in the letter of the law then certainly in spirit! is this stance any different to the high priests of ancient societies that used the control of knowledge to keep the population subservient? Huh? Where did I write that we keep things “closed source”?! is "gate keepers to ideas" not just a more prosaic way of saying 'closed source'? ideas, knowledge, being withheld or suppressed because those ideas are deemed 'bad' by a select group of experts. am getting close to a workable solution of the symbol versioning issue. turns out that just a single line in each of  si_c.inc and si_g.inc needs changing, and a 'dummy' replacement for libdl.so dropping into each of your project directories. the end result is a binary that has all GLIBC symbols versioned to 2.2.5. it works under linux mint 21.1 (glibc 2.35), producing a binary that runs fine under mint 19.3 (glibc 2.27). other folks are testing with other distros. i've only been testing with a couple of relatively simple GUI application so far, but the results look promising. i'm hoping to create a shell script to make the necessary small changes, and am considering if it is worth creating two branches in /usr/share/fpcsrc of 3.2.2 (3.2.2_old and 3.2.2_new) with a symlink used to switch between them. cheers, rob   :-) ---------------------------------------------------------------------- marcov · 2023-06-23 14:53 Quote from: robert rozee on June 23, 2023, 12:34:41 am is "gate keepers to ideas" not just a more prosaic way of saying 'closed source'? ideas, knowledge, being withheld or suppressed because those ideas are deemed 'bad' by a select group of experts. Gate keepers also select people/things coming in. That's what I think PascalDragon hinted on. IOW what patches/features to integrate. ---------------------------------------------------------------------- robert rozee · 2023-06-24 14:15 Quote from: marcov on June 23, 2023, 04:53:12 pmGate keepers also select people/things coming in. That's what I think PascalDragon hinted on. IOW what patches/features to integrate. perhaps i am being a bit harsh... i can understand the need to avoid 'feature creep', where a product has more and more features added to meet (usually sales executive) demands. i've worked on hardware projects which suffered from this, to the point of nearly being cancelled by management after several years of continual additions. but at the same time, i do feel that (open source) developers should also be open about the internal workings of their projects. if someone asks "how can i modify it to do this", the developers should not say "that is a 'frickle scheme' (maintaining PascalDragon's keyboard mishap) but rather help out (where practical) anyone who wants to dig into the source code themselves. even if their changes will never go into the main source. case in point is the alterations i am working on with some other forum members at the moment. turns out it is relatively easy to modify FPC to build against any version of libc that the currently installed glibc supports. my glibc (version 2.35) has internal support for glibc 2.34 symbols, as well as a complete base set of glibc 2.2.5 symbols. the glibc 2.2.5 support is complete, and likely to be with glibc for many years to come. but i couldn't care less about the alterations being incorporated into the main source of FPC. all i want to do is create a set of patches that anyone else is free to incorporate into FPC if they wish to. is this unreasonable? cheers, rob   :-) ---------------------------------------------------------------------- PascalDragon · 2023-06-25 18:48 Quote from: robert rozee on June 23, 2023, 12:34:41 amQuote from: PascalDragon on June 22, 2023, 09:36:58 pmQuote from: robert rozee on June 21, 2023, 05:35:05 amQuote from: PascalDragon on June 20, 2023, 09:56:18 pmwe devs are acting as gate keepers to ideas well... that makes it even worse. in effect, you are saying that the developers are keeping certain information 'closed source'. certainly this is a violation of the GPL et al, if not in the letter of the law then certainly in spirit! is this stance any different to the high priests of ancient societies that used the control of knowledge to keep the population subservient? Huh? Where did I write that we keep things “closed source”?! is "gate keepers to ideas" not just a more prosaic way of saying 'closed source'? ideas, knowledge, being withheld or suppressed because those ideas are deemed 'bad' by a select group of experts. The ideas, the knowledge can still be out there, you are after all free to do your own fork of FPC and implement any changes you envision. With FPC however we are working on our vision of FPC and if there are ideas that agree with our vision then we'll integrate them, if not, we simply won't. But nobody stops you from implementing them yourself and distributing them yourself. This is true for any open source project. Just think about Linus Torvalds as the ultimate gatekeeper of Linux. If he sees a change that he thinks doesn't fit with how Linux is supposed to work, he'll refuse the integration. Quote from: robert rozee on June 24, 2023, 04:15:50 pmbut at the same time, i do feel that (open source) developers should also be open about the internal workings of their projects. if someone asks "how can i modify it to do this", the developers should not say "that is a 'frickle scheme' (maintaining PascalDragon's keyboard mishap) but rather help out (where practical) anyone who wants to dig into the source code themselves. even if their changes will never go into the main source. I mainly wanted to make clear that even if you'd implement this that it's very unlikely that this would become part of the main source. It has happened often enough that people were disappointed that they weren't told at an early stage of their work that their ideas/changes aren't deemed useful for the official source. ---------------------------------------------------------------------- Bogen85 · 2023-09-10 16:34 FPC stable 3.2.2 and FPC development 3.3.1 both do work with musl-libc instead of glibc if compiled for it: See https://forum.lazarus.freepascal.org/index.php/topic,64570.msg Still a few minor issues to work out, but musl-libc can definitely be used in place of glibc.