Dynamic Library on Linux
lazarus / 32-Linux

----------------------------------------------------------------------
Kevan S. Hashemi · 2020-05-11 16:25

Dear FPC Community,

I have the following library program:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i library
d;function answer:integer; cdecl;begin        writeln('Hello from d
library, answer routine.');        answer:=42;end; exports answer;end.
And the following main program that loads the "d" library when it
starts up:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
m;{$linklib d}function answer:integer; cdecl; external;begin       
writeln('Answer: ',answer);end.
On MacOSX I compile and run like this (leaving out compiler messages):

Code: Bash  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i $ fpc d$
fpc m$ ./mHello from d library, answer routine.Answer: 42
On Linux 64-bit Scientific Linux 6.4, a Fedora variant used by CERN,
with "Free Pascal Compiler version 3.0.4 [2017/10/03] for x86_64"
downloaded from FPC site yesterday I get the following (again, I have
removed normal compiler output to leave only warnings):

Code: Bash  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i $ fpc
d/usr/local/bin/ld: warning: link.res contains output sections; did
you forget -T?$ fpc m/usr/local/bin/ld: warning: link.res contains
output sections; did you forget -T?$  file libd.solibd.so: ELF 64-bit
LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not
stripped$ ./m./m: error while loading shared libraries: libd.so:
cannot open shared object file: No such file or directory
The "cdecl" is required on both MacOSX and Linux, or else "m" won't
compile. The "stdcall" directive, or the absence of any directive,
results in a compiler error: fps cannot understand the "answer"
function.

What am I doing wrong on Linux?

Best, Kevan

----------------------------------------------------------------------
Fred vS · 2020-05-11 17:00

Hello.

With the directive Code: Pascal  [Select][+][-]window.onload =
function(){var x1 = document.getElementById("main_content_section");
if (x1) { var x = document.getElementsByClassName("geshi");for (var i
= 0; i {$linklib d} the linker will look for the library in the linker
library search path.

Did you copy libd.so in one of that library search path? (usually one
of this:) Code: Pascal  [Select][+][-]window.onload = function(){var
x1 = document.getElementById("main_content_section"); if (x1) { var x
= document.getElementsByClassName("geshi");for (var i = 0; i /lib
orCode: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i  /usr/lib
or Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
/usr/local/lib

If you want use a custom directory, then you should use LoadLibrary()
to dynamically load your library.

[EDIT] You may also set the linker library search path with the -Fl
command line option.

----------------------------------------------------------------------
jiaxing2 · 2020-05-11 18:12

Quote from: Kevan S. Hashemi on May 11, 2020, 06:25:20 pmDear FPC
Community,

I have the following library program:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i library
d;function answer:integer; cdecl;begin        writeln('Hello from d
library, answer routine.');        answer:=42;end; exports answer;end.
And the following main program that loads the "d" library when it
starts up:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
m;{$linklib d}function answer:integer; cdecl; external;begin       
writeln('Answer: ',answer);end.
On MacOSX I compile and run like this (leaving out compiler messages):

Code: Bash  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i $ fpc d$
fpc m$ ./mHello from d library, answer routine.Answer: 42
On Linux 64-bit Scientific Linux 6.4, a Fedora variant used by CERN,
with "Free Pascal Compiler version 3.0.4 [2017/10/03] for x86_64"
downloaded from FPC site yesterday I get the following (again, I have
removed normal compiler output to leave only warnings):

Code: Bash  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i $ fpc
d/usr/local/bin/ld: warning: link.res contains output sections; did
you forget -T?$ fpc m/usr/local/bin/ld: warning: link.res contains
output sections; did you forget -T?$  file libd.solibd.so: ELF 64-bit
LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not
stripped$ ./m./m: error while loading shared libraries: libd.so:
cannot open shared object file: No such file or directory
The "cdecl" is required on both MacOSX and Linux, or else "m" won't
compile. The "stdcall" directive, or the absence of any directive,
results in a compiler error: fps cannot understand the "answer"
function.

What am I doing wrong on Linux?

Best, Kevan

Everything is normal. FPC done it job fine. The problem is Linux can't
find libd.so when running m because Linux doesn't search the current
directory by default.

So, run ldconfig "absolute path to the directory contains libd.so" as
root and you could run m as normal.

More information: https://unix.stackexchange.com/questions/67781/use-
shared-libraries-in-usr-local-lib

----------------------------------------------------------------------
Kevan S. Hashemi · 2020-05-11 19:10

Dear Fred,

Thanks for your help.

I copied libd.so into /lib, /usr/lib, and then /usr/local/lib, but
program "m" still says it can't find the file. Please note: I compile
"m" without errors. The shared library libd.so is in the same
directory as m.pas and the compiler finds the library just fine. If I
delete libd.so and try to compile, I get an error saying the compiler
cannot find the library. After compiling libd.so and the executable
"m", I use ldd to look for dependancies:

Code: Bash  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
[hashemi@vmboxslc64 Build]$ ldd libd.so         /lib64/ld-
linux-x86-64.so.2 (0x00000038b7a00000)        linux-vdso.so.1 =>
 (0x00007ffe8cf7e000)[hashemi@vmboxslc64 Build]$ ldd m        linux-
vdso.so.1 =>  (0x00007ffe2374c000)        libd.so => not found$ file
libd.solibd.so: ELF 64-bit LSB shared object, x86-64, version 1
(SYSV), dynamically linked, not stripped
So it's not just FPC that can't find the file. But the file is
definitely present. Does my example code work for you, if you compile
with FPC on your Debian system?

Best, Kevan

----------------------------------------------------------------------
MarkMLl · 2020-05-11 19:32

I'm by no means an expert at this, and whenever I've used .so files
I've explicitly used LoadLibrary().

I find myself wondering whether ldconfig is relevant here, and I think
it might be worth considering what's in its manpage:

(on some 64-bit architectures such as x86-64, /lib and /usr/lib are
the trusted directories for 32-bit libraries, while /lib64 and
/usr/lib64 are used for 64-bit libraries).

Having said that, my last couple of attempts to use LoadLibrary() on
x86_64 (Debian "stable" v10) haven't worked out well, however they
were rather tricky setups with calls in both directions... there
/might/ be a compiler- or OS version-specific problem in there, but I
worked around it rather than investigating in depth. However your  ldd
m  output suggests that that's probably a red herring.

MarkMLl

----------------------------------------------------------------------
marcov · 2020-05-11 19:37

Compile with -st to stop at linking, and inspect the ppas.sh file that
calls the linker.

This way you can call the linker independent from FPC.

It's the linker and the dynamic linker that search libraries.

----------------------------------------------------------------------
Fred vS · 2020-05-11 19:57

QuoteDoes my example code work for you, if you compile with FPC on
your Debian system?
Yes, following my previous post, on Debian 10, 6' bit:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i fred@fiens
~> fpc dFree Pascal Compiler version 3.0.4+dfsg-22 [2019/01/24] for
x86_64Copyright (c) 1993-2017 by Florian Klaempfl and othersTarget OS:
Linux for x86-64Compiling d.pasLinking libd.so9 lines compiled, 0.2
sec
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i fred@fiens
~> fpc mFree Pascal Compiler version 3.0.4+dfsg-22 [2019/01/24] for
x86_64Copyright (c) 1993-2017 by Florian Klaempfl and othersTarget OS:
Linux for x86-64Compiling m.pasLinking m6 lines compiled, 0.1 sec 
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i fred@fiens
~> ./m./m: error while loading shared libraries: libd.so: cannot open
shared object file: No such file or directory 
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i fred@fiens
~> sudo cp libd.so /usr/lib
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i fred@fiens
~> ./mHello from d library, answer routine.Answer: 42

----------------------------------------------------------------------
Kevan S. Hashemi · 2020-05-11 20:25

Fred: Thanks for showing that moving the library into the standard
search path will get the program working.

Marcov: I love that -st option, which produces a script to perform
linking. I tried -sh as well.

MarkMLI: You are absolutely correct about /lib64. When I move my
libd.so into /lib64, my program runs. And "ldd" finds it there too.

jiaxing2: It would have taken me a long time to figure out that "ldd"
does not search the local directory.

The four of you have saved me a lot of trouble. Thank you. Here is a
working solution that uses the LD_LIBRARY_PATH environment variable,
and allows us to leave the shared object where it is, and we don't
have to reconfigure the system as root.

Code: Bash  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i $
LD_LIBRARY_PATH=.$ export LD_LIBRARY_PATH$ ./mHello from d library,
answer routine.Answer: 42$ ldd m        linux-vdso.so.1 =>
 (0x00007fff9fbfe000)        libd.so => ./libd.so
(0x00007fcc17a85000)        /lib64/ld-linux-x86-64.so.2
(0x00000038b7a00000)
Note that LD_LIBRARY_PATH adds to the library search path. It does not
over-write the library search path.

Best, Kevan

----------------------------------------------------------------------
Fred vS · 2020-05-11 20:38

Hello Kevan.

Thanks for the thanks!

I would recommend you to take a look at Loadlibrary - (dynamically
loading a dynamic library):

https://wiki.lazarus.freepascal.org/Lazarus/FPC_Libraries

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
LoadLibrary('/the/direcrory/of/libd.so');
I agree, there is a few more code needed but much less problem after.

Fre;D

----------------------------------------------------------------------
MarkMLl · 2020-05-11 20:50

Quote from: Kevan S. Hashemi on May 11, 2020, 10:25:15 pmMarkMLI: You
are absolutely correct about /lib64. When I move my libd.so into
/lib64, my program runs. And "ldd" finds it there too.

jiaxing2: It would have taken me a long time to figure out that "ldd"
does not search the local directory.

You really were very close to fixing it yourself. If you hadn't
mentioned ldd- which I always forget about- I wouldn't have had a
quick way of reminding myself the name of the ldconfig program, and if
you'd used  ldd -v  you'd probably have seen "x86_64" repeated often
enough to twig :-)

QuoteNote that LD_LIBRARY_PATH adds to the library search path. It
does not over-write the library search path.

Actually, that takes me /way/ back: the first Linux distro I used
(Yggdrassil, 25+ years ago) recommended LD_LIBRARY_PATH to add an
automatic text decompressor when files were being viewed from the live
CD, and more recently I've used it to add a shim layer to V4L. It's
not generally appreciated how much mischief it can cause...

MarkMLl

----------------------------------------------------------------------
Fred vS · 2020-05-12 01:21

Hello Kevan.

Here your code for m program using loadlibrary(), supposing that
libd.so is in the same directory as the program:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program m; 
 uses dynlibs, SysUtils;     type      TMyMethod=function :integer;
cdecl;         var       MyLib: TLibHandle = dynlibs.NilHandle;     
 MyMethod: TMyMethod;         begin       MyLib :=
LoadLibrary(ExtractFilePath(ParamStr(0)) + '/libd.so');       if MyLib
= 0 then Exit;        MyMethod := TMyMethod(GetProcedureAddress(MyLib,
'answer'));       if MyMethod <> nil then writeln('Answer:
',MyMethod);       FreeLibrary(MyLib);        MyLib:=
DynLibs.NilHandle;    end.
Note that if you use only the filename without the full path:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i MyLib :=
LoadLibrary('libd.so');
LoadLibrary will do like external does: looking in the linker library
search path.

Then it will work if libd.so is in /usr/lib or /usr/lib64 for example.

----------------------------------------------------------------------
jiaxing2 · 2020-05-12 05:46

Quote from: Fred vS on May 12, 2020, 03:21:07 amNote that if you use
only the filename without the full path:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i MyLib :=
LoadLibrary('libd.so');
LoadLibrary will do like external does: looking in the linker library
search path.

Then it will work if libd.so is in /usr/lib or /usr/lib64 for example.

On my system /usr/lib64 is not included by default and needs to be
added to /etc/ld.so.conf. The IUP library installs to this directory.

----------------------------------------------------------------------
MarkMLl · 2020-05-12 08:07

General point: note that LoadLibrary() etc. has a GetLoadErrorStr()
function which returns the last saved message from the OS.

MarkMLl

----------------------------------------------------------------------
trev · 2020-05-12 12:30

You can also use:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i uses 
SysUtils;...   // check whether loading was successful   if LibHandle
<> 0 then...   else     // error message on load failure   
 WriteLn('GetLastOSError = ', SysErrorMessage(GetLastOSError)); 
Many ways to skin an error

----------------------------------------------------------------------
marcov · 2020-05-12 12:41

Quote from: jiaxing2 on May 12, 2020, 07:46:09 amOn my system
/usr/lib64 is not included by default and needs to be added to
/etc/ld.so.conf. The IUP library installs to this directory.

Note that officially after changing ld.so.conf you must run ldconfig.
Linux never has been too critical on that, but some BSDs did.

----------------------------------------------------------------------
Kevan S. Hashemi · 2020-05-13 03:23

The loadlibrary method does look more reliable, especially with the
addition of ExtractFilePath(ParamStr(0)), which looks useful. Fred:
thanks for your code. I will try this out on Windows, Linux, and
MacOSX. Best, Kevan

----------------------------------------------------------------------
Fred vS · 2020-05-13 14:55

QuoteI will try this out on Windows, Linux, and MacOSX.
Mac OSX uses bundle for application.

https://developer.apple.com/library/archive/documentation/CoreFoundati
on/Conceptual/CFBundles/BundleTypes/BundleTypes.html#//apple_ref/doc/u
id/10000123i-CH101-SW1

You just have to add your libraries in the Ressource-files directory
of the bundle.
Those Apple Ressource files can only be loaded by dynamic loading, so
the good news is that LoadLibrary does it.

So, with MacOSX, it could be something like:

MyLib := LoadLibrary(ExtractFilePath(ParamStr(0)) +
'/Resources/libd.dylib');

[EDIT]  See next post of trev for good explanations.

Fre;D

----------------------------------------------------------------------
Kevan S. Hashemi · 2020-05-13 17:51

Fred,

I like your solution for MacOSX.

The linking problem in practice has another complication. Suppose we
have a generic program, call it 'prog', that we don't write ourselves,
but which we want to augment with our own libraries. We're expecting
'prog' to be able to load our dynamic library.

But prog knows nothing of our library when prog is compiled. Prog
cannot call any routine in our library unless our library provides a
routine with a pre-defined format, with a name that can be deduced
from the library file name. So prog might say that libd.so must
provide a routine D_Init that takes a pointer and returns an integer.

Now prog loads libd.so and calls D_Init. The job of D_Init is to
install in prog a list of new commands that prog can execute. To do
that, D_Init must call an "install command" routine defined by prog
itself, call it Prog_InstallCmd, which takes a pointer and a name for
the command. So now we see that libd.so must be linked against a
library prog.so that defines the structure of Prog_InstallCmd.

On Linux, I think we do this (have not tested fully yet):

fps -k-lprog d

On MacOSX, I'm doing the following to link libd.dylib with the TclTk
frameworks, where I'm running fpc in a directory adjacent to Prog.app,
the application bundle (fully tested):

fpc -k"-F../Prog.app/Contents/Frameworks -framework Tcl -framework Tk"
d

Best, Kevan

----------------------------------------------------------------------
Fred vS · 2020-05-13 19:27

Hello Kevan.

Sorry, but I did not catch the problem.

When you use external your "prog" must also know the names of the
methods of the linked library.

Sure you want to demonstrate something else but I dont catch what.

----------------------------------------------------------------------
Fred vS · 2020-05-13 19:38

Do you want to know the method names of a unknown library?

If it is that, you can retrieve it on Unix system with nm:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i $ nm
 libunknown.so
So you may imagine in your code, via a TProcess, retrieve the name of
the method but I dont know how to retrieve the parameters used by the
exported method.

Aslo I dont know to determine if a exported method is void (procedure)
or not (function).

----------------------------------------------------------------------
Fred vS · 2020-05-13 20:53

Hum, I think I get it (maybe).

You want to have a exported method that has the same name "init_lib"
and same parameter, for each library, and when the program load a new
library, he first call "init_lib".

That "init_lib" method will add all the stuffs needed by the prog to
deal with that new library.

Ha, ok, nice and good to experiment.

----------------------------------------------------------------------
Kevan S. Hashemi · 2020-05-13 21:17

I'm not sure I understand my own point now, it seemed so profound at
the time. I end up with two libraries that call routines both ways.
One library I make, the other is pre-existing and knows nothing about
the library I'm making. The only way to get that to work is to use the
dynamic loading scheme you recommend: where you load at run-time,
under instruction from the user, and specify the name of the library
you want to load. But we don't need to use the command-line "-k" flag
to sort this out, we can sort it out with the various directives and
identifiers available in the Pascal code.

----------------------------------------------------------------------
Fred vS · 2020-05-13 21:30

Hello Kevan.

Hum, maybe I did not understand too.

I was thinking that "prog" was already compiled, with instruction to
load some libraries (plugins?) from a directory.
After loading, "prog" call a method "lib_init" from each library that
will add all what needed for the library of the "plugins" directory.

But now it seems that you want to re-compile "prog" with "-k"  flag.

I am lost, sorry.

----------------------------------------------------------------------
Fred vS · 2020-05-13 21:51

Quote we can sort it out with the various directives and identifiers
available in the Pascal code.
Sure.

I dont know what kinds of method you want to export, must it be
compatible with C applications?

If you use "universal" variables, like integer or pchar, with few type
of methods, it can be predefined with type in the "prog" code.

----------------------------------------------------------------------
trev · 2020-05-13 23:24

Quote from: Fred vS on May 13, 2020, 04:55:47 pmSo, with MacOSX, it
could be something like:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i MyLib :=
LoadLibrary(ExtractFilePath(ParamStr(0)) + '/Resources/libd.dylib');
No, no, no 

1. For a start, any library in an application bundle should be in
Project1.app/Contents/Resources/Frameworks
2. Never use ExtractFilePath(ParamStr(0)) in any UNIX or UNIX-like OS,
it only leads to tears.
3. The correct method to use so that the executable can find the
library is:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
install_name_tool -add_rpath "@executable_path/../Frameworks/."
project1
I recently wrote fairly extensive Wiki articles: macOS Dynamic
Libraries along with macOS Static Libraries which explains it all (I
hope).

----------------------------------------------------------------------
Fred vS · 2020-05-13 23:32

Quote from: trev on May 14, 2020, 01:24:00 amQuote from: Fred vS on
May 13, 2020, 04:55:47 pmSo, with MacOSX, it could be something like:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i MyLib :=
LoadLibrary(ExtractFilePath(ParamStr(0)) + '/Resources/libd.dylib');
No, no, no 

1. For a start, any library in an application bundle should be in
Project1.app/Contents/Resources/Frameworks
2. Never use ExtractFilePath(ParamStr(0)) in any UNIX or UNIX-like OS,
it only leads to tears.
3. The correct method to use so that the executable can find the
library is:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
install_name_tool -add_rpath "@executable_path/../Frameworks/."
project1
I recently wrote fairly extensive Wiki articles: macOS Dynamic
Libraries along with macOS Static Libraries which explains it all (I
hope).

Ok, for my post about Mac OSX, I dont have a Mac, I should not add
that post.

But for that:

Quote2. Never use ExtractFilePath(ParamStr(0)) in any UNIX or UNIX-
like OS, it only leads to tears.
Sorry but I  never cried using this on Linux or FreeBSD.

[EDIT]
From https://www.freepascal.org/docs-html/rtl/system/paramstr.html

QuoteIn the interest of portability, the ParamStr function tries to
behave the same on all operating systems: like the original ParamStr
function in Turbo Pascal. This means even on Unix, paramstr(0) returns
the full path to the program executable. A notable exception is Mac OS
X, where the return value depends on how the application was started.
It may be that just the name of the application is returned (in case
of a command-line launch)
So, ok for Mac OS X, but for the others (the topic is about Dynamic
Library on Linux) ?

----------------------------------------------------------------------
Kevan S. Hashemi · 2020-05-14 00:51

My point is merely something I found interesting, and probably nothing
new to any of you. It applies to any operating system. I don't expect
you to keep reading, but I feel it my duty to attempt to be clear
rather than vague.

With FPC's LoadLibrary command, we can write a master program that
allows the user to select a dynamic library to load. So we load the
library, call it the "analysis library". Then what? We trust that the
analysis library contains a routine called Init, and the master
program runs that routine. The Init routine calls a routine made
available by the master program's own dynamic library, a routine like
"install_new_command", and installs a bunch of new analysis commands.
which the user can now run.

Thus the analysis library declares "install_new_command" as an
external, and it makes available "Init" as an export. So far as I can
tell, using the "LoadLibrary" command in the master program is the
only way to make this happen. So I'm grateful to Fred for showing me
how to use LoadLibrary.

It turns out that, on Linux, we can compile the analysis library
without linking to the library that declares "install_new_program"
being present. That makes sense to me. The compiler trusts that, when
the analysis library is loaded, the "install_new_program" routine will
be available for dynamic linking. If it isn't, the the program will
crash.

But on MacOSX, when we compile the analysis library, fpc wants to see
the library declaring "install_new_command", or else it aborts with an
error. That I don't understand. But I'm looking into it, and that's a
subject for the MacOSX board.

----------------------------------------------------------------------
Fred vS · 2020-05-14 01:08

QuoteThe compiler trusts that, when the analysis library is loaded,
the "install_new_program" routine will be available for dynamic
linking. If it isn't, the the program will crash.
Exactly!

But you may have a filter like this to not crash the program:

       Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i MyMethod :=
TMyMethod(GetProcedureAddress(MyLib, 'install_new_program'));       if
MyMethod = nil then writeln('No install_new_program routine found');
QuoteBut on MacOSX,
For this, I better shut up otherwise some people will cry...

----------------------------------------------------------------------
trev · 2020-05-14 01:17

Quote from: Fred vS on May 14, 2020, 01:32:53 amBut for that:
Quote2. Never use ExtractFilePath(ParamStr(0)) in any UNIX or UNIX-
like OS, it only leads to tears.
Sorry but I  never cried using this on Linux or FreeBSD.

[EDIT]
From https://www.freepascal.org/docs-html/rtl/system/paramstr.html

QuoteIn the interest of portability, the ParamStr function tries to
behave the same on all operating systems: like the original ParamStr
function in Turbo Pascal. This means even on Unix, paramstr(0) returns
the full path to the program executable. A notable exception is Mac OS
X, where the return value depends on how the application was started.
It may be that just the name of the application is returned (in case
of a command-line launch)
So, ok for Mac OS X, but for the others (the topic is about Dynamic
Library on Linux) ?

To quote Jonas:

QuoteParamstr(0) must only be used on Dos and Windows (and maybe
OS/2). On all other platforms the results are unpredictable and it
will not work as you expect (in the best case it will never work; in
the worst case it will only fail under certain circumstances,
resulting in hard-to-find bugs). Read the mailing list thread I quoted
above for more information.

Jonas details the "why" in this mailing list thread post.

----------------------------------------------------------------------
Fred vS · 2020-05-14 01:22

@trev.

OK, thanks for the info (from 2009), and so, what do you propose as
alternative to ParamStr(0)?

----------------------------------------------------------------------
Fred vS · 2020-05-14 01:53

This one is interesting.

https://stackoverflow.com/questions/4025370/can-an-executable-
discover-its-own-path-linux

To resume: normally no but yes it is possible.

One of the solution is to use readlink("/proc/self/exe", buf, bufsize)
and I see just this:

https://www.freepascal.org/docs-html/rtl/baseunix/fpreadlink.html

Hum, could it be used?

----------------------------------------------------------------------
Fred vS · 2020-05-14 03:24

Ok, I understood that in Unix OS it is not the good way to try to find
the path of the current running executable.

But, for the sport, what do you think of this?:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
readlink_test; usessysutils,
baseunix; beginwriteln(fpReadLink('/proc/self/exe'));end.
--> result:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i fred@fiens
~> fpc readlink_test.pas Free Pascal Compiler version 3.0.4+dfsg-22
[2019/01/24] for x86_64Copyright (c) 1993-2017 by Florian Klaempfl and
othersTarget OS: Linux for x86-64Compiling readlink_test.pasLinking
readlink_test8 lines compiled, 0.1 sec

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i fred@fiens
~> ./readlink_test/home/fred/readlink_test

Now create a symlink:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i fred@fiens
~> ln -s readlink_test readlink_link

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i fred@fiens
~> ./readlink_link/home/fred/readlink_test
[EDIT]

Ooops, it seems that fpc does use it already...

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i procedure
SysInitExecPath;var  i    : longint;begin  execpathstr[0]:=#0; 
i:=Fpreadlink('/proc/self/exe',@execpathstr[1],high(execpathstr));  {
it must also be an absolute filename, linux 2.0 points to a memory   
location so this will skip that }  if (i>0) and (execpathstr[1]='/')
then     execpathstr[0]:=char(i);end;
OK, more than time to go to bed.

----------------------------------------------------------------------
trev · 2020-05-14 06:41

@Fred vS

FreeBSD does not usually mount the /proc file system, although it can
be done.

The better solution is to use a kernel sysctl. I've just added an
example to do this to my FreeBSD sysctl article on the Wiki.

For macOS, see Locating the macOS Application Resources Directory.

I have little to almost no experience with Linux, but it seems from a 
quick look around that the readlink /proc method is the way to go.

----------------------------------------------------------------------
PascalDragon · 2020-05-14 07:29

Quote from: Kevan S. Hashemi on May 13, 2020, 07:51:39 pmThe linking
problem in practice has another complication. Suppose we have a
generic program, call it 'prog', that we don't write ourselves, but
which we want to augment with our own libraries. We're expecting
'prog' to be able to load our dynamic library.

But prog knows nothing of our library when prog is compiled. Prog
cannot call any routine in our library unless our library provides a
routine with a pre-defined format, with a name that can be deduced
from the library file name. So prog might say that libd.so must
provide a routine D_Init that takes a pointer and returns an integer.

Now prog loads libd.so and calls D_Init. The job of D_Init is to
install in prog a list of new commands that prog can execute. To do
that, D_Init must call an "install command" routine defined by prog
itself, call it Prog_InstallCmd, which takes a pointer and a name for
the command. So now we see that libd.so must be linked against a
library prog.so that defines the structure of Prog_InstallCmd.

This is not a problem if you do it correctly. Plugin systems are
something that has been solved many times already.

First of, yes, your library needs to follow a specific interface.
Let's say it needs to export a function Init. Now for the library to
interact with the program the program needs to provide the library
with an interface to its own functions. One possibility is the
following (for the sake of the example I assume we're writing an
editor):

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i type 
TGetCurrentFilename = function: PChar; cdecl;  TSetCurrentFilename =
procedure(aFilename: PChar); cdecl;  TGetCursorPos = function: TPoint;
cdecl;  TSetCursorPos = procedure(aPos: TPoint); cdecl;  TSaveFile =
function: Boolean; cdecl;   TProgramFuncs = record   
GetCurrentFilename: TGetCurrentFilename;    SetCurrentFilename:
TSetCurrentFilename;    GetCursorPos: TGetCursorPos;    SetCursorPos:
TSetCursorPos;    SaveFile: TSaveFile;  end;  PProgramFuncs =
^TProgramFuncs;   TLibraryFuncs = record    // functions that the
library provides  end;  PLibraryFuncs = ^TLibraryFuncs;   TInitFunc =
function(aProgFuncs: PProgramFuncs; aLibFuncs: PLibraryFuncs):
Boolean;
The library exports a function Init that matches the signature of
TInitFunc and the program passes in a filled in TProgramFuncs record
with its own functions (so that the library can use them to interact
with the program) and in turn fills in a TLibraryFuncs record with
functions that the program can use to interact with the library.

Of course this is just a simple example and one can create rather
complex systems with this.

One thing to keep in mind however is to implement this library
interface in such a way that for example a C library can be loaded as
well (so String is a nogo for example, aside from the problems with
memory management). Also you need to make sure that memory is freed by
that code who allocated it. This can be achieved by having the program
provide functions for its memory manager as well and have the library
use those for buffers provided by the program or to the program.

----------------------------------------------------------------------
MarkMLl · 2020-05-14 08:02

Quote from: PascalDragon on May 14, 2020, 09:29:42 amThis is not a
problem if you do it correctly. Plugin systems are something that has
been solved many times already.

I'd add two things to that if I may. First, as a general point,
Windows allows a binary to be both an executable and a loadable
library. Linux doesn't, it appears to be a library restriction rather
than the kernel so /could/ be fixed, but don't expect it to be fixed
since that limitation appears to be one of the mechanisms the FSF uses
to enforce the GPL.

Second, I've written code in the past that loads a plugin and allows
plugin code to invoke entry points in the main program by name. I
can't get that to work in current 64-bit FPC, I've not reported it yet
as a bug, so if anybody comes up against that do not automatically
assume that it's your application code that's at fault.

MarkMLl

----------------------------------------------------------------------
trev · 2020-05-14 08:49

Quote from: MarkMLl on May 14, 2020, 10:02:14 amFirst, as a general
point, Windows allows a binary to be both an executable and a loadable
library. Linux doesn't, it appears to be a library restriction rather
than the kernel so /could/ be fixed, but don't expect it to be fixed
since that limitation appears to be one of the mechanisms the FSF uses
to enforce the GPL.

I know I read somewhere that the Linux libc is also an executable...
found it,  see  this Stack Overflow post  for example which explains
how it's done.

----------------------------------------------------------------------
MarkMLl · 2020-05-14 09:20

Quote from: trev on May 14, 2020, 10:49:16 amQuote from: MarkMLl on
May 14, 2020, 10:02:14 amFirst, as a general point, Windows allows a
binary to be both an executable and a loadable library. Linux doesn't,
it appears to be a library restriction rather than the kernel so
/could/ be fixed, but don't expect it to be fixed since that
limitation appears to be one of the mechanisms the FSF uses to enforce
the GPL.

I know I read somewhere that the Linux libc is also an executable...
found it,  see  this Stack Overflow post  for example which explains
how it's done.

Yes, but I believe that's an exception: the normal stuff in libc etc.
that handles locating entry points doesn't like doing that.

The FSF's rationale, as far as I can see, is that the distinction
between a program (with a single entry point) and a library (with
multiple entry points) is relevant to whether the source has to be
disclosed if it's statically linked with something covered by the
LGPL. I forget some of the detail, but I dug into this sort of thing
in some depth when I was trying to work out whether I could get away
with linking one particular thing (the BFD library) into a plugin
without polluting the main program.

MarkMLl

----------------------------------------------------------------------
Fred vS · 2020-05-14 12:11

Quote from: trev on May 14, 2020, 08:41:11 am@Fred vS

FreeBSD does not usually mount the /proc file system, although it can
be done.

The better solution is to use a kernel sysctl. I've just added an
example to do this to my FreeBSD sysctl article on the Wiki.

For macOS, see Locating the macOS Application Resources Directory.

I have little to almost no experience with Linux, but it seems from a 
quick look around that the readlink /proc method is the way to go.

Hello trev.

Thanks for your clear infos.

About FreeBSD, and friends, I did find this:
https://stackoverflow.com/questions/1023306/finding-current-
executables-path-without-proc-self-exe/1024937#1024937

QuoteSome OS-specific interfaces:

    Mac OS X: _NSGetExecutablePath() (man 3 dyld)
    Linux: readlink /proc/self/exe
    Solaris: getexecname()
    FreeBSD: sysctl CTL_KERN KERN_PROC KERN_PROC_PATHNAME -1
    FreeBSD if it has procfs: readlink /proc/curproc/file (FreeBSD
doesn't have procfs by default)
    NetBSD: readlink /proc/curproc/exe
    DragonFly BSD: readlink /proc/curproc/file
    Windows: GetModuleFileName() with hModule = NULL

Quote from: trev on May 14, 2020, 08:41:11 am...with Linux, but it
seems from a  quick look around that the readlink /proc method is the
way to go.
And so I am happy that the code I gave was not totally rejected with
lot of your tears.


Fre;D

----------------------------------------------------------------------
Fred vS · 2020-05-14 12:19

About ParamStr(0).

Please update:

 https://www.freepascal.org/docs-html/rtl/system/paramstr.html

(or give a cristal ball or the phone number of Jonas to know what is
allowed).

Fre;D

----------------------------------------------------------------------
sash · 2020-05-15 19:19

tl;dr

How do I link shared libraries:
0. Library
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i exports
func1Name name 'func1Name';1. Caller project:
  Project -> Options -> Paths -> Libraries (-Fl) add
"/path/where/your/lib/is/stored" (just a path)
2. Caller (import part)
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i const//
 MyCodeIntf.pas  {$ifdef WINDOWS}    LIB_MYCODE = 'mycode.dll'; 
{$endif}  {$ifdef LINUX}    LIB_MYCODE = 'libmycode.so';  {$endif}//
--- function func1() : cint; cdecl; external LIB_MYCODE name
'func1Name'; // note, it could be imported with other namefunction
func2() : cint; cdecl; external LIB_MYCODE name 'func2Name'; // this
last `name` - is how it was exported in your lib//                   
      ^--- be it cdecl, if you want, but (imo) you could omit it or
make `pascal`, in accordance with library's signature of course 
How to run:
-- In lazarus, run params, environment, add user override:
 LD_LIBRARY_PATH = $LD_LIBRARY_PATH:/path/where/your/lib/is/stored

-- from shell
Code: Bash  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
#!/bin/bashexport LD_LIBRARY_PATH=$LD_LIBRARY_PATH:"/path/where/your/l
ib/is/stored"./executable

----------------------------------------------------------------------
Thaddy · 2020-05-15 19:42

Sigh
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
testpar;beginwriteln(paramstr(0));end.[code]You can also use writestr
to a variable of course. All modes, all platforms that support a
console or terminal. it is really that simple.

----------------------------------------------------------------------
trev · 2020-05-16 04:17

Quote from: Fred vS on May 14, 2020, 02:11:51 pmThanks for your clear
infos.

You're welcome!

Quote from: trev on May 14, 2020, 08:41:11 am...with Linux, but it
seems from a  quick look around that the readlink /proc method is the
way to go.
Quote from: Fred vS on May 14, 2020, 02:11:51 pmAnd so I am happy that
the code I gave was not totally rejected with lot of your tears.

----------------------------------------------------------------------
mpv · 2020-06-28 14:41

Program can be linked to a library using path relative to a program
location (at last on Linux)
To link to a library from the same folder as a program all we need is
to pass a -rpath=$ORIGIN option to linker. ( $ORIGIN/MyLibs etc.)

In Lazarus:
Project-> Options-> Compiler Options -> Compilation and Linking ->
check "Pass Optins to linker with "-k "" -> put --rpath=$ORIGIN into
edit box

From a command line:

fpc.sh .....  -k'-rpath=$ORIGIN'

In this case we do not need to use LoadLibrary / modify a
LD_LIBRARY_PATH etc.

See "Rpath token expansion" section in https://man7.org/linux/man-
pages/man8/ld.so.8.html

----------------------------------------------------------------------
devEric69 · 2020-06-29 07:38

For your information, there is a graph that summarizes the most common
methods of loading a library under Linux, here: https://wiki.freepasca
l.org/Lazarus/FPC_Libraries#Simplified_overview_of_the_system.2C_when_
loading_a_shared_library_under_Linux