[SOLVED] LAMW: Add custom shared library to project
lazarus / 43-Android

----------------------------------------------------------------------
tk · 2016-11-11 00:34

I would like to test my shared library (cross compiled to Android/arm)
in an Android app.

I managed to install LAMW and run a sample project.
However, I could not find an example how to load and manage custom
shared libraries in a LAMW application.

So I just placed the library file libmylib.so to
c:\lamw\projects\LamwGUIProject1\libs\armeabi-v7a\ where there already
was libcontrols.so and wrote:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i {
TAndroidModule1 } procedure TAndroidModule1.jButton2Click(Sender:
TObject);var  LibName: string;  LibModule: HMODULE;begin  LibName :=
'libmylib.so';  ShowMessage('Library: ' + LibName);  if
FileExists(LibName) then    ShowMessage('Library file exists!')  else 
  ShowMessage('Library file does not exists!');  LibModule :=
LoadLibrary(FileName);  if LibModule <> 0 then  begin   
ShowMessage('Library loaded!');  end else    ShowMessage('Library not
loaded!');end;
After downloading to Android emulator and clicking the button it says:
Library file does not exist.
When I rename the file LamwGUIProject1-debug.apk to a zip file and
inspect its content the libmylib.so is there.
So what made I wrong? Is there some path info I must specify?
Thank you

----------------------------------------------------------------------
jmpessoa · 2016-11-11 02:23

Hi tk!

QuoteSo I just placed the library file libmylib.so to
c:\lamw\projects\LamwGUIProject1\libs\armeabi-v7a\ where there already
was libcontrols.so 

Yes!


Now go to  folder ...\src\..... and edit  "Controls.java" like this:

Code: Java  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i      try {
 //your custom mylib ...        System.loadLibrary("mylib");    }
catch (UnsatisfiedLinkError e) {         Log.e("JNI_Load_LibMylib",
"exception", e);    }       try {     //default       
System.loadLibrary("controls");    } catch (UnsatisfiedLinkError e) { 
       Log.e("JNI_Load_LibControls", "exception", e);    } 
And to prevent your modification from being lost,

go to menu:

 "Tools" -----> "[Lamw] Android Module Wizard" ---> "[Configure]
CanUpdateJavaTemplates" 

and on dialog prompt choice "change to False".

----------------------------------------------------------------------
tk · 2016-11-11 10:10

Hi jmpessoa,
now library loads fine, got Library loaded and even I could call a
function from library!

But I still get Library file does not exist.

Everywhere else (Win, Linux, MAC) I first check with FileExists and
then call LoadLibrary, with the same (full) library path.

Is there some difference between the parameter for LoadLibrary (works)
and FileExists (doesn't work) on Android?

Thank you

----------------------------------------------------------------------
jmpessoa · 2016-11-11 16:46

Hi tk!

1. About "FileExists". The question is: "where android system save the
library"? Android use a not trivial internal app file structure...

2. You really need to use "LoadLibrary"?

3. If possible send me a simple project with a custom minimalistic
library. So
I will put it in "demos" project. If you have knowledge it would be
interesting one compiled with Free Pascal and another with "C".

Thank you!

----------------------------------------------------------------------
tk · 2016-11-12 09:44

Quote from: jmpessoa on November 11, 2016, 05:46:12 pm1. About
"FileExists". The question is: "where android system save the
library"? Android use a not trivial internal app file structure...

This I don't know! I only know the apk is saved under /data/app but
where does the system extract it before run?
I only know that on Linux LoadLibrary does not search in current
directory as on Windows.
You have to specify full path.
But Android is not Linux.


Quote from: jmpessoa on November 11, 2016, 05:46:12 pm2. You really
need to use "LoadLibrary"?

Yes absolutely. See this:
http://forum.lazarus.freepascal.org/index.php/topic,34431.0.html.
Still I would prefer pascalish IDE even for the mobile dev. I'll play
with LAMW a bit first to get the feeling.
For iOS it seems we have no other choice.

Quote from: jmpessoa on November 11, 2016, 05:46:12 pm3. If possible
send me a simple project with a custom minimalistic library. So
I will put it in "demos" project. If you have knowledge it would be
interesting one compiled with Free Pascal and another with "C".

Yes I will. I'll put then other (more complex) LAMW demo with our real
library to our commercial SW.

----------------------------------------------------------------------
tk · 2016-11-12 11:48

Further observations:
You can't add Forms unit to the library (mylib.so), the application
crashes immediately.
Clearly because of the initialization code in the Forms unit.
So you can't access Application variable at all in the library.
I have LCL package in dependencies of the library project though, and
customdrawn widgetset.

You can't add Forms unit even to the main app (libcontrols.so).

----------------------------------------------------------------------
jmpessoa · 2016-11-12 15:35

QuoteYou can't add Forms unit to the library...

Yes, we can't have LCL dependency in LAMW project..

but, we can use many many FCL stuff. There are some graphics [fcl-
image] e freetype examples.... see demos
"AppTFPNoGUIGraphicsBridgeDemoXXX" and here
is an [old]  "readme.txt" for help with freetype and fcl-image on
Android.

QuoteAbout "TFPNoGuiGraphicsBridge"

   ref. https://github.com/jmpessoa/tfpnoguigraphicsbridge

   "A wrapper over Free Pascal fcl-image ref.
http://wiki.freepascal.org/fcl-image"

   "FPImage (fcl-image) draw images which won't be displayed in the
screen [NoGUI !!!].
http://wiki.freepascal.org/Developing_with_Graphics


Hint: TFPNoGuiGraphicsBridge on Android [Lamw/Lazarus Android Module
Wizard project!]

   --->>> Cross compile [Lamw/arm-android] project fail .... NO PANIC!

PANIC I: Compiling ... [please, read lazarus or/as laz4android and
...\fpc\2.7.1 or/as ...\fpc\3.1.1 etc..]

   "(FTFont.PPU and freetype.PPU) units NOT FOUND in
"...\lazarus\fpc\2.7.1\units\arm-android\fcl-image" ???

   Solution:

   1. Goto "...\lazarus\fpc\2.7.1\source\packages\fcl-image\src" and
copy

      ftfont.pp (if need change to .pas)
      freetype.pp (if need change to .pas)
      freetypeh.pp (if need change to .pas)

      to folder "...\tfpnoguigraphicsbridge" package folder and build
AGAIN your project!

      Yes, now you got ftfont.ppu, ftfont.o, freetype.ppu etc... to
"arm-android" !

   2. Copy THEM to folder  "...\lazarus\fpc\2.7.1\units\arm-
android\fcl-image"
      So, others [future] projects will find its there!  [solved to
"arm-android" !!!]

PANIC II: [building Lamw project cross-arm]::

   ".... : cannot find -lfreetype"

   Solution:

      Copy "libfreetype.so" to NDK location   
"....\platforms\android-XX\arch-arm\usr\lib"
      where XX = 14 or 15 or 16... or 21 .. etc
      example: put "libfreetype.so" here:
"..\ndkplatforms\android-15\arch-arm\usr\lib"

      For Lamw project you can look for "XX" value in menu:
      "Project" --->> "Project Options" ---> "Compile Options" -->>
"Paths" --->> Libraries [-Fl]

PANIC III. Where I find a "libfreetype.so" for arm-android ?

   Go to demo "...\AppTFPNoGUIGraphicsBridgeDemo1\libs\armeabi"
[Eclipse compatible Project]

   You will find an "all ready" there!

PANIC IV.  Where "libfreetype.so" will be load in java code?

   Go to "Controls.java" [\src\...\..] and uncomment this line:

      --->> System.loadLibrary("freetype");

   The code now will stay that way:

   //Load Pascal Library
   static {  
            System.loadLibrary("freetype");  // <<---uncommented here!
            System.loadLibrary("controls");    
   }

Other example:  we can play with FCL "fphttpclient", too: simply copy
"fphttpclient.pp",  "httpprotocol.pp" and "httpprotocol.pp" to your
project   "...\jni" folder  and "uses" ...

Thank you!

----------------------------------------------------------------------
tk · 2016-11-12 20:50

Thank you, good to know.
My first feeling is LAMW is just a great tool, despite of the missing
debugging and some other things!
Wish there was a LiMW tool as well (for iOS)!

Now I'll try to crosscompile our Linux UDP driver, which is based on
Synapse.
If it is not running then I'll have to write a new one for Android
using udpsocket.
This will be another library loaded by the library I spoke off above.

So I'll make regular dll hell under Android. Great isn't it?

----------------------------------------------------------------------
jmpessoa · 2016-11-13 03:48

Ok. I was able to generate the customs android-arm ".so" from codes in
"Pascal"  and "C". And everything worked fine!

Here  are my findings:

(1)
Prepare to Linker:
Copy "libmylib.so" to NDK location    "....\platforms\android-XX\arch-
arm\usr\lib"
      where XX = 14 or 15 or 16... or 21 .. etc
      example: put "libmylib.so" here:
"..\ndkplatforms\android-15\arch-arm\usr\lib"

      For Lamw project you can look for "XX" value in menu:
      "Project" --->> "Project Options" ---> "Compile Options" -->>
"Paths" --->> Libraries [-Fl]

(2)
Put "libmylib.so" in folder "...\libs\armeabi"  where there already
was "libcontrols.so"

(3)
Edit "Controls.java" to load "mylib", too ["...\src\...\..."]:

Code: Java  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i     try { 
     System.loadLibrary("mylib");    } catch (UnsatisfiedLinkError e)
{         Log.e("JNI_Load_LibMyLib", "exception", e);    } 
(4)
Copy  "mylib.pas" interface unit  to "....\jni" folder

"mylib.pas" code example:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i unit mylib;
 interface     function addp(a: longint;  b: longint):longint; cdecl;
external 'libmylib.so' name 'addp'; implementation end. 
5. Add "mylib.pas" interface unit  to  "unit1.pas"  [uses]

6. Try call the function "addp"
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i procedure
TAndroidModule1.jButton1Click(Sender: TObject);var  sumPascal:
longint;begin  sumPascal:= addp(13, 15); 
ShowMessage(IntToStr(sumPascal));end; 
7. Success!

NOTE:  I did not need to use pascal code like "LibModule :=
LoadLibrary('libmylib.so');" This was handled by java [step "3"]

----------------------------------------------------------------------
tk · 2016-11-13 09:34

Hi jmpessoa thank you much for further help.

I have to use LoadLibrary because I need the library to be loaded
dynamically.
I generate the interfaces for the library with my tool (PWIG, is
already part of my KControls in the bitbucket repo) and they are the
same for Win, Linux, MAC, Android....

I think currently it works fine (on my Android 4 device, not on the
emulator but who cares, I can try putting it to the usr/lib folder
according to your point 1 so maybe it helps even for the emulator)

Actually I thought about some automation of the library management in
LAMW/Controls.java.

I mean not having to disable the "[Configure] CanUpdateJavaTemplates"
because it is needed when I add some new UI to the module.
I think I might be able to do this myself in LAMW, could you point me
in correct direction?

EDIT:
I thought about adding a new page to Android Project Options named
External Libraries, there a simple list and then generate the code to
Controls.java.
But maybe you'll be quicker than me, as with the demo libs written in
Pascal and C  I am still playing with my real project and want to test
as much as possible first.
Later I will send you a simple demo for a dynamically loaded library
as promised , where the interfaces for the caller (libcontrols.so) and
callee (libmylib.so) were generated by PWIG. Here you can use classes
from the library transparently as if they were declared in the main
app.

EDIT2:
Quote from: tk on November 12, 2016, 09:50:54 pmNow I'll try to
crosscompile our Linux UDP driver, which is based on Synapse.
If it is not running then I'll have to write a new one for Android
using udpsocket.

As I suspected Synapse does not work under Android. Question is now
how to use jUDPSocket in my custom library where I don't have (and
already can't have) the jApp reference.

----------------------------------------------------------------------
jmpessoa · 2016-11-13 16:11

QuoteI mean not having to disable the "[Configure]
CanUpdateJavaTemplates" because it is needed when I add some new UI to
the module....

No, You can continue adding new UI to the module.... no problem!
[In fact update only synchronizes the project templates com the
framework templates..]


QuoteI thought about adding a new page to Android Project Options
named External Libraries, there a simple list and then generate the
code to Controls.java.

Great idea! I will see what I can do...

QuoteAs I suspected Synapse does not work under Android. Question is
now how to use jUDPSocket in my custom library where I don't have (and
already can't have) the jApp reference. 

I will think about...But we should continue to seek a pure pascal
solution. In a similar case [http] I was able to use FCL stuff.  

Is there  a FCL UDP implementation?

Maybe someone here in the forum could point us to a [simple] pascal
UDP implementation...

----------------------------------------------------------------------
tk · 2016-11-13 17:46

Quote from: jmpessoa on November 13, 2016, 05:11:11 pmNo, You can
continue adding new UI to the module.... no problem!
[In fact update only synchronizes the project templates com the
framework templates..]

Did not work for me. When I tried to add jTimer for example I had to
turn it on and then again off, to get jTimer.java and some jTimer
stuff to Controls.java. Then I had to include the library loads again
to Controls.java.
So did I make sth. wrong?

Quote from: jmpessoa on November 13, 2016, 05:11:11 pmGreat idea! I
will see what I can do...

Ok. You are the master here

Quote from: jmpessoa on November 13, 2016, 05:11:11 pmIs there  a FCL
UDP implementation?

I tried SynCrtSock from MORMOT but that did not compile, then I tried
to activate linux (kylix) port for Synapse and took an extraction of
LibC for that (so that the linker did not complain about missing
libs), but app crashed immediately.

So now I'll try to use the socket API first. There is libc.so in the
ndk so the socket api should be basically supported.

And actually I can't use the jUDPsocket at all because the driver
library has to be completely standalone, would not work without the
java counterpart.

----------------------------------------------------------------------
jmpessoa · 2016-11-13 18:38

There is some FPC UDP stuff here:

QuoteC:\lazarus\fpc\3.1.1\source\packages\libenet

----------------------------------------------------------------------
tk · 2016-11-13 19:19

Quote from: jmpessoa on November 13, 2016, 07:38:55 pm
C:\lazarus\fpc\3.1.1\source\packages\libenet
Voila, this connects to ENET (http://enet.bespin.org/) which in turn
connects directly to BSD sockets.
This might work.
Finally I gave up to get Synapse running on Android. Too complex code
and I can't debug.
Hopefully the ENET lib helps me, will report then, thank you!

----------------------------------------------------------------------
jmpessoa · 2016-11-13 19:48

QuoteQuoteNo, You can continue adding new UI to the module.... no
problem!
[In fact update only synchronizes the project templates com the
framework templates..]
Did not work for me. When I tried to add jTimer for example I had to
turn it on and then again off, to get jTimer.java and some jTimer
stuff to Controls.java. Then I had to include the library loads again
to Controls.java.
So did I make sth. wrong?

I did some test here... works as expected ...The jTime [and others]
stuff was include and my "loadlibrary" was preserved in
"Controls.java"... try "save all"  or/and  "run-->>build" before "look
for" change in your code base...

----------------------------------------------------------------------
jmpessoa · 2016-11-14 03:59

Ok.

The github repository was updated!

Added support to custom library when "CanUpdateTemplate=True"

Thank you!

----------------------------------------------------------------------
yuriy_sydorov · 2016-11-14 15:32

Quote from: tk on November 13, 2016, 10:34:34 amAs I suspected Synapse
does not work under Android.
Synapse works great on Android when using FPC trunk version.
There was a problem with DNS resolution (netdb) and it is fixed in the
trunk.
See here the list of fixed issues:
http://wiki.freepascal.org/Android#Known_issues

----------------------------------------------------------------------
tk · 2016-11-15 00:34

Quote from: yury_sidorov on November 14, 2016, 04:32:37 pmSynapse
works great on Android when using FPC trunk version.
There was a problem with DNS resolution (netdb) and it is fixed in the
trunk.

I already made my own lightweight UDP client unit (no DNS and IP4
only, sufficient to what I need), see attachment, works very well for
me (Linux+Android).
But I'll look at the FPC trunk version later, good to know that it
already works.

----------------------------------------------------------------------
tk · 2016-11-15 00:38

Quote from: jmpessoa on November 13, 2016, 08:48:38 pmI did some test
here... works as expected ...The jTime [and others] stuff was include
and my "loadlibrary" was preserved in "Controls.java"...

Works for me now, don't know already what I made wrong before.

Quote from: jmpessoa on November 13, 2016, 08:48:38 pmAdded support to
custom library when "CanUpdateTemplate=True"

Great! I'll check it out and report then!

----------------------------------------------------------------------
tk · 2016-11-16 09:26

So I tried current git version but suddenly Controls.java was not
updated at all.

I found relevant code in smartdesigner.pas but then inspected my java
template path and it still pointed to the old location which I
deleted. So it did not replace Controls.java.

Where can I set new java template location?
I don't see such entry in Tools/LAMW path settings.

Thank you

EDIT: Success! Had to add Java template path edit field to
uformsettingspaths.pas and I was then able to change that path!
So works now, marking this solved.

Shall I send you my changes? The edit I added to that form is not
properly aligned as I was not able to move fields on that form, don't
know why... But the setting works.

PS. I was not able to find the location where the LAMW settings are
stored on disk...

EDIT2: I attached the patch.

----------------------------------------------------------------------
tk · 2016-11-16 17:55

And one more note to this (using the FPC trunk now):

GetAppConfigDir(False) returns correct config path only when called in
the main app.
For the library it returns /data/local/tmp/ which is not anymore
writable on recent Android versions.

It seems to me that GetHomeDir (which is called by GetAppConfigDir) is
badly defined in SysUtils.pas as it relies on the IsJniLibrary
variable. This variable is false in my library and IMO the
configuration path should not be dependent on that variable.

I could normally obtain the path by using the same code and removing
the IsJniLibrary check.

----------------------------------------------------------------------
jmpessoa · 2016-11-16 22:29

QuoteWhere can I set new java template location?
I don't see such entry in Tools/LAMW path settings.

go to foder "C:\lazarus\config" and edit "JNIAndroidProject.ini"

But, Ok I will apply your patch!

Thank you!

----------------------------------------------------------------------
yuriy_sydorov · 2016-11-18 20:26

Quote from: tk on November 16, 2016, 06:55:26 pmAnd one more note to
this (using the FPC trunk now):

GetAppConfigDir(False) returns correct config path only when called in
the main app.
For the library it returns /data/local/tmp/ which is not anymore
writable on recent Android versions.

Just set IsJniLibrary to True in your external library to force
GetAppConfigDir return correct path.

----------------------------------------------------------------------
yuriy_sydorov · 2016-11-20 10:55

Quote from: yury_sidorov on November 18, 2016, 09:26:49 pmQuote from:
tk on November 16, 2016, 06:55:26 pmAnd one more note to this (using
the FPC trunk now):

GetAppConfigDir(False) returns correct config path only when called in
the main app.
For the library it returns /data/local/tmp/ which is not anymore
writable on recent Android versions.

Just set IsJniLibrary to True in your external library to force
GetAppConfigDir return correct path.
This issue has been fixed in r34926 in trunk. GetAppConfigDir() works
for any shared library, if a package name can be detected.

----------------------------------------------------------------------
tk · 2016-11-20 11:18

Thanks yury, good to know!

Until now I used the copy of GetAppConfigDir code for Android, did not
want to alter IsJniLibrary variable in my library.
I'll pull the trunk via fpcupdeluxe again and try it!

----------------------------------------------------------------------
tk · 2016-11-21 00:02

One more important note for shared libraries on Android.
If the library is NOT a JNI library (which I suppose is always the one
created by LAMW) you have to simulate that, otherwise it will hang on
Android systems <4.2!
At least this blocker comes with recent FPC trunk (downloaded a week
ago).
The cause is well described in fpc source
(compiler/systems/t_android.pas).
I spent hours and hours isolating the cause.

If coming to this issue and your library hangs on such old device, add
this to your library:


Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i {$IFDEF
ANDROID}function JNI_OnLoad(VM: Pointer; {%H-}reserved: pointer):
Longint; cdecl;const  JNI_VERSION_1_6=$00010006;begin  Result :=
JNI_VERSION_1_6;end; exports  JNI_OnLoad name 'JNI_OnLoad';{$ENDIF}
Note: In FPC that came with Laz4Android 1.5 the libraries work fine
even without this fix. Clearly because there were huge changes made in
Android RTL during this time.

Yury, as it seems you are the author of the new android RTL, did you
make here something recently or what else would you recommend here?
Although this fix works (and even the GetAppConfigDir works because
the RTL sets IsJNILibrary to true), it is quite a dirty fix.

----------------------------------------------------------------------
yuriy_sydorov · 2016-11-21 12:05

Indeed, if you use the cwstring unit in a JNI shared library, your
Java application will hang when loading the library on Android 4.0 or
earlier. This also applies to any unit which dynamically loads shared
libraries during initialization.
It is caused by a bug in older Android versions.

The issue is fixed only in FPC trunk. And your JNI library must export
the JNI_OnLoad function in order the compiler be apple to apply needed
workaround.

----------------------------------------------------------------------
tk · 2016-11-21 14:06

Quote from: tk on November 21, 2016, 01:02:29 amNote: In FPC that came
with Laz4Android 1.5 the libraries work fine even without this fix.

So it was clearly only a lucky coincidence that the older RTL
(distributed with Laz4Android 1.5) did not call dlopen() during unit
initializations (which actually causes the freeze).
Haven't tried but if it was called then there was a freeze as well.

----------------------------------------------------------------------
yuriy_sydorov · 2016-11-22 10:47

Quote from: tk on November 21, 2016, 03:06:58 pmSo it was clearly only
a lucky coincidence that the older RTL (distributed with Laz4Android
1.5) did not call dlopen() during unit initializations (which actually
causes the freeze).
Haven't tried but if it was called then there was a freeze as well.
Probably the cwstring unit was not used in your library or dependent
units when you compiled using Laz4Android 1.5. That's why there was no
freeze on older Android versions. cwstring for Android preforms
dynamic loading of ICU libraries from the beginning.

I've discovered this bug quite recently, despite I had been working on
Android target for FPC for several years. I always performed FPC test
suite runs on Android versions 4.1+ and all worked fine.

But when my own Android application has matured enough to require
cwstring, I've found that it does not start on Android 2.3 anymore.
And after some investigation the bug was discovered and fixed.