Suggests: libssl-doc
BTW. You will also be safe to install openssl package because that one
also depends on libssl.
Code: Bash [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i :~ $ sudo
apt-cache depends opensslopenssl Depends: libc6 Depends: libssl1.1
Suggests: ca-certificatesAnd I think you may want the openssl
executable over the headers on a clean non-development system
----------------------------------------------------------------------
maurobio · 2023-09-14 13:18
Hi, @paweld, @rvk!
Many thanks for these other good suggestions, I will add some changes
to the code to conform to your suggestions.
@rvk pointed out a small detail which may explain why I didn't have
errors, but users who attempted to run the executable program had - it
has not occurred to me that such errors (for unknown reasons to me)
did not appeared when the program was running in the IDE (and I have
been too imprudent to not try it outside the development environment,
as a regular user would do)!
Just as an aside, I have no problems compiling the latest release
version (not the trunk version) of BGRA library. Anyway, I never use
OPM to install libraries - I prefer to download the latest release
files and install using the IDE package manager.
Also, the program also ran fine with the last (very old) release
version of Synapse (4.0).
With warmest regards,
----------------------------------------------------------------------
paweld · 2023-09-14 13:36
Quote from: rvklibssl-dev includes only header files. It doesn't
actually include the openssl library. But because of dependency it
will probably install the library itself too. Thank you for the
information, i didn't know about that
Quote from: maurobioAlso, the program also ran fine with the last
(very old) release version of Synapse (4.0) old version of synapse
(40, stable) does not support openssl 3
----------------------------------------------------------------------
maurobio · 2023-09-14 13:51
Hi, @paweld!
Quoteold version of synapse (40, stable) does not support openssl 3
I am using OpenSSL 1.0.2 as per @rvk suggestion.
Best regards,
----------------------------------------------------------------------
maurobio · 2023-09-14 14:01
Hi, gentlemen!
I have some further progress to report.
I have just tried the program on an older Windows 7 machine without
Lazarus installed. I just copied the executable (created with Lazarus
2.0.12 on a newer Windows 10 machine), the two OpenSSL dll's
(ssleay32.dll and libeay32.dll), and the "static" folder to the same
folder and voilà! The program ran smoothly with all species I entered
(Vicia faba, Loxodonta africana, Physeter catodon). It even ran
without errors when I entered a non-existent species ("Loxodota
africana", which was a typo, notice the missing "n" in the genus
name).
This test was done with the original code improved by @paweld,
without the further error checking just suggested by @rvk and that I
will add to the code soon.
This afternoon I will proceed with the tests, this time under three
Linux distros.
More soon...
With warmest regards,
----------------------------------------------------------------------
paweld · 2023-09-14 15:00
Quote from: rvkBut for a clean system, libssl-dev shouldn't be needed.
Just the libssl would be sufficient for the executable.I just now
checked on a freshly installed system (Debian 12). After the
installation I have openssl version 3.0.9, but the program does not
download over https. Only installing the libssl-dev package (also
version 3.0.9) makes all the data downloaded.
Uninstalling the package makes it fail again over https.
Quote from: maurobioI am using OpenSSL 1.0.2 as per @rvk suggestion.on
linux as I mentioned earlier the default is openssl version 3, so
using the old version of synapse will not work with the openssl
package installed
----------------------------------------------------------------------
delphius · 2023-09-14 16:12
Quote from: paweld on September 14, 2023, 05:00:48 pmon linux as I
mentioned earlier the default is openssl version 3, so using the old
version of synapse will not work with the openssl package installed
Please, pros, explain to me why, when developing Free Pascal or
Lazarus programs, we must necessarily rely on intermediary libraries
like LNet, Indy, Synapse, etc.?
Why is it impossible to work with OpenSSL directly, as tfphttpclient
does?
How will such code from the example working with OpenSSL over simple
sockets directly be worse?
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
opensslmail; {$mode objfpc}{$H+}{$Packrecords C} uses SysUtils,
sockets, resolve, openssl; type PSSL_CIPHER = ^SSL_CIPHER; // https:
//github.com/openssl/openssl/blob/ed4a71d18d939f557b430c4369d118b55c1c
0b6c/ssl/ssl_local.h#L397 SSL_CIPHER = record valid: LongWord;
name: PChar; stdname: PChar; id: packed array[0..1] of Char;
algorithm_mkey: LongWord; algorithm_auth: LongWord;
algorithm_enc: LongWord; algorithm_mac: LongWord; min_tls:
Integer; max_tls: Integer; min_dtls: Integer; max_dtls:
Integer; algo_strength: LongWord; algorithm2: LongWord;
strength_bits: LongInt; alg_bits: LongWord; end; const // The
read functions work based on the SSL/TLS records. The data are
received in records (with a maximum record size of 16kB) // For TLS
1.2 and earlier, that limit is 2^14 octets. TLS 1.3 uses a limit of
2^14+1 octets. // https://www.rfc-
editor.org/rfc/rfc8449.html#section-4 BUF_SIZE = 16 * 1024; Host =
'example.org'; Port = 443; {$IFDEF WINDOWS} ext = '.dll';
{$ELSE} ext = '.so'; {$ENDIF} var filepath, trustedCertFile:
string; //OpenSSL ctx: PSSL_CTX; ssl: PSSL; cipherPtr:
PSSL_CIPHER; server_cert:pX509; in_buf: array[0..BUF_SIZE - 1] of
char; request: string; nbytes_written, nbytes_read, ssl_error:
integer; cert_str: AnsiString; tempBuf: AnsiString; //Sockets
CSocket: TSocket; Address: TInetSockAddr; IPAddr: string; hrs:
THostResolver; procedure PrintSSLCipherInfo(const cipher:
SSL_CIPHER);begin Writeln('Cipher Name: ', cipher.name);
Writeln('RFC Name: ', cipher.stdname); //
https://www.iana.org/assignments/tls-parameters/tls-
parameters.xhtml#tls-parameters-4 Writeln('TLS Cipher ID: ', '0x' +
IntToHex(Ord(cipher.id[1]), 2) + ', 0x' + IntToHex(Ord(cipher.id[0]),
2)); Writeln('Key Exchange Algorithm: ', cipher.algorithm_mkey);
Writeln('Server Authentication Algorithm: ', cipher.algorithm_auth);
Writeln('Symmetric Encryption Algorithm: ', cipher.algorithm_enc);
Writeln('Symmetric Authentication Algorithm: ',
cipher.algorithm_mac); Writeln('Minimum SSL/TLS Protocol Version: ',
cipher.min_tls); Writeln('Maximum SSL/TLS Protocol Version: ',
cipher.max_tls); Writeln('Minimum DTLS Protocol Version: ',
cipher.min_dtls); Writeln('Maximum DTLS Protocol Version: ',
cipher.max_dtls); Writeln('Strength and Export Flags: ',
cipher.algo_strength); Writeln('Extra Flags: ', cipher.algorithm2);
Writeln('Number of Bits Used: ', cipher.strength_bits);
Writeln('Number of Bits for Algorithm: ', cipher.alg_bits);end; begin
filepath := ExtractFilePath(ParamStr(0)); //
https://curl.se/ca/cacert.pem trustedCertFile := filepath +
'cacert.pem'; // Load OpenSSL dynamic libraries from program folder
InitSSLInterface(filepath + 'libssl' + ext, filepath + 'libcrypto' +
ext); // Show libs version https://github.com/openssl/openssl/blob/8
f51b2279eda1e0cffb3400c2e5b5c3771f62ea7/include/openssl/crypto.h.in#L1
62 WriteLn('OpenSSL version: ' + OpenSSLGetVersion(7)); //
OPENSSL_FULL_VERSION_STRING // Create an SSL context. ctx :=
SSLCTXnew(SslTLSMethod); if not Assigned(ctx) then begin
WriteLn('Could not create SSL context'); Halt(1); end; // Load
trusted certificates. if SSLCTXloadverifylocations(ctx,
PChar(trustedCertFile), '') <= 0 then begin WriteLn('Could not
load trusted certificates'); Halt(1); end; // Set
verification options. SSLCTXsetverify(ctx, SSL_VERIFY_PEER, nil);
SSLCTXsetmode(ctx, SSL_MODE_AUTO_RETRY); // Create SSL connection
ssl := SSLnew(ctx); if ssl = nil then begin Writeln('Error
creating SSL connection'); Halt(1); end; // Create simple
socket CSocket := fpsocket(AF_INET, SOCK_STREAM, 0); if CSocket = -1
then begin WriteLn('Error creating socket'); Halt(1); end;
// Resolve domain name to IP adress hrs :=
THostResolver.Create(nil); if hrs.NameLookup(Host) then begin
IPAddr := hrs.AddressAsString; WriteLn('Resolved IP Address: ',
IPAddr); end else begin WriteLn('Failed to resolve the hostname:
', Host); Halt(1); end; // Connect to host with Address do
begin sin_family := AF_INET; //TCP/IP sin_port:=
htons(word(Port)); //Port
sin_addr:=StrToNetAddr(hrs.AddressAsString); // IP address end;
hrs.Free; if fpconnect(CSocket, @Address, SizeOf(Address)) < 0 then
begin WriteLn('Error connecting to server.'); Halt(1); end;
writeln('Connected ', Host, ':', Port); // Set file descriptor (fd)
of connected socket to SSL connection SSLsetfd(ssl, CSocket); //
Establish SSL connection if SslConnect(ssl) <= 0 then begin
Writeln('SSL connection error.'); Halt(1); end; Writeln('SSL
connection established.'); WriteLn('----' + #13#10 + 'SSL connection
cipher description:'); //
https://www.openssl.org/docs/manmaster/man3/SSL_get_cipher.html
cipherPtr := SSLgetcurrentcipher(ssl);
PrintSSLCipherInfo(cipherPtr^); WriteLn('----'); //
https://www.openssl.org/docs/manmaster/man3/SSL_get_version.html
WriteLn('SSL/TLS version: ' + SslGetVersion(ssl)); WriteLn('----');
// https://www.openssl.org/docs/manmaster/man3/SSL_get_peer_certificat
e.html server_cert := SslGetPeerCertificate(ssl); WriteLn('Server
certificate:'); SetLength(tempBuf, 1024); //
https://www.openssl.org/docs/man3.0/man3/X509_NAME_oneline.html // ht
tps://www.openssl.org/docs/manmaster/man3/X509_get_subject_name.html
cert_str := X509NameOneline(X509GetSubjectName(server_cert), tempBuf,
Length(tempBuf)); cert_str := StringReplace(cert_str, '\xC2\xA0', '
', [rfReplaceAll]); WriteLn('Subject: ', cert_str); //
https://www.openssl.org/docs/manmaster/man3/X509_get_issuer_name.html
cert_str := X509NameOneline(X509GetIssuerName(server_cert), tempBuf,
Length(tempBuf)); cert_str := StringReplace(cert_str, '\xC2\xA0', '
', [rfReplaceAll]); WriteLn('Issuer: ', cert_str + #13#10); // We
could do all sorts of certificate verification stuff here before
deallocating the certificate //
https://www.openssl.org/docs/manmaster/man3/X509_free.html
X509free(server_cert); // Create an HTTP GET request. request :=
'GET / HTTP/1.1'#13#10 + 'Host: ' + Host + #13#10 + 'Connection:
close'#13#10 + 'User-Agent: Example TLS client'#13#10#13#10; //
Send the request to the server. WriteLn('--> Sending to the
server:'); Write(request); //
https://www.openssl.org/docs/manmaster/man3/SSL_write.html
nbytes_written := SslWrite(ssl, PChar(request), Length(request)); if
nbytes_written <> Length(request) then begin WriteLn('Could not
send all data to the server'); Halt(1); end; WriteLn('-->
Sending to the server finished'); WriteLn('<-- Receiving from the
server:'); // Receive and display the server's response. repeat
// https://www.openssl.org/docs/manmaster/man3/SSL_read.html
nbytes_read := SslRead(ssl, @in_buf, BUF_SIZE); if nbytes_read <= 0
then begin ssl_error := SSLgeterror(ssl, nbytes_read); if
ssl_error = SSL_ERROR_ZERO_RETURN then Break else
begin WriteLn('Error ', ssl_error, ' while reading data from
the server'); Halt(1); end; end;
Write(UTF8Encode(Copy(in_buf, 0, nbytes_read))); until False;
WriteLn('<-- Receiving from the server finished'); // Send SSL/TLS
close_notify //
https://www.openssl.org/docs/manmaster/man3/SSL_shutdown.html repeat
ssl_error := SslShutdown(ssl); until ssl_error <> 0; if
ssl_error = 1 then WriteLn('SSL connection sussefully closed.')
else //
https://www.openssl.org/docs/manmaster/man3/SSL_get_error.html
WriteLn('SSL connection closed with error.', SSLgeterror(ssl,
ssl_error)); // Close socket CloseSocket(CSocket); // Clean up.
if Assigned(ssl) then SslFree(ssl); if Assigned(ctx) then
SSLCtxFree(ctx);end.
----------------------------------------------------------------------
toby · 2023-09-14 16:27
doing Vespa mandarinia works ...
delphius
for me i tried to use the fpc sockets unit and indy library a long
time ago and got nowhere with either - synapse just worked immediately
and never had any problems since (that rvk couldn't fix)
i can't wait to see if your code fills in this hole - it would be nice
to have an fpc sockets program
paweld
all the .h files are ususally installed when compiling source - so you
probably never noticed them not being there - but the distros separate
them all into different (bothersome) packages
would you be upset if i replaced your nice synapse code with curl
code? (just joking i don't have time to do it anyway)
----------------------------------------------------------------------
toby · 2023-09-14 17:10
delphius
these are the kind of compiler errors i got originally when i first
tried the fpc sockets unit
your line is 42 is my line 47
opensslget.pas(47,7) Error: Wrong number of parameters specified for
call to "InitSSLInterface"
openssl.pas(5644,10) Error: Found declaration:
InitSSLInterface:Boolean;
openssl.pas(1495,10) Error: Found declaration:
InitSSLInterface(Boolean):Boolean;
InitSSLInterface(filepath + 'libssl' + ext, filepath + 'libcrypto' +
ext);
-
your filepath also points to the current dir (i guess for where the
.dll will be) but my libssl.so is in /usr/local/lib64/
which links to /usr/local/lib64/libssl.so.3
i am using a debian based os with fpc-3.2.2
---
----------------------------------------------------------------------
paweld · 2023-09-14 17:20
@delphius: You can use fphttpclient and that will work (the original
version uses that). It's just that fpc 3.2 supports openssl at max
version 1.1, so it's not very universal (which is @maurobio's goal)
because most Linuxes already have openssl version 3.
Quote from: tobywould you be upset if i replaced your nice synapse
code with curl code?I don't mind, I would even like to see another
solution to this problem
----------------------------------------------------------------------
rvk · 2023-09-14 17:25
Quote from: paweld on September 14, 2023, 05:00:48 pmQuote from:
rvkBut for a clean system, libssl-dev shouldn't be needed. Just the
libssl would be sufficient for the executable.I just now checked on a
freshly installed system (Debian 12). After the installation I have
openssl version 3.0.9, but the program does not download over https.
Only installing the libssl-dev package (also version 3.0.9) makes all
the data downloaded.
Uninstalling the package makes it fail again over https.
I would be interested to see the apt-get install log for that. What
gets installed alongside libssl-dev and also what get deleted again
when uninstalling it.
Looking at the contents of libssl-dev there isn't really anything in
that package itself which would be used at runtime. So it must be
something that gets installed alongside.
(I also found it strange that on a clean mint with OpenSsl 3, it works
without, but I'll check that myself again later on)
----------------------------------------------------------------------
delphius · 2023-09-14 17:53
Quote from: paweld on September 14, 2023, 07:20:09 pm@delphius: You
can use fphttpclient and that will work (the original version uses
that).
I am not referring to the already implemented functionality
(tfphttpclient), but something that is not yet available in clean fpc,
for example, sending e-mail via smtps, etc.
Quote from: paweld on September 14, 2023, 07:20:09 pmIt's just that
fpc 3.2 supports openssl at max version 1.1, so it's not very
universal (which is @maurobio's goal) because most Linuxes already
have openssl version 3.
1. Mormot wraper is production ready, its a pro, but it pulls
dependencies from its core, its a con
2. Native fpc opensslpas it is a collection of support for all
conceivable and unimaginable versions of the library for all
platforms, which on the one hand is good, but on the other requires
constant painstaking work to keep it up to date, which, as we see now,
has led to a strong lag
3. Guaranteed solution is either static assembly or delivery of the
necessary versions of the library files with the application, which is
a blessing for windows, but for linux it is a bad thing
----------------------------------------------------------------------
delphius · 2023-09-14 18:00
Quote from: toby on September 14, 2023, 07:10:04 pmthese are the kind
of compiler errors i got originally when i first tried the fpc sockets
unit
This is just an example written on the knee under windows and not
adapted to linux, clearly repeating the functionality of ssl/tls
connections implemented under the hood of fphttpclient
Quote from: toby on September 14, 2023, 07:10:04 pmyour filepath also
points to the current dir (i guess for where the .dll will be) but my
libssl.so is in /usr/local/lib64/
which links to /usr/local/lib64/libssl.so.3
i am using a debian based os with fpc-3.2.2
Try just to comment out Quote// InitSSLInterface(filepath + 'libssl' +
ext, filepath + 'libcrypto' + ext);and it will be automatically
initiated in the next function OpenSSLGetVersion with InitSSLInterface
without params
----------------------------------------------------------------------
rvk · 2023-09-14 18:48
Quote from: delphius on September 14, 2023, 07:53:11 pm2. Native fpc
opensslpas it is a collection of support for all conceivable and
unimaginable versions of the library for all platforms, which on the
one hand is good, but on the other requires constant painstaking work
to keep it up to date, which, as we see now, has led to a strong lag
The code used in openssl.pas comes from Synapse. The only downside is
that it's compiled in FPC. And you can't easily change the openssl
unit without recompiling FPC (which is hard when using release
Synapse).
BTW. I showed earlier that you can manipulate DLLVersions array to add
your own without the need of recompiling FPC (you just need to replace
an older version with '.3'.). If they made it an dynamic array it
would heven be easier. But you can also use Synapse itself which is
updated for the latest versions on Windows and Linux and you don't
need to recompile FPC either for using that.
Yes, you can create the whole openssl.pas from Synapse yourself in
code (or do something like you showed) but why would you do that if
there is Synapse? And what to do with the special functions for using
sockets and openssl (which are all in Synapse)?
Just using Synapse seems much easier and stable to me.
Synapse on Android might be a challenge (although there is a version
for it I think) but changing that socket/ssl code to support Android
seems like a challenge too
PS. Your code is still very interesting for the purist under us who
want to use as few units as possible
But would also need to be expanded for 64 bit (which has another dll),
Linux, Android etc.
You'll probably end up with a unit similar to openssl.pas from
Synapse.
----------------------------------------------------------------------
delphius · 2023-09-14 19:07
Quote from: rvk on September 14, 2023, 08:48:12 pmJust using Synapse
seems much easier and stable to me.
It seems to me that this is the main reason, and for different people
you just need to substitute the name of the net library they are
familiar with
Quote from: rvk on September 14, 2023, 08:48:12 pm Your code is still
very interesting for the purist under us who want to use as few units
as possible
Thank you, I'm doing all this in the context of the solving task of
writing a collection of simple functions for sending emails from fpc,
now it's OpenSSL's turn, there is already implemented ssl/tls
functionality using Cryptlib (I still can't understand why it is not
popular)
Quote from: rvk on September 14, 2023, 08:48:12 pmBut would also need
to be expanded for 64 bit (which has another dll), Linux, Android etc.
You'll probably end up with a unit similar to openssl.pas from
Synapse.
It's too ambitious, I'm trying to solve feasible tasks that will
benefit here and now.
But if we approach this issue conceptually, then I would simply
transpile the last openssl lib c header file(s) for the current
library development branch (3.x) and rely on it, discarding all the
old versions, taking into account the end of support for old branches
----------------------------------------------------------------------
rvk · 2023-09-14 19:18
Quote from: delphius on September 14, 2023, 09:07:57 pmThank you, I'm
doing all this in the context of the solved task of writing a
collection of simple functions for sending emails from fpc, now it's
OpenSSL's turn, there is already implemented ssl/tls functionality
using Cryptlib (I still can't understand why it is not popular)The
last time I checked, Cryptlib isn't free to use in commercial closed-
source programs.
QuoteIn addition, cryptlib is often free for use in low-cost, non-
open-source applications such as shareware, and for personal and
research use.
----------------------------------------------------------------------
toby · 2023-09-14 19:27
i had commented it to see if it would compile but had a haltl after it
this is what i get when i remove the halt
OpenSSL version:
Could not create SSL context
i don't know if you plan or want to spend time having it work on linux
- but i am here for you
(bribe rvk to look at it
----------------------------------------------------------------------
maurobio · 2023-09-14 20:24
Hi, Gentlemen!
I have yet more news to report.
First, I did a small change to the code provided by @paweld. I did not
made any changes to the code itself, just moved four routines
(GetUrlToFile, GetImgUrlToBase64Png, GetAndMergeImages, IsOnline) from
the BioWS unit to the main program unit. This is because these
routines are used only by the e-Species Desktop application, and the
online searching routines in BioWS are intended to be used in other
programs too (as part of my BioLib project, about which I can tell
more in another moment).
In the process of moving these four routines between units, I found
what looks like a curious bug in GetImgUrlToBase64Png. Here is the
declaration of this function:
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i function
GetImgUrlToBase64Png(Url: string; InlineImg: boolean; newWidth:
integer; newHeight: integer; Proportional: boolean): string;
Before moving it from the BioWS unit to the main unit, it was called
by the main program as follows:
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
GetImgUrlToBase64Png(imgUrl, True, 94, 145)
Notice the missing last boolean parameter. But no error was issued!
However, when I moved the function declaration to the main program
unit, compilation failed because of that missing parameter; I then
just change the call to
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
GetImgUrlToBase64Png(imgUrl, True, 94, 145, True)
and now of course compilation was successful.
Second, I found what may be an "intermittent" bug which appears only
at times: when I search for the name "Panthera leo" (the African lion)
sometimes no bibliographic reference is returned; other times a list
of references related to that species is displayed as expected. I
didn't found this problem with any other species name which I tested.
Is there any conceivable way around this?
Implemented this minor modification as suggested by @rvk
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
Result1 := EvaluateXPathExpression('//Article/ArticleTitle',
Doc.DocumentElement); Result2 := EvaluateXPathExpression('//Pub
medData/ArticleIdList/ArticleId[@IdType="doi"]',
Doc.DocumentElement); NodeSet1 := Result1.AsNodeSet;
NodeSet2 := Result2.AsNodeSet; if NodeSet1.Count > 0 then
begin for i := 0 to NodeSet1.Count - 1 do try
if i <= NodeSet2.Count - 1 then
references.Add(TDomElement(NodeSet1.Items[i]).TextContent + '=' +
TDomElement(NodeSet2.Items[i]).TextContent); except
continue; end; end;
caused other names (eg., "Vicia faba") to return no reference (as in
the above mentioned "Panthera leo" search).
Quoteold version of synapse (40, stable) does not support openssl 3
Sure, I am using the trunk version, just commented about being able to
compile and run the program with the original stable version of
Synapse under Windows (it does not even compile under Linux).
I have not yet time to test the Linux version (probably will get it
done only by tomorrow).
With warmest regards,
----------------------------------------------------------------------
maurobio · 2023-09-14 20:38
Hi, @rvk! Ave, @delphius!
QuoteJust using Synapse seems much easier and stable to me.
I like the biologically-inspired name too... But now I have to tike
the time to learn how to use it, whereas I was already (more or less)
familiar with fhttpclient.
QuoteSynapse on Android might be a challenge (although there is a
version for it I think) but changing that socket/ssl code to support
Android seems like a challenge too
And now for an interesting coincidence... @rvk mentioned Android
precisely when I intended to post a message asking about that!
That's because I am crazy (or stupid, or both) enough to have start
developing an Android version (using LAWM) of eSpecies which partially
works! I am using the jHttpClient which comes with LAMW but my code
(based on the (now outdated) online searching routines used by
eSpecies Desktop is plagued probably by the very same problems already
pointed out by @rvk. If you wizards would like to take a look at my
humble and messy code, I can of course share the code anytime! Just
imagine being able to look up information on any biological species
anywhere in the world using a mobile device!
With warmest regards,
----------------------------------------------------------------------
paweld · 2023-09-14 21:12
Quote from: maurobioIn the process of moving these four routines
between units, I found what looks like a curious bug in
GetImgUrlToBase64PngParameters of functions / procedures can have
defined default values. The function you specified had them set like
this:Code: [Select]function GetImgUrlToBase64Png(Url: String;
InlineImg: Boolean; newWidth: Integer = 0; newHeight: Integer = 0;
Proportional: Boolean = False): String; Quotewhen I search for the
name "Panthera leo" (the African lion) sometimes no bibliographic
reference is returned; other times a list of references related to
that species is displayed as expected.I tested several times and did
not experience this problemQuoteCode: [Select]if NodeSet1.Count > 0
then
begin
for i := 0 to NodeSet1.Count - 1 do//...This `if` is not needed
here, because if the list is empty (count = 0) then the loop will not
execute (`for i := 0 to -1 do`)
----------------------------------------------------------------------
delphius · 2023-09-14 21:49
Quote from: toby on September 14, 2023, 09:27:33 pmi don't know if you
plan or want to spend time having it work on linux - but i am here for
you
1. Installed debian based linux in a virtual machine
2. Installed the latest version of fpc&laz (trunk) using fpcupdeluxe
3. Installed and configured VSCode (omnipascal + tasks.json)
4. Compiled and works without errors (screenshots attached).
Try it too on the latest version of fpc
----------------------------------------------------------------------
toby · 2023-09-14 22:30
paweld
a simple curl code that does away with direct openssl code etc
always save my curl to file in case they change stuff and don't have
to reget their pages
if don't want it and want it in a tstringlist directly use
tprocess.execute etc
uncomment line 25 to enable downloading
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
ecurl; uses sysutils, classes; // classes for astringlist label
10; const list : array[1..5] of ansistring = ('asian giant hornet',
'lima bean', 'apple', 'human', 'fava bean'); var en, s, d, d1, l :
ansistring; n, i : longint; sp : tstringlist; html :
text; begin writeln; sp := tstringlist.create; for i := 1 to
high(list) do begin en := stringreplace(list[i], ' ', '_',
[rfReplaceAll]); l := '-A Mozilla/4.0 -L -k -o ' + en + '.html ' +
'https://en.wikipedia.org/wiki/' + en; writeln('retrieving ', l);//
sysutils.executeprocess('/ram/curl', l); // line 25 uncomment to do
download assign(html, en + '.html'); reset(html); d := '';10:
readln(html, d1); d := d + d1; if not eof(html) then goto 10;
close(html); //Species:V. mandarinia ', d);
delete(d, 1, (n + 2)); n := pos('', d); s := copy(d, 1, (n -
1)); s := stringreplace(s, ' ', ' ', [rfReplaceAll]);
writeln('Species : ', s); sp.add(s);// writeln(sp.count); writeln;
end;writeln; for i := 1 to sp.count do writeln(list[i], ' : ', sp[i -
1]); sp.free; writeln; end.
now to concentrate on the code from delphius
----------------------------------------------------------------------
rvk · 2023-09-14 22:48
Quote from: paweld on September 14, 2023, 11:12:01 pmParameters of
functions / procedures can have defined default values. The function
you specified had them set like this:Code: [Select]function
GetImgUrlToBase64Png(Url: String; InlineImg: Boolean; newWidth:
Integer = 0; newHeight: Integer = 0; Proportional: Boolean = False):
String;
Yes, in the interface it was declared as:
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i function
GetImgUrlToBase64Png(Url: String; InlineImg: Boolean; newWidth:
Integer = 0; newHeight: Integer = 0; Proportional: Boolean = False):
String;But in the implementation it is like this:
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i function
GetImgUrlToBase64Png(Url: String; InlineImg: Boolean; newWidth:
Integer; newHeight: Integer; Proportional: Boolean): String;Although
it's not a problem with compiling, my preference is to always match
those two, so it's also clear in the implementation that the
Proportional parameter is optional.
Quote from: maurobio on September 14, 2023, 10:24:33 pmSecond, I found
what may be an "intermittent" bug which appears only at times: when I
search for the name "Panthera leo" (the African lion) sometimes no
bibliographic reference is returned; other times a list of references
related to that species is displayed as expected. I didn't found this
problem with any other species name which I tested. Is there any
conceivable way around this?Those intermittent problems come because
you have the exception handling on many places without ANY feedback.
You just return result = empty. But it's sometimes better to give
feedback to what exactly is going on with those exceptions. When
running in the IDE you WILL receive those exceptions dialogs (even if
you handle the exception afterwards). Then you can see where things go
wrong.
That's why I said it's best to examine exceptions and always give
feedback. If you get feedback for which you can explain the cause,
like that ResultSet1 being a different size than ResultSet2, you
should program around that, so that you don't get an exception for
that anymore. You end up only with REAL exceptions which are
unexpected, and it is good to give feedback to the user then.
Now the exceptions are all handled silently and you don't know when
and if they occur, end you just end up with an 'empty' result.
Quote from: maurobio on September 14, 2023, 10:24:33 pmImplemented
this minor modification as suggested by @rvk
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
Result1 := EvaluateXPathExpression('//Article/ArticleTitle',
Doc.DocumentElement); Result2 := EvaluateXPathExpression('//Pub
medData/ArticleIdList/ArticleId[@IdType="doi"]',
Doc.DocumentElement); NodeSet1 := Result1.AsNodeSet;
NodeSet2 := Result2.AsNodeSet; if NodeSet1.Count > 0 then
begin for i := 0 to NodeSet1.Count - 1 do try
if i <= NodeSet2.Count - 1 then
references.Add(TDomElement(NodeSet1.Items[i]).TextContent + '=' +
TDomElement(NodeSet2.Items[i]).TextContent); except
continue; end; end;
caused other names (eg., "Vicia faba") to return no reference (as in
the above mentioned "Panthera leo" search).
That's strange. That would also happen if you didn't add that if line.
Because when i > NodeSet2.Count - 1 you would get an exception and you
would continue.
So with that line the exception doesn't happen and should return the
same as when the exception did happen.
I think there is some other exception going on there (for example with
retrieving the data), but because you handle that silently, you don't
see that.
My advice... build showmessage's in ALL except parts and just let them
happen (visibly). Then, go through them and solve them all via code
(like I did here with the if line).
Don't handle exceptions silently (unless you know why the exception is
happening and it is intended, which is certainly not true for an empty
result)
----------------------------------------------------------------------
toby · 2023-09-14 23:16
i am not just trying to get
OpenSSLGetVersion to work
right after your begin i have
or i := 1 to 10 do // test all integers 7 fails?
WriteLn('OpenSSL version: ' + OpenSSLGetVersion(i)); //
OPENSSL_FULL_VERSION_STRING
halt;
i have the latest openssl.pas from
git clone https://gitlab.com/freepascal.org/fpc/source.git fpc
202841 Sep 14 2023 17:01:38 openssl.pas
the following openssl.rsjwas generated in the dir i am working in - to
be honest i don't think i've ever seen a .rsj file before
openssl.rsj
{"version":1,"strings":[
{"hash":251958569,"name":"openssl.sfailedtoloadopenssl","sourcebytes":
[70,97,105,108,101,100,32,116,111,32,108,111,97,100,32,79,112,101,110,
83,83,76,32,108,105,98,114,97,114,121],"value":"Failed to load OpenSSL
library"}
]
when i compile with my new -uvt toy i see
openssl.pas(1704,32) Hint: C arrays are passed by reference
----------------------------------------------------------------------
rvk · 2023-09-14 23:24
Quote from: toby on September 15, 2023, 01:16:29 ami am not just
trying to get
OpenSSLGetVersion to work
right after your begin i have
or i := 1 to 10 do // test all integers 7 fails?
WriteLn('OpenSSL version: ' + OpenSSLGetVersion(i)); //
OPENSSL_FULL_VERSION_STRING
And what openssl version do you have on your computer?
Code: [Select]openssl version
(I thought OpenSSLGetVersion was only available for 3+, or maybe
1.1+.)
----------------------------------------------------------------------
toby · 2023-09-14 23:35
openssl version
OpenSSL 3.0.5 5 Jul 2022 (Library: OpenSSL 3.0.5 5 Jul 2022)
----------------------------------------------------------------------
delphius · 2023-09-15 04:51
Quote from: toby on September 15, 2023, 01:16:29 amor i := 1 to 10 do
// test all integers 7 fails?
And why not from 0 to 9?
These constants are for version 3+, in 1.1.1 there were 6 of them
Quoteconst char *OpenSSL_version(int type);
# define OPENSSL_VERSION 0
# define OPENSSL_CFLAGS 1
# define OPENSSL_BUILT_ON 2
# define OPENSSL_PLATFORM 3
# define OPENSSL_DIR 4
# define OPENSSL_ENGINES_DIR 5
# define OPENSSL_VERSION_STRING 6
# define OPENSSL_FULL_VERSION_STRING 7
# define OPENSSL_MODULES_DIR 8
# define OPENSSL_CPU_INFO 9
QuoteOpenSSL_version() returns different strings depending on t:
OPENSSL_VERSION
The value of OPENSSL_VERSION_TEXT
OPENSSL_VERSION_STRING
The value of OPENSSL_VERSION_STR
OPENSSL_FULL_VERSION_STRING
The value of OPENSSL_FULL_VERSION_STR
OPENSSL_CFLAGS
The compiler flags set for the compilation process in the form
compiler: ... if available, or compiler: information not available
otherwise.
OPENSSL_BUILT_ON
The date of the build process in the form built on: ... if available
or built on: date not available otherwise. The date would not be
available in a reproducible build, for example.
OPENSSL_PLATFORM
The "Configure" target of the library build in the form platform: ...
if available, or platform: information not available otherwise.
OPENSSL_DIR
The OPENSSLDIR setting of the library build in the form OPENSSLDIR:
"..." if available, or OPENSSLDIR: N/A otherwise.
OPENSSL_ENGINES_DIR
The ENGINESDIR setting of the library build in the form ENGINESDIR:
"..." if available, or ENGINESDIR: N/A otherwise. This option is
deprecated in OpenSSL 3.0.
OPENSSL_MODULES_DIR
The MODULESDIR setting of the library build in the form MODULESDIR:
"..." if available, or MODULESDIR: N/A otherwise.
OPENSSL_CPU_INFO
The current OpenSSL cpu settings. This is the current setting of the
cpu capability flags. It is usually automatically configured but may
be set via an environment variable. The value has the same syntax as
the environment variable. For x86 the string looks like CPUINFO:
OPENSSL_ia32cap=0x123:0x456 or CPUINFO: N/A if not available.
For an unknown t, the text not available is returned.
----------------------------------------------------------------------
rvk · 2023-09-15 11:03
Quote from: paweld on September 14, 2023, 05:00:48 pmQuote from:
rvkBut for a clean system, libssl-dev shouldn't be needed. Just the
libssl would be sufficient for the executable.I just now checked on a
freshly installed system (Debian 12). After the installation I have
openssl version 3.0.9, but the program does not download over https.
Only installing the libssl-dev package (also version 3.0.9) makes all
the data downloaded.
Uninstalling the package makes it fail again over https.
I had some time to investigate this. Why was libssl-dev package
needed?
It seems that FPC/openssl doesn't have a problem and for openssl in
FPC the libssl-dev doesn't need to be installed.
The reason for this is that the openssl in FPC searches for ALL
possible library names of openssl (so 1.0.2, 1.1 and 3).
BUT... the openssl from Synapse only checks the basic library name:
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i DLLSSLName:
string = 'libssl.so';DLLUtilName: string = 'libcrypto.so';
So when you use the openssl from Synpase you need to have a symlink
pointing to the actual openssl library.
And the libssl-dev happens to makes that symlink:
Code: [Select]libssl-dev: /usr/lib/aarch64-linux-gnu/libcrypto.so
libssl-dev: /usr/lib/aarch64-linux-gnu/libssl.so
You could make those symlink yourself in your program but that's the
only thing that's needed, not the entire libssl-dev itself.
(But installing libssl-dev might be a bit easier )
----------------------------------------------------------------------
rvk · 2023-09-15 12:23
O, and @paweld and @maurobio.
I found a few other issues.
In Snippet() and Images() there is a construct to do a Get() on the
redirect URL.
If it succeeds then there is a assignment to queryStr (from the
redirect or direct from searchStr).
But if it fails, then queryStr is empty. Normally the Get() won't fail
but you can't be certain of that.
So queryStr should be initialized to searchStr anyway at the start of
those procedures.
And to @paweld... I saw you catch every except in the LPDNetU.
For a 'low-level' library, maybe it should not catch those exceptions
but pass them on to the caller.
Now it wasn't even possible to catch the exception that OpenSSL wasn't
found because the exception was ignored silently.
----------------------------------------------------------------------
maurobio · 2023-09-15 13:18
Hi, Gentlemen!
I have some weird news...
Just completed my tests on Windows 10, Linux Mint 19.1 (32-bits) and
Lubuntu Linux 18.04 (64-bits), with Lazarus 2.0.12 and OpenSSL 1.1.1
installed in Linux and OpenSSL 1.0.2 installed in Windows.
For comparison, I tested @paweld modified code using Synapse and my
original version of eSpecies, with a fix for the problem with
Wikipedia, substituting the previous code with the one below:
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i function
TWikiSearch.Snippet(const searchStr: string): string;var JsonData:
TJsonData; Client: TFPHttpClient;begin Client :=
TFPHttpClient.Create(nil); try try { Allow redirections }
Client.AllowRedirect := True; JsonData :=
GetJson(Client.Get(WIKIPEDIA_URL + StringReplace(searchStr, '
', '_', [rfReplaceAll]))); Result :=
JsonData.FindPath('extract').AsUnicodeString; except Result :=
''; end; finally JsonData.Free; Client.Free; end;end;
With @paweld code, the program worked fine under Windows but did not
work under either Linux distro: no errors were issued, but no search
results were returned (see the screenshot below)!
However, on all platforms, my original code worked fine (with just the
above modification).
I also tested the program in two machines without Lazarus installed,
one with Windows 7 (32-bits) with OpenSSL 1.0.2 -- where both my
original version and @paweld's worked -- and another one with Linux
Mint 21 (64-bits) with OpenSSL 3.0.2 -- where neither my original
version nor @paweld version worked.
To sum up: it seems that in the end the really serious issue is the
version of OpenSSL installed on a given machine; it seems that the
program does not work with version 3.x. Maybe, it does not matter if
one is using Synapse or the FPC networking routines, because if
neither support OpenSSL 3.x, the program will not work anyway.
I obviously cannot oblige users to downgrade their Linux systems by
removing OpenSSL 3.x and replacing it with OpenSSL 1.x (because that
would break their systems), but would it be possible to have both
versions (1.x and 3.x) installed on the same machine? And if so, how
to instruct the program to use the older version instead of the newer
one?
With warmest regards,
With warmest regards,
----------------------------------------------------------------------
delphius · 2023-09-15 13:19
In order not to clog up the topic of the respected maurobio with
unnecessary information, I took the discussion of the direct use of
OpenSSL in a separate topic
----------------------------------------------------------------------
maurobio · 2023-09-15 13:24
Ave, @delphius!
QuoteIn order not to clog up the topic of the respected maurobio with
unnecessary information, I took the discussion of the direct use of
OpenSSL in a separate topic
I just noticed it. I will follow that discussion, because your
proposal could perhaps make things simpler. But in this matter I am
just an amateur and can only follow the discussions in the hope that
some useful code arises from them!
With warmest regards,
----------------------------------------------------------------------
rvk · 2023-09-15 13:29
Quote from: maurobio on September 15, 2023, 03:18:00 pmI also tested
the program in two machines without Lazarus installed, one with
Windows 7 (32-bits) with OpenSSL 1.0.2 -- where both my original
version and @paweld's worked -- and another one with Linux Mint 21
(64-bits) with OpenSSL 3.0.2 -- where neither my original version nor
@paweld version worked.Read my last post.
For the openssl in Synapse to work you need a symlink to the openssl
libraries.
The package libssl-dev will provide that. Do a apt-get and it should
work.
sudo apt-get install libssl-dev
You can create the symlinks yourself but installing that package is
the easiest solution.
Unfortunatly the exception for OpenSSL not being present doesn't
surface because now a lot of exceptions are handled silently (they are
just ignored). Not the best practice in my opinion.
----------------------------------------------------------------------
delphius · 2023-09-15 13:31
Quote from: maurobio on September 15, 2023, 03:24:46 pmbecause your
proposal could perhaps make things simpler.
It is for this reason that I got into the discussion, because in my
opinion, in your case no complications are required and there should
be enough features provided by the standard fpc libraries.
And suggestions, like the working, but odious use of curl directly,
only throw you off course.
----------------------------------------------------------------------
rvk · 2023-09-15 13:35
Quote from: delphius on September 15, 2023, 03:31:38 pmIt is for this
reason that I got into the discussion, because in my opinion, in your
case no complications are required and there should be enough features
provided by the standard fpc libraries.
The problem with the standard library is that it doesn't yet support
OpenSSL 3.
You could program code like your showed but that only works for
Windows (and here both Linux and Windows is needed, and even Android
in the future).
So if you can provide the same code cross-platform it can be used.
(But you'll end up with roughly the code as openssl.pas )
The downside of openssl.pas is that the DLLVersions is not dynamic but
you can replace the oldest version with .3 and it will work for the
newer OpenSSL 3 too (as I showed).
(but at that point we already moved on to Synapse which works fine
too, I use that a lot too, also in Delphi.)
----------------------------------------------------------------------
maurobio · 2023-09-15 13:45
Hi, @rvk!
QuoteThe package libssl-dev will provide that.
Well, I can understand this, but I did not did it because of another
post where in a reply to @paweld you asked if libssl-dev should be
really necessary... I just got confused!
QuoteUnfortunately the exception for OpenSSL not being present doesn't
surface because now a lot of exceptions are handled silently (they are
just ignored). Not the best practice in my opinion.
I do understand and agree. I intend to improve the code for handling
the exceptions in a more careful way, but I first need to have working
versions of the program, so that I can proceed from them! So, I first
needed to understand that some of the problems with my program were
simply these issues with OpenSSL (and it took me days to do so...).
With warmest regards,
----------------------------------------------------------------------
maurobio · 2023-09-15 13:49
Ave, @delphius!
QuoteAnd suggestions, like the working, but odious use of curl
directly, only throw you off course.
I do admit that in fact, sometimes I get lost along these discussions,
because I don't know if the problem is with my code, the networking
libraries used in the code, the system's required libraries, or a
combination of all these ...
With warmest regards,
----------------------------------------------------------------------
rvk · 2023-09-15 14:42
Quote from: maurobio on September 15, 2023, 03:45:37 pmQuoteThe
package libssl-dev will provide that.Well, I can understand this, but
I did not did it because of another post where in a reply to @paweld
you asked if libssl-dev should be really necessary... I just got
confused!
Yes, that was a reply from me. I was puzzled as to why libssl-dev was
needed.
But I checked it out and it seems Synpase/openssl needs the symlink
libssl-dev provides (besides the header files).
I was puzzled because the openssl.pas version from FPC doesn't need
that symlink
----------------------------------------------------------------------
paweld · 2023-09-15 14:55
Quote from: rvkI had some time to investigate this. Why was libssl-dev
package needed?
It seems that FPC/openssl doesn't have a problem and for openssl in
FPC the libssl-dev doesn't need to be installed.
The reason for this is that the openssl in FPC searches for ALL
possible library names of openssl (so 1.0.2, 1.1 and 3).
BUT... the openssl from Synapse only checks the basic library name:
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i DLLSSLName:
string = 'libssl.so';DLLUtilName: string = 'libcrypto.so';
So when you use the openssl from Synpase you need to have a symlink
pointing to the actual openssl library.
And the libssl-dev happens to makes that symlink:
Code: [Select]libssl-dev: /usr/lib/aarch64-linux-gnu/libcrypto.so
libssl-dev: /usr/lib/aarch64-linux-gnu/libssl.so
You could make those symlink yourself in your program but that's the
only thing that's needed, not the entire libssl-dev itself.
(But installing libssl-dev might be a bit easier )
@rvk thank you very much for this valuable information.
Quote from: rvkAnd to @paweld... I saw you catch every except in the
LPDNetU.
For a 'low-level' library, maybe it should not catch those exceptions
but pass them on to the caller.
Now it wasn't even possible to catch the exception that OpenSSL wasn't
found because the exception was ignored silently.But these are all
functions, so they return False in case of any failure. But you are
right, it would be useful to know about the inability to load openssl
for encrypted connections.
Quote from: maurobioWith @paweld code, the program worked fine under
Windows but did not work under either Linux distro: no errors were
issued, but no search results were returned (see the screenshot
below)!as @rvk said - you must install libssl-dev package.
When I find some time I will try to make some changes to the code,
implementing the ideas suggested by @rvk.
----------------------------------------------------------------------
toby · 2023-09-15 23:25
delphius
how are you using
InitSSLInterface(filepath + 'libssl' + ext, filepath + 'libcrypto' +
ext);
when the function in openssl.pas is
function InitSSLInterface(AVerboseLoading: Boolean): Boolean ;
overload; deprecated;
your rude comment about my curl code is not appreciated in the least -
i'm pretty sure more people use curl then use fpc sockets and don't
think using it is odious
---
rvk
is my problem the the openssl.pas doesn;t support openssl-3 - i didn't
see amy UNIX or LINUX defs in it?? is it just fopr ps2 os2 and
windows?
after upgrading to fpc-3.1.1 the openssl.rsj file never came back
looks like it supports libressl - i've tried libressl on another
system and i think it was slower?? so i went back to openssl
---
----------------------------------------------------------------------
delphius · 2023-09-16 05:20
Quote from: toby on September 16, 2023, 01:25:14 amyour rude comment
about my curl code is not appreciated in the least - i'm pretty sure
more people use curl then use fpc sockets and don't think using it is
odious
I deeply apologize if I offended you by this comment, since I use
machine translation and I am not a native speaker, I do not fully
understand how correct the output phrases are
In fact, I just wanted to warn the author of the topic against rash
steps, because at one time I tried to help him with setting up the
first version of his program (cgi) on a free hosting, which of course
was not successful due to significant restrictions to such programs on
such hostings.
He relies entirely on our competent opinion, but the task was
extremely simple and did not require anything other than the
capabilities available in fpc and it remained so in my opinion.
As long as I follow the discussion, problems in the code itself are
now surfacing, not related to language capabilities or technologies
(cgi, desktop/web app, curl or something else), but related to the
processing of all possible situations and options, errors, etc.
And I still think that the author of the topic could safely use the
very first option (cgi), although for minimal money, for example in a
docker container on cloud hosting, but he needs an absolutely free
open hosting method, so he went further
And I also understand that everything that is not being done is for
the best, since the business logic of his application does not change
as a result of these studies, but only improves (now with the help of
the community), and the author himself gains knowledge and skills of
new technologies by implementing various wrappers for his business
logic
However, whatever way we would ultimately choose to provide this
amazing free open source service to the world, the essence remains the
essence, we just need to give it round-the-clock uninterrupted access
and it does not matter for the end user how it will work under the
hood.
Quote from: toby on September 16, 2023, 01:25:14 amhow are you using
InitSSLInterface(filepath + 'libssl' + ext, filepath + 'libcrypto' +
ext);
when the function in openssl.pas is
function InitSSLInterface(AVerboseLoading: Boolean): Boolean ;
overload; deprecated;
I use a newer version of FPC than you, so my openssl.pas is different
from yours, you can make sure that the function I use is present in
the latest version, and the function version you are using is there
too for compatibility with old versions
But the method I use (forced using dll/so from the program folder) is
not approved under linux (although I can't understand who can forbid
me to distribute an application for any operating system this way or
even by statically compiling the necessary libraries inside the
program, since the final task for me is just to make it work right
away and not require no additional actions from the user)
P.S. Colleagues are already doing a separate header file for openssl3
for the amiga while we are discussing our problems here
----------------------------------------------------------------------
maurobio · 2023-09-16 11:56
Hi, @toby!
Quoteyour rude comment about my curl code is not appreciated in the
least - i'm pretty sure more people use curl then use fpc sockets and
don't think using it is odious
Gentlemen, please! Seeing you, wise wizards, quarrelling because of my
humble program causes me as much anxiety and consternation as that
which afflicted Alberto Santos Dumont when he saw his great invention,
the airplane, being used as a killing machine in the First World
War... :'(
I am sure that @delphius had no intention to offend anyone and, as he
himself explained, his use of a less than polite term to refer to Curl
(which we all do agree is a really great tool!) was just a
misunderstanding.
Peace & Love.
With warmest regards,
----------------------------------------------------------------------
maurobio · 2023-09-16 12:07
@Ave, delphius!
QuoteHe relies entirely on our competent opinion, but the task was
extremely simple and did not require anything other than the
capabilities available in fpc and it remained so in my opinion.
Absolutely true!
QuoteAs long as I follow the discussion, problems in the code itself
are now surfacing, not related to language capabilities or
technologies (cgi, desktop/web app, curl or something else), but
related to the processing of all possible situations and options,
errors, etc.
Quite right!
QuoteAnd I also understand that everything that is not being done is
for the best, since the business logic of his application does not
change as a result of these studies, but only improves (now with the
help of the community), and the author himself gains knowledge and
skills of new technologies by implementing various wrappers for his
business logic
Fully agree!!!
QuoteHowever, whatever way we would ultimately choose to provide this
amazing free open source service to the world, the essence remains the
essence, we just need to give it round-the-clock uninterrupted access
and it does not matter for the end user how it will work under the
hood.
Indeed!
QuoteBut the method I use (forced using dll/so from the program
folder) is not approved under linux (although I can't understand who
can forbid me to distribute an application for any operating system
this way or even by statically compiling the necessary libraries
inside the program, since the final task for me is just to make it
work right away and not require no additional actions from the user)
And that is the method I would unashamedly embrace and use if any of
you provide me with the instructions for doing so!! In my humble
(and no doubt even amateurish) opinion, this would make everything
simpler, both for the end users and the developers. Why not just
include the required libraries (.so files) in the distribution package
for Linux and forcing the program to use these files, no matter what
version of OpenSSL is installed in a user's system? This would
provide for much the same simple solution as in Windows? BTW, why some
deem a "bad idea" to install DLL's in the Windows system folder (and
many programs, including Lazarus itself, do just that) but consider
the "right thing" to install .so files in Linux file system?
With warmest regards,
----------------------------------------------------------------------
rvk · 2023-09-16 13:30
Quote from: maurobio on September 16, 2023, 02:07:37 pmWhy not just
include the required libraries (.so files) in the distribution package
for Linux and forcing the program to use these files, no matter what
version of OpenSSL is installed in a user's system? This would
provide for much the same simple solution as in Windows?Have you ever
tried to take one little pas file source from a very big library?
One of the procedures from the Jedi JVCL and JCL libraries for
example.
Then you come to the conclusion you need to include another unit.
But that unit makes use of another unit. And another. And another.
You end up with including the complete library anyway.
The same goes (mostly) for .so files in Linux.
They depend on eachother.
The libssl.so depends on a certain version of libc6 which depends on a
certain version of yet another lib, etc. etc.
Ever tried to install PHP 5 on a newer machine?
It asks for the dependency of an older version of a certain library.
But you can't install that library because you have a newer version of
it and downgrading isn't possible because other components of the
system depend on the newer one.
No, it's not that easy. But you are welcome to try.
You could copy those .so files and set them in the correct
LD_LIBRARY_PATH (or add you directory to it).
Quote from: maurobio on September 16, 2023, 02:07:37 pmBTW, why some
deem a "bad idea" to install DLL's in the Windows system folder (and
many programs, including Lazarus itself, do just that) but consider
the "right thing" to install .so files in Linux file system?The
copying to the system directory was once the idea in Windows but
became a problem when every program copied there own version (old or
new) to it.
What if your program depends on the newer version but another program
copies an older version to the system directory.
No... copying to the system directory is discouraged for a long time
now.
You can let the official releases handle the system directory. That's
also the case on Linux. Install via apt-get and let the newest known
version install.
----------------------------------------------------------------------
maurobio · 2023-09-16 13:42
Hi, @rvk!
I can understand what you point out, but on the other side what I want
is to have a reliable way of distributing the binary executable of the
Linux version of my application (as a Debian package) without having
the user to type any further commands in a terminal window!
Someone suggested that I add the line:
openssl (>=1.0.2k)
on the dependencies list of the package, but shouldn't it be libssl1.1
(or 1.0.2, or 3.0.2, whatever)? And what about libssl-dev? Should it
be included also in the dependencies list? If so, what version number?
With warmest regards,
----------------------------------------------------------------------
Thaddy · 2023-09-16 13:46
Quote from: rvk on September 16, 2023, 03:30:34 pmNo... copying to the
system directory is discouraged for a long time now.
That is your most unprofessional answer ever... I will send it to
Guinness.
----------------------------------------------------------------------
rvk · 2023-09-16 13:56
Quote from: Thaddy on September 16, 2023, 03:46:46 pmQuote from: rvk
on September 16, 2023, 03:30:34 pmNo... copying to the system
directory is discouraged for a long time now.
That is your most unprofessional answer ever... I will send it to
Guinness.
Really? Do you encourage it then?
----------------------------------------------------------------------
maurobio · 2023-09-16 14:02
Gentlmen,
QuoteThat is your most unprofessional answer ever... I will send it to
Guinness.
With all due respect, isn't this beyond the point of the discussion?
Even if it ware just an opinion (and I'm not saying it is, I take it
as a very professional advice), isn't he entitled to his opinion?
Maybe I will regret these words, but I fail to see where can we go
amid such harshness. :'(
Peace!
With best wishes,
----------------------------------------------------------------------
toby · 2023-09-16 15:33
delphius
you are correct
i have upgraded to 3.3.1
writeln(InitSSLInterface(filepath + 'libssl' + ext, filepath +
'libcrypto' + ext));
writeln(InitSSLInterface());
both return true in 3.3.1 but gives the error on 3.2.2
you can practice your english on us - we'll understand - more
important the language we share in common is fpc (sorry i entered say
something schmaltzy into my AI translator and this is what came out)
---
i tried this on computer with openssl-1.0.2 but with fpc-3.2.2
(same output with fpc-3.3.1 with openssl-3.0.5)
for i := 0 to 9 do WriteLn(i, ' : OpenSSL version: ' +
OpenSSLGetVersion(i)); // OPENSSL_FULL_VERSION_STRING
0 : OpenSSL version:
1 : OpenSSL version:
2 : OpenSSL version:
3 : OpenSSL version:
4 : OpenSSL version:
5 : OpenSSL version:
6 : OpenSSL version:
7 : OpenSSL version:
8 : OpenSSL version:
9 : OpenSSL version:
---
rvk
i also have the openssl-1.0.2 libs installed on a computer with
fpc-3.3.1 (but with openssl vesion 3.0.5 installed - no problem with
any synapse code or curl or httpd or dillo ssl etc)
but i do have these libs installed along with the 3.0.5
-rw-r--r-- 1 root root 768714 May 17 2018 07:05:51 /lib/libssl.a
lrwxrwxrwx 1 root root 15 Sep 07 2022 09:41:03 /lib/libssl.so ->
libssl.so.1.0.0
-r-xr-xr-x 1 root root 517144 May 17 2018 07:05:51
/lib/libssl.so.1.0.0
-rw-r--r-- 1 root root 4556376 May 17 2018 07:05:51 /lib/libcrypto.a
lrwxrwxrwx 1 root root 18 Sep 07 2022 09:41:03 /lib/libcrypto.so
-> libcrypto.so.1.0.0
-r-xr-xr-x 1 root root 2713248 May 17 2018 07:05:51
/lib/libcrypto.so.1.0.0
-
am i correct that synapse doesn't even use openssl.pas unit itself but
only uses ssl_openssl.ppu and sssl_openssl_lib.ppu
so why isn't fpc using them instead of openssl.pas?
---
rvk you actually read what thaddy posts? that is funny - he never
responds to his many bogus posts when challenged
---
maurobio
you don't need to mess with copying dlls to windows system directory -
having them in the current directory you are working in is the best
way
---
----------------------------------------------------------------------
delphius · 2023-09-16 18:36
Quote from: toby on September 16, 2023, 05:33:40 pmdelphius
you are correct
i have upgraded to 3.3.1
writeln(InitSSLInterface(filepath + 'libssl' + ext, filepath +
'libcrypto' + ext));
writeln(InitSSLInterface());
both return true in 3.3.1 but gives the error on 3.2.2
OK, but we have to remember what rvk explained above, that we can't
just bring our own .so into a linux system by simply using them from a
program folder, as they are themselves! depend on the specific!
versions of other libraries, which usually does not happen in windows
(except for dependencies on various runtimes).
Therefore, a valid way in windows would be to force the dll use from
the program folder, and in the case of linux, you need to entrust
openssl.pas with the help of calling the initialization of the OpenSSL
library without parameters, independently find and load the version of
the .so libraries installed in the system into the address space of
our program (or, accordingly, solve the dependencies automatically by
the package installer when installing the application distribution)
There is always a second option to build all dependencies statically
in the binary file
And the third option is to build .so files from sources on the system,
then all dependencies will be taken into account and the collected
dynamic libraries will work from the program folder as well.
Quote from: toby on September 16, 2023, 05:33:40 pmmore important the
language we share in common is fpc
Absolutely true!))))
----------------------------------------------------------------------
maurobio · 2023-09-16 18:58
Hi, @toby!
Quoteyou don't need to mess with copying dlls to windows system
directory - having them in the current directory you are working in
is the best way.
Sure! I also prefer this way. Under Windows, my application now runs
fine with the required DLLs present in the binary executable folder.
What I would like it to be able to do the same thing with the
respective Linux shared libraries (.so)....
Best regards,
----------------------------------------------------------------------
TRon · 2023-09-16 19:12
Quote from: maurobio on September 16, 2023, 08:58:57 pmWhat I would
like it to be able to do the same thing with the respective Linux
shared libraries (.so)....
Already wrote about that. It will not be accepted in the Linux world.
fwiw it works in exactly the same way under Linux as it does on
Windows.
----------------------------------------------------------------------
maurobio · 2023-09-16 19:57
Hi, @TRon!
Quote fwiw it works in exactly the same way under Linux as it does on
Windows.
Now it works, after I have installed (on a machine without the Lazarus
IDE) the libssl-dev as you wizards have suggested.
With best regards,
----------------------------------------------------------------------
maurobio · 2023-09-16 20:07
Hi, Gentlmen!
Finally, I have some good news!
I gad already managed to get the eSpecies application running under
Windows (on a machine without the Lazarus IDE installed), but just
including the required OpenSSL DLL files in the same folder of the
application.
Now, I got the Linux versions (32- and 64-bits) also working (again,
on machines without the Lazarus IDE installed), by installing the
libssl--dev package as suggested by all.
In the case of Linux, I added three dependencies to my .deb
distribution package (besides other already required by Lazarus
applications):
libssl (>=1.1.1), libssl-dev (>=1.1.1), openssl (>=1.1.1)
I hope this will work on most distros.
It looks that the problem is solved and this topic can now be closed.
Thank you very much!
With warmest regards,
----------------------------------------------------------------------
delphius · 2023-09-17 06:41
Quote from: delphius on September 16, 2023, 08:36:05 pmOK, but we have
to remember what rvk explained above, that we can't just bring our own
.so into a linux system by simply using them from a program folder, as
they are themselves! depend on the specific! versions of other
libraries, which usually does not happen in windows (except for
dependencies on various runtimes).
I looked at the dependencies of both dynamic libraries, except for
mutual dependence and dependence on libc (on which the application
binary itself depends too), there are no more dependencies.
Then the question is, what could be the problems with distribution?
----------------------------------------------------------------------
delphius · 2023-09-17 06:50
Quote from: maurobio on September 16, 2023, 10:07:02 pmIt looks that
the problem is solved and this topic can now be closed.
Great news! It's nice to hear that everything worked out!
----------------------------------------------------------------------
rvk · 2023-09-17 07:04
Quote from: delphius on September 17, 2023, 08:41:25 amThen the
question is, what could be the problems with distribution?
Could you install libc6 if there is also a newer or older (g)libc
installed?
The symlink libc.so.6 to glibc would seem to always be there, so you
might get away with this for now.
Ok. How would you add you directory to the search path?
You would need to add it to LD_LIBRARY_PATH to load the library.
Make sure not to do that in .bashrc or somewhere else global because
then every program is going to load your older openssl while some of
those might need OpenSSL 3.
It might be better and safer to use the .a option and link the openssl
version statically into your program.
----------------------------------------------------------------------
delphius · 2023-09-17 07:29
Quote from: rvk on September 17, 2023, 09:04:30 amOk. How would you
add you directory to the search path?
You would need to add it to LD_LIBRARY_PATH to load the library.
I don't need to add it to LD_LIBRARY_PATH because I forcibly loading
them from program folder with
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
InitSSLInterface(filepath + 'libssl' + ext, filepath + 'libcrypto' +
ext);
Quote from: rvk on September 17, 2023, 09:04:30 amMake sure not to do
that in .bashrc or somewhere else global because then every program is
going to load your older openssl while some of those might need
OpenSSL 3.
No global, nor touching anything outside program folder
Quote from: rvk on September 17, 2023, 09:04:30 amIt might be better
and safer to use the .a option and link the openssl version statically
into your program.
I have already said this above, that it would be ideal to encapsulate
all dependencies inside the binary by static linking.
This would simultaneously reduce both the volume of distribution
(since static linking would take only the necessary parts, and not the
entire volume of the library) so is the number of distribution units,
reducing them to one (in the simplest case). And we have not yet
touched on reducing the size of the binary by packers.
P.S. further research led to the finding of a regular way to include
the path of the initial search for shared libraries inside executable
file, by passing to the linker -rpath or -R
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i -rpath
{path to lib(s) folder}
for example for folder `lib` in current folder
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i -rpath lib
or for folder `lib` in program executable binary's folder (relative,
in most cases we must use this method)
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i -rpath
$ORIGIN/lib
or for absolute path to folder `lib' (remember, when putting your
shared libs to places where other programs can search of such libs, it
can affect problems, we discussed above)
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i -rpath
/usr/lib
In order to pass this to the linker, it is necessary to set the
appropriate parameter in the project properties in Lazarus (see
attachment)
In fpc, accordingly, you just need to pass the -k key with these
parameters to the compiler
At the output, we get in the executable file with readelf -d
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i (RUNPATH)
Library runpath: [lib]or
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i (RUNPATH)
Library runpath: [$ORIGIN/lib]
Thus, in my opinion, this is an acceptable way to distribute the
program together with shared libraries, without affecting the system
environment and without affecting other programs.
So if we want to distribute .so files in program folder, then we must
set rpath just to $ORIGIN
In case that for some reason the libraries in the specified folder are
not found, then the search for shared libraries will follow the usual
route.
As for the versions of OpenSSL itself, as it was said, support for the
1.x branch ended on September 11, 2023, the 3.0.x branch is
experiencing problems with speed, although it has a longer lifetime.
Therefore, in my opinion, a reasonable choice would be to use the
3.1.x branch (currently the latest version 3.1.2). The most optimal
would be to compile libssl.so.3 and libcrypto.so.3 from the library
source codes yourself and rely on them when developing and testing
your program.
I am waiting for competent comments and opinions.
----------------------------------------------------------------------
paweld · 2023-09-17 14:00
I posted new version on github: https://github.com/paweld/especies
changes:
- an entirely new Bio unit with threads (downloading infos and images
in few threads), error handling, load from/save to json
- adding error handling to httpclient
- adding openssl3 unit to httpclient - libssl-dev installation is not
required
- few minor changes
With support for threaded downloads, download times have been reduced
by about 70%.
----------------------------------------------------------------------
maurobio · 2023-09-17 14:05
Hi, @paweld!
Thank you very much for the updated code!
I will study it in detail, to learn how things are done (in
particular, I am not familiar with the use of threads).
With warmest regards,
|