fphttpclient - ESocketError, time out on localhost ElasticSearch
lazarus / 38-Networking and Web Programming

----------------------------------------------------------------------
LeadGuit · 2021-03-30 19:19

Hi,

As I'm diving in, I thought I could write a small unit for doing some
basic stuff with Elasticsearch.

The Elasticsearch server run on localhost, default port 9200. Curl
from win and Linux etc work flawlessly, but fphttpclient seems to
struggle a little with the mentioned ESocketError. Other ports work
properly. Minimal Code is here:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i unit
Elasticsearch; {$mode objfpc}{$H+} interface procedure
testget; implementation uses  Classes, SysUtils, opensslsockets,
fphttpclient; procedure testget;  var    S: Rawbytestring;  begin   
with TFPHTTPClient.Create(nil) do        s :=
SimpleGet('http://localhost:9200');    WriteLn(S);    ReadLn; 
end; end. 
If I e.g. spin up a python httpserver, even on the same port (after
stopping Elasticsearch) works nicely. I have no idea why that
ESocketError is raised - for all I know, if I curl from the command
line with no arguments added, that should be almost identical to the
SimpleGet (or Get for that matter) call?

Best,
Mat

----------------------------------------------------------------------
marcov · 2021-03-30 19:57

Without any detailed investigation:

9 times out of 10, the url has a redirection, and you haven't
registered an ondirect(ion) handler.

----------------------------------------------------------------------
LeadGuit · 2021-03-30 20:18

Thanks,

I set AllowRedirect := True, but that didn't work (same Error as
before).

It's hard to find an explanation for the handler you mentioned - the
wiki only mentioned the AllowRedirect for that - if you could help me
out there?

I'm not sure what fphttpclient does under the hood - both linux and
Windows curl and browsers display the default JSON right away. If I
can trust the DevTab in Chrome, it's a straight call. from wget
Code: Bash  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i Resolving
localhost (localhost)... 127.0.0.1Connecting to localhost
(localhost)|127.0.0.1|:9200... connected.HTTP request sent, awaiting
response... 200 OKLength: 536 [application/json]....

----------------------------------------------------------------------
Gustavo 'Gus' Carreno · 2021-03-30 20:46

Hey LeadGuit,

Could you please provide us with your FPC version please?

There's been a mounting number of posts on the forum due to issues
that arose from the update from FPC 3.0.4 to 3.2.0, and SSL, in terms
of fphttpclient.
Some, if not most, are problems with SSL. Which is not completely the
case here.
I've solved most of the version and SSL issues on the example project
attached.

Have a test drive and see if the errors is still present. If so, then
you have a problem particular to your installation that needs to be
drilled down.

Cheers,
Gus

----------------------------------------------------------------------
trev · 2021-03-31 00:24

Quote from: marcov on March 30, 2021, 09:57:32 pmWithout any detailed
investigation:

9 times out of 10, the url has a redirection, and you haven't
registered an ondirect(ion) handler.

The other 1 out of 10 the issue is the user agent being used. Does
ElasticSearch check user agents?

----------------------------------------------------------------------
LeadGuit · 2021-03-31 04:27

Quote from: Gustavo 'Gus' Carreno on March 30, 2021, 10:46:23 pmHey
LeadGuit,

Could you please provide us with your FPC version please?

Stable, 3.2.0

Quote from: Gustavo 'Gus' Carreno on March 30, 2021, 10:46:23
pmThere's been a mounting number of posts on the forum due to issues
that arose from the update from FPC 3.0.4 to 3.2.0, and SSL, in terms
of fphttpclient.
Some, if not most, are problems with SSL. Which is not completely the
case here.
I've solved most of the version and SSL issues on the example project
attached.

Have a test drive and see if the errors is still present. If so, then
you have a problem particular to your installation that needs to be
drilled down.

Cheers,
Gus

The sample project works flawlessly. As said, other things work well
too, it only concerns Elasticsearch.
I tested it with my "production" instance, which lies on another
server behind a VPN, and that works as well. Both my local one and the
production operate without SSL.

So my  guess is that there is a special constellation that troubles
FPC. To elaborate - essentially what I'm doing is running a WSL2
(Windows Subsystem for Linux) on my Win 10. I use the Linux VM for all
my dayjob development, so e.g. Elasticsearch is also installed on
there. The development in Lazarus takes places on Windows (WSL is only
CLI, or GUI via an installed XServer on windows). Could it be that
fphttpclient goes "another way" contrary to curl et al? The ES
installation is relatively Vanilla. Other thing served from the Linux
part are "gettable", on various ports. Just ES is a little reluctant,
and only with FP.
E.g. my usual Python scripts on the linux machine work as expected,
and a Chrome-Plugin to voew ES Data (on Windows) also connects without
issue, curl on the linux subsystem as well as on windows are able to
fire the get request.


Quote from: trev on March 31, 2021, 02:24:44 am
The other 1 out of 10 the issue is the user agent being used. Does
ElasticSearch check user agents?


Not that I know, no - ES mainly offers an API for programmatic
consume. The test with my production instance shows that it should
work out of the box.

----------------------------------------------------------------------
engkin · 2021-03-31 06:45

FYI, your usage in the first post is wrong. SimpleGet is a "class
method" it does not need an instance, nor does it take any changes you
make to the instance you create. Use Get instead.

My guess your problem might be an antivirus or firewall.

----------------------------------------------------------------------
Gustavo 'Gus' Carreno · 2021-03-31 11:22

Hey LeadGuit,

Quote from: LeadGuit on March 31, 2021, 06:27:56 amThe sample project
works flawlessly. As said, other things work well too, it only
concerns Elasticsearch.

Just one more spot of curiosity: Did you point my sample project at
your ES instance?
I'm sorry to ask this, but it's not clear from your answer if you did?
Either that or I'm very thick

Cheers,
Gus

----------------------------------------------------------------------
LeadGuit · 2021-03-31 15:04

Quote from: Gustavo 'Gus' Carreno on March 31, 2021, 01:22:03 pmHey
LeadGuit,

Quote from: LeadGuit on March 31, 2021, 06:27:56 amThe sample project
works flawlessly. As said, other things work well too, it only
concerns Elasticsearch.

Just one more spot of curiosity: Did you point my sample project at
your ES instance?
I'm sorry to ask this, but it's not clear from your answer if you did?
Either that or I'm very thick

Cheers,
Gus

Oh, whoops - I'm sorry. No, I didn't - I just ran it - I thought it
would be just to check if connections work. I changed one line to the
ES address and the same error occurs:
"Project DownloadProgressBar raised Exception class 'ESocketError'
with message:
Connection to localhost:9200 timed out.

At address 100180718"
On Break, it shows me line 165 of the Sequential unit
(http.HTTPMethod('HEAD......)

----------------------------------------------------------------------
Gustavo 'Gus' Carreno · 2021-03-31 15:39

Hey LeadGuit,

Quote from: LeadGuit on March 31, 2021, 05:04:55 pmOh, whoops - I'm
sorry. No, I didn't - I just ran it - I thought it would be just to
check if connections work. I changed one line to the ES address and
the same error occurs:
"Project DownloadProgressBar raised Exception class 'ESocketError'
with message:
Connection to localhost:9200 timed out.

At address 100180718"
On Break, it shows me line 165 of the Sequential unit
(http.HTTPMethod('HEAD......)

Yeap, that kinda makes it officially rather fishy!!

There is absolutely something rather strange with that connection.

Tell me LeadGuit, do you have experience with netcat(nc) and/or the
verbose side of cURL?

Is it that the server is serving data in chunked instead of normal and
that fphttpclient doesn't grok that? This just occurred to me...
I need to investigate if fphttpclient can manage chunked or even
HTTP/2.0

Can you investigate what version of HTTP(1.0 or 1.1 or 2.0, plain,
chunked or gzip'ed) is the ES server using and report back, please?
Thanks!!

Cheers,
Gus

----------------------------------------------------------------------
LeadGuit · 2021-03-31 16:31

Yeah - the funny thing is, that the ES version on another server
works. Potentially a "breaking" change between versions could be a
culprit too:
Server that works: ES 6.8.11
Localhost: ES 7.10.2

Curl-wise, the headers are pretty identical:
Code: Bash  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i *   Trying
127.0.0.1:9200...* TCP_NODELAY set* Connected to localhost (127.0.0.1)
port 9200 (#0)> GET / HTTP/1.1> Host: localhost:9200> User-Agent:
curl/7.68.0> Accept: */*>* Mark bundle as not supporting multiuse<
HTTP/1.1 200 OK< content-type: application/json; charset=UTF-8<
content-length: 536<{.....}* Connection #0 to host localhost left
intact Only differences are the various versions in use (ES itself,
lucene etc)

Although I am suspicious of the Windows Hyper-V, where the WSL runs
in. Interestingly, a verbose curl from the windows CLI connects to the
IPv6:

Code: Bash  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i * Rebuilt
URL to: http://localhost:9200/*   Trying ::1...* TCP_NODELAY set*
Connected to localhost (::1) port 9200 (#0)> GET / HTTP/1.1> Host:
localhost:9200> User-Agent: curl/7.55.1> Accept: */*>< HTTP/1.1 200
OK< content-type: application/json; charset=UTF-8< content-length: 536

----------------------------------------------------------------------
LeadGuit · 2021-03-31 17:09

The problem isn't solved per se now - but since ES also isn't
supported on WSL, I wasn't able to quickly uninstall it (I'd have to
hunt down the directories and remove manually etc).

I installed the 6.8 Version via windows msi, adjusted the Listening
address and at least I can access it now, even though, from my dayjob
environment on WSL not as localhost. Minor inconvenience.

So, at least you can ease, since it generally works, and WSL (likely
due to HyperV Networking or the like) is a kind-of edge case from "dev
setup" perspective I think.

----------------------------------------------------------------------
Gustavo 'Gus' Carreno · 2021-03-31 18:28

Hey LeadGuit,

That is a serious edge case that you stumbled on.
Now the million dollar question is: What side is it from? fphttpclient
or ES?

Are you aware of the synapse network library?
It's present on the OPM(No visual components, just put on uses) and
you could do some more debugging by it's http client.
And if you feel you have some time to spare, there's also Indy(has
visual components for most protocols), another package under the OPM.

These edge cases really intrigue me. And if you stumbled on a
reportable bug of the fphttpclient, we maybe need to report it
And if it's not so edge, we might see more people stumbling on it and
we need to be aware in order to help or direct somewhere.

Cheers,
Gus

----------------------------------------------------------------------
LeadGuit · 2021-03-31 18:49

Quote from: Gustavo 'Gus' Carreno on March 31, 2021, 08:28:02 pmHey
LeadGuit,

That is a serious edge case that you stumbled on.
Now the million dollar question is: What side is it from? fphttpclient
or ES?

Are you aware of the synapse network library?
It's present on the OPM(No visual components, just put on uses) and
you could do some more debugging by it's http client.
And if you feel you have some time to spare, there's also Indy(has
visual components for most protocols), another package under the OPM.

These edge cases really intrigue me. And if you stumbled on a
reportable bug of the fphttpclient, we maybe need to report it
And if it's not so edge, we might see more people stumbling on it and
we need to be aware in order to help or direct somewhere.

Cheers,
Gus

Definitely intriguing!
I'm extremely new to Pascal and I'm still trying to piece together
where to find info on what, as that is a major difference between more
"popular" contemporary languages ;-)

I know of synapse and indy (meaning - I know that exists and I likely
installed it. I will have a look how they behave with the WSL-
Localhost.

----------------------------------------------------------------------
Gustavo 'Gus' Carreno · 2021-03-31 18:55

Hey LeadGuit,

Quote from: LeadGuit on March 31, 2021, 08:49:31 pmDefinitely
intriguing!

You can say that again...

Quote from: LeadGuit on March 31, 2021, 08:49:31 pmI'm extremely new
to Pascal and I'm still trying to piece together where to find info on
what, as that is a major difference between more "popular"
contemporary languages ;-)

I have this sense that you're a really fast learner!!
And I think you have the strength to persevere!

Quote from: LeadGuit on March 31, 2021, 08:49:31 pmI know of synapse
and indy (meaning - I know that exists and I likely installed it. I
will have a look how they behave with the WSL-Localhost.

I would be eternally grateful if you do. Many, many thanks!!

Cheers,
Gus

----------------------------------------------------------------------
LeadGuit · 2021-03-31 19:15

Thank you for the encouragement ;-) Learning the Syntax and Libraries
is always a little journey. especially coming from Python with maybe a
little too much convenience to to "you can find a package for
everything". Hiding the whole logic from you. On the plus side, Pascal
let's me feel more like a Programmer, while the stuff I do in Py let's
me feel more like a SysAdmin or Data Scientist xD

So I tried out Synapse in a very quick and hacky way

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i procedure
Tform1.Button1click(Sender: TObject);  var    httpsender: Thttpsend; 
 Httpgetresult: boolean;    ss: TStringStream;  begin    ss :=
TSTringStream.Create('');    httpsender := THTTPSend.Create;   
HTTPGetResult := HTTPSender.HTTPMethod('GET',
'http://localhost:9200');    ss.CopyFrom(httpsender.document, 0);   
Memo1.Lines := (httpsender.Headers) ;   
Memo1.Lines.Add(IntToStr(httpsender.ResultCode));    
Memo1.Lines.Add(ss.DataString);  end; 
And yeah - it again can't connect to the instance running in WSL, but
can easy get on the one running on Windows.
Interesting is the ResultCode 500 I get back:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i GET /
HTTP/1.0Host: 127.0.0.1:9200Keep-Alive: 300Connection: keep-aliveUser-
Agent: Mozilla/4.0 (compatible; Synapse) 500
vs on the working instance:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i HTTP/1.0
200 OKconnection: Keep-Alivecontent-type: application/json;
charset=UTF-8content-length: 501 200{.....

----------------------------------------------------------------------
LeadGuit · 2021-03-31 19:39

Ok, a bit further:

Setting ES to "production mode" - or rather development mode with a
small - totally insecure and not recommended - hack, I can access it
through the "Network IP" the WSL2 machine has assigned. Meaning, I
expose the ES instance to the whole network, instead of the normal
"development mode" which would only expose it to localhost (aka
0.0.0.0 in production vs localhost in development)

WSL2 should forward to localhost on Windows, which in most instances
work. For Pascal though, in fphttpclient and synapse, The forward
isn't recognized as it seems, instead, the WSL instance is seen as a
"separate computer". Which is technically not wrong since it's a
lightweight VM, but still is interesting.
Maybe Windows is doing some behind-the-scenes work with HyperV to get
that localhost-forward going, which Pascal doesn't have access to (or
rather, doesn't use without explicitly stating so)?
I've tested quickly with Python on Windows using the requests library,
and as expected the localhost-forward works as expected.

----------------------------------------------------------------------
Gustavo 'Gus' Carreno · 2021-03-31 19:48

Hey LeadGuit,

Quote from: LeadGuit on March 31, 2021, 09:15:11 pmThank you for the
encouragement ;-) Learning the Syntax and Libraries is always a little
journey. especially coming from Python with maybe a little too much
convenience to to "you can find a package for everything". Hiding the
whole logic from you. On the plus side, Pascal let's me feel more like
a Programmer, while the stuff I do in Py let's me feel more like a
SysAdmin or Data Scientist xD

Oh, I so feel you!!

Quote from: LeadGuit on March 31, 2021, 09:15:11 pmSo I tried out
Synapse in a very quick and hacky way

Apart from a try...finally block to make sure ss and httpsender get
freed, that's not hacky at all. Actually quite on the nose.

Quote from: LeadGuit on March 31, 2021, 09:15:11 pmAnd yeah - it again
can't connect to the instance running in WSL, but can easy get on the
one running on Windows.

So that confirms that there is something in common with fphhtpclient
and synapse that doesn't work in FPC land but does in cURL land. How
utterly intriguing!!
But hey, synapse is quite old. Wanna try with Indy? One click away on
the OPM... Okay, maybe 2...

Quote from: LeadGuit on March 31, 2021, 09:15:11 pmInteresting is the
ResultCode 500 I get back:

That's very intriguing.
I suspect that synapse is probably eating an Exception and then just
spitting out a 500.
I think if you debug and enter the code(F7 enter code, F8 go over) and
go into the synapse code you'll find out the ESocketError. But not
100% sure. Talking outta me arse, here...

Cheers,
Gus

----------------------------------------------------------------------
Gustavo 'Gus' Carreno · 2021-03-31 19:58

Hey LeadGuit,

Quote from: LeadGuit on March 31, 2021, 09:39:01 pmOk, a bit further:

Setting ES to "production mode" - or rather development mode with a
small - totally insecure and not recommended - hack, I can access it
through the "Network IP" the WSL2 machine has assigned. Meaning, I
expose the ES instance to the whole network, instead of the normal
"development mode" which would only expose it to localhost (aka
0.0.0.0 in production vs localhost in development)

WSL2 should forward to localhost on Windows, which in most instances
work. For Pascal though, in fphttpclient and synapse, The forward
isn't recognized as it seems, instead, the WSL instance is seen as a
"separate computer". Which is technically not wrong since it's a
lightweight VM, but still is interesting.
Maybe Windows is doing some behind-the-scenes work with HyperV to get
that localhost-forward going, which Pascal doesn't have access to (or
rather, doesn't use without explicitly stating so)?
I've tested quickly with Python on Windows using the requests library,
and as expected the localhost-forward works as expected.

Hummm, so what this is kinda telling me is that python and cURL are
initiating sockets in a manner that enable the WSL redirect(let's call
it that), to work.
On the other hand, FPC is initiating sockets in such a way that it
pretty much effes with the WSL redirection.

Now is it a binding issue? Does FPC bind and cURL not, or vice-versa?
It's not DNS related because the error is ESocketError Timeout, not
DNS resolution error.
Is it the diff between 0.0.0.0 and 127.0.0.1?

AHHHHH, you're killing me!! So many questions!!

Cheers,
Gus

----------------------------------------------------------------------
LeadGuit · 2021-03-31 20:42

Quote from: Gustavo 'Gus' Carreno on March 31, 2021, 09:58:00 pmHey
LeadGuit,

Quote from: LeadGuit on March 31, 2021, 09:39:01 pmOk, a bit further:

Setting ES to "production mode" - or rather development mode with a
small - totally insecure and not recommended - hack, I can access it
through the "Network IP" the WSL2 machine has assigned. Meaning, I
expose the ES instance to the whole network, instead of the normal
"development mode" which would only expose it to localhost (aka
0.0.0.0 in production vs localhost in development)

WSL2 should forward to localhost on Windows, which in most instances
work. For Pascal though, in fphttpclient and synapse, The forward
isn't recognized as it seems, instead, the WSL instance is seen as a
"separate computer". Which is technically not wrong since it's a
lightweight VM, but still is interesting.
Maybe Windows is doing some behind-the-scenes work with HyperV to get
that localhost-forward going, which Pascal doesn't have access to (or
rather, doesn't use without explicitly stating so)?
I've tested quickly with Python on Windows using the requests library,
and as expected the localhost-forward works as expected.

Hummm, so what this is kinda telling me is that python and cURL are
initiating sockets in a manner that enable the WSL redirect(let's call
it that), to work.
On the other hand, FPC is initiating sockets in such a way that it
pretty much effes with the WSL redirection.

Now is it a binding issue? Does FPC bind and cURL not, or vice-versa?
It's not DNS related because the error is ESocketError Timeout, not
DNS resolution error.
Is it the diff between 0.0.0.0 and 127.0.0.1?

AHHHHH, you're killing me!! So many questions!!

Cheers,
Gus

Haha - so my current hypothesis is that really FPC binds differently -
but only on ES, since a simple HTTP Server on WSL2 works...or a
uvicorn server as well
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i HTTP/1.1
200 OKdate: Wed, 31 Mar 2021 20:29:33 GMTserver: uvicorncontent-type:
text/html; charset=utf-8Connection: close 200It's only the local ES
that goes haywire. Which would mean in the end - ES is screwing with
the sockets in some way, which only would affect FPC/Synapse.

I tried to do something with indy, but I couldn't find how - I can't
see a http component, and every question seems to skip the "use" part
and whatnot. And it's getting too late over here to read question
thread through and make code work ;-)

----------------------------------------------------------------------
Gustavo 'Gus' Carreno · 2021-03-31 21:04

Hey LeadGuit,

Quote from: LeadGuit on March 31, 2021, 10:42:40 pmHaha - so my
current hypothesis is that really FPC binds differently - but only on
ES, since a simple HTTP Server on WSL2 works...or a uvicorn server as
well
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i HTTP/1.1
200 OKdate: Wed, 31 Mar 2021 20:29:33 GMTserver: uvicorncontent-type:
text/html; charset=utf-8Connection: close 200
I'm glad you cleared this issue. I was gonna mention you did this
exact test(great minds and all that), but I was also pushing for the
Indy test, so I didn't wan to seem bully-ish


Quote from: LeadGuit on March 31, 2021, 10:42:40 pmIt's only the local
ES that goes haywire. Which would mean in the end - ES is screwing
with the sockets in some way, which only would affect FPC/Synapse.

Indeed. But what still baffles me is exactly that: Why is only FPC
tripping on that?!?!?

Quote from: LeadGuit on March 31, 2021, 10:42:40 pmI tried to do
something with indy, but I couldn't find how - I can't see a http
component, and every question seems to skip the "use" part and
whatnot. And it's getting too late over here to read question thread
through and make code work ;-)

Yeah, with fphttpclient and synapse there are no visual components.
Indy needs a bit of a paradigm shift since it's ALL visual components.
This means looking for the Indiy Clients component tab and finding the
HTTP one, dropping on the... Why am I patronising you? You know this
by now: Find lill picture on tabs, click on it, click or drag on form,
look at object inspector. Or at least you shouldda have mastered it
already

And, yes, please go and have a rest. I also need one. I'm finding
myself staring into the void and drooling just a bit...

Cheers,
Gus

----------------------------------------------------------------------
sstvmaster · 2021-03-31 21:57

I have installed elasticsearch from:
https://www.elastic.co/de/downloads/elasticsearch
I use Windows 10 and Lazarus 2.0.12

With this code i got response:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i unit
Unit1; {$mode objfpc}{$H+}  interface uses  Classes, SysUtils, Forms,
Controls, Graphics, Dialogs, StdCtrls,   fpHttpClient; const 
HttpErrorMsg = 'Error %d (%s)'; type   { TForm1 }   TForm1 =
class(TForm)    Button1: TButton;    Memo1: TMemo;    procedure
Button1Click(Sender: TObject);  private   public   end; var  Form1:
TForm1; implementation {$R *.lfm} { TForm1 } procedure
TForm1.Button1Click(Sender: TObject);var  http: TFPHTTPClient; 
Response: TMemoryStream;begin  http := TFPHTTPClient.Create(nil); 
Response := TMemoryStream.Create;  try    try     
http.Get('http://127.0.0.1:9200', Response);      Response.Seek(0,
soFromBeginning);      Memo1.Lines.LoadFromStream(Response);   
except      Memo1.Clear;      Memo1.Lines.Add(Format(HttpErrorMsg,
[http.ResponseStatusCode,http.ResponseStatusText]));    end;  finally 
  http.Free;    Response.Free;  end;end; end.

----------------------------------------------------------------------
PierceNg · 2021-04-01 01:39

Quote from: LeadGuit on March 31, 2021, 10:42:40 pmHaha - so my
current hypothesis is that really FPC binds differently - but only on
ES, since a simple HTTP Server on WSL2 works...or a uvicorn server as
well
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i HTTP/1.1
200 OKdate: Wed, 31 Mar 2021 20:29:33 GMTserver: uvicorncontent-type:
text/html; charset=utf-8Connection: close 200It's only the local ES
that goes haywire. Which would mean in the end - ES is screwing with
the sockets in some way, which only would affect FPC/Synapse.

Does this Python server listen on localhost only in WSL though? Or
does it listen on 0.0.0.0?

----------------------------------------------------------------------
LeadGuit · 2021-04-01 05:03

Quote from: sstvmaster on March 31, 2021, 11:57:55 pmI have installed
elasticsearch from: https://www.elastic.co/de/downloads/elasticsearch
I use Windows 10 and Lazarus 2.0.12

With this code i got response:
...

Which Linux are you running on your WSL?
The problem is connecting to an ES instance running on WSL, not
Windows (in case you installed ES on Windows instead). Your code
results in the same ESocketError for me.

Quote from: PierceNg on April 01, 2021, 03:39:17 amDoes this Python
server listen on localhost only in WSL though? Or does it listen on
0.0.0.0?

The tested uvicorn listens on 127.0.0.1:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i INFO:    
Uvicorn running on http://127.0.0.1:8001 (Press CTRL+C to quit)

----------------------------------------------------------------------
sstvmaster · 2021-04-01 06:21

Quote from: LeadGuit on April 01, 2021, 07:03:46 am... ES instance
running on WSL...
I'm on Windows only. And what is an "ES instance" and "WSL" ?

rgr

----------------------------------------------------------------------
PascalDragon · 2021-04-01 06:57

Quote from: sstvmaster on April 01, 2021, 08:21:41 amQuote from:
LeadGuit on April 01, 2021, 07:03:46 am... ES instance running on
WSL...
I'm on Windows only. And what is an "ES instance" and "WSL" ?

ES instance = the Elastic Search server
WSL = Windows Services for Linux (aka "Linux on Windows")

----------------------------------------------------------------------
LeadGuit · 2021-04-01 07:06

Quote from: sstvmaster on April 01, 2021, 08:21:41 amQuote from:
LeadGuit on April 01, 2021, 07:03:46 am... ES instance running on
WSL...
I'm on Windows only. And what is an "ES instance" and "WSL" ?

rgr

Ah ok.
ES Instance: Simply an instance of Elasticsearch - meaning "one
running version of it somewhere"
WSL: Windows Subsystem for Linux - Microsoft integrated a linux kernel
into Windows some time ago, running as very lightweight virtual
machine under Windows. I do all my dayjob coding now in there, as the
stuff I need runs like 99% smoothly and I can properly develop on
Linux without having to dual-boot, a separate laptop or other
shenanigans. Additional benefit, due to the forwarding mentioned in
other posts here, my Browser on Windows can access the server running
on this Linux as localhost. So if I am developing a webapp, I use
VSCode's remote function to write the code, execute the command line
of linux to run it, and my windows browser to access it. The Linux
doesn't need any GUI facilities this way - best of both worlds imo.

For my problem in this thread this means - the architecture is like
this:
Windows -> Developing Lazarus apps for fun, to learn a new language
and quick GUI stuff
Linux inside Windows -> Dayjob coding, machine learning, SysAdmin
automation etc

Which in turn means -> ES is hosted inside the Linux VM, since I need
it for the dayjob - that enables convenience using the elasticsearch
package with Python in my linux dev setup, as the default is
"localhost", meaning when I write my data into the "production
server", I simply input it's IP and off we go, while I can test,
delete and adjust my logic on my local ES instance - just to be sure I
don't accidentally change/delete something important. At the same
time, I can easily use a chrome extension on windows to do simple ES
queries to check if stuff gets done right - by querying localhost as
well.
So what I tried to do for the original question was creating a unit
for elasticsearch access - similarly as I would create a python
module, to conveniently just call a couple functions to interact and
write/read data. The problem seems to arise in the way Windows handles
that port forwarding - as said, Chrome, Python, Curl etc all get
properly forwarded to the Linux VM underneath, the FPC app does not -
but only in the case of Elasticsearch. Other servers I run under my
linux can be accessed as intended.
Unfortunately ES isn't supported on WSL (meaning - it runs, I've been
working that way for over 2 years xD, but they don't give official
support for it). So if you ask th eES guys a question - the answer is
"it's not supported, sorry". The interesting part is merely, that it
just concern that forwarding constellation - as mentioned, connecting
to the ES Instance via the Linux VM's URL does work (with ES in an
"expanded development" mode).

I hope that clears it all up a bit :-)

----------------------------------------------------------------------
sstvmaster · 2021-04-01 07:26

Thank you for your explanation. So then i think if your elasticsearch
server running as localhost (127.0.0.1), then you only have access
from this machine (with curl it works).

So you must run the elasticsearch server with ip mode 0.0.0.0 so he
can accept requests from outside.
see: https://discuss.elastic.co/t/network-host-0-0-0-0-is-not-working-
in-elasticsearch-7-3-0/195697 and https://www.elastic.co/guide/en/elas
ticsearch/reference/current/modules-network.html

----------------------------------------------------------------------
LeadGuit · 2021-04-01 07:56

Quote from: sstvmaster on April 01, 2021, 09:26:25 amThank you for
your explanation. So then i think if your elasticsearch server running
as localhost (127.0.0.1), then you only have access from this machine
(with curl it works).

So you must run the elasticsearch server with ip mode 0.0.0.0 so he
can accept requests from outside.
see: https://discuss.elastic.co/t/network-host-0-0-0-0-is-not-working-
in-elasticsearch-7-3-0/195697 and https://www.elastic.co/guide/en/elas
ticsearch/reference/current/modules-network.html

Exactly, that's was the "kind of solution" that I came up with a few
posts above (the not recommended hack I talked about, since it exposes
the ES to the whole network). But the problem is only circumvented, as
I have to access it through the VMs IP instead of localhost. So to
speak - it "kind of solved", but the problem is still there and now
it's more of a bug hunt to find out why this problem exists :-)