Way to know if fphttpserver is working lazarus / 38-Networking and Web Programming ---------------------------------------------------------------------- fjabouley · 2020-04-15 15:28 Hi all !! I added a http server in my main program, it works well, serving everything I want without issues. But unfortunately, and randomly, it stops working, without any error. When I try to access it, I got in chrome "Err_empty_response", and it doesn't respond anymore. I don't know how to debug it, since it doesn't show any error... I tried to see if the thread was suspended, or if thread.server was active, but they seem to keep working... Is there a way to know what can happen or to see if the server has an issue, or stopped working, in order, as a workaround , to be able to restart it ? Best regards ---------------------------------------------------------------------- Leledumbo · 2020-04-16 16:56 Easiest way is to start it through something like Supervisor, you can even customize the behavior like maximum number of retries. It will also capture stderr in case the server crash and it dumps something there, like a stacktrace ---------------------------------------------------------------------- fjabouley · 2020-04-18 10:19 @leledumbo : Thanks for your answer. Unfortunately, the program is running under windows. I think I misunderstand how the server works. Actually it seems to be less buggy when setting queuesize with larger number, but I don't know why. As I said, it works very well, but at some random point it stops responding, without stressing the CPU nor displaying any error.. I wished there were (and perhaps there is) some kind of internal setting to see if the server was working. But even if server.active = true it doesn't work. Best regards ---------------------------------------------------------------------- Leledumbo · 2020-04-19 11:51 Quote from: fjabouley on April 18, 2020, 12:19:03 pmUnfortunately, the program is running under windows. It has a Windows port. ---------------------------------------------------------------------- fjabouley · 2020-04-25 15:46 @leledumbo : thanks for your answer. The http server is a part of my main program, do you think supervisor can help me debug this part ? Is there another way to catch errors, or debug fphttpserver ? like checking if the server is still listening for new connections or even running ? Thanks a lot for your answers. Regards ---------------------------------------------------------------------- zamronypj · 2020-04-26 05:39 I suggest wireshark. It allows you to inspect http protocol request response messages. ---------------------------------------------------------------------- lucamar · 2020-04-26 07:00 Quote from: zamronypj on April 26, 2020, 07:39:37 amI suggest wireshark. It allows you to inspect http protocol request response messages. But it doesn't tell you if the server is working. As the name suggests, it's a tool for looking what is passing "through the wires", not what's happening at the extremes. ---------------------------------------------------------------------- Thaddy · 2020-04-26 07:14 What fpc version are you using? There have been quite a few changes and fixes in FPC 3.2.0 and current trunk ( I believe almost all trunk changes for fcl-web are back-ported to 3.2.0 ) ---------------------------------------------------------------------- fjabouley · 2020-04-26 07:51 Thx all for your answers @thaddy : I'm still on 3.0.4, do you think upgrading could help? I'm quite lost, because all seems to be working but for some reason, and at some random point the server stops responding, without errors, and although the server says it is still active. ---------------------------------------------------------------------- fjabouley · 2020-04-26 12:13 May I submit the unit I made for http server ? There must be something in it that may cause the server to stop working, but cannot manage to find where... As I tried to make sessions by myself, etc... there must be many newbie errors in it. Thanks again to all ! ---------------------------------------------------------------------- zamronypj · 2020-04-26 12:17 Quote from: lucamar on April 26, 2020, 09:00:31 amQuote from: zamronypj on April 26, 2020, 07:39:37 amI suggest wireshark. It allows you to inspect http protocol request response messages. But it doesn't tell you if the server is working. As the name suggests, it's a tool for looking what is passing "through the wires", not what's happening at the extremes. It does tell you if server is working. If http server receives proper http request but not respond proper http response then server is not working. If somehow incorrect http request is received, if server is working it should response with proper http 400 response, otherwise it is not working. By looking at data that is passed through the wire you can get overview if the server is working or not. ---------------------------------------------------------------------- fjabouley · 2020-04-29 07:14 @all : thanks again for your answers. @leledumbo : do you think I should use the HTTPRouter.RegisterRoute() you mentioned in this thread https://forum.lazarus.freepascal.org/index.php/topic,49527.0.html instead of using @Onrequest in a http server (like in the example I posted ) ? Thanks again to everybody ! Best regards ---------------------------------------------------------------------- devEric69 · 2020-04-29 11:20 QuoteI suggest wireshark. It allows you to inspect http protocol request response messages. This can be a "real true false good" idea: I mean, it may be better to improve the resilience of the TfpHTTPClient or TfpHTTPCServer program. I say this, because having tried such solutions like those, I've ended up with a "skewed" TCP\IP stack (anyway, the MS-DOS commands to put the TCP\IP stack back, in the same state as the one delivered by Microsoft are: Code: Bash  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i >ipconfig /flushdns>netsh winsock reset catalog ). ---------------------------------------------------------------------- Leledumbo · 2020-04-30 17:58 Quote from: fjabouley on April 29, 2020, 09:14:04 am@leledumbo : do you think I should use the HTTPRouter.RegisterRoute() you mentioned in this thread https://forum.lazarus.freepascal.org/index.php/topic,49527.0.html instead of using @Onrequest in a http server (like in the example I posted ) ? It's your call. I personally would always use what's considered the present and future by the package developer. ---------------------------------------------------------------------- fjabouley · 2020-04-30 19:56 @leledumbo : thank you for your answer. Actually, before I saw your post in the thread I mentioned, and because I'm a noob, I didn't know anything about the HTTPRouter registerroute function. It seems to be really cool and quite easy to use.  Do you think I can replace my current http server with this, and instead of making a thread with fphttpserver, I just put HTTPRouter.RegisterRoute('*', @Onrequest); it seems magical and too easy ---------------------------------------------------------------------- Leledumbo · 2020-05-01 19:19 Quote from: fjabouley on April 30, 2020, 09:56:30 pm@leledumbo : thank you for your answer. Actually, before I saw your post in the thread I mentioned, and because I'm a noob, I didn't know anything about the HTTPRouter registerroute function. It seems to be really cool and quite easy to use.  Do you think I can replace my current http server with this, and instead of making a thread with fphttpserver, I just put HTTPRouter.RegisterRoute('*', @Onrequest); it seems magical and too easy I don't see why not. The httprouter unit was made for convenience anyway, it's comparable to any modern way to implement a http server application quickly and easily, such as in Go's standard http library (the difference is, Go version supports HTTP/2, but that's supposed to be a silent upgrade anyway, at least it should be). ---------------------------------------------------------------------- fjabouley · 2020-05-02 06:46 @leledumbo: thank you for your answers. Is it possible to thread it or is it already threaded? I use global variables for my users' sessions. Can I also do it? with critical sections? Last question, but does it use fphttpserver? If so, do you think I could have the same kind of problems as with my current server? I have to look at this, I haven't had time yet. Thanks again !!!! ---------------------------------------------------------------------- Leledumbo · 2020-05-02 19:42 Quote from: fjabouley on May 02, 2020, 08:46:22 am@leledumbo: thank you for your answers. Is it possible to thread it or is it already threaded? The routing mechanism isn't threaded nor it has a need for it. This is the HTTP server handler's job and the server already has that capability built-in. Quote from: fjabouley on May 02, 2020, 08:46:22 amI use global variables for my users' sessions. Can I also do it? with critical sections? Ugh, bad idea. Use proper session implementations instead like a(n in- memory) database like Redis or something. Delegate what you can delegate to a dedicated service to keep your own code slim. Quote from: fjabouley on May 02, 2020, 08:46:22 amLast question, but does it use fphttpserver? If so, do you think I could have the same kind of problems as with my current server? The router isn't tied to the server (but tied to the application object), it's just an alternative implementation of the previous router, which was already decoupled from the server. ---------------------------------------------------------------------- fjabouley · 2020-05-03 17:13 Thanks very much leledumbo dor all your answers!!! I'll try to use this. I had some time today, I tried to see if all the requests where handled correctly by my current server.... and then I noticed it was like I was DoSed (sometimes, I got many many many requests in very short period of time)! For example (requests I didn't make): Code: [Select]03/05/2020 18:53:49 [] : /manager/html? 03/05/2020 18:53:41 [] : /websql/index.php? 03/05/2020 18:53:37 [] : /SQL/index.php? 03/05/2020 18:53:33 [] : /sql/index.php? 03/05/2020 18:53:29 [] : /sqladmin/index.php? 03/05/2020 18:53:13 [] : /mysql/dbadmin/index.php? 03/05/2020 18:53:09 [] : /mysql/admin/index.php? 03/05/2020 18:52:57 [] : /shopdb/index.php? 03/05/2020 18:52:53 [] : /program/index.php? 03/05/2020 18:52:49 [] : /__phpMyAdmin/index.php? 03/05/2020 18:52:47 [] : /? 03/05/2020 18:52:45 [] : /phpMyAdmin_ai/index.php? 03/05/2020 18:52:41 [] : /phpMyAdmln/index.php? 03/05/2020 18:52:37 [] : /WWW/phpMyAdmin/index.php? 03/05/2020 18:52:33 [] : /sbb/index.php? 03/05/2020 18:52:29 [] : /phpMyAdminhf/index.php? 03/05/2020 18:52:25 [] : /phpMyAdminn/index.php? 03/05/2020 18:52:21 [] : /123131/index.php? 03/05/2020 18:51:52 [] : /? .... : /Joomla/> : /joomla/> : /> : /?a=fetch&content=die(@md5(HelloTh inkCMF))>a=fetch&content=die(@md5(HelloThinkCMF)) : / index.php?s=/module/action/param1/${@die(md5(HelloThinkPHP))}>s=/modul e/action/param1/${@die(md5(HelloThinkPHP))} : /index.php/module/action/param1/${@die(md5(HelloThinkPHP))}> : /App/?content=die(md5(HelloThinkPHP))>content=die(md5(HelloThinkPHP)) : /elrekt.php?s=%2f%69%6e%64%65%78%2f%5c%74%68%69%6e%6b%5c%61%70%70%2f %69%6e%76%6f%6b%65%66%75%6e%63%74%69%6f%6e&function=%63%61%6c%6c%5f%75 %73%65%72%5f%66%75%6e%63%5f%61%72%72%61%79&vars[0]=%6d%645&vars[1][]=% 48%65%6c%6c%6f%54%68%69%6e%6b%50%48%50>s=%2f%69%6e%64%65%78%2f%5c%74%6 8%69%6e%6b%5c%61%70%70%2f%69%6e%76%6f%6b%65%66%75%6e%63%74%69%6f%6e&fu nction=%63%61%6c%6c%5f%75%73%65%72%5f%66%75%6e%63%5f%61%72%72%61%79&va rs[0]=%6d%645&vars[1][]=%48%65%6c%6c%6f%54%68%69%6e%6b%50%48%50 : /ind ex.php?s=%2f%69%6e%64%65%78%2f%5c%74%68%69%6e%6b%5c%61%70%70%2f%69%6e% 76%6f%6b%65%66%75%6e%63%74%69%6f%6e&function=%63%61%6c%6c%5f%75%73%65% 72%5f%66%75%6e%63%5f%61%72%72%61%79&vars[0]=%6d%645&vars[1][]=%48%65%6 c%6c%6f%54%68%69%6e%6b%50%48%50>s=%2f%69%6e%64%65%78%2f%5c%74%68%69%6e %6b%5c%61%70%70%2f%69%6e%76%6f%6b%65%66%75%6e%63%74%69%6f%6e&function= %63%61%6c%6c%5f%75%73%65%72%5f%66%75%6e%63%5f%61%72%72%61%79&vars[0]=% 6d%645&vars[1][]=%48%65%6c%6c%6f%54%68%69%6e%6b%50%48%50 : /weathermap/editor.php> : /cacti/plugins/weathermap/editor.php> : /plugins/weathermap/editor.php> : /MyAdmin/scripts/db___.init.php> : /myadmin/scripts/db___.init.php> : /pma/scripts/db___.init.php> : /MyAdmin/scripts/setup.php> : /PMA/scripts/setup.php> : /pma/scripts/setup.php> : /phpMyAdmin/scripts/db___.init.php> : /scripts/db___.init.php> : /scripts/setup.php> : /shell.php> : /t6nv.php> : /wpo.php> : /wpc.php> : /lala-dpr.php> : /lala.php> : /z.php> : /desktop.ini.php> : /sane.php> : /b.php> : /htdocs.php> : /shell.php> : /pmd_online.php> : /hell.php> : /license.php> : /help-e.php> : /logon.php> : />: />: />: />: />: />: />: />: />: />: />: />: />: />: />: />:  : /PMA/scripts/setup.php> : /pma/scripts/setup.php> : /phpMyAdmin/scripts/db___.init.php> : /scripts/db___.init.php> : /scripts/setup.php> : /shell.php> : /t6nv.php> : /wpo.php> : /wpc.php> : /lala-dpr.php> : /lala.php> : /z.php> : /desktop.ini.php> : /sane.php> : /b.php> : /htdocs.php> : /shell.php> : /pmd_online.php> : /hell.php> : /license.php> : /help-e.php> : /logon.php> : />: />: />: />: />: />: />: />: />: />: />: />: />: />: /: />: />: />: />: />: />: />: />c: /logon.php> :   : /yuyang.php> : /xiaoxia.php> : /mjx.php> : /doudou.php> : /ftmabc.php> : /floaw.php> : /yu.php> : /up.php> : ip.ws.126.net:443> : /d7.php> : /?a=echo%20-n%20HelloNginx%7Cmd5sum>a=echo%20-n%20HelloNginx%7Cmd5sum : /Joomla/> : /joomla/> : /> : /?a=fetch&content=die(@md5(HelloTh inkCMF))>a=fetch&content=die(@md5(HelloThinkCMF)) : / index.php?s=/module/action/param1/${@die(md5(HelloThinkPHP))}>s=/modul e/action/param1/${@die(md5(HelloThinkPHP))} : /index.php/module/action/param1/${@die(md5(HelloThinkPHP))}> : /App/?content=die(md5(HelloThinkPHP))>content=die(md5(HelloThinkPHP)) : /elrekt.php?s=%2f%69%6e%64%65%78%2f%5c%74%68%69%6e%6b%5c%61%70%70%2f %69%6e%76%6f%6b%65%66%75%6e%63%74%69%6f%6e&function=%63%61%6c%6c%5f%75 %73%65%72%5f%66%75%6e%63%5f%61%72%72%61%79&vars[0]=%6d%645&vars[1][]=% 48%65%6c%6c%6f%54%68%69%6e%6b%50%48%50>s=%2f%69%6e%64%65%78%2f%5c%74%6 8%69%6e%6b%5c%61%70%70%2f%69%6e%76%6f%6b%65%66%75%6e%63%74%69%6f%6e&fu nction=%63%61%6c%6c%5f%75%73%65%72%5f%66%75%6e%63%5f%61%72%72%61%79&va rs[0]=%6d%645&vars[1][]=%48%65%6c%6c%6f%54%68%69%6e%6b%50%48%50 : /ind ex.php?s=%2f%69%6e%64%65%78%2f%5c%74%68%69%6e%6b%5c%61%70%70%2f%69%6e% 76%6f%6b%65%66%75%6e%63%74%69%6f%6e&function=%63%61%6c%6c%5f%75%73%65% 72%5f%66%75%6e%63%5f%61%72%72%61%79&vars[0]=%6d%645&vars[1][]=%48%65%6 c%6c%6f%54%68%69%6e%6b%50%48%50>s=%2f%69%6e%64%65%78%2f%5c%74%68%69%6e %6b%5c%61%70%70%2f%69%6e%76%6f%6b%65%66%75%6e%63%74%69%6f%6e&function= %63%61%6c%6c%5f%75%73%65%72%5f%66%75%6e%63%5f%61%72%72%61%79&vars[0]=% 6d%645&vars[1][]=%48%65%6c%6c%6f%54%68%69%6e%6b%50%48%50 : /weathermap/editor.php> : /cacti/plugins/weathermap/editor.php> : /plugins/weathermap/editor.php> : /MyAdmin/scripts/db___.init.php> : /myadmin/scripts/db___.init.php> : /pma/scripts/db___.init.php> : /MyAdmin/scripts/setup.php> : /PMA/scripts/setup.php> : /pma/scripts/setup.php> : /phpMyAdmin/scripts/db___.init.php> : /scripts/db___.init.php> : /scripts/setup.php> : /shell.php> : /t6nv.php> : /wpo.php> : My ISP doesn't provide me static public IP, so for now I used changeIP service, but there must be some bots that scan range of IPs and URL. I just deleted my account and I now use DNSTube with Cloudflare... I currently don't get annoyed by unwanted requests.. Do you think my server problem was linked to that (perhaps some requests made fphttpserver crash at some random point?) But for sure, I'll never use that dyndns provider again.... ---------------------------------------------------------------------- Leledumbo · 2020-05-03 19:06 Quote from: fjabouley on May 03, 2020, 07:13:36 pmFor example (requests I didn't make): Yep, that's a common basic attack. Every starting attackers think every website has access to phpmyadmin on the same domain, just with different request URI. The lower part is slightly more advanced, it's trying to pass a PHP code hoping the server has a silly use of eval() somewhere inside. Quote from: fjabouley on May 03, 2020, 07:13:36 pmDo you think my server problem was linked to that (perhaps some requests made fphttpserver crash at some random point?) Probably, but no one can't be sure without you compiling the application with -gl and dump the stacktrace somewhere examinable (see Logging_exceptions). I personally don't put my http app directly to the public, I have haproxy as the frontman then nginx as the middleman, the app will only start working once the request passes the nginx barrier. ---------------------------------------------------------------------- fjabouley · 2020-05-07 20:08 Thanks again for all leledumbo. Actually the issue seems to be the fact my ip was under heavy attack... Since I switched to cloudflare (updating dns with dnstube), the server hasn't crashed anymore, it now runs for many days without issue. I still receive random requests but they are far less important than before... I'll try to use httprouter, that seems to be more clear code than mine. Do you think the sessions/login I created in my example are enough to be safe using a SSL connection ? it uses cookies to check users, the password is stored in local database (hashed when received) In the login page : Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i  Veuillez entrer le mot de passe de connexion pour continuer