[SOLVED] TFPHTTPClient.Get(URL) / SimpleGet(URL) do not get the full
HTML
lazarus / 38-Networking and Web Programming
----------------------------------------------------------------------
lebao3105 · 2023-12-01 03:22
I am new to fphttpclient. Here is my code:
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i function
downloadItem(const URL: ansistring): string;var http:
TFPHTTPClient;begin // http := TFPHTTPClient.Create(nil); //try
// result := http.Get(URL); //finally // http.Free;
result := TFPHTTPClient.SimpleGet(URL);end;
When I try this for this site:
https://cdn.nickchan.lol/palera1n/c-rewrite/releases/, all I see that
is the downloaded website HTML as expected, but partial:
Code: Text [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
Index of
/palera1n/c-rewrite/releases/Index of
/palera1n/c-rewrite/releases/
../v2.0.0-beta.1/
12-Feb-2023
The same goes for TFPHTTPClient.SimpleGet(URL). However: this works:
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
dl_fphttp_b; {$mode delphi}{$ifdef windows}{$apptype
console}{$endif} uses fphttpclient, opensslsockets; begin
writeln(TFPHttpClient.SimpleGet('https://freepascal.org')); end.
Using TFPHTTPClient with a TStream object works.
Any ideas?
EDIT: Solved by adding {$H+} into my unit
----------------------------------------------------------------------
dseligo · 2023-12-01 04:03
It works on Windows 11, Lazarus 2.2.6, FPC 3.2.2, you didn't write
what OS and version you work on.
I attached test project.
You forgot to free TFPHTTPClient.
----------------------------------------------------------------------
lebao3105 · 2023-12-01 16:24
Quote from: dseligo on December 01, 2023, 05:03:08 amIt works on
Windows 11, Lazarus 2.2.6, FPC 3.2.2, you didn't write what OS and
version you work on.
I attached test project.
You forgot to free TFPHTTPClient.
Sorry for my mistakes.
I’m using Arch Linux with Lazarus 3.0rc2 and fpc 3.2.2-9. For the
missing http.Free, I forgot to add it to my post. However it still
didn’t work.
----------------------------------------------------------------------
TRon · 2023-12-01 17:01
Quote from: lebao3105 on December 01, 2023, 05:24:55 pmI’m using Arch
Linux with Lazarus 3.0rc2 and fpc 3.2.2-9. For the missing http.Free,
I forgot to add it to my post. However it still didn’t work.
Since the resulting number of characters that you do receive seem to
be around 250 ... are you perhaps using shortstring ?
Full compilable example please (including used compiler/build flags) .
Anything else is just a guessing game.
----------------------------------------------------------------------
dseligo · 2023-12-01 20:14
Quote from: lebao3105 on December 01, 2023, 05:24:55 pmI’m using Arch
Linux with Lazarus 3.0rc2 and fpc 3.2.2-9. For the missing http.Free,
I forgot to add it to my post. However it still didn’t work.
Did you try project I attached?
----------------------------------------------------------------------
lebao3105 · 2023-12-02 04:24
Quote from: dseligo on December 01, 2023, 09:14:00 pmQuote from:
lebao3105 on December 01, 2023, 05:24:55 pmI’m using Arch Linux with
Lazarus 3.0rc2 and fpc 3.2.2-9. For the missing http.Free, I forgot to
add it to my post. However it still didn’t work.
Did you try project I attached?
I did, and it worked. But not in my project.
Quote from: TRon on December 01, 2023, 06:01:42 pmQuote from:
lebao3105 on December 01, 2023, 05:24:55 pmI’m using Arch Linux with
Lazarus 3.0rc2 and fpc 3.2.2-9. For the missing http.Free, I forgot to
add it to my post. However it still didn’t work.
Since the resulting number of characters that you do receive seem to
be around 250 ... are you perhaps using shortstring ?
Full compilable example please (including used compiler/build flags) .
Anything else is just a guessing game.
Shortstring... you made me realized that I need the {$H+}. That's why
the result is that short.
Firstly I tried dseligo's code, then just run TFPHTTPClient.Get(URL).
I tried to assign the result to a variable and writeln() it, and yet
the download result still that short.
Indeed I did not used any compiler/build flag else than {$mode objfpc}
in my file (which is -Mobjfpc on command-line) for now (I don't use
Lazarus). Here is my code (not fixed):
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i unit
resoures;{$mode objfpc} Interface function downloadItem(const URL:
ansistring): string;function downloadItem(const URL: ansistring,
outPath: ansistring): boolean; function getReleases:
TStringArray; Implementation function downloadItem(const URL:
shortstring): shortstring;var http: TFPHTTPClient;begin
http := TFPHTTPClient.Create(nil); try result :=
http.Get(URL); finally http.Free;
end;end; function downloadItem(const URL: ansistring; outPath:
ansistring): boolean;var fs: TFileStream; http:
TFPHTTPClient; begin http := TFPHTTPClient.Create(nil);
FS := TFileStream.Create(outPath, fmCreate or fmOpenWrite);
try http.Get(URL, FS); finally
fs.Free; http.Free; result := true;
end;end; function getReleases: TStringArray;var URL: string;
content: string; begin if usesNightly = true then
URL :=
'https://cdn.nickchan.lol/palera1n/artifacts/c-rewrite/main'
else URL :=
'https://cdn.nickchan.lol/palera1n/c-rewrite/releases/';
content := downloadItem(URL); writeln(content); //
unimplemented html parseend; end.
Currently my program just calls getReleases at first.
The problem is fixed for now, thanks everyone for trying to help me
out! I will need to learn more...
----------------------------------------------------------------------
dseligo · 2023-12-02 09:38
Quote from: lebao3105 on December 02, 2023, 05:24:06 amThe problem is
fixed for now, thanks everyone for trying to help me out! I will need
to learn more...
If you use {$mode objfpc}, then add {$H+} to it so that it looks like
that:
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i {$mode
objfpc}{$H+}
In mode Delphi Ansistrings are turned on by default.
Also change Shortstring type to String in your program.
----------------------------------------------------------------------
TRon · 2023-12-02 12:28
Quote from: lebao3105 on December 02, 2023, 05:24:06 amShortstring...
you made me realized that I need the {$H+}. That's why the result is
that short.
Thank you for reporting back and good to hear that you got that sorted
out.
$H+ is a small (as in number of characters) but very important
directive fir these cases. dseligo's advise to always define it when
using mode objfpc is solid (for now, until/once using unicode rtl
which is a new upcoming feature and for which default string type
settings might change and/or be different).
----------------------------------------------------------------------
Thaddy · 2023-12-02 13:30
Quote from: dseligo on December 02, 2023, 10:38:26 amAlso change
Shortstring type to String in your program.
Imnsho opinion it is the other way around, specify your string type
exactly....
That way you do not mix up all the alternative string types available
that modern pascal provides. Just string is horrid. Especially for
beginners that do not know about modes.
BAD advice.