[SOLVED] IdHTTP and BAD REQUEST
lazarus / 38-Networking and Web Programming

----------------------------------------------------------------------
DanishMale · 2017-05-05 23:01

I've got a problem which drives me nuts....

I have googled and searched for a couple of days now and still not
found any solution which solves my problem

I keep getting the error HTTP/1.1 - 400 Bad Request

My code is as follows:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i function
GetRemoteData(web_url: String): String;var  HTTP1: TIdHTTP;  StrList,
Data: TStringList;begin try   try   HTTP1 := TIdHTTP.Create(nil); 
 HTTP1.Request.UserAgent:= 'APP
Name/'+GetAppVer(GetAppPath+'emcd.exe')+ ' +http://app url'; 
 HTTP1.Request.CharSet := 'utf-8';   if (POS('per_page',web_url) > 0)
then   begin     HTTP1.Request.ContentType := 'application/x-www-form-
urlencoded';     HTTP1.IOHandler :=
TIdSSLIOHandlerSocketOpenSSL.Create(nil);     HTTP1.Request.Accept :=
'application/vnd.discogs.v2.plaintext+json';     StrList :=
TStringList.Create;     SplitText('&', web_url, StrList);   
 StrList[0] := StringReplace(StrList[0],' ','%20',[rfReplaceAll]);   
 Data := TStringList.Create;     Data.Values['q'] :=
copy(StrList[0],POS('=',StrList[0])+1, Length(StrList[0]));   
 Data.Values['key'] := copy(StrList[1],POS('=',StrList[1])+1,
Length(StrList[1]));     Data.Values['secret'] :=
copy(StrList[2],POS('=',StrList[2])+1, Length(StrList[2]));   
 Data.Values['per_page'] := copy(StrList[3],POS('=',StrList[3])+1,
Length(StrList[3]));     Data.Values['page'] :=
copy(StrList[4],POS('=',StrList[4])+1, Length(StrList[4]));   
 StrList.Destroy;     Result :=
HTTP1.Post('https://api.discogs.com/database/search',data);   
 Data.Destroy;   end   else   begin     HTTP1.Request.ContentType :=
'text/html';     Result := HTTP1.Get(web_url, IndyTextEncoding_UTF8); 
 end;    except     on Error: Exception do        ShowMessage('My
Errorcatcher: ' +Error.ClassName+' - '+Error.Message);   end; finally 
 HTTP1.Free; end;end; 
The error only occurs when I try to POST (e.g using the part where
"per_page" is the query and the main url is "hardcoded" hence it is
static, any other request works perfectly using the GET (e.g.
everything w/o "per_page")

I have the correct SSL dlls installed (hence they are reuse from
another program)

I hope some you intelligent guys out there can look thru and point out
my mistake

----------------------------------------------------------------------
Leledumbo · 2017-05-06 06:32

HTTP 4XX is ALWAYS your fault (invalid credentials, invalid request
method or parameter, etc.), in which without any valid and up to date
(as in: the one the server is running) reference to discogs API, I
can't tell much. I don't usually return empty data in 4XX HTTP
response for API, naturally at least a reason should be given
(probably with fix suggestion), but it all comes down to the API
developer.

----------------------------------------------------------------------
DanishMale · 2017-05-07 11:25

I don't know what happened ... today I switched back to one of my
previous solutions and now it works as expected ....


Thx for your time and answer Leledumbo

----------------------------------------------------------------------
Remy Lebeau · 2017-05-09 18:38

Quote from: DanishMale on May 06, 2017, 01:01:59 amCode: Pascal
 [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
SplitText('&', web_url, StrList);
What is the actual value of web_url?

Quote from: DanishMale on May 06, 2017, 01:01:59 amCode: Pascal
 [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i StrList[0]
:= StringReplace(StrList[0],' ','%20',[rfReplaceAll]);
Why are you doing that?  And only for the first string and not the
other strings?

In "application/x-www-form-urlencoded", spaces are encoded as "+", not
as "%20".  And besides, TIdHTTP.Post() already encodes spaces for
you.  What you are doing is problematic if any "%" characters
belonging to urlencoded sequences end up in your TStringList strings,
because TIdHTTP.Post() will re-encode them as "%25" instead of leaving
them as "%", thus corrupting the data the server sees.  Unless you
disable the hoForceEncodeParams flag in the TIdHTTP.HTTPOptions
property (it is enabled by default).

Quote from: DanishMale on May 06, 2017, 01:01:59 amCode: Pascal
 [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
Data.Values['q'] := copy(StrList[0],POS('=',StrList[0])+1,
Length(StrList[0]));Data.Values['key'] :=
copy(StrList[1],POS('=',StrList[1])+1,
Length(StrList[1]));Data.Values['secret'] :=
copy(StrList[2],POS('=',StrList[2])+1,
Length(StrList[2]));Data.Values['per_page'] :=
copy(StrList[3],POS('=',StrList[3])+1,
Length(StrList[3]));Data.Values['page'] :=
copy(StrList[4],POS('=',StrList[4])+1, Length(StrList[4]));
You are assuming the StrList values are in the same order as the Data
values.  I would do it more dynamically instead:

Code: [Select]for I := 0 to StrList.Count-1 do
begin
  J := Pos('=', StrList[I]);
  Data.Values[Copy(StrList[I], 1, J-1)] := Copy(StrList[I], J+1,
MaxInt);
end;
Or simpler:

Code: [Select]for I := 0 to StrList.Count-1 do
begin
  Data.Values[StrList.Names[I]] := StrList.ValueFromIndex[I];
end;
Or, since the strings are clearly already in "name=value" format, just
add them as-is instead:

Code: [Select]for I := 0 to StrList.Count-1 do
  Data.Add(StrList[I]);
Or simpler:

Code: [Select]Data.Assign(StrList);
In which case, you don't need two separate TStringLists at all.  Get
rid of Data and post StrList as-is instead:

Code: [Select]Result :=
HTTP1.Post('https://api.discogs.com/database/search', StrList);
Or, if web_url is already in "application/x-www-form-urlencoded"
format to begin with (such as if are extracting it from the query
portion of a URL), then just post it as-is without parsing it at all:

Code: [Select]Strm := TStringStream.Create(web_url);
Result := HTTP1.Post('https://api.discogs.com/database/search', Strm);
Strm.Destroy;
Quote from: DanishMale on May 06, 2017, 01:01:59 amThe error only
occurs when I try to POST

That usually means you are corrupting the data you are posting.

Quote from: DanishMale on May 06, 2017, 01:01:59 amany other request
works perfectly using the GET (e.g. everything w/o "per_page")

Sure, because you are not sending any data to the server, you are
requesting data from the server instead.

----------------------------------------------------------------------
DanishMale · 2017-05-12 01:31

@Remy Lebeau:

SplitText & StringReplace : The actual value of web_url is static and
ONLY the first part is dynamic (StrList[0])

Data.Values are ALWAYS in the same order due to it's hardcoded

I am grateful for your suggestions how to improve my coding and your
fantastic explanation on this issue. You pointed out something I
should have remembered (e.g. diff between GET and POST)  

Thanks again to all for their great contributions