[RESOLVED] FreeSpider Response Code
lazarus / 38-Networking and Web Programming

----------------------------------------------------------------------
jamhitz · 2011-12-18 20:13

Good Evening People,

I have written a small CGI application using FreeSpider but all
responses return HTTP Response Code 200, and there does not appear to
be a way of changing the Response Code. I am willing to dig into the
code, but before I do, I was wondering if there is an existing method
that I just can't seem to find.

I presently rely on Response.CustomHeader.Add('...'), but that is just
not the same thing.

Any ideas?

Thanks.

Hitz

----------------------------------------------------------------------
jamhitz · 2011-12-21 23:42

Well... As currently implemented, FreeSpider does not have custom
Response Codes, so the browser interprets all as "200 Ok".  Delving
deeper into the code, I realized it was quite trivial and decided to
implement this feature. If the FreeSpider author, and/or anyone else
is interested, this is how I did it:

Edit SpiderUtils.pas

Code: [Select]
TSpiderResponse = class(TObject)
private
        fCookiesList:TStringList;
        fContent:TStringList;
        ...
        ...
        fResponseCode:Integer; // <-- New
        fResponseString:String; // <-- new
public
        ....
        property ResponseCode:Integer read fResponseCode write
fResponseCode; // <-- new
        property ResponseString:String read fResponseString write
fResponseString; // <-- new
end;

...
implementation

...
constructor TSpiderResponse.Create;
begin
        ....
        fResponseCode := 200;           // <-- new
        fResponseString := 'OK';        // <-- new
end;

Also, edit file SpiderCGI.pas:

Code: [Select]   procedure TSpiderCGI.Execute;
   var
         ....
         httpVer:String; // <-- Add New Variable
   begin

    ....

     if (Not Found) and (Assigned(fOnRequest)) then
        fOnRequest(Self, fRequest, fResponse);
  
    //<new code>
    httpVer := 'Status: ' + IntToStr(Response.ResponseCode) + ' ' +
Response.ResponseString;
    Writeln(httpVer);
    //</new code>

     ....

In the Request Handler of the TSpiderAction you can then assign
Responses as is appropriate (if not 200 OK) eg:

Code: [Select]Response.ResponseCode := 404;
Response.ResponseString := 'Not Found';

With a little more work, we can maintain a list of known standard
response codes and make this assignment a little more intelligent so
that we don't get mismatched codes (eg 404 OK)

Well... That's it. Hope it helps

----------------------------------------------------------------------
motaz · 2011-12-29 16:36

Hi Jamhitz
could you please send to me the modified SpiderUtils.pas to make it
available for all

Thanks

Motaz
motaz@code.sd

----------------------------------------------------------------------
jamhitz · 2012-01-10 21:37

Will do. Sorry for the delay.

----------------------------------------------------------------------
myopera9 · 2013-02-07 08:35

Could you please show me how to implement this feature not in
spidercgi just in apache module(spiderapache execute).
I have changed spiderapache.pas as like you did, but it did not work
at all.
Thanks in advance!

----------------------------------------------------------------------
jamhitz · 2013-02-08 13:05

@myopera9

I have not used the Apache modules, but looking at the code, I think
you will need to edit SpiderApache.pas and modify the
TSpiderApache.Execute function. The function returns a TSpiderResponse
object, so you will also have to edit SpiderUtils.pas.

I have not done this practically but I trust this should be relatively
easy to do.

Thank you.
JamHitz

----------------------------------------------------------------------
motaz · 2013-02-09 14:35

Now I've added Jamhitz ResponseCode to Response object, and I have
implemented it in CGI version, but I don't know how to set it in
Apache module.
You can check the latest version in GitHub:
https://github.com/motaz/freespider