REST Call to TIKA using INDY
lazarus / 38-Networking and Web Programming

----------------------------------------------------------------------
majlumbo · 2020-02-20 21:10

I am trying to call Apache-TIKA via their REST API.

I have successfully been able to upload a PDF document and return the
document's text via CURL
Code: [Select]curl -X PUT --data-binary @<filename>.pdf
http://localhost:9998/tika --header "Content-type: application/pdf"
That translated to INDY like so:
Code: [Select]function GetPDFText(const FileName: String): String;
var
  IdHTTP:  TIdHTTP;
  Params: TIdMultiPartFormDataStream;
begin
  IdHTTP := TIdTTP.Create;
  try
    Params := TIdMultiPartFormDataStream.Create;
    try
      Params.Add('file', FileName, 'application/pdf')
      Result := IdHTTP.PUT('http://localhost:9998/tika', Params);
    finally
      Params.Free;
    end;   
  finally
    IdHTTP.Free;
  end;
end;
Now I want to upload a word document (.docx) I assumed that all I
would need to do is change the Content-type when I add the file to
Params, but that doesn't seem to produce any results, although I get
no error reported back. I was able to get the following CURL command
to work correctly
Code: [Select]CURL -T <myDOCXfile>.docx http://localhost:9998/tika
--header "Content-type: application/vnd.openxmlformats-
officedocument.wordprocessingml.document"
Any ideas on how to modify the INDY code?

----------------------------------------------------------------------
Remy Lebeau · 2020-02-21 20:41

Quote from: majlumbo on February 20, 2020, 10:10:43 pmI have
successfully been able to upload a PDF document and return the
document's text via CURL
Code: [Select]curl -X PUT --data-binary @<filename>.pdf
http://localhost:9998/tika --header "Content-type: application/pdf"
That translated to INDY like so:

Your translation is wrong.  That CURL command does not send the file
in 'multipart/form-data' format, it sends the file as-is with no
wrapping around it at all.  That would translate to the following in
Indy:

Code: [Select]function GetPDFText(const FileName: String): String;
var
  IdHTTP:  TIdHTTP;
  FS: TFileStream;
begin
  IdHTTP := TIdHTTP.Create;
  try
    FS := TFileStream.Create(FileName, fmOpenRead or
fmShareDenyWrite);
    try
      IdHTTP.Request.ContentType := 'application/pdf';
      Result := IdHTTP.Put('http://localhost:9998/tika', FS);
    finally
      FS.Free;
    end;   
  finally
    IdHTTP.Free;
  end;
end;

If you want to send data in 'multipart/form-data' format, you need to
use TIdHTTP.Post() instead of TIdHTTP.Put(), and you need to send it
to 'http://localhost:9998/tika/form' instead of to
'http://localhost:9998/tika'.  Read the CURL and Apache-Tika
documentations more carefully.

Quote from: majlumbo on February 20, 2020, 10:10:43 pmwhen I add the
file to Params, but that doesn't seem to produce any results, although
I get no error reported back.

Because you are not sending the file correctly to begin with.

Quote from: majlumbo on February 20, 2020, 10:10:43 pmI was able to
get the following CURL command to work correctly
Code: [Select]CURL -T <myDOCXfile>.docx http://localhost:9998/tika
--header "Content-type: application/vnd.openxmlformats-
officedocument.wordprocessingml.document"
Any ideas on how to modify the INDY code?

That CURL command also uses PUT to upload the file as-is, not in
'multipart/form-data' format.  Using the '-T <filename>' parameter for
an HTTP url is similar to using the '-X PUT --data-binary @<filename>'
parameters, eg:

Code: [Select]function GetPDFText(const FileName: String): String;
var
  IdHTTP:  TIdHTTP;
  FS: TFileStream;
begin
  IdHTTP := TIdHTTP.Create;
  try
    FS := TFileStream.Create(FileName, fmOpenRead or
fmShareDenyWrite);
    try
      IdHTTP.Request.ContentType := 'application/vnd.openxmlformats-
officedocument.wordprocessingml.document';
      Result := IdHTTP.Put('http://localhost:9998/tika', FS);
    finally
      FS.Free;
    end;   
  finally
    IdHTTP.Free;
  end;
end;

----------------------------------------------------------------------
Remy Lebeau · 2020-02-21 20:50

Seems your same question on StackOverflow got a more detailed answer. 
You really shouldn't post the same question to multiple forums at the
same time.

----------------------------------------------------------------------
Thaddy · 2020-02-21 21:00

This is the best introduction, btw:
https://medium.com/@marcusfernstrm/create-rest-apis-with-
freepascal-441e4aa447b7
Neat is that it only uses standard provided libraries.
Downside is medium.com, but that does not really matter, because the
content is spot-on.

If you understand that, you can write your own rest api with ease.

Complements to the forum member who wrote that.
This actually belongs to the wiki?