Oauth2 Redirect URI, best practices lazarus / 38-Networking and Web Programming ---------------------------------------------------------------------- Maris · 2019-04-18 10:01 Hello everybody! I wanted to ask an advice about best practices to implement redirect from web browser to desktop application during Oauth2 login process. REST API provider is allegro.pl. Here is the test code for login process: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i {AUTH_URI - Allegro endpoint for user authentication in OAuth;TOKEN_URI - Allegro endpoint which enables to obtain an OAuth token;CLIENT_ID - Client ID (obtained upon application registration)CLIENT_SECRET - Client Secret (obtained upon application registration)REDIRECT_URI - Return address where the code is sent}procedure TForm1.ButtonLoginClick(Sender: TObject);var  HTTPClient: TFPHTTPClient;begin  if Length(EditCode.Text) > 0 then  begin    HTTPClient := TFPHTTPClient.Create(nil);    try      HTTPClient.AllowRedirect := True;      HTTPClient.AddHeader('Authorization', 'Basic ' + EncodeStringBase64(CLIENT_ID + ':' + CLIENT_SECRET));      HTTPClient.Post(TOKEN_URI + '?grant_type=authorization_code&code=' + EditCode.Text + '&redirect_uri=' + REDIRECT_URI, MemoResponse.Lines);    finally      HTTPClient.Free;    end;  end  else    begin      OpenURL(AUTH_URI + '?response_type=code&client_id=' + CLIENT_ID + '&redirect_uri=' + REDIRECT_URI + '&prompt=confirm');    end;end; Process here is very simple. After initial login and redirect to specified Redirect URI (http://localhost), with provided code in URL as get parameter, user should take this code from URL, switch back to program, paste it into particular field, and press Login button. Program makes POST request to REST service provider and gets access token. So question is, what is the correct way to implement reading of code from Redirect URI within desktop application? After some research I found few ways to do that, not sure which is best though. 1) Implementing lightweight web server within application which listens on some custom port (80 is not nice idea here). Not sure how to do that yet, but probably this tutorial can be used. 2) Custom URI Scheme. Something similar like "callto:" links. Never tried this before. Also there are no unique implementation method for such links on various platforms. For Linux it's even different between various Desktop Environments. 3) Integrate web browser into application. Probably OLE automation can be used here, but this would be Windows only solution, and probably not good idea in general, because making initial login step without system web browser and hiding this process from user is not welcome from REST API providers. I think lightweight web server would be best way. Or maybe there are other ways to do that? Thank you in advance for any thoughts about this question! ---------------------------------------------------------------------- lucamar · 2019-04-18 13:16 If there is no other way I would include a very small server in my app* but ... are you sure there is no other way to do it? Some process specific for deskop apps? Also, note that sending "http://localhost:9090" (for example) as redirect URI will make the (external) OAuth server try to redirect to its own port 9090! Use your IP instead of localhost. * Note that, probably, the best solution migth be embedding a browser in your app but as of today that basically means distributing most of a browser's libs with your app: multimegabytes of third-party code you'll never use. ---------------------------------------------------------------------- Leledumbo · 2019-04-21 09:08 Indeed the first solution is the best ATM. Google Cloud SDK uses it, for instance. It's as simple as: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i uses  fphttpapp, httpdefs, httproute; procedure GetAuthCode(ARequest: TRequest; AResponse: TResponse);begin  WriteLn(ARequest.QueryFields.Values['code']);  AResponse.Content := '';end; begin  HTTPRouter.RegisterRoute('/',@GetAuthCode);  Application.Port := 9000;  Application.Initialize;  Application.Run;end; Note that this is standalone web app, which is expected to run from your desktop app separately using TProcess or one of its wrappers. It writes the code to stdout, which your desktop app should catch before it can issue http requests to the web service. It's possible to fully embed this piece of code and run it in a separate thread, killing the server by calling Application.Terminate from the main thread (your desktop app), but I'll let that for your own improvement. Quote from: lucamar on April 18, 2019, 03:16:57 pmAlso, note that sending "http://localhost:9090" (for example) as redirect URI will make the (external) OAuth server try to redirect to its own port 9090! Use your IP instead of localhost. No, it won't. Redirection happens on client side, server side really does nothing with it except sending 301 with it as location header, plus the auth code in url params. ---------------------------------------------------------------------- Maris · 2019-04-30 21:27 Thanks for your answers! I decided to use embedded web server in separate thread. I tested only starting and stopping of web server so far, to understand how it works, and already have some questions. Server starts without any problems in separate thread, but can't stop it without workaround. Tested on Windows and Linux. Easy to check with netstat. After setting TFPHttpServer.Active := False, web server is still running and stops only after next request to web server. There is no response anymore of course. I found discussion about this problem here. But similar code does not work... Terminating thread directly with Terminate method would not work here either, because this method sets Terminated property of thread to True, and this property would have to be checked periodically within web server object, which is not implemented of course. Workaround is to make request to web server with TFPHTTPClient after TFPHttpServer.Active := False. Request in browser would work either. Here is  web server code. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i unit http_server; {$mode objfpc}{$H+} interface uses  Classes, SysUtils, fphttpserver, fphttpclient; type  TShowStatusEvent = procedure(Status: String) of Object;   THTTPServerThread = Class(TThread)  Private    FHTTPServer: TFPHTTPServer;    Port: Word;    FStatusText: string;    FOnShowStatus: TShowStatusEvent;    Procedure ShowStatus;  Protected  Procedure Execute; override;  Public    Constructor Create(APort : Word{; Const OnRequest: THTTPServerRequestHandler});    Destructor Destroy; override;    Procedure Stop;    Property HTTPServer: TFPHTTPServer Read FHTTPServer;    Property OnShowStatus: TShowStatusEvent Read FOnShowStatus Write FOnShowStatus;  end; implementation constructor THTTPServerThread.Create(APort: Word{; const OnRequest: THTTPServerRequestHandler});begin  Inherited Create(False);  Self.Port := APort;  Self.FreeOnTerminate := True;  FHTTPServer := TFPHTTPServer.Create(Nil);  HTTPServer.Port := Port;  {HTTPServer.OnRequest := OnRequest;}end; procedure THTTPServerThread.ShowStatus;//this method is executed by the mainthread and can therefore access all GUI elements.begin  if Assigned(FOnShowStatus) then  begin    FOnShowStatus(FStatusText);  end;end; procedure THTTPServerThread.Execute;begin  FStatusText := 'Starting server';  Synchronize(@Showstatus);  try    HTTPServer.Active := True;  Finally  end;  FStatusText := 'Server stopped';  Synchronize(@Showstatus);end; procedure THTTPServerThread.Stop;var  HTTPClient: TFPHTTPClient;begin  HTTPServer.Active := False;   {Workaround. Make request to web server, otherwise it will not stop.}  HTTPClient := TFPHTTPClient.Create(nil);  try    try      HTTPClient.SimpleGet('http://localhost:' + IntToStr(Port));    finally      HTTPClient.Free;    end;  except  end;end; destructor THTTPServerThread.Destroy;begin  inherited Destroy;  FreeAndNil(FHTTPServer);end; end. And code in main program, just necessary parts. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure TFormMain.ShowStatus(Status: String);begin  MemoLog.Append(Status);end; procedure TFormMain.ButtonStartWebClick(Sender: TObject);begin  HTTPServerThread := THTTPServerThread.Create(10000);  HTTPServerThread.OnShowStatus := @ShowStatus;end; procedure TFormMain.ButtonStopWebClick(Sender: TObject);begin  HTTPServerThread.Stop;end; Is there a bug in TFPHTTPServer, or this is normal behavior? Maybe there is better workaround? Thanks in advance! ---------------------------------------------------------------------- lucamar · 2019-05-01 05:11 I'm not exactly an expert (quite the contrary, in fact) but, isn't freeing the server enough to stop it? I mean, when you terminate the thread, can't you just call Myserver.Free, without setting Active to False? ---------------------------------------------------------------------- Maris · 2019-05-01 07:55 Thanks, Lucamar! No, not really. Thread is not being terminated, thread stops itself when HTTPServer is stopped. Correct way to terminate thread in the middle of the work would be to use Terminate method, which is not possible for TFPHTTPServer. Here is description how Terminate works. So Terminate method sets Terminated property to True, this property should be checked periodically within Execute method in-between executed commands. For web server it's not possible to check Terminate property periodically, because when server is launched it blocks thread, Execute method stops at line "HTTPServer.Active := True;" and waits while server will be stopped to proceed to next command. So Terminated property would have to be checked inside TFPHTTPServer object. So to stop execution of thread, it's necessary to stop execution of TFPHTTPServer by setting Active to False. The problem is that TFPHTTPServer does not stop until next request is made. In browser or programmatically, does not matter, just need to make request, which will show that server is not accessible. Actually that's why Try Finally Except is used in code to hide unnecessary exception. I also tried to use KillThread. Didn't work either. But this is not correct way to stop threads, so I didn't want to use it and didn't search for solution why it does not work. I also tried to FreeAndNil object right after setting TFPHTTPServer.Active to False, as you sugested, but program raised an exception, which probably is logical, as processes within object is still executing until next request which stops server. I don't know maybe it's a bug in TFPHTTPServer component. Not sure. So far probably best solution would be to leave workaround. Another solution would be to check other components which probably has objects to use as embedded web server, maybe Synapse, Indy or MORMOT. Anyway, thanks for help! Any new ideas would be greatly appreciated! ---------------------------------------------------------------------- lucamar · 2019-05-01 19:55 Thanks for the explanation. Now I finally "get" what the problem is ---------------------------------------------------------------------- Leledumbo · 2019-05-03 05:17 As I said above, executing the web server as external program and catch the stdout is a much simpler solution IMO. It's even a one-liner thanks to the wrapper function (2nd variant with only 2 parameters). ---------------------------------------------------------------------- Maris · 2019-05-03 13:12 Hi, Leledumbo! Yes, it would be simpler. But not so convenient for end user. As an idea is to make Oauth2 login, it would be better if end user have to use only application where he works in, and browser for initial login of course. (See comment in next post) Anyway I tested Web Server which is made by using Sockets. I used Synapse and examples here and here, and any other useful examples and forum topics I was able to find about Sockets programming especially with Synapse library. Here is the code of http server I have so far. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i unit http_listen; {$mode objfpc}{$H+} interface uses  Classes, SysUtils, blcksock, sockets,  Synautil; type  TLogEvent = procedure(LogText: String) of Object;   TLightWeb = class(TThread)  private    Port: word;    ListenerSocket: TTCPBlockSocket;    ConnectionSocket: TTCPBlockSocket;    FLogText: string;    FOnLog: TLogEvent;    procedure AttendConnection(ASocket: TTCPBlockSocket);    procedure Log;  protected    procedure Execute; override;  public    constructor Create(APort: word);    destructor Destroy; override;    Property OnLog: TLogEvent Read FOnLog Write FOnLog;  end; implementation constructor TLightWeb.Create(APort: word);begin  inherited Create(False);  Self.Port := Aport;end; {This method is executed by the mainthread and can therefore access all GUI elements.}procedure TLightWeb.Log;begin  if Assigned(FOnLog) then    FOnLog(FLogText);end; procedure TLightWeb.Execute;begin  ListenerSocket := TTCPBlockSocket.Create;  ConnectionSocket := TTCPBlockSocket.Create;   try    ListenerSocket.CreateSocket;    ListenerSocket.setLinger(True, 10);    ListenerSocket.bind('127.0.0.1', IntToStr(Port));    ListenerSocket.listen;    FLogText := 'Listening';    Synchronize(@Log);     repeat      if ListenerSocket.canread(1000) then      begin        ConnectionSocket.Socket := ListenerSocket.accept;        FLogText := 'Attending Connection. Error code (0=Success): ' + IntToStr(ConnectionSocket.LastError);        Synchronize(@Log);        AttendConnection(ConnectionSocket);        ConnectionSocket.CloseSocket;        FLogText := 'Connection socket closed';        Synchronize(@Log);      end;    until Terminated;  finally;    FreeAndNil(ListenerSocket);    FreeAndNil(ConnectionSocket);  end;   FLogText := 'Thread done';  Synchronize(@Log);end; procedure TLightWeb.AttendConnection(ASocket: TTCPBlockSocket);var  Timeout: integer;  s: string;  OutputDataString: string;begin  Timeout := 10000;   try    try      {Read request line}      FLogText := 'Reading request line';      Synchronize(@Log);      s := ASocket.RecvString(timeout);      FLogText := s;      Synchronize(@Log);       {Read request headers}      FLogText := 'Readinging request headers';      Synchronize(@Log);      repeat      s := ASocket.RecvString(Timeout);        FLogText := s;        Synchronize(@Log);      until s = '';       {Read body}      FLogText := 'Reading body';      Synchronize(@Log);      while ASocket.WaitingDataEx > 0 do      begin        s := ASocket.RecvPacket(timeout);        FLogText := s;        Synchronize(@Log);      end;       {Output HTML}      OutputDataString := '' +        CRLF + '
' + CRLF;       {Write the headers back to the client}      FLogText := 'Writing the headers back to the client';    Synchronize(@Log);      ASocket.SendString('HTTP/1.0 200' + CRLF);    ASocket.SendString('Content-type: Text/Html' + CRLF);      ASocket.SendString('Content-length: ' + IntToStr(Length(OutputDataString)) + CRLF);      ASocket.SendString('Connection: close' + CRLF);      ASocket.SendString('Date: ' + Rfc822DateTime(now) + CRLF);      ASocket.SendString('Server: Lazarus Synapse' + CRLF);      ASocket.SendString(CRLF);       FLogText := 'Socket error code (0=Success): ' + IntToStr(ASocket.LastError);      Synchronize(@Log);      if ASocket.lasterror <> 0 then      begin        FLogText := 'Socket error: ' + ASocket.LastErrorDesc;        Synchronize(@Log);    end;      ASocket.SendString(OutputDataString);    except      on E: Exception do      begin        FLogText := E.Message;        Synchronize(@Log);      end;    end;  finally  end;end; destructor TLightWeb.Destroy();begin  inherited;end; end. This is test code of course, with all logging messages almost everywhere, and maybe some unnecessary lines, so clean up is needed. But it is pretty much ok for task I need. It reads also HTTP body, which is not necessary for Oauth2, so made it for testing purpose only. Oauth2 needs only get parameters, which are in starting request line. As implementation of server is made inside Thread's Execute method, Terminate can and should be used to turn off web server. I noticed if request was made to server, then connection is still present in netstat output for a while after thread was terminated, but probably it should be like this. Not sure how sockets works in details, but probably there is some king of timeout until connection is closed by system. Also not sure how such server would work with concurrent connections, but again, it's good enough for Oauth2 implementation. I'll try to implement full Oquth2 log-in now, will see how it works. ---------------------------------------------------------------------- Maris · 2019-05-03 13:16 Quote from: Leledumbo on May 03, 2019, 07:17:48 amIt's even a one- liner thanks to the wrapper function (2nd variant with only 2 parameters). Ah, sorry, didn't pay attention about using external program within main application. Yes, that would be ok for end user either, as he don't have to think about what's going on in background.