N-tier multiuser application development
lazarus / 38-Networking and Web Programming

----------------------------------------------------------------------
JD · 2010-08-10 16:15

I'm working on the development of a multiuser application using a 3
tier design - a rich client, an application server & a database
backend. I have the following options:

a) Build everything in Lazarus

    Client: Lazarus Indy application
    Application Server: Custom Lazarus Indy application
    Database backend: Firebird/PostgreSQL

    The client side could use the TMemDataset to send streamed data
using Indy TCP components to the application server which uses Zeos to
communicate with the database. The problem is TMemDataset cannot
handle master/detail relationships unlike the ClientDataset in Delphi.
In addition, passing data over sockets can be blocked by firewalls on
the client machine & in addition, sensitive data must be encrypted
before sending it over the wire. Encrypting & then decrypting data may
have a negative impact on application speed.

b) Build a hybrid solution (I)
   
    Client: A rich client based on Lazarus/Delphi
    Application Server: Apache Geronimo/Glassfish
    Database backend: Firebird/PostgreSQL/JavaDB (also called Apache
Derby)
    Database server: Apache Server

    The client side will use Habari ActiveMQ Client to communicate
with the application server. The problem is I've never used it before
& I'm wondering about the ease of deployment of this solution. It
however looks very promising & is one for the future.

c) Build a hybrid solution (II)

    Client: A rich ExtPascal client
    Application Server: Custom Lazarus application
    Database backend: Firebird/PostgreSQL
    Database server: Apache Server

    The client could use CGI or FastCGI to communicate with the
application server. The problem here again is that I've never tried
this before but ExtPascal (based on ExtJS) looks promising & is also
likely to be one for the future.

I'm open to suggestions on the best way to approach this problem. I
don't want to use a browser based client side because I don't think
they are good at handling data displayed in grids. It is absolutely
essential for me that the client resembles a normal desktop
application but its inner workings make it a web application. I want
the users to have a rich desktop experience. 

I know Lazarus does not have anything like DataSnap or WebSnap, so
what can I use as alternatives? I've looked at the excellent
FreeSpider component but like I said the grid displays are an issue
for me.

I'll greatly appreciate your advice, links, articles etc.

----------------------------------------------------------------------
jixian.yang · 2010-08-10 16:39

Option A is appreciated.
Use Memorytable of Rxlib with source code modified, we can handle the
modified, deleted and inserted data, like delta data in delphi.

With SSL support, Synapse and lNet are popular.

----------------------------------------------------------------------
JD · 2010-08-10 17:04

Quote from: jixian.yang on August 10, 2010, 06:39:06 pmUse Memorytable
of Rxlib with source code modified, we can handle the modified,
deleted and inserted data, like delta data in delphi.

I have RxLib but what does "use Memorytable of RxLib with source code
modified..." mean? What source code was modified?

----------------------------------------------------------------------
jixian.yang · 2010-08-11 01:33

The modified code can handle old value and new value of record. If
just handle the modified, deleted and inserted data, the override
InternalOpen, etc can do, not need to modify source code.

The following is the change:

Comparing files RxMemDsEx.pas and RXMEMDS.PAS
***** RxMemDsEx.pas

unit RxMemDsEx;

{$IFNDEF LINUX}
{$I rxl\rx.inc}
{$ELSE}
{$I RXL/rx.inc}
{$ENDIF}

***** RXMEMDS.PAS

unit rxmemds;


{$I rx.inc}

*****

***** RxMemDsEx.pas

uses SysUtils, Classes, Controls, DB, dbutils;
***** RXMEMDS.PAS


uses SysUtils, Classes, Controls, DB, dbutils;
*****

***** RxMemDsEx.pas

{ TRxMemoryData}

***** RXMEMDS.PAS

{ TRxMemoryData }

*****

***** RxMemDsEx.pas

  TRxMemoryData= class(TDataSet)
  private
    FOldBuf:PChar;
    FOnFilterRecordEx: TFilterRecordEvent;
***** RXMEMDS.PAS

  TRxMemoryData = class(TDataSet)
  private
    FOnFilterRecordEx: TFilterRecordEvent;
*****

***** RxMemDsEx.pas
    procedure InternalDelete; override;
    procedure InternalEdit; override;
    procedure InternalPost; override;
***** RXMEMDS.PAS
    procedure InternalDelete; override;
    procedure InternalPost; override;
*****

***** RxMemDsEx.pas
    procedure InternalOpen; override;
    procedure InternalCancel; override;
    procedure OpenCursor(InfoQuery: Boolean); override;
***** RXMEMDS.PAS
    procedure InternalOpen; override;
    procedure OpenCursor(InfoQuery: Boolean); override;
*****

***** RxMemDsEx.pas

{ TRxMemoryData}

***** RXMEMDS.PAS

{ TRxMemoryData }

*****

***** RxMemDsEx.pas
  FRecords := TList.Create;
  FOldBuf := nil;
end;
***** RXMEMDS.PAS
  FRecords := TList.Create;
end;
*****

***** RxMemDsEx.pas

procedure TRxMemoryData.InternalEdit;
begin
     inherited InternalEdit;
     GetMem(FOldBuf,FRecBufSize);
     Move(ActiveBuffer^,FOldBuf^,FRecBufSize);
end;
***** RXMEMDS.PAS

{ Records Management }

function TRxMemoryData.GetCapacity: Integer;
begin
  if FRecords <> nil then Result := FRecords.Capacity
  else Result := 0;
end;
*****

***** RxMemDsEx.pas

{ Records Management }

function TRxMemoryData.GetCapacity: Integer;
begin
  if FRecords <> nil then Result := FRecords.Capacity
  else Result := 0;
end;
***** RXMEMDS.PAS

procedure TRxMemoryData.SetCapacity(Value: Integer);
begin
  if FRecords <> nil then FRecords.Capacity := Value;
end;
*****

***** RxMemDsEx.pas

procedure TRxMemoryData.SetCapacity(Value: Integer);
begin
  if FRecords <> nil then FRecords.Capacity := Value;
end;
***** RXMEMDS.PAS

function TRxMemoryData.AddRecord: TMemoryRecord;
begin
  Result := TMemoryRecord.Create(Self);
end;
*****

***** RxMemDsEx.pas

function TRxMemoryData.AddRecord: TMemoryRecord;
begin
  Result := TMemoryRecord.Create(Self);
end;

function TRxMemoryData.FindRecordID(ID: Integer): TMemoryRecord;
***** RXMEMDS.PAS

function TRxMemoryData.FindRecordID(ID: Integer): TMemoryRecord;
*****

***** RxMemDsEx.pas
    dsCalcFields: RecBuf := CalcBuffer;
    dsOldValue:
    begin
    if FOldBuf=nil then
    RecBuf:=ActiveBuffer
    else
    RecBuf:=FOldBuf;
    end;
    dsFilter: RecBuf := TempBuffer;
***** RXMEMDS.PAS
    dsCalcFields: RecBuf := CalcBuffer;
    dsFilter: RecBuf := TempBuffer;
*****

***** RxMemDsEx.pas
  SetMemoryRecordData(Buffer, Rec.Index);

  GetMem(FOldBuf,FRecBufSize);
  Move(ActiveBuffer^, FOldBuf^, FRecBufSize);
end;
***** RXMEMDS.PAS
  SetMemoryRecordData(Buffer, Rec.Index);
end;
*****

***** RxMemDsEx.pas
  end;
  FreeMem(FOldBuf);
  FOldBuf:=nil;
end;
***** RXMEMDS.PAS
  end;
end;
*****

***** RxMemDsEx.pas
//  OpenCursor(false);
  //           俞疣螯 镱耠?桉镳噔脲龛
{  Fields.Clear;
***** RXMEMDS.PAS
//  OpenCursor(false);
  //           俞疣螯 镱耠?桉镳噔脲龛
{  Fields.Clear;
*****

***** RxMemDsEx.pas

procedure TRxMemoryData.InternalCancel;
begin
     FreeMem(FOldBuf);
     FOldBuf:=nil;
     inherited InternalCancel;
end;
***** RXMEMDS.PAS

procedure TRxMemoryData.InternalHandleException;
begin
  Application.HandleException(Self);
end;
*****

***** RxMemDsEx.pas

procedure TRxMemoryData.InternalHandleException;
begin
  Application.HandleException(Self);
end;
***** RXMEMDS.PAS

procedure TRxMemoryData.InternalInitFieldDefs;
begin
end;
*****

***** RxMemDsEx.pas

procedure TRxMemoryData.InternalInitFieldDefs;
begin
end;
***** RXMEMDS.PAS

function TRxMemoryData.IsCursorOpen: Boolean;
begin
  Result := FActive;
end;
*****

***** RxMemDsEx.pas

function TRxMemoryData.IsCursorOpen: Boolean;
begin
  Result := FActive;
end;

{ Informational }
***** RXMEMDS.PAS

{ Informational }
*****

----------------------------------------------------------------------
herux · 2010-08-11 03:03

Go to Option
 D. Using http://wiki.freepascal.org/tiOPF, its support HTTP Proxy-
Remote.

----------------------------------------------------------------------
JD · 2010-08-11 07:26

Quote from: herux on August 11, 2010, 05:03:40 amGo to Option
 D. Using http://wiki.freepascal.org/tiOPF, its support HTTP Proxy-
Remote.

Thanks I already know about tiOPF. I skipped it because it is not
clear if it works with ZeosLib. I'll take another look at it but
having the Zeos option is absolutely important for me.

----------------------------------------------------------------------
herux · 2010-08-11 08:37

I use zeoslib in tiopf, and that's ok. you just need a little editing

----------------------------------------------------------------------
JD · 2010-08-11 09:06

Quote from: herux on August 11, 2010, 10:37:14 amI use zeoslib in
tiopf, and that's ok. you just need a little editing

What needs to be edited?

----------------------------------------------------------------------
JD · 2010-08-13 07:54

Waiting for inspiration.....  Anybody there ???

----------------------------------------------------------------------
herux · 2010-08-13 08:23

QuoteWhat needs to be edited?
create objects from tiopf, use zeosdbo as the data connection.
I think you'll know what needs to be corrected.
just something simple, such as the use of execsql and Open method

----------------------------------------------------------------------
herux · 2010-08-13 08:24

oh sorry you have to grab the source from svn

----------------------------------------------------------------------
JD · 2010-08-13 08:57

Quote from: herux on August 13, 2010, 10:24:58 amoh sorry you have to
grab the source from svn

You're right there because the official release cannot be compiled. I
kept getting errors.

----------------------------------------------------------------------
sandeep_c24 · 2010-08-20 03:12

So what have you decided to use?

Sandeep

----------------------------------------------------------------------
JD · 2010-08-21 18:18

Quote from: sandeep_c24 on August 20, 2010, 05:12:10 amSo what have
you decided to use?

Sandeep

I've decided to proceed this way

1) For small to medium scale projects, I'll adopt a mixture of my
option (a) & the tiOPF package as the application server.

2) For larger scale projects, I'll have to use a Java application
servers because of freely available reporting tools (BIRT & iReports).
This also makes sense because some companies already have them in
place & they are proven solutions.

I really like ExtPascal (it looks beautiful) but I'm not too clear
about licensing. ExtJS has both an open source license & a commercial
license. How does this affect ExtPascal? I'm not too sure.

----------------------------------------------------------------------
bobo · 2010-09-29 23:32

Kind of late to join the discussion here, but I am now successfully
using ExtJS directly with FPC/Lazarus with the latest fcl-web/fpweb
with JSON communication (works with XML too, but XML was a big
overhead I did not need), also using Ajax calls from the ExtJS side to
call the web server CGI/FCGI actions, also using html templates with
fptemplate, etc. All works.

You need to have the latest FPC 2.5.1 SVN sources and the latest
Lazarus SVN sources at the moment, but it works nicely.

No licensing issues or anything...

----------------------------------------------------------------------
JD · 2010-09-30 00:02

Hi bobo,

Quote from: bobo on September 30, 2010, 01:32:32 amNo licensing issues
or anything...

Ext JS is not free as far as I know. This is from Wikipedia
QuoteOn 20 April 2008, Ext 2.1 was released under a new dual license
which allowed the options of the full GPL 3.0 license or a commercial
license.[10]

The change in license over time, from a permissive open source license
to a restrictive dual license, caused controversy in the ExtJS user
community.

I'm wary about rolling out an ExtPascal solution only to discover that
the Ext JS license will choke me or my clients.

Are you really sure when you say there are no licensing issues?

----------------------------------------------------------------------
bobo · 2010-09-30 00:30

Checking out  http://www.sencha.com/products/license-faq.php

I am not distributing extjs with my projects, the clients can download
and extract extjs themselves on their web server as part of the
installation process.

As for extending ExtJS functionality - that happens rarely by you -,
usually someone already done it and you just download the extensions,
etc. so it is already published.
In the rare cases you do new extensions (I never needed it) to ExtJS,
you can publish those small individual javascript files (or collect
them into one), publish them, it does not affect your FPC/Lazarus
project at all.

As for ExtPascal, I don't need one additional layer between the client
and my server app, I am not using it.
The current fpweb/FCL-web provided functions are perfectly sufficient.
Of course, using ExtJS directly means touching some JavaScript
"things" but that should not be an issue for any serious developer,
because it is really small with ExtJS, mostly giving values to
properties only.

----------------------------------------------------------------------
JD · 2010-09-30 09:15

Hi Bobo,

I thought using Ext JS in commercial applications required the
purchase of a license. I'll check out the link for clarification.

Using Javascript is not an issue because I already work in Java (using
Eclipse IDE). I've never tried fpweb/FCL-web because examples &
documentation are hard to find. I tend to gravitate towards Java
whenever I need a solution that makes extensive use of Internet
technology.

This is the first time I'm trying to build something that big in
FPC/Lazarus for a market that does not have the hardware to run Java
apps. A case of market segmentation if you will.

So do you have examples of the use of fpweb/FCL-web and secondly can
you make a diagram of the way you used it in the layers of your
application for me.

Thanks

----------------------------------------------------------------------
bobo · 2010-09-30 19:54

According to the linked ExtJS licensing FAQ, you have obligations to
publish the ExtJS JavaScript code that YOU have modified AND(!) you
are distributing it. So, in-house projects are excluded by default, as
well as all extensions you have downloaded from the Internet - since
those are already published back to the community. As I've noted, I
never needed to extend ExtJS so far, always found that someone else
did what I needed already.
Either case, it does not affect your FPC/Lazarus project parts.

True, that FPC/Lazarus fpweb/FCL-web documentation is not abundant  -
not surprising, considering that the latest ExtJS functionality is
only in the 2.5.1 branch of FPC and only in the trunk of Lazarus, both
from SVN sources, obviously - , but there are some - trivial -
examples and demos available to get you over the first few bumps.
For the core Lazarus components you can find demos under
...your Lazarus dir.../components/fpweb/demo
(sessions, how to use templates, etc.)

and for the core FCL parts you can find some demos under
...your FPC dir.../packages/fcl-web/examples
(extjs, json, etc., I'd look at the webdata mini-demos there)

And there are the FPC and Lazarus mailing lists where you can ask your
questions from the community or the developers themselves.

----------------------------------------------------------------------
jl · 2010-10-19 03:05

Hi bobo, this is interesting, by the way, is your web app multi-user? 
If so, how are you implementing the multi-logon sessions?


Quote from: bobo on September 30, 2010, 09:54:29 pmTrue, that
FPC/Lazarus fpweb/FCL-web documentation is not abundant  - not
surprising, considering that the latest ExtJS functionality is only in
the 2.5.1 branch of FPC and only in the trunk of Lazarus, both from
SVN sources, obviously - , but there are some - trivial - examples and
demos available to get you over the first few bumps.
For the core Lazarus components you can find demos under
...your Lazarus dir.../components/fpweb/demo
(sessions, how to use templates, etc.)

and for the core FCL parts you can find some demos under
...your FPC dir.../packages/fcl-web/examples
(extjs, json, etc., I'd look at the webdata mini-demos there)

And there are the FPC and Lazarus mailing lists where you can ask your
questions from the community or the developers themselves.

----------------------------------------------------------------------
bobo · 2010-10-19 06:14

For web applications I have used both cookie sessions and URL sessions
(and posted the basic examples of them in the fpweb fptemplate
examples above) in different projects depending on the need.
I store the user login/pwd data in a table and assign a unique session
id to the web clients after login. That session id is used throughout
their communication with the web application until they log out or
their session expires.
The session id is either in a cookie (cookie sessions) or embedded in
every link (encrypted usually) on the response pages, so any following
server calls will pass them to the server side CGI/FCGI/Apache_module
back-end app therefore identifying the user that has logged in
earlier.

If not web pages are used, and the communication is stateless, then
you still need some kind of session id to identify the people and only
give the clients data they can access.

For native client/server apps, you do not need sessions because the
client application just directly can access the back-end server
database (VPN or secure tunnel over tcp/ip, etc.), or calls functions
there to get data back (the client is not stateless). Usually in these
cases the client does the authentication at least at start.

AB

----------------------------------------------------------------------
jl · 2010-10-19 14:18

Thanks for explaining.  Does session work the same for ExtJS?  Note: I
still haven't got the chance to work on the demos as I'm still stuck
at compiling Laz with FPC 2.5.1.  

Chanced upon this article on session with ExtJS -
http://ffzhuang.blogspot.com/2007/12/ajax-based-extjs-
authentication.html

It mentions "The HTTP stack in the browser maintains the sessionid for
each successive invocation to the web server that it's talking to and
ONLY for that web server. As long as the browser is running, the
sessionid is maintained. Once the browser goes away, the sessionid
goes away with it. There are known session hijacking hacks out there,
but they can be averted by providing a bit more information within the
variables attached to the sessionid on the server."

This seems quite different from cookies or sessionid embedded in link.

Quote from: bobo on October 19, 2010, 08:14:28 amFor web applications
I have used both cookie sessions and URL sessions (and posted the
basic examples of them in the fpweb fptemplate examples above) in
different projects depending on the need.
I store the user login/pwd data in a table and assign a unique session
id to the web clients after login. That session id is used throughout
their communication with the web application until they log out or
their session expires.
The session id is either in a cookie (cookie sessions) or embedded in
every link (encrypted usually) on the response pages, so any following
server calls will pass them to the server side CGI/FCGI/Apache_module
back-end app therefore identifying the user that has logged in
earlier.

----------------------------------------------------------------------
bobo · 2010-10-19 16:25

Well, I did not read the link you put there, but from the quote you
provided, it is the same session cookie that fpweb has built in. Those
also go away after the browser is closed, therefore closing the
session on the client side due to destroying the cookie there, so no
request will come to the server from then on from any client for that
session id.

As for "URL sessions", those are totally handled from server side, so
either a log-out action or time-out is the way to exit/destroy a
session.

What I do with ExtJS is to have an additional "KeepAlive" action
available on the server side that refreshes the session time-out clock
on the server in case the user spends a lot of time client side
actively without communicating to the server.
Every now and then, from the ExtJS client this action is called on the
server, therefore refreshing the session there (ex: every time the
user clicks on a grid cell but 5 minutes hasn't been passed since the
last keepalive call, etc.).

AB