External Process streaming terminal to Memo, line by line?
lazarus / 21-General

----------------------------------------------------------------------
tatamata · 2021-03-15 08:59

Hello,

I am using following function to execute external command-line
program. Function returns the output as string which can be then
appended to Memo. This works.
However, it is a long-running process, so I need real-time progress
added to Memo, line by line, so I added this part into the repeat
loop, but it does not update Memo...
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i        
//Transfer stream to Memo_Terminal, line by line - THIS DOES NOT
WORK?!?       
Form1.Memo_Terminal.Lines.LoadFromStream(OutputStream);       
Form1.Memo_Terminal.Refresh; 
The function:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i function
ProcessExecuteSync(exepath: string; params: array of string):
string;{$IFDEF Verbose}const  BUF_SIZE = 2048; // Buffer size for
reading the output in chunks{$ENDIF Verbose}varProcess:
TProcess;i:integer;{$IFDEF Verbose}s: string;OutputStream :
TStream;BytesRead    : longint;Buffer       : array[1..BUF_SIZE] of
byte;AStringlist: TStringList;{$ENDIF Verbose}begin 
ProcessExecuteSync:='';  Process := TProcess.Create(nil);  try   
Process.Executable := exepath;    for i:=0 to high(params) do begin  
  if (params[i]<>'') then begin       
 Process.Parameters.Add(params[i]);      end;    end;    {   
 Process.Options := Process.Options + [poWaitOnExit];  //if you wish
synchronous, otherwise asynchronous     Process.Options :=
Process.Options + [poNoConsole];     Process.Options :=
Process.Options + [poUsePipes]; //if you wish to load output into
string list....      showmessage('process.Parameters: ' +
process.Parameters.Text);    }    {$IFDEF Verbose}    
showmessage('Parameters: ' + process.Parameters.Text);     {$ENDIF
Verbose}     {$IFDEF Verbose}    Process.Options := Process.Options +
[poUsePipes];  //[poWaitOnExit] must not be used!!!    {$ELSE}   
Process.Options := Process.Options + [poWaitOnExit];    {$ENDIF
Verbose}     //Process.Options := Process.Options + [poNoConsole];   
Process.Execute;     {$IFDEF Verbose}    //Output to string    begin 
   OutputStream := TMemoryStream.Create;      // All generated output
from AProcess is read in a loop until no more data is available     
repeat        // Get the new data from the process to a maximum of the
buffer size that was allocated.        // Note that all read(...)
calls will block except for the last one, which returns 0 (zero).    
  BytesRead := Process.Output.Read(Buffer, BUF_SIZE);         // Add
the bytes that were read to the stream for later usage       
OutputStream.Write(Buffer, BytesRead);         //Transfer stream to
Memo_Terminal, line by line - THIS DOES NOT WORK?!?       
Form1.Memo_Terminal.Lines.LoadFromStream(OutputStream);       
Form1.Memo_Terminal.Refresh;       until BytesRead = 0;  // Stop if no
more data is available       AStringList:=TStringList.Create;     
OutputStream.Position := 0; // Required to make sure all data is
copied from the start      AStringList.LoadFromStream(OutputStream);  
    //Transfer whole output to string list      s:= AStringList.Text; 
    ShowMessage('Process Output:' + sLineBreak + s);       //Return
whole output      ProcessExecuteSync:=s;     end;    {$ENDIF Verbose} 
finally    Process.Free;    {$IFDEF Verbose}    AStringList.Free;   
OutputStream.Free;    {$ENDIF Verbose}  end;end; 
Can you help me, how can I get real-time progress of command-line
program execution being transferred line by line to TMemo?

----------------------------------------------------------------------
bytebites · 2021-03-15 09:28

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
Application.ProcessMessages

----------------------------------------------------------------------
Gustavo 'Gus' Carreno · 2021-03-15 10:39

Hey tatamata,

Instead of having the memo refresh, try making the application process
the messages, like bytebites suggests:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i  
//Transfer stream to Memo_Terminal, line by line - THIS DOES NOT
WORK?!?  Form1.Memo_Terminal.Lines.LoadFromStream(OutputStream); 
Application.ProcessMessages;
Cheers,
Gus

----------------------------------------------------------------------
Roland57 · 2021-03-15 11:12

Hello!

There is an example here, and another one here.

You could also use the UnTerminal library, as explained here.

Regards.

Roland

----------------------------------------------------------------------
tatamata · 2021-03-15 14:51

Quote from: gcarreno on March 15, 2021, 11:39:33 amHey tatamata,

Instead of having the memo refresh, try making the application process
the messages, like bytebites suggests:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i  
//Transfer stream to Memo_Terminal, line by line - THIS DOES NOT
WORK?!?  Form1.Memo_Terminal.Lines.LoadFromStream(OutputStream); 
Application.ProcessMessages;
Cheers,
Gus
Hi, thanks, but it didn't solve it.

----------------------------------------------------------------------
tatamata · 2021-03-15 14:52

Quote from: bytebites on March 15, 2021, 10:28:07 amCode: Pascal
 [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
Application.ProcessMessagesHi! Where to put it in code?

----------------------------------------------------------------------
bytebites · 2021-03-15 17:39

As shown in another post.

----------------------------------------------------------------------
engkin · 2021-03-15 22:01

Quote from: tatamata on March 15, 2021, 03:52:27 pmQuote from:
bytebites on March 15, 2021, 10:28:07 amCode: Pascal
 [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
Application.ProcessMessagesHi! Where to put it in code?

Inside your repeat...until loop in your first post.

But that is not the proper way. If you have a long activity, you run
it in a thead and use synchronize to send updates to the main (GUI)
thread.

----------------------------------------------------------------------
tatamata · 2021-03-15 22:25

Quote from: engkin on March 15, 2021, 11:01:24 pmQuote from: tatamata
on March 15, 2021, 03:52:27 pmQuote from: bytebites on March 15, 2021,
10:28:07 amCode: Pascal  [Select][+][-]window.onload = function(){var
x1 = document.getElementById("main_content_section"); if (x1) { var x
= document.getElementsByClassName("geshi");for (var i = 0; i
Application.ProcessMessagesHi! Where to put it in code?

Inside your repeat...until loop in your first post.

But that is not the proper way. If you have a long activity, you run
it in a thead and use synchronize to send updates to the main (GUI)
thread.
Anyway, this with Application.ProcessMessages does not work. I suppose
there must be something else wrong in the code...
Do you have a proper example with thread and synchronize, but with
focus on external process?

----------------------------------------------------------------------
lucamar · 2021-03-15 22:58

That can't work, because LoadFromStream() will keep reading, before
returning, until it reaches the end of the stream, which won't happen
until the process has completed.

What you should do is read a line from the stream and update the memo,
and keep doing it until the stream ends. IIRC, there is an example
doing something similar in the section about TProcess of the wiki page
on Executing External Programs, though you'll have to adapt it quite a
bit.

----------------------------------------------------------------------
Jurassic Pork · 2021-03-15 23:17

hello,
Quote from: Roland57 on March 15, 2021, 12:12:42 pmHello!
There is an example here, and another one here.
tatamata have you read the links given in this post ?
in this one you have the use of TProcessEx :

QuoteQuote from: gnu_trisquel on July 19, 2018, 08:51:38 pm
I have managed to use TProcess to run a command-line tool and output
all messages in a component in my application.

you have better than TProcess to do that :  TProcessEx in the unit
ProcessUtils of fpcUpDeluxe project.
TprocessEx includes Real time output management.
Example under linux to have the output of my biglines executable in
realtime in a Tmemo
Friendly, J.P

----------------------------------------------------------------------
engkin · 2021-03-15 23:51

Drop a TAsyncProcess from the System components. Use its events to
read the data.

----------------------------------------------------------------------
Edson · 2021-03-16 00:02

I think the easy way is to use https://github.com/t-edson/UnTerminal

It controls the output using events.

----------------------------------------------------------------------
engkin · 2021-03-16 00:59

Quote from: Edson on March 16, 2021, 01:02:55 amI think the easy way
is to use https://github.com/t-edson/UnTerminal

It controls the output using events.

TAsyncProcess is event based.

----------------------------------------------------------------------
tatamata · 2021-03-16 16:21

Quote from: Edson on March 16, 2021, 01:02:55 amI think the easy way
is to use https://github.com/t-edson/UnTerminal

It controls the output using events.
Hi, it might be, but I have difficulties to properly understand your
UnTerminal, you haven't documented this case.
Anyway, I found some of your posts and try to follow it, but
apparently I am doing something wrong?
Code that executes external program with parameters:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i          
//By using unterminal unit          try             ProcCon :=
TConsoleProc.Create(nil);           
 ProcCon.OnLineCompleted:=@Form1.procLineCompleted;           
 ProcCon.OnReadData:=@Form1.procReadData;           
 ProcCon.RunInLoop(vExeName,                                       
vParam1 + ' ' + vParam2  + ' ' + vParam3,                            
          -1, s);             Form1.Memo_Terminal.Append(s); //This
works, Memo is properly populated, but only after the execution is
finished....             Form1.Memo_Terminal.Refresh;         
finally             ProcCon.Free;          end; 
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i   TForm1 =
class(TForm)    ....    procedure procLineCompleted(const lin:
string);    procedure procReadData(nDat: integer; const lastLin:
string);    ....  private   public     LinPartial: boolean; //added
for purpose of UnTerminal  end;                               
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i procedure
TForm1.procLineCompleted(const lin: string);begin  if LinPartial then
begin    Memo_Terminal.Lines[Memo_Terminal.Lines.Count-1] := lin;   
LinPartial := false;  end else begin    Memo_Terminal.Lines.Add(lin); 
end;end; procedure TForm1.procReadData(nDat: integer; const lastLin:
string);begin  LinPartial := true; 
Memo_Terminal.Lines.Add(lastLin);end;     
Can you please check what am I doing wrong here?

----------------------------------------------------------------------
bytebites · 2021-03-16 18:15

How about this?
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i          
//By using unterminal unit          try             ProcCon :=
TConsoleProc.Create(nil);           
 ProcCon.OnLineCompleted:=@Form1.procLineCompleted;           
 ProcCon.OnReadData:=@Form1.procReadData;           
 ProcCon.RunInLoop(vExeName,                                       
vParam1 + ' ' + vParam2  + ' ' + vParam3,                            
          0);          finally             ProcCon.Free;          end;

----------------------------------------------------------------------
Edson · 2021-03-16 18:45

Quote from: tatamata on March 16, 2021, 05:21:22 pmHi, it might be,
but I have difficulties to properly understand your UnTerminal, you
haven't documented this case.
Anyway, I found some of your posts and try to follow it, but
apparently I am doing something wrong?
Code that executes external program with parameters:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i          
//By using unterminal unit          try             ProcCon :=
TConsoleProc.Create(nil);           
 ProcCon.OnLineCompleted:=@Form1.procLineCompleted;           
 ProcCon.OnReadData:=@Form1.procReadData;           
 ProcCon.RunInLoop(vExeName,                                       
vParam1 + ' ' + vParam2  + ' ' + vParam3,                            
          -1, s);             Form1.Memo_Terminal.Append(s); //This
works, Memo is properly populated, but only after the execution is
finished....             Form1.Memo_Terminal.Refresh;         
finally             ProcCon.Free;          end; 

You are mixing several ways to launch a process 

UnTerminal have different ways to get the output for a Process.
RunInLoop() is the most simple, but don't use events. It get all the
output after finishing.

To get the output line by line, just use the event
procOnLineCompleted:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i ...  proc
:= TConsoleProc.Create(nil);  //Create connection 
proc.OnLineCompleted:=@procOnLineCompleted;  proc.Open('cmd /c
dir','');... procedure TForm1.procOnLineCompleted(const lin:
string);begin  Memo1.Lines.Add(lin);end;
This is a simple way to get output. Maybe not the best. Anyway,
UnTerminal have several methods to capture the output.

I need to update the documentation.

----------------------------------------------------------------------
engkin · 2021-03-17 05:33

While using AsyncProcess unit:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i procedure
TForm1.AsyncProcess1ReadData(Sender: TObject);var  AsyncProcess:
TAsyncProcess absolute Sender;  s: string = '';  Len: Integer;begin 
Len := AsyncProcess.Output.NumBytesAvailable;  Setlength(s, Len);  Len
:= AsyncProcess.Output.Read(s[1],Len);  Memo1.Append(s);end;
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i procedure
TForm1.Test(Sender: TObject);var  AsyncProcess: TAsyncProcess;begin 
Memo1.WordWrap:=False;  Memo1.ScrollBars:=ssBoth;   AsyncProcess :=
TAsyncProcess.Create(Self);  AsyncProcess.OnReadData :=
@AsyncProcess1ReadData;  //AsyncProcess.OnTerminate:=
@AsyncProcess1Terminate;//<-- optional 
AsyncProcess.Options:=[poUsePipes,poNoConsole]; 
AsyncProcess.ParseCmdLine('cmd /C "dir /S c:\"'); 
AsyncProcess.Execute;end;
The highlighted lines are the needed lines, as the rest in this proc
can be handled by the form designer.

----------------------------------------------------------------------
tatamata · 2021-03-17 14:42

Hi engkin, thanks, it does work, however, process output is not
received into Memo line by line, but in big (too big) groups of lines.
Not really real-time emulation of the terminal output. I supposed it
was related  to AsyncProcess.PipeBufferSize property, so I
experimented with different values, but there is absolutely no
difference whether I set it to default 1024 or 16 or 16384, results
are always acquired in very big chunks of lines, not line by line.
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
AsyncProcess1.PipeBufferSize:=16;  //Default is 1024Do you know how to
control this behavior? Where is it defined?

----------------------------------------------------------------------
Jurassic Pork · 2021-03-17 15:31

hello,
Tatamata you ignore me but with TProcessEx launching this program :
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
biglines;uses sysutils;var myline : String;beginmyline :=
StringofChar('a',120);writeln(myline);sleep(1000);myline :=
StringofChar('b',120);writeln(myline);sleep(1000);myline :=
StringofChar('c',120);writeln(myline);sleep(1000);myline :=
StringofChar('d',120);writeln(myline);end.Result in Attachment  (click
to see animation)  


Friendly, J.P

----------------------------------------------------------------------
engkin · 2021-03-17 17:16

Hi J.P.

According to the source code1 of TProcessEx, it uses Sleep(10), and
ProcessMessages. While TAsyncProcess does not use Sleep, and it runs
in the application OnIdle. Because of that, I argue that TAsyncProcess
is faster, or at least the app is faster. What do you think?

Another problem, tatamata is asking for "real time" results.

@tatamata, I assume you realize how the OS gives you the perception of
all your apps are working by giving them small amount of time
frequently and fast enough that you don't notice the mostly serial
work. Probably you need to explain more of what you mean when you say
real time.

1- The source code is attached to this link.

----------------------------------------------------------------------
tatamata · 2021-03-17 17:26

By "realtime" I mean Memo showing lines appearing in similar manner as
while program is executing in command line, one line in a time. Here I
get huge chunk of lines at once and then another one huge chunk
several minutes later, etc. There is pretty big time gap between
receiving these chunks. This is a problem because user might conclude
that the process stopped working and might decide to run another async
process. This I must prevent.
So, beside controlling these chunks, another question related to
TAsyncProc:
- How do I stop, i.e. kill such async process?

----------------------------------------------------------------------
engkin · 2021-03-17 18:42

Several minutes is unacceptable, it shouldn't be more than a fraction
of a second. What OS are you on?

----------------------------------------------------------------------
tatamata · 2021-03-17 19:16

Quote from: engkin on March 17, 2021, 07:42:14 pmSeveral minutes is
unacceptable, it shouldn't be more than a fraction of a second. What
OS are you on?
It's on Windows server (hosted).
But it is not due to lack of resources (CPU and RAM are more than
enough).

----------------------------------------------------------------------
engkin · 2021-03-17 20:29

What about your application? the reason am asking is because I
remember seeing a limit of 10 entries being serviced per cycle, I have
to double check, but it seems *your* application is overwhelmed, to
need minutes to process the data. Also, how much data are we talking
about, how many lines per second?

----------------------------------------------------------------------
engkin · 2021-03-17 20:44

Can you test TAsyncProcess in a new application, and try it with the
test I posted above, to display the directories recursively? Does it
take minutes as well? if yes, then TMemo *might* be the problem, not
sure.

In any case, I am sure you will receive a number of lines, but it
should not take minutes, unless your app is overwhelmed, or TMemo has
a huge number of lines.

----------------------------------------------------------------------
tatamata · 2021-03-17 21:17

Quote from: engkin on March 17, 2021, 09:44:13 pmCan you test
TAsyncProcess in a new application, and try it with the test I posted
above, to display the directories recursively? Does it take minutes as
well? if yes, then TMemo *might* be the problem, not sure.

In any case, I am sure you will receive a number of lines, but it
should not take minutes, unless your app is overwhelmed, or TMemo has
a huge number of lines.
Indeed, your example (test) flies like a rocket!

My external exe is a python skript compiled with nuitka into exe...

----------------------------------------------------------------------
tatamata · 2021-03-17 21:23

Quote from: engkin on March 17, 2021, 09:29:29 pmWhat about your
application? the reason am asking is because I remember seeing a limit
of 10 entries being serviced per cycle, I have to double check, but it
seems *your* application is overwhelmed, to need minutes to process
the data. Also, how much data are we talking about, how many lines per
second?
There are not many lines at all, it's not about how many lines per
second, it's rather how many lines per minute....But they come in
chunks, not continuosly...

----------------------------------------------------------------------
Roland57 · 2021-03-17 21:41

@engkin

Hello! Your example works fine here (Linux).

Is there a reason why you declare the TAsyncProcess as a local
variable? To build your example, I did put a TAsyncProcess component
on the form, to be able to create automatically the OnRead method, and
after that I deleted it.

P.-S. I made this modification:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i  
Memo1.Append(Trim(s));

----------------------------------------------------------------------
engkin · 2021-03-17 21:58

Hi Roland,

No, not at all. There is no reason, I even highlighted the part that
is needed, the rest, as you said, can be handled using the form
designer.

----------------------------------------------------------------------
engkin · 2021-03-17 22:08

@tatamata, just to be sure, in a fresh app you tried the test and it
is fast, you tried your python app and it takes minutes? am I right?
both tested in a fresh app, right?

----------------------------------------------------------------------
tatamata · 2021-03-17 22:35

Quote from: engkin on March 17, 2021, 11:08:10 pm@tatamata, just to be
sure, in a fresh app you tried the test and it is fast, you tried your
python app and it takes minutes? am I right? both tested in a fresh
app, right?
Hm, both tested in the same Lazarus app on which I am working right
now....

----------------------------------------------------------------------
engkin · 2021-03-17 23:21

How to replicate the problem?

Is you Laz app 32 or 64bit?

What about your compiled Python app?

What Python script would show the same problem?

----------------------------------------------------------------------
Gustavo 'Gus' Carreno · 2021-03-18 00:04

Hey tatamata,

Can I butt in and ask you a question? It may seem out of nowhere, but
did you ever used fpcupdeluxe in the past month or even last year?

Why do I ask you? Well, fpcuplazarus deals with a ton of TProcess
calls and, even before DonAlfredo did it with a thread, it always felt
that it was rather quick.

I had a look at his code before he went on the TThread bandwagon
because I really loved his UI and wanted to replicate it for a
personal project and this is what I came up with:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
interfacetype{ TfrmMain }  TfrmMain = class(TForm)  {...}    seLog:
TSynEdit;  private    function RunWithOutput(const ACommand: String;
const AParams: array of String): Boolean;  {...} 
end; implementation function TfrmMain.RunWithOutput(const ACommand:
String; const AParams: array of String): Boolean;var  i, bytes:
integer;  s: String;  p: TProcess;  buff: array[1..2048] of
byte;begin  Result := False;  p := TProcess.Create(nil);  try   
p.Options := [poUsePipes];    for i := 1 to
GetEnvironmentVariableCount do    begin      s :=
GetEnvironmentString(i);      if Pos('path', LowerCase(s)) > 0 then  
  begin      {$IFDEF LINUX}        s := s +':.'      {$ENDIF}     
{$IFDEF WINDOWS}        s := s +';.'      {$ENDIF}      end;     
p.Environment.Add(s);    end;    p.Executable := ACommand;    for s in
AParams do    begin      p.Parameters.Add(s);    end;    p.Execute;  
repeat      bytes := p.Output.Read(buff, 2048);      s := EmptyStr;  
  for i := 1 to bytes do      begin        s := s + Char(buff[i]);   
 end;      seLog.Lines.AddText(s);      seLog.CaretY :=
seLog.Lines.Count;      Application.ProcessMessages;    until bytes =
0;    Result := p.ExitStatus = 0;  finally    p.Free;  end;end;
I'm sure I copied about 99% of his code and it works wonders on my
test thing. I guess I'll have to give him a ton of credit if I publish
it!! (Sorry DonAlfredo!!)

I'm also aware that I'm using a TSynEdit and not a TMemo, but from my
experience on Linux, the output string always comes with line endings
CRLF.

TMemo will respect the host architecture in regards to line endings,
e.g., Linux LF and Windows CRLF
But I've noticed that TSynEdit is always in CRLF mode for some reason.
That actually bit me on the butt on laz-JSON-Viewer when I implemented
the F12 feature.

I sincerely hope that this code can help. And I'm quite baffled that
DonAlfredo himself hasn't chimed in, odd.

I'm sorry if it's another code dump and then passing you the hot
potato, but I've been following the thread and, just a moment ago did
a bulb turned on and I got the DonAlfredo, fpcupdeluxe and my copy of
the code connection. You know how neurons firing go, pretty erratic

I sincerely hope this helps, but if it doesn't, no harm done, just
ignore me

Cheers,
Gus

----------------------------------------------------------------------
marcov · 2021-03-18 19:28

Most of the examples in these thread are for old FPC versions. (though
not yet using the onidle event)

I tried to quickly patch something together using old test code that I
made for the 3.2.0 FPC TProcess, moving as much code as possible to a
simple TProcess derivative.

For 3.2.0 TProcess I looked at various TProcess derivatives (Lazarus
TUF8Process which is now mostly obsolete, Jurassic Pork's unit and
fpcdeluxe.  The Runcommand code is related to the fpcdeluxe code to
start with)

The code plugs into events in the RunCommandLoop that implements 
largeoutput and polling.

The zip also contains a simple test program to run (testit.lpr) that
simply runs a loop and prints some strings with a sleep between it.

Mostly that is to test that the results don't come back in one big
batch at the end, but trickle in, and to allow testing that the GUI
remains snappy (I type text in the memo to test). Also it writes text
just before exiting, so you can verify the last bits are captured.

p.s. another thread about FPC 3.2.0+ tprocess is here implementing a
runcommand with a timeout

----------------------------------------------------------------------
engkin · 2021-03-19 10:17

@marcov, the core load goes 100% on my side. Is it expected?
I am using Laz 2.0.10, maybe a known issue? or did I miss with your
code.

----------------------------------------------------------------------
engkin · 2021-03-20 16:10

Quote from: engkin on March 19, 2021, 11:17:19 am@marcov, the core
load goes 100% on my side. Is it expected?

To answer my own question, yes expected. The code calls
RunCommandLoop:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i function
TProcessnamemacro.RunCommandLoop(out outputstring:string;......   
while Running do      begin...      end;
So we are stuck in this loop until the other application is done.
Meanwhile, the Event handler used here, TForm1.ProcessEvent, calls
application.ProcessMessages to keep the GUI responsive.

@Marcov, I don't think you are happy with this solution. I tried your
test project, testit, and it seems to be working normal with
TAsyncProcess. I do not see bursts of lines.

By the way, closing the form while testit is still running will not
close the form.

----------------------------------------------------------------------
marcov · 2021-03-20 16:54

Quote from: engkin on March 19, 2021, 11:17:19 am@marcov, the core
load goes 100% on my side. Is it expected?
I am using Laz 2.0.10, maybe a known issue? or did I miss with your
code.

Yeah, add a small sleep in the processevent method after the
application.processmessages.

In the code where I ported it from the sleep was still in
runcommandloop, but in 3.2.0 that is delegated to the handler for more
control/choice.

Exiting here is refused till the program finishes, but indeed that is
something to look into. Another problem to fix is if testit doesn't do
writeln() but only write() at the end.

I'm thinking about creating a similar skeleton that keeps the state
more in TProcess so that it can be used for onidle using code too.

The whole idea is to do away with umpteen variants that all have
slightly different variants of code that all have their own bugs and
put validated code into tprocess itself that can be plugged with
events.