TProcess does not work in Linux
lazarus / 32-Linux

----------------------------------------------------------------------
svd71 · 2023-04-24 17:50

Hi All,

I had make little example of the program, that does not work.
How somebody advice me show what I did wrong?

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
testProcess; {$mode objfpc}{$H+} uses  {$IFDEF UNIX}{$IFDEF
UseCThreads}  cthreads,  {$ENDIF}{$ENDIF}  Classes, SysUtils, CustApp,
process, FileUtil  { you can add units after this }; type   { TTest
}   TTest = class(TCustomApplication)  protected    procedure DoRun;
override;  public    ScriptProcess : TProcess;    procedure Execute; 
 constructor Create(TheOwner: TComponent); override;    destructor
Destroy; override;    end; { TTest } procedure TTest.Execute;var   
ras,cmd : String;    sl : TStringList;      i : Integer;begin  sl :=
TStringList.Create;    try    ScriptProcess := TProcess.Create(nil);//
   ScriptProcess.ShowWindow:= swoNone;     {$ifdef unix}    cmd :=
FindDefaultExecutablePath('ls');     {$endif}      ras := '--
help';       ScriptProcess.Executable:= cmd;   
ScriptProcess.Parameters.add('-c');   
ScriptProcess.Parameters.add(ras);      ScriptProcess.Options :=
[poNoConsole,         poUsePipes     ];     try     
ScriptProcess.Execute;    except on E: Exception do     
 Writeln('SE:E:' + E.Message);    end;    while ScriptProcess.Running
do    begin        if ScriptProcess.Output.NumBytesAvailable > 0
 then      begin        sl.LoadFromStream(ScriptProcess.Output);     
   Writeln(sl.Text);      end;        if
ScriptProcess.Stderr.NumBytesAvailable > 0  then      begin       
sl.LoadFromStream(ScriptProcess.Stderr);          Writeln(sl.Text);  
  end;       Sleep(100);    end;   finally    ScriptProcess.Free;   
sl.free;  end;  Writeln('');end; procedure TTest.DoRun;var  ErrorMsg:
String;begin  // quick check parameters  ErrorMsg:=CheckOptions('h',
'help');  if ErrorMsg<>'' then begin   
ShowException(Exception.Create(ErrorMsg));    Terminate;    Exit; 
end;    Execute;   // stop program loop  Terminate;end; constructor
TTest.Create(TheOwner: TComponent);begin  inherited
Create(TheOwner);end; destructor TTest.Destroy;begin  inherited
Destroy;end; var  Application: TTest;begin 
Application:=TTest.Create(nil);  Application.Title:='My Application'; 
Application.Run;  Application.Free;end.

----------------------------------------------------------------------
Fred vS · 2023-04-24 18:50

Hello.

With this it works here (XUbuntu 64 bit):

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i  procedure
TTest.Execute;   var    ras,cmd : String;    sl : TStringList;     i :
Integer;begin  sl := TStringList.Create;   try    ScriptProcess :=
TProcess.Create(nil);     {$ifdef unix}    cmd :=
FindDefaultExecutablePath('ls');    {$endif}    ras := '--help';     
   ScriptProcess.Executable:= cmd;   
ScriptProcess.Parameters.add('-c');   
ScriptProcess.Parameters.add(ras);         ScriptProcess.Options :=
[poNoConsole, poUsePipes];    ScriptProcess.Execute;   
sl.LoadFromStream(ScriptProcess.Output);    Writeln(sl.Text);        
  finally        ScriptProcess.Free;        sl.free;      end;     
Writeln('');    end;

----------------------------------------------------------------------
marcov · 2023-04-24 19:01

sl.LoadFromStream(ScriptProcess.Output)  doesn't get the number of
bytes saved, so it is probably not safe. Also calling it potentially
multiple times doesn't seem something that TStringlist.loadfromstream
supports. So badly designed code.

The base principle is to read _all_ bytes indicated by
numbytesavailable(*), and never read more.

----------------------------------------------------------------------
Fred vS · 2023-04-24 19:27

Quote from: marcov on April 24, 2023, 09:01:07
pmsl.LoadFromStream(ScriptProcess.Output)  doesn't get the number of
bytes saved, so it is probably not safe. Also calling it potentially
multiple times doesn't seem something that TStringlist.loadfromstream
supports. So badly designed code.

The base principle is to read _all_ bytes indicated by
numbytesavailable(*), and never read more.

I try to catch the goal of ScriptProcess.Output.NumBytesAvailable

With this:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i   ... 
ScriptProcess.Options := [poNoConsole, poUsePipes];     
ScriptProcess.Execute;   
writeln(scriptProcess.Output.NumBytesAvailable);   
sl.LoadFromStream(ScriptProcess.Output);   
writeln(scriptProcess.Output.NumBytesAvailable);
I get this:
Code: Bash  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i 00
But with this ( adding poWaitOnExit ):

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i   ... 
ScriptProcess.Options := [poNoConsole, poWaitOnExit, poUsePipes];    
ScriptProcess.Execute;   
writeln(scriptProcess.Output.NumBytesAvailable);   
sl.LoadFromStream(ScriptProcess.Output);   
writeln(scriptProcess.Output.NumBytesAvailable);
I get this:
Code: Bash  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i 77530
What are those 7753 bytes available for output after
ScriptProcess.Execute and why 0 after sl.LoadfromStream ?

----------------------------------------------------------------------
marcov · 2023-04-25 11:02

As said, sl.loadfromstream tries to read with 1024 bytes increments,
and thus will read beyond numbytes. Therefore this is nonsense code.

----------------------------------------------------------------------
Fred vS · 2023-04-25 11:54

Quote from: marcov on April 25, 2023, 01:02:38 pmAs said,
sl.loadfromstream tries to read with 1024 bytes increments, and thus
will read beyond numbytes. Therefore this is nonsense code.

Sorry but I still dont catch it (or maybe your answer is not related
to my question) 

Why this code with ScriptProcess.Options := [poNoConsole,
poWaitOnExit, poUsePipes];

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i   ScriptPro
cess.Execute; writeln(scriptProcess.Output.NumBytesAvailable);
gives 7753 as result ? What represent those 7753 bytes, is it a random
number?

----------------------------------------------------------------------
marcov · 2023-04-25 12:02

I've no idea, popipes and powaitonexit are not ideal

----------------------------------------------------------------------
Fred vS · 2023-04-25 12:09

Ha, ok, I never used Output.NumBytesAvailable, I just try to
understand his utility after post of OP.
When I have to get output from a TProcess, I use code like in Reply
#1, is it ok or do I miss something?

----------------------------------------------------------------------
gucao · 2023-04-25 13:23

some suggestions that might help:
1、increase ScriptProcess.PipeBufferSize.such as:
ScriptProcess.PipeBufferSize:=20480;
2、ScriptProcess.Output  ScriptProcess.Stderr Need to check whether it
has been Assigned. ScriptProcess.Stderr may not exist.
3、ScriptProcess.ExitCode ScriptProcess.ExitStatus These two values
​​may be helpful.

----------------------------------------------------------------------
Fred vS · 2023-04-25 13:55

Quote from: gucao on April 25, 2023, 03:23:03 pmsome suggestions that
might help:
1、increase ScriptProcess.PipeBufferSize.such as:
ScriptProcess.PipeBufferSize:=20480;
...

Thanks for th infos Gucao.

But there is still something that I dont catch.
If you dont set ScriptProcess.PipeBufferSize:=something, the default
is 1024 and all the sl.text is there.

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i    
ScriptProcess.Options := [poNoConsole, poUsePipes];   
ScriptProcess.Execute;    writeln(ScriptProcess.PipeBufferSize);    //
result is 1024    sl.LoadFromStream(ScriptProcess.Output);   
Writeln(sl.Text); // all is written
But if set ScriptProcess.PipeBufferSize:=16, with this little buffer
size, all sl.text is also there and with much more bytes than 16:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i    
ScriptProcess.Options := [poNoConsole, poUsePipes];   
ScriptProcess.Execute;    ScriptProcess.PipeBufferSize:=16;   // or
even 0 !    writeln(ScriptProcess.PipeBufferSize);    // result is 16 
  sl.LoadFromStream(ScriptProcess.Output);    Writeln(sl.Text); // all
is written
[EDIT] Even with ScriptProcess.PipeBufferSize:=0 all the text is
written.

So I dont understand what ScriptProcess.PipeBufferSize is for...

----------------------------------------------------------------------
marcov · 2023-04-25 18:52

Quote from: Fred vS on April 25, 2023, 02:09:39 pmHa, ok, I never used
Output.NumBytesAvailable, I just try to understand his utility after
post of OP.
When I have to get output from a TProcess, I use code like in Reply
#1, is it ok or do I miss something?

It is wrong. Basically it only works if the outputs fits in the OS'
pipe buffer (whatever that is) AND you use powaitonexit to make sure
it is actually all written to the buffer when you loadfromstream. In
other cases there might be output, but the stability of such schemes
is random.

In short: tstringlist.loadfromstream() usage in TProcess cases is a
code smell.

----------------------------------------------------------------------
Fred vS · 2023-04-25 19:07

Quote from: marcov on April 25, 2023, 08:52:08 pmQuote from: Fred vS
on April 25, 2023, 02:09:39 pmHa, ok, I never used
Output.NumBytesAvailable, I just try to understand his utility after
post of OP.
When I have to get output from a TProcess, I use code like in Reply
#1, is it ok or do I miss something?

It is wrong. Basically it only works if the outputs fits in the OS'
pipe buffer (whatever that is) AND you use powaitonexit to make sure
it is actually all written to the buffer when you loadfromstream. In
other cases there might be output, but the stability of such schemes
is random.

In short: tstringlist.loadfromstream() usage in TProcess cases is a
code smell.

Yep, we are going forward 
May I ask you what you would propose to use instead of
tstringlist.loadfromstream() ?

[EDIT] And the use of PipeBufferSize is, for me, still a mystery, even
using ScriptProcess.PipeBufferSize := 0 there seems to have no
change.   But it is not the question from the OP, so, once again,
sorry for the noise.

----------------------------------------------------------------------
Fred vS · 2023-04-25 20:00

Ok, ok, ok, from the wiki:
https://wiki.freepascal.org/Executing_External_Programs#TProcess

QuoteAn improved example (but not correct yet)

That's nice, but how do I read the Output of a program that I have
run?

Well, let's expand our example a little and do just that: This example
is kept simple so you can learn from it. Please don't use this example
in production code, but use the code in #Reading large output.

It is like my post and they say it is NOT CORRECT, in wiki example,
they use  :
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
AStringList.LoadFromStream(AProcess.Output);
The good way is to use OutputStream : TStream and after
TProcess.execute:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
 AProcess.Options := [poUsePipes];   // Start the process (run the
dir/ls command)  AProcess.Execute;  // Create a stream object to store
the generated output in. This could  // also be a file stream to
directly save the output to disk.  OutputStream :=
TMemoryStream.Create;   // All generated output from AProcess is read
in a loop until no more data is available  repeat    // Get the new
data from the process to a maximum of the buffer size that was
allocated.    // Note that all read(...) calls will block except for
the last one, which returns 0 (zero).    BytesRead :=
AProcess.Output.Read(Buffer, BUF_SIZE);     // Add the bytes that were
read to the stream for later usage    OutputStream.Write(Buffer,
BytesRead)   until BytesRead = 0;  // Stop if no more data is
available 
And after the process is finish, then you may use
sl.LoadFromStream(OutputStream).

OK, thanks Marcov and I hope OP get it too.

Fre;D

----------------------------------------------------------------------
MarkMLl · 2023-04-25 20:19

Quote from: Fred vS on April 24, 2023, 09:27:54 pmWhat are those 7753
bytes available for output after ScriptProcess.Execute and why 0 after
sl.LoadfromStream ?

I assume that you are referring there to your second and first
examples respectively.

Leaving aside Marco's point for the moment, I think you have to assume
that by default .Execute is asynchronous hence an adjacent
.LoadfromStream will behave unpredictably.

I'd hardly claim to be an expert at this, but looking back at code
which interfaced with a number of programs (using a number of
character encodings) I see this:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i function
FetchLineCodeProgram: word; begin  result := Lc_NoData;  if not
CheckConnectionAndPollProgram(false) then   
PseudoEventQueue.Enqueue($ffffffff)         (* Adapter signals line
loss    *)  else begin    if ProcessObject.Output.NumBytesAvailable >
0 then begin      ProcessObject.Output.Read(ch, 1);      result :=
Ord(ch)    end;    if ProcessObject.Stderr.NumBytesAvailable > 0 then
begin      ProcessObject.Stderr.Read(ch, 1);      programStderrBuffer
+= ch;... 
So I'm tempted to suggest that to do this entirely reliably we need to
need Dave Cutler's criticism of unix:

QuoteGet a byte, get a byte get a byte, byte, byte...

However if TProcess really isn't compatible with a StringList, I hope
the fact is properly documented somewhere.

MarkMLl

----------------------------------------------------------------------
Fred vS · 2023-04-25 20:30

Hello Mark.
Yes, sometime things are strange. 

For OP svd71, this code should make everybody happy ( I hope ):
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i    
procedure TTest.Execute;    const      BUF_SIZE = 2048; // Buffer size
for reading the output in chunks    var        ras,cmd : String;     
 sl : TStringList;        ScriptProcess : TProcess;       
OutputStream : TStream;        BytesRead    : longint;        Buffer 
    : array[1..BUF_SIZE] of byte;         begin            try       
ScriptProcess := TProcess.Create(nil);               {$ifdef unix}   
   cmd := 'ls';        {$endif}          ras := '--help';         
ScriptProcess.Executable:= cmd;   
ScriptProcess.Parameters.add('-l');   
ScriptProcess.Parameters.add(ras);         ScriptProcess.Options :=
[poNoConsole, poUsePipes];    ScriptProcess.Execute; // Create a
stream object to store the generated output in. This could  // also be
a file stream to directly save the output to disk.    OutputStream :=
TMemoryStream.Create;   // All generated output from AProcess is read
in a loop until no more data is available  repeat     // Get the new
data from the process to a maximum of the buffer size that was
allocated.    // Note that all read(...) calls will block except for
the last one, which returns 0 (zero).    BytesRead :=
ScriptProcess.Output.Read(Buffer, BUF_SIZE);     // Add the bytes that
were read to the stream for later usage    OutputStream.Write(Buffer,
BytesRead)   until BytesRead = 0;  // Stop if no more data is
available     // Now data can be shown on screen    sl :=
TStringList.Create;    OutputStream.Position := 0; // Required to make
sure all data is copied from the start   
sl.LoadFromStream(OutputStream);    writeln(sl.Text);   finally      
ScriptProcess.Free;        OutputStream.free;        sl.free;     
end;    end;

----------------------------------------------------------------------
marcov · 2023-04-25 20:46

Quote from: Fred vS on April 25, 2023, 10:00:16 pmOK, thanks Marcov
and I hope OP get it too.

To do it properly, you need to drain both output and stderr streams.
See e.g. TProcess.RunCommandLoop, the basis for Runcommand.

If you don't know what you are doing, use runcommand, get a string and
assign that to .text.

----------------------------------------------------------------------
Fred vS · 2023-04-25 21:10

Quote from: marcov on April 25, 2023, 10:46:20 pmQuote from: Fred vS
on April 25, 2023, 10:00:16 pmOK, thanks Marcov and I hope OP get it
too.

To do it properly, you need to drain both output and stderr streams.

Something like this: ?

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i    
procedure TTest.Execute;    const      BUF_SIZE = 2048; // Buffer size
for reading the output/errors in chunks    var        ras,cmd :
String;        sl, slError : TStringList;        OutputStream,
ErrorStream : TStream;        ScriptProcess : TProcess;       
BytesRead    : longint;        Buffer, BufferError  :
array[1..BUF_SIZE] of byte;         begin     try        ScriptProcess
:= TProcess.Create(nil);          {$ifdef unix}        cmd := 'ls';  
    {$endif}          ras := '--help';         
ScriptProcess.Executable:= cmd;   
ScriptProcess.Parameters.add('-l');   
ScriptProcess.Parameters.add(ras);     ScriptProcess.Options :=
[poNoConsole, poUsePipes]; // Create a stream object to store the
generated output in.    OutputStream := TMemoryStream.Create;// Create
a stream object to store the generated errors in.     ErrorStream :=
TMemoryStream.Create;     ScriptProcess.Execute; while
ScriptProcess.Running do  begin    // All generated output from
AProcess is read in a loop until no more data is available  repeat   
// Get the new data from the process to a maximum of the buffer size
that was allocated.    // Note that all read(...) calls will block
except for the last one, which returns 0 (zero).    BytesRead :=
ScriptProcess.Output.Read(Buffer, BUF_SIZE);     // Add the bytes that
were read to the stream for later usage    OutputStream.Write(Buffer,
BytesRead)   until BytesRead = 0;  // Stop if no more data is
available  repeat   // the same for errors    BytesRead :=
ScriptProcess.Stderr.Read(BufferError, BUF_SIZE);     // Add the bytes
that were read to the stream for later usage 
 ErrorStream.Write(BufferError, BytesRead)    until BytesRead = 0;  //
Stop if no more data is available   end;     // The output can be
shown on screen       sl := TStringList.Create;      
OutputStream.Position := 0; // Required to make sure all data is
copied from the start    sl.LoadFromStream(OutputStream);   
writeln(sl.Text);    // And the errors too:       slError :=
TStringList.Create;       ErrorStream.Position := 0; // Required to
make sure all data is copied from the start   
slError.LoadFromStream(ErrorStream);   
writeln(slError.Text);  finally        ScriptProcess.Free;       
sl.free;        slError.free;        ErrorStream.Free;       
OutputStream.Free;      end;      Writeln('');    end;

----------------------------------------------------------------------
Fred vS · 2023-04-25 21:29

Quote from: marcov on April 25, 2023, 10:46:20 pmQuote from: Fred vS
on April 25, 2023, 10:00:16 pmOK, thanks Marcov and I hope OP get it
too.
See e.g. TProcess.RunCommandLoop,

Ooops: https://www.freepascal.org/docs-
html/fcl/process/tprocess.runcommandloop.html
I did not know. 
Thanks for the tip!  (but now that I am a TProcess.Output/Stderr guru,
I dont need it anymore   )

----------------------------------------------------------------------
Fred vS · 2023-04-25 21:52

Quote from: marcov on April 25, 2023, 10:46:20 pmQuote from: Fred vS
on April 25, 2023, 10:00:16 pmOK, thanks Marcov and I hope OP get it
too.
... use runcommand, get a string and assign that to .text.

OK, that is, indeed, much more easy (but less fun).

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i procedure
TTest.Execute;  var       ScriptProcess : TProcess;     
 outputstring, stderrstring: string;       anexitstatus: integer; 
  begin    ScriptProcess := TProcess.Create(nil);       cmd := 'ls'; 
 ras := '--help';          ScriptProcess.Executable:= cmd;   
ScriptProcess.Parameters.add('-l');   
ScriptProcess.Parameters.add(ras);    ScriptProcess.Options :=
[poNoConsole, poUsePipes];    
ScriptProcess.RunCommandLoop(outputstring, stderrstring,
anexitstatus);          // The output    writeln(outputstring);      
// The errors    writeln(stderrstring);       // The exit status   
writeln(anexitstatus);     ScriptProcess.free;    end;

----------------------------------------------------------------------
gucao · 2023-04-26 00:50

Quote from: Fred vS on April 25, 2023, 09:07:14 pm[EDIT] And the use
of PipeBufferSize is, for me, still a mystery, even using
ScriptProcess.PipeBufferSize := 0 there seems to have no change.   But
it is not the question from the OP, so, once again, sorry for the
noise.
When use poWaitOnExit,If the output of the pipeline is greater than
PipeBufferSize,it will be blocking,program will become unresponsive.
This is my little experience of using TProcess on win.
Can try TAsyncProcess,It is in the AsyncProcess unit.

----------------------------------------------------------------------
SymbolicFrank · 2023-04-26 07:32

If they run in different threads, you don't know if the amount of
bytes waiting is accurate. Do they write the data first, or the
amount? Perhaps the last thing they do is write the pointer to the
buffer. In that case, data might be reported as available, while it
isn't.

Ok, generally this will be handled by OS functions, so you could look
at the documentation to find out what guarantees are given. But it's
safest to assume that sometimes it's broken. Because, it can work now,
but the behavior can change in a future release.

----------------------------------------------------------------------
MarkMLl · 2023-04-26 07:50

Quote from: SymbolicFrank on April 26, 2023, 09:32:54 amIf they run in
different threads, you don't know if the amount of bytes waiting is
accurate. Do they write the data first, or the amount? Perhaps the
last thing they do is write the pointer to the buffer. In that case,
data might be reported as available, while it isn't.

Ok, generally this will be handled by OS functions, so you could look
at the documentation to find out what guarantees are given. But it's
safest to assume that sometimes it's broken. Because, it can work now,
but the behavior can change in a future release.

It's not just different threads: it is, by definition, different
/processes/ with different memory contexts. It's safe to assume that
the way that the processes are scheduled will vary depending on the
OS, since the OS designers will have their own ideas as to how best to
move or copy the data between the contexts. In fact on some OSes- MS-
DOS being a specific example- the process generating the data might
run to completion with its output being stored in a file: THIS IS NOT
HOW THINGS ARE DONE ON UNIX where results are piped as they're
generated (with some amount of buffering and granularity).

MarkMLl

----------------------------------------------------------------------
SymbolicFrank · 2023-04-26 10:53

It doesn't matter, processes or threads, they get suspended at
intervals. You never know when the pipe-buffers are updated, or if the
amount of data available you just read is still valid. Both can have
changed already.

----------------------------------------------------------------------
marcov · 2023-04-26 11:51

Quote from: gucao on April 26, 2023, 02:50:01 amWhen use
poWaitOnExit,If the output of the pipeline is greater than
PipeBufferSize,it will be blocking,program will become unresponsive.

That is basically the wait for large output from the wiki scenario

----------------------------------------------------------------------
marcov · 2023-04-26 11:53

Quote from: Fred vS on April 25, 2023, 11:10:54 pmSomething like this:
?

Compare with runcommandloop. You probably need some loop around the
both repeats.

Also look at readinputstream https://www.freepascal.org/docs-
html/fcl/process/tprocess.readinputstream.html

----------------------------------------------------------------------
Fred vS · 2023-04-26 12:21

Quote from: marcov on April 26, 2023, 01:53:57 pmQuote from: Fred vS
on April 25, 2023, 11:10:54 pmSomething like this: ?
Compare with runcommandloop.

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i function
TProcessnamemacro.RunCommandLoop(out outputstring:string;            
              out stderrstring:string; out
anexitstatus:integer):integer;var    bytesread : integer;   
outputlength, stderrlength : integer;    stderrbytesread : integer;  
gotoutput,gotoutputstderr : boolean;begin  result:=-1;    try   
Options := Options + [poUsePipes];    bytesread:=0;   
outputlength:=0;    stderrbytesread:=0;    stderrlength:=0;   
Execute;    while Running do      begin        // Only call
ReadFromStream if Data from corresponding stream        // is already
available, otherwise, on  linux, the read call        // is blocking,
and thus it is not possible to be sure to handle        // big data
amounts bboth on output and stderr pipes. PM.        gotoutput:=ReadIn
putStream(output,BytesRead,OutputLength,OutputString,1);        // The
check for assigned(P.stderr) is mainly here so that        // if we
use poStderrToOutput in p.Options, we do not access invalid memory.  
    gotoutputstderr:=false;        if assigned(stderr) then          
gotoutputstderr:=ReadInputStream(StdErr,StdErrBytesRead,StdErrLength,S
tdErrString,1);         if (porunidle in options) and not gotoutput
and not gotoutputstderr and Assigned(FOnRunCommandEvent) Then        
FOnRunCommandEvent(self,Nil,RunCommandIdle,'');      end;    // Get
left output after end of execution   
ReadInputStream(output,BytesRead,OutputLength,OutputString,250);   
setlength(outputstring,BytesRead);    if assigned(stderr) then      Re
adInputStream(StdErr,StdErrBytesRead,StdErrLength,StdErrString,250); 
 setlength(stderrstring,StderrBytesRead);   
anexitstatus:=exitstatus;    result:=0; // we came to here, document
that.    if Assigned(FOnRunCommandEvent) then          // allow
external apps to react to that and finish GUI     
FOnRunCommandEvent(self,Nil,RunCommandFinished,'');     except      on
e : Exception do         begin           result:=1;         
 setlength(outputstring,BytesRead);         
 setlength(stderrstring,StderrBytesRead);           if
Assigned(FOnRunCommandEvent) then           
 FOnRunCommandEvent(self,Nil,RunCommandException,e.Message);       
 end;     end;end; 
OK, you win, much easier to use the out-of-the-box RunCommandLoop, he
does all the dirty work.


Quote from: marcov on April 26, 2023, 01:53:57 pmQuote from: Fred vS
on April 25, 2023, 11:10:54 pmSomething like this: ?
You probably need some loop around the both repeats.

Not sure to understand, do you mean this from RunCommandLoop?

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
...Execute;    while Running do   // This ?      begin...

----------------------------------------------------------------------
marcov · 2023-04-26 13:19

yes

----------------------------------------------------------------------
Fred vS · 2023-04-26 13:21

Quote from: marcov on April 26, 2023, 03:19:28 pmyes

Thanks for your science and patience Marco.

----------------------------------------------------------------------
PascalDragon · 2023-04-26 13:45

Quote from: SymbolicFrank on April 26, 2023, 12:53:23 pmIt doesn't
matter, processes or threads, they get suspended at intervals. You
never know when the pipe-buffers are updated, or if the amount of data
available you just read is still valid. Both can have changed already.

While one should always protect against errors from - in this case -
read functions on pipes, if the OS tells you that there are X Byte
available in a pipe you can generally assume that the read function
will indeed be able to return X Byte before blocking (and assuming you
use blocking reads it doesn't matter anyway, cause the time difference
between the OS noting the amount of data available in the pipe and
providing said data in the pipe's buffer would in nearly all cases not
be relevant).

In general pipes do not count as an API that needs to be handled in a
thread safe manner despite different threads and even processes being
involved, because the OS will ensure correct behavior here (by
synchronizing access to the pipe itself).