Piping Console Output into GUI Application
lazarus / 32-Linux

----------------------------------------------------------------------
ThomasK · 2021-06-09 09:36

I need some inspiration...

Is it possibe to pipe the console output of a GUI application into
this application itself?

I attach the GUI itself and the Console Input/Output of the debugger.

The question is now, can I somehow get the Console Output istelf to
show it to the user without debugger?


Thanks Thomas

----------------------------------------------------------------------
marcov · 2021-06-09 09:44

Close(output);
stdoutstream:=TMemoryStream.Create;
streamio.assignstream(output,stdoutstream);

Now using a timer, poll stdoutstream to see if something is changed,
and append it to the memo.

----------------------------------------------------------------------
MarkMLl · 2021-06-09 10:26

Quote from: ThomasK on June 09, 2021, 11:36:53 amIs it possibe to pipe
the console output of a GUI application into this application itself?

Tell us more about what /exactly/ you are trying to do, what OS you're
running etc.

MarkMLl

----------------------------------------------------------------------
ThomasK · 2021-06-09 10:50

I am posting in the Linux Part of the forum-> os is Linux, or better
Raspbian.
I run a GUI application, when calls receive a -1 as return, error
messages of the called underlying os processes are posted on the
console.
I can see that the Lazarus debugging session in the console window of
the debugger.
I want to be able to show those messages in my application.
Without debugger.

----------------------------------------------------------------------
MarkMLl · 2021-06-09 11:43

Quote from: ThomasK on June 09, 2021, 12:50:02 pmI am posting in the
Linux Part of the forum->

Your message appeared in the "Recent" pane, and I've got more pressing
things to do than check its provenance.

Like helping.

Quote os is Linux, or better Raspbian.
I run a GUI application, when calls receive a -1 as return, error
messages of the called underlying os processes are posted on the
console.
I can see that the Lazarus debugging session in the console window of
the debugger.
I want to be able to show those messages in my application.
Without debugger.

I presume that when you say "console" you mean a shell session, rather
than the physical serial console or the kernel log.

The first thing you need to check is whether the message is going to
stdout or stderr.

I've definitely been able to run a shell session as an external
process and capture its output to go into a terminal emulator session
which existed as a pane on a program's form. This was under Debian, I
think it was working with both the gtk2 and Qt widget sets but have no
immediate reason to think that it was a problem.

I'd have used TProcess for that, I suggest reading up on that and/or
experimenting, then if you need more help I'll try to dig out my code
as an example.

MarkMLl

----------------------------------------------------------------------
engkin · 2021-06-09 11:51

Why, what's wrong with the code posted above by macrov in the *first*
reply?

----------------------------------------------------------------------
marcov · 2021-06-09 12:41

My solution will only trap Pascal output.

If he means gtk errors etc, I don't know.

----------------------------------------------------------------------
MarkMLl · 2021-06-09 12:57

Quote from: marcov on June 09, 2021, 02:41:57 pmMy solution will only
trap Pascal output.

If he means gtk errors etc, I don't know.

Apologies if it looked like I was intentionally treading on your toes.

I've got working code that will run a shell, I think it also ran
arbitrary programs without an intervening shell, and I put various
things in there that allowed me to run a nested copy of Lazarus hence
gdb to debug child programs... so at that point I'm fairly confident
it will catch just about anything.

As a slight aside: Marco, am I correct in that the Lazarus IDE doesn't
have an interactive shell-out facility?

MarkMLl

----------------------------------------------------------------------
ThomasK · 2021-06-09 14:23

Quote from: marcov on June 09, 2021, 02:41:57 pmMy solution will only
trap Pascal output.

If he means gtk errors etc, I don't know.

Maybe to make it more clear, if I start the application in a console
"./libsockuse" or "sudo ./libsockuse" (this is at least required if I
want to use it with root rights to set CAN configuration), it will
show the (error) messages in that shell window. This is what I want to
catch and show it in the application itself.

This is not necessarily Pascal output, as you can see in the screen
shot.

----------------------------------------------------------------------
Gustavo 'Gus' Carreno · 2021-06-09 15:24

Hey ThomasK,

I hope what I'm about to suggest make sense or is even
correct/possible.

What Marco V. suggested is good to capture anything that comes out of
the Pascal side the app. I'm not sure it will capture the GTK2 side of
things...

But if one starts the app with both STDOUT and STDERR piped to a file,
then we can use the timer technique that Marco V. suggested but
directed at the file at the end of that pipe:
Code: Bash  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i $
./MyGTK2App 1>&2 > output.txt
One then just has to check if the file exists and then update the
TMemo with it's contents.

All the time crossing our fingers that the buffer for that file is
flushed with enough regularity that something shows before the closing
of the app.

OR I'm completely misunderstanding the way pipes work and I just
should shut up and crawl back under my cozy little rock

Cheers,
Gus

----------------------------------------------------------------------
MarkMLl · 2021-06-09 15:25

Quote from: ThomasK on June 09, 2021, 04:23:16 pmMaybe to make it more
clear, if I start the application in a console "./libsockuse" or "sudo
./libsockuse" (this is at least required if I want to use it with root
rights to set CAN configuration), it will show the (error) messages in
that shell window. This is what I want to catch and show it in the
application itself.

Right, so you're running from a shell: standard TProcess stuff IIRC.
There's other possibilities e.g. using a pty, if you're stuck I'll
look for examples later.

You don't need to use root, you can use POSIX capabilities although
you might still need gdbserver for debugging.

MarkMLl

----------------------------------------------------------------------
Gustavo 'Gus' Carreno · 2021-06-09 15:37

Hey MarkMLI,

I'm not sure the he wants the output of a third application.

If I've read him well, he wants to show the output of the application
ITSELF, not of another one.

He mentions A debugger just for the fact that he can see the output of
the debugged app in some window of the debugger.
I think that's where is interest on the debugger ends.

But again, I may be completely out of order and just muddying the
waters a bit more. If so, I deeply apologise!!

Cheers,
Gus

----------------------------------------------------------------------
ThomasK · 2021-06-09 16:14

Hi Mark,

Quote from: MarkMLl on June 09, 2021, 05:25:55 pmRight, so you're
running from a shell: standard TProcess stuff IIRC. There's other
possibilities e.g. using a pty, if you're stuck I'll look for examples
later.

No I am not running from a shell, this was just for the purpose to
test all possible calls in the application. I use TProcess shell calls
as sudo for the functions requiring root rights, here I get the stdout
&stderr.

This because these functions are only used for setting up baudrate,
start & stop the interface.

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i begin   
OutputLines := TStringList.Create;    AProcess    :=
TProcess.Create(nil);    AProcess.Executable := '/bin/sh';   
AProcess.Parameters.Add('-c');    AProcess.Parameters.Add(cmds2exec); 
  AProcess.Options := AProcess.Options + [poWaitOnExit, poUsePipes]; 
 AProcess.Execute;    OutputLines.LoadFromStream(Aprocess.Output);   
if length(Outputlines.text)>0 then     
FCANTest.mlog.lines.addstrings(Outputlines.text);   
OutputLines.LoadFromStream(AProcess.Stderr);    if
length(Outputlines.text)>0 then     
FCANTest.mlog.lines.addstrings(Outputlines.text);    AProcess.Free;  
OutputLines.Free;end;   As also decribed in https://forum.lazarus.free
pascal.org/index.php/topic,54783.msg408024.html#msg4080

As Gus mentions, I want the outputs of processes which show up in the
console if I would start from the console.

They are also shown to me in the Debug window if I start the
application in Lazarus with debugging. (screenshot in the initial
post)

Maybe due to the fact, that Lazarus is the caller of the application
it can catch the console output and show it in the debug console input
/ output window.

thx.

Thomas

----------------------------------------------------------------------
avra · 2021-06-09 16:28

Quote from: ThomasK on June 09, 2021, 06:14:18 pmI want the outputs of
processes which show up in the console if I would start from the
console.Take a look at RunProcess() at
https://bitbucket.org/avra/ct2laz/src/master/utils.pas. It runs
external process and captures it's output to a memo on both linux and
windows.

----------------------------------------------------------------------
marcov · 2021-06-09 20:50

Quote from: MarkMLl on June 09, 2021, 02:57:20 pmAs a slight aside:
Marco, am I correct in that the Lazarus IDE doesn't have an
interactive shell-out facility?

Afaik indeed it doesn't. It was designed for systems that could run
more tasks in parallel, so there was no need for functionality to
suspend the IDE to run a shell like old DOS IDEs did.

----------------------------------------------------------------------
marcov · 2021-06-09 20:54

Quote from: ThomasK on June 09, 2021, 04:23:16 pm Quote from: marcov
on June 09, 2021, 02:41:57 pmMy solution will only trap Pascal output.

If he means gtk errors etc, I don't know.

Maybe to make it more clear, if I start the application in a console
"./libsockuse" or "sudo ./libsockuse" (this is at least required if I
want to use it with root rights to set CAN configuration), it will
show the (error) messages in that shell window. This is what I want to
catch and show it in the application itself.

Those might not even be application errors, but kernel (module) errors
redirected to the console.

Maybe directly interacting with sysconsole pty or so?

Anyway I suggest to rethink that. It sounds simple, but it probably
isn't. (and by that I mean that it might be hard to do that from _any_
process, not just a FPC compiled one)

----------------------------------------------------------------------
Fred vS · 2021-06-09 21:18

Hello.

Thanks to Warfley (sorry I did not re-find the post), this will
redirect all message-console, including gtk errors etc, into a text
file.

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
errorlog;  uses   ... ,  Classes, BaseUnix;   var     fs:
TFileStream;   begin     fs := TFileStream.Create('error.log',
fmOpenReadWrite or fmCreate);     FpDup2(fs.Handle, StdErrorHandle); 
    ...  // all your program stuff     Application.run; // Inside
application, catch fs data     fs.free;    end.
Maybe you could catch the 'error.log' data in the program himself
(close fs, then read error.log).

----------------------------------------------------------------------
engkin · 2021-06-10 00:34

Yes, I agree with Fred. I found a SO post that solved this problem
using Dup2 and pipes. This way you don't need a "file". I tested the
idea on an Android terminal. Seemed to work:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
Project1; {$mode objfpc}{$H+} uses  Classes, SysUtils, BaseUnix 
{$IFDEF UNIX}  {$IFDEF UseCThreads}  ,cthreads  {$ENDIF}  {$ENDIF}  {
you can add units after this
};{https://stackoverflow.com/questions/955962/how-to-buffer-stdout-in-
memory-and-write-it-from-a-dedicated-thread}function
Test(AHandle:cInt; var APipe: tfildes):cint;var  saved: cint;begin 
Result:=-1;  saved:=FpDup(AHandle);  if FpPipe(APipe)<>0 then  begin 
 WriteLn('FpPipe failed.');    exit;  end;  FpDup2(APipe[1],
AHandle);  FpClose(APipe[1]);  Result:=saved;end;  const 
MAX_LEN=40;type  TBuffer=array[0..MAX_LEN-1] of char;var  Pipe:
tfildes=(0,0);  saved_, TestHandle: cint;  buffer:TBuffer;begin 
buffer:=Default(TBuffer);   TestHandle:=StdOutputHandle; 
//TestHandle:=Test(StdErrorHandle);   WriteLn('Test Before'); 
saved_:=Test(TestHandle, Pipe);  WriteLn('Test After'); //<---- this
will be sent to the pipe  Flush(StdOut);   fpread(Pipe[0], buffer[0],
MAX_LEN); // read from pipe into buffer   FpDup2(saved_,
StdOutputHandle);  // reconnect stdout for testing  
WriteLn(Format('read: %s', [buffer]));  readLn;end.
To test on Android:
Code: Text  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i adb push
project1 "/data/local/tmp"adb shell chmod 755
/data/local/tmp/project1adb shell /data/local/tmp/project1
The output:
QuoteTest Before
read: Test After
On Linux, StdErrorHandle is hard coded:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i const 
UnusedHandle    = -1;  StdInputHandle  = 0;  StdOutputHandle = 1; 
StdErrorHandle  = 2;
I believe it should work with kernel errors as well.

----------------------------------------------------------------------
MarkMLl · 2021-06-10 06:52

Quote from: marcov on June 09, 2021, 10:50:39 pmQuote from: MarkMLl on
June 09, 2021, 02:57:20 pmAs a slight aside: Marco, am I correct in
that the Lazarus IDE doesn't have an interactive shell-out facility?

Afaik indeed it doesn't. It was designed for systems that could run
more tasks in parallel, so there was no need for functionality to
suspend the IDE to run a shell like old DOS IDEs did.

Thanks for that, just checking I've not overlooked anything. I've got
a vague recollection of a couple of cases where it would have been
useful although I can't remember the exact details... and of course
there's the "shell-like" facilities of "change current directory" and
"set shell/environment variable".

MarkMLl

----------------------------------------------------------------------
Edson · 2021-06-10 18:10

Maybe I don't understand well the problem but in case you need to
capture the output or error of a process , you can try my library:
https://github.com/t-edson/UnTerminal

If you need to emulate a terminal, to control special inputs like
"sudo" needs, I wrote an article about that:
http://blogdetito.com/2016/12/04/el-inicio-de-un-terminal-con-linux-y-
free-pascal/

----------------------------------------------------------------------
ThomasK · 2021-06-25 17:49

Quote from: engkin on June 10, 2021, 02:34:38 amYes, I agree with
Fred. I found a SO post that solved this problem using Dup2 and pipes.
This way you don't need a "file". I tested the idea on an Android
terminal. Seemed to work:

I modified the code a little bit:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
test_pipe; {$mode objfpc}{$H+} {$Define std} uses  Classes, SysUtils,
BaseUnix  {$IFDEF UNIX}  {$IFDEF UseCThreads}  ,cthreads  {$ENDIF} 
{$ENDIF}  { you can add units after this
};{https://stackoverflow.com/questions/955962/how-to-buffer-stdout-in-
memory-and-write-it-from-a-dedicated-thread}function
Test(AHandle:cInt; var APipe: tfildes):cint;var  saved: cint;begin 
Result:=-1;  saved:=FpDup(AHandle);  if FpPipe(APipe)<>0 then  begin 
 WriteLn('FpPipe failed.');    exit;  end;  FpDup2(APipe[1],
AHandle);  FpClose(APipe[1]);  Result:=saved;end;  const 
MAX_LEN=40;type  TBuffer=array[0..MAX_LEN-1] of char;var  Pipe:
tfildes=(0,0);  saved_, TestHandle: cint;  buffer:TBuffer;begin 
buffer:=Default(TBuffer);{$ifdef std} 
TestHandle:=StdOutputHandle;{$else} 
TestHandle:=StdErrorHandle;{$endif}  WriteLn('Test Before'); 
saved_:=Test(TestHandle, Pipe);  WriteLn('Test After'); //<---- this
will be sent to the pipe{$ifdef std}  Flush(StdOut);{$else} 
Flush(StdErr);{$endif}   fpread(Pipe[0], buffer[0], MAX_LEN); // read
from pipe into buffer{$ifdef std}  FpDup2(saved_, StdOutputHandle);
 // reconnect stdout for testing{$else}  FpDup2(saved_,
StdErrorHandle);  // reconnect stderr for testing{$endif}  
WriteLn(Format('read from pipe: %s', [buffer]));  readLn;end. 
when the compiler switch is set for 'std' it works.

Code: Bash  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
pi@RasPiHW:~/FreePascal/Projekte/Konsolenapps $ ./test_pipeTest
Beforeread from pipe: Test After Finished as pogrammed with <Enter>

But when stderr should be used, the program stucks.
Code: Bash  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
pi@RasPiHW:~/FreePascal/Projekte/Konsolenapps $ ./test_pipeTest
BeforeTest After   ^Cpi@RasPiHW:~/FreePascal/Projekte/Konsolenapps $
 It has to be terminated with Ctrl-c.

Any ideas?

----------------------------------------------------------------------
engkin · 2021-06-25 19:09

It is waiting for you to hit enter (or somehow pass it if you use
Android through ADB). The code for testing is more like:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
Project1; {$mode objfpc}{$H+} uses  Classes, SysUtils, BaseUnix 
{$IFDEF UNIX}  {$IFDEF UseCThreads}  ,cthreads  {$ENDIF}  {$ENDIF}  {
you can add units after this
};{https://stackoverflow.com/questions/955962/how-to-buffer-stdout-in-
memory-and-write-it-from-a-dedicated-thread}function
Test(AHandle:cInt; var APipe: tfildes):cint;var  saved: cint;begin 
Result:=-1;  saved:=FpDup(AHandle);  if FpPipe(APipe)<>0 then  begin 
 WriteLn('FpPipe failed.');    exit;  end;  FpDup2(APipe[1],
AHandle);  FpClose(APipe[1]);  Result:=saved;end; procedure
Test_StdText(var f:Text);const  MAX_LEN=40;type 
TBuffer=array[0..MAX_LEN-1] of char;var  Pipe: tfildes=(0,0);  saved_,
TestHandle: cint;  buffer:TBuffer;begin  buffer:=Default(TBuffer); 
TestHandle:=TextRec(f).Handle;   WriteLn(f,'Test Before'); 
saved_:=Test(TestHandle, Pipe);  WriteLn(f,'Test After'); //<---- this
will be sent to the pipe  Flush(f);   fpread(Pipe[0], buffer[0],
MAX_LEN); // read from pipe into buffer   FpDup2(saved_,
TextRec(f).Handle);  // reconnect stdout for testing  
WriteLn(Format('read: %s', [buffer]));  //readLn;end; begin 
WriteLn('StdOut:');  Test_StdText(StdOut);   WriteLn('StdErr:'); 
Test_StdText(StdErr);end.
Pressing Ctrl+C is ending the local process.