Close/reopen text file lazarus / 66-Unix ---------------------------------------------------------------------- MarkMLl · 2022-01-09 12:46 Is it possible to close and then reopen the "Output" device on Linux? The situation is that I've got potential output to the unnamed "Output" device, to a named file, to a unix-domain pipe or to a FIFO (named pipe). The output format does not, as standard, have any form of EOF marker so any cooperating programs have to use e.g. the inotify (FAM) API to monitor the writer's state. If the writer program is looping, e.g. to process successive runs of an instrument capture, then it can close output routed to any of the named file types and then reopen it immediately before starting another run. For the purpose of orthogonality, I would like to be able to handle "Output" in the same way. I appreciate that if this is being redirected or piped (by the shell) that the behaviour might be undefined, but for the moment I'm focussing on the writer and the relationship between "Output" and the "stdout" handle: any signals or error states seen by the reader is for another day. MarkMLl ---------------------------------------------------------------------- Thaddy · 2022-01-09 13:38 Quote from: MarkMLl on January 09, 2022, 01:46:10 pm but for the moment I'm focussing on the writer and the relationship between "Output" and the "stdout" handle: any signals or error states seen by the reader is for another day. MarkMLl FYI stdout is simply an alias for output. They are interchangeble. https://www.freepascal.org/docs-html/rtl/system/stdout.html So relation part solved  As for eof, dunno. Afaik that is indeed not standard for output/stdout. ---------------------------------------------------------------------- MarkMLl · 2022-01-09 13:47 Quote from: Thaddy on January 09, 2022, 02:38:20 pmFYI stdout is simply an alias for output. https://www.freepascal.org/docs-html/rtl/system/stdout.html So relation part solved  As for eof, dunno. Afaik that is indeed not standard. No, stdout is the unix handle with the default value 1. I anticipate that closing/reopening "Output" hence stdout will break something if e.g. it's being piped to another program. I'm not all that bothered, "somebody else's problem". MarkMLl ---------------------------------------------------------------------- Kays · 2022-01-09 14:00 Quote from: MarkMLl on January 09, 2022, 01:46:10 pmIs it possible to close and then reopen the "Output" device on Linux?No, not the way you want it: Once /dev/stdout has been closed it is not connected to the respective terminal [or file myProgram > outfile] anymore. You cannot, at least not trivially, regain access to user’s console (or the file specified on the command line upon program invocation).Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i uses      baseUnix;var        FD: text;begin        fpClose(stdOutputHandle);      assign(FD, '/dev/stdout');        rewrite(FD); { causes RTE 2 "File not found" }end. ---------------------------------------------------------------------- MarkMLl · 2022-01-09 14:14 Quote from: Kays on January 09, 2022, 03:00:01 pmNo, not the way you want it: Once /dev/stdout has been closed it is not connected to the respective terminal [or file myProgram > outfile] anymore. You cannot, at least not trivially, regain access to user’s console (or the file specified on the command line upon program invocation). In practice, I think that's exactly what I want: I'd forgotten that special device. I anticipate that either the inotify API, a signal or a runtime error condition (EPIPE?) will alert the reader to the fact that the writer's closed its end. If in practice the reader can't recover... that's not the writer's problem, the user should have set things up to use a unix-domain socket or a FIFO. It might be possible for the reader to examine the writer's /proc/something/fd/1 and to reconnect, but again that's not the writer's problem. MarkMLl ---------------------------------------------------------------------- Thaddy · 2022-01-09 16:07 Quote from: MarkMLl on January 09, 2022, 02:47:43 pmNo, stdout is the unix handle with the default value 1. Am I right that you suggest that the official documentation is incorrect?  You can also reverse it: Output is really an alias for stdout. Clear? If you really feel bad about it, file a bug against documentation. The point is that from the language point of view the documentation is correct, but from the platform point of view it is the other way around. I would stick to the language. You will find that the handle - on unix flavors - is indeed 1 as expected in most cases. ---------------------------------------------------------------------- MarkMLl · 2022-01-09 16:37 Quote from: Thaddy on January 09, 2022, 05:07:15 pmIf you really feel bad about it, file a bug against documentation. If /you/ really feel bad about it, file a bug against unix. MarkMLl ---------------------------------------------------------------------- PascalDragon · 2022-01-10 13:17 You two are talking about two different things: Thaddy says that StdOut is an alias of Output which in the context of FPC is definitely correct. MarkMLI says that StdOut is an alias for the Unix handle with value 1, which is also correct, but in the context of Unix. In the context of FPC that is named StdOutputHandle. ---------------------------------------------------------------------- MarkMLl · 2022-01-10 13:40 I know that, and I suspect he knows that as well :-) And I did carefully say at the start 'relationship between "Output" and the "stdout" handle'... and since this is the unix topic I'd have thought that the reference to "handle" needed no further expansion. This does of course xref to my slightly older https://forum.lazarus.freepascal.org/index.php/topic,57706.0.html , and when considering closing/reopening it looks as though the preferred behaviour should be slightly different depending on whether output is to a regular file (where reopening will truncate) vs anything else. However all I'm trying to do at present is rough out "reasonable" behaviour for the various types... and as discussed I anticipate that doing this to a named pipe will have undesirable consequences (which I'm documenting but not prohibiting). MarkMLl ---------------------------------------------------------------------- Kays · 2022-01-10 16:50 Quote from: MarkMLl on January 09, 2022, 03:14:08 pm[…] I anticipate that either the inotify API,Inotify does not work (reliably) on pseudo file systems such as procfs(5) or devfs/devtmpfs. Quote from: MarkMLl on January 09, 2022, 03:14:08 pma signal or a runtime error condition (EPIPE?) will alert the reader to the fact that the writer's closed its end.Files are only really deleted if they are not opened (confer output of lsof(8)) and their (hardlink) reference count hits zero. Similarly, as long as someone somewhere (e.g. another process) holds a valid FD to stdout, this stdout is not closed/“deleted”.Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i uses        baseUnix;var        FD: text;begin        assign(FD, '/dev/stdout');        rewrite(FD);        fpClose(stdOutputHandle);        writeLn(FD, 'Hi!'); { still works as expected }        writeLn('Bye!'); { causes RTE 101 }end.On FreeBSD you could use revoke(2) to really close /dev/stdout for good. ---------------------------------------------------------------------- MarkMLl · 2022-01-10 20:46 Quote from: Kays on January 10, 2022, 05:50:49 pmInotify does not work (reliably) on pseudo file systems such as procfs(5) or devfs/devtmpfs. True, but what I'm trying to do is get the writer "doing the right thing", or at least "doing the best under the circumstances". QuoteFiles are only really deleted if they are not opened (confer output of lsof() and their (hardlink) reference count hits zero. Similarly, as long as someone somewhere (e.g. another process) holds a valid FD to stdout, this stdout is not closed/“deleted”. True, and that's one reason why unix doctrine has settled on a comparatively complex rename-and-copy sequence to handle reopening a file when a backup is to be saved. But I'd prefer not to rely on it when a Pascal-style Rewrite() is used, since this implies that a file is truncated and (potentially) that the space is reused. So what I've coded for the time being is: * When the instrument has generated data the file or pipe/FIFO/socket is closed. * If it's a pipe/FIFO/socket it's reopened promptly, since this gives both ends the best chance of sorting things out. * If it's a regular file it's not reopened until the next instrument run in case that causes data to be lost. The interpretation of "promptly" will no doubt need to be adjusted, depending on the reliability of the inotify API and/or file timestamps etc. As I've said though, this is really an attempt to "do the right thing" under different circumstances. MarkMLl ---------------------------------------------------------------------- Kays · 2022-01-10 21:11 Quote from: MarkMLl on January 10, 2022, 09:46:03 pm[…] Rewrite() There is append (in Extended Pascal, ISO 10206, known as extend). For character devices such as your terminal it’s evidently irrelevant, I think,.