https://github.com/flonatel/pipexec Skip to content Toggle navigation Sign in * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions For + Enterprise + Teams + Startups + Education By Solution + CI/CD & Automation + DevOps + DevSecOps Resources + Learning Pathways + White papers, Ebooks, Webinars + Customer Stories + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} flonatel / pipexec Public * Notifications * Fork 17 * Star 406 * Handling pipe of commands like a single command License GPL-2.0, Unknown licenses found Licenses found GPL-2.0 LICENSE Unknown COPYING 406 stars 17 forks Branches Tags Activity Star Notifications * Code * Issues 0 * Pull requests 1 * Actions * Projects 0 * Wiki * Security * Insights Additional navigation options * Code * Issues * Pull requests * Actions * Projects * Wiki * Security * Insights flonatel/pipexec This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. master BranchesTags Go to file Code Folders and files Name Name Last commit Last commit message date Latest commit History 180 Commits .github/workflows .github/ workflows build build doc doc src src test test .gitignore .gitignore .travis.yml .travis.yml COPYING COPYING LICENSE LICENSE Makefile.am Makefile.am README.md README.md ReleaseNotes.md ReleaseNotes.md configure.ac configure.ac examples.txt examples.txt version.sh version.sh View all files Repository files navigation * README * GPL-2.0 license * License pipexec Build a network of processes and connecting pipes - and have them act like a single process. Build Status Code Analysis Release License Issues Introduction and Purpose pipexec has two major use cases. Use Case 1: Handling Arbitrary Pipes between Processes Basics When it comes to pipes in shells many tutorials introduce stdin, stdout and stderr which map to file descriptors 0, 1 and 2 respectively. If you want to know how many lines contains the word bird in chapter 1 and 2 of your text, you can use a command like: $ cat Chap1.txt Chap2.txt | grep bird | wc -l And pictures like this are shown to explain what happens internally: Simple Pipe Advanced The more advanced sections provide information how to use constructs like 2>&1 to redirect stderr to stdout. And then you might come to the sections for the pros and nerds. There is explained that you can build up a whole tree of processes like $ find / 1> >(grep .txt) 2> >(wc >/tmp/w.log) Simple Tree The Hidden Universe of File-Descriptors, Processes and Pipes Nobody will tell you: 1. stdin, stdout and stderr are artificial definitions. 2. Also the relation to file descriptors 0, 1 and 2 is artificial. 3. There are more than three file descriptors. On a typical Linux system each process has by default 1024 - which can be increased if needed. 4. From starting up processes and generating pipes between them there is mostly no limitation on system level; shells only support this in a very limited way. This is the moment when pipexec drops in: with pipexec you can start up any kind of processes and build up pipes between them as you want. Cyclic $ pipexec -- [ A /usr/bin/cmd1 ] [ B /usr/bin/cmd2 ] "{A:1>B:0}" "{B:1>A:0}" gives Pipexec Cycle Complex pipexec supports any directed graph of processes and pipes like Pipexec Complex Use Case 2: Handle Bunch of Processes like one single Process Most systems to start and run processes during system start-up time do not support pipe. If you need to run a pipe of programs from an / etc/init.d script you are mostly lost. Depending on your distribution you can be happy if it starts up - but when it comes to stopping, at least the current Debian start-stop-daemon and RHEL 6 daemon function fail. Also here pipexec comes in: it starts up processes piped together, acts like a single process and supports pid file handling. Usage $ ./pipexec -h pipexec version 2.5.5 (c) 2014-2015,2022 by Andreas Florath License GPLv2+: GNU GPL version 2 or later . Usage: pipexec [options] -- process-pipe-graph Options: -h display this help -k kill all child processes when one terminates abnormally -l logfd set fd which is used for logging -p pidfile specify a pidfile -s sleep_time time to wait before a restart process-pipe-graph is a list of process descriptions and pipe descriptions. process description: '[ NAME /path/to/proc ]' pipe description: '{NAME1:fd1>NAME2:fd2}' Example: $ pipexec -- [ LS /bin/ls -l ] [ GREP /bin/grep LIC ] '{LS:1>GREP:0}' -rw-r--r-- 1 florath florath 18025 Mar 16 19:36 LICENSE Be sure to escape pipe descriptions. Brackets for the command '[]' must be separated by space! Definitions for pipes '{}' must not contain spaces! It is possible to specify a fd for logging. $ pipexec -l 2 -- [ LS /bin/ls -l ] [ GREP /bin/grep LIC ] '{LS:1>GREP:0}' 2014-05-15 16:30:35;pipexec;23978;pipexec version 2.4 2014-05-15 16:30:35;pipexec;23978;Number of commands in command line [2] 2014-05-15 16:30:35;pipexec;23978;Number of pipes in command line [1] 2014-05-15 16:30:35;pipexec;23978;[LS] command_info path [/bin/ls] 2014-05-15 16:30:35;pipexec;23978;[GREP] command_info path [/bin/grep] 2014-05-15 16:30:35;pipexec;23978;{0} Pipe [LS] [1] > [GREP] [0] 2014-05-15 16:30:35;pipexec;23978;Cannot set restart flag - process will terminate 2014-05-15 16:30:35;pipexec;23978;Start all [2] children [...] Or $ pipexec -l 7 -- [ LS /bin/ls -l ] [ GREP /bin/grep LIC ] '{LS:1>GREP:0}' 7>/tmp/pipexec.log -rw-r--r-- 1 florath florath 18025 Mar 16 19:53 LICENSE $ head -2 /tmp/pipexec.log 2014-05-15 16:30:35;pipexec;23978;pipexec version 2.4 2014-05-15 16:30:35;pipexec;23978;Number of commands in command line [2] Installation From Packages The following Linux distributions include the package. You can install pipexec with the distribution's package manager: * Debian * Ubuntu * Archlinux * Raspbian * Kali * AOS From Source Download the latest tar ball $ tar -xf pipexec-X.Y.Z.tar.xz $ mkdir PIPEXECBUILD $ cd PIPEXECBUILD $ ${PWD}/../pipexec-X.Y.Z/configure $ make There will be three binaries in the bin directory: pipexec, ptee and peet. You can copy / install them as you need. Copyright copyright 2015,2022 by Andreas Florath License: see LICENSE file About Handling pipe of commands like a single command Resources Readme License GPL-2.0, Unknown licenses found Licenses found GPL-2.0 LICENSE Unknown COPYING Activity Stars 406 stars Watchers 13 watching Forks 17 forks Report repository Releases 11 Fix 32 bit build Latest Jun 20, 2022 + 10 releases Packages 0 No packages published Contributors 6 * @florath * @flonatel * @ColinH * @jirutka * @majewsky * @d-frey Languages * C 89.1% * M4 4.8% * Shell 3.7% * Makefile 2.4% Footer (c) 2024 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact * Manage cookies * Do not share my personal information You can't perform that action at this time.