[HN Gopher] Nq - a simple Unix job queue system
___________________________________________________________________
Nq - a simple Unix job queue system
Author : unhammer
Score : 121 points
Date : 2021-01-26 20:13 UTC (2 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| txdv wrote:
| awesome tool leah!
| swlkr wrote:
| Yeah this is really cool, I especially like the idea of using it
| as a nohup alternative
| RickHull wrote:
| Note, this is from the creator of Ruby's Rack
| https://github.com/rack/rack
| andrey_utkin wrote:
| Having skimmed the README, I failed to grasp what it is. I would
| appreciate it if the README featured a clear one-sentence summary
| of what the tool does. Like...
|
| > cat - concatenate files and print on the standard output
|
| > nohup - run a command immune to hangups, with output to a non-
| tty
|
| > at -- execute commands at a later time
|
| Anyway, it's good to see new conceptual tools being developed for
| the Unix Toolbox, keep up!
| ww520 wrote:
| From the reading of the text, it's a serialized job queue for
| arbitrary commands submitted from the command line. It's the
| equivalent of "cmd1 & cmd2 & cmd3" except all are put to the
| background and will be run serially one after another. And new
| command is automatically added to the end.
| gigatexal wrote:
| at looks cool. https://linux.die.net/man/1/at
| [deleted]
| jbotz wrote:
| > I failed to grasp what it is
|
| It queues up Unix (shell) jobs. Do you understand what's meant
| by a shell "job"? Like "cmd &"?
|
| Typing "cmd1 &", "cmd2 &", etc, runs jobs in parallel. "nq
| cmd1", "nq cmd2", runs them serially, in a queue.
| jolmg wrote:
| I think the first example is pretty good:
|
| > Build targets clean, depends, all, without occupying the
| terminal
|
| This accommodates a workflow where one wouldn't prefer to just
| open up another terminal to do other stuff (maybe you're not in
| a graphical environment, maybe there's no tmux, etc.). With
| just the shell features, one could do `make ... && make ... &&
| make ... &`, but that would cause bothersome output that would
| prevent you from effectively working in the same terminal. One
| could redirect the output of those commands, but then you lose
| it, or you have to think of where to collect it. This provides
| an alternative where you can background and still have
| convenient access to the output when you need it.
|
| The other use is something like:
|
| $ nq mpv $youtube_url
|
| $ #work... work... work....
|
| $ nq mpv $next_youtube_url
|
| so that the next starts when the first finishes.
| Zhyl wrote:
| The mpv example was my main thought. If you want to add
| something to a playlist using mpv's playlist feature you have
| to stop and restart mpv.
| jolmg wrote:
| Doesn't help if you like to loop the playlist, though.
| waynesonfire wrote:
| It also offers a better history UX. For example, if you
| issued,
|
| make .. && make foobar && make ..
|
| and you wanted to just run 'make foobar' again you'd search
| your history for the last foobar invocation, and be required
| to do some editing. With nq this wouldn't be necessary.
|
| It's also not clear to me if nq has '&&' or ';' semantics in
| the event a command fails. I suspect it's ';'.
|
| Clever and fun tool but unlikely I'll adopt it.
| epistasis wrote:
| "Lightweight job queue system" is in the first line of the
| README. I'm struggling to come up with any sort of better
| description than that.
|
| Some attempts:
|
| * shell-controllable queue of commands * queue for running
| commands one after another
|
| but I still think the README's is the best.
| umvi wrote:
| You queue up terminal commands to be run in the background so
| that you can continue doing other things in the meantime.
| monadic3 wrote:
| How do you even describe what a job queue does in once
| sentence? I don't think your standard is possible for most unix
| commands, let alone most software.
|
| "sed -- stream editor" is, frankly, a useless description.
| blablabla123 wrote:
| The README is really not ideal and not reflecting how
| orthogonal/concise the tool is.
|
| Obviously it's a queue and it's as simple as `nq cmd`. Also one
| can see the status with `fq`
| gmfawcett wrote:
| > nq tries hard (but does not guarantee) to ensure the log file
| of the currently running job has +x bit set. Thus you can use ls
| -F to get a quick overview of the state of your queue.
|
| Is that a typo? Why would you need {ugo}+x to tail a file?
|
| [edit -- sorry, it's not tailing; somehow I read `tail -F`
| instead of `ls -F`.]
| pavpanchekha wrote:
| Color mode (-F) in ls will highlight executables, usually in
| green.
| gmfawcett wrote:
| Ah! Thanks. I guess if they need a second color, they could
| mark the file `setuid` as well. :P This is clever, but I
| wouldn't trust any tool that marks not-intentionally-
| executable things as executable.
| wnoise wrote:
| They are in fact executable!
|
| > Due to the initial exec line in the log files, you can
| resubmit a job by executing it as a shell command file,
| i.e. running sh $jobid.
|
| (and many shells have a legacy behavior of running a file
| as a script if it doesn't start with a #! line.)
| jmgao wrote:
| > (and many shells have a legacy behavior of running a
| file as a script if it doesn't start with a #! line.)
|
| execlp and execvp are actually guaranteed to do this in
| POSIX-2017, although shells might also do this themselves
| if they resolve $PATH by themselves.
| icedchai wrote:
| Sounds like they are repurposing the bit for _status._
| wiml wrote:
| "ls", not "tail". I think the idea is that "ls" will visually
| mark files with the execute bit set, which makes them easy to
| pick out in a directory listing.
| teddyh wrote:
| No comparison with the venerable at(1) and batch(1)?
| Areading314 wrote:
| Gnu parallel is another great one
| silasdavis wrote:
| Apart from it being academic citation ransomware:
| https://news.ycombinator.com/item?id=15319715
| watersb wrote:
| I always find great stuff when I read @leahneukirchen's blog
| posts.
| wyldfire wrote:
| Maybe a public domain alternative to LSF/bsub?
| amelius wrote:
| How is this different from simply serializing your commands by
| putting them, in the desired order, in a batch script?
| [deleted]
| sanxiyn wrote:
| It is not, but REPL is useful even if you can run script from
| file. It's like that.
| jolmg wrote:
| You can also use `;` and `&` as well as redirections and job
| control in an interactive shell session.
| aidenn0 wrote:
| Because you can append commands at any point you like without
| worrying if previous commands have completed or not.
| jolmg wrote:
| https://unix.stackexchange.com/questions/572616/run-a-
| bash-c...
|
| You can do that with the regular shell, just `^Z%;
| $next_command`, but I agree that nq might be more practical
| if that kind of workflow is done often.
| aidenn0 wrote:
| That requires you still have the original shell open that
| ran the earlier commands; nq doesn't appear to have that
| limitation.
| jolmg wrote:
| Ok, that sheds light into the kind of workflow this might
| be useful in, like calling stuff from a launcher instead
| of a terminal or something like that and having them go
| through nq automatically. Personally, I would just keep
| the shell open, but options are cool.
| sickygnar wrote:
| nohup detatches the process from the shell
| tyingq wrote:
| Gearman is popular if your needs are more complex than this.
| http://gearman.org/
| jpalomaki wrote:
| Taskspooler provides similar service. Install and submit tasks
| from command-line. You can choose how many tasks run in parallel.
|
| http://manpages.ubuntu.com/manpages/xenial/man1/tsp.1.html
| curun1r wrote:
| Also Pueue (https://github.com/Nukesor/pueue)
| garettmd wrote:
| This. Taskspooler is a powerful but simple tool that seems to
| be fairly unknown.
| xaduha wrote:
| I knew about it, but years go by and I can't honestly say
| that I ever needed it.
| gigatexal wrote:
| I was just thinking there should be a tool like this. Thanks!
| I'll check it out!
___________________________________________________________________
(page generated 2021-01-26 23:00 UTC)