[HN Gopher] Show HN: Nerdlog - Fast, multi-host TUI log viewer w...
___________________________________________________________________
Show HN: Nerdlog - Fast, multi-host TUI log viewer with timeline
histogram
For more background and technical details, I wrote this up as well:
https://dmitryfrank.com/projects/nerdlog/article
Author : dimonomid
Score : 94 points
Date : 2025-04-21 11:38 UTC (11 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| Zopieux wrote:
| journalctl is mentioned once in the landing page and it seems to
| imply that journalctl is not supported per se, as logs need to be
| stored plaintext to legacy syslog (?).
|
| I do not want to store plaintext logs and use ancient workarounds
| like logrotate. journald itself has the built-in ability to
| receive logs from remote hosts (journald remote & gateway) and
| search them using --merge.
| tstack wrote:
| The article makes it sound like it uses various command-line
| tools (bash/awk/head/tail) to process the logs. So, I imagine
| it's not a huge leap to extend support to using journalctl to
| do that work instead.
| mamcx wrote:
| One small hitch I found is that this kind of tools are fixes
| in what to process, so for example I can't use them for
| structured logging. If it has an escape hatch where I can
| supply my own pipe (for example `process = 'vector ....'`)
| then it will be enough.
| dimonomid wrote:
| That's true, as of today nerdlog doesn't use journalctl, and
| needs plain log files. There were a few reasons of that,
| primarily related to the sheer amount of logs that we were
| dealing with.
|
| As mentioned in the article, my original use case was: having a
| fleet of hosts, each printing pretty sizeable amount of logs,
| e.g. having more than 1-2GB log file on every host on a single
| day was pretty common. My biggest problem with journalctl is
| that, during some intensive spikes of logs, it might drop logs;
| we were definitely observing this behavior that some messages
| are clearly missing from the journalctl output, but when we
| check the plain log files, the messages are there. I don't
| remember details now, but I've read about some kind of
| ratelimiting / buffer overflow going on there (and somehow the
| part which writes to the files enjoys not having these limits,
| or at least having more permissive limits). So that's the
| primary one; I definitely didn't want to deal with missing
| logs. Somehow, old school technology like plain log files keeps
| being more reliable.
|
| Second, at least back then, journalctl was noticeably slower
| than simply using tail+head hacks to "select" the requested
| time range.
|
| Third, having a dependency like journalctl it's just harder to
| test than plain log files.
|
| Lastly, I wanted to be able to use any log files, not
| necessarily controlled by journalctl.
|
| I think adding support for journalctl should be possible, but I
| still do have doubts on whether it's worth it. You mention that
| you don't want to store plaintext logs and using logrotate, but
| is it painful to simply install rsyslog? I think it takes care
| of all this without us having to worry about it.
| whalesalad wrote:
| can't you just read from stdin?
|
| i use lnav in this way all the time: journalctl -f -u service
| | lnav
|
| this is the ethos of unix tooling
| dimonomid wrote:
| Not really, at least not yet, because nerdlog's focus is
| very different than that of lnav. There is a section about
| it in the article as well.
|
| In fact nerdlog doesn't even support anything like -f
| (realtime following) yet. The idea to implement it did
| cross my mind, but I never really needed it in practice, so
| I figured I'd spend my time on something else. Might do it
| some day if the demand is popular, but still, nerdlog in
| general is not about just reading a continuous stream of
| logs; it's rather about being able to query arbitrary time
| periods from remote logs, and being very fast at that.
| lenova wrote:
| I appreciate this response, and want to say I really like
| your tool's UI over something like lazyjournal. But like the
| above commentor, I would love to see journald support as
| well, just because it's the default these days on the distros
| I use, and seems like the direction the Linux system industry
| has headed in.
| dimonomid wrote:
| Thanks for the feedback. I'll see what I can do. But for
| now, do you think the workaround of having to install
| rsyslog is not good enough?
| lenova wrote:
| I think it will impact first-time users giving nerdlog a
| quick test/trial run, and cause them to bounce to another
| tool when it doesn't show them logs from journald out of
| the box. Users can be finicky and impatient with new
| tools ;-)
|
| Example: I'm running an Arch-based Linux desktop.
| Installing ryslog took several minutes to build and
| install. If I wasn't highly motivated to try out nerdlog,
| I would have canceled the install.
|
| Also, can the SSH requirement for localhost be bypassed?
| Most users won't be running an SSH server on their
| desktop, and this would improve nerdlog's use-cases and
| make it easier for new users to give it a quick local
| test run.
|
| Final suggestion: add `go get` support to your repo, so
| that I can install nerdlog from a single command and not
| have to clone the repo itself.
| dimonomid wrote:
| Yeah you're right, agree with both of your points.
|
| The `go get` one should be easy to solve though, and my
| bad for not thinking of it before, thanks. I'll look into
| it.
| lenova wrote:
| Appreciate your receptiveness, and sorry about all of the
| edits... I was rethinking my thoughts in real-time ;-)
| dimonomid wrote:
| I first responded before your edit about ssh and
| localhost, so: yeah, as briefly mentioned in the article,
| as of today there's no shortcut even for localhost. I was
| debating whether I should implement this feature before
| open sourcing it, but I had to draw the line somewhere (I
| have TONS of ideas what could be implemented), and since
| reading local logs isn't the primary focus of nerdlog, I
| decided to skip it for now.
|
| But yes the bypass for localhost can definitely be
| implemented.
| mcint wrote:
| Yeah, I'm bouncing for now on the localhost requirement.
| Or, on a related issue of not parsing my .ssh/config, a
| Match directive, and not wanting it to parse it yet. I
| grep'ed for an env var to override, but only USER and
| SSH_AUTH_SOCK are pulled in.
|
| I did go get install ...nerdlog/cmd/nerdlog-tui@latest
| just fine.
|
| Thanks for hacking in the open, and releasing early.
| dimonomid wrote:
| Sorry to hear you're having issues. I'll try to reproduce
| and fix the issue with the Match.
|
| Not sure if that "Thanks" for releasing early is
| sarcastic, but regardless, I appreciate the feedback.
| tstack wrote:
| Nice work! The TUI looks really sharp and I like the histogram on
| top. Going to play with this today.
|
| TIL awk patterns can be more than just regexes and can be
| combined with boolean operators. I've written a bit of awk and
| never realized this.
| esafak wrote:
| Looks nice. You might want to get help from the community to get
| it packaged for major linux distros, if you want more users.
| dimonomid wrote:
| Thanks, and yeah getting distro packages would be dope.
| Hopefully, some day.
| tomerbd wrote:
| Can i view logs from aws cloudwatch?
| mdaniel wrote:
| You'll go broke doing that, as those API calls are not free.
| Best to configure cloudwatch to dump into some sane place (S3,
| SigNoz, whatever) so you only pay the api call once and not
| every time for interactive viewing
|
| I went spelunking around in the codebase trying to get the
| actual answer to your question and it seems it's like many
| things: theoretically yes with enough energy expended but by
| default it seems to be ssh-ing into the target hosts and
| running a pseudo agent over its own protocol back through ssh.
| So, "no"
| openWrangler wrote:
| Seconded - it sounds like compatibility isn't there yet with
| AWS, but it would be great if there was a way to use nerdlog
| with other OSS dashboard tools like Signoz or Coroot like you
| mentioned. Still a really interesting graylog altnerative.
| knowitnone wrote:
| Nice. I needed this a few years ago. No license file?
| dimonomid wrote:
| Yeah right, my bad, and thanks for reminding me. Just added one
| (the BSD 2-clause).
| adityavinodh wrote:
| Looks great! Was just looking for something like this.
| ryanhecht wrote:
| I definitely intend on playing around with this later! I see that
| [gzipped log archives aren't supported](https://dmitryfrank.com/p
| rojects/nerdlog/article#depends_on_...), minimizing the use case
| for me personally. You've at least thought enough about that to
| bring it up as a limitation you think people will call attention
| to -- any plans to eventually support it?
| dimonomid wrote:
| Thanks for the feedback!
|
| Yeah it would be great, and I do want to support it, especially
| if the demand is popular. In fact, even if you ungzip them
| manually, as of today nerdlog doesn't support more than 2 files
| in a logstream, which needs to be fixed first.
|
| Specifically about supporting gzipped logs though, the UX I'm
| thinking about is like this: if the requested time range goes
| beyond the earliest available ungzipped file, then warn the
| user that we'll have to ungzip the next file (that warning can
| be turned off in options though, but by default I don't want to
| just ungzip it silently, because it can consume a signficant
| amount of disk space). So if the user agrees, nerdlog ungzips
| it and places somewhere under tmp. It'll never delete it
| manually though, relying on the regular OS means of cleaning up
| /tmp, and will keep using it as long as it's available.
|
| Does it make sense?
| ryanhecht wrote:
| Definitely makes sense!
|
| > In fact, even if you ungzip them manually, as of today
| nerdlog doesn't support more than 2 files in a logstream
|
| Ah, interesting! I read the limitation as "we don't support
| zipped files," not "we only support two files!"
|
| Best of luck, this is neat!
| nodesocket wrote:
| Very nice work. Anyway to specify a group of log files in the
| config that are shared across many hosts? For example:
| log_files: mygroup: - /var/log/syslog
| - /var/log/foo - /var/log/bar log_streams:
| myhost-01: hostname: actualhost1.com port:
| 1234 user: myuser log_files: mygroup
| myhost-02: hostname: actualhost2.com port:
| 7890 user: myuser log_files: mygroup
| myhost-03: hostname: actualhost3.com port:
| 8888 user: myuser log_files: mygroup
| dimonomid wrote:
| Thanks. And no, as of today, there's no way to define a group
| like that. Might be a viable idea though.
|
| However, before we go there, I want to double check that we're
| on the same page: this `log_files` field specifies only files
| _in the same logstream_; meaning, these files need to have
| consecutive logs. So for example, it can be ["/var/log/syslog",
| "/var/log/syslog.1"], or it can be ["/var/log/auth.log",
| "/var/log/auth.log.1"], but it can NOT be something like
| ["/var/log/syslog", "/var/log/auth.log"].
| mdaniel wrote:
| At the very grave risk of scope creep, I'll point out that
| the GP's yaml is very close to an Ansible inventory file so
| rather than just making up a new structure one could leverage
| any existing muscle memory (and create helpful defaults for
| folks who have not yet seen Ansible but have seen your tool)
|
| https://docs.ansible.com/ansible/11/collections/ansible/buil.
| ..
|
| e.g. all: children:
| mygroup: hosts: myhost-01:
| hostname: actualhost1.com port: 1234
| user: myuser myhost-02:
| hostname: actualhost2.com port: 7890
| user: myuser myhost-03:
| hostname: actualhost3.com port: 8888
| user: myuser vars: files:
| - /var/log/syslog - /var/log/foo
| - /var/log/bar
|
| That first "children" key is because in ansible's world one
| can have "vars" and "hosts" that exist at the very top, too;
| the top-level "vars" would propagate down to all hosts which
| one can view as "not necessary" in the GP's example, or
| "useful" if those files are _always_ the same for every
| single host in the whole collection. Same-same for the
| "user:" but I wasn't trying to get bogged down in the DRY for
| this exercise
| piterrro wrote:
| This is a really cool project -- love the simplicity and the TUI
| approach, especially with the timeline histogram and remote-first
| design. I had similar pain points at my end when dealing with
| logs across many hosts, which led to building Logdy[1] -- a tool
| with a slightly different philosophy.
|
| Logdy is more web-based and focuses on live tailing, structured
| log search, and quick filtering across multiple sources, also
| without requiring a centralized server. Not trying to compare
| directly, but if you're exploring this space, you might find it
| useful as a complementary approach or for different scenarios.
| Although we need to work more on adding ability to query multiple
| hosts.
|
| Anyway, kudos on nerdlog--always great to see lean tools in the
| logging space that don't require spinning up half a dozen
| services.
|
| [1] https://logdy.dev
| johnisgood wrote:
| This looks good. Any way to use date/time in RFC 3339 format
| without changing the source code?
|
| Does this work with runit (Void Linux)?
| dimonomid wrote:
| Re: the date/time format, I was thinking about implementing
| support for an option like timefmt, so you'd be able to do :set
| timefmt=2006-01-02T15:04:05Z07:00 , but postponed for now.
|
| That's not hard to implement, however to make it persistent
| requires implementing some config / scriptability, which is a
| whole other thing and requires more thought.
|
| Re: runit, I never tested it, but after looking around briefly,
| it sounds like there is no unified log file, and not even
| unified log format? I mean it's possible to make it work,
| treating every log file as a separate logstream, but I've no
| idea what these logs look like and whether supporting the
| formats would be easy.
| johnisgood wrote:
| Void Linux uses svlogtail for viewing logs.
|
| It is a simple script: https://github.com/void-linux/socklog-
| void/blob/master/svlog...
|
| I think everything is in _/
| var/log/socklog/everything/current_, so this could be
| considered united.
|
| Before you add the timefmt, it may be better to add a
| configuration file if one does not already exist, but it
| seems like it does? You already have _~
| /.config/nerdlog/logstreams.yaml_, so might as well have
| _config.yaml_?
|
| For more about logging on Void:
| https://docs.voidlinux.org/config/services/logging.html
| dimonomid wrote:
| That's good news that we have
| /var/log/socklog/everything/current, but I'm also trying to
| figure the format. Is it like this? (sourced from chatgpt)
|
| 2025-04-21 12:34:56 myhostname myservice: Something
| happened
|
| If so, then yeah it's totally doable to make this format
| supported.
|
| Re: config.yaml, yeah I thought of that, but in the long
| term I rather wanted it to be nerdlogrc.lua, so a Lua
| script which nerdlog executes on startup. Similar to vim
| (or rather, more like neovim in this case since it's Lua).
| Certainly having config.yaml is easier to implement, but in
| the longer term it may make things more confusing if we
| also introduce the Lua scripting.
| johnisgood wrote:
| Sadly no. The format is (with examples):
| 2025-04-21T19:18:15.09577 user.notice: Apr 21 21:18:15
| root: ACPI group/action undefined: jack/lineout / LINEOUT
| 2025-04-21T19:18:15.98845 daemon.debug: Apr 21 19:18:15
| rtkit-daemon[1368]: Supervising 0 threads of 0 processes
| of 1 users.
|
| And yes! That is even better for configuration!
| dloss wrote:
| Very nice! Added to my little list of log viewers at
| https://github.com/dloss/klp#alternative-tools
| dimonomid wrote:
| Thanks, appreciate that!
___________________________________________________________________
(page generated 2025-04-21 23:01 UTC)