[HN Gopher] Lnav - An advanced log file viewer for the small-scale
       ___________________________________________________________________
        
       Lnav - An advanced log file viewer for the small-scale
        
       Author : thunderbong
       Score  : 155 points
       Date   : 2023-01-04 09:01 UTC (1 days ago)
        
 (HTM) web link (lnav.org)
 (TXT) w3m dump (lnav.org)
        
       | LinuxBender wrote:
       | The package is available in Alpine so I tried it out. It seems to
       | work best if the terminal is using en_US.UTF-8. My default is C
       | but I have a function to set UTF-8. I was curious if it could
       | handle stdin and it does. It saves the output in ~/.config/lnav/
       | something to probably add to my clean up script. It is convenient
       | that it can prepend timestamps to logs. I could see this being
       | nice for people that do QA on build logs all day or that have to
       | debug access logs often. If used with sensitive data one should
       | probably add a shell function and log rotate scripts to clean up
       | ~/.config/lnav/ using the shred command.
        
         | pinkorchid wrote:
         | This will depend on the specific threat model for the sensitive
         | data you are dealing with, but generally files in an SSD can't
         | be destroyed securely (and shred specifically does not help due
         | to wear leveling).
         | 
         | Instead, you may want to encrypt your disk so that as soon as
         | the key is gone it all becomes unreadable. For a bigger threat,
         | maybe you need to follow the NIST destroy guidelines [1] to
         | "Disintegrate, Pulverize, Melt, and Incinerate" the media.
         | 
         | 1:
         | https://nvlpubs.nist.gov/nistpubs/specialpublications/nist.s...
        
         | c0wb0yc0d3r wrote:
         | I was under the impression that the shred command was pretty
         | hard on SSDs. Did that change?
        
           | LinuxBender wrote:
           | It would be as hard as writing the file {n} number of times
           | again. So if one created a 1MB log file and then did a 3 pass
           | wipe that would write 3MB. Number of passes would really just
           | depend on the system being worked on. If PCI I would expect 7
           | passes and I am sure that would just be factored into the
           | cost of the servers for that environment.
           | 
           | Shred also isn't perfect as it has no concept of the file
           | systems journal and does not clean that data but I think it
           | is still good practice for sensitive data, in addition to
           | filesystem or file level encryption on systems with highly
           | sensitive data.
           | 
           | With sensitive data one must look at the value of the data vs
           | the value of the SSD. If I go out of my way to extend the
           | life of my SSD, do I risk losing $5billion? That is how I
           | factor in whether or not extra wear-and-tear on a SSD makes
           | sense.
        
             | electroly wrote:
             | Shred doesn't work _at all_ on SSDs because wear leveling
             | will spray the writes all over the drive, rather than
             | overwriting the blocks you intended to. Your random bytes
             | will end up in new blocks, not overwriting the original
             | blocks. Trying to shred individual files is a totally
             | pointless exercise on SSDs. It isn 't doing what you want
             | it to do.
        
               | LinuxBender wrote:
               | I've also seen people say that and to instead use
               | "blkdiscard -sz" but that also does not write to specific
               | blocks. I still can not find a definitive source saying
               | that shred is not able to write to specific blocks on a
               | SSD and it is not clear to me that wear leveling actually
               | prevents shred from writing to specific blocks. But I
               | agree that I have seen people say this repeatedly on
               | StackExchange, ServerFault and Reddit.
               | 
               | Either way one should also use filesystem and file level
               | encryption for sensitive files and encrypt sensitive
               | attributes inside databases. Swap must also be encrypted
               | as it may contain sensitive data. Datacenter bad-drive
               | mishaps do happen _as in not shredding the physical
               | drive_ as a few government and military agencies have
               | recently been embarrassed by.
               | 
               | I do agree with pinkorchid that ultimately drives should
               | be destroyed physically.
        
               | j0057 wrote:
               | It's also explained in the shred manual itself, that
               | seems like a decently authoritative source to me: https:/
               | /www.gnu.org/software/coreutils/manual/html_node/shre...
               | -- halfway the page all the caveats are listed up to &
               | including forensic analysis of magnetic traces of the
               | deleted data.
        
               | electroly wrote:
               | Wear leveling prevents _anyone_ from writing to specific
               | blocks. It decouples the OS 's block numbering from the
               | physical blocks. That indirection allows it to distribute
               | writes evenly across the drive even if the OS is writing
               | to consecutive blocks (or the same block).
               | 
               | This behavior is fundamental; it's what wear leveling
               | _is_ and why it exists. Wear leveling exists to prevent
               | you from writing to the same block repeatedly, which is
               | exactly what you 're trying to do with shred.
               | 
               | I'm not sure what other source you're looking for, but I
               | hope you find it. Perhaps, at least, the number of people
               | "on StackExchange, ServerFault and Reddit" and now also
               | HackerNews telling you the same thing is sufficient
               | evidence to stop spreading the idea that shred might work
               | on SSDs. Do what you want on your own systems, of course,
               | but this is dangerously misleading guidance to be
               | offering on an open forum.
        
               | LinuxBender wrote:
               | So my concern with following this logic is that such
               | tools that are writing to specific block locations would
               | then be wiping unrelated inodes depending on the low
               | level logic. I tried emailing Colin but his last known
               | email is invalid now. He may be working for the NSA so I
               | probably won't be able to reach him. Some people there
               | are responsive, some are not and a few are on here.
               | 
               | So even if wear leveling prevents overwriting a file then
               | such tools should in theory be a risk of data corruption.
               | If this is the case then the tool should be updated to
               | detect if the target is an SSD and abort with a scary
               | message. Perhaps another route is to reach out to the
               | coreutils team.
        
               | sgc wrote:
               | Does that mean that partitions on an SSD therefore do not
               | affect the long term wear on the SSD?
        
         | tstack wrote:
         | The data consumed from stdin is stored in a file to support
         | scrollback.
         | 
         | > something to probably add to my clean up script.
         | 
         | These capture files are deleted if they are older than a day
         | the next time lnav is run.
         | 
         | (I realize they should've been stored in ~/.local/state instead
         | of ~/.config ...)
        
       | hericium wrote:
       | Do we need lnav thread every 3 weeks?
        
         | mhitza wrote:
         | Yes /s
         | 
         | Not everyone checks out daily HN to see all the reposts.
         | Pulling numbers out of the air, I'd say that 5-10% of daily
         | content on HN is a repost, should get used to it. Just hide it
         | an carry on is what I would suggest.
        
         | qbasic_forever wrote:
         | Keep a log of how often it occurs and we can review it... in
         | lnav
        
         | xdfgh1112 wrote:
         | First I'd heard of it. I have wanted a tool like this for a
         | while.
        
           | asah wrote:
           | Same, and I'm working on an IoT project that precisely needs
           | this (superhigh$ per device, don't need scale or efficiency)
        
         | joshka wrote:
         | lnav is pretty great. I wish it had a library that could be
         | pulled out into an app instead of a terminal. There are a few
         | things that make more sense with a GUI like standard keyboard
         | shortcuts, filtering, saving exports, etc.
        
           | hericium wrote:
           | Not saying there's something wrong with lnav - just that
           | spamming HN every 3 weeks isn't great.
           | 
           | There were submissions with links to lnav.org main site:
           | 
           | - 21 days ago
           | 
           | - 22 days ago
           | 
           | - 46 days ago
           | 
           | - 68 days ago
           | 
           | It's a spam campaign from my pov.
        
             | __float wrote:
             | Do you browse new? None of these have had more than a
             | couple points since
             | https://news.ycombinator.com/item?id=9294622 in 2015.
             | 
             | It seems very reasonable that most visitors simply never
             | saw any of those submissions.
        
             | chalst wrote:
             | Five submissions in 70 days is rather less than a spam
             | campaign in my view.
        
       | Thaxll wrote:
       | I mean if the goal is to read logs the default color scheme looks
       | pretty bad, why there are so many colors in the example?
       | 
       | For example the http 403/404 is red but it does not stand out.
       | Also why a 404 would be red in the first place.
        
         | tstack wrote:
         | > why there are so many colors in the example?
         | 
         | Identifiers, like process-names/pids/IPs, are semantically
         | highlighted by default to make it easier to visually match up
         | values that are the same. And, it's this way because I like it
         | this way, so that's the default.
         | 
         | > For example the http 403/404 is red but it does not stand
         | out.
         | 
         | To me, the red stands out. But, I would also have a much wider
         | window, so the red would stand out even more with the rest of
         | the text.
         | 
         | > Also why a 404 would be red in the first place.
         | 
         | Because 4XX are error codes.
         | 
         | Of course, lnav is pretty customizable at this point, so you
         | can adjust it to your liking. The theme can be changed to
         | something less colorful and the log format definitions can be
         | patched to change their behavior.
        
       | mihaigalos wrote:
       | Nice work! I've written something similar based on regexes:
       | 
       | https://github.com/mihaigalos/pipeview
        
       | _bax wrote:
       | I'm using every day on multiple erver and application. It a MUST
        
       | haspre wrote:
       | Works fine on my pi.. thx
       | 
       | With caddy server
        
       | calmoo wrote:
       | I used this extensively when working in a cloud support role,
       | incredible tool.
        
       | lathiat wrote:
       | Does anyone have a trick for connecting to remote hosts as a non-
       | root user but having it elevate to root to read the logs?
        
         | prmoustache wrote:
         | For systemd/journald managed log, best is to add non privileged
         | user to the systemd-journal group.
         | 
         | For logs managed by syslog daemons, most implementations allows
         | you to set the owner and group of the log files. You could
         | decide on a specific log group and add the desired dedicated
         | user to this group.
         | 
         | In the end it is usually better to ship the logs to a dedicated
         | machine/space/database.
        
       | yellow_lead wrote:
       | It's cool you can SSH to demo it. More CLI tools should have
       | that.
        
         | tstack wrote:
         | The SSH demo was inspired by git.charm.sh
        
       | zulu-inuoe wrote:
       | Say I'm building a logging library, is there any sort of
       | generally agreed-upon standard per what a log file should look
       | like? I know there's several formats that lnav supports, but I'm
       | not familiar with them.
       | 
       | I've always had trouble before figuring out what a good
       | compromise between log format and flexibility looks like.
       | Especially wrt newlines in the log message itself (eg if I want
       | to log a stack trace)
       | 
       | Any thoughts for a format that would generally work ootb in lnav
       | or other log viewers?
        
         | 10987654321 wrote:
         | Not sure about lnav but most log aggregation systems support
         | json and logfmt-formatted logs, and there are many standard
         | logging libraries that supports emitting those formats
         | 
         | Of those json is better if you want to be able to do more
         | advanced stuff (nest dictionaries, use lists, ...) and logfmt
         | is better if you want to have human-readable logs without
         | external tools as well, an example line can look like
         | msg="Request finished" tag=request_finish status=200
         | user=brandur@mutelight.org user_id=1234 app=mutelight
         | app_id=1234
         | 
         | Some more info here https://www.brandur.org/logfmt
        
           | GordonS wrote:
           | You can use logfmt with Serilog on dotnet too:
           | 
           | https://github.com/serilog-contrib/Serilog.Logfmt
        
           | ithkuil wrote:
           | Has anybody written an actual spec for logfmt? I noticed that
           | different implementations handle escaping of quoted strings
           | subtly differently
        
           | jteppinette wrote:
           | Agree with logfmt. I wrote the logfmter python library:
           | https://github.com/jteppinette/python-logfmter. You can
           | quickly have all of your logs (including 3rd party) converted
           | to this style.
        
           | tstack wrote:
           | lnav does support JSON-lines and logfmt logs. For JSON-lines,
           | it will pretty-print the log messages to make them human
           | readable.
           | 
           | For logfmt, I seem to remember the spec not being very clear
           | on quoting semantics (maybe I'm wrong). Anyhow, I would
           | suggest using JSON since it has pretty broad support at this
           | point.
        
           | zulu-inuoe wrote:
           | Thank you! Now when you say json logging, are there any
           | common patterns you can guide me to look at? Just looking
           | around for JSON logging gives me results like [1] which talk
           | about JSONL (object per line, presumably JSON serializer
           | makes sure to not emit literal newlines).
           | 
           | But some other places describe this a bit more liberally. And
           | [2] notes that I should include a time stamp in each log
           | entry which makes sense.
           | 
           | [1]: https://stackoverflow.com/questions/10699953/format-for-
           | writ... [2]:
           | https://www.papertrail.com/solution/tips/8-essential-tips-
           | fo...
        
         | capableweb wrote:
         | There is a bunch of them (log formats). I don't think any is
         | like ISO/ANSI standarized, but many tools work with many of the
         | different formats.
         | 
         | Probably the most common one (or at least was most common,
         | maybe not anymore) would be "NCSA Common log format" (or just
         | CLF) which looks something like this:
         | 127.0.0.1 user-identifier frank [10/Oct/2000:13:55:36 -0700]
         | "GET /apache_pb.gif HTTP/1.0" 200 2326
         | 
         | https://en.wikipedia.org/wiki/Common_Log_Format
         | 
         | Many tools for getting analytics out of server-side logs can
         | work with CLF and the various variations. But probably today
         | there are more "modern" formats as well.
        
       | shae wrote:
       | This tool saved me weeks of work once. You gotta try it.
        
       | hoyd wrote:
       | Related to this, is there a tool like this, but that can show
       | some kind of simple analytics, for example page views over a
       | period of time from an access log produced by python html server.
        
         | guessmyname wrote:
         | > _[...] a tool like this, but that can show some kind of
         | simple analytics_
         | 
         | Yes, you can use - https://goaccess.io/
        
         | kiliancs wrote:
         | You can run queries on the log files and write them to files. I
         | have used these files to feed pre-built spreadsheets with
         | charts, etc.
        
         | tstack wrote:
         | As mentioned in a sibling comment, https://goaccess.io is
         | probably best for simple analytics on log files.
         | 
         | If you want to write more complicated queries, lnav exposes log
         | data through SQLite vtables[1]. So, you can do a SQL query and
         | get a simple bar chart visualization.
         | 
         | [1] - https://docs.lnav.org/en/latest/sqlext.html#sqlite-
         | interface
        
         | tobylvp wrote:
         | I mentioned this below, but you could try LogViewPlus. It has a
         | built in SQL based reporting feature you can use.
         | 
         | You can see it in action here:
         | https://www.logviewplus.com/docs/create_a_custom_report.html
        
       | tobylvp wrote:
       | For Windows users, there is LogViewPlus. (I am the author).
       | https://www.logviewplus.com/
       | 
       | It has a similar feature set (tail, syntax highlighting, SQL
       | reporting) with a focus on accessing files remotely.
        
         | tstack wrote:
         | I'm the author of lnav and had never heard of logviewplus
         | before, thanks for mentioning it. It looks like we've
         | independently arrived at a very similar feature set, that's
         | pretty neat. I will now shamelessly steal some of your ideas :)
        
           | tobylvp wrote:
           | It looks like you have done a great job with lnav - it's
           | impressive. I know how much work has gone into LogViewPlus
           | and you must have made a similar commitment. I thought I was
           | the only one crazy enough to want to solve this problem. :-)
           | 
           | I would be happy to chat to a like mind if you are
           | interested. You can contact me here:
           | https://www.logviewplus.com/contact.aspx
           | 
           | Steal away! :-)
        
         | winter_blue wrote:
         | Is LogViewPlus heavily tied to Windows? How hard would a port
         | to Linux be?
         | 
         | I love that so many Windows tools get a nice-looking GUI,
         | versus a heavy CUI lean to *nix tools.
        
           | tobylvp wrote:
           | As things currently stand - hard. LogViewPlus is written in
           | .Net using WinForms, so it is heavily tied to Windows.
           | However, Microsoft is evolving. The bulk of the code can now
           | be made to be cross-platform easily, but the front-end
           | remains a problem.
           | 
           | I would love to do a Linux port as soon as there is a viable
           | cross-platform GUI technology with strong 3rd party vendor
           | support. The controls in LogViewPlus are very rich.
        
         | kcbanner wrote:
         | I've been using https://glogg.bonnefon.org/. The mark / matches
         | feature is really handy. However there are a few bugs with
         | highlighting and it hasn't been updated in a while. Will have
         | to check this out!
        
           | tobylvp wrote:
           | Glogg is a great tool. We take a bit of a different approach
           | in that we parse the log file. This enables some more in-
           | depth features such as SQL reporting. We also support remote
           | file access (for example, SFTP).
           | 
           | Let me know if you have any questions or feedback. You can
           | reach me on the contact page of the site.
        
       ___________________________________________________________________
       (page generated 2023-01-05 23:02 UTC)