[HN Gopher] Clean mount lists in Linux
___________________________________________________________________
Clean mount lists in Linux
Author : networked
Score : 158 points
Date : 2023-08-26 10:54 UTC (12 hours ago)
(HTM) web link (dbohdan.com)
(TXT) w3m dump (dbohdan.com)
| chrisco23 wrote:
| Great post, thanks, also 'duf' was new to me, like it.
|
| One question: I have 2 ntfs drives and they show as type
| "fuseblk". They don't show now with the new "mount" alias. I
| tried removing "fuse" from the list but no change. How can I
| bring these back to displaying when I enter "mount"?
| nemetroid wrote:
| Sounds like a Snap/Ubuntu thing. On my Arch system:
| $ df -h | wc -l 7 $ lsblk | wc -l 6 $
| mount | wc -l 26
|
| (in the article: 25, 56, 77)
| [deleted]
| semiotic1 wrote:
| Why not just use the designated `findmnt` command from util-linux
| for the majority of this?
| saurik wrote:
| This article covers three commands, and the two I use the most
| --df and lsblk--don't seem easily covered by "findmnt"?
| semiotic1 wrote:
| > df -x tmpfs
|
| findmnt -Dt notmpfs
|
| > lsblk -e 7
|
| findmnt -iS 7:0 (excludes mounted loop devices, not
| functionally equivalent to lsblk)
|
| > mount -l -t
| btrfs,fat,exfat,ext2,ext4,iso9660,ntfs3,ufs,vfat,xfs,zfs
|
| findmnt -lt
| btrfs,fat,exfat,ext2,ext4,iso9660,ntfs3,ufs,vfat,xfs,zfs
|
| > mount | awk '$5 !~ /(autofs|binfmt_misc|bpf|cgroup2|configf
| s|debugfs|devpts|devtmpfs|fuse|hugetlbfs|mqueue|nfsd|nsfs|pro
| c|pstore|ramfs|rpc_pipefs|securityfs|squashfs|sysfs|tmpfs|tra
| cefs)/'
|
| findmnt -O rw --real
|
| findmnt -lnt noautofs,nobinfmt_misc,nobpf,nocgroup2,noconfigf
| s,nodebugfs,nodevpts,nodevtmpfs,nofuse,nohugetlbfs,nomqueue,n
| onfsd,nonsfs,noproc,nopstore,noramfs,norpc_pipefs,nosecurityf
| s,nosquashfs,nosysfs,notmpfs,notracefs
|
| Or simply invert the target filesystems with --invert instead
| of prefixing "no".
|
| That counts as the majority for me.
| saurik wrote:
| Ok, the df one works, so--assuming the mount ones work (I
| haven't checked)--I guess you are up to 2/3 and so "the
| majority": touche ;P. The lsblk one though isn't even
| close... I'm not sure whether you are trying to cover this
| with your parenthetical that the functionality isn't the
| same, but then I don't get why you are trying to slip it in
| that list.
|
| The basic/key functionality of lsblk is that it shows me
| all the things that look like disks; which, to me,
| shouldn't include all of these squashfs mounts, but--very
| very critically--includes disks that haven't been mounted,
| as the whole point is that we are looking at block devices,
| not mount points, or we would be using mount instead of
| lsblk.
|
| Another important feature of lsblk is how it then takes
| those block devices and builds them into a tree, showing me
| how the various layers of--again: potentially as yet
| unmounted--block devices are "built": I can see that this
| device isn't in a filesystem, but is being used by an md
| array which is then used by a bcache or (conversely) that
| this filesystem isn't just some opaque bcache but is a
| cache of an md array of some set of specific disks.
|
| The findmnt command you provided thereby simply doesn't
| seem to do anything related to lsblk; again: you said it
| wasn't functionally the same... but then what are you
| trying to show me? I will note your findmnt version also
| doesn't even succeed in filtering all of the loopback
| disks, much less all of the random mount trash... 7:0 is
| just the first /dev/loop, but I have a giant pile of them
| (due to the stupidly slow and wasteful way Ubuntu insists
| on using squashfs). I don't think -S even supports
| something akin to filtering '7:*'?
|
| (FWIW, I do thank you for at least causing me to think
| about my lsblk usage, as I usually _hated_ how the tree was
| to-me _inverted_ and I was going to say I 'd love to have a
| tool which didn't do that... but after decades of using
| lsblk I finally looked at --help and it turns out I can use
| -s to get the output I actually want, so I am amazingly
| happy! findmnt definitely isn't a useful alternative,
| enough.)
| semiotic1 wrote:
| The block post is titled "Clean mount lists in Linux" not
| "Clean block device lists in Linux". I was demonstrating
| that findmnt can filter the source based on MAJOR:MINOR
| as a tangential, there was no attempt to "slip" anything
| in, hence the disclaimer.
|
| You are correctly pointing out that it filters only the
| first loop device and doesn't support globbing. lsblk is
| clearly the better suited tool for the job you are
| describing and the reason I said "majority" in my first
| comment.
| saurik wrote:
| That's the article's _title_ , but the actual content of
| the article is about: 1) df, 2) lsblk, and 3) mount. And
| yes: I clearly caught that you said "majority", as my
| first comment was about how you only seemed to have
| 1/3... but, now, showing -D for the df functionality
| (which was you showing me wrong, absolutely), I did agree
| that you have hit the majority (2/3). Adding the lsblk
| one continues to confuse me as to the intent.
|
| (FWIW, I frankly just didn't like how you flippantly came
| in and off-handedly dismissed the article without much
| explanation and by redirecting people to a Linux-specific
| tool that has a million options that you apparently still
| have to configure. I found the article quite valuable as
| it somehow never occurred to me to try to get these tools
| to filter these crazy Ubuntu devices, and it felt as if--
| honestly still kind of does--you were viewing the article
| extremely narrowly, maybe based on just the title.)
| pixelmonkey wrote:
| Yea, seems like:
|
| findmnt --options rw --real
|
| ... gets you most of the way there.
| networked wrote:
| Thanks. I have added a section about `findmnt` to the page.
| I'll expand the section later.
|
| Edit: It seems an adequate replacement for `df` and `mount`.
| Can it print a block device tree organized by device name
| like `lsblk`? I have skimmed the man page and not found
| options for it.
| cf100clunk wrote:
| dfc may be of interest to some:
|
| https://github.com/rolinh/dfc
|
| https://www.linux-magazine.com/Online/Blogs/Productivity-Sau...
| flas9sd wrote:
| something I came to use later in my cli life - using `df .` as in
| most cases the working directory I'm in is the filesystem I want
| to get the info on
| zh3 wrote:
| Sort of related, 'du | sort -n' to figure out what's using the
| space.
| Calzifer wrote:
| Even better 'du -h | sort -h' for human readable sizes.
| petecooper wrote:
| Somewhat related - `duf` is "a better `df` alternative":
|
| https://github.com/muesli/duf
| 9659 wrote:
| replacement?
|
| why not make df better...
| imiric wrote:
| And miss the chance to rewrite it in Rust or Go? :)
|
| Jest aside, it's much easier to write a small tool from
| scratch, than to contribute to coreutils, where this kind of
| improvement is likely to be rejected, for various arguably
| valid reasons.
|
| Besides, having alternatives is healthy for the Linux
| ecosystem, and writing something from scratch is a good
| learning experience.
| thayne wrote:
| Making a new tool means you can break backwards
| compatibility.
|
| Better is also subjective.
| gkfasdfasdf wrote:
| I also like https://github.com/Byron/dua-cli, does disk usage
| like du but also has an interactive mode which makes finding
| and freeing up space a breeze.
| jplona wrote:
| I've always used ncdu for this. It also allows deleting
| files/directories for when you're trying to figure out how to
| free up space.
|
| https://en.wikipedia.org/wiki/Ncdu
| tinco wrote:
| Even though it's not supposed to be my day job, I've ran `df
| |grep -v <super long list` hundreds of times in the past year,
| yet it never occurred to me that I could try and find a better
| tool.
| hackmiester wrote:
| If it's possible to include (rather than exclude), you can:
| $ df /dev/sda1 /dev/nvme0n1p1
|
| And get only those file systems.
| jrm4 wrote:
| Preach. This _happens to me all the time and keeps
| happening._
|
| There's a lot of Unix/Linux stuff that I wish I could
| UNlearn, I'm aware of other tools but muscle memory keeps me
| using worse things.
|
| (honestly, I also feel this way about vim as well as 'writing
| in cursive')
| entropie wrote:
| I feel you.
|
| I still `cat foo | grep bar` instead of `grep foo bar`
| nielsole wrote:
| The reason I stick to `cat foo | grep bar` is that I am
| never quite sure if `grep bar foo` would be correct
| instead (it is).
| btgeekboy wrote:
| I remember it as "grep for a needle in a haystack"
| im3w1l wrote:
| The file argument is usually the last so that you can do
| eg `grep foo -- *`
| jrm4 wrote:
| But to me -- this is different?
|
| I believe "useless use of cat" is FAR MORE useful than
| what the ultra-nerds say; I could care less about "saving
| a few characters."
|
| The mental model of the pipeline --in one end and out the
| other -- is entirely too usefully intuitive and way
| better than some imaginary geek-cred.
|
| (I was more thinking about the new rust tools, e.g. rg
| instead of grep)
| systems_glitch wrote:
| Plus sometimes you want to throw something like `pv` in.
| simon04 wrote:
| Plus the regex is more likely incorrect in the first
| attempt and needs to be fixed.
| gumby wrote:
| `df|grep` might be "the better tool" because the cognitive
| overhead of knowing an extra program's interface.
|
| You're just saying "show me the subset of this other list"
| and you automatically select your go-to solution for that:
| `|grep`
|
| The reason for seeking a better tool is when too much
| excessive work (== time) is involved, or list generation has
| side effects. But in this case you don't have two million
| mount points, so no need to worry.
| t-3 wrote:
| Installing a new tool just to exclude certain parts of the
| list has much more friction involved than just making a shell
| alias/function/script though. It's also much more portable.
| ggambetta wrote:
| And `dust` as a `du` replacement.
| Given_47 wrote:
| Started using dust yesterday and like it so far. Also a big
| fan of diskus by sharkdp as a _blazingly fast_ du -hs
| alternative
| 2bluesc wrote:
| Also `dua`[0] is a great `du` replacement which is must
| faster on modern NVMe drives. It includes an interactive mode
| `dua i` which I'd frame as a `ncdu` replacement.
|
| [0] https://github.com/Byron/dua-cli
| backendanon wrote:
| I'm OK with how the disk tools show stats on Linux, thank you.
| leslielurker wrote:
| Off topic but the favicon for this website is giving me big
| Windows 3.1 File Manager nostalgia. Whenever I see a file icon
| from that era it takes me back. </getting-old>
| networked wrote:
| I am glad someone noticed. :-) It _is_ the File Manager icon. I
| have always liked this icon and the aesthetic of graphics made
| for Windows 3.x, even though my first Windows was 95 OSR2.
| Since File Manager and the icon are now free
| (https://github.com/microsoft/winfile), there is no copyright
| reason not to use it verbatim.
|
| If you liked it, you may enjoy my page https://dbohdan.com/gui-
| games.
| cyfex wrote:
| I have been using `duf` [1] for a while, as a "cleaner"
| alternative to `df`.
|
| [1] https://github.com/muesli/duf
| rollcat wrote:
| Here we are, one of the primary reasons why I adore the BSDs.
| Linux seems to be just neverending layers of indirection,
| abstraction, and a kitchen sink of features that less than 1% of
| its users actually need.
|
| The full command to get a "clean" mount list on OpenBSD,
| including disk usage with human-friendly units, is:
| $ df -h
|
| https://man.openbsd.org/df
| liendolucas wrote:
| In Linux is no longer: df -h
|
| but df -h | egrep SOMETHING
|
| or maybe df -h | less
|
| and yes I still don't understand why dozens of mount points
| reported.
| [deleted]
| bayindirh wrote:
| Well, if you're seasoned enough, you start to read df output
| from right to left.
|
| Find the mountpoint you're looking for, read to left to see the
| stats. When you're managing a storage box, no amount of tmpfs
| entries can create the noise stated in the blog post.
|
| P.S.: If you know the device names like your name, you can
| quickly skim that list, too.
| rollcat wrote:
| I prefer, in this order:
|
| 1. For the computing device to do the more reasonable thing;
|
| 2. For the computing device to adapt to and serve my needs,
| not the other way around.
| backendanon wrote:
| You want the mount points first? In school we were taught
| to think in what then where order. That's how the df -h
| command works on Linux, the way most people think. When you
| want to eat, most of us think what do I want to eat, then
| where.
| bayindirh wrote:
| I think Linux is doing the more reasonable thing. The
| features are implemented with the existing facilities and
| extended to the bare minimum as required.
|
| On top of that the commands are adaptable, and you can
| always write the tools you need. There are no
| hidden/forbidden interfaces.
|
| I commend BSDs for what they are, but blindly praising any
| OS is wrong IMHO. Is Linux perfect? Heck no. Is BSDs are
| squarely better at everything, again no.
|
| These are different OSes shine at different things. We use
| all of them, but they are not that interchangeable.
| rollcat wrote:
| My praise is far from blind; OpenBSD lacks a modern
| filesystem, FreeBSD still uses /usr/local/etc for some
| configuration, and all of them suffer to a various degree
| from simply not being x86 Linux with glibc and systemd.
| What you do get in return though is some sanity. When I
| type "df -h", what I mean is: "show me the remaining free
| space on the internal disks and attached media". It's not
| a big ask for that information to not be littered with
| half a dozen lines of useless noise?
| derefr wrote:
| You have a point, but I don't think you have the point
| you think you have. It seems like df(1) as a tool should
| be about showing you "free space, on things where the
| amount of free space might ever be a concern." (Otherwise
| you'd just use mount(1)!) So df(1) itself, should be
| _filtering_ the mounts list for only devices capable of
| _having_ free space.
| rollcat wrote:
| That is exactly my point. But my other point is that
| keeping things simple (what is /dev/shm? I thought /dev
| is already a synthetic FS?) helps avoid the problem in
| the first place.
| derefr wrote:
| > what is /dev/shm? I thought /dev is already a synthetic
| FS?
|
| /dev/shm is a tmpfs mount. It's not synthetic. It's
| basically "/tmp but it uses your RAM so it's not
| essentially-unbounded, so we didn't put it on /tmp
| because then stupid legacy programs might OOM the
| kernel."
|
| It's mounted under /dev presumably because you can think
| of "a tmpfs mount" as a system _facility_ for "named
| shared-memory allocations" -- in the same way that e.g.
| /dev/audio is a system facility, or /dev/random is a
| system facility. It's a facility that presents itself as
| a virtual filesystem, but it's still fundamentally a "low
| level system API" meant for consumption by the systems
| programmers writing e.g. glibc, rather than something the
| user is meant to see and browse into as part of the
| filesystem abstraction. (I believe that /dev/mapper is
| another thing that ended up under /dev for this reason.)
|
| Also, /dev _was_ a synthetic FS for a little while --
| devfs -- but this long post-dates the creation and
| conventional-ization of these other mountpoints inside
| /dev. Until 2009, /dev was just a directory in the
| rootfs, with special-device inodes created manually
| during rootfs creation with mknod(8).
|
| These days, under udev, /dev is once again "just a
| directory" -- just one with a daemon that automatically
| creates and destroys the special-device inodes in it to
| match what's actually available on the system.
| rollcat wrote:
| Thank you for the explanation. The implementation details
| and the history are both interesting, but again -
| completely irrelevant to the question of "am I running
| out of disk space".
|
| Having thought of your suggestion again (df should hide
| irrelevant mounts), I think it would be unfortunate. I
| might have legitimate reasons to check on my tmpfs mounts
| (e.g. as you suggested, a stupid program dumping cruft
| into /tmp). Changing df also doesn't help clean up my
| Grafana dashboard.
|
| Again, I think the root cause of the problem lies
| somewhere between exposing implementation details to the
| user, and layering on more uninvited complexity. Between
| a bunch of Linux boxes I have at hand, /run, /run/lock,
| /run/wrappers, /run/user/1000 (and /run/user/1001,
| 1002...), are all separate mount points. If /run is
| already a tmpfs mount point, why does /run/lock have to
| be one as well? The mount flags are pretty much
| identical. If uid 1000 shouldn't be able to deny uid 1001
| the ability to write to their XDG_RUNTIME_DIR, why not
| implement some form of quotas?
|
| I'm sure there are good answers to all of these
| questions, but my main point still stands - even when
| this information _is_ relevant, it 's being drowned in
| the noise.
| derefr wrote:
| > I might have legitimate reasons to check on my tmpfs
| mounts
|
| Yeah, but the output of df(8) re: tmpfs is meaningless,
| as 1. _every_ tmpfs mount is considered to _each_ have as
| much free space as there is RAM in the system ( _total_
| RAM, not _free_ RAM!); and 2. the "used space" stat will
| track unlinked-but-open files, which for a tmpfs
| shouldn't be thought of as even being "in" the tmpfs at
| that point, but rather just being reserved IPC memory
| held by one or between several processes.
|
| If you want to know the _disk usage_ of a tmpfs dir, use
| du(1). That 's what it's for: seeing what the on-disk
| space taken by the tree of inodes in a directory or
| mountpoint aggregates up to.
|
| df(1) is an abbreviation for "disk free" for a reason --
| the output of df(1) only makes sense insofar as there's a
| bounded, independent, reserved pool of "free space" of a
| resource that you want to measure and manage.
|
| Since df(1) _used to_ exist in an environment where the
| only mount points _were_ of such resources, df(1) never
| _previously_ had to do any filtering of the mounts table
| to achieve its stated job. But the "spirit of its
| semantics" would today imply filtering.
|
| > If /run is already a tmpfs mount point, why does
| /run/lock have to be one as well?
|
| I believe this is a long-term deprecation caught in mid-
| transition -- in theory, /run itself doesn't actually
| need to be a tmpfs any more; according to the FHS and the
| newer versions of XDG, all the tmpfs-es should be
| subdirectories of /run. Once that actually happens, /run
| would just become a regular directory, like /dev is.
|
| > If uid 1000 shouldn't be able to deny uid 1001 the
| ability to write to their XDG_RUNTIME_DIR, why not
| implement some form of quotas?
|
| I believe these dirs are actually done this way for
| efficiency, not security: one magical thing about a tmpfs
| mount is that it somewhat acts like a memory arena for
| the allocations within it; so _unmounting_ it will _batch
| free_ all those memory allocations in a very efficient
| manner. (Think: the time it takes to `rm -rf` a million
| tiny files in a scratch partition, vs. just reformatting
| said partition.) /run/user/... is created at user login;
| users expect logout [i.e. session refcount dropping to 0]
| -- and/or graceful shutdown -- to not hang on unlinking a
| million tiny files; and having /run/user/... be a
| separate tmpfs that gets unmounted on logout, helps with
| that.
|
| Separately, I believe "application users" for
| containerized services get /run/user/... mounts created
| for them, _within their separate mount namespace_ (and
| this is the only way that that can work); having non-
| containerized users also use mountpoints here allows code
| reuse, rather than two mostly-redundant and potentially-
| buggy codepaths.
|
| > even when this information is relevant, it's being
| drowned in the noise
|
| Yes, I agree. My point is more that only certain
| _filesystem mounts_ are actually _mounts of bounded-size
| writable disks_. And that it 's only _mounts of bounded-
| size writable+ disks_ that a system administrator would
| be concerned about "managing" by _asking the question_
| "how much of this disk is free?"
|
| (+ One example you didn't mention: read-only squashfs
| images, used for e.g. Ubuntu snaps -- which show up under
| df(1) as always-100%-utilized mounts.)
| yyyk wrote:
| >/dev/shm is a tmpfs mount. It's not synthetic. It's
| basically "/tmp but it uses your RAM so it's not
| essentially-unbounded
|
| /dev/shm is for the shared-memory facility. It's not used
| as a temporary filesystem by anything (possibly excepting
| the user). FreeBSD's shm implementation actually does
| support the read/write interface, but it doesn't expose a
| filesystem.
|
| >, so we didn't put it on /tmp because then stupid legacy
| programs might OOM the kernel."
|
| >Yeah, but the output of df(8) re: tmpfs is meaningless,
|
| /tmp is tmpfs on Fedora and other distros. The size
| parameter on mount can limit available 'storage' so the
| free 'space' df output can be useful.
| drewg123 wrote:
| To be fair, FreeBSD with ZFS (especially with the poudriere pkg
| build system, and 2 different linux compat environs) has quite
| a few (40+) filesystems mounted. See my workstation:
| https://dpaste.org/bnwdt
| backendanon wrote:
| That command works fine on Linux, what am I missing?
| rollcat wrote:
| The fine article.
| backendanon wrote:
| The article is useless. What am I missing, I'll ask again.
| rollcat wrote:
| The article describes the steps necessary to get the
| short and relevant answer: how much free space is there
| left on my physically attached disk devices? On Linux,
| the answer you're actually looking for is hidden among
| half a dozen tmpfs/devtmpfs/udevfs/whateverfs "synthetic
| filesystem" mounts; the situation is even more dire when
| you type "mount" which will list another couple dozen,
| with the relevant information (such as, which external
| media is mounted?) hidden in that noise.
|
| The BSDs, rather than drowning the user in noise, and
| creating the problem that the article tries to address,
| provide just the relevant information.
| johannes1234321 wrote:
| On FreeBSD `du` also doesn't necessarily go e the right
| info due to us, where having multiple filesystems on the
| same pool is easy and simple. This leads to many
| filesystems reporting same free space. Right way there is
| to check the zpool directly ...
| rollcat wrote:
| That's 100% a ZFS problem, not a FreeBSD problem ;) I
| have the same issue on my Ubuntu and NixOS systems. Well
| that's the price you have to pay for ZFS, it doesn't
| exactly fit in a traditional model - but the space
| reported by "du" is still accurate e.g. wrt the remaining
| quota.
| johannes1234321 wrote:
| True, but given the role of ZFS in FreeBSD this hits most
| FreeBSD installs, thus the claim of any BSD solving that
| isn't true.
|
| OpenBSD solves that by limiting their features.
| akira2501 wrote:
| It's down to the distribution. If you roll without all the
| systemd and canonical nonsense, then you get a pretty clean
| system that's on par with the BSDs.
___________________________________________________________________
(page generated 2023-08-26 23:00 UTC)