[HN Gopher] In POSIX, you can theoretically use inode zero
___________________________________________________________________
In POSIX, you can theoretically use inode zero
Author : mfrw
Score : 63 points
Date : 2025-05-31 08:57 UTC (2 days ago)
(HTM) web link (utcc.utoronto.ca)
(TXT) w3m dump (utcc.utoronto.ca)
| Animats wrote:
| It's been a long time since what user space sees as an "inode"
| has anything to do with the representation within the file
| system.
| nulld3v wrote:
| Also, there seems to be an effort brewing in the kernel to push
| userspace away from depending on inode #s due to difficulty in
| guaranting uniqueness and stability across reboots.
| https://youtu.be/TNWK1zbTMOU
| AndrewDavis wrote:
| They definitely aren't unique even without reboots. Postfix
| uses the inode number as a queue id. At $dayjob we've seen
| reuse surprisingly quickly, even within a few hours. Which is a
| little annoying when we're log spelunking and we get two sets
| of results because of the repeating id!
|
| (there is now a long queue id option which adds a time
| component)
| amiga386 wrote:
| ...but it's unique while the file exists, right?
|
| The combination of st_dev and st_ino from stat() _should_ be
| unique on a machine, while the device remains mounted and the
| file continues to exist.
|
| If the file is deleted, a different file might get the inode,
| and if a device is unmounted, another device might get the
| device id.
| the_mitsuhiko wrote:
| > The combination of st_dev and st_ino from stat() should
| be unique on a machine
|
| It should, but it seems no longer to be the case. I believe
| there was an attempt to get a sysctl flag in to force the
| kernel to return the same inode for all files to see what
| breaks.
| AndrewDavis wrote:
| Yes! It's reusable, but not duplicated.
| londons_explore wrote:
| > ...but it's unique while the file exists, right?
|
| I don't think all filesystems guarantee this. Especially
| network filesystems.
| the_mitsuhiko wrote:
| It's effectively impossible to guarantee this when you
| have a file system that unifies and re-exports. Network
| file systems being an obvious one, but overlayfs is in a
| similar position.
|
| Even if inodes still work nowadays they will eventually
| run into issues a few years down the line.
| amiga386 wrote:
| That's a problem for programs that do recursive fs
| descent (e.g. find, tar) because they use st_dev and
| st_ino alone for remembering what directories they've
| been in. They can't just use the absolute path, because
| symbolic links allow for loops.
|
| find:
|
| * https://cgit.git.savannah.gnu.org/cgit/findutils.git/tr
| ee/fi...
|
| * https://cgit.git.savannah.gnu.org/cgit/findutils.git/tr
| ee/fi...
|
| tar:
|
| * https://cgit.git.savannah.gnu.org/cgit/tar.git/tree/src
| /crea...
|
| * https://cgit.git.savannah.gnu.org/cgit/tar.git/tree/src
| /name...
|
| * https://cgit.git.savannah.gnu.org/cgit/tar.git/tree/src
| /incr...
|
| In particular, I'm intrigued by the comment in the last
| link: /* With NFS, the same file can
| have two different devices if an NFS
| directory is mounted in multiple locations,
| which is relatively common when automounting.
| To avoid spurious incremental redumping of
| directories, consider all NFS devices as equal,
| relying on the i-node to establish differences. */
|
| So GNU tar expects an inode to be unique across _all_ NFS
| mounts...
| the_mitsuhiko wrote:
| You are not wrong, but the issues with tar are well
| known. Linus himself had this to say [1]:
|
| > Well, the fact that it hits snapshots, shows that the
| real problem is just "tar does stupid things that it
| shouldn't do".
|
| > Yes, inode numbers used to be special, and there's
| history behind it. But we should basically try very hard
| to walk away from that broken history.
|
| > An inode number just isn't a unique descriptor any
| more. We're not living in the 1970s, and filesystems have
| changed.
|
| You might still get away with it most of the time today,
| but it's causing more and more issues.
|
| [1]: https://lkml.iu.edu/hypermail/linux/kernel/2401.3/04
| 127.html
| amiga386 wrote:
| That sounds like blaming userspace.
|
| If it's not the 1970s anymore, then _update the POSIX
| standard_ with a solution that works for all OSes
| (including the BSDs) and can be relied upon. Definitely
| don 't suggest a Linux-only solution for a Linux-only
| problem.
| dwattttt wrote:
| Have you checked what POSIX has to say about inode
| numbers? It may say less than you think.
| amiga386 wrote:
| https://pubs.opengroup.org/onlinepubs/009696799/basedefs/
| sys...
|
| > The st_ino and st_dev fields taken together uniquely
| identify the file within the system.
|
| It says exactly what it ought to say.
| dwattttt wrote:
| Issue 8 (2024, https://pubs.opengroup.org/onlinepubs/9799
| 919799/basedefs/sy...) relaxed the language there. st_ino
| and st_dev still uniquely identify a file, but it now
| notes that the duration it identifies it for is not
| indefinite.
|
| As an example, it offers that the identity of a file
| that's deleted could be reused.
| amiga386 wrote:
| It says pretty much what I said at the start of the
| thread (https://news.ycombinator.com/item?id=44157026),
| and yet this is what Linux is having problems complying
| with:
|
| > A file identity is uniquely determined by the
| combination of st_dev and st_ino. At any given time in a
| system, distinct files shall have distinct file
| identities; hard links to the same file shall have the
| same file identity. Over time, these file identities can
| be reused for different files. For example, the st_ino
| value can be reused after the last link to a file is
| unlinked and the space occupied by the file has been
| freed, and the st_dev value associated with a file system
| can be reused if that file system is detached
| ("unmounted") and another is attached ("mounted").
|
| I still think POSIX says exactly what it needs to say,
| and Linux ought to either comply with it, or lead the
| standardisation process on what should be done instead.
|
| Don't say "tar is old". Tar's problems with Linux are the
| same problems that find, zip, rsync, cp and all other fs
| walking programs have. If memorising st_dev and st_ino
| are no good, tell us what cross-platform approach should
| be taken instead.
| Brian_K_White wrote:
| This. You can't break a fundamental assumption without
| providing it's replacement, and call anyone _else_
| stupid.
|
| "A centimeter is no longer based on anything and has an
| unpredictable length. Rulers always did stupid things
| relying on that assumption."
| the_mitsuhiko wrote:
| > You can't break a fundamental assumption without
| providing it's replacement, and call anyone else stupid.
|
| Sure, within the bounds of what's documented you are
| right. However tar is going beyond what either standard
| or Linux guarantee so a lot of bets are off.
|
| The guarantee that tar wants is not given by any FS that
| recycles inodes and most importantly, tar already
| completely disregards the file-system locality when
| network drives are involved.
|
| The actual issue here is that both tar and Linux are just
| in a tough situation because a) the POSIX spec is
| problematic b) no alternative API exists today. Something
| has to give.
| the_mitsuhiko wrote:
| Again, you are not wrong. This is all clearly not
| intended. However it has become a challenge to map things
| like Btrfs subvolumes (when seen from a Btrfs mount) onto
| POSIX semantics [1].
|
| You are absolutely right that ideally there is an update
| to the POSIX standard. But things like this take time and
| it's also not necessarily clear yet what the right path
| here is going forward. You can consider a lot of what is
| currently taking place as an experiment to push the
| envelope.
|
| As for if this is a Linux specific problem I'm not sure.
| I'm not sufficiently familiar with the situation on other
| operating systems to know what conversations are taking
| place there.
|
| [1]: https://lwn.net/Articles/866582/
| db48x wrote:
| ZFS has the same problem, for the same reasons. But it
| also has additional reasons. The simplest of them is that
| inode numbers are 64-bit integers but ZFS filesystems can
| have up to 2128 files.
| jcranmer wrote:
| There is no solution, much less one that is portable
| across different Unixen.
|
| The core problem is that, because of the ability of
| filesystems to effectively contain other filesystems
| within them, the number of bits to uniquely identify a
| file within a filesystem is not a constant number across
| different filesystem types. It's a harder problem on
| Linux because Linux is also full of filesystems that
| aren't really filesystems, where trying to come up with a
| persistent, unique identifier for people to use is a lot
| more bother than it's really worth.
| koverstreet wrote:
| The combination of st_ino and the inode generation is
| guaranteed to be unique (excepting across subvolumes, because
| snapshots screw everything up). Filesystems maintain a
| generation number that's incremented when an inode number is
| being used, for NFS.
|
| Unfortunately, it doesn't even seem to be exposed in statx
| (!). There's change_cookie, but that's different.
|
| If anyone wants to submit a patch for this, I'll be happy to
| review it.
| quotemstr wrote:
| The problem isn't relying on inode numers; it's inode numbers
| being too short. Make them GUIDs and the problems of uniqueness
| disappear. As for stability: that's just a matter of filesystem
| durability in general.
| the_mitsuhiko wrote:
| > The problem isn't relying on inode numers; it's inode
| numbers being too short.
|
| It's a bit of both. inodes are conflating two things in a
| way. They are used by the file system to identify a record
| but they are _also_ exposed in APIs that are really cross
| file system (and it comes to a head in case of network file
| systems or overlayfs).
|
| What's a more realistic path is to make inodes just an FS
| thing, let it do it's thing, and then create a set of APIs
| that is not relying on inodes as much. Linux for instance is
| trying to move towards file handles as being that API layer.
| bastawhiz wrote:
| You could make it bigger, but then your inode table gets
| pretty big. If an inode number is 32 bits today, then UUIDs
| would take up four times the space. I'd also guess that the
| cost of hashing the UUIDs is significant enough that you'd
| see a user-visible performance hit.
|
| And really, it's not even super necessary. 64-bit inode
| numbers already exist in modern file systems. You don't need
| UUIDs to have unique IDs forever: you'll never run out of
| 64-bit integers. But the problem wasn't really ever that
| you'd run out, the problem is in the way they're handled.
| quotemstr wrote:
| > You could make it bigger, but then your inode table gets
| pretty big.
|
| You could do it like Java's Object.identityHashCode() and
| allocate durable IDs only on demand.
|
| > If an inode number is 32 bits today, then UUIDs would
| take up four times the space.
|
| We probably waste more space on filesystems that lack tail-
| packing.
|
| > I'd also guess that the cost of hashing the UUIDs is
| significant enough that you'd see a user-visible
| performance hit.
|
| We're hashing filenames for H-tree indexing anyway, aren't
| we?
|
| > you'll never run out of 64-bit integers
|
| Yeah, but with 128-bit ones you'll additionally never
| collide.
| the_mitsuhiko wrote:
| The OpenBSD UFS documentation says this:
|
| > The root inode is the root of the file system. Inode 0 can't be
| used for normal purposes and historically bad blocks were linked
| to inode 1 (inode 1 is no longer used for this purpose; however,
| numerous dump tapes make this assumption, so we are stuck with
| it). Thus the root inode is 2.
|
| This is also echoed on the wikipedia page for it.
|
| The linux Kernel also has this comment for why it does not dish
| out that inode for shmem for instance:
|
| > Userspace may rely on the the inode number being non-zero. For
| example, glibc simply ignores files with zero i_ino in unlink()
| and other places.
|
| On macOS it's pretty clear that inode 0 is reserved:
|
| > Users of getdirentries() should skip entries with d_fileno = 0,
| as such entries represent files which have been deleted but not
| yet removed from the directory entry
| duckerude wrote:
| See also: https://internals.rust-lang.org/t/can-the-standard-
| library-s...
|
| A file descriptor can't be -1 but it's not 100% clear whether
| POSIX bans other negative numbers. So Rust's stdlib only bans -1
| (for a space optimization) while still allowing for e.g. -2.
| jcalvinowens wrote:
| It used to happen on Linux with tmpfs, but kernel doesn't allow
| it anymore:
| https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds...
|
| It turns out that glibc readdir() assumes inode zero doesn't
| happen and the files are "invisible" to anything using libc. But
| you can call getdents() directly and see them.
|
| I actually ran into this on a production machine once a few years
| ago, a service couldn't restart because a directory appeared to
| be "stuck" because it had one of these invisible zero inode files
| in it. It was very amusing, I figured it out by spotting the
| invisible filename in the strace output from getdents().
___________________________________________________________________
(page generated 2025-06-02 23:02 UTC)