[HN Gopher] The File Filesystem (2021)
___________________________________________________________________
The File Filesystem (2021)
Author : wegwerff
Score : 161 points
Date : 2024-04-30 17:43 UTC (5 hours ago)
(HTM) web link (mgree.github.io)
(TXT) w3m dump (mgree.github.io)
| isoprophlex wrote:
| For fuck's sake! Not everything needs to be a file!
|
| (This is a joke. I love the idea and execution.)
| mypalmike wrote:
| I see what you did there.
| vidarh wrote:
| Daniel Stenberg, of cURL fame, co-write an Amiga-editor called
| FrexxEd where the open buffers were exposed as files in the
| filesystem.
|
| Meaning you could write any shell script to manipulate an open
| buffer (not _that_ important as it also exposed all editor
| functionality both via IPC via Arexx and via FPL - a C-like
| scripting language), and that you could e.g. compile without
| saving ( _that_ was very helpful on a system where a lot of
| people might only even have a single floppy drive, and where
| being able to have the compiler in the drive and compile
| straight from the in-memory version in RAM so you didn 't have
| to keep swapping floppies was highly useful (just remember to
| save before actually trying to _run_ the program - no memory
| protection...)
| salgernon wrote:
| Classic MacOS in the 80s had "MPW" macintosh programmers
| workshop - that treated open text windows as files _and
| selections within the windows as files_ , so it wasn't
| uncommon to have a portion of an otherwise documentation file
| have a "click here and hit enter", which would use the
| selected text as stdin for some semi-ported unix tool. (no
| memory protection or multitasking, so true pipelines with
| backpressure didn't work)
| freeney wrote:
| This looks awesome, I need to give it a try asap. I can very well
| see myself using this to navigate or search inside JSON files
| yjftsjthsd-h wrote:
| I would gently suggest naming it filefs or something; ffs already
| means https://man.freebsd.org/cgi/man.cgi?ffs(7)
|
| That said - good idea/approach; seems like an excellent way to
| cleanly extend the unix approach to structured file formats:)
| wegwerff wrote:
| also https://en.wikipedia.org/wiki/Formatted_File_System
|
| and https://en.wikipedia.org/wiki/Flash_file_system
|
| and kind of
| https://en.wikipedia.org/wiki/Amiga_Fast_File_System
|
| Four file systems named ffs; ffs is popular for file systems
| galkk wrote:
| Op's ffs do not target FreeBSD and it l seems like referenced
| system is FreeBSD only. Claiming naming rights is a stretch
| here
| eichin wrote:
| That's ahistorical. FFS is the _berkeley_ Fast File System,
| from BSD 4.2, in 1983.
| yjftsjthsd-h wrote:
| FFS predates FreeBSD and is in some capacity supported by all
| 3 major BSDs. I'm fairly confident that Linux actually
| supports it through the ufs driver (
| https://github.com/torvalds/linux/tree/master/fs/ufs );
| whether the use of different names in different places makes
| it better or worse is an exercise for the reader.
| tambourine_man wrote:
| There's another expansion to that acronym that I can think of.
| I think the joke is implied.
| jasonpeacock wrote:
| You can also mount your git commits as a filesystem:
|
| https://jvns.ca/blog/2023/12/04/mounting-git-commits-as-fold...
|
| (previously: https://news.ycombinator.com/item?id=38527866 )
| amiga386 wrote:
| All you need now is a giant pile of rules for which revisions
| to select and you have the unholy demon that is Rational
| ClearCase
| TheGlav wrote:
| What!? you didn't add a versioned database layer on a server
| with code stored in clearcase that stored those ClearCase
| config specs to manage the configuration of your config specs
| to manage the configuration of your version control system
| that had your application configuration in it?! How did you
| even operate? /s
| skissane wrote:
| I always thought Oracle ADE was a cooler demon. Shame the
| internal talk about productising it never went anywhere.
| Edmond wrote:
| If you're intrigued by this then Solvent-Configr might be of
| interest:
| https://aws.amazon.com/marketplace/pp/prodview-i3ym46leenag4
|
| It uses file system mechanics to model objects, meaning you can
| design object based solutions that support file system style
| navigation.
|
| Demo: https://youtu.be/XgTgubZQPHw
| sambeau wrote:
| Ha. I did this back in 2003. It's surprisingly fast, and makes it
| simple to do granular locking.
|
| I used it as a per-user database for a web-templating language
| for a giant web-site building tool.
| compressedgas wrote:
| Also the Parts-of-file File System:
| https://www.usenix.org/conference/2005-usenix-annual-technic...
| ThinkBeat wrote:
| No XML, Excel,PDF or CSV support yet.
| RetroTechie wrote:
| Useful enough that it should be an OS-level standard feature,
| imho.
|
| Unix-like OSes allow mounting disk images to explore their
| contents. But there's many more file formats where exploring
| files-inside-files is useful. Compressed archives, for one. Some
| file managers support those, but (imho) application-level is not
| the optimal layer to put this functionality.
|
| Could be implemented with a kind of driver-per-filetype.
| frizlab wrote:
| Honest question: How is this useful? I don't see any use-case
| where this would come in handy.
| TimeBearingDown wrote:
| You could seed compressed archives of massive text files or
| similar via BitTorrent while making the contents available to
| your apps in read-only mode.
| RetroTechie wrote:
| It allows you to use any tool available for regular files, on
| the files-in-files as well.
|
| As opposed to extract contents and _then_ work on that
| (requiring extra steps + disk space). Or be limited to what
| specialized utilities support.
| jraph wrote:
| It exists :-)
|
| For zip archives, there are fuse-zip and mount-zip which are
| FUSE filesystem.
|
| As an intermediate between OS level and application-level,
| there are desktop environment level: gvfs for GNOME and KIO for
| KDE, but they are compatible only in their own ecosystems.
| duped wrote:
| Really what you'd like to see is a way to write the mount
| command for each file type (do one thing well) and another
| command to detect the file type and dispatch accordingly
| (probably similar to the `file` command), all in user space.
|
| The only thing standing in the way of this today is that MacOS
| doesn't expose a user space file system API. You can do this on
| Linux, Windows, and BSDs today.
|
| (No, file provider extensions don't cut it, Apple devs who read
| this, please give us a FUSE equivalent, we know it exists).
| Groxx wrote:
| Does https://osxfuse.github.io/ cover this? Or is there some
| fundamental issue? (beyond "it's not built in")
| skissane wrote:
| Recent macOS versions do have a general purpose built-in
| API for user mode filesystems. That API is incompatible
| with FUSE. The big problem is it is undocumented and you
| need an entitlement from Apple to use it, and Apple won't
| give you that.
|
| Apple do have a publicly available API for cloud file
| systems (Dropbox-style products), but it makes a lot of
| assumptions which makes it effectively unusable for other
| use cases.
|
| Then there are third party solutions like osxfuse. These
| have the problem that they rely on kernel extensions and
| Apple keeps on making those harder and harder, and is
| aiming to get rid of them; plus, they are all now
| proprietary licensed, albeit often with a free license for
| open source use.
|
| One approach that does work without any kernel extensions
| or private APIs is to make your user filesystem an NFS
| server and then mount that. One competitor to osxfuse does
| that, but it also is proprietary
| duped wrote:
| Well that requires a kext so it's a nonstarter, and fuse-t
| uses NFS which is extremely janky and unreliable on MacOS.
|
| The fundamental issue is that macOS doesn't provide an API
| for this natively.
| kybernetikos wrote:
| It's not exactly the same, but nushell provides ways of
| exploring inside files.
| lmm wrote:
| This was a core design feature of reiserfsv4, but Linux
| ultimately refused to merge it, probably not helped by the
| whole murdering-his-wife thing.
| chuckadams wrote:
| Neat. Now how about a filesystem that takes a directory of files
| and exposes it as a single json file? You could call it the
| Filesystem File, and mount it in the File Filesystem if you
| wanted...
| paulgb wrote:
| This is really neat, but when I saw the headline I got excited
| that it was something I have been looking for / considering
| writing, and I figure the comments here would be a good place to
| ask if something like this exists:
|
| Is there a FUSE filesystem that runs in-memory (like tmpfs) while
| mounted, and then when dismounted it serializes to a single file
| on disk? The closest I can find are FUSE drivers that mount
| archive files, but then you don't get things like symlinks.
| khc wrote:
| does it have to be fuse? cant you mount a disk image with
| loopback
| metadat wrote:
| But will the disk image be fully stored in memory? No.. not
| with loopback. Either that, or it won't be mutable in memory
| with commit on unmount.
| andrewflnr wrote:
| Wouldn't a disk image have a fixed size? It could be a pain
| to resize.
| ranger_danger wrote:
| qcow2 is auto-expanding
| Scaevolus wrote:
| Not purely in-memory, but something like
| https://github.com/jrwwallis/qcow2fuse maybe? It's clunky
| compared to OSX's DMGs, but if you squint it achieves similar
| ends.
|
| Otherwise you could achieve this with a tmpfs wrapped to
| serialize to a tarball (preserving symlinks) when unmounted.
| ranger_danger wrote:
| Oh nice, I didn't even know that existed. I've been using
| qemu-nbd and parted by hand and it gets cumbersome, so this
| might help a lot. Thanks!
| ranger_danger wrote:
| I can't think of anything _exactly_ like that, but I think you
| can get close by just copying some type of image file to /tmp
| and then moving it to disk when you're done after unmounting.
| AgentME wrote:
| /tmp isn't stored in memory; it's usually a normal on-disk
| filesystem that's cleared regularly. You want /dev/shm
| instead, which is a purely in-memory filesystem on normal
| Linux systems.
| codetrotter wrote:
| > /tmp isn't stored in memory
|
| It is if your system uses tmpfs for /tmp
|
| https://en.wikipedia.org/wiki/Tmpfs
| throwway120385 wrote:
| The point they were trying to make is that it doesn't
| have to be, and it isn't in several of the Linux systems
| I've used over the years. Assuming that it is is a bad
| idea.
| speps wrote:
| Closest I found: https://github.com/guardianproject/libsqlfs
|
| > The libsqlfs library implements a POSIX style file system on
| top of an SQLite database. It allows applications to have
| access to a full read/write file system in a single file,
| complete with its own file hierarchy and name space. This is
| useful for applications which needs structured storage, such as
| embedding documents within documents, or management of
| configuration data or preferences. Libsqlfs can be used as an
| shared library, or it can be built as a FUSE (Linux File System
| in User Space) module to allow a libsqlfs database to be
| accessed via OS level file system interfaces by normal
| applications.
| agumonkey wrote:
| All I see is a generic tree walk mechanism, here implemented in
| folders/files, but in plan9 it's .. plumber ?
| alephaleph wrote:
| Reminds me of Omar Rizwan's TabFS <https://omar.website/tabfs/>
___________________________________________________________________
(page generated 2024-04-30 23:00 UTC)