[HN Gopher] Lessons from Ancient File Systems
___________________________________________________________________
Lessons from Ancient File Systems
Author : zdw
Score : 105 points
Date : 2024-07-27 23:33 UTC (2 days ago)
(HTM) web link (madcompiler.blogspot.com)
(TXT) w3m dump (madcompiler.blogspot.com)
| bediger4000 wrote:
| This is the type of analysis I always hope for about file
| systems, but almost never see. Looks like some of these file
| systems had linked list files, which is an interesting contrast
| to inode based file systems like the original UFS, or BSD FFS.
| jhallenworld wrote:
| Perhaps they did it to save RAM since you are buffering the
| current sector anyway. An alternative is CP/M's extent-based
| filesystem, where the meta-data is all in the directory entry.
| But I think space allocation algorithm is more complex for it.
|
| I like the OS/8 file allocation scheme: files are always
| contiguous. Only one growing file can be open for writing and
| it allocates space out of the contiguous free space (actually
| it owns all of the free space until the file is closed, where
| the unused space is then given back). If you run out of disk
| space, you have to compact the disk, eliminating space taken up
| by deleted files.
|
| I like it because it's illustrates the surprising fact that you
| can get away with only one file open for writing for most
| things.
| Someone wrote:
| > I like it because it's illustrates the surprising fact that
| you can get away with only one file open for writing for most
| things.
|
| https://en.wikipedia.org/wiki/CP/M#CP/M_Plus:
|
| _"The last 8-bit version of CP /M was version 3, often
| called CP/M Plus, released in 1983. Its BDOS was designed by
| Brown. It incorporated the bank switching memory management
| of MP/M in a single-user single-task operating system
| compatible with CP/M 2.2 applications"_
|
| Note the "single task" part. That excludes a lot from what we
| nowadays would be included in "most things". That, I think,
| is the reason you could get away with that. I also would
| guess it was more that programmers would either not know what
| they missed or accepted it as a limitation that kept the OS
| small.
| inkyoto wrote:
| On the modern hardware (i.e. SSD's) files are always
| <<contiguous>>, thanks to the transparent block remapping in
| the SSD firmware.
|
| Compacting/squeezing was only necessary because magnetic
| disks were contiguous in their nature and had to be erased
| and rewritten, which was a time consuming (sometimes very
| time consuming) and potentially dangerous operation leading
| to a complete data loss if power flickered.
|
| To a certain extent, the compacting is still there and is
| done in the SSD's (now called <<garbage collection>>),
| although it is also transparent to the user and to the
| operating system.
| jdougan wrote:
| This might scratch that itch aloso. Beneath Apple DOS:
| https://mirrors.apple2.org.za/Apple%20II%20Documentation%20P...
| jftuga wrote:
| Conclusion: "So the big lesson here is to always plan for the
| future. Listen to the requirements for the current product, but
| then design with the assumption that you'll be asked to expand
| the requirements in the future. If you don't, users may be
| cursing you when the code is released. But who knows? If you do
| it right, people may still be using your code in 40 years."
| saulpw wrote:
| Unfortunately, "plan for the future" is too general to be
| workable; this is how you get architecture astronauts using
| UUIDs and multiple levels of indirection for a 160KB floppy.
|
| The future you can plan for, is specifically 10x more. If you
| have 160KB floppies, then design as though there might be 1.6MB
| floppies. You'll have to revisit your design anyway for the
| emergent complexities of the next order of magnitude, but this
| will at least give you some breathing room.
|
| Designing for 100x more than your system has is
| overengineering. You have no idea what the needs of the future
| are, and the design for a 100x system will be detrimental for
| your current 1x system, and likely sink it. So ironically
| planning for 100x means you probably won't get there.
| antonkochubey wrote:
| Windows XP lived thorough 6 GB (minimum requirement - 1.5 GB,
| but I don't recall seeing drives less than six gigs at the
| time) to 2 TB HDD's. That's literally a 333x increase, which
| NTFS handled just fine. From min requirement it'd be a 1000x
| increase.
| akira2501 wrote:
| > That's literally a 333x increase
|
| Yet that's only going from 2^32 to 2^40.
| jeroenhd wrote:
| Windows XP's default clustering (well, until SP2 at least)
| "only" allowed for 128GB disks. The cluster size was later
| increased. The 2TB limit was generally a limit for the BIOS
| code rather than an OS limit, as NTFS will happily scale
| beyond 2TB.
|
| Windows NT 4 already contained the code necessary to allow
| for up to 16 exabyte partitions (https://web.archive.org/we
| b/20010208131204/http://support.mi...), but the hardware it
| was running on probably didn't support anything bigger than
| a few terabytes. Sure, every text file takes up at least
| 2MB, but with an exabyte disk, you probably don't care
| about wasting sectors like that.
| paulmd wrote:
| a system that can be scaled 333x or pushed into radically
| different allocation patterns by changing one or two
| constants seems like an odd choice for arguing that YAGNI
| and you shouldn't plan for anything beyond 10x/should
| plan for a system rearchitecture at that scale. That
| seems very much like a system that thought ahead and
| picked meaningful knobs to allow drastic changes in use-
| case to suit the situation.
| kalleboo wrote:
| When XP came out, you already had 100 GB consumer drives on
| the market that it had to support out of the box, so that's
| where I'd put the design requirements "starting point".
| That makes it only one order of magnitude bigger, similar
| to the 10x.
| banannaise wrote:
| Designing for 10x will probably get you to 100x just fine
| anyway. Scaling up the first 10x is already in the design, so
| that design only needs to scale another 10x. Scaling that
| second 10x will probably be _just fine_ , if not ideal.
| taneq wrote:
| Exactly. Perfect is the enemy of good, and good often the
| enemy of pragmatic (assuming we're using 'good' in the
| sense of 'what most engineers would think was a complete
| solution' not in the enlightened sense of 'meeting our
| actual needs for the near future').
| didgetmaster wrote:
| Backwards compatibility can be the bane of innovation. There
| are many instances where we are living with arcane systems or
| limitations simply because it was too cumbersome to break
| compatibility.
|
| Deciding what are 'acceptable' limitations given current
| constraints vs 'planning for future expansion' is an art form.
| Unfortunately, too little thought is actually put into that
| decision in too many cases.
|
| I remember working on a system (NetWare) when the server error
| code was a single byte. It didn't take long to run out and they
| started assigning the same error code to multiple conditions.
| The last one (0xFF) was so widely used, internal docs referred
| to it as 'something bad happened'.
| jclulow wrote:
| Seems like a lost opportunity to introduce 0xFF as the error
| code that means "look in the extended error field we've since
| added to the message, which includes a u32 _and_ a human
| readable string for display" to be honest.
| didgetmaster wrote:
| It's been many years ago; but if memory serves...the error
| code was part of a header in a server response packet.
| Expanding that header to include another field would break
| all the clients and handler software.
|
| Eventually, a major release fixed the problem with a
| different header with expanded fields; but that was not a
| simple fix. Like I said, backwards compatibility CAN be the
| bane of innovation.
| munchler wrote:
| This is the opposite of today's "You aren't going to need it"
| (YAGNI). I think the best approach might be somewhere in
| between.
|
| https://martinfowler.com/bliki/Yagni.html
| jujube3 wrote:
| Yes. This would be "You ARE going to need it" (YAGNI)
| daotoad wrote:
| The hard part of applying YAGNI is figuring out which value
| of A is appropriate.
| benreesman wrote:
| This is like the Halting Problem for Cloud.
|
| Well put.
| m463 wrote:
| thankfully there was a flexible future-proof design in
| place for the acronyms allowing re-use and backward-
| compatible upgrade.
| kristopolous wrote:
| How are the design intentionalities determined? Are they
| documented somewhere? I'm not disputing anything, I just want to
| know the provenance in case there's some really great book or
| code I should be reading.
| pwg wrote:
| The book "Inside Atari DOS" by Bill Wilkinson (author of Atari
| DOS) goes into some of the rationale:
|
| https://www.atariarchives.org/iad/
| skissane wrote:
| > Instead, it was to support "note" and "point" commands. Atari
| believed people would use files as a database, and with the
| sector chaining, it was impossible to jump around in a file
| without reading it linearly. So a program could "note" its
| position in a file (sector and offset) and then return there
| later with the point command. This is where verifying that the
| file number in the user-supplied sector number matched the entry
| used when opening the file came in. Without that number, there
| would be no verification that a "point" would end up in the right
| file.
|
| Interesting to hear this terminology used in the context of 8-bit
| Atari computers. The terms NOTE and POINT come from IBM
| mainframes, where there are OS/360 macros by those names. NOTE
| essentially does an fgetpos and POINT an fsetpos
| jhallenworld wrote:
| It answers the question of where experienced programmers then
| got their experience. In the mid 70s, Sigma-7 was another
| popular 32-bit machine, but I don't know anything about its OS.
| mrandish wrote:
| Brings back fond memories of the Amiga DOS floppy disk structure
| which had some clever things I hadn't seen before (coming from
| 8-bit ROM-based floppy disk OSes). Such as having each file
| sector link to the next so if the file allocation table got
| corrupted, the files on the disk could be recovered by a utility
| that crawled every sector and rebuilt a new file allocation
| table.
|
| IIRC, it also would occasionally write a backup copy of the file
| allocation table on another normally unused track as well as
| parking the disk head away from the track where the directory was
| after some default period of inactivity. Floppy drives were
| notorious for occasionally crashing the heads into whatever track
| they happen to be over if there was a power glitch or software
| crash - and of course, statistically, they spent most of their
| time over the directory track, thus greatly increasing to odds of
| blasting part of the file allocation table.
|
| When I moved to PCs I remember being surprised that NTFS didn't
| seem to do any of this (at least early on) despite being more
| recent and supposedly advanced.
___________________________________________________________________
(page generated 2024-07-30 23:02 UTC)