[HN Gopher] The weird world of Windows file paths
___________________________________________________________________
The weird world of Windows file paths
Author : sharevari
Score : 287 points
Date : 2023-04-21 14:47 UTC (8 hours ago)
(HTM) web link (www.fileside.app)
(TXT) w3m dump (www.fileside.app)
| naikrovek wrote:
| eh some of these are slightly wrong but yeah it's weird.
|
| MSDOS included some compatibility for CP\M and everything since
| has maintained compatibility with the version prior, except for a
| few exceptions.
|
| so even today we have compatibility built into Windows for things
| that don't exist anymore in any real capacity.
|
| Microsoft is very serious about backwards compatibility.
| pjmlp wrote:
| Old Microsoft was very serious, young blood Microsoft not so
| much.
| naikrovek wrote:
| I am not sure where you are coming from with that comment.
| Microsoft hasn't broken backwards compatibility on anything
| that I'm aware of in some time.
| pjmlp wrote:
| .NET Framework to .NET Core transition, Xamarin.Forms to
| MAUI, XNA, the multiple rewrites on the WinRT platform
| since Windows 8, Windows 8 to 10 users pace drivers
| framework, .NET Native, C++/CX, Win2D, WinRT for XBox
| replaced by WinGDK, .NET Sharepoint Framework via a
| JavaScript one,
| fisian wrote:
| > Disallowed characters
|
| > < > " / \ | ? * Never allowed
|
| A friend once somehow crated a file called <HTML>.
|
| I don't know how, but he also couldn't delete or do anything with
| it.
| kps wrote:
| They should have fixed that when they went to long file names.
| It's ridiculous that you can't name a file with its contents'
| actual title. Random example:
| http://doi.org/10.1145/327070.327153
| dark-star wrote:
| So many things wrong with this article. Some things that I
| noticed by skimming over it:
|
| > UNC paths can also be used to access local drives in a similar
| way:
|
| > \\\127.0.0.1\C$\Users\Alan Wilder
|
| > UNC paths have a peculiar way of indicating the drive letter,
| we must use $ instead of :.
|
| This is actually incorrect... He's actually accessing some random
| share that has no real connection to a drive. Yes, sometimes
| (quite often), the C$ share corresponds to the C: drive's root,
| but this is by no means given, as one can easily either delete
| the C$ share, or have it pointing to somewhere else entirely
|
| > When the current directory is accessed via a UNC path, a
| current drive-relative path is interpreted relative to the
| current root share, say \\\Earth\Asia.
|
| This is also wrong. There is no "current directory" on an UNC
| share (which can easily be shown by trying to open a command
| prompt on a UNC share, it will show an error and start you
| somewhere on C:\users), and the example he gives just tries to
| access the share "Asia" on the server "Earth"
|
| > Less commonly used, paths specifying a drive without a
| backslash, e.g. E:Kreuzberg, are interpreted relative to the
| current directory of that drive. This really only makes sense in
| the context of the command line shell, which keeps track of a
| current working directory for each drive.
|
| Also wrong, it's not the command line shell that keeps track of
| the current directories, it's the Windows kernel itself. But I
| agree that such a scenario is quite useless as you can never be
| quite sure on what CWD you are on a given drive
|
| > For the most part, : is also banned. However, there is an
| exotic exception in the form of NTFS alternate data streams.
|
| Yeah, well, surprise: the ":" is not part of the file name, it's
| just a separator between filename and stream name. This is like
| saying that "you cannot have \ characters in a file name, but in
| directory names it is allowed". No, it's not. It's a separator
| ChrisSD wrote:
| > There is no "current directory" on an UNC share
|
| SetCurrentDirectory allows setting the current directory to a
| UNC share. https://learn.microsoft.com/en-
| us/windows/win32/api/winbase/...
|
| > Also wrong, it's not the command line shell that keeps track
| of the current directories, it's the Windows kernel itself. But
| I agree that such a scenario is quite useless as you can never
| be quite sure on what CWD you are on a given drive
|
| Not for a long time. It's set as a special (hidden) environment
| variable like `=C:=C:\current\directory`.
| https://devblogs.microsoft.com/oldnewthing/20100506-00/?p=14...
| muststopmyths wrote:
| >SetCurrentDirectory allows setting the current directory to
| a UNC share.
|
| Exactly. The cmd prompt not setting UNC paths as current
| directory was introduced around Windows 2000 (or maybe post-
| XP, it's been a while) to help legacy batch files being run
| from a share and then getting confused by being on a UNC path
| instead of one beginning with a drive letter.
|
| This was also why, when you do a pushd \\\server\share
| cmd.exe puts you on a mapped drive instead of directly on a
| UNC path.
|
| If you use the Windows native version of tcsh, for example,
| you can happily use UNC paths as current directories and run
| commands (provided they don't try to parse drive letters from
| their CWD)
| tpmx wrote:
| This got me to try out out the Fileside app (an Explorer/Finder
| alternative) from the blog post author. It's available for
| Windows and Mac.
|
| I think this is an interesting space with room for innovation.
|
| Fileside starts out with a grid of four directories: Home,
| Documents, Desktop and Downloads. You can customize and name new
| grid layouts that are shown in a sidebar for quick switching.
| This seems like a neat idea for specific recurring manual
| workflows.
|
| It's doesn't seem to be targetted to the minimalist crowd.
| Directory entries beginning with a dot are visible (but greyed
| out). Full Unix-style permissions are shown for each entry, etc.
|
| It looks like it's Electron-based and implemented in a javascript
| SPA framework. It doesn't use the default system font (SF Pro) on
| Mac. A bunch of other things also don't look or behave as you
| expect.
|
| The font weight in the size column maps to each file's relative
| size. All the way from very thin to very bold. Kind of cute.
|
| The path completion seems pretty good - as could be guessed from
| the blog post.
|
| I think this app sometimes confuses power with details/verbosity.
| There are some gold nuggets in there though.
| ChrisSD wrote:
| I think what's missing from this discussion is an emphasis on how
| layered Windows paths are.
|
| The Win32 paths are like an emulation layer. They parse the given
| path and produce a kernel path. Win32 implements all the weird
| history you know and love as well as things like `.` and `..`.
| You can use the `\\\?\\` prefix to escape this parsing and pass
| paths to the kernel.
|
| The NT kernel has paths like
| `\Device\HarddiskVolume2\path\to\file`. NT paths are much
| simpler. There are no restrictions on paths except that they
| can't contain empty components (notably they can contain nul). At
| this layer, `.` and `..` are legit filenames.
|
| However, it's the filesystem driver ultimately says what's a
| valid filename and what isn't.
| janvdberg wrote:
| These path shenanigans get even trickier when invoking WSL or
| Cygwin.
|
| Here's an alias from my Cygwin .bashrc (which took me way too
| long to figure out) where both the *nix and Windows style paths
| are invoked:
|
| alias ms='/cygdrive/c/Windows/System32/OpenSSH/ssh.exe -A -i
| 'C:\Users\me\\.ssh\mm-id_rsa' me@myserver'
| Calzifer wrote:
| Or MSYS2. Other tricky variants is using Windows commands
| (which typically use '/' as option marker), e.g.
| $ tasklist /V ERROR: argument/option invalid -
| "C:/msys64/V"
|
| or the time I wanted to use xmlstarlet and any xpath expression
| was interpreted as Windows path :(
|
| At least for MSYS2 the environment variable MSYS2_ARG_CONV_EXC
| can be used to prevent conversion.
| https://www.msys2.org/docs/filesystem-paths/#process-argumen...
| mynameisvlad wrote:
| I believe it's because it's executing a Windows program
| (ssh.exe) located in Cygwin's mount for your C: drive and that
| program therefore expects Windows-style paths.
| janvdberg wrote:
| I know now :)
| throwaway09223 wrote:
| The bit about allowing / as a path separator is one of my
| favorite bits of DOS/Windows trivia. As a unix guy it's fun to
| give a windows person a path with the slashes wrong like
| "z:/foo/bar", being corrected for a unix-ism, then having it
| actually work!
|
| In practice I think the biggest problem with using forward
| slashes on Windows is confusing programs which expect "/" to
| indicate program switches. The non-uniformity of shell parsing is
| also a big unix/win design difference.
| naikrovek wrote:
| yeah almost no one knows that forward slashes have been
| acceptable as path separators since (I wanna say) Win95.
| perhaps MSDOS.
| layer8 wrote:
| It doesn't work everywhere. For example tab completion in
| cmd.exe doesn't work for a path containing forward slashes
| (even when quoted), because forward slash is the prefix
| character for command-line options.
| naikrovek wrote:
| right but that's a cmd.exe thing, not a Windows thing.
|
| Windows supports it, CMD doesn't. programs that you run
| from a CMD prompt support other options flag syntaxes, so
| it's just a cmd.exe feature.
|
| CMD.exe is its own thing with its own backwards
| compatibility requirements and the case could be made that
| cmd.exe is "Windows" as much as anything else is, so I get
| it.
| chungy wrote:
| Since MS-DOS 2.0.
| throwaway09223 wrote:
| It actually predates MSDOS, and I believe dates back to PCDOS
| 2 when support for directories was first added.
| zwieback wrote:
| I'd qualify that as "almost no non-programmers know". Forward
| slashes are so useful in languages that use \ as an escape
| sequence that most programmers do know this.
| ambicapter wrote:
| Can you mix the two in one filepath though? '/' and '\\\'?
| levidos wrote:
| Depends on the interpreter, batch and PowerShell accepts
| both.
| [deleted]
| layer8 wrote:
| Yes. You can even write stuff like "cd foo/\bar\/baz".
| usrusr wrote:
| It's not really a new version of Windows if it does not introduce
| a new variation of universal path addressing to end all
| variations of universal path addressing.
| usefulcat wrote:
| > That's what sticking to a policy of total backwards
| compatibility for several decades gets you!
|
| Kind of? Surely unix file path conventions are of a similar age
| (if not older), yet somehow they seem to exhibit much less
| weirdness..
| NelsonMinar wrote:
| Add \\\wsl$ to the list of "weird paths". This one fits in the
| general schema the article is about but has the special meaning
| of "your files in the Linux VM hiding in the Windows box" (aka
| WSL). It's running as a Plan9 network filesystem.
| jeroenhd wrote:
| Is it a weird path? It's just a simulated computer with
| hostname wsl$. It looks like this is just a network share
| server running on the loopback interface. Similarly,
| \\\wsl.localhost\ will also show you your WSL files. The dollar
| sign has some weird special meaning to the Windows security
| system (trusted account, local computer, etc., in AD it's
| related to this mess:
| https://techcommunity.microsoft.com/t5/core-
| infrastructure-a...) but it's essentially just a standard
| Windows UNC paths.
| helloooooooo wrote:
| This is a better explainer:
| https://googleprojectzero.blogspot.com/2016/02/the-definitiv...
| sim7c00 wrote:
| you can create files and folders with illegal names like '.' or
| '..' with python. comedy ensues if you try to recursively go into
| . directory and explorer crashesh -_- . despite windows having
| rules, appearently not everything needs to play by it despite
| giving these expectations
| cryptonector wrote:
| > Say you, for whatever incomprehensible reason, need to access a
| file named .., which would normally be resolved to the parent
| directory during normalisation, you can do so via a literal
| device path.
|
| Oh no. No. Windows allows files to be named `..`?!
| Karellen wrote:
| > Windows allows files to be named `..`?!
|
| Maybe not?
|
| Under unix, if you create a symlink to a directory, e.g.
| `~/syslogs` is a symlink to `/var/log`, then `..` can be used
| to traverse the "true" parent directory. So `~/syslogs/../lib`
| will traverse `/var/log/..` and refer to `/var/lib`, not to
| `~/lib`.
|
| However, a "normalising" path interpreter will just take
| something like `~/syslogs/../lib` and change it to `~/lib`
| without consulting the filesystem.
|
| Given that (AIUI) Windows has supported symlinks for a while
| now (?), it's possible that files called `..` aren't actually
| allowed, but the ability to access `..` is still necessary.
|
| (Notably, the article does point out that filenames ending in
| `.` are disallowed - which should exclude `..` as a name one
| can give a file.)
| rxtdc wrote:
| Preferring a minimal look (and being immature) my desktop
| shortcuts for "This PC" and "Recycle Bin" have been renamed
| with two of the many invisible characters that windows allows.
|
| I also routinely use single extended unicode characters as root
| folder names and identifiers for various purposes.
|
| Using a search programme 'Everything", it's a lot easier to
| find things if I use something like pilcrow symbol as the root
| folder for any directory dedicated to text documents, when the
| alternative is to wade through results for 'documents', 'text',
| 'reading' or any combination of those words.
|
| For the same reason, I find I can make much more memorable
| associations. It helps me harness things relationally. I can
| preserve uncertainty and avoid the frustration and negativity
| of trying to make shades of grey and rose fit black and white
| patterns. It does sound a bit new age, but there's no doubt in
| my mind, flat heirachical alphanumeric patterns are
| restrictive, prescriptive, insufficient. For example, a lot of
| artists actively work to defy pidgeon holing. I still need
| identifiers.
|
| I mean, even if I wasn't into 'bleeding edge' culture,
| restrictions, problems and frustrations are the normal
| experience. I think this is illustrated by the unsatisfactory
| experiences that people find when they try to make id3 tagging
| "work".
|
| It's as close as I can get to banishing the pervasive 'what-if'
| heartbreak of WinFS being cancelled. Sadly it doesn't help at
| all make up for what 'Semantic Web' promised. But that's
| probably why I'm a believer in GPT and the like.
|
| Is it just me that can't help thinking they are products that
| have arisen from the need to make non-semantic computing useful
| again?
| jaclaz wrote:
| There are even more "strange" cases, JFYI:
|
| https://msfn.org/board/topic/131103-win_nt~bt-can-be-omitted...
| Jap2-0 wrote:
| I can't find a way to create one - so if Windows allows that,
| it's begrudgingly at best.
|
| You can put `..` earlier in the file name, though.
| jagged-chisel wrote:
| Seems like that violates the "can't end in a period" rule
| chungy wrote:
| Just using Cygwin will show that file names can indeed end in
| periods (and spaces). The article is very much restraining
| itself to the standard limitations imposed by the Win32 API,
| but not what the operating system actually allows. Case
| sensitivity has always been a thing, since Windows NT 3.1,
| for example; the "forbidden" characters are not so forbidden
| with the right file access APIs.
| ChrisSD wrote:
| Yes you can create files named `.` and `..`. However, any
| sensible filesystem driver will reject that name (spoiler:
| there does exist drivers that aren't sensible).
| [deleted]
| garaetjjte wrote:
| Maybe? https://i.imgur.com/ebo4Nd8.png
|
| But seriously: no, at least not on NTFS. This filename does
| have trailing space. Though it is enough to defeat Explorer,
| you cannot move or delete it and properties window is broken.
| Calzifer wrote:
| One thing that make Windows paths wired is that Windows API, NTFS
| and most Windows tools have different restrictions on file paths.
|
| NTFS would accept almost anything. The Windows API (I think of
| the old Win32 one) would apply most restrictions the article
| mentions.
|
| But for example not the normalization part. A filename can end
| with a space, no problem. That lead me once to a minor bug in
| .NET Framework. One of the path related functions (I think it was
| Directory.move) did not correctly apply this normalization and
| could produce directories with trailing whitespace. Good luck
| removing/fixing those in Windows Explorer.
| kelsolaar wrote:
| The 260 characters limit has been the bane of my existence. Even
| though it can be disabled in the registry there are gazillions
| software built against old APIs that will still not work. You
| also get really odd bugs when that plague hits you.
| simonsarris wrote:
| One of these tripped me up, the "Disallowed Names" section has a
| bunch of not-too-wordlike names, except one.
|
| When I made carefulwords.com, which I made because I wanted a
| thesaurus where you could just write eg
| https://carefulwords.com/book for "book" and get the results, I
| found out the hard way that you cannot make a file named "con" on
| Windows. Or "con.html", or any file extension. You can try to
| force this, make it via a script, but then programs like Git will
| hang when they come across it. So in my thesaurus the actual page
| is /con-word.html and I just have it rewrite to /con
| ChrisSD wrote:
| This has actually changed in Windows 11. You can use "con.html"
| without fear. "con" is still a bit of a problem. ".\con" will
| work but not a bare "con".
| noAnswer wrote:
| There is also this can of worms regarding translations. For
| example C:\Users will be shown as C:\Benutzer on a German system
| but still be C:\Users on the FS. "C:\Program Files (x86)" will
| show up as "C:\Programme (x86)" BUT "C:\Program Files" will stay
| the same (non translated).
|
| (I forgot how this topic is called though and on what layer it
| takes places.)
|
| Windows XP already could do it, but didn't for the most part. I
| remember, like ~18 years ago, I was a sysadmin. One user out of
| 50 got his "My Doc/Pictures/etc." in English but for me on the
| File-Server it was all in German. Very confusing.
| softfalcon wrote:
| After years of working exclusively on Windows, I took a job that
| required me to build file management, except now, on macOS and
| Linux (along with Windows).
|
| All I can say is, this article is the tip of the ice berg on
| Windows I/O weirdness. You really don't realize how strange it is
| until you are actively comparing it to an equivalent
| implementation on the two other competing operating systems day-
| to-day.
|
| Each of them has their quirks, but I think Windows takes the cake
| for "out there" hacks you can do to get things running. I
| sometimes like to ponder what the business case behind all of
| them was, then I Google it and find the real reasons are wilder
| than most ideas I can imagine.
|
| Fun stuff!
| ChuckNorris89 wrote:
| From my experience as an EE, working with serial ports is much
| nicer on Windows (COM1, COM2, etc.) than on Linux where serial
| ports are abstracted to files like /dev/ttyACM0 and has a lot
| more gotchas.
|
| PowerShell is also quite a powerful alternative to Bash/Mingw,
| although it came out much later.
|
| Windows might do some things differently than UNIX-like OSs,
| but it does them really well.
| mikub wrote:
| I just started with using serial ports on windows while doing
| some Raspberry Pico hobby projects. Something that I find
| strange is that every new device gets assigned a new comport,
| I mean let's say I do this for a while one day I will have a
| comport 100, 200 and so on. Is that right, or does it somehow
| reset the comports?
| zwieback wrote:
| That's how it works and generally it's to the user's
| advantage. We often set specific parameters based on the
| device's serial number so getting the same COM port is
| nice, sometimes the devices are so simple that you cannot
| query its serial number.
|
| Sometimes I'll do a "blank slate" and delete all my
| accumulated COM ports in Device Manager (need to enable
| "Show Hidden Devices").
| wolletd wrote:
| Technically, COM1, COM2 etc. are filenames as well. They are
| just special in that they are available everywhere. That's
| why you are not allowed to create any file named COM1 or
| such.
|
| But it's a DOS relic. Actually, Windows has a "Win32 Device
| Namespace" under "\\\\.\", which is something like /dev/ in
| Unixoids. COM1 is actually \\\\.\COM1:
| https://learn.microsoft.com/en-
| us/windows/win32/fileio/namin...
| pixl97 wrote:
| I deal with software that processes files on a Windows
| system... loves to break when people on other OS's subnet
| AUX, PRN, COM, File:Name, and tons of other unacceptable
| names (like 'file ').
|
| I'm glad our new releases work on Linux and we don't have
| to deal with that crap in 99.99% of cases now.
| riffic wrote:
| it's a CP/M relic of i'm not mistaken
|
| https://en.wikipedia.org/wiki/CP/M
| blooalien wrote:
| Wow! There's an operating system I ain't heard tell of in
| a good long while. That's the very first OS I used in a
| professional context. Got me my first computer store job
| (in my late "teens") on a CP/M system.
| throwaway09223 wrote:
| I've done quite a bit of work with serial ports on Windows,
| Linux and other unixes. I've also written a serial device
| driver.
|
| Your comment is very confusing to me. The serial ports are
| abstracted to a file on Windows just like on unixes - the
| file is actually discussed in the above article: \COM1
|
| Maybe you're talking about the old days where you would just
| outb 0x3f8? The modern interfaces are actually fairly
| similar.
| zwieback wrote:
| 0x3f8 IRQ4, 0x2f8 IRQ3 - still hardcoded in my brain 30yrs
| on!
| blooalien wrote:
| My "burned in" code snippet is "call -151" from Apple ][+
| days, to drop to the built-in 6502
| (dis)assembler/debugger.
| zwieback wrote:
| MONZ
|
| I spent a lot of time reading the disassmbly listing in
| the back of the manual to see what happens when I jump to
| the monitor.
| qalmakka wrote:
| COM ports on Windows are crap nowadays due to how crappy USB
| to serial adapters have become. I've seen Windows reassigning
| different COM names to the same device every single time it
| was unplugged due to it "remembering" what COM port was used
| previously. Needless to say, that was an anti-feature if
| there ever was one.
| dfox wrote:
| Windows tries to keep a long term identity of all of the
| device instances that it knows about (and in the idela
| world assign the same COM port numer to the same physical
| serial adapter). For USB this is supposed to be done by
| combination of VID, PID and serial number in device
| descriptor. But even early on there was a lot devices that
| had the serial number empty and thus Windows came up with
| some heuristics about when this method is unreliable and
| switches to identifying the device by its physical position
| on the USB bus. The whole mechanism is well intentioned,
| but in the end probably somewhat misguided because it is
| not exactly reliable and has surprising consequences even
| if it were reliable.
|
| As a side note: on modern Windows NT implementations the so
| called "registry bloat" is non-issue (anyone who tells your
| otherwise is trying to sell you something), but keeping
| list of every device that was ever plugged into the
| computer in there is somewhat ridiculous.
| InitialLastName wrote:
| > As a side note: on modern Windows NT implementations
| the so called "registry bloat" is non-issue
|
| How modern? I manage Windows 7 (transitioning to 10)
| machines that are used for QC in a hardware manufacturing
| environment that enumerate hundreds of devices (with
| mostly identical VID/PID) every week. We find that if we
| don't clear old devices out of the registry every so
| often, the enumeration time slows to a crawl.
| dfox wrote:
| That is kind of a niche use case ;)
|
| In the times when it was a real issue (I would hazard a
| guess that that means "before XP") the reason was that
| the original registry on disk format made every registry
| access more or less O(n) in bunch of things like the
| overall on disk hive size, total number of keys, number
| of subkeys in each of the keys along the path...
| mmis1000 wrote:
| It also do this for monitor or usb/bluetooth earphones.
| So you end up get earphone(2), monitor(2) even you never
| have a second one. The only way to fix it is delete the
| hidden device in device monitor and rename it back in
| monitor/audio manager.
|
| It's really a confusing thing to me that the script I use
| to change sound output and leveling suddenly didn't work
| after a bios/mobo software/whatever windows update and
| noticing the device have an appended (2).
| yndoendo wrote:
| And this is why I hate Windows in an industry automation
| environment. Dislike having to troubleshoot why that USB
| NIC or Serial device being destroyed by plugging it into
| another port. Had to write a PowerShell script for the
| USB NIC issue to reapply NIC settings with a reboot.
|
| Also, always locking an open file is repulsive. Other OSs
| allow for renaming an open file. Not Windows! Thumbs.db
| being locked because File Explorer keeps the file open /
| locked preventing deleting an empty folder and wastes so
| much time waiting for Windows to unlock the file.
|
| You have to pay me to use Windows!
| scoutt wrote:
| COM1 = CreateFile("COM1", ...) => Nice!
|
| COM9 = CreateFile("COM9", ...) => Nice!
|
| COM10 = CreateFile("\\\\\\\\.\\\COM10", ...) => NOT nice!
| Kwpolska wrote:
| How often do you end up with 10 COM ports?
| zwieback wrote:
| We do all the time. In industrial automation COM ports
| are still shockingly popular, although it's usually the
| USB emulated variety. On a lot of our development and on
| some of our production tools we end up with COM20 or
| COM30, not because we have that many running at one time
| but because over time we've plugged in that many distinct
| devices. Nowadays most drivers will assign COM(n+1) when
| they see a device with a new serial number.
| connicpu wrote:
| UART is available on nearly every microcontroller under
| the sun, and USB<->UART serial chips are super cheap, so
| it makes complete sense to me that'd become the defacto
| system for interfacing the automation controller with a
| computer
| InitialLastName wrote:
| Even beyond that, USB is available on many
| microcontrollers, a USB CDC device is dead simple to
| implement, the drivers are baked into every modern OS,
| and all the software developers operating at that layer
| already know how to interact with text streams. Add in
| the ease of debugging when you can just manually
| send/receive ASCII to operate a device, and you've got
| the makings of a standard practice.
| rogerbinns wrote:
| If you use USB dongles for serial adapters, then each
| path through USB is assigned a different COM number when
| you plug it in. For example if you plug into USB
| controller 2, port 3 which goes to a hub, and then you
| plug into port 2 that gets a number. Now plug the same
| thing into a different port and it will get another COM
| number.
|
| Under the hood this is because the USB devices do not
| have the (optional) unique serial number (or in some
| cases they all get the same serial number).
|
| https://devblogs.microsoft.com/oldnewthing/20041110-00/?p
| =37...
| JohnFen wrote:
| Interesting. I very much prefer working with serial ports
| under Linux than Windows. It's more straightforward and
| easier to engage in fine control.
| BrandoElFollito wrote:
| So do I, I find the addressing more consistent, too.
|
| It used to be completely predictable when I was working
| with drivers on 1994 (patching the code), then less
| predictable when hardware for more diverse, and predictable
| again (or at least "always the same") with UUIDs.
|
| It was always amateur/hobby dev or sysadmin so I may have
| had the wrong impression.
| klysm wrote:
| I've found the assignment of com ports in windows really
| annoying
| fsckboy wrote:
| "get-a-byte, get-a-byte, get-a-byte, byte, byte" - Dave Cutler
| https://retrocomputing.stackexchange.com/questions/14150/how...
|
| > _Windows I /O weirdness. You really don't realize how strange
| it is until you_
|
| "can I get a wut... WUT?" - Developers, Developers, Developers
|
| ====
|
| it's just hard for people today to understand what a revolution
| stdin and stdout was, full 8 bit but sticking with ASCII as
| much as possible. There was nothing about it that limited
| Unices from having whatever performant I/O underneath, but it
| gave programmers at the terminal the ability to get a lot done
| right from the command line.
|
| The web itself is an extension of stdin and stdout, and the
| subsequent encrusting of simple HTTP/HTML/ _et al_ standards
| with layer upon layer of goop that calls for binary blobs can
| be seen as the invasion of Cutlerian-like mentalities. It 's
| sad that linux so successfully took over that all the people
| who we were happy to let use IIS and ASP and ActiveX had to
| come over to this side with their ideas. No idea of which is
| bad, but which together are incoherent.
|
| client server FTW, bring it back
| chasil wrote:
| At least Windows paths are not as clunky as Cutler's prior
| operating system, VAX/VMS Version 1.00 21-AUG-1978 15:54.
| $ create/directory dm0:[foobar] $ set default
| dm0:[foobar] $ copy DM0:[SYSMGR]SYSTARTUP.COM
| example.txt $ dir/full DIRECTORY
| DM0:[FOOBAR] 21-APR-23 14:42 EXAMPLE.TXT;1
| (615,2) 0./0. 21-APR-23 14:42 [1,4] [RWED,RWED,RWED,RE]
| TOTAL OF 0./0. BLOCKS IN 1. FILE
| dhosek wrote:
| I rather liked VMS conventions as it was immediately
| obvious what was the directory part of the path and what
| was the filename and the device. You could create virtual
| devices as well, so for example, with VMS TeX, I had
| TEX_ROOT defined to point to the root of the TeX
| distribution and you would have TEX_ROOT:[INPUTS] for the
| input directory, TEX_ROOT:[MF] for Metafont files,
| TEX_ROOT:[EXE] for executables, etc. and everything was
| logically arranged. CLD files were another wonderful thing
| where you could define your CLI interface in an external
| file and let the OS handle argument and option parsing for
| you.
| hilbert42 wrote:
| _" All I can say is, this article is the tip of the ice berg on
| Windows I/O weirdness."_
|
| Well, then, is there a more detailed summary than this one
| that's accessible?
|
| This one looks very useful and I'll use it, but to make the
| point about more info it'd be nice to know how they differ
| across Windows versions.
|
| For example, I've never been sure about path length of 260 and
| file length of 255. I seem to recall these were a little
| different in earlier versions, f/l being 254 for instance. Can
| anyone clear that up for me?
|
| Incidentally, I hit the 255/260 limit regularly, it's damn
| nuisance when copying stops because the path is say 296 or 320,
| or more.
| ChrisSD wrote:
| There are some APIs that have a lower limit than 260. But the
| limits can be bypassed using `\\\?\\` prefixed paths (except
| when using SetCurrentDirectory) or by enabling long paths
| https://learn.microsoft.com/en-
| us/windows/win32/fileio/maxim...
| arka2147483647 wrote:
| Windows.h defines MAX_PATH as 260.
|
| Many apps do something like
|
| char path[MAX_PATH]
|
| In that case, no amount of prefixing will help you, if
| random app enforces the limit.
| rightbyte wrote:
| > All I can say is, this article is the tip of the ice berg on
| Windows I/O weirdness. You really don't realize how strange it
| is until
|
| In one way it is beautiful. "Laid cards lies", you know. Don't
| mess with the user for your conception of Agile Clean Extreme
| Code (tm). Each stupid design decision is forever.
|
| Windows .bat files win the awfulness and quirkyness contest
| with sh with a razor thin margin. And both are awesome.
| Kwpolska wrote:
| Windows has PowerShell these days, and it is actually usable
| and not at all awful (if a little quirky) compared to bat
| files.
| maccard wrote:
| Powershells biggest problem is error handling. There's just
| no equivalent to set -euo pipefail
|
| It ends up with manual error handling to handle the case of
| bat, ps1 and exe's
| WorldMaker wrote:
| `$ErrorActionPreference = "Stop"` is very similar and
| does that for all of PS1 cmdlets. You still have to check
| $LastExitCode manually for BAT/EXEs, though.
|
| `$PSNativeCommandUseErrorActionPreference = $true` is an
| experimental flag [1] as of PowerShell 7.3 that applies
| the same $ErrorActionPreference to BAT/EXEs (native
| commands), stopping (if $ErrorActionPreference is "Stop")
| on any write to stderr or any non-zero return value.
|
| [1] https://learn.microsoft.com/en-
| us/powershell/module/microsof...
| arka2147483647 wrote:
| Sadly, the powershell windows ships with is 5.1.
|
| Why?
|
| Because of backwards Compatibilytytytyy
| WorldMaker wrote:
| `winget install Microsoft.PowerShell`
|
| 5.1 was the last "Windows-specific"/"Windows-only"
| PowerShell (and is still branded "Windows PowerShell"
| more than "PowerShell") before it went full cross-
| platform (and open source). It's an easy install for
| PowerShell 7+ and absolutely worth installing. If you are
| using tools like the modern Windows Terminal and VS Code
| they automatically pick up PowerShell 7+ installations
| (and switch to them as default over the bundled "Windows
| PowerShell"), so the above command line really is the one
| and only step.
| pxc wrote:
| You can also install the latest PowerShell Core (the
| open-source, cross-platform releases we're talking about)
| via Scoop, which is a package manager for Windows that
| works even if you don't have admin rights:
| https://scoop.sh/#/apps?q=pwsh&s=0&d=1&o=true
| moremetadata wrote:
| [dead]
| _a_a_a_ wrote:
| A bit like finding in favour of london's old, awful slums
| by comparing them to the Somme.
|
| "Well yes, kind of..."
| 2devnull wrote:
| "Powershell(tm): Not Entirely Awful (if a little quirky)!"
|
| Seems worth investing a lot time into given Microsoft's
| history of not rug pulling developers.
| connicpu wrote:
| The first release was 16 years ago and they're still
| making new releases of it so I'd say it's definitely here
| to stay ;)
| WorldMaker wrote:
| Also it is MIT-licensed open source today:
| https://github.com/PowerShell/PowerShell
| blibble wrote:
| powershell is still built on crap
|
| I had it randomly throwing exceptions the other day that a
| path was too long
|
| (it was only about 300 characters...)
| dingdingdang wrote:
| It also adds it's own special brand of crap .. as in
| after trying 10x different ways (not kidding: https://soc
| ial.technet.microsoft.com/wiki/contents/articles/...) of
| executing an external ffmpeg command over several hrs I
| eventually wrote a one line .bat file* and was done with
| it. Never again.
|
| * for %%a in ("*.mp4") do ffmpeg -i "%%a" -vcodec libx265
| -crf 26 -tune animation "%%~na.mkv"
| mdaniel wrote:
| The irony of commenting that under an article that cites
| the maximum path cannot exceed 260 characters is not lost
| on me
| mauvehaus wrote:
| The maximum path length of the NT kernel is
| 3276-something UCS-2 characters. 260 is a limit imposed
| by the legacy Win32 interfaces IIRC. I believe the
| W-interfaces get you the full-fat version, it's just that
| they're so inconsistently used as to all but guarantee
| that something you need will have problems.
|
| The user-mode stuff is kind of a mess. The kernel-mode
| stuff is comparatively orthogonal.
| blibble wrote:
| how is that ironic?
| inferiorhuman wrote:
| It's like rain on your wedding day
| noAnswer wrote:
| But the path length can be longer! They just don't care
| and offload the work.
| levidos wrote:
| PowerShell is an absolutely amazing scripting language; it
| gets things done way quicker than Bash, because it's object
| oriented, and you don't have to call external tools to get
| anything done (sed, grep, find, touch, curl, etc.). It can
| even run raw C# code for you.
| xp84 wrote:
| This definitely falls into the category for me of "things
| that I wasn't there for."
|
| Because I learned computers when DOS was a thing, I will
| always be able to write a .bat or use CMD when necessary,
| but having been on the UNIX/Linux side since 2003, I
| didn't learn C# or PowerShell but rather bash, php, ruby.
| So while I'm friendly with modern Windows now that they
| closed up the "is it a stable OS" gap with Apple, I don't
| really know what to do in PowerShell and am more likely
| to use WSL!
| WorldMaker wrote:
| It's worth learning at any age, especially now that it is
| an open-source, cross-platform shell. The PS Koans [1]
| that recently showed up on HN seemed an interesting way
| to try to learn it.
|
| [1] https://github.com/vexx32/PSKoans
| klodolph wrote:
| I think the margin of bat vs sh is a larger margin. Batch
| files use gotos, the entire file is reread for each line,
| etc.
|
| My main complaint about sh and Unix is that environment
| variables are mostly inherited from parent processes, and
| that can be awkward.
|
| (And Windows at least, these days, is seeing a lot more
| PowerShell adoption.)
| rightbyte wrote:
| > I think the margin of bat vs sh is a larger margin [...]
| the entire file is reread for each line
|
| Ye agreed. Separation of data and code always was a
| mistake.
|
| I wonder if this feature of bat files was like a thing once
| as a "best practice"? Practically, you should only append
| lines I guess. When I close my eyes, I can see a DOS batch
| file doing actual batch job processing, appending to itself
| and becoming a intertwined log and execution script.
| conductr wrote:
| I'm not familiar with inner workings, but simply moving files
| feels odd in Windows compared to macOS. If it's a big lift in
| terms of data size or file/folder counts it's most obvious but
| it feels like Windows literally copies the files into memory,
| then rewrites them on disk or something similar that has
| results in negative performance and a long running
| cut/copy/paste dialog box. I've had some of these run for hours
| on decent hardware (SSD, etc) for what I consider small
| datasets (couple GB). It's been a major Windows gripe of mine
| for years now.
|
| Meanwhile macOS appears to just change an internal link to the
| data that's already written on disk. As such, it's usually so
| very fast compared to Windows.
| fencepost wrote:
| A lot of this depends on whether you're crossing devices. If
| you think of drive letters as mount points it may make more
| sense - if you're moving between mountable filesystems
| obviously a move has to be a copy-then-delete; if you're
| remaining on the same filesystem a move can typically be a
| rewriting of indexing information only with very limited data
| rewriting.
|
| One other thing that can be an issue particularly on NTFS
| with ACLs is that moving files typically retains their
| ownership and permissions, while copying files typically
| inherits the ownership and permissions of the destination.
| This can bite you if as an administrator you're moving data
| from one user's account to another because a move will leave
| the original owner still owning the files.
| Strom wrote:
| Windows does literally copy (parts of) files into memory.
| More precisely it's Windows Defender Real-Time Protection.
| It's a real menace when you're dealing with a lot of small
| files, e.g. node_modules.
|
| Windows Explorer is also slow for an unknown reason.
|
| Doing file operations through the API with Real-Time
| Protection turned off is several orders of magnitude faster
| in the case of small files. It's crazy stuff.
| pixl97 wrote:
| Eh, with moving files on Windows in the same security context
| it is 'generally' pretty fast on the same drive.... Are you
| sure you didn't paste in a directory that is setting new
| security permissions on all files?
| WorldMaker wrote:
| Windows File Explorer does a lot of extra work to get a sense
| of file sizes and other metadata to try to keep the UI
| looking fresh/interesting/useful to someone watching the job
| in real time.
|
| If you need to seriously move/copy lots of files or lots of
| data in Windows it is generally a good idea to use shell
| commands. Robocopy [1], especially, is one of the strongest
| tools you can learn on Windows. (It gets very close to being
| Windows' native rsync.)
|
| [1] https://learn.microsoft.com/en-us/windows-
| server/administrat...
| ripley12 wrote:
| Been a while since I looked at the details, but Explorer's
| file management is generally slow compared to what you can do
| with the actual Win32 APIs.
| armarr wrote:
| That's likely to be due to the NTFS file system. Another
| piece of legacy Windows drags along
| e4m2 wrote:
| A lower-level, more security oriented look at some of the same
| issues: https://googleprojectzero.blogspot.com/2016/02/the-
| definitiv...
| nicetryguy wrote:
| As a related anecdote, you NEVER want to put whitespace in
| filepaths. Ever. Tons of programs will just break, even in 2023,
| even Microsoft ones (looking at you Powershell), and i'd imagine
| forevermore.
|
| Your life will end up as a series of awful MYPATH~1\ kludges. You
| have been warned.
|
| EDIT: There seems to be different camps here and perhaps a
| generational divide. Maybe the kids haven't (or never will be)
| burned with this one, but i've seen too much, wasted too many
| hours, wrote too many workarounds, and will forever remain
| #TeamNoWhiteSpace.
| dmd wrote:
| Counterpoint: Nearly all of my (thousands of) users use
| whitespace in basically every single folder name and file name
| every single day (which are usually titles of papers or
| experiments), and none of them ever have any problems.
|
| When writing software I always make sure that all my test data
| has spaces - and not just the normal one but weird unicode ones
| too - in paths and filenames.
| wvenable wrote:
| Microsoft called the directory where programs are installed
| "Program Files" and documents in "My Documents" specifically so
| that developers would have to learn to deal with spaces
| immediately.
| ygra wrote:
| Curious about PowerShell misbehaving there. Care to share any
| details? Typically you never want to handle paths to files as
| strings, or at least resolve them to File/DirectoryInfo objects
| as soon as possible. But the only real blunder regarding paths
| in PowerShell I'm aware of is the Path parameter to various
| cmdlets and having [] in file names (which is why LiteralPath
| always exists alongside Path parameters).
| 13of40 wrote:
| Came here to mention []. For those who haven't looked it up,
| in addition to * and ? as wildcard characters, PowerShell
| supports "[xyz]" and "[a-z]", meaning "x y or z" and
| "anything from a to z", so "[a-z]alls" would match calls,
| falls, walls, etc.
|
| I knew the guy who tested the feature in PowerShell and he
| was deeply frustrated over the fact that there was no way to
| escape some sequences. For example, you can loosely match
| "[abc]" with something like "?abc?" or "[[]abc?" because the
| [[] indicates exactly one opening square bracket but there's
| no way (as far as I know) to say "this section ends with a
| square bracket".
|
| The PM really wanted that feature, though, even though you
| could probably count on one hand how many times it's been
| used in real life.
|
| Edit: That guy down there corrected my example.
| justeleblanc wrote:
| Uh... You can escape these sequences. But the escape
| character isn't the backslash for obvious reasons (it's the
| path separator). It's the backtick. > ni
| '[abc]' Directory: C:\Users\<user>
| Mode LastWriteTime Length Name
| ---- ------------- ------ ----
| -a--- 21/04/2023 12:23 0 [abc]
| > gi '[abc]' # no output - would have matched
| files named a, b, or c > gi '`[abc`]'
| Directory: C:\Users\<user> Mode
| LastWriteTime Length Name ----
| ------------- ------ ---- -a---
| 21/04/2023 12:23 0 [abc] > gi
| -LiteralPath '[abc]' Directory:
| C:\Users\<user> Mode
| LastWriteTime Length Name ----
| ------------- ------ ---- -a---
| 21/04/2023 12:23 0 [abc] > gi
| -lp '[abc]' Directory: C:\Users\<user>
| Mode LastWriteTime Length Name
| ---- ------------- ------ ----
| -a--- 21/04/2023 12:23 0 [abc]
|
| As far as I know that's been in powershell forever.
| 13of40 wrote:
| Ah, OK. This works because you're using single quotes on
| your strings, meaning the escape sequence `[ is preserved
| until it reaches the globbing layer. This will not work,
| for example... gi "`[abc`]"
|
| ...because the `[ will be processed before the string
| gets sent to globbing, meaning the back tick will be
| removed. This would work with double quotes:
| gi "``[abc``]"
|
| ...because at the command line level this will evaluate
| to `[abc`] and the globbing will know that the square
| brackets are literal. So I will concede and downgrade my
| complaint from "impossible" to merely "overly complicated
| for the layman".
| justeleblanc wrote:
| Anyone used to scripting languages knows there's a
| difference between single quotes and double quotes for
| escaping.
| LoganDark wrote:
| > Anyone used to scripting languages knows there's a
| difference between single quotes and double quotes for
| escaping.
|
| Not necessarily. There are all sorts of scripting
| languages where single and double quotes are identical.
|
| It is, however, super common for there to be escaping
| differences in _shell_ scripting languages.
| nicetryguy wrote:
| $args[x] breaks. Get-Content breaks. probably more.
|
| ...admittedly i'm not the best with Powershell. There are
| probably workarounds or things that i did wrong.
| justeleblanc wrote:
| Just put them in quotes. PowerShell is infinitely better
| than bash/zsh/... at handling spaces. You're spreading
| nonsense.
| nicetryguy wrote:
| > Just use quotes bro
|
| Man, it's never that simple. How about generating
| Powershell scripts with quotes? Sometimes you are dealing
| with escape characters \", sometimes you aren't. Why
| bother with all that? Hell, versions of software and
| compilers change such behaviors either on purpose or by
| accident all the time. It just gets messy dude.
| C:\NEVER_~1\AGAIN_~1.NOP
|
| Perhaps i'm a dinosaur still scarred by the golden olden
| days, but, there is no convincing me that whitespace in
| filenames or URLs is ever a good idea.
| justeleblanc wrote:
| I don't know what to tell you. You gave two examples that
| aren't actually issues. But if there's no convincing you,
| I won't bother. Have a nice day.
| nicetryguy wrote:
| > You gave two examples that aren't actually issues.
|
| Man, i've had config files break because i saved them in
| UTF-8 instead of ANSI. I hope to god you never have to
| experience the horror... You also have a nice day.
| LoganDark wrote:
| > Man, i've had config files break because i saved them
| in UTF-8 instead of ANSI.
|
| Isn't this because random Windows tools add garbage bytes
| to the start of the file (a "byte order marker") and then
| _don 't tell you they did this_?
|
| It's not the encoding that's the problem, it's the
| garbage bytes.
|
| P.S. I do know of exactly one program that only accepts
| UTF-16 for its config file, and won't accept UTF-8 _or_
| ANSI.
| justeleblanc wrote:
| This is getting less and less relevant.
| numpad0 wrote:
| Non-ASCII characters too, if situation allows. Lots of apps
| aren't tested to work with that whatever "new" API.
| jeroenhd wrote:
| I know someone with an e in his name. He once made the
| mistake of using that character in his username.
|
| Windows broke. Programs broke. The registry broke. Special
| characters work well enough, but never use them in standard
| directory paths.
|
| If you want your folder to be written correctly in your own
| language, you can use desktop.ini to correct the name of the
| folder in Windows Explorer:
| [.ShellClassInfo] LocalizedResourceName=Fancy Name
|
| Put that into a desktop.ini file, assign it the right
| attributes (`attrib +S +H desktop.ini`) and your folder will
| look right in any program that uses the shell API without any
| risk of breaking programs.
| eru wrote:
| Tools in the Unix world aren't too robust in that regard
| either. Gnu Make is particularly egregious.
| Joker_vD wrote:
| There is also gnu patch: the filenames in the unified patch
| file format can't contain spaces and can't be quoted so...
| just don't use spaces, I guess.
| mdaniel wrote:
| how old is your copy of patch? $ cat >
| my.patch <<PATCH diff --git a/alpha beta/charlie
| b/alpha beta/charlie new file mode 100644
| index 0000000..3b18e51 --- /dev/null +++
| b/alpha beta/charlie @@ -0,0 +1 @@
| +hello world PATCH $ patch -p1 < my.patch
| patching file 'alpha beta/charlie' $ patch
| --version GNU patch 2.7.6
| mynameisvlad wrote:
| > maybe the kids
|
| Really? You posted a divisive and absolute statement and people
| pointed out how it's not accurate. Considering "Program Files",
| a _very_ common Windows path, was introduced back in Windows 95
| (https://devblogs.microsoft.com/oldnewthing/20120307-00/?p=81..
| .), almost 30 years ago, it's pretty hard to blame it on "the
| kids".
|
| That doesn't sound like "those damn kids and their newfangled
| apps that can handle spaces, they'll never know the pain I went
| through!!!" and more like "I said something that was clearly
| not accurate and people pushed back".
|
| The whole thing reeks of
| https://www.mouser.com/blog/Portals/11/mrb-singularity-f1.pn...
| NikolaNovak wrote:
| This is a fascinating weekly topic for me :-)
|
| I grew up in days of DOS 3 and onward, so I am basically
| physically incapable of using whitespace in filenames. And
| frequently I'll replace white spaces with underscores on files
| shared with me.
|
| Fascinatingly though (and sometimes irritatingly), several of
| my mentors have cautioned me to drop this habit as I get
| promoted. At management level, they all use white space and
| dots haphazardly, and they apparently perceive underscored
| filenames negatively. This goes up drastically at executive
| level.
| dr-detroit wrote:
| [dead]
| Joker_vD wrote:
| That's... definitely not as dramatic as you try to make it
| sound. "Program Files" and "Documents and Settings" have been
| around for almost three decades, and most programs work just
| fine with files living somewhere inside those paths.
| mynameisvlad wrote:
| I mean, let's go past "Program Files" to "Program Files
| (x86)" which has both spaces and special characters, and
| which was moved from the former.
|
| And most programs are able to handle _that_ just fine.
| mdaniel wrote:
| And the "cherry on top" is there's an _env-var_ named
| `%ProgramFiles(x86)%` (aka `${env:ProgramFiles(x86)}` in
| PS) that points to where that directory actually lives
| since AFAIK it 's controlled by a registry entry
| c0nsumer wrote:
| This is a great article and really illustrates just how hard
| Windows works to be backwards compatible.
|
| Lots of these (eg: the COM/LPT stuff) could be dropped and
| wouldn't affect most people either way, but for those things
| depending on it, it would be a profoundly breaking change.
| tragomaskhalos wrote:
| No coverage of this nonsense would be complete without also
| mentioning that CON, AUX and PRN and a couple of others are
| verboten as filenames in Win. Although apparently you can defeat
| this via e.g. \\\?\C:\con
| jeroenhd wrote:
| One of the early stupid annoying teenager programs I wrote was
| a tool that would spam your desktop with CON.001, CON.002, and
| so on through the \\\?\ trick.
|
| Windows explorer could not delete the file. You have to specify
| the \\\?\ path to get the delete call to work, but that didn't
| work well with cmd.exe's `del` command.
|
| I've since used these files to create directories that can't be
| deleted by automated cleanups and such, like a special folder
| in %TEMP% that one program needed but didn't create on its own.
| dr-detroit wrote:
| [dead]
| loudgas wrote:
| The article mentions this in the Disallowed Names section.
| 0xbadcafebee wrote:
| If they could slowly start adopting Unix file paths and slowly
| phase out Windows file paths, I think more people would start to
| use Windows. I would love to _want_ to use Windows. It 's a
| platform that has first class hardware support and paid support,
| and it's designed (well, in theory) with users and application
| platforms in mind. And it actually has a bunch of advanced
| features that Linux and Mac doesn't have.
| naikrovek wrote:
| people don't choose to use Windows or not based on the file and
| path syntax.
| chasil wrote:
| Windows was built with a POSIX layer; it already does this.
|
| "Broad software compatibility was initially achieved with
| support for several API 'personalities', including Windows API,
| POSIX, and OS/2 APIs - the latter two were phased out starting
| with Windows XP."
|
| https://en.wikipedia.org/wiki/Windows_NT
| wvenable wrote:
| Why would it matter? 99% of these are effectively edge cases
| and feature-wise they're pretty equivalent.
| proactivesvcs wrote:
| It'd be nice if Microsoft read this list and adjusted their
| software, like perhaps File Explorer, to be able to read and
| write this data. Or at least delete it.
| antx wrote:
| "Legal" characters is a fun topic.
|
| I realized at some point that there is a discrepancy between
| what's allowed on the file system, and "Windows" itself (or, more
| exactly, the programs running on Windows and using its APIs to
| communicate with said file system.
|
| In this case, NTFS, totally allows for "illegal" characters such
| as < > : " | ? * etc... pretty much everything except / and \,
| and \0, I think.
|
| This makes for funny situations, where sometimes Windows programs
| cannot deal with that. At best, they can't read, write or rename
| them... at worse they'll crash, which is always fun.
| ambicapter wrote:
| This might be some run-of-the-mill weirdness, but I was using a
| Microsoft file globbing library recently
| (https://learn.microsoft.com/en-us/dotnet/api/microsoft.exten...)
| and it handed back filepaths with forward slashes (instead of
| double-backslash), even on Windows (and in .Net Framework). I
| don't know if this is a library someone developed for .Net Core
| and forgot that it also was going to be used in .Net Framework?
| Anyways, another reason I don't like the occasional dip into
| Windows I have to do at my job (which is 80% mac/linux).
| dataflow wrote:
| The (second-)worst offense I'm aware of here is that alternate
| data stream names can have otherwise special characters in them,
| like backslashes. So if you (for example) want to strip off the
| last path component, you technically cannot do this by just
| stripping everything after the last backslash.
|
| In fact this probably isn't the worst thing - it's even worse
| than this. Because you first need to strip off the prefix that
| represent the volume (like C:\\) before you can look for a colon.
| But the prefix can be something like \\\\.\C:\ or
| \\\\.\HarddiskVolume2\, or even
| \\\?\GLOBALROOT\DosDevices\HarddiskVolume2\\. Or it can be any
| mount point _inside_ another volume! (Remember that feature
| inside Disk Management?)
|
| Moreover you can't even assume the colon and alternate data
| streams are even a thing on the file system - it's an NTFS
| feature. So you gotta query the file system name first. And if
| the file system is something else with its own special syntax you
| don't know, then in general you can't find the file name and
| strip the last component at all.
|
| All of which I think means it's impossible to figure out the
| prefix length without performing syscalls on the target system,
| and that the answer might vary if the mounts change at run time.
| flangola7 wrote:
| What is an alternate data stream?
| dataflow wrote:
| C:\foo has a default (primary) data stream; the name of that
| stream is empty, so it's omitted entirely when writing the
| name. But the file can also have C:\foo:bar on NTFS. It's a
| different stream that's part of the original. (Look up "NTFS
| ADS" or just "NTFS streams".) These are often used to store
| information tied to a file that shouldn't affect the file
| contents.
| KMag wrote:
| In the late 1990s, there was a bug in MS IIS where if you
| requested http://example.com/page.php , it would execute
| the PHP script, but if you requested
| http://example.com/page.php: , it would give you the PHP
| source code. Even more than today, it was common to hard-
| code database connection details, including passwords, into
| the source code.
| Calzifer wrote:
| Any data stream which is not the first one.
|
| Data stream is basically the file content and on NTFS a file
| can have more than one. In practice it is comparable to
| extended attributes in the Linux world but somewhat superior.
|
| But like extended attributes it doesn't seem to have too much
| real world use. The only use case for alternate data streams
| I can remember are the "this file was downloaded from the
| internet, do you really want to run it" warnings. In such
| cases the browser attached a standardized marker as alternate
| data stream to the file.
| ajcoll5 wrote:
| If you have macOS clients connecting to an SMB file share
| hosed by a Windows server they use alternate data streams
| to store resource forks - like fonts. Makes for a fun 'oh
| shit' moment if you go to zip up files on Windows to
| archive, then realize you're missing data when you later
| unzip as most compression applications don't keep them.
| LoganDark wrote:
| > they use alternate data streams to store resource forks
| - like fonts
|
| macOS stores fonts in resource forks? I'm confused, what
| use does this have and what happens when you accidentally
| miss them?
| jra_samba wrote:
| Oh contraire :-) .Alternate data streams are widely used by
| virus writers and spies using them to exfiltrate data from
| foreign (to the spy) government and corporate Windows IT
| systems.
|
| You think I jest ? Look up the leaked source code for the
| US government spy tooling. They hide data to be exfiltrated
| in an ADS on the root directory of the share :-).
|
| I finally realized ADS were the mother of bad ideas when
| Ted Tso responded to me asking why I couldn't have them in
| Linux for the umpteenth time by showing me a Windows task
| manager screenshot of Myfile.txt as an actively running
| process.
|
| If the ADS ends in .exe then Windows will happily run it
| :-).
| nereye wrote:
| That standardized marker is also known as the 'Mark of the
| Web' (MOTW) in case you want to search for more details
| about it.
|
| In general, there is a tool which comes with the
| SysInternals suite that allows you to see which files have
| streams and their size:
|
| https://learn.microsoft.com/en-
| us/sysinternals/downloads/str...
| dataflow wrote:
| It's used for other things too. Like modern file
| compression (compact /exe:lzx).
| Alupis wrote:
| And used to be used to hide malicious software back in
| the early days.
| Calzifer wrote:
| Are you sure we mean the same thing?
|
| You seem to talk about a specific command line argument
| of the compact command with a Windows typical (and IMO
| ugly) option style with '/' instead of '--' as option
| marker and ':' instead of '=' as option value separator.
|
| But that would not be directly related to ADS and I
| cannot imagine a good use case where the compact command
| should use ADS.
| dataflow wrote:
| Yes, look up WofCompressedData. It's the stream name
| ultimately used by that command.
| Calzifer wrote:
| Thanks, with this keyword I found https://devblogs.micros
| oft.com/oldnewthing/20190618-00/?p=10...
|
| In context of ADS the first thing I imagined was storing
| the compressed and uncompressed file alongside. (which is
| rather silly, why compress at all)
|
| This use case is also kinda strange. Have the compressed
| content as ADS, the primary contend filled with 0 as
| sparse and fill it when needed/accessed. :/
| ChrisSD wrote:
| A stream name is somewhat more limited than that:
|
| > All Unicode characters are legal in a streamname component
| except the following:
|
| > * The characters \ / :
|
| > * Control character 0x00.
|
| > * A streamname MUST be no more than 255 characters in length.
|
| >
|
| > A zero-length streamname denotes the default stream.
|
| https://learn.microsoft.com/en-us/openspecs/windows_protocol...
| dataflow wrote:
| Oops, thanks for the correction! I must've seen this with
| other characters (most likely double quotes) and not realized
| slashes and backslashes are an exception.
|
| Though ironically that still doesn't help you strip the last
| component, since it could still be a volume mount point. Like
| you don't want C:\mnt\\..\foo to suddenly become C:\foo, just
| like how you don't want \\\\.\Server\Share1\\..\Share2 to
| become \\\\.\Server\Share2, or for
| \\\\.\C:\\..\HarddiskVolume1 to become \\\\.\HarddiskVolume1,
| etc.
| cryptonector wrote:
| > Moreover you can't even assume the colon and alternate data
| streams are even a thing on the file system - it's an NTFS
| feature. So you gotta query the file system name first. And if
| the file system is something else with its own special syntax
| you don't know, then in general you can't find the file name
| and strip the last component at all.
|
| If the :stream syntax is not FS-specific then you can parse the
| data stream name out statically in almost every case. Yes, you
| have to work out the prefix, but you can mostly do that
| statically too, I _think_ :
|
| > In fact this might not even be the worst thing - it's even
| worse than this because you first need to strip off the prefix
| that represent the volume (like C:\\) before you can look for a
| colon. But the prefix can be something like \\\\.\C:\ or
| \\\\.\HarddiskVolume2\, or even
| \\\?\GLOBALROOT\DosDevices\HarddiskVolume2\\. Or it can be any
| mount point inside another volume! Which I think means it's
| impossible to figure out the prefix length without performing
| syscalls on the target system, and that the answer might vary
| if the mounts change at run time.
|
| The prefix of `\\\\.\C:\Foo:Bar` is `\\\\.\C:` as `C:` couldn't
| be a file name. The prefix of `\\\\.\HarddiskVolume2\Foo:Bar`
| is `\\\\.\HarddiskVolume2` because the volume name ends at the
| backslash. The prefix of
| `\\\?\GLOBALROOT\DosDevices\HarddiskVolume2\Foo:Bar`... can be
| harder to determine but it doesn't matter because clearly there
| is no letter drive name in sight since a letter drive name
| would be... a single letter, but if the volume name were a
| single letter then it might require using system calls to
| resolve it (`\\\?\GLOBALROOT\DosDevices\X\Y:Z\A:B` is harder to
| parse because X might be the volume name, or maybe Y: might be
| the letter drive and X might be part of the path prefix).
| dataflow wrote:
| > If the :stream syntax is not FS-specific
|
| It is, I believe, as I alluded to in the comment.
|
| > `\\\?\GLOBALROOT\DosDevices\X\Y:Z\A:B` is harder to parse
|
| As in, this is impossible to do statically in the general
| case - those names aren't guaranteed to look like that. See
| the note I had added about mount points. Remember C:\mnt can
| itself be the mount point of a volume instead of a drive
| letter. (Junctions present a similar problem, but at least
| for those, you can make an argument that they're intended to
| look like physical folders, and treat them similarly. With
| mount points, you might not have that intention - you might
| be just trying to go over 26 drive letters.)
| fifticon wrote:
| the worst thing about windows paths is how unintegrated it all
| is. You can navigate into all kinds of weird paths in the win32
| com shell (file explorer), which is itself possibly the pinnacle
| of executed design MS ever achieved. But those paths you build..
| You can't copy them to the clipboard, you can't serialize them,
| you can't move them between various tools, and particularly not
| to the command prompt nor to socalled Power"shell. If there ever
| was a continent of independent fiefdoms, windows is it :-/. If
| you don't know what i am talking about, try navigating to your
| android phone's image folder in file explorer. Next, then try to
| USE that path in powershell or cmd, to copy those image files..
| good luck. there must certainly have been some moron in charge to
| make SURE things couldnt interoperate on windows.. in spite of
| them having explorers design.
| [deleted]
| jaclaz wrote:
| Another lesser known fact:
|
| The volume id string (what you get with mountvol) is - at least
| up to Windows 10[1] a UUID version 1 according to RFC 4122, i.e.
| time and node based:
|
| https://www.famkruithof.net/guid-uuid-make.html
|
| https://www.famkruithof.net/guid-uuid-timebased.html
|
| Since windows creates the UUID the first time it "sees" a volume,
| and - usually - uses the network card MAC as node, by decoding
| the UUID you can get the MAC address of the PC and the time the
| volume was seen (this can be useful for forensics, expecially
| with removable devices and to verify there has been no
| manipulation of the MountedDevices in the Registry).
|
| [1]possibly windows 11 changed that, or at least the UUID's shown
| in the article are type 4
| planetafro wrote:
| Was Dave Gahan left off purposely?
| mixmastamyk wrote:
| The period as the final character limit has gotten me when
| copying albums to a fat32 drive. Turns out a few album names end
| in a period. :-/
| EvanAnderson wrote:
| In UNC paths you can append "$NOCSC$" to the hostname to force
| the client to bypass the "Offline Files" cache. (There are
| probably other wild undocumented bits like this one hiding in
| other places in the Windows stack.)
| mdaniel wrote:
| Do you happen to have a source where you learned that? I'm
| always interested in "teaching myself to fish"
| EvanAnderson wrote:
| I don't recall. Like the other reply to you says, these get
| leaked in support, etc. I'll also run "strings" or even
| Ghidra on closed-source binaries when I'm troubleshooting
| issues. There's usually good fun to be had from Microsoft
| binaries doing that. I've discovered undocumented debugging
| switches, registry entries, etc.
|
| (In version 10.0.19041.985 of cscsvc.dll in Windows 10 I'm
| seeing the string "If you hit this breakpoint, send debugger
| remote to BrianAu." Presumably that's "Brian Aust",
| referenced in a chat[0] re: Offline Files.)
|
| [0] https://techcommunity.microsoft.com/t5/storage-at-
| microsoft/...
| c0nsumer wrote:
| Lots of this stuff we just find out while working various
| deep MS cases, and then the info just leaks out.
| xiwenc wrote:
| Recently we had a bug in an electron app that couldnt call exec
| on a filepath that contains a space in windows. After 2 days of
| researching and trying, i still don't have a solution. I reckon
| adding nodejs to the fantastic world of windows file IO does not
| make it any easier.
|
| Gonna give it another try this weekend.
| [deleted]
| unnouinceput wrote:
| Ahhh, the memories of naming my folder AUX and then having my Uni
| professors with administrative privileges not being able to
| access my files on that folder. Drove them crazy because they
| thought there is the secret of the Universe hidden there and they
| demanded I let them see what's inside, when in reality I only
| wanted to show off and had nothing at all. Novel Netware on top
| of DOS, year is 1994 - good times.
| layer8 wrote:
| The part about trailing periods and spaces being disallowed isn't
| quite correct. On an NTFS drive, you can actually do
| echo > \\?\C:\path\to\file.
|
| and echo > "\\?\C:\path\to\file "
|
| Similarly, files with such names can be created with Cygwin.
| mdaniel wrote:
| > UNC paths have a peculiar way of indicating the drive letter,
| we must use $ instead of :.
|
| I don't believe that's true, I am almost positive they're SMB
| shares, just like any other, but are created by the system, which
| is why "accessing drives in this way will only work if you're
| logged in as an administrator."
| kunwon1 wrote:
| The dollar sign indicates that the share is 'hidden' and can't
| be enumerated by traditional means. The C$ share is created by
| default and provides root level access to the system drive, and
| is locked down by default for this reason
|
| you are correct that they are just SMB shares like any other.
| They can be removed, though many management processes across
| different applications assume that those shares will be present
| banana_giraffe wrote:
| The bit about "UNC Paths" is a bit simplified. The "$" shares are
| administrative shares. They're created by default, you can delete
| or disable them (though, if you delete them, they'll be recreated
| on a reboot). You can also add normal users to them.
|
| It should also be noted that while the single driver letter ones
| are automatically created, the "$" at the end just marks them as
| hidden. You can create your own hidden shares if you ever want
| to.
| masfuerte wrote:
| You can also change the permissions on them so you don't need
| to be an admin to access them.
| sam1r wrote:
| Wow, incredible write up. Not only did I learn a lot - but it was
| also the perfect product pitch.
|
| The "and this is why fileside exists" transition at the end -
| perfect. No "sign up" or anything. Awesome!
| dmarinus wrote:
| GitHub Actions in 2023 has issues with checking out a repository
| with 256ish path names on Windows runners. whups
| zwieback wrote:
| Needs mention of auto-generated short file names (8.3 alias,
| typically with ~ in them) on volumes that don't support long
| names
| Someone wrote:
| Not only there. Windows originally created them on disks that
| supported long names, too.
|
| That was necessary to support the use case where an older OS
| tried to read the disk (could happen because the user rebooted
| into an old DOS, for example, or if an external disk was moved
| to a different computer)
|
| https://en.wikipedia.org/wiki/8.3_filename#VFAT_and_computer...
| :
|
| _"VFAT, a variant of FAT with an extended directory format,
| was introduced in Windows 95 and Windows NT 3.5. It allowed
| mixed-case Unicode long filenames (LFNs) in addition to classic
| 8.3 names by using multiple 32-byte directory entry records for
| long filenames (in such a way that only one will be recognised
| by old 8.3 system software as a valid directory entry).
|
| To maintain backward-compatibility with legacy applications (on
| DOS and Windows 3.1), on FAT and VFAT filesystems an 8.3
| filename is automatically generated for every LFN, through
| which the file can still be renamed, deleted or opened,
| although the generated name (e.g. OVI3KV~N) may show little
| similarity to the original. On NTFS filesystems the generation
| of 8.3 filenames can be turned off. The 8.3 filename can be
| obtained using the Kernel32.dll function GetShortPathName"_
| zwieback wrote:
| Right. Back in the 90s I worked on a network server to allow
| AppleTalk clients into DOS or OS/2 based networks. The Mac
| users enjoyed their filename freedom but the PC clients had
| trouble with the super-weird 8.3 short names. You couldn't
| really tell what the Mac filenames were.
|
| The other direction worked great, though, DOS filenames
| always worked on the Mac side of the network.
| Dwedit wrote:
| Here is an excellent article about Paths on Win32:
| https://googleprojectzero.blogspot.com/2016/02/the-definitiv...
| QuercusMax wrote:
| I knew windows filesystem layout was super bonkers when I had to
| explain to fellow devs that on a 64-bit machine, you put the
| 32-bit libraries in SysWow64, and the 64-bit libraries in
| system32.
| eduction wrote:
| The fact these paths are considered at all "weird" just
| underlines how much we live in a Unix world.
|
| Filesystem paths used to all be weird in the sense that there was
| more OS diversity. I'm sure some people here remember that
| classic MacOS paths used colon as the separator:
| Hard Drive:My Folder:My Document
|
| VMS (designed by the same person as Windows NT by the way) had
| paths that looked like this (per Wikipedia):
| NODE"accountname
| password"::device:[directory.subdirectory]filename.type;ver
|
| RSTS/E had [project,user] in the filename.
|
| Multics paths: >dir1>dir2>dir3>filename
|
| Apple Lisa used dashes as the separator. Etc.
| bobbylarrybobby wrote:
| macOS still uses colons as the path separator, it just does a
| great job of hiding them from the user. If you try to open a
| file with a slash in its name in a shell, though, you'll need
| to use a colon.
| dfox wrote:
| I suspect that it is the other way around and the Finder and
| standard dialogs (both of which use slashes as path separator
| when you type the path) simply shows colons in filenames as
| slashes.
| soraminazuki wrote:
| macOS's kernel has BSD roots, so I'd be surprised if its VFS
| code accepts anything other than unix paths. Just a guess,
| but it's probably the Cocoa APIs accepting colon paths and
| translating it to unix paths internally.
| m463 wrote:
| thank goodness unix has cleared this up, with paths,
| mountpoints, overlay filesystems, chroot, device trees, bind
| mounts, loopback mounts and probably a few I forgot
| latexr wrote:
| > I'm sure some people here remember that classic MacOS paths
| used colon as the separator
|
| In modern macOS (previously OS X), you'll eventually bump into
| those if you need to work with paths in AppleScript. You have
| to specify when you're using a POSIX path so it is properly
| converted. Example: $ osascript -e 'POSIX
| file "/System/Applications/Mail.app/Contents/MacOS/Mail"'
| => file Macintosh
| HD:System:Applications:Mail.app:Contents:MacOS:Mail
| jeroenhd wrote:
| As someone who grew up with Windows, I don't think these paths
| are that weird at all. Drive letter working directories just
| make sense, for example. The weirdest part is the (edit: HFS)
| compatibility mode (file.ext:substream).
|
| One fun surprise is that because of codepage reasons, the
| Windows will use Y= as a path separator in Japanese. In Korean,
| it's W. These characters represent U+005C, which is \ in Latin-
| compatible character sets.
| naikrovek wrote:
| that's not a compatibility thing, is it? it's just the
| alternate streams feature that NTFS implemented.
| jeroenhd wrote:
| NTFS implemented it to be compatible with Mac. They then
| started using it for storing the Mark of the Web and other
| special system properties, but practical came much later.
| robertoandred wrote:
| It's pretty weird that Windows drives use random letters
| instead of just the name of the disk.
| kemotep wrote:
| It's a legacy from the IBM CP/CMS days.
|
| First floppy drive was A, Second B, and when internal Hard
| Drives came along they defaulted to C to be compatible with
| computers that had at least 2 disk drives.
|
| https://en.wikipedia.org/wiki/Drive_letter_assignment?wprov
| =...
| YakBizzarro wrote:
| If I rembember correctly, you could use the B drive even
| if you have just one unit. It was useful to copy files
| from one disk to another, even if you didn't had an hard
| drive as temporary storage
| layer8 wrote:
| This was ultimately inherited from IBM's CMS system [0]
| (from 1968 I believe) via CP/M and DOS.
|
| [0] https://en.wikipedia.org/wiki/CMS_file_system
| jeroenhd wrote:
| I tend to use /dev/sda1 more than /dev/disk/by-
| path/pci-0000:00:17.0-ata-1.0-part1. Disk names are nice,
| but also often longer than 8 characters and usually not
| very unique.
|
| Starting from A and iterating on through Z makes sense, for
| an OS that's designed for two drives at most. /dev/sda and
| /dev/sdb are no less arbitrary than A: and B:.
|
| One major difference was that Unix was used on big servers
| and couldn't fit itself onto a single disk, so /usr had to
| be created. DOS and Windows never needed a second drive to
| boot, so they didn't need to embed their resources into the
| drive hierarchy.
|
| Of course, you can mount NTFS volumes at any directory you
| wish since at least somewhere in the early 2000s. Very few
| people do it, but you can!
|
| For example: $Disk = Get-Disk 2
| $Partition = Get-Partition -DiskNumber $Disk.Number
| $Partition | Add-PartitionAccessPath -AccessPath
| "G:\Folder01"
| rlkf wrote:
| > The weirdest part is the HPFS compatibility mode
| (file.ext:substream).
|
| HPFS had extended attributes, but not substreams. You are
| thinking about HFS; substreams were added to NTFS to support
| storing resource forks on network shares used by Macs.
| jeroenhd wrote:
| Oops, you're right, added an extra letter to the acronym.
| eduction wrote:
| that's amazing. and point taken that Windows is probably
| extra weird due to its longevity, evolution, and backward
| compatibility.
| [deleted]
| Karellen wrote:
| As another example, with ADFS (Advanced Disk Filing System) on
| the Acorn/BBC computer family, the root directory was specified
| with `$`, and the directory separator was `.`.
| $dir1.dir2.dir3.filename
| Symbiote wrote:
| A full path on RISC OS included the filesystem:
| ADFS::IDEDisc4.$.Games.!Repton.Arctic
|
| ADFS is the filesystem, IDEDisc4 is the disc name, $ is the
| root directory, Games is a subdirectory, !Repton is an
| application directory (since it begins with !) and Arctic is
| a file within the application directory, not normally
| referenced by users. Resources:$.Apps.!Edit
|
| is the application !Edit from the built-in ROM.
___________________________________________________________________
(page generated 2023-04-21 23:00 UTC)