[HN Gopher] What it takes to pass a file path to a Windows API i...
___________________________________________________________________
What it takes to pass a file path to a Windows API in C++
Author : AshleysBrain
Score : 136 points
Date : 2023-09-24 11:45 UTC (11 hours ago)
(HTM) web link (mastodon.gamedev.place)
(TXT) w3m dump (mastodon.gamedev.place)
| BazookaMusic wrote:
| Ah yes, the summoner of outages if you have production systems
| with windows server and allow user input in paths.
| grishka wrote:
| > most modern software uses UTF-8
|
| [citation needed]
|
| UTF-8 is used for storage and serialization, yes, but most
| mainstream programming languages store unicode strings in memory
| as UTF-16. The notable exception is Rust, which does indeed use
| UTF-8 even in memory.
| chrismorgan wrote:
| > _most mainstream programming languages store unicode strings
| in memory as UTF-16_
|
| This is increasingly frequently false. For stupid historical
| reasons, far too many languages adopted potentially-ill-formed
| UTF-16 semantics ( _never_ well-formed, sadly), but this makes
| for such a bad _representation_ that they've mostly abandoned
| it as a sole representation, in favour of more complicated
| arrangements. For example, Java strings can internally be
| single-byte Latin-1 or two-byte UTF-16. JavaScript engines do
| similar tricks these days, and although Python 3 just never did
| UTF-16 at all (it uses _code point_ semantics, which is almost
| worse than UTF-16 semantics) it does a 1-byte /2-byte/4-byte
| hybrid representation thing too.
|
| For one of the most extreme examples of becoming unglued from
| UTF-16, the Servo browser engine manages to use WTF-8 (UTF-8
| plus lone surrogates, to allow representing ill-formed UTF-16)
| despite being forced by these historical reasons to expose
| UTF-16 semantics, and although the performance of _some_
| operations are harmed by it, others are improved, and in the
| balance it was a pretty convincing win when they tested it
| (which many people did not expect).
|
| PyPy is also a great example of changing your encoding to one
| that has mismatched semantics: it applies UTF-8 (or I suppose
| it must actually be UTF-8 plus surrogates) to Python, and
| likewise they found it surprisingly good for performance.
|
| These things give me hope that even environments like the JVM
| and .NET CLR might eventually manage to switch their internal
| string representations to UTF-8 or almost-UTF-8.
|
| But also I emphasise that these are _historical_ languages that
| are stuck with the horrendous 16-bit decisions of the early-to-
| mid-'90s. When you look at newer languages, they almost always
| choose something more sensible, and UTF-8 is almost always at
| the heart of it.
| grishka wrote:
| > it uses code point semantics, which is almost worse than
| UTF-16 semantics
|
| What's so bad about storing a unicode string as a series of
| codepoints? UTF-8 is also a series of codepoints in essence.
|
| > When you look at newer languages, they almost always choose
| something more sensible, and UTF-8 is almost always at the
| heart of it.
|
| I still think that UTF-8 for memory is a bad idea. For
| example, it wastes a lot of bits to make sure that a string
| can be decoded starting from an arbitrary byte offset. This
| is generally a desirable property for files and some
| networking applications, but it's an absolute waste of space
| when used in memory.
| chrismorgan wrote:
| _Code point_ semantics, not scalar value semantics which is
| what Unicode strings are supposed to be. That is, a Python
| string can contain surrogate code points, which cannot be
| encoded in any UTF-*: >>>
| '\udead'.encode('utf-8') Traceback (most recent call
| last): File "<stdin>", line 1, in <module>
| UnicodeEncodeError: 'utf-8' codec can't encode character
| '\udead' in position 0: surrogates not allowed >>>
| '\udead'.encode('utf-16') Traceback (most recent call
| last): File "<stdin>", line 1, in <module>
| UnicodeEncodeError: 'utf-16' codec can't encode character
| '\udead' in position 0: surrogates not allowed
|
| (UTF-8 is a sequence of 8-bit code _units_ , representing a
| sequence of Unicode _scalar values_.)
|
| --***--
|
| > _For example, it wastes a lot of bits to make sure that a
| string can be decoded starting from an arbitrary byte
| offset._
|
| If you ditch self-synchronisability and ASCII purity of
| extension by ditching the first two bits ("1-0") on
| continuation bytes, you _can_ encode some more briefly:
| roughly, U+0800-U+1FFF go from three-byte to two-byte, and
| U+10000-U+FFFFF go from four-byte to three-byte, leaving
| just U+100000-10FFFF as four-byte. But honestly the
| difference isn't often _that_ much (unlike UTF-16 - UTF-8,
| which roughly halves memory usage most of the time,
| commonly even saving when dealing with text that makes
| heavy use of other scripts that are more compact in UTF-16
| than in UTF-8 since you commonly mix things in with ASCII
| markup), and if you're really trying to shave memory, other
| techniques like compression are far more effective.
|
| I love microoptimisation (and spent a large number of hours
| on my Casio GFX-9850GB PLUS, where having only 30KB of
| memory encouraged learning to shave single bytes here and
| there!), but there's also a _lot_ to be said for
| consistency. Yes, some purposes could have an encoding that
| is more efficient than UTF-8. But doing this complicates
| matters quite a bit, and I think we're better off with just
| UTF-8.
|
| Also I will note that the self-synchronising nature of
| UTF-8 _is_ useful for some operations: you can scan through
| strings much more quickly that way. So it's not pure waste
| as a memory representation.
| [deleted]
| loeg wrote:
| > most mainstream programming languages store unicode strings
| in memory as UTF-16
|
| Uh. What languages do you have in mind? C/C++ don't have any
| preference for UTF-16. Python3 doesn't use UTF-16. As you
| mentioned, Rust doesn't.
| grishka wrote:
| Java, C# and the rest of .net, JavaScript, Swift/ObjC.
|
| C/C++ don't have a preference but, for example, if you're
| writing for Windows, you'd probably want to use wide
| characters. I'm not sure what kind of strings Linux GUIs use
| but I suspect it's wide characters as well.
| 7jjjjjjj wrote:
| Qt uses UTF-16, Gtk uses UTF-8.
| Longhanks wrote:
| Swift is UTF-8 since 2019
|
| https://www.swift.org/blog/utf8-string/
| MatmaRex wrote:
| In addition to the examples already given, JavaScript as well
| (although modern implementations do wacky stuff with the
| internal representations, the APIs are required to expose
| only UTF-16).
| steeleduncan wrote:
| java, .net, apple ecosystem (i.e. swift/objc),...
| Longhanks wrote:
| Swift is UTF-8 since 2019
|
| https://www.swift.org/blog/utf8-string/
| IshKebab wrote:
| That's arguably because those languages are not modern. Nobody
| would design a modern language today that uses UTF-16 (unless
| they really need compatibility with Java or JavaScript).
|
| Go also uses UTF-8 btw.
|
| I think you're being needlessly pedantic (I know it's unusual
| for HN).
| hgs3 wrote:
| It gets worse. Windows doesn't use UTF-16 for file paths it uses
| UCS-2. Example: Windows paths can contain unpaired UTF-16
| surrogates which is illegal in UTF-16. Unix paths are no better;
| they are "bags of bytes" and not necessarily UTF-8. File paths
| are not guaranteed to be Unicode normalized. Even if they are
| canonically equivalent (equal graphemes) it doesn't mean they are
| binary equivalent (equal code points). You need to treat file
| paths as their own special thing and not strings.
| chrismorgan wrote:
| Windows uses _potentially-ill-formed_ UTF-16.
|
| _No_ language or environment that uses UTF-16 validates it,
| ever. Not a single one that I know of. But that doesn't make it
| UCS-2, it just makes it potentially-ill-formed UTF-16.
| hnfong wrote:
| I just have the impression that they s/UCS-2/UTF-16/g without
| changing anything. Was that not the case? Genuinely curious
| whether there were UTF-16-specific changes made before they
| declared that Windows uses/supports UTF-16.
| qsdf38100 wrote:
| Wrong. Windows uses UTF-16. It used to use UCS-2 a long time
| ago.
| pornel wrote:
| Unpaired surrogates are not allowed in UTF-16, so whatever
| Windows uses is not quite it.
| jeroenhd wrote:
| Windows doesn't bother checking every string after every
| modification, but neither does Linux with UTF-8. You can
| pass invalid UTF-8 to tons of APIs and almost all of them
| will just work, just like you can with UTF-16 in Windows.
|
| Doing a string validation check in every single API call
| would waste cycles for no good reason.
| Karellen wrote:
| Linux (the kernel) doesn't claim (AFAIK) to use UTF-8. It
| takes NUL-terminated strings in any charset you choose
| (or none) and either compares them with other NUL-
| terminated strings, or spits them back out again later.
|
| Interpreting a string as encoding text in particular
| character set is, as far as the kernel is concerned, a
| problem for userspace.
|
| Windows does make claims of "supporting" UTF-16.
| Guvante wrote:
| And Linux paths can contain anything that isn't a nul character
| or a /.
|
| Your points about normalization only matter in so far as
| Windows is generally setup to be case insensitive (which can be
| disabled).
|
| All UTF strings face normalization issues for a whole host of
| reasons.
| jeroenhd wrote:
| > (which can be disabled)
|
| Wouldn't that break spectacularly? The whole "the fs is case
| insensitive" assumption has kind of been a given on Windows
| for about 40 years now.
|
| I remember accidentally creating an NTFS flash drive with two
| names with difference cases in Linux. Windows programs sure
| didn't like that.
| Guvante wrote:
| It was added in 2018 as a folder attribute. There isn't a
| global option but you can annotate something as case
| sensitive.
| Affric wrote:
| So, a lot of what people think of when they think of OS is UI
| based (GUI/TUI/CLI) but kernel APIs are the real bread and butter
| and this is where the UNIX philosophy really shines.
|
| Passing a string as a file name in C++ with macOS or with Linux
| in my experience was simple. The permitted length of using ASCII
| characters is about 4 times as long (may god have mercy on your
| soul).
|
| I am not here to shit on Windows but the Windows devs clearly
| have a very different set of priorities (e.g. backwards
| compatibility) than the other breeds of modern OS devs.
|
| I guess to a large extent we all expect to be in the browser
| (gross) in some number of years but Windows seems so much harder
| from the perspective of someone that has programmed for Unixen
| and studied Windows as an OS.
| cerved wrote:
| It's not all backwards comparability. I'm willing to bet that
| some (a large part?) is just sloppy software development.
|
| SQL Server (2017?) breaks if you update it on a UTF-8 Windows
| because it runs a TSQL that doesn't work with that code page.
| That script is a mess. Some of it is indented using tabs, some
| space. Trailing whitespace. Yuck
| nikanj wrote:
| My hot take: Code quality is not measured by formatting
| issues, but by error resilience and number of actual bugs.
|
| Much of modern linting and commit hooking is dedicated to
| checking whitespace placement, variable naming and function
| lengths but the well-formatted newly rewritten code is still
| buggy as hell - it just looks pretty
| lpapez wrote:
| True, but personally for me at least it is easier to find
| bugs in neatly formatted consistent code than something
| written in multitude of styles.
|
| It is kind of like "pattern matching" on the error
| patterns.
| edejong wrote:
| Formatting doesn't remove bugs, but it'll help you detect
| them. Linted code helps you scan the code faster and
| provides valuable pattern recognition, allowing us to
| detect common mistakes.
|
| There have been numerous bugs caused by incorrect code
| formatting, most notably an SSL security bug from 2014:
| https://dwheeler.com/essays/apple-goto-fail.html
|
| Another reason for formatting is the "minimal diff"
| paradigm. If a formatting rule would not be followed, in
| the next commit hitting this code, the format would also be
| affected, causing a larger diff than necessary.
|
| There are other reasons for simple format linting, but the
| reasons above are the most profound.
|
| Lastly, formatting is part of a range of static code
| analysis tools. Generally, formatting inconsistencies are
| the easiest to detect and resolve, as opposed to more
| sophisticated tools.
| inferiorhuman wrote:
| Passing a string as a file name in C++ with macOS or with Linux
| in my experience was simple. The permitted length of
| using ASCII characters is about 4 times as long (may god
| have mercy on your soul).
|
| Macs are simple enough too if you ignore the quirks. HFS (which
| was never seen on a modern MacOS) usually stores no information
| about what encoding was used for filenames. It's entirely
| dependent upon how the OS was configured when the file was
| named (although some code I've seen suggests that something in
| System 7 would save encoding info in the finderinfo blobs). So
| non-latin stuff gets mangled pretty easily if you're not
| careful. Filenames are pretty short (32 bytes) minus the one
| byte because (except for the volume name) they're Pascal
| strings with the length at the front.
|
| HFS+ (which is what you'll find on OSX volumes) uses UTF-16 but
| then mandates its own quirky normalization and either Unicode
| 2.1 or 3.2 decomposition depending... which can create
| headaches because most HFS+ volumes are case-insensitive. It's
| been so long since I've touched anything Cocoa, but I assume
| the file APIs will do the UTF-16 dance for you and the POSIX
| stuff is obviously OK with ASCII.
|
| And, of course, let's not forget the heavily leveraged resource
| forks. Of course NTFS has forks but nobody seems to use them.
|
| APFS standardized on Unicode 9 w/ UTF-8.
|
| CDs? Microsoft's long filenames (Joliet) use big endian UTF-16
| (via ISO escape sequences that theoretically could be used to
| offer UTF-8 support). Which sounds crazy until you realize
| their relative simplicity (a duplicate directory structure)
| compared to the alternative Rockridge extensions which store
| LFNs in the file's metadata with no defined or enforced
| encoding. UDF? Yeah that's more or less UTF-16 as well.
|
| I think we're perhaps forgetting just how young UTF-8 is.
| Affric wrote:
| Thanks for the comment. HFS/HFS+ is a fascinating bit of
| history.
|
| It strikes me how developer ergonomics have improved as
| computers have become cheaper/increased in power.
|
| As to UTF-8, we may say it's young but in 14 months it will
| be old enough to purchase and consume alcohol in the United
| States. From other comments it seems like Microsoft don't
| think the tech debt is too great so long as they have good
| libraries in C#
| sn_master wrote:
| In fairness, not that many people (including Microsoft) write
| native C++ apps for Windows anymore, certainly not without
| tried and tested libraries.
|
| You can write C# code dealing with reading/writing files once
| and compile it on Linux/Windows/Mac and it'll work pretty much
| the exact same.
| pjmlp wrote:
| Microsoft does write native C++ apps for Windows all the
| time.
|
| First of all, games are apps, second even if apps unit keeps
| mostly ignoring WinUI/UWP (written in C++), whatever they do
| with Web widgets is mostly backed by C++ code, not C#.
|
| On of the reasons why VSCode is mostly usable despite being
| Electron, is exactly the amount of external processes written
| in C++.
|
| Applications being written in .NET is mostly on the Azure
| side.
| jabits wrote:
| "Applications being written in .NET is mostly on the Azure
| side."
|
| You are of course, wrong about this. Most .Net/C# code is
| _not_ Azure (yet anyway) -related; it is the billions of
| lines of enterprise application code across businesses
| around the world (for me, since 2001)...
| pjmlp wrote:
| You are not Microsoft apps unit, the subject of what is
| being discussed here.
| sn_master wrote:
| Microsoft has literal teams with budgets of several
| millions USD just for the file open/save in Office which is
| written in C++.
| ilrwbwrkhv wrote:
| But despite that they cannot fix it. Consistently they
| make perhaps the worst APIs of any major tech company.
| sn_master wrote:
| True. They broke the basic Windows search functionality
| some time in 2007 and broke Outlook search around 2013
| and neither of which have been fixed since.
| pjmlp wrote:
| I will take Win32 over anything related to X Windows,
| OpenGL and Vulkan, with pleasure.
| shortrounddev2 wrote:
| Maybe for file handling in C++, but DirectX/HLSL is the
| best Graphics API I've worked with and C# is easily my
| favorite language to develop in. It's easy for us to talk
| shit about Win32 today, 30 years after it was initially
| developed, but there are myriad historical reasons why
| UTF-16 is used by Java, Windows, and other
| languages/runtime environments and why it's not simple to
| just break compatibility with decades of software running
| at hospitals and financial trading firms because the 32
| year old armchair experts at HN said so.
|
| According to wikipedia:
|
| https://en.wikipedia.org/wiki/Universal_Coded_Character_S
| et
|
| > The UCS has over 1.1 million possible code points
| available for use/allocation, but only the first 65,536,
| which is the Basic Multilingual Plane (BMP), had entered
| into common use before 2000. This situation began
| changing when the People's Republic of China (PRC) ruled
| in 2006 that all software sold in its jurisdiction would
| have to support GB 18030. This required software intended
| for sale in the PRC to move beyond the BMP.
| pjmlp wrote:
| Those file/save dialogs are an application of their own,
| and with multiple versions across all supported
| platforms.
| cm2187 wrote:
| I never understood what backward compatibility was met by
| windows api not supporting >260 chars file paths. It will work
| just in the same was if you pass any short path and no old
| application expects a long path anyway.
| mappu wrote:
| MAX_PATH is a #define. So its value is baked in to old
| binaries.
|
| In RAM-constrained world of the past, you would stack-
| allocate `char buff[MAX_PATH]` and do all your strcpy/strspn
| in there with no problems.
|
| Now, if that app receives a long path into a too short
| buffer, it will instantly stack overflow and may cause
| exploitable problems.
| Arainach wrote:
| Decades of binaries are in use doing something like
|
| ``` wchar_t filename[MAX_PATH]; CreateFileW(...) ```
|
| in both first part and third party Windows code, often in
| deep callstacks passing file names around. Changing the
| length requires fixing them all.
|
| See comments in https://archives.miloush.net/michkap/archive/
| 2006/12/13/1275...
| TonyTrapp wrote:
| Your example isn't problematic API-wise, because
| CreateFileW doesn't need to care if you pass in 16
| characters or 1600 - if it does, that is mostly a matter of
| refactoring and not inherent to how the function works. The
| real problem are APIs that inherently assume that you pass
| in a string of at most MAX_PATH characters, because you
| provide a pointer but no size, and the API is expected to
| write to that pointer. This affects most shell32 getter
| functions, e.g. SHGetKnownFolderPath.
|
| But for functions outside of Windows itself, this is the
| exact reason why the long path feature is hidden behind an
| opt-in flag.
| Someone wrote:
| They have API calls that fill user-supplied buffers that have
| room for _MAX_PATH_ characters.
|
| See for example https://learn.microsoft.com/en-
| us/windows/win32/api/fileapi/..., which also shows how they
| gradually made the input argument more flexible:
|
| - _"By default, the name is limited to MAX_PATH characters.
| To extend this limit to 32,767 wide characters, prepend
| "\\\?\" to the path"_
|
| - _"Starting with Windows 10, Version 1607, you can opt-in to
| remove the MAX_PATH limitation without prepending "\\\?\""_
|
| I also guess there's lots of code that sees those paths
| (anti-virus software, device drivers)
| tester756 wrote:
| Which is interesting because for example C#/.NET base class
| library provides very high quality APIs
|
| They even wrote a book about API design "Framework Design
| Guidelines: Conventions, Idioms, and Patterns for Reusable .Net
| Libraries"
| toyg wrote:
| C# was effectively built from scratch with the benefit of
| hindsight from both C++ and Java. Decent practices should be
| considered been the bare minimum, surely.
| thehias wrote:
| Whats even worse: Trying to figure out if the current
| user/process is allowed to write to a file in a specific path.
| Someone1234 wrote:
| Because you shouldn't. Either try and write, and see if it
| fails, or if need-be ask for an exclusive lock on the file and
| see if the OS gives it to you. That isn't just permissions
| either, it actually picks up a lot of potential problems (e.g.
| device disconnected, lock conflict, et al).
| colejohnson66 wrote:
| This. Seeing if you can do something before actually doing it
| falls victim to TOCTOU[0] bugs. Until one actually tries to
| perform the security-sensitive operation, there's no
| guaranteeing that you will succeed.
|
| [0]: https://en.wikipedia.org/wiki/Time-of-check_to_time-of-
| use
| Cold_Miserable wrote:
| Windows uses unicode because NTFS uses unicode instead of
| efficient ascii.
| becurious wrote:
| When it was designed there was no UTF-8. You had to know the
| code point; and file systems like FAT did not encode the code
| point, so file names would be interpreted by whatever the
| current code point was.
|
| So having a file system and other APIs support Unicode natively
| made a lot of sense at the time; it simplifies many things
| including deploying into organizations that have global users.
| This is now over 30 years ago. And the A vs the W suffixed APIs
| were not simple ascii; they were multi-byte, so to work with
| strings you still had to know the code page. Obviously UTF-8 is
| much simpler for that now, but that wasn't the world when this
| was designed.
| daemin wrote:
| For game development this should only matter on the game
| developers' machine, where there is more interaction with loose
| files on disk.
|
| For any shipped game being played on a player's machine the data
| files should be some form of packed files, which are close to the
| executable and only a handful need to be opened.
| h2odragon wrote:
| It's possible to get a file into a windows filesystem whose name
| contains "*" or ":" characters. Windows reacts quite poorly when
| it find those in file names; but only in certain APIs, others
| pass them without concern. Makes for fun explosions.
|
| The "260 character limit" is a good one; mostly seen that one in
| "I can do anything to this file but delete it!" complaints.
| quickthrower2 wrote:
| I had the can't delete a file problem the other day. Got around
| it my moving it's parent folder to a less nested location
| huhtenberg wrote:
| It's also possible to create files whose names start (or
| contain) a NUL character. Doing so will trigger every single
| antivirus, but it's doable nonetheless and messes a lot of
| things very thoroughly.
| Someone1234 wrote:
| Also, possible in the Windows Registry, which no normal UI
| tools (inc. Reg Edit) could remove.
| 13of40 wrote:
| A similar quirk of the registry API is that it has a
| simplified set of methods that don't specify the data type
| of the value, so you can have an application that expects
| to read a null terminated string value, but substitute a
| binary value with no null termination. If the app doesn't
| handle that right, it's buffer overrun time.
|
| Usually that's foot-shooting, but sometimes you can do that
| in HKCU as a low privileged user to a value that gets read
| by a higher privileged process and causes it to
| "misbehave".
| ale42 wrote:
| You can also create a file (or even better a folder) named
| CON or COM4. Good luck doing anything with it using the
| Explorer. (15 years ago I've seen some malware using that to
| make removal difficult)
| dark-star wrote:
| you can also get "files into a Linux filesystem" that contain
| the "/" character, which also leads to fantastic kernel
| crashes...
|
| Admittedly, it's (probably) harder to pull off than on Windows,
| but still...
| badrabbit wrote:
| Most windows software uses utf16. This guy is used to web and
| *nix software where UTF-8 is popular. All your strings should
| default to UTF-16 on Windows so you don't have to deal with this
| against specific APIs.
|
| The longpath thing shouldn't be something you have to deal with
| each time you call an API. You should do what Git for windows and
| other software do which is enable long paths (assuming you
| actually need it) during setup/install.
| DarkmSparks wrote:
| when I first ran into the utf8/utf16 etc a lot of windows stuff
| was "recommending" utf16. I see quite a few windows text
| editors default to it.
|
| After a day or so of testing and trying it with various
| different SDKs across different platforms I settled on utf8.
| did everything needed, compatible with ASCII, all the
| functionality of 16 none of the _absolute insanity_.
|
| looking back, these kind of issues are why its now some 20
| years since I wrote any windows software (rather than porting
| linux/mac stuff or just using cross platform VMs like
| Javascript, Lua or Java).
|
| MFC..... Oh my.... I actually feel sorry for the poor souls
| that still have that in their life.
| n2d4 wrote:
| A lot of people would disagree with you:
|
| _> This document also recommends choosing UTF-8 for internal
| string representation in Windows applications, despite the fact
| that this standard is less popular there, both due to
| historical reasons and the lack of native UTF-8 support by the
| API. We believe that, even on this platform, the following
| arguments outweigh the lack of native support._
|
| https://utf8everywhere.org/
| jeroenhd wrote:
| Linux and MacOS use UTF-8. Qt, Windows itself, Java,
| Javascript, and (on Windows) C# will all use UTF-16.
|
| Most computers run Windows, by a rather big margin. Even more
| computers use Javascript. Are you running KDE? Don't be
| surprised if half the text in your screen is secretly UTF-16!
|
| UTF-8 is definitely _better_ , but if you're on Windows,
| you're making your own life harder by using UTF-8. Microsoft
| is migrating APIs to UTF-8 but it'll take years before that's
| finished. Just look at all the conversion code you need
| according to the page you linked.
|
| Now, if you're writing a cross platform library, you'll have
| to decide what encoding(s) you use. I personally don't really
| see why you wouldn't make your library encoding agnostic, but
| if you want to stick to one single encoding, you'll have to
| decide.
|
| If you're writing a program for Windows, technical
| superiority is a mediocre argument for making your own life
| that much harder. There are tons of formats and design
| decisions that may be technologically superior, but just end
| up wasting programmers' time, and this is one of them.
| badrabbit wrote:
| Windows is not open source, I don't prefer either encoding I
| am just telling you that as that document acknowledges,
| UTF-16 is the default whether "everyone" (tech bubble) likes
| it or not. If you write windows software assume that default,
| not your favorite utf8. Just like how you assume endianess on
| a system because of the default.
| WirelessGigabit wrote:
| 1607 has reached End of Servicing. So unless you need to support
| software (= $$$) on older versions that shouldn't be a concern.
|
| Second, I don't see a risk in setting `LongPathsEnabled` on a
| machine as it's only 1 piece of the puzzle. You still need to
| opt-in at the application level:
|
| https://learn.microsoft.com/en-us/windows/win32/fileio/maxim...
| hohoharvey wrote:
| [dead]
| LargeTomato wrote:
| One time I installed a buggy version of Arduino and it installed
| recursively, creating file paths far larger than 260 char. I was
| unable to delete the files using explorer, cmd line, or power
| shell. I had to use cygwin. That was the most disappointed I've
| ever been with Windows.
| MatmaRex wrote:
| The built-in "robocopy" tool (normally used for making backups
| / mirrors) is capable of deleting any messed up directories
| like that, by mirroring an empty directory over it.
| kmeisthax wrote:
| Half a decade ago anything that needed Node packages - even
| just a Gulp build system - would wind up constructing extremely
| deep directory trees that Explorer would choke on. So you would
| npm install the project, realize you needed a different version
| of something, try to delete node_modules, and fail horribly
| with a bunch of long path errors.
| jeroenhd wrote:
| You can remove these files using the command line by using the
| \\\?\ syntax to bypass all normalisation and compatibility
| tricks Windows employs. Definitely works in CMD, never checked
| in Powershell (but I assume it works).
|
| I've had a similar issue with a recursive folder and
| transferring it to another host in Linux. A megabyte on my
| machine filled up the server pretty quick.
| wzdd wrote:
| Alternative sane approach:
|
| - Forget about long paths unless you really need to care. You
| need to care if you're accepting a file path from another app,
| for example. You don't need to care if you're using your own
| files in your install directory; too much of Windows doesn't
| support long files (including explorer, as mentioned, not to
| mention third party apps), so it's very unlikely that you'll need
| to deal with them if you're just handling your own data.
|
| - Work in utf-16 when dealing with file paths. Convert back and
| forth at the point of entry / exit to / from your app. It's
| unsurprising that you need to call the conversion function twice
| in order to find out how much memory to allocate, that's just how
| that works.
|
| Don't change random registry settings or code pages on the user's
| computer, that's mad.
| Aardwolf wrote:
| > Work in utf-16 when dealing with file paths. Convert back and
| forth at the point of entry / exit to / from your app.
|
| But that's not compatible with other OSes, so you can't write
| native multiplatform C++ code just because windows keeps
| insisting on their worst-of-both-worlds UTF-16
|
| (worst of both worlds because it's not single value per unicode
| character like UTF-32, and not ASCII compatible like UTF-8)
| fsloth wrote:
| Our native multiplatform c++ code uses wstring (win, mac,
| android, wasm). It's not the worst option. AFAIK There is
| _no_ 100% method to have single codebase tackle the
| differences in platforms. My recipe for keeping sane is to
| have _single_ implementation for storage (wstring), and then
| very explicit points of converting paths from 'platform' to
| program internal.
|
| Platform differences unfortunately go way deeper than just
| using UTF-16. Lots of gnarly details. No platform is worse
| than another - they just have different kinds of good and bad
| in different places.
| plonk wrote:
| Have a few functions taking UTF-8 paths for the filesystem
| operations you need, have separate versions for Windows and
| for other systems, and enable one or the other version with
| the preprocessor?
| shortrounddev2 wrote:
| This is how 99% of cross-platform C++ works. Very few C++
| is actually "cross platform" in that it compiles the same
| functions for different platforms
| [deleted]
| bartwe wrote:
| linux uses utf8 for display of filenames but, the paths
| themselves allow non-utf8 byte sequences.
| loeg wrote:
| It's an oversimplification to say Linux uses UTF-8 for
| display. Linux just stores bags of bytes and leaves
| interpretation to userspace. You could store paths in
| ISO-8859-1 if you wanted. The only special bytes are '\0'
| and '/'.
| jeroenhd wrote:
| Not only could you, this actually happens in practice.
| Not necessarily ISO-8859-1, but specifically SHIFT-JIS, a
| Japanese encoding that you will run into if you run old
| Japanese software. To make things even worse, SHIFT-JIS
| is almost entirely incompatible with any form of UTF
| based encoding, and depending on the attempted
| normalisation you can quickly end up with paths that have
| been messed up multiple times in a row.
|
| I forgot what Japanese emulator I tried to run when I
| found all of this out, ut sufficed to say I didn't enjoy
| the experience.
| Tigress8780 wrote:
| I buy digital Japanese Doujin music on sites like
| booth.pm, and their provided zip files extracts
| "beautifully" on Linux if you simply `unzip` them.
| charcircuit wrote:
| Don't you mean `unzip -O shift-jis` them?
| AshamedCaptain wrote:
| Except that Linux does support several filesystems that
| do claim to store the filenames in a specific encoding
| and therefore the kernel must do conversion. Mostly
| Windows FSes, but nowadays case-insensitive ext4 also
| applies.
| loeg wrote:
| These are exceptions, not the norm. The VFS layer does
| not care.
| mmis1000 wrote:
| linux don't display them, the shell (emulator) do. Linux
| just send the bytes back to userland and let shell
| interpret them with a proper format to human. And even
| then. Tons of distro default the global LANG to c for
| some reason. So utf8 display isn't even working by
| default.
| c4mpute wrote:
| Even better: Each user can have his own locale and
| charset, and may even change that per
| program/shell/session. One may save filenames as UTF-8,
| one as ASCII, one as ISO8859-13, one as EBCDIC.
|
| However, the common denominator nowadays is UTF-8, which
| has been a blessing overall getting rid of most of the
| aforementioned mess for international multi-user systems.
| And there is the C.UTF-8 locale which is slowly gaining
| traction.
| shortrounddev2 wrote:
| If you're calling Win32 functions it's already not compatible
| with other OSes... If you want to use cross platform code
| with files, use the C or C++ standard library's file handling
| functions.
|
| C++ also supports wstring, but its implementation is
| dependent on the platform for god knows what reason.
|
| utf-16 is also supported in C11:
|
| https://en.wikipedia.org/wiki/C11_(C_standard_revision)#Chan.
| ..
|
| > Improved Unicode support based on the C Unicode Technical
| Report ISO/IEC TR 19769:2004 (char16_t and char32_t types for
| storing UTF-16/UTF-32 encoded data, including conversion
| functions in <uchar.h> and the corresponding u and U string
| literal prefixes, as well as the u8 prefix for UTF-8 encoded
| literals).
|
| You can use uchar.h for converting instead of the Win32
| functions.
|
| Here's a helper I wrote to wrap these string types in
| std::string and std::wstring: bool
| mbStrToWideChar(string const& in, wstring& out) {
| bool status = true; i32 bufferSize =
| MultiByteToWideChar( CP_ACP, 0, in.c_str(),
| -1, nullptr, 0 ); WCHAR* buffer
| = new WCHAR[bufferSize]; i32 conversionResult =
| MultiByteToWideChar( CP_ACP, 0,
| in.c_str(), -1, buffer, bufferSize
| ); if (!conversionResult) {
| cerr << "Failed to convert ANSI string [" << in << "] to Wide
| string!" << endl; status = false;
| goto Delete; } out =
| wstring(buffer); Delete: if (buffer)
| { delete[] buffer; }
| return status; }
|
| and bool wideStrToMbStr(wstring const& in,
| string& out) { // Get buffer size
| i32 bufferSize = WideCharToMultiByte(
| CP_UTF8, 0, in.c_str(), (i32)in.size(),
| nullptr, 0, nullptr, nullptr );
| // Set outstring to new buffer size out =
| string(bufferSize, '\0'); //Convert
| i32 res = WideCharToMultiByte( CP_UTF8, 0,
| in.c_str(), (i32)in.size(), &out[0],
| (i32)out.size(), nullptr, nullptr
| ); if (!res) { wcerr
| << L"Failed to convert Wide string [" << in << L"] to ANSI
| string!" << endl; return false; }
| return true; }
| beached_whale wrote:
| Store it as std::path
| [deleted]
| bruce511 wrote:
| Not to be that guy, but utf-32 is fixed length code point,
| not fixed length character.
|
| Characters can consist of multiple code points.
|
| I agree that utf-8 is preferable to utf-16 though in most
| cases where the root script is English/Latin.
| usrusr wrote:
| Makes me wonder how bad utf-8 in off-latin environments
| really is: what's the fraction of strings that is "language
| content", and how much is colons, quotes, decimal numbers
| and the like? What fraction of coding cultures embraces
| home locale script for identifiers, and how many continue
| on the trajectory set in motion by pre-unicode programming
| languages? My guess would be that utf-8 is still quite bad,
| but I wouldn't be entirely surprised if actual numbers
| turned out considerably less bad than one might expect. And
| how much of the "utf-8 tax" remains if the actual format of
| most data at rest and in transit isn't json but gzipped
| json?
| vlovich123 wrote:
| Experiments I recall from a while back from browsers and
| JDK indicated the vast majority of memory is consumed by
| latin1 strings (even for apps in Asia). Of course, those
| experiments I recall are something like 20 years old and
| likely biased (websites were smaller, content was still
| latin1 dominated). It would be interesting to know the
| results from Chinese apps and websites today. For
| websites, the markup used the most space and that stuff
| remains latin1. For apps I don't know. But also, text
| size in memory is typically a joke because the expensive
| bit is all the multimedia that accompanies text these
| days. Remember, 100mib is 25million characters even in
| utf32. That's 1 hour or so of high quality audio or a few
| seconds of video. In terms of compression, the smaller
| initial file will typically compress better so not sure
| what you mean by "utf-8" tax. UTF8 is not slower to
| process than utf32 as far as I'm aware except for one
| small corner case of random codepoint access (but then in
| UTF text processing, you really shouldn't be doing that I
| think but I'm not a utf expert ).
| 3PS wrote:
| > Characters can consist of multiple code points.
|
| I enjoy being "that guy", and so, to be unfathomably
| pedantic, I will point out that the Unicode standard
| actually uses the word "character" to refer to _encoded
| characters_, i.e. the mapping from a code point to an
| abstract character [0]. So 1 code point = 1 character for
| Unicode. Of course, in real life, character is a silly
| made-up word that means whatever you think it means and
| using it to refer to an extended grapheme cluster or glyph
| is probably closer to how people really think of them
| anyway.
|
| [0]: See section 3.4 of https://www.unicode.org/versions/Un
| icode15.0.0/UnicodeStanda...
| gavinhoward wrote:
| Speaking as someone who did what GP suggested, it _can_ be
| cross-platform; I just abstracted the string to path and path
| to string conversions, as well as any path operations that
| are different across platforms.
|
| The first thing to realize to make it work is that paths are
| _not_ strings, so you actually don 't want to treat them like
| they are.
| actionfromafar wrote:
| If you need to consider slightly older Windows versions, it's
| WTF-16, not UTF-16.
| chrismorgan wrote:
| There's no such thing as WTF-16--or if you want to use that
| label for "potentially-ill-formed UTF-16", then _everything_
| that says it does UTF-16 is actually WTF-16, because _no one_
| validates UTF-16.
| actionfromafar wrote:
| Yes and no - some programs or systems of programs regularly
| transform between UTF-8 and UTF-16. A file path in Windows
| can be "valid" (or whatever we wanna call it) and still not
| convert cleanly to UTF-8.
|
| Edit to clarify: many systems which deal with non-path
| UTF-16 strings, nowadays are so intertwined with the UTF-8
| world, that they get the validation "for free" so to speak,
| here and there, wherever they interact with the rest of the
| world.
|
| Paths though, remain a common little corner case, working
| fine until it suddenly doesn't, for some weird combination.
| chrismorgan wrote:
| That's just potentially-ill-formed UTF-16, which is what
| _all_ UTF-16 is in practice. The novel thing about UTF-8
| environments is that they (or most of 'em, anyway)
| finally actually validate stuff.
| loeg wrote:
| Isn't it UCS-2 rather than any form of UTF-16?
| protomolecule wrote:
| Not after Windows 2000.
| gavinhoward wrote:
| This is exactly the approach I took in my own cross-platform
| "standard" library. It works.
|
| In particular, paths must be a different type than strings.
|
| As a bonus: it becomes easier to implement niceties like path
| concatenation and realpath.
| cbsmith wrote:
| You can also just over-allocate to avoid having to call it
| twice, or do some really basic heuristics to fast path the call
| (it's very specific UTF-8 sequences that need more than 2 bytes
| in UTF-16).
| smarkov wrote:
| > Don't change random registry settings or code pages on the
| user's computer, that's mad.
|
| A million times this.
|
| Multiple Valve games set the microphone volume in Windows to
| 100% whenever you launch the game, which, for my setup, makes
| me sound unbearably loud. Unless your app has a valid reason to
| change my settings, like if I've set it up to do that on a
| schedule or with some condition, sure, otherwise don't mess
| with my settings.
| formerly_proven wrote:
| Story time
|
| When the USB audio device class was created [1], it had
| support for AVR-like audio controls (volume, balance, tone
| etc.), and, like would be reasonable, made all that work with
| decibels. But, as far as I know, that part of the standard
| got ignored by basically everyone, including some of whom
| wrote it (Microsoft), by presenting e.g. the on-device volume
| control as a 0-100 scale, like Windows and ALSA still do.
| Except... some hardware devices actually implemented the
| spec, including the PCM29xx family of audio codecs. These,
| and clones of them, have been used in quite a few USB audio
| cards and some very low-end audio interface over the decades.
| And for those, "100" in Windows means something like "+30
| dBfs" [2] - so you'll get extreme distortion with basically
| any signal.
|
| [1] 1.0 is actually impressively complex and basically models
| the entire stack of audio devices someone could have possibly
| had in their TV cabinet in the 1995, including stuff like
| dolby and surround decoders.
|
| [2] This is defined around 5.2.2.2.3 and/or 5.2.2.4.3.2 in
| https://www.usb.org/sites/default/files/audio10.pdf
| sn_master wrote:
| Now try doing this for Windows 98 or older, specially the non-
| ASCII file paths part (which was a common use case outside of the
| US).
| ormax3 wrote:
| https://en.wikipedia.org/wiki/Microsoft_Layer_for_Unicode
| sn_master wrote:
| > The MSLU was announced in March 2001, and was first made
| available as a compatibility layer for Unicode-supporting
| code written for the then-new Windows XP RC1 in the July 2001
| edition of Microsoft's Platform SDK.
|
| People had to deal with local language file names since
| Windows 3.1 at least and it became very common with Windows
| 95. Good luck if you wanted to deal with files named in more
| than 1 non-ASCII language (very common in Europe or the
| Middle East).
| herf wrote:
| In addition to maxpath (which can be ignored in most regular
| apps), I remember short filenames used to have some severe
| performance implications. For instance, with Picasa we tried
| never to stat a file over Samba that didn't exist, because it
| would make Windows enumerate all the other files in the folder
| (presumably to see if they were a SFN match). That was a while
| ago (XP), but I just strace'd and it still seems to do it.
| laurent123456 wrote:
| Every time I created software for Windows in recent years I used
| Qt, WxWidgets or even Delphi, which all have wrappers to handle
| system-specific quirks. Is it common these days to code directly
| to the Windows API? It feels like it would lead to code that's
| harder to maintain and certainly less portable.
| ormax3 wrote:
| Like explained in https://utf8everywhere.org/#windows , you can
| write simple wrapper functions `narrow`/`widen` used when you are
| about to call window api functions.
| ::SetWindowTextW(widen(someStdString).c_str());
|
| Implementation is straightforward, relying on
| `WideCharToMultiByte`/`MultiByteToWideChar` to do the conversion:
|
| https://github.com/neacsum/utf8/blob/master/src/utf8.cpp
| mkoubaa wrote:
| I have good experiences with this approach. It isolates the
| weirdness of windows and let's the rest of your code do things
| idiomatically
| huhtenberg wrote:
| You are missing OP's point - this still costs you 2 extra
| calls.
|
| If this cost really matters (and practically speaking it never
| does), then, as the other commenter said, the correct solution
| is to just use OS-native encoding for all file system paths and
| names used by the program, hidden behind an abstraction layer
| if needs be. UTF16 for Windows, UTF8 elsewhere.
| ormax3 wrote:
| The above manifesto makes the argument to use UTF-8
| *everywhere*, even on windows where the internal
| representation is not native utf8.
|
| The conversion overhead is really negligible:
| https://utf8everywhere.org/#faq.cvt.perf
|
| (note: the two api calls per conversion is because how those
| specific functions work, first call to get the size to
| allocate, second to do the actual conversion, but you can
| always use another library in the implementation for the
| utf8<->utf16 conversion that might be more optimized than
| those windows api functions)
| donatj wrote:
| Especially negligible versus the trip to the file system
| you are setting up for.
| jiggawatts wrote:
| Not all API calls are for filesystem access.
| Someone1234 wrote:
| I thought the OP's point is there are too many considerations
| when doing this?
|
| Someone is suggesting a way of making it less tedious, and
| your response is "performance?!" even though in both
| scenarios you're running the same code and it is likely the
| compiler in release would remove the intermediary.
| huhtenberg wrote:
| > your response is "performance?!"
|
| No, that's not what I said.
| ynik wrote:
| "2 extra calls" is a weird metric here. Some calls are vastly
| more expensive than others. Syscalls come with a significant
| cost, encoding conversion of short strings (esp. filenames)
| does not. Hiding just the syscalls behind an abstraction
| layer is vastly simpler than doing that and additionally
| hiding the string representation, so "UTF-8 everywhere" is
| IMHO the right solution.
| eco wrote:
| Microsoft added a setting in Windows 10 to switch the code page
| over to utf-8 and then in Windows 11 they made it on by
| default. Individual applications can turn it on for themselves,
| so they don't need to rely on the system setting being checked.
|
| I haven't tried it yet but with that you can just use the -A
| variants of the winapi with utf-8 strings. No conversion
| necessary.
| adzm wrote:
| Do you have any references about it being enabled by default
| in Windows 11? I've seen conflicting reports and often seems
| to vary depending on the system locale whether it gets
| enabled or disabled by default.
| gumby wrote:
| C++'s filesystem package (part of the standard library) should
| take care of all that nonsense for you. It's whole _point_ is so
| that you don 't have to pay attention to format of the underlying
| filenames, directories, etc.
| kouteiheika wrote:
| > C++'s filesystem package (part of the standard library)
| should take care of all that nonsense for you. It's whole point
| is so that you don't have to pay attention to format of the
| underlying filenames, directories, etc.
|
| Unfortunately _by default_ it doesn 't, because the
| constructors which take `const char *`/`std::string` will
| default to the local codepage, instead of just using UTF-8. So
| you have to use `u8path` or C++20's `u8` string literals to get
| sane portable behavior.
|
| So yes, you technically don't have to pay attention to the
| format, but as it is typical of many features in C++ the
| default is wrong, and you have to manually make sure you don't
| write code that's subtly broken.
|
| (Or alternatively use a linter which would warn about this, but
| does such linter exist?)
| LegionMammal978 wrote:
| I think some people might be wary of the std::filesystem APIs
| because the standard allows implementations to completely
| disregard TOCTOU issues internally, to the point of breaking
| memory safety [0]:
|
| > A _file system race_ is the condition that occurs when
| multiple threads, processes, or computers interleave access and
| modification of the same object within a file system. Behavior
| is undefined if calls to functions provided by subclause
| [filesystems] introduce a file system race.
|
| It's not just implementation-defined behavior, but full UB!
| You're utterly at the mercy of your implementation to do
| something reasonable when it encounters a TOCTOU issue, or, for
| that matter, any kind of concurrent modification to a file or
| directory. And C++ has a long history of implementations being
| unreliable in their behavior when UB is encountered.
|
| [0] https://wg21.link/fs.race.behavior#1
| gumby wrote:
| The more limited C API (system calls) is also full of race
| opportunities. The windows file APIs are better, but not a
| lot better.
|
| Anyway, the problem described in the post (scanning the
| string twice when converting) is as race prone, or not, as
| the the filesystem API.
| LegionMammal978 wrote:
| > The more limited C API (system calls) is also full of
| race opportunities. The windows file APIs are better, but
| not a lot better.
|
| Sure, there are opportunities for races, but POSIX and the
| Windows API limit the possible outcomes of concurrent
| modification, and they also provide tools (fixed file
| handles, etc.) for well-written programs to prevent TOCTOU
| bugs. Meanwhile, the std::filesystem API just throws its
| hands in the air and says that filesystem races are UB,
| period, making it unusable if the program isn't in complete
| control of the directory tree it operates on.
|
| > Anyway, the problem described in the post (scanning the
| string twice when converting) is as race prone, or not, as
| the the filesystem API.
|
| I don't see what you mean? The typical scenario here is,
| you receive some absolute or relative path from an external
| source, encoded in ASCII or UTF-8 or some other non-UTF-16
| encoding, and you want to operate on the file or directory
| at that path. Your internal path string is never going to
| change, only the filesystem can change. So there aren't any
| races from scanning your internal string twice to re-encode
| it; races can only come from _using_ the re-encoded path in
| multiple calls to the filesystem API.
| forrestthewoods wrote:
| Ehhhhhh. Passing ascii file paths to a Windows API is really
| straight forward and trivial. No conversion needed.
|
| MaxPath is real though. Exceeding it not recommended.
| londons_explore wrote:
| Tl:DR: there are many different ways to send file paths to system
| library functions. That's mostly because of Windows' long legacy
| support history. Don't worry about it - just pick one and go.
| c4mpute wrote:
| No. Because the API is fragmented into old, not-so-old and new
| parts, some higher-, some lower-level[0]. You need things from
| all of them, the new API isn't complete enough and has warts in
| places where using one of the older ones is
| better/easier/safer. So in the end you will have to use all the
| ways, and know when to use which one.
|
| All this can of course be fixed with another layer of
| indirection. But if you ever wanted to know why Windows file
| APIs and filesystems are so dog-slow, here is (part of) your
| answer.
|
| [0] e.g. there are lower level APIs that can create and access
| files and filenames that higher level APIs will filter out or
| disallow.
| HeckFeck wrote:
| I feel inclined to bypass all that Win32 gubbins and talk to
| the Kernel directly.
|
| Back in my day we just walked into the office of the man in
| charge and shook hands, we struck deals like real men. If we
| ask ask the NT kernel politely in a language it understands,
| we will get our filepath. Right?
|
| http://undocumented.ntinternals.net/index.html?page=UserMode.
| ..
| huhtenberg wrote:
| NT API is now documented by Microsoft. The coverage is not
| complete, but enough to be useful. E.g.
| https://learn.microsoft.com/en-
| us/windows/win32/api/winternl....
| sn_master wrote:
| > new API isn't complete enough and has warts in places where
| using one of the older ones is better/easier/safer
|
| Describes Windows Forms vs everything else MS has been trying
| to do in the past 15 years.
| shortrounddev2 wrote:
| > Alternatively you can set the process code page to UTF-8 and
| call the 'A' variant API directly, but only sometimes, and only
| with Windows 10 v1903+, and you might still have to change the
| system locale setting and reboot
|
| There is no need to do this; you can just call the A variant by
| name. Wide char and ascii functions are distinguished with W or A
| at the end, such as CreateFileA or CreateFileW. CreateFile is a
| macro.
| daemin wrote:
| I always directly call the A or W versions of these functions.
| It makes things unambigious.
|
| Additionally I have my own Windows header file which undefines
| all of the macros so my function names do not get mangled with
| A and W suffixes.
| civilized wrote:
| Why are people obsessing about the overhead of two function
| calls? What application are you writing that needs to optimize
| passing millions of long file names in real time? Is this some
| kind of file path based video game?
| gpderetta wrote:
| Git, for example, used by millions of developers every day.
| civilized wrote:
| It's still going to be a tiny fraction of what git does, so
| speeding it up will have a negligible impact on performance.
| cerved wrote:
| Git reads and writes a lot of files in the working tree as
| well as the object store. It's a large part of what it does
| civilized wrote:
| I'm aware that passing a file name is a part of working
| with a file. I'm just skeptical that it's such a large
| fraction of the computation that optimizing it would
| provide a noticeable performance boost.
| im3w1l wrote:
| Crazy idea... but what if you stored the files in a fast
| purpose-built database instead of the normal file system. You
| would have to mount it I suppose, to be able edit the files.
| But I think it could be very performant.
| gpderetta wrote:
| The filesystems is a database for storing files already.
| There is no particular reason a different approach be
| faster in principle.
| kmeisthax wrote:
| Git actually does have a separate disk format that's way
| more database-like, but it's only used for older data
| that doesn't need to be shuffled around as often. Git was
| written assuming the filesystem is decently fast at
| metadata mutation, which is a good assumption on Linux
| and less so on Windows.
| im3w1l wrote:
| Git needs some queries that filesystems arent very good
| at natively, like "get all modified files". There are
| workarounds but yeah.
| mdaniel wrote:
| https://en.wikipedia.org/wiki/WinFS
|
| I was actually secretly cheering for that project, because
| I thought it could be actually innovative
|
| But, as I look back on that with my modern understanding of
| Microsoft's backward compatibility focus, I realize such a
| thing would never, ever fly
| Tigress8780 wrote:
| Talking about how strange the current state of Windows API is,
| there are actually 3 different APIs that can move the mouse
| cursor: SetCursorPos [1], SendInput [2], and mouse_event [3].
|
| Applications might react differently to input events generated by
| these APIs. For example, in Windows 11's display settings, the
| position of screens can not be dragged if input events are coming
| from SetCursorPos, but works fine if using SendInput. Microsoft's
| own PowerToys uses a mixture of both under certain (complex)
| conditions, but I never found out the actual difference between
| them.
|
| I was writing an application that sends mouse input from a Linux
| machine to a Windows one (similar to Synergy), and I originally
| received mouse movement events that are accelerated (with user or
| system-defined acceleration factors), and I found out that there
| are no Windows API (three of them) that accepts relative
| movements without further acceleration (i.e. Windows will always
| apply further acceleration, making the mouse hard to use). I
| ended up directly hook into evdev to get raw mouse movements and
| let Windows accelerate them.
|
| [1] https://learn.microsoft.com/en-
| us/windows/win32/api/winuser/... [2]
| https://learn.microsoft.com/en-us/windows/win32/api/winuser/...
| [3] https://learn.microsoft.com/en-
| us/windows/win32/api/winuser/...
| lights0123 wrote:
| What do remote desktop tools do? TeamViewer allows mouse
| movement from even a phone touchscreen just fine, and unless
| they're computing the inverse of Windows's mouse acceleration,
| they must have another solution.
| Tigress8780 wrote:
| On desktop clients, they fetch the cursor bitmap and renders
| it locally, then send absolute movements. For mobile clients,
| it is possible (and better) to send relative coordinates and
| let Windows accelerate that action, since touch events are
| not accelerated. Doing this acceleration actually makes the
| client feel more like a laptop touchpad if you are using
| relative mode.
| tjoff wrote:
| Some at least don't even bother to send relative mouse
| movements, what matters is where you click. So they disable
| the guest mouse-pointer and rely on the host instead and then
| only send positions every now and then and when you are
| actually doing something interactive.
|
| Synergy types of applications don't have that freedom because
| the host mouse cursor don't extend to the other devices
| display.
| jeroenhd wrote:
| Only three ways to move the mouse seems relatively tame
| compared to the forest of Linux API calls when it comes to
| input (depends on X11/Wayland among other things). Also, your
| last link clearly states that the API call has been superseded,
| so if you follow the documentation you only have two options.
|
| I don't think the difference between the two is all that
| strange. One sets the position of the cursor, the other
| interacts with the system like a normal mouse. The mouse and
| the cursor are separate things, and they're handled at
| different levels in the API stack, like XSendEvent and sending
| data to libinput.
| ilrwbwrkhv wrote:
| I don't know why but Microsoft programmers seem to make bizarre
| code choices. I worked with a lot of principal developers in my
| life but the ones from Microsoft wrote the most convoluted code I
| have ever seen. Even though it's anecdotal I wonder what is going
| on in the culture that Microsoft keeps producing these shabby
| programs all over the place.
|
| Casey has also spoken out about the horrible state of affairs in
| Microsoft in the past
| https://news.ycombinator.com/item?id=27728177 so I'm sure it's
| not a one off phenomena.
| patmorgan23 wrote:
| I think I read on another hacker news post that some windows
| API where intentionally made convoluted/hard to work with AND
| poorly documented in the official documentation. So that the
| API author's could go off and write a "win32 explained" book,
| and become fabulously wealthy.
|
| Allegedly.
| ilrwbwrkhv wrote:
| Yes! I remember that too. I wouldn't be surprised.
| [deleted]
___________________________________________________________________
(page generated 2023-09-24 23:01 UTC)