[HN Gopher] Managarm: Pragmatic microkernel-based OS with asynch...
___________________________________________________________________
Managarm: Pragmatic microkernel-based OS with asynchronous I/O
Author : ksp-atlas
Score : 139 points
Date : 2024-07-27 19:06 UTC (1 days ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| jcelerier wrote:
| people always say that C++ is not a good fit for kernels, but so
| far this is the only language I know where small teams or
| individuals are regularly able to create non-trivial hobby OSes
| from scratch that go from zero to GUI:
|
| - Serenity (https://github.com/SerenityOS/serenity) - not really
| a small team, but it managed to get to GUI as pretty much a one-
| man-show
|
| - Skift (https://github.com/skift-org/skift)
|
| - hhu: https://github.com/hhuOS/hhuOS
|
| - MaxOS: https://github.com/maxtyson123/MaxOS
|
| - MorphiOS: https://github.com/syedtaqi95/morphiOS
|
| - Macaron: https://github.com/MacaronOS/Macaron
|
| - Ghost: https://github.com/maxdev1/ghost
|
| most big ones in C don't manage to get to the GUI level, except
| toaruos: https://github.com/klange/toaruos
| mananaysiempre wrote:
| HelenOS[1] is in C and has a GUI. I don't know how many people
| participated over its (quite extensive) history.
|
| Axle[2] is a one-man project with a GUI that the author has
| been gradually transitioning from C to Rust.
|
| Among C++ projects, I think Essence[3] also merits a mention.
|
| [1] http://www.helenos.org/
|
| [2] https://github.com/codyd51/axle
|
| [3] https://gitlab.com/nakst/essence
| codyd51 wrote:
| Hi, author of axle here - thank you for the shout out! It's
| been a wonderfully fun and enriching project to work on over
| the years. I'm now working on XNU at Apple, so won't be
| working further on axle for the foreseeable future.
| ksp-atlas wrote:
| Congrats on the job
| ironhaven wrote:
| There is Redox written in rust. https://gitlab.redox-
| os.org/redox-os/redox
| fbdab103 wrote:
| Is the kernel really the differentiator there? seL4 is a Proven
| microkernel that is some 9000 lines of C. This ancient SO
| post[0] claims Linux is 140k lines. The kernel is just a tiny
| component of the many things required to get an OS up and
| running. I suspect most projects just peter out as the enormity
| of the complexity becomes apparent.
|
| [0] https://unix.stackexchange.com/questions/223746/why-is-
| the-l...
| monocasa wrote:
| That's just the architecture and driver independent part of
| the kernel. The kernel source tree is 15M lines or so.
| asveikau wrote:
| I think C++ is especially good for a GUI.
|
| The two counter-examples I can think of with plain C:
|
| * GTK+, though honestly speaking I think quality has dipped
| over the decades where C++ equivalents have flourished.
|
| * Old school Win32 style. Counter to the bad reputation, I find
| it easy to be productive with in plain C once you adjust your
| mental model to its expectations. Though it's probably better
| from C++ than C, for a few convenience features to reduce
| boiler plate.
| mananaysiempre wrote:
| Old Win32 style received less attention than it deserves, I
| think. It's an object system in its own right--a Smalltalkish
| one, at that, with an (admittedly muddled) E-style separation
| of synchronous and asynchronous calls!--but one rarely sees
| it mentioned in the same sentence as Objective-C, C++, and
| GObject/Vala. Part of the blame for that undoubtedly rests on
| Microsoft's incompetence at documenting concepts, but
| whatever the causes I'd definitely like to see that corner of
| the object-system design space explored more, with or without
| GUIs.
| asveikau wrote:
| When win32 was dominant they didn't need to document it,
| people had the motivation to figure it out.
|
| Successive waves of Microsoft people then tried to make it
| easier to use or replace it entirely, which, they never did
| so with the same quality as the original.
|
| Unfortunately some of those misadventures became synonymous
| with building a windows UI.
| tomcam wrote:
| > Part of the blame for that undoubtedly rests on
| Microsoft's incompetence at documenting concepts
|
| I mean Charles Petzold did that so successfully redoing it
| would be superfluous
| mananaysiempre wrote:
| Kind of. He stops somewhat short of admitting that
| Win16/32 windows _are_ objects and their graphical
| representation is mostly incidental. That's not a bad
| thing--the book is introductory on many topics, in a good
| way, and just dropping such an idea somewhere in the
| beginning portions would be more confusing than helpful
| for the intended audience. That audience would almost
| certainly not be helped by the observation that they are
| dealing with a sort of Smalltalkish /Actorish system with
| badly bolted-on multithreading--most of it wouldn't have
| knowm what that meant, at the time.
|
| Still, this is kind of a common theme. Perhaps "you had
| to have been there" worked as an approach to
| documentation for the first ten to fifteen years, but
| afterwards any newcomer is just guaranteed to get lost.
|
| If you want to learn Win32, you need to read Petzold. If
| you're interested specifically in how the dialog manager
| works, you need to invent a time machine and read a
| series of blog posts Raymond Chen wrote in 2005[1]. If
| you want the particulars on message queueing, you need to
| invent a time machine and read the chapter about then in
| Raymond Chen's blogobook[2]. If you want to learn about
| STRICT and the very useful message cracker macros in
| WINDOWSX.H, first, you were probably reading Raymond
| Chen, because the official documentation is basically mum
| about it, but second, you need to track down that one KB
| article about porting to Win32 that accompanied one
| particular Windows 3.1-era Microsoft compiler[3].
|
| If you want to learn DCOM95-vintage COM, you need to read
| Box and Brockschmidt and that _COM Programmer's Cookbook_
| [4] thing that was somebody's own private initiative
| inside Microsoft and is buried somewhere in the technotes
| and _then_ the COM Specification[5] will be accessible to
| you, and _only after that_ will the MSDN specification
| make some sort of sense. If you want to learn the how or
| why of COM marshalling in particular, you will perhaps be
| helped by some of the above, but really you need to
| invent a time machine and read a series of blog posts
| Raymond Chen wrote in _2022_ [6]. Only then will the MSDN
| reference be even slightly helpful (and even then not
| that much over the IntelliSense hints).
|
| If you want to learn ActiveX ... I have no idea what you
| need to read! And that itself is indicative of a problem
| (took me five years to stumble upon Brockschmidt).
|
| If you want to learn MSMQ/MTS/COM+/whatever, you need to
| read that one book[7] that's still going to leave you
| with more questions than answers, and then maybe track
| down some MSJ articles from after COM+ was only a vague
| marketing term for something internal, but before .NET
| completely overtook all Microsoft communications
| channels. If you want to learn about using COM contexts,
| first, my condolences, second, AFAIK your _only_ source
| for this Win2K-era, and I quote, "backdoor into the very
| low-level COM infrastructure" is Raymond Chen's post from
| two decades later[8]. There's no motivating spec for any
| of this, even one as scarce on details as the later parts
| of the COM one.
|
| If you want to learn about WinRT internals, well, there
| are some blog posts random people on the Internet have
| written[9], and maybe you can spelunk some in the
| Win8-era legacy MSDN docs that are somewhat more explicit
| about what's happening. If you want to learn about
| WinRT's headline feature of "metadata-driven
| marshalling", _fuck you_ , it's not explained anywhere,
| nor is an up-to-date MIDL grammar that would include the
| ways to adjust it a "priority" for that team--or maybe
| it's just that I don't have a time machine, but Raymond
| Chen is mortal too, and it's been over a decade.
|
| And it's all like that. (Eric Lippert copied _some_ of
| the articles on Active Scripting / WSH from his now-
| deleted MSDN blogs, but not all of them. Michael Kaplan's
| blog, one of the best resources on Windows i18n, was
| speed-deleted from MSDN after he published a Vista leak,
| and the man himself died shortly afterwards, so for
| anything after that you're out of luck. Etc., etc. Need I
| say there's no type system spec for TypeScript?)
|
| Again, if you were there at the time and you subscribed
| to MSDN, until 2005 or so I think you would've gotten
| basically all of the things I listed on the CDs, and more
| besides (Dr. GUI and Dr. International anyone?). But if
| you only joined in 1995 or 1998 or 2000, God help you.
| (WinRT happened after 2005, though, so the sum total of
| in-depth, under-the-hood narrative stuff about it is fuck
| all.) The references from that era are excellent, but as
| far as, again, narrative docs are concerned-- C'mon,
| guys, you had _Ted Chiang_ at your disposal and that's
| what you managed?..
|
| (Then again, it probably had very little to do with the
| technical writers themselves and a lot to with the
| preposterously hectic development and a management
| culture that just did not prioritize this sort of thing.
| But still.)
|
| [1] http://web.archive.org/web/20231204102018/https://byt
| epointe... (wait, has bytepointer.com died? that would
| make me very sad...)
|
| [2] https://openlibrary.org/works/OL9256722W/The_old_new_
| thing
|
| [3] Nope, can't find it right now! Good thing Microsoft
| deleted most of the old KB articles from their website...
| _--an hour passes--_ Found it! Q83456: https://raw.github
| usercontent.com/jeffpar/kbarchive/master/t... (thank
| goodness these were numbered).
|
| [4] https://learn.microsoft.com/en-us/previous-
| versions/ms809982...
|
| [5] What, were you expecting a microsoft.com link?
| Hahaha, nope! Just imagine you're listening to Rick
| Astley, I guess. Anyway, there are two versions in
| circulation (originally as DOC). The "Component Object
| Model specification", version 0.9, is dated 1995 and is
| easier to find, e.g.: https://groups.csail.mit.edu/medg/f
| tp/emjordan/COM/THE%20COM.... The "COM core technology
| specification", version 1.0, is dated 1998 and is, as far
| as I know, only available from this one Russian-language
| homepage, last updated 2004 (and you can certainly tell):
| https://thegercog.narod.ru/index.htm?u=https://thegercog.
| nar..., direct link: https://thegercog.narod.ru/Files/Doc
| s/ds/com/com_spec.rar. A lot of the same material is in
| the DCOM Internet-Draft and the ActiveX Spec drafts on
| the Open Group website.
|
| [6] https://devblogs.microsoft.com/oldnewthing/20220615-0
| 0/?p=10...
|
| [7] https://thrysoee.dk/InsideCOM+/
|
| [8] https://devblogs.microsoft.com/oldnewthing/20191128-0
| 0/?p=10...
|
| [9] https://www.interact-
| sw.co.uk/iangblog/2011/09/25/native-win...
| actionfromafar wrote:
| Completely amazed that you could write this comprehensive
| reply just like that.
| mananaysiempre wrote:
| I guess I'm just a bit obsessed with Golden Age
| Microsoft, having grown up during that time without
| understanding much of what was going on. My original
| point, though, was that they had some very nice, perhaps
| thesis-chapter-worthy points that ended up buried in the
| specifics of (and on more than one occasion, in the same
| grave as) their products.
|
| Windows's, well, windows as a object/concurrency system
| are one (even if the Win32 transition to multithreading
| is IMO conceptually botched). The actor model admittedly
| already existed at that point, but Mark Miller et al.'s
| work on E (the only other source I know for having
| separate synchronous and asynchronous sends) hadn't even
| started at that point. (There are innumerable papers on
| various versions of actors, though, so I might have
| missed it.)
|
| COM aggregation is another. It would probably have been
| much more successful had they used fat pointers, and
| there are a couple of old papers pointing this approach
| is a viable way to do implementation (as opposed to
| interface) inheritance. But I know of no precedent in the
| literature for the inside-out way WinRT builds its
| allegedly more conventional inheritance mechanism on it.
| (I guess I would've referenced BETA if I were writing a
| paper on it?)
|
| DCOM's "causality IDs" for reentrancy control are yet
| another. You can basically have calls to your object
| guarded by a _recursive_ mutex whose ownership is passed
| along with any remote calls that that object itself
| makes. I'm not sure they are a _good_ idea (recursive
| mutexes are perilous in the simplest of situations, and
| this is most definitely not one of those), but I'm almost
| certain it's an _original_ one.
|
| That's it, for now, but I'm almost certain I could find
| more, if only I knew where to look.
| amluto wrote:
| Recursive mutexes are a terrible idea, except maybe in
| context. In a world where you can SendMessage a message
| to yourself or to another window in the same thread,
| process the message synchronously, and then return back
| to where you were, you have essentially uncontrolled
| reentrancy. If you want to throw mutexes into the mix,
| they pretty much need to be recursive. Then you get to
| acquire the mutex in a message handler to protect against
| other threads, and everything sort of works.
|
| In my own code, I try fairly hard to avoid this sort of
| reentrancy.
| tomcam wrote:
| Agreed. It is a tour de force.
| asveikau wrote:
| I learned a bunch of these things largely on my own, but
| it probably helped that I was at MSFT for a few years.
| tomcam wrote:
| Thanks for debunking my answer so comprehensively. Cannot
| disagree.
| kragen wrote:
| this is such a valuable comment. thank you for writing
| what may be the best guide to microsoft windows
| documentation i've ever seen
| WesolyKubeczek wrote:
| OOP is especially good for making UI toolkit. If you look at
| GTK+ and Win32 closely, you'll notice that they all sport
| some kind of a homegrown class system, complete with
| inheritance and polymorphism.
|
| It's popular to dunk on OOP and its concepts nowadays, but I
| think that languages that straight away shun them because
| "OOP sucks" are an example of their authors overreacting to
| OOP's dominance back in 1990s-early 2000s and tendencies to
| shove OOP into every nook and cranny, with a notion that if
| you don't do it, or do not enough of it, your solution is
| inferior.
|
| Holy hell. Linux kernel is object-oriented. Because it's damn
| convenient for a lot of things.
| bsder wrote:
| You are seeing a creation date bias. And it doesn't help that
| there just aren't that many systems programming languages.
|
| Until Rust in 2010, there basically was almost no motion in the
| "system programming space". Even afterward, there needs to be
| enough uptake to actually have critical mass. Only then can
| people start using it for projects.
|
| Side note: D actually predates a lot of this by being from
| 2001, but, sadly, never seemed to get any traction. It seems
| like it had the misfortune of being about 10 years too early
| and that programmers just weren't ready for a new systems
| programming language at that point.
| pezezin wrote:
| > Side note: D actually predates a lot of this by being from
| 2001, but, sadly, never seemed to get any traction. It seems
| like it had the misfortune of being about 10 years too early
| and that programmers just weren't ready for a new systems
| programming language at that point.
|
| I was only a kid starting my university degree at that time,
| but I had been coding since I was 9 and I was very excited
| about D. What I remember is that getting it up and running
| was a cumbersome, fully manual process. The compiler was
| closed-source, delivered as a tarball without any kind of
| installation script, ditto for the libraries. I wrote some
| little programs and I liked the language, but in the end I
| gave up.
|
| On the other hand, Rust has rustup and Cargo which are just
| amazing. I am sure that a big part of Rust's popularity comes
| from Cargo.
| voidfunc wrote:
| Yea D stagnated due to some terrible decisions by Digital
| Mars.. it never built a serious community.
| gnufx wrote:
| There have been a number of OS-type systems in Ada/SPARK,
| though at least mainly in the embedded space. E.g. the Muen
| verified separation kernel: https://muen.sk/
| mrbluecoat wrote:
| Are ARM and RISC supported or are the errors on
| https://builds.managarm.org placeholder stubs?
| no92 wrote:
| The ARM and RISC-V ports are in different states of being a
| work-in-progress.
| dang wrote:
| Apparently very little past discussion:
|
| _Managarm: August 2022 Update_ -
| https://news.ycombinator.com/item?id=32515546 - Aug 2022 (3
| comments)
|
| _The Managarm Operating System_ -
| https://news.ycombinator.com/item?id=24689727 - Oct 2020 (1
| comment)
| npalli wrote:
| The github repo was updated just 5 days back. Not sure what the
| state was previously, so not much discussion.
| ksp-atlas wrote:
| Managarm has gained a bunch since then, like the ability to
| run a modern web browser
| tomcam wrote:
| Very neat project! It appears to run on qemu. Can someone tell me
| at the 30,000' view what's required to get it running on actual
| hardware? I'm imagining something like signed boot manager, hard
| disk drivers, etc. They seem to have a ton of USB stuff working
| which seems amazing to me, but I've been out of the low-level PC
| loop for a couple decades and don't know what prevents working on
| bare metal in 2024.
| boricj wrote:
| At 30,000' I'm not seeing anything that would prevent this from
| booting on hardware. It seems to have a reasonable complement
| of modern, real-world drivers. On the other hand, physical
| hardware tends to have bugs, quirks and diversity that VMs
| don't have, so hobby operating systems may have trouble with it
| if they aren't tested on it.
| toast0 wrote:
| I've got a hobby OS that runs on real hardware (sometimes, on
| some hardware). Mostly it's just patience and debugging effort
| to get things going. qemu defaults to doing some things quickly
| instead of accurately, so it's easy to do things wrong as an OS
| and if you don't test regularly, you can end up with a lot of
| breakage.
|
| I don't have a boot manager, my OS is multiboot compatible, so
| I can boot with qemu's multiboot support or PXElinux, or grub
| or whatever. IMHO, with the multiboot standard, there's no need
| to write a boot loader if you want to build an OS, and no need
| to write an OS if you want to build a boot loader.
|
| Looks like this OS has a good selection of storage drivers, so
| should be good to go there.
| no92 wrote:
| I have managarm installed on a partition on my testing
| ThinkPad. The steps needed to get it set up were creating the
| partition, copying over the sysroot, and adding an entry to my
| grub configuration.
| tomcam wrote:
| Awesome. How is the overall experience?
| ksp-atlas wrote:
| Theoretically, since it has a few real world drivers, it may be
| possible to run on real world hardware, but in practice,
| Managarm tends to get hung up very low level, at Eir or Thor
| (the two components of the microkernel). Since basically all
| the testing is currently done on QEMU, real world testing isn't
| really a thing for Managarm
| dilippkumar wrote:
| This is mostly a note to myself.
|
| It's interesting to consider "what next" once an OS project
| reaches this stage. There are soo many directions a team can
| take, but also, none of those direction lead to a clear path
| towards massive user adoption.
|
| There are obvious holes/gaps in what mainstream OSes offer today,
| however it is not clear how a project goes from here to
| addressing those gaps, and even if those were to be addressed, it
| is not clear how it could displace mainstream OSes.
|
| We are clearly better off having projects like this, that give us
| options in case something were to go horribly wrong with
| mainstream OSes. However, what is the incentive to keep projects
| like this alive when the path ahead is soo unclear?
|
| How does one disrupt the operating systems market?
| lasiotus wrote:
| Motor OS (https://motor-os.org) attempts to do exactly that, by
| focusing on a rather narrow, from a "mainstream OS", point of
| view, niche. Kind of "do this one thing better" approach.
| globalnode wrote:
| rather than displace mainstream os's what id rather have is
| some way to easily switch os depending on my task, gaming?
| windows, anything else? linux (for now), and have that run
| directly on the hardware with little interference... like a
| meta task switcher os.
|
| someone is going to come on here and tell me we already have
| this (i hope).
|
| ive been out of computing for so long i dont know if this
| reliably exists, but back in the day this is what i would have
| wished for.
| globalnode wrote:
| and since i dont trust windows to host my work os securely,
| and performance of games would be abysmal with the hosting
| roles reversed, what i really need is 2 boxes and a kvm
| switch!. the question now becomes, are there physical systems
| that turn on and off at the same time... i.e. share a psu but
| host different os's!
|
| suppose i could build my own
|
| am i looking at building a rack mounted system? heh, think i
| just found a project for myself.
|
| 1 psu, 1 gpu (for gaming), 1 kb/monitor, 2 hdd, 2
| motherboards, 1 kvm switch, build my own rack out of wood,
| done! so the only extra cost really should be the 2nd
| motherboard and the kvm switch.
| paholg wrote:
| You can avoid the kvm with a cheap USB switch and software.
|
| https://github.com/haimgel/display-switch
| globalnode wrote:
| I was actually starting to think about how I could make a
| KVM due to the costs but this link looks really
| interesting. Just have a PC and a work laptop and switch
| between the two. Thank you!
| ksp-atlas wrote:
| A hypervisor does what you want, though note that the low
| level ones that run as the OS are generally geared towards
| servers, not desktops
| globalnode wrote:
| You're talking about a type 1 hypervisor from what I've
| just read. Linux KVM/QEMU looks interesting. Do you think
| win 11 would work at near native speeds under this?
| Rochus wrote:
| Is this a hobby project or intended for real applications? How
| does it compare performance-wise to Minix 3 or the L4 family (one
| of which, Hazelnut, was also written in C++)?
| kragen wrote:
| epoll, wayland, usb, and smp on a microkernel is already a very
| promising feature list
___________________________________________________________________
(page generated 2024-07-28 23:10 UTC)