[HN Gopher] Axle OS
___________________________________________________________________
Axle OS
Author : mmphosis
Score : 228 points
Date : 2023-05-20 01:13 UTC (21 hours ago)
(HTM) web link (axleos.com)
(TXT) w3m dump (axleos.com)
| josephg wrote:
| This is super cool!
|
| I've had an idea I've been wanting to build into an OS for ages -
| where we use versions, message passing and structured patches to
| pass information between the kernel and application processes. So
| an application can subscribe to a file / USB device state / list
| of running processes / anything using a common, standard API. But
| it sounds really daunting to try and hack something like into the
| Linux kernel or something in one go.
|
| If I want to experiment with something like this, what do you
| recommend? How hard would it be to do something like that in a
| hobby kernel like Axle?
| ww520 wrote:
| Linux exposes system information like processes and devices in
| the psuedo file system hosted at the mount point /proc. Any
| user mode programs can use the standard file IO API to access
| them. Some of the "files" like /proc/self/mounts are pollable,
| i.e. a program can use blocking select to wait for changes like
| when a new file system has been mounted.
| josephg wrote:
| Yeah, I know how it works in linux. I just hate it. Polling
| and writing a parser for a pseudo-file always seems like a
| super roundabout, inefficient, brittle way to send
| information from kernel to userspace.
|
| Think about it - the kernel stores some data in a nice, neat
| struct. And the kernel knows exactly when that struct is
| modified. Userspace wants that data, and wants to know when
| the data changes. So every second, userspace "reads" a pseudo
| file. The kernel packs the struct's contents into a big
| string with some sloppy, hand written text format and passes
| it to userspace. Then in userspace you need to write a
| brittle parser for whatever the kernel gives you, knowing
| that in different versions of the kernel you'll get slightly
| differently formatted strings.
|
| When the data changes in the kernel, the application has no
| way of being notified. It just polls and parses in a loop.
| Its both inefficient (when there are no changes, its all
| wasted effort). And its laggy (when changes happen, it'll
| take your application up to the polling interval to find out
| about the changes).
|
| For things like filesystem watching, there are special APIs
| applications can consume. But those APIs are different for
| every kind of data the kernel manages. Devices, mount points,
| USB connectivity, filesystem watching, CPU usage, thermal
| sensors, open network connections and on and on. All of this
| stuff has totally different kernel-level APIs for "watching"
| changes. (If you're lucky - a lot of it can only be polled).
| Its a big fat mess. I want to try replacing all of that with
| a nice clean API for fetching & subscribing to any kind of
| data. So, the kernel has a standard API for fetching (maybe
| it returns JSON). And a standard API for subscribing to
| changes - such that your application is notified when a
| change occurs, and the change is sent in a standard, easily
| digestible patch format.
|
| And ideally, eventually, I'd like applications to be able to
| share their own changes back with the kernel in the same form
| & format. And maybe, share data objects between applications
| like that - with the kernel acting as a broker for sending
| patches back and forth, and syncing them with the filesystem.
| I think that would be really neat and useful for all sorts of
| applications.
|
| But I want to prototype it all first in something thats not
| linux. I'm just not sure where to start. I don't really want
| to write an OS from scratch just to try out this generic
| state passing idea.
| ww520 wrote:
| I agree with you mostly. In some cases the performance
| might not be desirable to deliver the events. One minor
| nitpick, pollable endpoints on /proc means that a program
| can run select() or epoll() on the file descriptor. Those
| can be blocked to sleep, no need to re-poll at intervals.
|
| Windows has WMI and other systems have SNMP MIB as a
| uniform interface to access system resource. Those never
| get anywhere.
| grepfru_it wrote:
| In Unix everything is a file. Linux is not Unix but tries
| to be like Unix. This all files represent a resource on the
| system. It's hard to separate this from POSIX requirements.
| Lose POSIX compatibility and you end up with a half
| implemented system like Windows. And then you must recreate
| the entire application ecosystem yourself or you end up
| with this funky BeOS/Haiku style POSIX compatibility layer
| that excludes a good subset of applications that already
| exist.
|
| PS /proc isn't a file system, it is an api (that can be
| accessed by the file system)
| nerdponx wrote:
| How do you access the contents of /proc programmatically?
| ww520 wrote:
| cat, fopen, fread, fwrite, etc.
| nurettin wrote:
| read/select loop? Or am I too outdated?
| [deleted]
| rajeevk wrote:
| There are a lot of resources here in case you want to write a
| kernel from scratch. https://wiki.osdev.org/Main_Page
|
| I wrote a unix-like kernel from scratch long back. It was not a
| fully functional unix but had a lot of components ready to the
| extent it would provide a command line interfaceI from which
| you can run commands like ls, cat etc by loading elf
| executables.
|
| I would highly recommend to read this tutorial
| http://www.osdever.net/bkerndev/Docs/title.htm
|
| Feel free to reach out to me (email in my profile page) if you
| want to discuss the topic with me.
| blown_gasket wrote:
| I've been pondering this for a moment and if I were to write an
| application for such a system I don't think I'd enjoy it. If
| you want to pass information from the kernel and bubble that
| all the way to the application to do something with it, I would
| think you now have to write signal handling in your
| application.
| codyd51 wrote:
| Hi, author here!
|
| Thanks for the kind words! I'll echo rajeevk's recommendation:
| the OSDev wiki is full of useful documentation and guides for
| setting up a bare-metal toolchain. There's also a community
| forum with a wealth of knowledge just waiting to be read.
|
| In terms of boots on the ground, I remember once seeing a
| mantra from someone on the OSDev forums that went something
| like this: building an x86 OS is first about getting the CPU
| into a usable state, then about getting as far away from it as
| possible. This is really true! There's a lot of work to do to
| set up the features that any modern environment needs as a
| baseline: paging, segmentation, interrupt descriptors, etc, but
| once you're done with all that you're free to roam in the
| design space and build whatever you like, however you imagine,
| building abstractions and then abstractions on top of those.
|
| I'd liken the early stages of OS development to a tech tree in
| a video game: you need serial output so that you can debug your
| kernel. You need to be able to allocate memory so that you can
| create pixel buffers, then need drawing primitives to fill them
| up. You need processes, and a scheduler to coordinate them.
| Every new feature you add opens up a myriad of possibilities
| that weren't there before, and although many things have a
| serial list of dependencies, there's always so much to do that
| you can just focus on whatever feels particularly inspiring to
| you at any given time. It's really a lot of fun.
|
| axle also uses structured messages for making requests to the
| kernel, as well as everything else. Here's an example of some
| structured requests that can be sent to the kernel:
| https://github.com/codyd51/axle/blob/paging-
| demo/kernel/kern.... Here's some structured data that an
| application (in this case, the window manager) uses to interact
| with other apps: https://github.com/codyd51/axle/blob/paging-
| demo/rust_progra...
|
| All the best, and feel free to reach out to me as well!
| josephg wrote:
| Thanks for the links and encouragement! I suppose my main
| hesitation is that it seems like an awful lot of work to
| build all that from scratch. I really want to - it looks
| super fun. But I've got my hands totally full at the moment
| building a CRDT & data engine for local first software.
|
| I really just have one idea I want to explore, which is this
| idea around sharing state between the kernel and userland via
| subscriptions and patches. (By taking some ideas from event
| sourcing and things like that). I guess I'm trying to figure
| out where I would even start prototyping something like that.
| I guess I could just fork axle and hack on it a bit to see
| how it goes?
| bionsystem wrote:
| You can probably also hack into some of those educationnal
| OSes that already have a workable base, i.e xv6, os161,
| maybe even minix ? This way you wouldn't have to implement
| the initial tedious layers yourself.
| wilg wrote:
| I love that hobby OSes are a thing!
| metadat wrote:
| See also:
|
| _I wrote a GameBoy emulator for my hobby OS (axleos.com)_
|
| https://news.ycombinator.com/item?id=36004553 (13 comments)
| pkphilip wrote:
| What I would love is an alternative to Android and IOS for mobile
| devices! Something with a simple UI and SDK which allows apps to
| be developed for it easily without jumping through the hoops and
| without losing all the control to the device manufacturers and
| the OS company
| waboremo wrote:
| Hasn't there been several attempts at this? Vaguely recall
| firefoxOS and sailfishOS.
|
| I believe the problem always came down to hardware, due to it
| being essentially dominated by Android (ignoring iOS due to how
| tightly wound hardware and software are) you always ran into
| trouble requiring Android in some way before being able to
| switch to that new OS, even if the drivers themselves were
| freely and openly available.
|
| Which is a much harder problem to solve since there is so much
| upfront cost involved with hardware that most wouldn't dare
| bothering not prioritizing Android.
| tpoacher wrote:
| FirefoxOS is dead, but Sailfish is very much alive and
| kicking, with a reasonably thriving community; some of which
| inevitably overlapping with projects like pinephone etc.
| CrampusDestrus wrote:
| Why is it always something Unix-like?
|
| It's a hobby project! Be bold and creative and make something
| nobody has ever seen!
| pjmlp wrote:
| Sadly I can only upvote once.
|
| We need new BeOs, Inferno, Xerox like workstations,...
|
| Naturally the answer to why is clear, reuse existing UNIX
| software.
|
| However Apple and Google OSes also show that one can innovate,
| even at the bottom there is an UNIX like kernel.
| rollcat wrote:
| > Naturally the answer to why is clear, reuse existing UNIX
| software.
|
| POSIX can be added as a layer on top of (or besides) the
| native interface. macOS/Darwin does just that, with Mach as
| the microkernel.
|
| It creates opportunities for applications to use a non-
| standard, but more powerful or simpler interface (e.g. some
| sort of SpawnProcess, than can be more lightweight or less
| quirky than fork+exec). Unfortunately that's a tough call to
| make for applications that want to remain portable, and the
| world is already too fed up just dealing with
| Windows+Mac+Linux (+Android+iOS+Web+...).
| gumballindie wrote:
| My favourite alt os was something called "cube os". I cant find
| it anymore. Had it on a pirated disk in the early 2000s.
|
| Edit: found it, turns out its an msdos, freedos and maybe?
| linux gui called "qube":
|
| https://archive.org/details/DOS-GUI-DOS-QUBE-2001#ia-carouse...
|
| https://www.osnews.com/story/75/enter-the-qube-a-new-graphic...
| codyd51 wrote:
| Unix-like is definitely a misnomer, and I've gone ahead and
| removed it from the home page. It might've been true many years
| ago, but it's no longer the case.
|
| axle is fundamentally organised around structured messages.
| This mechanism is used both for interaction between userspace
| applications, and for nearly all interaction between kernel and
| userspace.
|
| For example, creating a process, allocating virtual memory
| regions, profiling the system and registering process hooks,
| are all just special messages that are sent to the kernel,
| rather than to another userspace app. Want to transmit a
| packet? If you have the rights, send a message to the NIC. Want
| to create a window on the desktop, or update its contents? Send
| a message to the window manager. It'll send some messages back
| to you when the user interacts with your content. The GUI
| toolkits use custom idioms. It's certainly true that axle
| reuses some common ideas (like featuring a desktop environment,
| and a process model), but I'm happy to go my own way from
| there!
|
| It's also true, as pjmlp mentioned, that some level of POSIX
| compatibility is necessary if you want to run external
| software, which I certainly do! DOOM, for example, requires
| read() and write(), among other common syscalls. If you want to
| run, say, GCC, you need fork().
| ianai wrote:
| Sounds very similar to objective-c.
| pavlov wrote:
| QNX is the canonical example of this kind of OS kernel.
| Everything happens by calling MsgSend().
|
| https://en.wikipedia.org/wiki/QNX
|
| It's quite successful in the embedded space.
| ksaj wrote:
| QNX is where I initially learned my Unix-like ways.
| Because of that exposure, I was never enthusiastic about
| what I saw as extreme limitations of Microsoft DOS and
| then Windows when I discovered them.
|
| One thing that really set it apart at the time, is that
| you could have a fully functional desktop environment
| boot off a single diskette. So much power in so little
| space. And it had features I still never see in current
| OS's. (For example, in QPaint you could save graphical
| images as a C header file. Weird perhaps, but awesome if
| you were making a graphical application.)
|
| It's all arguable nowadays I suppose, but I've grown so
| much with Unix-like OS's that it's a very hard sell.
| ianai wrote:
| It's awesome for me to find out QNX is message based. It
| makes sense though - I knew it was a microkernel. It'd be
| a lot easier to have a solid, real time OS if the kernel
| is minimal and just accepts/serves requests. Sounds like
| an interesting engineering project.
|
| As for "why posix". Any OS gains decades and decades of
| work by simply adhering to posix. It might not be perfect
| with only that but doing only that does have huge
| benefits. It might be boring to some, but it's not like
| there's a Windows standard a new OS can easily target and
| at no financial cost.
| CrampusDestrus wrote:
| Thank you for listening and making this correction. This is
| your personal project so there ks really no need to "sell"
| this OS to anybody, but maybe a clearer elevator pitch in the
| home page about what makes this project different (a bit like
| you wrote in your reply) might dispel a few doubts
| unixhero wrote:
| I love the picture of a woman, to show off the window
| manager/gui. Very 1990s :)
| thih9 wrote:
| Context about the picture: https://en.wikipedia.org/wiki/Lenna
| throwme_123 wrote:
| https://en.wikipedia.org/wiki/Lenna
|
| Edit: Maybe someone should edit the Wikipedia intro that overly
| insists on very modern issues on gender. Lena herself happily
| participated to several research conferences on image
| processing: https://www.icip2015.org/banquet.html
| fnimick wrote:
| Times change and people realize how inappropriate it is. She
| herself has called for an end to the use of the image as a
| relic of a male dominated field.
|
| https://www.sfgate.com/news/article/How-a-Nude-Playboy-
| Photo...
| dfee wrote:
| Inappropriate selling yourself as a centerfold for Playboy
| or using the image?
| pavlov wrote:
| The latter, obviously.
| matthrtly wrote:
| Awesome to see it so far along!
|
| I've been interested in OS dev since my teenage years. I remember
| reading about BeOS in 2005 and wanting to run it. I didn't even
| have the internet so that wasn't going to happen.
|
| I am extremely intimidated but I've been working through some of
| the tutorials on the osdev wiki.
|
| How much did you know about OS theory before you started on this?
| codyd51 wrote:
| Thank you! I didn't know anything about working with bare metal
| when I started out, and relied on the OSDev Wiki and other
| online resources. The work of those early years isn't as mature
| as now, but I learned a lot. Later on, I graduated to being
| comfortable reading (for example) the CPU manuals, and realised
| that I could now really get value out of them! Overall, I feel
| a lot more comfortable in the domain and like I can tackle a
| new device driver, or set off making the hardware do something
| obtuse and wrangle it out, but I only got there through trying
| to do it with less expertise under my belt.
|
| There's definitely a bootstrapping problem with the lower-level
| knowledge, but there are enough resources out there for the
| initial steps that anyone can get started, and you can do a lot
| if you enjoy the journey!
| vrglvrglvrgl wrote:
| [dead]
| ksaj wrote:
| And it runs Doom. Very important feature.
|
| I like how you've given the desktop a unique look. Doesn't look
| like it's trying to clone anything in particular.
| grepfru_it wrote:
| To be fair, a lot of hobbyist operating systems have a fairly
| unique 1990s lock
| birdyrooster wrote:
| [flagged]
| ugh123 wrote:
| Amazing work!
___________________________________________________________________
(page generated 2023-05-20 23:02 UTC)