[HN Gopher] A completely-from-scratch hobby operating system
___________________________________________________________________
A completely-from-scratch hobby operating system
Author : tirrex
Score : 222 points
Date : 2021-10-31 14:06 UTC (8 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| junon wrote:
| This is by Klange, the owner of the #osdev channel on Libera
| chat, associated with osdev.org. Super nice individual and always
| posts updates and interesting tidbits about ToaruOS.
|
| Strange, but welcome, to see it on the frontpage! :)
| klange wrote:
| I wouldn't say I "own" that channel in any respect, I'm just
| responsible for dragging it over from Freenode during the
| exodus... we've got the same same staff as we did before the
| move and I'm not even the "founder" according to ChanServ.
| behnamoh wrote:
| I like that the main website looks like the OS itself!
|
| https://toaruos.org/
| shiftoutbox wrote:
| https://web.archive.org/web/20210423232715/https://toaruos.o...
| _tom_ wrote:
| Looks like the website has been hugged to death, for now.
|
| I wonder if the web server is hosted on toarus?
| bogomipz wrote:
| Under features, the first bullet point is "Dynamically linked
| userspace." Can someone say what this means? How is this
| different than a userspace made up of dynamically linked
| utilities and shell?
| afr0ck wrote:
| Libraries are shared by programs and refrences to library
| routines are automatically resolved at program load time.
| kenferry wrote:
| It's not different, but it's a feature that this is included vs
| only supporting static linking.
| bluecatswim wrote:
| No inbuilt documentation system called Index?
| picture wrote:
| Oh that would be awesome
| fouc wrote:
| Nice. This came before SerenityOS and yet seems similar in
| approach.
| amitport wrote:
| How this compares with xv6 for first OS course purpose? Is it
| viable to use this to teach the same things? (Paging, inter
| process communication, threading...)
| Semaphor wrote:
| Back in 2014 this was posted before, so before the first
| "release" version and when there were external dependencies:
| https://news.ycombinator.com/item?id=8566217 (26 comments).
| kevincox wrote:
| At that time the repo probably looked like this:
|
| https://github.com/klange/toaruos/tree/9f34619078f92bd7d9815...
| spicybright wrote:
| I really like the UI! Most hobby OS's just do the same bland
| patterns most of the time.
| picture wrote:
| I love that the OS seems to be named after Toaru Majutsu no Index
| and Toaru Kagaku no Railgun. Misaka the kernel and Kuroko the
| interpreter are named after iconic characters from the series.
| ulzeraj wrote:
| Agreed. At first I thought my weeb brain was just making
| associations and then I read the component names I realized its
| all based on Raildex.
| rurban wrote:
| Then why in the heck is he going for POSIX compatibility, when he
| can afford the luxury of not having to deal with blocking
| syscalls and all this crap? Much easier and safer multithreading.
| Also faster.
|
| And why we are there, why not a safer microkernel, keeping
| everything in userspace? Questions over questions.
| craigmart wrote:
| >why in the heck is he going for POSIX compatibility
|
| Because existing desktop applications can be ported to ToaruOS
|
| >why not a safer microkernel, keeping everything in userspace?
|
| This is a design choice, microkernels aren't necessarily better
| than hybrid, they're slower, harder to debug and process
| management can be complicated
| eggy wrote:
| Just curious how hard it would be to forego POSIX entirely if
| you were building an OS. I know TempleOS is entirely from
| scratch. I'd like to implement a small LISP like SectorLISP
| [1] (see yesterday's posts too on HN). I don't know much
| about building my own OS, so I'd like to start with something
| like MenuetOS (my first PL was asm), SerenityOS, TempleOS, or
| this one. I'd like it to be completely an 'island', i.e.
| POSIX not a requirement. I want to use it to hack on in
| isolation without any easy copy/paste shortcuts. I know
| Mezzano exists, and it has booted on bare metal, but I would
| like to start with the OS's above, implement my own LISP, and
| go from there.
|
| Any other OS recommendations base on my ignorant, but
| wishful, reqs above? I realize there are some others in Rust
| too. Thanks!
|
| [1] https://github.com/jart/sectorlisp
| adrian_b wrote:
| Someone who would make a new OS, should define a completely
| new system call interface, as it is likely that now it is
| possible to conceive a better interface than 50 years ago
| and anyway if it would not be different there would be no
| reason to make a new OS, instead of modifying an existing
| one.
|
| Nevertheless, the first thing after defining a new OS
| interface must be writing a POSIX API translation layer, to
| be able to use without modifications the huge number of
| already existing programs.
|
| Writing a new OS is enough work, nobody would have time to
| also write file systems, compilers, a shell, a text editor,
| an Internet browser and so on.
|
| After having a usable environment, one can write whatever
| new program is desired, which would use the new native OS
| interface, but it would not be possible to replace
| everything at the same time.
|
| Besides having a POSIX translation layer, which can be
| written using as a starting point one of the standard C
| libraries, where the system calls must be replaced with the
| translation layer, some method must be found for reusing
| device drivers made for other operating systems, e.g.
| either for Linux or for one of the *BSD systems.
|
| Nobody would have time to also write all the needed device
| drivers. So there must exist some translation layer also
| for device drivers, maybe by running them in a virtual
| machine.
|
| The same as for user applications, if there is special
| interest in a certain device driver, it should be rewritten
| for the new OS, but rewriting all the device drivers that
| could be needed would take years, so it is important to
| implement a way to reuse the existing device drivers.
| md8z wrote:
| At that point, why not just contribute to Linux?
| AnIdiotOnTheNet wrote:
| > Nevertheless, the first thing after defining a new OS
| interface must be writing a POSIX API translation layer,
| to be able to use without modifications the huge number
| of already existing programs.
|
| I disagree. POSIX sucks. Build a hypervisor so people can
| run their applications in a VM and insist that native
| programs use the non-garbage API. It's the only way
| you'll ever unshackle yourself.
| PaulDavisThe1st wrote:
| > Someone who would make a new OS, should define a
| completely new system call interface, as it is likely
| that now it is possible to conceive a better interface
| than 50 years ago and anyway if it would not be different
| there would be no reason to make a new OS, instead of
| modifying an existing one.
|
| For an example of how things like this can be done
| incrementally, you can look at io_uring on linux.
| spicybright wrote:
| > Writing a new OS is enough work, nobody would have time
| to also write file systems, compilers, a shell, a text
| editor, an Internet browser and so on.
|
| > So there must exist some translation layer also for
| device drivers, maybe by running them in a virtual
| machine.
|
| > ... but rewriting all the device drivers that could be
| needed would take years, so it is important to implement
| a way to reuse the existing device drivers.
|
| I'd think most people making a hobby OS specifically want
| to do these things.
|
| I also think most don't care about wide hardware
| compatibility.
| adrian_b wrote:
| Even if you do not want the new OS to run on anything
| else but your own laptop, that still needs a huge amount
| of drivers, for PCIe, USB, Ethernet, WiFi, Bluetooth,
| TCP/IP, NVME, keyboard / mouse / trackpad, sound, GPU,
| sensors, power management, ACPI and so on.
|
| The volume of work for rewriting all these is many times
| larger than writing from scratch all the core of a new
| OS.
|
| Rewriting them requires studying a huge amount of
| documentation and making experiments for the cases that
| are not clear. Most of this work is unlikely to present
| much interest for someone who wants to create an original
| OS, so avoiding most of it is the more likely way leading
| to a usable OS.
| a-dub wrote:
| redox is one i've been following from afar. rust, not
| posix, microkernel, s/everything is a file/everything is a
| url/
|
| it looks pretty cool, although the url thing seems yet to
| prove its utility. they seem to be playing around a bit
| with using the protocol component (net, disk, etc), but
| it's unclear what this adds over just using paths. although
| maybe if they used the protocol to describe the encoding of
| the data, it would add something?
| fouc wrote:
| Isn't QNX a microkernel? I remember it being known for being
| quite fast?
| guenthert wrote:
| As a real-time OS it is known for deterministic response
| times. If it were exceptionally fast (and licenses cheap
| enough), you'd see hosts in the TOP500 using it.
| p_l wrote:
| TOP500 is chock full of microkernels though, even if the
| "I/O nodes" would run Linux
| eggy wrote:
| I agree 100%. QNX is lurking in products you may use. It
| was the OS for the show control system that is used
| throughout entertainment where real-time is necessary for
| the safety of the devices it controls, which also have
| hardware safety at the lower level. I would drop into the
| QNX terminal for certain tasks. Unfortunately, you used
| to be able to download the show control software and play
| with it, but it has since been bought buy a company that
| sells it with the equipment they rent, so you need to buy
| trainging and it is behind their wall now. Not QNX, but
| the show control software that runs on QNX.
| bregma wrote:
| No, it's more of a nanokernel. It's very fast.
|
| Full disclosure: I maintain QNX toolchain.
| md8z wrote:
| I'm very confused by this comment. There are a ton of other
| things you need to implement if you want to have desktop
| applications. POSIX does not specify any APIs for graphical
| applications. You might be thinking of something else.
|
| If you want to support the lion's share of desktop
| applications, it would actually be better to implement the
| Win32 API...
| craigmart wrote:
| Sorry, I meant software in general but wrote "desktop
| applications" instead. Anyway the sentence is still valid,
| even if you'll have to implement other things such as the
| graphical interface, the POSIX compliant code won't need
| modification
| md8z wrote:
| If you're taking an app built for Linux or GNU or BSD,
| then it probably will need modification, as those systems
| have various extensions on top of POSIX.
| yjftsjthsd-h wrote:
| > microkernels aren't necessarily better than hybrid, they're
| slower, harder to debug and process management can be
| complicated
|
| I was basically on board, but how are they harder to debug?
| I'd think being able to run components in userspace would
| make debugging way easier.
| mjgerm wrote:
| You are now debugging a distributed system.
| yjftsjthsd-h wrote:
| Oh, good point; I was thinking at the component level
| the-dude wrote:
| Shallow dismissal of other people's work.
| teddyh wrote:
| Shallow dismissal of other people's comments.
| the-dude wrote:
| Fine, I will provide the link again :
| https://news.ycombinator.com/newsguidelines.html
| turbinerneiter wrote:
| You can do all of this in your own hobby project.
| ape4 wrote:
| You need POSIX to compile nearly any program
| rurban wrote:
| That his advantage. He doesn't want to, rather write
| everything anew from scratch.
|
| Also signals. With a microkernel you won't need signals
| mdp2021 wrote:
| So, a preliminary problem for many OS-from-scratch projects
| intending to create "New Different Great OS" will be to have
| a language implementation which does not expect posix.
|
| It depends on the goal. If you want to compile pre-existing
| software for it, or if you want to really "start anew".
| spijdar wrote:
| True, and as another commenter hinted at, you're free to do
| your own hobby project however you please, there's not a
| wrong way to do so if you find it fulfilling.
|
| But at the same time, it does make me sad that most hobby
| OSes end up seeking POSIX compatibility, because that means
| being destined to essentially either be another unix-variant
| or develop a unix-variant inside some subsystem of your OS.
|
| Yes, being unix-like means you gain access to a trove of
| software and libraries that makes porting applications much
| easier, but it also limits the potential be truly different
| and experiment, as your end result will look like "yet
| another unix" with misc. improvements.
|
| Since I think the enjoyment of building something like this
| comes from the satisfaction of building an OS from the ground
| up, I don't think it matters, but it would be cool to see
| more hobby OSes try more exotic ideas and runtimes.
| spicybright wrote:
| Fully agree. We need more Temple OS's!
| ranger_danger wrote:
| For *nix maybe, but there are many other non-POSIX operating
| systems in use today.
| klange wrote:
| When I started the project a decade ago, my aim was
| specifically to understand POSIX/Unix and to "learn by doing",
| so of course there's some POSIX-like elements underpinning the
| design. Back before I decided that literally anything can be in
| scope and was using third-party components, this aided in
| porting software, as other commenters have pointed out. These
| days, it gives more purpose to things I build for the OS if
| they can also reasonably be built for Linux or macOS - things
| like my editor (which I wrote for the OS and now use as my
| daily driver in Linux), or my Python knock-off.
|
| As for the microkernel bit, this might sound like circular
| reasoning but I didn't go for a microkernel because no one
| really uses microkernels. It's not that I think microkernels
| are a bad idea, ToaruOS does push plenty of stuff into
| userspace. Rather, my main goal at the moment is to provide an
| educational resource that more accurately models the way "real"
| OSes work than the typical academic OS projects.
| VWWHFSfQ wrote:
| And why in the heck didnt he write it in Rust!
|
| ...
| fouc wrote:
| At least he didn't write it in node.js /s
|
| Other interesting alternatives to c or rust might be vlang or
| zig.
| jstx1 wrote:
| I'm very impressed any time I see people pouring so much work
| over long periods of time into hobby projects.
| [deleted]
___________________________________________________________________
(page generated 2021-10-31 23:00 UTC)