[HN Gopher] Project Oberon: Design of an operating system, a com...
___________________________________________________________________
Project Oberon: Design of an operating system, a compiler, and a
computer [pdf]
Author : JoachimS
Score : 330 points
Date : 2024-01-04 16:59 UTC (1 days ago)
(HTM) web link (people.inf.ethz.ch)
(TXT) w3m dump (people.inf.ethz.ch)
| zengid wrote:
| RIP Prof Wirth
|
| Side note: can we get a better tag on this to show it's a PDF
| file? I think the source is reputable but I always freak out when
| a url goes to a pdf.
| forgotpwd16 wrote:
| Usually links leading to pdf are denoted with [pdf] at the end.
| zengid wrote:
| it wasn't formatted like this at the time
| userbinator wrote:
| The URL already ends in .pdf .
| gjvc wrote:
| What, specifically, worries you about pdf files?
| cmplxconjugate wrote:
| His plea for lean software [1] is always worth a read.
|
| [1] https://cr.yp.to/bib/1995/wirth.pdf
| i_don_t_know wrote:
| That's only the first half of the book. You can find all parts
| and software at: https://people.inf.ethz.ch/wirth/ProjectOberon/
| jll29 wrote:
| p. 59-75 covers the fast implementation of piece lists (recently
| covered on HN).
|
| (Just saying because there were many threads on how to represent
| text buffers for fast editing on HN:
|
| https://news.ycombinator.com/item?id=15381886
| https://news.ycombinator.com/item?id=11244103
| https://news.ycombinator.com/item?id=14129543
| https://news.ycombinator.com/item?id=15387672
| https://news.ycombinator.com/item?id=14046446 )
| zvmaz wrote:
| > The largest chapter of the 1992 edition of this book dealt with
| the compiler translating Oberon programs into code for the
| NS32032 processor. This processor is now neither available nor is
| its architecture recommendable. Instead of writing a new compiler
| for some other commercially available architecture, I decided to
| design my own in order to extend the desire for simplicity and
| regularity to the hardware. The ultimate benefit of this decision
| is not only that the software, but also the hardware of the
| Oberon System is described completely and rigorously. The
| processor is called RISC. The hardware modules are decribed
| exclusively in the language Verilog.
|
| I wonder how it compares to Nand To Tetris. Other than that, it
| seems really interesting. Has anyone read it?
| 082349872349872 wrote:
| It's much less pedagogical than Nand to Tetris; if you need the
| background, read that first, then read this.
|
| (he eventually ported the hardware description to his own HDL,
| Lola)
| kragen wrote:
| oh i had no idea, i have to see this. does lola support any
| existing fpgas?
| 082349872349872 wrote:
| see https://people.inf.ethz.ch/wirth/Lola/index.html
|
| IIUC, in its 1990s incarnation Wirth had been able to get
| bitstream formats, but this century everything was closed
| off, so now it transpiles to Verilog.
| i_don_t_know wrote:
| Project Oberon describes the final system as it is. It's not a
| tutorial that takes you step by step from the basics to a
| complete system.
| hnthrowaway0328 wrote:
| Nand2Tetris is the perfect introduction project of this field
| that spans Hardware, OS and compiler design. That said, I
| completed the project and found it lacking depths and in-depth
| investigation of all topics. But this is what makes it the
| perfect introduction project.
|
| The University of Tokyo has a similar project that I believe
| can serve as a stepping stone. Check out the CPU experiment:
| https://ytsmiling.tech/2017/04/02/cpuex.html
|
| If you don't like it (probably due to lack of documentation),
| at this stage you can also design your own projects. Basically
| a CPU sub-project that uses Verilog or any HDL, which leads to
| an OS and compiler project.
| retrac wrote:
| As another user said, it's not oriented towards a beginner. You
| would want some, maybe most, of the background in NAND 2 Tetris
| first. Chapter 16 about that mentioned processor for example,
| and it just throws you in the deep end with the Verilog code.
| Here's the CPU interface with the system bus, the registers,
| and how the multiplier unit works.
|
| The good part is the commentary about the design decisions and
| trade-offs. That is invaluable, because it is the wisdom of
| Niklaus Wirth, towards the end of his career, drawing on a
| lifetime of experience. He was one of those rare polymaths,
| with a broad and deep understanding, of both the circuitry, and
| the more abstract parts of CS. He always generalized, tried to
| understand the principle, and places things in their historical
| context and explains how they developed. Because it's Wirth,
| the history lesson is often based on personal experience. It
| produces a good synthesis, in my opinion. For example:
|
| > The second [interface] (MouseX) is included here for
| historical reasons. It was used by the computer Lilith in 1979,
| and used the same Mouse as its ancestor Alto (at PARC, 1975).
| It is distinguished by a very simple hardware without its own
| microprocessor, which is currently contained in most mice. This
| goes at a cost of a 9-wire cable. But today, microprocessors
| are cheaper than cables. We include this interface here,
| because it allows for a simple explanation of the principle of
| pointing devices.
| kragen wrote:
| the wirth-the-risc processor is immensely simpler to describe
| and program than the tecs/nand2tetris processor, which borders
| on unusable. i've gone through the process of 'designing' the
| nand2tetris processor on nandgame, and i'm pretty sure the
| nand2tetris processor is simpler to wire up from gates. but the
| wirth-the-risc processor is a lot easier to get running on an
| fpga or, i bet, to simulate with verilator, because it uses a
| real hdl instead of something that someone who's never designed
| hardware thinks an hdl might look like. probably the
| nand2tetris processor would require less code in verilog if you
| coded it up, quite similar to chuck thacker's 'a tiny computer
| for teaching'
| https://www.cl.cam.ac.uk/teaching/1112/ECAD+Arch/files/Thack...
| which is, like the nand2tetris processor, based on the dirty
| genitals nova architecture
|
| nand2tetris will get you from nand gates to tetris and to
| bytecode interpreters. but oberon will get you from
| synthesizable verilog (which can be easily converted into nand
| gates but almost never is) to a fully usable gui operating
| system that can recompile its own source code. sadly it cannot
| resynthesize its own fpga bitstream because you cannot run
| vivado on it (though i see 71bae0447c737f454371dcf3b84fc62c
| says below it can at least simulate its own hardware)
|
| a thing they both have in common is the lack of a usable name
| for the processor architecture
| pjmlp wrote:
| After this follow up on the Oberon System 3, with its Gadgets
| component model, and a mix of JIT/AOT compilers,
|
| "The Oberon companion - a guide to using and programming Oberon
| System 3"
|
| https://www.semanticscholar.org/paper/The-Oberon-companion-a...
|
| You can access the source code from Rochus fork,
|
| https://github.com/rochus-keller/OberonSystem3
|
| Or the Active Oberon evolution, although not directly done by
| Niklaus Wirth,
|
| https://gitlab.inf.ethz.ch/felixf/oberon
|
| Screenshots and OS Manual,
|
| https://gitlab.inf.ethz.ch/felixf/oberon/-/blob/main/ocp/Doc...
|
| Or Component Pascal and Blackbox IDE, created by a startup out of
| ETHZ
|
| https://blackboxframework.org/index.php
| Rochus wrote:
| The Active Oberon evolution started with the Active Object
| System (AOS) and the Bluebottle window manager; the source code
| of the last "Current" release can be accessed e.g. here:
| https://github.com/OberonSystem3/AOS_Bluebottle_Sources. If you
| want to study the source code, you can use this tool:
| https://github.com/Rochus-Keller/ActiveOberon.
| guidoism wrote:
| I re-typeset the text in TeX and have a nicer looking PDF for
| those that care: https://github.com/guidoism/tex-oberon
| zvmaz wrote:
| I really liked reading the README. It shows that you are
| passionate about Project Oberon and the book. This in turn
| motivates me to read it. Thank you for your work.
| sydbarrett74 wrote:
| Thank you for your hard work and dedication in keeping this
| alive.
| loup-vaillant wrote:
| Awesome! I'm guessing there's a not-too-hard way to change
| change the paper format? I'm thinking of going to A5 so I can
| bind myself a nice book.
| kragen wrote:
| thank you so much
|
| this must have been a huge amount of work
| berkeleynerd wrote:
| Is it possible to use this version of the book to follow along
| and build everything without having to source the original FPGA
| board Wirth was using?
| cxr wrote:
| Yes, but if you want to actually run the programs, you'll
| need an emulator.
| ikhare wrote:
| Back when I took compilers in college we wrote a compiler for
| Oberon. I couldn't quite find my original class site, but this
| one seems roughly right (from a few years before I took the
| class): https://cseweb.ucsd.edu/~wgg/CSE131B/
|
| It was a great teaching language.
| I_am_tiberius wrote:
| In an interview with Niklaus he was talking about a 500 page
| document...
| whartung wrote:
| The original 1992 book is longer. As mentioned earlier the code
| generation chapter is smaller because the RISC CPU is much
| simpler than the original.
|
| But, also, the original, I think, has much more source code. I
| don't know if the original book contains the entirety of the
| system, but the new one seems to contain more highlights of
| interfaces and selected examples. It likely relies on the
| ubiquity of source availability on internet, which clearly was
| not the case in 1992.
| fmeyer wrote:
| The idea of an education computer is bugging my mind in the last
| 5 years.
|
| If you consider modern hardware and OS is practically impossible
| to have a simple enough machine that you could teach the young
| generation. Fantasy consoles like pico-8 are good options for
| programming, but not for understanding the hardware underneath.
| That way you still have school who use old architectures for
| teaching.
| bpye wrote:
| A minimal RISC-V implementation is quite simple. There is a
| RISC-V implementation of xv6 - though that'll require slightly
| more than the absolute minimum RISC-V implementation,
| specifically you'll need CSRs, M, S and U mode and paging.
|
| If you don't care about paged memory you could do with just M
| and U mode. I have a small rtos that targets some of the WCH
| microcontrollers with that configuration. It does use the PMP
| but even that isn't really necessary.
| ysofunny wrote:
| why would anybody NOT care about paged memory?
|
| or maybe a better question is, why care about it? as far as
| I've understood, paged memory is a legacy from a time where
| cheap and fast memory wasn't a thing
|
| how feasible is it to get rid of memory pages? I guess the
| hardest thing would be untangling interprocess memory safety
| from pages??
| Veserv wrote:
| The answer is memory fragmentation. Process memory safety
| can theoretically be done with a different scheme such as
| hardware memory capabilities. You could probably even do
| demand paging if the memory capability is not a true
| pointer, but some sort of memory system coherent handle.
| But, as soon as a process wants to allocate a 1 GB chunk
| and you have 2 GB, but only in 64 KB chunks you have a
| problem. You could go around copying data to compact the
| physical memory, but now you have serious interference
| problems.
|
| You then run into the next problem of using say 32 MB in a
| "hot loop" in the middle of a 64 GB demand-loaded data
| structure on a 32 GB machine. You can not greedily load in
| the entire data structure from disk, so you need some sort
| of subset feature on your memory handles. But then what do
| you do about using two disjoint sections separated by over
| 32 GB? You need some way of having multiple subsets that
| correspond to physical addresses that do not respect the
| handle offset. Subset 1 corresponds to physical address
| range A and subset B corresponds to a uncorrelated physical
| address range B. Congratulations, you have reinvented
| memory mapping with extra steps.
| kragen wrote:
| adding an mmu to a basic rv32e processor might double its
| size and power consumption, and more than double the
| verification effort; moreover, if you're targeting
| applications where deterministic execution time or even
| worst-case execution time (wcet) is a concern, the mmu is a
| very likely source of nondeterminism
|
| so it's not so much that you don't care about it as that
| you might not be able to afford it. 'cheap and fast'
| depends on what scale of machine you're talking about; a
| 1C/ computer (not yet available) can afford less than a
| 10C/ computer like the pms150c or (rv32e) ch32v003, which
| can afford less than a 1-dollar computer like the
| stm32f104c8t6 or (rv32i) gd32vf104, which can afford less
| than a 10-dollar computer like a raspberry pi, which can
| afford less than a 100-dollar computer like a cellphone,
| which can afford less than a 1000-dollar computer like a
| gaming rig, which can afford less than a 10-kilobuck
| computer like a largish cpu server, which can afford less
| than a 100-kilobuck computer like a petabox
|
| unix originally ran on the pdp-11, the relevant models of
| which had interprocess memory safety in the form of
| segmentation, but no paging. i've never used a pdp-11.
| adding paging (for example, on the vax the sun-1, and the
| i386) enabled a variety of new unix features:
|
| - as you point out, it enables a process to be larger than
| physical memory;
|
| - fork() became immensely faster because it didn't have to
| copy all of process memory, just the page table, and mark
| the existing pages copy-on-write;
|
| - execve() became immensely faster for a similar reason: it
| could 'demand-page' the program into memory as you executed
| parts of it, instead of waiting to start executing it until
| the whole thing had been loaded from disk;
|
| - shared libraries became possible, so that executable code
| used by many programs at once could exist as only a single
| copy in memory (though you could do this without paging if
| all the processes share the same address space, perhaps
| with different permissions imposed by an mpu -- this wasn't
| considered an option for unix in part because it would
| involve either giving up fork() or only having one process
| in memory at a time);
|
| - similarly, it became possible for processes to
| communicate through shared memory buffers, which is
| commonly used to get images onto the screen quickly;
|
| - it became possible to memory-map files, like on multics,
| so you can access data in them without copying it, which
| normally takes about twice as long as accessing it;
|
| - it became possible for user programs to use the paging
| hardware to implement the write barriers for their garbage
| collectors by using mprotect(), though that's never been a
| very popular thing to do because sigsegv handlers are slow
| and usually nonportable;
|
| - and, as veserv pointed out, it eased fragmentation.
|
| non-unix systems used paging for a variety of even more
| creative purposes:
|
| - efficient system image checkpointing as in keykos or
| eumel, by way of atomically marking all pages on the system
| copy-on-write and then streaming out the dirty ones to
| disk, so you never had to reboot; after a power failure or
| system crash, all the same programs would be running in the
| same state as at the last checkpoint. qemu can do this too,
| i think
|
| - distributed single-address-space oses, where memory pages
| migrate around a cluster over a network according to where
| they're being accessed, so every program on the cluster is
| running in a single shared 64-bit address space; this
| didn't turn out to be as useful as it sounds
|
| - insert your mindblowing creative idea here
|
| anyway it's totally possible to implement memory protection
| without paging, and lots of computers have, past (with
| segmentation) and present (with mpus). but paging gives you
| a lot more than just memory protection
| eterps wrote:
| Project Oberon does not use paged memory. For the most part
| memory defragmentation is done by a relatively simple
| system-wide garbage collector (by copying live objects to
| one end of the heap).
| kragen wrote:
| this sort of exists at
| https://github.com/cksystemsteaching/selfie
| (http://selfie.cs.uni-salzburg.at/)
|
| > _Selfie is a self-contained 64-bit, 12KLOC C implementation
| of: (...) a tiny (...) subset of C called C Star (C*) (...)
| to a tiny (...) subset of RISC-V called RISC-U[;] a[n] (...)
| emulator (...) that executes RISC-U code[;] (...) a (...)
| hypervisor (...) that provides RISC-U virtual machines_
|
| so they have a self-hosted instruction set architecture,
| compiler, and operating system, though the operating system
| is much simpler than xv6. because the instruction set is a
| subset of risc-v you can run its code on actual risc-v
| hardware (or qemu-system-riscv), but presumably you could
| also design risc-u hardware in verilog that was simpler than
| a full implementation of rv64i with whatever extensions the
| hypervisor needs
| ysofunny wrote:
| I think that somehow a computer than can learn to talk to other
| computers would be the pinacle solution
|
| this is within reach now with LLMs, the remaining challenges
| would be somehow connecting the computers (a hardware
| compatibility issue) but the software should be able to figure
| the other software out somehow
| TillE wrote:
| Vintage 8-bit hardware is extremely comprehensible, and it
| teaches you fundamentals which absolutely still apply today.
| Ben Eater's YouTube videos are fantastic for this, both his
| 6502 project and his homebrew "from scratch" breadboard
| computer.
| flamedoge wrote:
| pretty good to save for post apocalyptic renaissance
| abdellah123 wrote:
| This is a jewel. Is there an attempt to repurpose oberon as a
| mobile "computer"?
| berkeleynerd wrote:
| Has anyone updated the book to allow one work with newer FPGA
| boards?
___________________________________________________________________
(page generated 2024-01-05 23:02 UTC)