[HN Gopher] In Praise of QEMU
___________________________________________________________________
In Praise of QEMU
Author : Tomte
Score : 204 points
Date : 2022-09-02 10:08 UTC (12 hours ago)
(HTM) web link (drewdevault.com)
(TXT) w3m dump (drewdevault.com)
| drewg123 wrote:
| QEMU is incredibly helpful when debugging issues very early in OS
| startup (like before console prints work). This sort of debugging
| was invaluable when working on FreeBSD NUMA stuff and doing
| things like backing the vm page array with node-local physical
| memory.
|
| My colleague, who is working on kboot for FreeBSD (to run the
| FreeBSD bootloader as a linux binary that kexec's into FreeBSD
| from a linuxboot environment) also found QEMU an invaluable tool.
|
| In both cases, I think QEMU probably cut development time by 50%
| or more.
| dinom wrote:
| It's truly awesome and has nearly become synonymous with
| emulation on Linux.
|
| Of course without things like KVM, and assorted accelerators,
| etc. it would lose a fair bit of luster. We all stand on the
| shoulders of giants to some degree.
|
| Nonetheless, I don't know of another free emulator that even
| comes close... are there any to speak of?
| dijit wrote:
| QEMU is still amazing even without KVM; KVM is just a
| performance improvement for native platforms.
|
| But with QEMU you can emulate extremely exotic CPUs on standard
| x86's; that's the stuff that Console Emulators struggled with
| immensely!
| ch_123 wrote:
| Not to trivialize QEMU (I use it often and I love it) but
| usually the challenge with console emulators are A) emulating
| supporting hardware such as graphics chips which are often
| undocumented and B) achieving cycle accurate emulation for
| platforms which need that. QEMU emulates relatively standard
| hardware for each platform. Even for x86, sometimes I find
| 86box or PCem more desirable for certain older
| hardware/software vs QEMU.
| anthk wrote:
| Well, it depends, the i440FX chipset with an SB16 will run
| most if not all DOS games.
| Koshkin wrote:
| ... and, since you need the OS, there's always FreeDOS.
| jiripospisil wrote:
| QEMU is fantastic, I use it pretty much every day through UTM on
| macOS (and it doesn't need any 3rd party kernel extensions
| because it can take advantage of Apple's Hypervisor framework). I
| used to just use shell scripts to launch VMs but configuring it
| via the UI is just much easier, maybe I'm getting old.
|
| https://mac.getutm.app
| otachack wrote:
| I always preferred a UI when messing with VMs. It's why I've
| used VirtualBox for the longest time. Part of my hesitancy on
| using QEMU has been how command line heavy it is where it has
| just enough friction that I don't want to devote too much time
| to it. I'll check out your suggestion!
| solarkraft wrote:
| QEMU is great, I'm just annoyed by its UI. I'd much rather have a
| config file I can point it towards than an endlessly long shell
| script I can't comfortably use comments in (maybe this is a
| criticism of shell scripts in general).
|
| I guess libvirt would be the "docker compose" front-end, but I
| remember considering that more complex than would be necessary.
|
| I know Proxmox has some QEMU management tools (which I find
| decent), but they don't seem to be very popular outside of it.
|
| One cool QEMU thing I recently discovered is the microvm machine
| type (https://qemu.readthedocs.io/en/latest/system/i386/microvm.h
| t...), which enables it to spawn VMs just as quickly as
| Firecracker!
|
| It would be nicer yet if there was some way to say "give me a
| shell in this ephemeral VM" like docker does it, but sadly Kata
| Containers appear to have dropped docker support and I haven't
| been able to get it to run with the alternative runtimes.
| nerdponx wrote:
| Zsh (and to a lesser extent Bash) has support for _arrays_ ,
| which you can use to represent long command line invocations in
| relative safety. E.g. instead of this sort of madness:
| #!/bin/sh set -eu # Argh I can't
| write comments in between lines... just email me if you have
| q's: nerdponx@mail.example.net exec /usr/sbin/boop \
| --enable-foobars \ --this-is-a-really-long-option \
| --sound=quack:.:loader=org.example.soundfactory-
| lib.SoundFactory.wrapped_loader \
| --sound=woof:.:loader=net.quibbler.QuibblerWoofMaker \
| --jvm-opts='-Xms512M -Xmx1G -Xquux=1234 -Xbootclasspath:.'
|
| But you can preserve what's left of your brain cells without
| switching to a "non-shell" language if you use Zsh, and then
| you don't have to sweat over shell quoting and parameter
| expansion either: #!/usr/bin/env zsh
| # Enable "strict" features for typo-safety emulate zsh
| setopt err_exit pipe_fail warn_create_global warn_nested_var
| no_unset boop_args=( # We always want
| foobars, why isn't this on by default? --enable-
| foobars # See:
| https://stackoverflow.com/questions/4973 --this-is-a-
| really-long-option # Use the good sounds! Please
| see README for obtaining these deps.
| --sound=quack:.:loader=org.example.soundfactory-
| lib.SoundFactory.wrapped_loader
| --sound=woof:.:loader=net.quibbler.QuibblerWoofMaker
| # Trust me, don't touch these. --jvm-opts='-Xms512M
| -Xmx1G -Xquux=1234 -Xbootclasspath:.' )
| exec /usr/sbin/boop ${(@)boop_args}
| Groxx wrote:
| aaah, that's a great idea for interleaving comments, I'll
| have to use that! thanks!
|
| previously I've done this, which works, but is a bit.......
| yeah. some command \ `# did you
| know comments work inside backticks like this?` \
| --complex_flag_needing_descriptions=... \ `#
| literally any ffmpeg flag` \ --lol no
|
| now if only shellcheck would support zsh....
| remram wrote:
| For completeness, the other option is:
| CMD="some command" # Some comment CMD="$CMD
| --verbose --complex-flag=..." # Other stuff
| CMD="$CMD --lol=\"no thanks\"" eval $CMD
| Groxx wrote:
| yeah, I just don't like dealing with the possibly-
| multiple-levels of escaping that sometimes requires :\
|
| there are ways to deal with that of course, but it's
| kinda unavoidably more complex.
| anthk wrote:
| Virt-Manager (and virsh) has option for both config files and
| scripts creation.
| troutwine wrote:
| > I'd much rather have a config file I can point it towards
| than an endlessly long shell script I can't comfortably use
| comments in (maybe this is a criticism of shell scripts in
| general).
|
| I end up designing a lot of CLI tools and this is a design
| problem I run into a lot with my own stuff: how many flags are
| too many? Introducing a config file works, but now it makes
| your tool much less feasible to run in a shell script but, like
| you say, too many flags makes your tool unwieldy to run anyway.
| The problem is harder if you have a venerable tool that's
| always been flag heavy. Do you keep lumping flags in or do you
| steer the community toward the use of a config file? Or a
| hybrid?
| rcoveson wrote:
| OpenVPN's approach is to have every config key imply a valid
| config file entry and a long-arg. So if you see the line
| `tun-mtu 1500` in a config file, you know you could use the
| arg `--tun-mtu 1500` on the command line, and vice versa; if
| you see somebody use the CLI arg `--proto udp` you know you
| could put the line in your config file `proto udp`.
|
| I think mysqld has a similar approach, though I don't think
| it's as complete. I'll bet both of those projects would
| answer that there's no such thing as a tipping point where
| you tell users to use a config file. They're just two ways of
| doing the same thing; use whichever makes sense for your
| deployment.
| nyanpasu64 wrote:
| The mpv media player takes a similar approach.
| troutwine wrote:
| That's an interesting approach.
| anthk wrote:
| Virt-Manager it's a must over Qemu, too. I mean, scripts are
| nice, but qemu has zillions of options.
|
| Also: virsh domxml-to-native qemu-argv --domain
| "your_wm_name"
|
| To get the Qemu launching script if you plain to deploy a
| prototype over thousands of machines.
| vbezhenar wrote:
| What's wrong with scripts anyway? I always call qemu from shell
| script and don't understand why would I need more layers for
| that.
| bitwize wrote:
| For the same reason you don't develop front-end JavaScript
| without a framework, or write a modern video game without
| using an engine. Unless you're willing to sign up for added
| friction and pain, you just don't.
| vbezhenar wrote:
| If all I need is to write 100 lines of code, framework will
| add friction while vanilla JS is perfectly fine.
| anthk wrote:
| Well, sometimes it helps to visually see your device tree
| setup and then when everything it's perfectly configured, you
| connect the virsh shell to your KVM domain and you begin to
| spit out all the Qemu scripts so you can edit them for tweaks
| not seen under Virt-Manager.
| emmelaich wrote:
| https://github.com/lima-vm/lima is also excellent
| nortonham wrote:
| QEMU is fantastic. I think for spinning up a VM just to try out
| something, virtualbox is still a good option because of its gui
| and its simplicity. QEMU is extremely powerful however, and I'm
| glad it exists.
| dinom wrote:
| It's a bit of an apples vs. oranges comparison though. AFAIK
| Virtualbox is a hypervisor for x86-64 only. Qemu, otoh, can
| emulate a number of different hw platforms.
| mongol wrote:
| I never understood the interaction between QEMU and KVM. I use
| the libvirt tools and it works great, but the internals confuse
| me.
| pydry wrote:
| I dont know why but many of the problems I have with
| podman/docker running on M1 seem to stem from QEMU.
|
| E.g. problems suspending/unsuspending the mac, problems
| starting/restarting machines, etc.
|
| Digging out logs to figure out what went wrong seems to be a pain
| too.
| haolez wrote:
| One of the best programmers I know always say that QEMU is the
| greatest piece of C code that he has ever come across. It seems
| that there is some very cool preprocessor magic as well, but I've
| never dug any deeper than hearing him out :)
| vegetablepotpie wrote:
| As someone who works with QEMU, I must say QEMU is a shining
| example of how to do object oriented programming in C.
|
| That said, trying to follow the QEMU code using static analysis
| is impossible. Every function call is a function pointer stored
| as a parameter in a struct that was set up by a macro called in
| initialization. The only way I could figure out how things are
| tied together is with GDB.
|
| I feel like using C++ would have been a better language choice.
| beagle3 wrote:
| The modern QEMU is magic, but I find the _original_ QEMU even
| more magical: It did not contain any assembly, and was easily
| portable to new architectures while still providing
| (relatively) decent performance -- the new TCG backend
| delivers much beter performance, at the cost of requiring
| more porting work for new host architectures.
|
| The original QEMU had an implementation of the required
| machine ops in C, but instead of calling those ops, it
| memcpy()d their implementation, thus enabling binary
| translation without interpreter overhead -- basically using
| the underlying "gcc" as a code-generator at _runtime_ without
| including it. Pure elegance and /or madness.
|
| And it worked very well.
| avg_dev wrote:
| I've either never heard of or never bothered to pay attention to
| SourceHut. https://sourcehut.org/
|
| How did GitHub ever get so big when this exists? Is it because GH
| predates SourceHut? I'm not clear on when SH came into existence,
| but they have entries on their blog from 2019.
|
| > * Absolutely no tracking or advertising
|
| > * All features work without JavaScript
|
| > * Many features work without an account
|
| > * The fastest & lightest software forge [links to
| https://forgeperf.org/ - which is very impressive]
|
| > * 100% free and open source software
|
| Edit: here's the actual "forge" site: https://sr.ht/
| Tomte wrote:
| I'm a paying customer, so I like SourceHut a lot, but it's easy
| to see why it won't be nearly as big as GitHub.
|
| * GitHub was much earlier
|
| * GitHub was backed by venture capital, and is now backed by
| Microsoft
|
| * GitHub doesn't force you to adopt an email-only workflow.
| Pull request are much maligned in tech circles (following Linus
| Torvald's rant), but people all over dev teams _love_ them.
| They are a huge argument why companies don 't just stand up a
| git repo and call it done, devs want all those workflows and
| issue trackers and stuff. SourceHut is very barebones even
| where it has them. (The exception would be build actions, where
| SourceHut has no real integrations into big tools, but the core
| build action system is superior to GitHub's, IMO).
|
| Incidentally, the build system depends on qemu:
| https://drewdevault.com/2018/09/10/Getting-started-with-qemu...
| avg_dev wrote:
| I am definitely a big PR user, but I believe that PRs could
| be implemented as a stand-alone app.
|
| That link was very interesting, I read it all.
|
| The footnote made me laugh, which is good, because apparently
| I make poor life choices, and am an embarrassment to us all.
| ;)
| krupan wrote:
| Thanks, Drew, for calling out the author of QEMU and linking to
| his sight: Fabrice Bellard https://bellard.org/ I didn't know he
| had done so much! Incredible!
| avg_dev wrote:
| https://en.wikipedia.org/wiki/Fabrice_Bellard
|
| Wow.
| gtirloni wrote:
| Calling out on what specifically?
| knob wrote:
| "shout out" was what he meant.
| gtirloni wrote:
| Ah, thanks. I thought I was missing something.
| versteegen wrote:
| He mostly seems to list his hobby projects on his website, not
| his paid work (except https://textsynth.com and I don't know
| what else). E.g. he is CTO of Amarisoft. His actual
| productivity is greater still.
| anthk wrote:
| From Bellard the things I like most are the "toy" projects.
| Qemu, tinyemu and such began like that.
| Uehreka wrote:
| His list of hobby projects includes QEMU and ffmpeg. I'm sure
| Amarisoft is a fine company, but mentioning them before those
| two would be burying the lede.
| pookeh wrote:
| He also wrote QuickJs a complete Javascript VM.
| ZeroNilNanNada wrote:
| I agree.
|
| Few years ago there was an uprise of blog posts explaining how to
| run Linux as a primary system and virtualize Windows machine with
| GPU passthrough for gaming. ~ 1 year ago I took a dive and did a
| lot of reading and attempts to make it work, the final results
| made me really happy. QEMU/KVM/VFIO is one of the best software
| combinations I've ever used. Witch CPU isolation
| and scheduling properly setup my Linux part of machine runs
| Docker with 25 services, while Windows part happily plays games
| without stutter at the same time. That said, quite a
| few gaming companies are anti-virtualization (with reasonings
| being that mostly those who use it are using it for cheating
| purposes) and ban your account, without rights to appeal (ex.
| Rainbow Six). I honestly find this stance blunt and it feels like
| discrimination. But currently such use of virtualization is
| really niche, so what one can do?
| sm_ts wrote:
| I've been using VFIO and maintaining a guide for a few years*,
| then I've realized that VFIO _kinda_ works, but it 's not a
| reliable technology, and I moved (back) to dual boot.
|
| There are very significant pain points, specifically:
|
| 1. if one reserves the video card for VFIO, it won't have any
| power management; this means that it will run hot while doing
| nothing; in order to work this around:
|
| 1a. first has to battle with X, which has an option not-to-
| take-over-a-card-but-it-takes-it-over-nonetheless
|
| 1b. then one can give exclusive access to the graphic card
| driver, which can be switched out/in when starting/stopping the
| VM; this unfortunately works, but not reliably
|
| 2. the points above apply to nvidia; AMD is worse, as it hasn't
| supported soft GPU reset until very recently (I think it was
| added on 5.19 or so)
|
| 2a. this means that one starts the VM, then stops it, and most
| of the times the card will hang
|
| 2b. there resize BAR functionality is not supported by VFIO (at
| least, last year it wasn't), which means, one loses additional
| performance (I could be ok with it, as the loss is not
| significant, but performance losses compound)
|
| The problem is that all the points above are not in control of
| the user; the problems happen at driver level (if, say, there
| is no reset support, one can't add it out of thin air).
|
| If one uses Nvidia, and they're ok with the card running hot
| all the time, then definitely, VFIO works wonder. But this lead
| me to abandon VFIO, as I don't want that (and the alternative
| of the card having a most-of-the-time-malfunctiong driver was
| not appealing, either).
|
| Big shame! I loved VFIO :)
|
| * https://github.com/64kramsystem/vga-passthrough
| nullwarp wrote:
| I built my current desktop with the whole goal of being a tank
| of a VFIO setup and I have no regrets. Best of both worlds for
| me as I used to just have two separate computers.
|
| QEMU/VFIO with Looking Glass is just such an amazing setup for
| my use cases.
| knob wrote:
| Drew is one of my internet personalities where I just follow and
| read everything I can.
|
| I still have to install sway though!
| NelsonMinar wrote:
| I recently discovered the magic that is qemu-user-static on an
| Ubuntu system. Install that and you can instantly run binaries
| compiled for other architectures. No need for a big VM or
| anything (*), just run the binary standalone. Combined with a
| cross-compiler toolchain and you have a nice way to do
| development for other platforms.
|
| (*) the trick being the environment. I'm writing assembly code
| that just invokes Linux system calls. If you need shared
| libraries or other OS services, it gets more complicated.
| jbirer wrote:
| The QEMU project and it's developers are responsible for making
| my research and hobby OS development on RISC-V and Aarch64
| platforms possible, where I learned so much at a pace I otherwise
| couldn't. Thanks.
| 0x0203 wrote:
| Agreed. I'm currently using qemu and gdb from an x64 Linux
| system to debug an aarch64 interrupt problem I'm having. Not
| the sort of thing you can easily debug on real hardware. I've
| done similar things with VMware, but as far as I know, that
| can't do cross architectures like this. It's pretty amazing.
| zen_1 wrote:
| Yeah embedded debugging is an absolute PITA, even with
| OpenOCD + JTAG exposing a remote gdb server to the target
| it's just always been so fragile in my experience. Like
| reloading a changed executable causing gdb to segfault
| sometimes, and you need to restart every part of the debug
| chain (device being debugged, openocd, gdb) in order to get
| back to working on the problem you were originally trying to
| solve.
| 0x0203 wrote:
| I hadn't seen OpenOCD yet, so thanks for that. On the
| boards I'm currently playing with, while they technically
| have JTAG, it's pin muxed with pins used for the sd card.
| So I'd have to build a new u-boot to reconfigure the right
| pins, and get an adapter to break the right jtag pins out
| of the sd slot, and figure out booting from emmc or spi
| since I'm currently booting everything from the sd card...
| QEMU seemed much easier!
| zen_1 wrote:
| OpenOCD is a massive godsend for any kind of board
| bringup/embedded development.
|
| While I can't speak for your board specifically, if the
| pin assignment is software reconfigurable (and is reset
| to sd-card access on reboot!) and you're not using the SD
| card once your program is running, you could reconfigure
| the pins in software and then wait for a JTAG debugger to
| attach (by having an infinite loop you use openOCD to
| continue past). Of course if you can debug your issue
| using QEMU that's indeed much easier, thanks Fabrice!
| ta988 wrote:
| For macos people, that's what your docker runs on.
| ch_123 wrote:
| If you are emulating another architecture (e.g. running x86
| images on top of an M1 chip) then yes. For native images,
| Docker Desktop uses xhyve, which is a port of the FreeBSD bhyve
| hypervisor to macOS
| whb07 wrote:
| I ran OSX on QEMU/KVM on a Ryzen emulated to be an IceLake Intel
| cpu and got close to ~90%+ performance of my native machine
| according to geekbench.
|
| So my QEMU OSX instance got better performance than a Mac Pro
| max, very well near the top ~5 scores for macs in geekbench.
|
| Props
| metadat wrote:
| How did you get the Mac VM created? I tried this once but
| eventually gave up because whatever path I tried it was
| challenging at every point to create a macOS VM from scratch.
___________________________________________________________________
(page generated 2022-09-02 23:02 UTC)