[HN Gopher] Porting SBCL to the Nintendo Switch
___________________________________________________________________
Porting SBCL to the Nintendo Switch
Author : todsacerdoti
Score : 278 points
Date : 2024-09-13 12:53 UTC (10 hours ago)
(HTM) web link (reader.tymoon.eu)
(TXT) w3m dump (reader.tymoon.eu)
| colingw wrote:
| I have being using Trial[1] for the past few weeks to test out
| game development in Common Lisp, and have been having a great
| time. Being able to alter (almost) all aspects of your game while
| it's running is a blessing.
|
| I hope this port succeeds.
|
| [1]: https://github.com/Shirakumo/trial
| sinker wrote:
| Lisp languages seem well-suited for building games. The ability
| to evaluate code interactively without recompilation is a huge
| deal for feature building, incremental development, and bug-
| fixing. Retaining application state between code changes seems
| like it would be incredibly useful. Common Lisp also appears to
| be a much faster language than I would have blindly assumed.
|
| The main downside for me (in general, not just for game
| programming) is the clunkiness in using data structures - maps
| especially. But the tradeoff seems worth it.
| pjmlp wrote:
| There are 1980's papers about Lisp compilers competing with
| Fortran compilers, unfortunately with the AI Winter, and the
| high costs of such systems, people lost sight of it.
| coliveira wrote:
| Well, I imagine at the time they had some LISP
| implementations that were very well tuned for specific high
| end machines, which essentially duplicated Fortran
| functionality. This is difficult to do for general purpose
| Lisps like SBCL. It was also probably very expensive.
| lispm wrote:
| One of the downsides is that implementations like SBCL have a
| deep integration and need things like a well performing GC
| implementation - to get this running on specialized game
| hardware is challenging. The article describes that. Getting
| over the hurdle of the low-level integration is difficult.
| The reward comes, when one gets to the point, where the rapid
| incremental development cycles of Common Lisp, even with
| connected devices, kicks in.
|
| For the old historic Naughty Dog use case, it was a
| development system written in Common Lisp on an SGI and a C++
| runtime with low-level Scheme code on the Playstation.
|
| > Common Lisp also appears to be a much faster language than
| I would have blindly assumed.
|
| There are two modes:
|
| 1) fast optimized code which allows for some low-level stuff
| to stay with Common Lisp
|
| 2) unoptimized, but natively compiled code, which enables
| safe (-> the runtime does not crash) interactive and
| incremental development -> this mode is where much of the
| software can run nowadays and which is still "fast enough"
| for many use cases
| koito17 wrote:
| > The ability to evaluate code interactively without
| recompilation
|
| SBCL and other implementations compile code to machine code
| _then_ execute it. That is to say, when a form is submitted
| to the REPL, the form is not interpreted, but first compiled
| then executed. The reason execution finishes quickly is
| because compilation finishes quickly.
|
| There are some implementations, like CCL, with a special
| interpreter mode exclusively for REPL-usage.[1] However, at
| least SBCL and ECL will compile code, not interpret.
|
| [1] https://github.com/Clozure/ccl/blob/v1.13/level-1/l1-read
| loo...
| Shinmera wrote:
| I specifically talk about the fast evaluator for SBCL. But
| even without that contrib, SBCL does have another evaluator
| as well that's used in very specific circumstances.
| taeric wrote:
| I think a lot of this is confusion between online versus
| batch compilation? Most of us have only ever seen/used
| batch compilation. To that end, many people assume that JIT
| in an interpreter is how online compilation is done.
|
| I probably am more guilty of that than I should be.
| akira2501 wrote:
| > the form is not interpreted, but first compiled then
| executed
|
| That's TempleOS technology right there.
| Filligree wrote:
| Other way around.
| darby_nine wrote:
| Do either CCL or SBCL have any kind of partial evaluation
| or tracing compilation?
| Jach wrote:
| If that's your main downside, that's pretty good, since
| clunkiness is in many ways fixable. Personally with standard
| CL I like to use property lists with keywords, so a "map
| literal" is just (list :a 3 :b 'other). It's fine when the
| map is small. The getter is just getf, setting is the usual
| setf around the getter. There's a cute way to loop by #'cddr
| for a key-and-value loop, though Alexandria (a very common
| utility library) has some useful utils for
| looping/removing/converting plists as well.
|
| If typing out "(list ...)" is annoying, it's a few lines of
| code to let you type {:a 3 :b 4} instead, like Clojure. And
| the result of that can be a plist, or a hash table, or again
| like Clojure one of the handful of immutable map structures
| available. You can also easily make the native hash tables
| print themselves out with the curly bracket syntax.
|
| (On the speed front, you might be amused by
| https://renato.athaydes.com/posts/how-to-write-slow-rust-
| cod... But separately, when you want to speed up Lisp (with
| SBCL) even more than default, it's rather fun to be able to
| run disassemble on your function and see what it's doing at
| the assembly level, and turn up optimization hints and have
| the compiler start telling you (even on the individual
| function level) about where it has to use e.g. generic
| addition instead of a faster assembly instruction because it
| can't prove type info and you'll have to tell it/fix your
| code. It can tell you about dead code it removed. You can
| define stack-allocation if needed. Simple benchmarking that
| also includes processor cycles and memory allocated is
| available immediately with the built-in time macro...)
| spsesk117 wrote:
| Thanks to the author for the fascinating and detailed write up.
| It feels like a lot of the time this level of detail around the
| specifics of 'blessed' (not homebrew) console porting are only
| revealed years after the end of the consoles lifetime.
|
| As an aside, reading about this kind of deeply interesting work
| always makes me envious when I think about the rote software I
| spend all day writing :)
| cschep wrote:
| well said!
|
| as I was just sitting down to another day of ruby on rails
| (that I am grateful for!) I was thinking.. I wonder what
| hobby/open source projects could use some of my attention
| later..
|
| .. what projects my attention could use later .. :D
| whizzter wrote:
| At least back when I was working these "blessed" tools were
| usually a tad hacked together, modern homebrew toolchains for
| many older platforms are better except for debugging support
| (since the devkits for the machines usually had better hooks
| available but also avoiding the entire GDB focus).
|
| Having been in both worlds, i'm not entirely sure there's that
| much to be envious of.
| fouric wrote:
| This is super neat - SBCL is an awesome language implementation,
| and I've always wanted to do CL development for a "real" game
| console.
|
| I'm also surprised (in a good way) that Shinmera is working on
| this - I've seen him a few times before on #lispgames and in the
| Lisp Discord, and I didn't know that he was into this kind of
| low-level development. I've looked at the guts of SBCL briefly
| and was frightened away, so kudos to him.
|
| I wonder if SBCL (+ threading/SDL2) works on the Raspberry Pi
| now...
| f1shy wrote:
| - Is it not her?
| anisoco wrote:
| It is
|
| https://reader.tymoon.eu/article/436
| f1shy wrote:
| Oh I did not know that she transitioned. I just remebered
| it was the author of portacle, that was a woman.
|
| What a world of pain must be to have to go through it.
|
| Kudos to her. Also what she does for lisp is amazing.
| Shinmera wrote:
| I'm not doing the SBCL parts, that's all Charles' work that I
| hired him for. My work is the portability bits that Trial
| relies on to do whatever and the general build architecture for
| this, along with the initial runtime stubbing.
|
| And, as mentioned, *her :)
| cellularmitosis wrote:
| Holy cow, Kandria looks amazing. Is it also developed using
| Trial? https://www.youtube.com/watch?v=usc0Znm-gbA
| Shinmera wrote:
| Yes, it's also open source:
| https://github.com/shirakumo/kandria
|
| My current unannounced project is a lot more ambitious
| still, being a 3D hack & slash action game. I post updates
| about that on the Patreon if you're interested.
| ducktective wrote:
| I wish the likes of Nintendo and Sony themselves finance such
| efforts. I mean it's one another way to create games (IP) for
| your console, what possibly could be the downside of starting
| something similar to Github Accelerator for your platform?
| jsheard wrote:
| Because it's well established that game developers can and will
| jump through whatever hoops the platform holder demands at
| their own expense, they don't have the leverage to be picky
| about the technical details when deciding which platforms to
| ship on. Nintendo doesn't need to create new incentives to
| release on the Switch when they already have the biggest
| incentive of all: 140+ million units sold, and a high attach
| rate.
|
| At least there isn't as much hoop jumping as there used to be,
| since the systems have all converged on using commodity CPU and
| GPU architectures with at most minor embellishments.
| Arelius wrote:
| Yeah, also whatever they would build, the other platform
| vendors won't choose the same thing, and it wont be the exact
| variant of lisp or w/e that even the few nice developers
| would want.
|
| I wish vendors would be just more supportive of different
| llvm tool chains. Rust isn't even well supported.
| Retr0id wrote:
| > The answer to that is that while I would desperately like to
| share it all publicly, the NDA prevents us from doing so.
|
| I'm curious what the rationale here was for using the official
| SDK, rather than the unencumbered "homebrew" ones[0].
|
| As a complete guess, maybe Nintendo doesn't let you officially
| publish games built using 3rd party SDKs?
|
| [0]
| https://switchbrew.org/wiki/Setting_up_Development_Environme...
| Shinmera wrote:
| You cannot publish games with homebrew, it has to use the
| official SDK. Besides that, almost nobody has a jailbroken
| Switch, so it would make it extremely hard to play any games on
| anything but an emulator.
| Retr0id wrote:
| > You cannot publish games with homebrew, it has to use the
| official SDK
|
| This doesn't surprise me much, but does Nintendo state this
| explicitly anywhere public?
|
| If Nintendo chose to sign an application developed using a
| 3rd party toolchain, there's no _technical_ reason why it
| couldn 't run on retail consoles.
| Arelius wrote:
| Yeah, maybe. But also the official SDKs are pretty good and
| you get support from Nintendo. It seems like a pretty big
| risk to use an unsupported SDK... for what benefit?
| Retr0id wrote:
| I don't have access to Nintendo's SDK so I can't compare
| directly, but the article cites an inability to map
| executable pages. libnx supports this (but of course,
| this is moot if Nintendo wouldn't let you ship it). But
| the main benefit is being able to talk about and share
| your work without worrying about violating an NDA.
|
| https://switchbrew.github.io/libnx/jit_8h.html
|
| https://switchbrew.org/wiki/JIT_services
| Shinmera wrote:
| The OS can do it, and some Nintendo titles on the Switch
| do use this capability, but I have talked to Nintendo
| directly about using it, and it's a hard No. I can't even
| use the JIT feature purely for dev.
| nightpool wrote:
| Really? What is the justification for allowing Nintendo
| titles to use it but not third parties? Security
| concerns?
| Shinmera wrote:
| That's what they claim, but ultimately it's their thing,
| so they do with it whatever they like.
| favorited wrote:
| > almost nobody has a jailbroken Switch
|
| This isn't really my scene so I don't know the details, but I
| remember reading that the first 10+ million Switches produced
| have an unpatchable bootloader exploit. I'm sure you're
| correct that almost nobody actually has a hacked console, but
| my understanding is that they're readily available for people
| who want one.
| ArneBab wrote:
| You don't let your kids jailbreak their Switch. Because it's
| a damn online system, so any leaked info and Nintendo can
| brick the Switch. And their game states are far too valuable
| for the kids for that.
| noobermin wrote:
| This is what I come to HN for. Kudos to OP and their colleague. I
| know it's impossible but what a blessing it would be if Nintendo
| could be a little more open about their system.
| whitten wrote:
| I probably missed it, but since the switch doesn't have a
| keyboard how do you handle text input ?
| jsheard wrote:
| The Switch can support a USB keyboard, which would be the nice
| way to do it. There's already a fair number of officially
| published games with keyboard and/or mouse support, including a
| couple of programming ones. It has an on-screen keyboard too of
| course but you wouldn't want to rely on that more than
| absolutely necessary.
| Arelius wrote:
| Over the network most likely?
| Shinmera wrote:
| It does have a HID keyboard API, but typically you're expected
| to present an on-screen keyboard.
| not_your_vase wrote:
| Somewhat offtopic, just flashed through my mind: you know what
| would be amazing and absolutely useless at the same time?
|
| Porting Yuzu to Nintendo Switch
| laidoffamazon wrote:
| This has been done, and it's been possible to run Switch on
| Switch for several months [0] (about 39 minutes into the
| video).
|
| [0] https://youtu.be/H1gveQUBIKk
| not_your_vase wrote:
| Shoot, it was a suspiciously genius idea* for a Friday 13th.
|
| *: Compared to my usual ideas
| jfim wrote:
| Since I had to look it up, Yuzu is a Nintendo switch emulator.
| sleepycatgirl wrote:
| Her work is just amazing. And makes me incredibly happy, as I
| like to write some CL here and there.
| Shinmera wrote:
| Aw, thank you. Happy hacking!
| trenchgun wrote:
| Does anybody know what is the status of Trial on Mac OS?
| Specifically Apple Silicon
| nanna wrote:
| I've just bought Kandria. I'm not much of a game player so I
| probably won't get much play out of it, but Shinmera is clearly
| pushing the bounds of the Lisp world, and that's something to
| support.
| hoten wrote:
| b/c it isn't described anywhere...
|
| SBCL - "Steel Bank Common Lisp"
|
| > Steel Bank Common Lisp (SBCL) is a high performance Common Lisp
| compiler. It is open source / free software, with a permissive
| license. In addition to the compiler and runtime system for ANSI
| Common Lisp, it provides an interactive environment including a
| debugger, a statistical profiler, a code coverage tool, and many
| other extensions.
|
| https://www.sbcl.org/
| varjag wrote:
| "Steel Bank" in turn is a homage to "Carnegie Mellon" of CMUCL.
| ArneBab wrote:
| According to the Benchmarks Game, SBCL is roughly as fast as
| Node.
|
| https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
| drmeister wrote:
| Awe-inspiring work Shinmera and Charles.
___________________________________________________________________
(page generated 2024-09-13 23:00 UTC)