[HN Gopher] Technical overview of Kandria, a game and game engin...
___________________________________________________________________
Technical overview of Kandria, a game and game engine developed in
Common Lisp
Author : trocado
Score : 326 points
Date : 2022-07-10 09:08 UTC (2 days ago)
(HTM) web link (reader.tymoon.eu)
(TXT) w3m dump (reader.tymoon.eu)
| gaze wrote:
| I'd really like to understand how live reloading works a bit
| better, especially when editing variables that some running
| thread depends upon. Is there an overall strategy here? Do I
| implement locking around variables that I intend to play with at
| runtime?
| User23 wrote:
| I don't believe the Common Lisp specification says anything
| about threads. For example here[1] is the relevant section of
| the SBCL manual. As you can see it's quite low level. If you're
| using those primitives then yes you'll have to manage barriers,
| synchronization, use atomic operations, or otherwise maintain
| sanity. The REPL (or SWANK) thread is just another thread.
|
| Edit: I should add that the usual pattern is Bordeaux threads
| or some other higher level thing.
|
| [1] http://www.sbcl.org/manual/index.html#Threading
| whartung wrote:
| All of the contention issues that we've had for ages across all
| platforms exist in CL. Loading and compiling new code isn't any
| different than two separate threads calling setq on the same
| element at the same time.
|
| The reality, though, is that the vast majority of the time when
| this is done, the system is idle. If you live update your
| server that's serving 1000 connections and isn't specifically
| coded to handle this potential event, then, for sure, there can
| be trouble.
|
| But if you're hot reloading your dev server, with an idle
| connection, then you're likely to have fewer problems.
|
| However, there's the potential impact of captured state that is
| not properly refreshed. For example, if you have some long
| running state that, say, captured a function via #'my-func,
| then that routine has, indeed, captured the pointer to the
| routine. But when you redefined it, the symbol (i.e. my-func)
| is now pointing to the new code, while your existing state is
| pointing to the old code. Any code that uses the symbol to call
| the routine will update automatically, but anything that does
| not, is stuck with stale data.
|
| And that rock potentially sends different ripples through the
| still pond of your running image. Just something to be aware
| of.
|
| It's a great feature, just need to be aware of what's going on
| when you do it. Most of the time, it's a non-issue.
| [deleted]
| aidenn0 wrote:
| With the threading model of most CL implementations, you really
| only need locking if you need to change _two_ (or more) places
| and have the update appear atomic, or if you are doing a read
| /modify/write. But you would already need such infrastructure
| in place if you were going to modify them in you program as
| well.
| newaccount2021 wrote:
| whartung wrote:
| This is a nice write up.
|
| The dark side is that they've put years into developing the core
| infrastructure to empower their Lisp game journey. Ideally that's
| all usable by others to "stand on the shoulders", so to speak,
| but that's always a challenge.
|
| I honestly have never had to work on a "running image" of any
| consequence. I've never worked on anything more than a few
| thousand lines of code. I've never needed any kind of source
| level "step" debugging. The ability to redefine a function
| quickly, I just throw in some prints to figure something out.
|
| Reloading the entire source file was always fast enough to now
| warrant hunting down any of the more interactive mechanisms
| available. This is all across using simple vi, or emacs, or even
| LispWorks. (defun l () (load "src.lisp"))
|
| Just save the file, and type that.
|
| All that said, my current low level fantasy is something akin to
| Electron, only with Common Lisp. First class DOM, first class
| event hooks, into a CL runtime that you can deploy cross
| platform. Dunno if that exists or not.
|
| I'm sure at some point my codebase will grow to the point where I
| would need to "level up" and become more intimate with SLIME,
| packages, system definitions, etc.
|
| I've just not crossed that threshold yet.
| jaccarmac wrote:
| > my current low level fantasy is something akin to Electron,
| only with Common Lisp. First class DOM, first class event
| hooks, into a CL runtime that you can deploy cross platform.
|
| Have you checked out https://github.com/rabbibotton/clog ? I
| have only played with a couple of the tutorials, but it seems
| to be targetting something like that, albeit without an
| included browser.
| dannyobrien wrote:
| The other direction is https://nyxt.atlas.engineer/ , which
| is a CL-driven browser. It's presented as a general-purpose
| browser, but you can also treat it as a standalone CL with a
| browser front-end, and I think that's the developer's long-
| term goal -- a bit like how emacs is a lisp with a text
| editor front-end.
| jomjomv wrote:
| > Reloading the entire source file was always fast enough
|
| The point is that you might want to keep the state.
| nine_k wrote:
| What kind of software do you usually create this way?
| whartung wrote:
| Essentially listener utilities (vs command line utilities).
| Code designed to solve specific problems, typically for their
| side effects and artifacts vs deployed code in production
| systems. One off simulations, etc.
|
| No UI but the listener.
| Shinmera wrote:
| Heya, I'm the author of the article. I've been working on Kandria
| full time for close to two years now, and on-and-off for some
| years while I was doing my Master's at ETH. The engine it's
| running on, Trial, is a few years older still, and initially just
| started out of the good ole university boredom and curiosity.
| Games is how I got into programming when I was about 6, and that
| particular fever hasn't left me yet it seems, ha ha!
|
| Kandria itself started as a "more serious" attempt at a game
| after I'd been making jam games with Trial for some years. I
| wanted to create something bigger, and had just finished playing
| through Celeste, so I just set out to replicate its mechanics in
| my own way. You can still find some demos of those very early
| days on my YouTube channel, if you're curious. In any case, the
| project went through several big changes until it turned into
| what it is now. During the pandemic in 2020 I finally decided to
| drop out of my Master's to pursue the project full time. At that
| point I also brought on a couple more people to help with the
| writing, art, and sound.
|
| We're currently also running a Kickstarter to get us some extra
| funds for the project. It ends on Thursday 14th, 12:00 CEST, so
| there's not that much time left in it. If you'd like to support
| our work, please have a look:
| https://www.kickstarter.com/projects/shinmera/kandria The
| campaign page has a lot more details on the project and the team,
| if you're curious about that.
|
| In the article I primarily cover the programming parts, giving a
| surface overview of the technologies involved and at least giving
| a brief hint at the methodology I employ to create games. I'm
| sure I've left out a lot of details that you might be curious
| about, so if you have any questions, please feel free to ask and
| I'll try my best to answer!
| doty wrote:
| From the post:
|
| > The same doesn't apply to the player sprite, as Aseprite
| takes minutes to compile the atlas for the nearly 1000 frames
| we have.
|
| Why does compiling a sprite atlas for the player take
| 60ms/frame?
| Shinmera wrote:
| I'm sure Aseprite could be further optimised to press down
| the constant factor, but it is performing a rect packing
| algorithm among other things, which is an NP-hard problem.
| rednab wrote:
| The bin packing problem is NP-hard, but rectangle packing
| is "merely" NP.
|
| However, most people just use a recursive biggest-fit-first
| 1), a simple heuristic that is surprisingly hard to beat
| for most workloads.
|
| I couldn't figure out what Aseprite is doing from their
| website or documentation, but if it's not that then it
| might be worth it writing your own sprite packer.
|
| 1) https://codeincomplete.com/articles/bin-packing/
| Shinmera wrote:
| Yeah I don't know what they use either, but so far
| repacking that atlas has not been frequent enough that
| writing my own packer would save time.
| tmountain wrote:
| This project looks awesome! I really like art, and it's
| exceptionally cool that you're using Common Lisp. As someone
| that often gets lost in the engineering (aka "fun") aspects of
| building things, my one piece of advice would be to make sure
| you put enough energy into marketing your game to ensure it
| reaches as wide an audience as possible and none of your
| efforts go to waste. Looking on reddit, I see a lot of posts in
| r/lisp, which is cool, but I'd work towards building as much
| buzz as you can in the indie gaming community (little teaser
| videos, etc). I'd also consider posting on the Tigsource
| forums, as that's a good way to build roots in the indie
| community. Anyway, looking great, and best of luck with your
| launch!
| Shinmera wrote:
| Thanks! Yeah, marketing is definitely a weak point, in large
| part because it's just something that goes very much against
| my character. I've tried posting to reddit before, but most
| communities' strict posting rules about promotions make that
| a pain, or simply not worth bothering with due to low
| conversion rates.
| dboon wrote:
| How have you funded yourself (prior to Kickstarter)?
|
| I'm in a similar position to you (indie developer, working on
| my first project of size that I am polishing with intent to
| release). I haven't quite been able to square up a way to work
| on my game full time -- I suspect out of a lack of ingenuity or
| resourcefulness on my part. You seem to have solved this
| problem, though.
| Shinmera wrote:
| By being lucky enough to have savings, and having a room at
| my parents' house that doesn't require any rent payments.
| Without those, it definitely wouldn't have been possible.
|
| Recently I've also received some grants, namely the Pro
| Helvetia interactive media grant, and the Poland Digital
| Dragons Accelerator. Both of those only offset a part of the
| money I had already personally invested though, and the
| Kickstarter barely makes a dent in anything.
|
| Selling games is such a gamble, too. If the stats are to be
| believed, only about 15% of games on Steam recoup their
| investment, let alone making profit. For the next project's
| funding, I'm going to try and look at some other source like
| Patreon, to at least offset my own measly costs so I can
| spend longer in pre-production.
|
| Making games is just really expensive, and any funding source
| is heavily fought over, or comes with huge caveats attached.
| I don't think I have this figured out yet either, so all I
| can say is best of luck!
| CraigJPerry wrote:
| >> One thing I do have to mention though is that the workflow in
| Lisp allows me to create these support libraries much faster than
| I can in other languages.
|
| I'm at that stage in my lisp journey (~18 months) where i know
| enough to be dangerous to others around me, i.e. i'm still a lisp
| n00b but have completed a few lisp & scheme books and written
| enough projects that i feel fairly productive. BUT i haven't
| reached, never mind come out the other side of, the "trough of
| disillusionment" yet. So everything lisp is unreasonably tainted
| with a positive glow in my mind.
|
| TL;DR i still think lisp is pretty awesome. It's my newest hammer
| and everything looks like a lisp-shaped nail these days. So i
| have to be deliberate and careful not to over-use lisp but it's
| totally changed how i write js for example. It's infected my
| brain.
|
| Here's the thing - that workflow statement by the OP is a fair
| chunk of the attraction to lisp for me. Before lisp i "knew" what
| a REPL was - i've been using python since i switched wholesale
| from Perl in the 00's, i thought you couldn't teach me anything
| new about REPLs, even quasi-REPLs like Jupyter were entirely
| familiar and "basic" to me. If you're a lisper, you know i was
| wrong about that, if you're not a lisper you're probably
| wondering what's the big deal.
|
| The other chunk - and i'm only really scratching the surface so
| far - has been macros. I think so far i prefer the non-hygenic
| type, but i haven't gone through Let over Lambda yet (although
| the book is on my bookshelf and is only 2 titles away from being
| started)
|
| For me, lisp opened my eyes to a way of working that i just wish
| i could fully replicate in other languages. Lisp has made me re-
| appreciate python and js in different ways now.
|
| If you want a really short version - Brett Victor's famous
| inventing on principle talk sums it up for me (he was using JS
| there, so there is hope...)
| nequo wrote:
| > it's totally changed how i write js for example
|
| This may be a silly question. But could you give a couple
| examples of how you implement something in JS now vs. how you
| would have implemented it 18 months ago?
| zasdffaa wrote:
| Learning a little haskell a million years ago was a
| revelation. In my next bit of JS (a large, non-web project) I
| did lots of FP in it - and for the better, not just because i
| could. I really cut down on boilerplate, made it more
| readable and reusable, and... it was just great.
| chii wrote:
| i would wager that after having used LISP, you start thinking
| bottoms up, rather than top down.
|
| i.e, instead of thinking about a program as a large one, and
| break it down into smaller, and smaller components or pieces,
| you'd start with the most bottom-est, most small piece first.
| Even if that piece isn't useful on its own - or it's only of
| limited use. Then you'd build another small, may be even
| orthogonal piece.
|
| Once you have these small pieces of individual functions, you
| might end up building a glue (sort of a language or custom
| macro?) to bind them together.
| newaccount2021 wrote:
| math-dev wrote:
| Great write-up. Indeed, the journey with Lisp is an interesting
| one and what you have witnessed seems to parallel many others
| experiences.
|
| What I found on my journey with Lisp:
|
| - I learnt Emacs well and learnt the beauty of its codebase and
| the thoughtfulness put into the program. Its' levels of
| documentation and customisation and being a lisp machine in its
| own rights was an enlightening experience for me when designing
| my own software (i.e. document well, document in a standardised
| way, document thoughtfully (think of what the users would want
| to know) and providing customisability to end users (which to
| be honest, dynamic langauges like Lisp excel at because they
| are much better suited for runtime manipulation and
| optimisation
|
| - The power of macros to change one's language. Now admittedly,
| this took some time to get a handle on. You kinda need to be
| proficient in Lisp before you start doing some super funky
| stuff and mould your language and program together into one
| (Ala PG's On Lisp). But having the ability to change the
| language at _compile_ time is simply, utterly, most imaginably
| amazing!
|
| - Writing clearer code: Because Lisp has very little
| punctuation syntax (not to be confused with the syntax of
| certain special forms), you cannot hide behind boilerplate code
| as much and you end up having to face your own thoughts more
| directly in the code you write. It is tough going at the start
| (who would have thought a _simple_ language would actually be
| hard to program in well - basically the more complexity in a
| language, the more restriction to your expressiveness, the more
| simple a language, the more power to you to fully express
| yourself by combining orthogonal blocks...speaking generally
| here as sometimes complexity does help in other ways (e.g. see
| CLOS))
|
| - Learning from the greats in Computer Science - there is so
| many good ideas and thoughts embedded into the lisps, that you
| inevitably learn a lot of good things by immersing yourself in
| the language and its history (of course over time)
|
| And finally, 'Life Beyond Lisp'. I no longer program in Common
| Lisp as my main language because I do need some libraries that
| I cannot do without. But as you alluded in your post on writing
| better JS after Lisp, I too have significantly improved my
| programming and code clarity in these other languages, after
| learning Lisp. Of course, one day I hope to program exclusively
| in CLOS, but until that day, Swift remains my near friend (at
| least it has generic functions).
|
| I can wholeheartedly recommend Common Lisp to anyone who has
| some spare time and curiosity to improve their skills in our
| profession of computer science. Also (and this is something
| Lispers don't seem to do that well), try other languages! Learn
| from them all in my humble opinion. Computer Science is a
| wonderful thing :)
|
| p.s. Congrats to Kandria on their progress. Truly a wonderful
| team and their main author is a wonderful open-source lisper.
| slekker wrote:
| Could you expand on the REPL bit? Having worked with JS, Python
| and Elixir, they feel more or less the same. What's Lisps have
| that these don't?
| bjoli wrote:
| So, I would say the idea of dynamic redefinition is probably
| the big thing: In cl I can redefine a class and have all
| instances updated while the program is running.
|
| I would also say that the error management is also beyond
| fantastic. Say you run a function, and deep inside the call
| stack there is an error. In CL you can actually update the
| faulty function and just continue executing at the point of
| the error.
|
| For someone like me without any intellectual rigour at all,
| this lends itself to a kind explorative programming I really
| like.
| phoe-krk wrote:
| The Lisp instinct is to make a change in a file and
| immediately hit C-c C-c in Emacs, at which point the change
| goes live in the system - no matter if it's a variable or a
| function that you have changed or something else.
|
| You don't need to reload the full module like in BEAM
| languages like Erlang or Elixir, you can modify a single
| function or a form without the need to make a version upgrade
| to the processes running your code - the next time any
| thread, including e.g. an already running game loop, ends up
| calling the new function, it's the new version of the
| function that will end up being called.
| ramchip wrote:
| Version upgrades are not needed on BEAM anymore than they
| are in Lisp, you can just reload a module and code will
| start using the new version.
| phoe-krk wrote:
| TIL, that's a big thing! When was that change made?
| ramchip wrote:
| I don't think it's recent, as far as I know. Versioning,
| appup / relup, etc. are really for upgrading production
| deployments live, when you need special instructions to
| pause processes or update their internal state before
| loading new code. In development, you can ignore all that
| and just reload modules directly. e.g. with "r" in the
| Elixir REPL.
|
| Once that's done any external call (`MyModule.add(1,2)`)
| will go to the new version of the code, while local calls
| (`add(1,2)`) stay on the same version.
|
| The Phoenix web framework will also auto-reload any
| module that has changed on disk when handling a web
| request in dev mode.
| ravi-delia wrote:
| I personally prefer SLIME over any IEX client I've ever seen,
| but that's not a limitation of Elixir. Since Erlang has a
| Smalltalk-esq image just like CL does, the Elixir REPL is in
| principle as powerful as a Lisp's. In practice CL has lots of
| additional little things which are nice, like the ability to
| update instances in place after changing a class or DEFVAR
| forms which don't clobber their contents on reloads, but for
| me most of the advantage comes from just how well designed
| SLIME is.
| alfiedotwtf wrote:
| Thanks for that. Are you able to tell us what books got you to
| where you are now? I've just started Emacs Lisp An Introduction
| as I'm wanting to dive right into Emacs (coming from a life-
| long Vimer), but I'd also want to expand to other Lisps.
___________________________________________________________________
(page generated 2022-07-12 23:02 UTC)