[HN Gopher] Show HN: Bomberland - An AI competition to build the...
___________________________________________________________________
Show HN: Bomberland - An AI competition to build the best Bomberman
bot
Author : joooyzee
Score : 92 points
Date : 2021-09-24 11:01 UTC (12 hours ago)
(HTM) web link (www.gocoder.one)
(TXT) w3m dump (www.gocoder.one)
| b20000 wrote:
| another competition to make people work on hard problems for free
| or a small prize, or at best a job in some company.
| armanboyaci wrote:
| Some similar "AI" challenges: - https://www.pommerman.com/ -
| https://play.battlesnake.com/ - http://fruitbots.org/ -
| https://nethackchallenge.com/
| joooyzee wrote:
| Speaking of Pommerman, we caught up recently with the
| organizer. While unfortunately Pommerman is no longer running,
| he was super helpful in giving us some advice for Bomberland.
|
| It'd be awesome if there were previous participants of
| Pommerman here who could share some feedback on how we could
| improve Bomberland, since there are some obvious parallels.
| te wrote:
| Where can I view recorded or real-time matches by existing bots?
| Where's the leaderboard?
| airza wrote:
| Is the code for these gonna be shared after the competition?
| There is a real dearth of complex reinforcement learning code
| publically available for tensorflow :(
| joooyzee wrote:
| We usually encourage people to open-source their code after the
| competition so that the community improves over time (but only
| if they're open to it).
|
| Speaking of Tensorflow, we're working on some ML starter kits
| and would love some feedback on how to improve the workflow for
| people using TF, PyTorch etc! If you do end up trying it out
| and get stuck anywhere, please feel free to ping either myself
| or Matt (@thegalah) on our Discord
| (https://discord.gg/tRUMgdfC).
| joooyzee wrote:
| Hey HN,
|
| I'm Joy from Coder One. This is an early version of our upcoming
| Bomberman-inspired AI competition. Bomberland is an intentionally
| challenging environment for ML featuring non-trivial problems
| like real-time decision-making, large search space, and both
| adversarial + cooperative play.[1]
|
| Longer term, we're building a place where anyone can explore
| cutting-edge algorithms like deep RL, GAN, MCTS etc on meaningful
| real-world challenges. Think OpenAI Gym, but with active
| competitions and open-ended multiplayer simulations.[2]
|
| We'd love to hear what you think!
|
| [1] Bomberland write-up: https://www.gocoder.one/blog/bomberland-
| multi-agent-artifici...
|
| [2] About us: https://www.gocoder.one/blog/next-generation-open-
| ai-gym
| agravier wrote:
| What other environments / games do you have in the works?
| joooyzee wrote:
| No others (at the moment). We're a small team so Bomberland
| is our current focus - we want to improve the tooling first
| so that it's easy for people to dive into ML before we
| introduce other environments.
|
| We do have a mini-project called Ultimate Volleyball
| (https://github.com/CoderOneHQ/ultimate-volleyball) built on
| Unity ML-Agents. It's intended more as an introduction to
| deep reinforcement learning, and we wrote some tutorials for
| it here if anyone's interested:
| https://www.gocoder.one/blog/hands-on-introduction-to-
| deep-r...
|
| But if there are any games/environments you're interested in,
| please let us know!
| mysterydip wrote:
| If powerups are randomly placed, it's possible for a "better"
| player to lose to a "worse" player simply by luck of the draw.
| How is that accounted for in determining the best AI?
| joooyzee wrote:
| Yes this could happen. Ideally, we would run enough matches
| so that over time this would average out and the better AI
| should come out on top.
| fxtentacle wrote:
| As-is, I wouldn't consider participating, because doing so would
| require a lot of boring manual work from me. The reason why we
| have good AI models for some games is that they are easy and fast
| to evaluate. So you can just let your AI-in-training play
| 100*1000 rounds of the game to establish a base policy scoring.
|
| For this competition, however, it appears that the gym
| environment is not available. So to get started, I would need to
| build my own Bomberman clone while trying to mimic your graphics
| style... I'll pass on that. The headline on the blog post says
| "open Bomberland arena" but I couldn't find any way to actually
| download it. I do like the idea of having an always-on AI
| competition running online, but that type of competitive AI play
| is usually only helpful after hundreds of GPU hours of offline
| training.
|
| So that would be my one big suggestion to you, joooyzee: Put a
| small TensorFlow / PyTorch script on GitHub that just runs the
| Bomberman environment with random inputs.
|
| Once I have such a script, I can then quite easily drop in my
| reinforcement learning research and get started with the actual
| AI.
| thegalah wrote:
| Matt from coderone here, by gym do you mean a open ai gym
| wrapper.
|
| Feel free to reach out to me on our discord:
| https://discord.com/invite/NkfgvRN @thegalah or reach out to me
| directly via email matt@gocoder.one
|
| Definitely there are some misses with the environment would be
| happy to patch it up to get it into a good state. We have the
| game engine available as a binary (outside of the docker flow
| too) available here:
| https://github.com/CoderOneHQ/bomberland/releases
| fxtentacle wrote:
| Thanks for sharing that GitHub repo :) That's what I was
| looking for.
| thegalah wrote:
| Anytime, please feel free to leave issues on the repo if
| there are things or dm me and tell me what sucks
| shantnutiwari wrote:
| Sorry, not sure what this is-- an online version of OpenAI?
|
| Also, in the getting started section, it assumes docker. I
| clicked on "docker alternative" but it takes me back to the same
| page.
|
| As it is, Im not sure what this is, or how I can get started. Do
| I also have to create an account to try it out?
| joooyzee wrote:
| Thanks for the feedback! We're working on improving the
| onboarding flow. Sorry about the Docker link issue - it should
| link you to a copy of the environment binary so that you can
| play around without Docker (no docs for this workflow just yet
| unfortunately).
|
| It's essentially a Bomberman-inspired game, where you program
| the agents to play in it and can play against other users'
| agents. You can try it out without an account by cloning one of
| the starter kits here: https://github.com/CoderOneHQ/bomberland
| and following the usage instructions (but you'll need to create
| an account to use the visualizer and to submit agents).
|
| We recommend the Docker flow, but if you get stuck feel free to
| reach out to me (Joy) or @thegalah (Matt) on our Discord:
| https://discord.gg/NkfgvRN
| bvanvugt wrote:
| I'm one of the devs behind Battlesnake
| (https://play.battlesnake.com), and what Joy and the Coder One
| team are building looks awesome :)
|
| Anything that gets more folks coding, especially at non-beginner
| levels, is a huge win imo. Congrats on the launch!
| joooyzee wrote:
| Thanks Brad! Battlesnake is pretty awesome too ;) A really fun
| programming project, and great Twitch streams too.
| davidkunz wrote:
| If the game is completely symmetrical, it would be funny if
| someone comes up with an "AI" which just mirrors the opponent's
| moves. If it acts faster than the frames, we would always get a
| draw.
|
| Edit: Nevermind, just read about the power ups.
| joooyzee wrote:
| That is definitely a strategy :) Have seen people try imitation
| learning in other AI programming challenges too -- although
| usually they perform worse than the original agent they
| imitate.
| Uberphallus wrote:
| What's the endgame of this, given platforms like CodinGame,
| Kaggle or Topcoder? What sets it aside?
| joooyzee wrote:
| Thanks for the question!
|
| Where CodinGame and TopCoder are great platforms for
| competitive programming, solvable programming problems and
| short-form competitions, our longer-term focus is more on open-
| ended, ongoing sandbox simulations that evolve over time. We
| think this format will lend itself more to challenging real-
| world simulations and ML approaches (think self-driving cars,
| drones, and challenging games like StarCraft II).
|
| While Kaggle is great for classification-type problems and even
| recently started running their own simulation competitions, we
| feel there's a lot of room for a platform that is 100% purpose-
| built for simulation-type competitions (e.g. better
| visualisers, Twitch streams, matchmaking).
| Buttons840 wrote:
| Will you make a fast implementation of the environment available?
| The best AIs right now are model based (AlphaGo), so the best
| bots will probably have to reimplement the environment, unless
| you make a model available for everyone?
|
| I've been trying to create a Slay the Spire AI and am burned out
| on reimplementing environments, it's rather boring code, but
| there sure is a lot of it, and it takes a lot of work trying to
| figure out subtle details. It would be nice to be able to spend
| more than 20% of my time on actual AI stuff, rather than trying
| to reverse engineer the game so I can make a good model.
| nicoco wrote:
| Oh, I have been thinking about learning about reinforcement
| learning by trying to make a STS AI too, nice! I eventually
| gave up, but would still be interested in seeing what can be
| done. Do you plan on releasing something at some point?
|
| About re-implementing the environment, it is probably worth
| getting in touch with STS major modders and even streamers
| (jorbs comes to mind...). In case you did not do that already.
| Buttons840 wrote:
| https://github.com/DevJac/solve_the_spire
|
| I stretched the truth a bit, I'm actually doing something
| like "hierarchical model-free reinforcement learning", even
| so, figuring out how to break the game down to create a
| hierarchy of agents is a lot of work. Basically, the AI is
| composed of about 8 different traditional RL agents (neural
| networks), each deciding a different thing. One chooses which
| cards to draft, one chooses which actions to take in combat,
| one chooses which path to take on the map, etc.
|
| Simple rules like "play random cards until your energy is
| used up" alone can sometimes beat the act 1 boss. My AI is
| barely above that, and still far from solving the game. I'm
| not convinced even DeepMind or other researchers could solve
| Slay the Spire right now.
|
| It shows definite signs of improvement, but has only reached
| a point where it can beat the act 1 boss about 50% of the
| time. I think that is its limit right now. I'm doing policy
| gradient which is very sample inefficient. I'm going to
| implement soft-actor-critic and see if it can do better with
| better sample efficiency.
|
| One thing I like about Slay the Spire is it's an environment
| to solve, not a competition. Gamers like to talk about PvP
| and PvE, well, I prefer AI vs environment over AI vs AI. In
| the end, an AI will win the competition, no surprise. An AI
| solving a new kind of environment is much more exciting IMHO.
| chongli wrote:
| I feel like a traditional expert system would work a lot
| better in Slay the Spire at this stage. The choices you
| make in the game are all highly interrelated so I'm not
| sure they can be broken down into separate agents like
| that.
|
| For example, when deciding what cards to play you often
| need to take into account what is coming up next on the
| map; it is not sufficient to consider only how to win the
| current fight. Relics such as incense burner carry over
| their turn counters between fights and so it's a strong
| strategy to delay the end of the current fight in order to
| set up an optimal incense burner number for the next fight.
| What number that counter should be is highly dependent on
| which enemies/elites/bosses you'll be facing in the next
| fight.
|
| An expert system would have a database of every opponent in
| the game and when they are likely/guaranteed to appear and
| then seek to optimize the various conditions at the end of
| the current fight so that the next fight goes as smoothly
| as possible. I don't see how this could be accomplished
| with separate agents each attempting to play a different
| component of the game in isolation.
| Madmallard wrote:
| Who is to say there isn't a simple strategy that's also
| optimal? It's not exactly a complex game.
| Buttons840 wrote:
| Maybe. All approaches can be tried, that's part of the fun.
| I'm just saying that the best algorithm we know of for
| solving games in general requires a model, and so if a model
| is made available to everyone it will save people some work.
| joooyzee wrote:
| Getting the platform to the point where people can spend most
| of the time on the actual training and experiments (and less on
| the infrastructure) is our current goal. We do have a forward
| model simulator which should let you step through the
| environment without re-implementing it, but if that's not what
| you're after, we'd love to chat more on what we could do to
| make this easier (feel free to ping any of us on Discord
| https://discord.gg/tRUMgdfC).
|
| P.S. Sounds like a cool project! Have you heard of the
| Hearthstone AI competition (https://hearthstoneai.github.io/)?
| Might be of interest to you.
| thegalah wrote:
| Matt from Coderone here: have made a forward model of the
| environment available by default through the game engine so
| there shouldn't need to "much" work. There are definitely
| friction points and some more abstractions can definitely be
| made, happy to iterate off any feedback provided
| amiadams wrote:
| This looks interesting, Are there any resources you can share for
| anyone like me who will start learning ML?
| AYBABTME wrote:
| I made a similar game a few years ago while in undergrad:
| https://github.com/aybabtme/bomberman
|
| Nowhere as polished or ambitious, but it just came to mind.
| Bomberman is a fun game!
| joooyzee wrote:
| Nice work looks awesome! Looks like there are some parallels in
| our websocket implementation too.
___________________________________________________________________
(page generated 2021-09-24 23:02 UTC)