↓Skip to main content

Building An AI That Can Play Games - Part II

·832 words·4 mins
Neural Networks Game development
Tzur Eliya
Author
Tzur Eliya
Developer, Gamer, Dad

if you haven’t yet, check out part I here

So, what’s all that has to do with an AI that can play games? #

Meet - Our AI Player
#

Alt text

We want him to reach the end of this simple level: #

Alt text

… so he can have some dinner

Alt text

The Problem #

He has no idea how to reach the end of the level.

  • should he go left? right?
  • when should he jump?
  • how much time does he need to move in one direction in order to get somewhere?

He does not have a brain to help him navigate.

The Solution #

I’m going to help him by using a Neural Network as his brain. His Neural Network will learn how to overcome obstacles through a process of trial and error. I’ll add the genetic algorithm On top of that, to make the learning process easier. Basically, like we saw before, our neural network’s learning process will require us to evolve the brain of the AI player each time, to reach its full potential.

Step I #

Initiate a population of N untrained AI players (randomly generated brains) - first Generation

Alt text

Step II #

Let these players do what they want and explore the level for 35 seconds - AKA “Time to live”

Step III #

Sort the players by a fitness rank, and choose who got the highest score.

Alt text

95 is the highest score, so we choose that AI player

Alt text

Step IV #

Destroy 50% of the population - those with the lowest score.

Alt text

Step V - end of genaration flow: #

  • Take the best player, and clone him instead of the half we killed.
  • Mutate a trait (10% chance) in one of the new cloned players - change something in their Neural Network.
  • Return to step 2 (Let them run around and calculate new scores).

This goes on, until we decide the results are good enough. We take the best Neural Network as our AI.

Creating the Neural Network #

Now that the flow of the design is done, we need to get down to the specifics - how do we build the Neural Network for our AI? What does our AI needs in order to play the level?

  • He needs to see the spaces between the platforms.
  • He needs to sense that he is standing on a platform.
  • He needs to see (mid fall) if he has a platform underneath him - “Am I going to land on a platform?”

Alt text

  • Forward ray is intended for the “when is the next space between platforms”
  • “groundCheck” is intended to sense he is standing on a platform
  • Downward ray is to see if he’s going to land on a platform

Fitness function #

Crucial traits we want to consider when choosing the best AI player -

  • Which player got the farthest.
  • Which player survived most of the time
  • Which was fastest player.

The First Try #

  • I did not trust the player with knowing the correct way - so I made him automatically move to the right of the level.
  • I did not let him see what’s underneath him (downward ray did not exist yet)

Neural Network had 2 inputs: #

  1. Distance to the next space between platforms (pit)
  2. Sense if he is on the ground - boolean(true/false)

Neural network output: #

  1. “Should I jump, or not?”

Fitness Function #

  • How far did a player get?
Here’s the run (there’s background music! beware) #

That did not go so well #

So, the player’s brain was still missing a few things. Maybe Free will? Let him choose the direction and speed he should go

Second Try #

Neural Network had 3 inputs: #

  1. Distance to the next space between platforms (pit)
  2. Sense if he is on the ground - boolean(true/false)
  3. Am I going to land on a platform?

Neural network now had 2 outputs: #

  1. Direction + speed to go, which was float in this segment [-1, 1]
  2. When to jump

Fitness Function #

  • How far did a player get? (No change)

I added another 5 seconds to reach the end of the level.

Here’s the run (there’s background music! beware) #

Results of Second Try #

In 23 Generations - the network finished the level! But the work is not done. The network can learn faster. So I made some changes: Now I’ve calculated the fitness to have more exact stats. Up until this point, the fitness was not accurate enough.

Third Try #

So I gave it a third try, which was like the second try, with better a better fitness function.

Fitness Function #

  • How far did a player get? (No change)
  • Did the player fall very early on?
  • how much time did it take the player to reach his current end point?
Here’s the run (there’s background music! beware) #

And now, our player can play the game easily

Not only that, it can finish similar levels as well, as long as they are in the same format.