Flappy Bird Which Learn And Play By Itself
In this article, we will explore the fascinating world of neural networks and game development by creating a Flappy Bird game that is powered by a neural network.
The game will be built using JavaScript, and we will also create a separate file for the neural network code, which we will call neuroevolution.js.
We will start by discussing the basics of neural networks and how they work. We will then delve into the details of the game mechanics and how we can implement them using JavaScript. We will also discuss how we can train the neural network to play the game and keep on playing infinitely, until we manually stop it.
Throughout the article, we will be using a step-by-step approach that will make it easy for readers to follow along, even if they have no prior experience with neural networks or game development. We will also provide explanations and code snippets to help readers understand the concepts and implement the code.
By the end of the article, readers will have a solid understanding of neural networks and how they can be used in game development. They will also have a working Flappy Bird game that is powered by a neural network, which they can customize and experiment with.
First we create and html file, in which we can see what’s happening:
<html>
<head>
<title>NeuroEvolution : Flappy Bird</title>
<link href='https://fonts.googleapis.com/css?family=Oswald' rel='stylesheet' type='text/css'>
</head>
<body>
<canvas id="flappy" width="500" height="512"></canvas> <br/>
<button onclick="speed(60)">x1</button>
<button onclick="speed(120)">x2</button>
<button onclick="speed(180)">x3</button>
<button onclick="speed(300)">x5</button>
<button onclick="speed(0)">MAX</button>
<br/>
<a href="http://onepagecode.substack.com">Substack Repository</a>
<script src = "Neuroevolution.js"></script>
<script src = "game.js"></script>
</body>
</html>Now let’s start coding the game:
Game.js
(function() {
// Create the timeouts array
var timeouts = [];
// Create a random message name
var messageName = "zero-timeout-message";
// Create the setZeroTimeout function
function setZeroTimeout(fn) {
timeouts.push(fn);
window.postMessage(messageName, "*");
}
// Create the handleMessage function
function handleMessage(event) {
// Check if the source of the message is the window
// and the data matches the messageName
if (event.source == window && event.data == messageName) {
// Stop propagation of the event
event.stopPropagation();
// Check if there are any timeouts
if (timeouts.length > 0) {
// Get the first timeout from the array
var fn = timeouts.shift();
// Execute the function
fn();
}
}
}
// Add the event listener
window.addEventListener("message", handleMessage, true);
// Set the setZeroTimeout function on the window object
window.setZeroTimeout = setZeroTimeout;
})();This code defines a setZeroTimeout function that allows a function to be executed in the next event loop iteration, but with zero delay, essentially providing a way to bypass the minimum delay enforced by setTimeout and setInterval in some browsers.
The setZeroTimeout function works by pushing the function to be executed into an array of timeouts, and then posting a message to the window object. When the window receives the message, it checks if there are any timeouts in the array, executes the first one, and removes it from the array.
The code also creates a handleMessage function that handles the message posted by setZeroTimeout, and adds an event listener for the message event to the window object. Finally, the setZeroTimeout function is attached to the window object, making it available for use throughout the script.
var Neuvol;
var game;
var FPS = 60;
var maxScore=0;
var images = {};
var speed = function(fps){
FPS = parseInt(fps);
}In the above code, we create some variables which will be useful in building he game.
var loadImages = function(sources, callback){
// nb is the number of images to load
var nb = 0;
// loaded is the number of images that have been loaded so far
var loaded = 0;
// imgs is an array of images
var imgs = {};
// for each source in sources
for(var i in sources){
// increment nb
nb++;
// set imgs[i] to a new image
imgs[i] = new Image();
// set the source of imgs[i] to the current source
imgs[i].src = sources[i];
// add an onload event on imgs[i]
imgs[i].onload = function(){
// increment loaded
loaded++;
// if loaded == nb
if(loaded == nb){
// call callback(imgs)
callback(imgs);
}
}
}
}The loadImages function takes two arguments: sources, an object containing the source paths of the images to be loaded, and callback, a function to be called once all images are loaded.
Within the function, nb is set to the number of images to be loaded, which is the length of the sources object. loaded is initialized to 0, which will be incremented each time an image is loaded. imgs is an empty object that will be populated with the loaded images.
A for loop is used to iterate over the sources object. Within each iteration, nb is incremented and a new Image object is created and assigned to imgs[i]. The source of the image is set to the current source in the loop. An onload event listener is added to the image, which will be triggered once the image is loaded. Within the event listener, loaded is incremented and if loaded is equal to nb, the callback function is called with the imgs object as an argument.
Essentially, the loadImages function asynchronously loads multiple images and executes a callback function when all the images are loaded.
Creating the bird object:
// this function creates a new bird object
var Bird = function(json){
// the position of the bird
this.x = 80;
this.y = 250;
// the size of the bird
this.width = 40;
this.height = 30;
// this variable determines if the bird is alive
this.alive = true;
// the gravity of the bird
this.gravity = 0;
// the velocity of the bird
this.velocity = 0.3;
// the jump of the bird
this.jump = -6;
this.init(json);
}
// Make a new Bird object using the data from the JSON object
Bird.prototype.init = function(json){
// Loop through each property in the JSON object
for(var i in json){
// Copy the value from the JSON object to this object
this[i] = json[i];
}
}
// Add a new method to the Bird object called flap.
Bird.prototype.flap = function(){
// Set the gravity speed to the jump speed.
this.gravity = this.jump;
}
// Add more detailed comments to your code to describe each step
Bird.prototype.update = function(){
// This function updates the bird's position on the screen
this.gravity += this.velocity;
this.y += this.gravity;
}
Bird.prototype.isDead = function(height, pipes){
// If the bird is off the screen, return true
if(this.y >= height || this.y + this.height <= 0){
return true;
}
// For each pipe
for(var i in pipes){
// If the bird is not touching the pipe
if(!(
// If the bird is to the right of the pipe
this.x > pipes[i].x + pipes[i].width ||
// If the bird is to the left of the pipe
this.x + this.width < pipes[i].x ||
// If the bird is below the pipe
this.y > pipes[i].y + pipes[i].height ||
// If the bird is above the pipe
this.y + this.height < pipes[i].y
)){
// Return true
return true;
}
}
}var Pipe = function(json){
// Define the default values for our pipe
this.x = 0;
this.y = 0;
this.width = 50;
this.height = 40;
this.speed = 3;
// Initialize the pipe with the values passed in
this.init(json);
}This is a constructor function for a Pipe object in JavaScript. The Pipe function takes in a single argument, json, which is expected to be a JSON object containing properties to initialize the Pipe object with.
Inside the Pipe function, there are several default properties being defined for the Pipe object, i ncluding x, y, width, height, and speed. These properties are initialized with default values.
After defining the default properties, the init method is called on the Pipe object with the json argument passed in. The purpose of this method is to update the default properties with any values passed in via the json argument.
So essentially, this constructor function creates a Pipe object with default properties, and then initializes those properties with any values passed in via the json argument.
Pipe.prototype.init = function(json){
// iterate through each property in the json object
for(var i in json){
// add the property and value to the Pipe object
this[i] = json[i];
}
}This is the implementation of the init method of the Pipe object. As mentioned in the previous explanation, this method is responsible for updating the default properties of the Pipe object with any values passed in via the json argument.
The init method takes in the json argument, which is expected to be a JSON object containing properties to update the Pipe object with.
Inside the method, a for loop is used to iterate through each property in the json object. For each property, the method sets the corresponding property of the Pipe object to the value of the property in the json object. This is achieved by using bracket notation to access the property of the Pipe object with the same name as the property being iterated over in the json object.
In essence, this method allows you to initialize or update the properties of a Pipe object by passing in a JSON object containing key-value pairs of properties and their corresponding values.
// this function will be called every frame
Pipe.prototype.update = function(){
// move the pipe to the left
this.x -= this.speed;
}This is the implementation of the update method of the Pipe object. This method is responsible for updating the position of the Pipe object.
The update method doesn’t take in any arguments, it simply updates the x property of the Pipe object by subtracting the speed property from it. This moves the Pipe object to the left.
In summary, the update method is called every frame (or at a specified interval) to update the position of the Pipe object by moving it to the left by an amount determined by the speed property.
Pipe.prototype.isOut = function(){
// if the pipe is out of the canvas, return true
// if the pipe is in the canvas, return false
if(this.x + this.width < 0){
return true;
}
}This is the implementation of the isOut method of the Pipe object. This method is responsible for determining whether the Pipe object is outside of the visible area of the canvas.
The isOut method doesn’t take in any arguments, it simply checks whether the sum of the x and width properties of the Pipe object is less than 0. If this condition is true, then the Pipe object is outside of the visible area of the canvas and the method returns true. Otherwise, the Pipe object is still visible and the method returns false.
In essence, the isOut method is called every frame (or at a specified interval) to check whether the Pipe object has moved outside of the visible area of the canvas. If it has, then the game can take appropriate action, such as removing thePipe object or resetting its position.
var Game = function(){
//create an array of pipes
this.pipes = [];
//create an array of birds
this.birds = [];
//initial score is 0
this.score = 0;
//canvas is the flappy canvas
this.canvas = document.querySelector("#flappy");
//the context of the canvas is 2d
this.ctx = this.canvas.getContext("2d");
//width of the canvas
this.width = this.canvas.width;
//height of the canvas
this.height = this.canvas.height;
//spawn interval is 1.5 seconds
this.spawnInterval = 90;
//interval is 0
this.interval = 0;
//generation is 0
this.generation = 0;
//create an array of generations
this.gen = [];
//number of birds alive is 0
this.alives = 0;
//background speed is 0.5
this.backgroundSpeed = 0.5;
//background x is 0
this.backgroundx = 0;
//max score is 0
this.maxScore = 0;
}This is the implementation of the Game object constructor. This constructor initializes a new Game object with default values for its properties.
The Game object has several properties:
pipes: an array that will contain Pipe objects.
birds: an array that will contain Bird objects.
score: the current score of the game, which is initially set to 0.
canvas: a reference to the HTML canvas element that the game will be rendered on.
ctx: the 2D rendering context of the canvas.
width: the width of the canvas.
height: the height of the canvas.
interval: a counter to keep track of the number of frames since the last pipe was spawned.
generation: the current generation of birds.
gen: an array that will contain all of the generations of birds.
alives: the number of birds that are still alive.
backgroundSpeed: the speed at which the background will move.
backgroundx: the x position of the background.
maxScore: the highest score achieved in the game.
In summary, the Game object constructor sets up the initial state of the game by initializing default values for its properties.
Game.prototype.start = function(){
// reset the game
this.interval = 0;
this.score = 0;
this.pipes = [];
this.birds = [];
// get the new generation from Neuvol
this.gen = Neuvol.nextGeneration();
for(var i in this.gen){
var b = new Bird();
this.birds.push(b)
}
this.generation++;
this.alives = this.birds.length;
}This is the implementation of the start method of the Game object. This method is responsible for resetting the game and starting a new generation of birds.
When the start method is called, it resets the interval, score, pipes, and birds properties of the Game object to their default values. It then retrieves a new generation of birds from the Neuvol object (presumably an external neural network library), and creates a new Bird object for each bird in the generation. These Bird objects are then added to the birds array of the Game object.
After creating the new generation of birds, the generation property of the Game object is incremented, and the alives property is set to the number of birds in the birds array.
In summary, the start method of the Game object resets the game state and starts a new generation of birds, which will be used to play the game.
Game.prototype.update = function(){
// Move the background
this.backgroundx += this.backgroundSpeed;
// Keep track of the distance to the next pipe
var nextHoll = 0;
// Make sure there is at least one bird alive
if(this.birds.length > 0){
// Loop over all pairs of pipes
for(var i = 0; i < this.pipes.length; i+=2){
// Check if the next pipe is after the bird
if(this.pipes[i].x + this.pipes[i].width > this.birds[0].x){
// Calculate the distance to the next pipe
nextHoll = this.pipes[i].height/this.height;
// Stop looping
break;
}
}
}This is the implementation of the update method of the Game object. This method is responsible for updating the game state during each frame of the game.
Firstly, the method moves the background by updating the backgroundx property of the Game object by the backgroundSpeed value.
Next, the method checks if there is at least one bird alive. If there is, the method loops over all pairs of pipes (presumably representing the obstacles in the game). For each pair of pipes, the method checks if the next pipe is after the bird. If it is, the method calculates the distance to the next pipe by dividing the height of the pipe by the height of the canvas. Finally, the loop is stopped using the break statement.
The purpose of the nextHoll variable is to keep track of the distance to the next pipe, which will be used later to determine the fitness of each bird in the game.
for(var i in this.birds){
// if the bird is alive
if(this.birds[i].alive){
// get the inputs for the bird
var inputs = [
this.birds[i].y / this.height,
nextHoll
];
// compute the output of the neural network
var res = this.gen[i].compute(inputs);
// if the output is greater than 0.5, flap
if(res > 0.5){
this.birds[i].flap();
}
// update the bird
this.birds[i].update();
// if the bird is dead (out of the screen or collided with a pipe)
if(this.birds[i].isDead(this.height, this.pipes)){
// the bird is dead
this.birds[i].alive = false;
// decrease the number of birds alive
this.alives--;
// compute the score of the bird
Neuvol.networkScore(this.gen[i], this.score);
// if this was the last bird alive
if(this.isItEnd()){
// start a new generation
this.start();
}
}
}
}
// Update the pipes
for(var i = 0; i < this.pipes.length; i++){
this.pipes[i].update();
// If the pipe is out of the screen
if(this.pipes[i].isOut()){
// Remove it from the list
this.pipes.splice(i, 1);
// Move one step back in the list
i--;
}
}This block of code is responsible for updating the state of the game. It loops through each bird and checks if it is alive. If it is, it gets the inputs for the bird, which includes its current y position and the distance to the next hole in the pipes. It then computes the output of the neural network and flaps the bird if the output is greater than 0.5. The bird’s position is then updated, and if it is dead (i.e., out of the screen or collided with a pipe), it is marked as dead, its score is computed, and the number of birds alive is decreased. If there are no more birds alive, the game starts a new generation.
After updating the state of the birds, the code updates the state of the pipes. It loops through each pipe, updates its position, and removes it from the list if it is out of the screen.
if(this.interval == 0){
// Create a new pipe each 100 frames
var deltaBord = 50;
var pipeHoll = 120;
// Compute a random position for the pipe holl
var hollPosition = Math.round(Math.random() * (this.height - deltaBord * 2 - pipeHoll)) + deltaBord;
// Create the two pipes
this.pipes.push(new Pipe({x:this.width, y:0, height:hollPosition}));
this.pipes.push(new Pipe({x:this.width, y:hollPosition+pipeHoll, height:this.height}));
}
// Add one to the interval.
this.interval++;
// If the interval has reached the spawn interval.
if(this.interval == this.spawnInterval){
// Reset the interval.
this.interval = 0;
}
// Add 1 to the current score
this.score++;
// Update the maximum score if the current score is greater than the max score
this.maxScore = (this.score > this.maxScore) ? this.score : this.maxScore;
// Store the current object in the variable self - this is needed to avoid a conflict
// with the keyword "this" in the setTimeout function
var self = this;
// If the FPS is set to 0, call the update function using setZeroTimeout
if(FPS == 0){
setZeroTimeout(function(){
self.update();
});
// Otherwise, call the update function using setTimeout
}else{
setTimeout(function(){
self.update();
}, 1000/FPS);
}
}This is the end of the code for the Flappy Bird game. The update function updates the state of the game and calls itself again after a certain amount of time using either setTimeout or setZeroTimeout.
The first section of the update function moves the background, computes the distance to the next pipe, and checks if there is at least one bird alive. Then, for each bird that is alive, it computes the inputs for the neural network (the y-position of the bird and the distance to the next pipe), computes the output of the neural network (flap or not), updates the bird’s position, and checks if the bird is dead. If a bird is dead, its score is computed using the Neuvol object, and if all birds are dead, a new generation is created.
The second section of the update function updates the position of each pipe, removes pipes that are out of the screen, and creates new pipes every certain number of frames.
The last section of the update function updates the score and calls the update function again after a certain amount of time. If FPS is set to 0, it uses setZeroTimeout, which is a more accurate version of setTimeout that uses the browser’s event loop to avoid long-running JavaScript code from blocking the UI.
Game.prototype.isItEnd = function(){
// loop through all the birds
for(var i in this.birds){
// if any of the birds are still alive
if(this.birds[i].alive){
// then the game is not over
return false;
}
}
// if we got here then all the birds are dead
// so the game is over
return true;
}The isItEnd() function in the Game class checks if any bird is still alive in the game. It loops through all the birds and checks if any of them have the alive property set to true. If it finds any such bird, it returns false, indicating that the game is not over yet. If it loops through all the birds and none of them are alive, it returns true, indicating that the game is over. This function is used in the update() function to determine if a new generation needs to be started.
Game.prototype.display = function(){
this.ctx.clearRect(0, 0, this.width, this.height);
for(var i = 0; i < Math.ceil(this.width / images.background.width) + 1; i++){
this.ctx.drawImage(images.background, i * images.background.width - Math.floor(this.backgroundx%images.background.width), 0)
}
for(var i in this.pipes){
if(i%2 == 0){
this.ctx.drawImage(images.pipetop, this.pipes[i].x, this.pipes[i].y + this.pipes[i].height - images.pipetop.height, this.pipes[i].width, images.pipetop.height);
}else{
this.ctx.drawImage(images.pipebottom, this.pipes[i].x, this.pipes[i].y, this.pipes[i].width, images.pipetop.height);
}
}
this.ctx.fillStyle = "#FFC600";
this.ctx.strokeStyle = "#CE9E00";
for(var i in this.birds){
if(this.birds[i].alive){
//save the current drawing state
this.ctx.save();
//translate the canvas origin to the center of the bird
this.ctx.translate(this.birds[i].x + this.birds[i].width/2, this.birds[i].y + this.birds[i].height/2);
//rotate the canvas origin around the center of the bird
this.ctx.rotate(Math.PI/2 * this.birds[i].gravity/20);
//draw the bird, offset so that the canvas origin is at the top left corner of the image
this.ctx.drawImage(images.bird, -this.birds[i].width/2, -this.birds[i].height/2, this.birds[i].width, this.birds[i].height);
//restore the previous drawing state
this.ctx.restore();
}
}
// Set the text color
this.ctx.fillStyle = "white";
// Set the font
this.ctx.font="20px Oswald, sans-serif";
// Draw the score
this.ctx.fillText("Score : "+ this.score, 10, 25);
// Draw the max score
this.ctx.fillText("Max Score : "+this.maxScore, 10, 50);
// Draw the generation
this.ctx.fillText("Generation : "+this.generation, 10, 75);
// Draw the population
this.ctx.fillText("Alive : "+this.alives+" / "+Neuvol.options.population, 10, 100);
var self = this;
requestAnimationFrame(function(){
// 1. Request the next animation frame for the display function
self.display();
});
}This function is responsible for displaying the game on the canvas. It first clears the canvas and draws the background, pipes, and birds on the canvas. It then displays the score, maximum score, generation, and population information on the canvas. The birds are drawn rotated based on their gravity value. If the bird is alive, it is drawn with its center at its (x,y) position. The canvas origin is first translated to the center of the bird and then rotated around that point. The bird image is then drawn with its top left corner at the new canvas origin. Finally, the previous drawing state is restored. The function uses the requestAnimationFrame() method to request the next animation frame for the display function, which allows for smooth animation.
window.onload = function(){
var sprites = {
bird:"./img/bird.png",
background:"./img/background.png",
pipetop:"./img/pipetop.png",
pipebottom:"./img/pipebottom.png"
}
var start = function(){
Neuvol = new Neuroevolution({
population:50,
network:[2, [2], 1],
});
game = new Game();
game.start();
game.update();
game.display();
}
loadImages(sprites, function(imgs){
images = imgs;
start();
})
}This code sets up the game by loading images and creating a new instance of the Neuroevolution and Game objects. The code is run when the window finishes loading. The sprites object defines the file paths for the images used in the game. The start function creates a new instance of Neuroevolution with a population of 50 and a neural network with 2 input neurons, 2 hidden neurons, and 1 output neuron. It also creates a new instance of the Game object, starts the game, updates it, and displays it. The loadImages function loads the images defined in the sprites object and passes them to the start function when they are finished loading. Overall, this code sets up the basic infrastructure for the game and initializes the necessary objects to run it.
Now let’s start working on our neural network. The file is called neuroevolution.js
First we declare the the variable Neuronevolution.
/**
* Provides a set of classes and methods for handling Neuroevolution and
* genetic algorithms.
*
* @param {options} An object of options for Neuroevolution.
*/
var Neuroevolution = function (options) {
var self = this; // reference to the top scope of this module// Declaration of module parameters (options) and default values
self.options = {
/**
* Logistic activation function.
*
* @param {a} Input value.
* @return Logistic function output.
*/
activation: function (a) {
ap = (-a) / 1;
return (1 / (1 + Math.exp(ap)))
},
/**
* Returns a random value between -1 and 1.
*
* @return Random value.
*/
randomClamped: function () {
return Math.random() * 2 - 1;
},
// various factors and parameters (along with default values).
network: [1, [1], 1], // Perceptron network structure (1 hidden
// layer).
population: 50, // Population by generation.
elitism: 0.2, // Best networks kepts unchanged for the next
// generation (rate).
randomBehaviour: 0.2, // New random networks for the next generation
// (rate).
mutationRate: 0.1, // Mutation rate on the weights of synapses.
mutationRange: 0.5, // Interval of the mutation changes on the
// synapse weight.
historic: 0, // Latest generations saved.
lowHistoric: false, // Only save score (not the network).
scoreSort: -1, // Sort order (-1 = desc, 1 = asc).
nbChild: 1 // Number of children by breeding.
}This code block initializes an object called options with various properties and their default values that are used to configure and control the behavior of the neural network.
The activation property specifies a logistic activation function, which takes an input value a and returns the output value of the logistic function applied to a.
The randomClamped property returns a random value between -1 and 1, which is used to initialize the weights of the synapses in the neural network.