Coding period week 12

1 minute read

Week 12

This week lots of new things are being implemented and documented, things should be clear for any user. Therefore, adding documentation for brains is very important since it will allow users to implement their own brains for formula 1.

Dockerfiles for GPU has been tested and updated to contain the latest changes ready to be tested and used.

An important part of this project is the documentation comparing all the different reinforcement learning methods used with formula 1 and which suits best for the tasks, how hyperparameters affect performance and how the brain could train faster.

With the recent changes is easier to explore and train in multiple setups a line following fomulua 1 race car using q learning, and it has been documented how to add a new brain with a different reinforcement learning approach.

Hyperparameters

Now I added the option to directly control some parameters for reinforcement learning can be set up directly from the yml file below the type of the robot which in this example is f1rl.

BrainPath: 'brains/f1rl/train.py'
Type: 'f1rl'
Parameters:
    action_set: 'simple'
    gazebo_positions_set: 'pista_simple'
    alpha: 0.2 
    gamma: 0.9
    epsilon: 0.99
    total_episodes: 20000
    epsilon_discount: 0.9986 
    env: 'camera'

Experiments

Q-learning Laser Formula 1

Usually training in a computer with RTX 2080 to get to the 1000 epochs takes around 6 hours, at this point the model has a good policy. Using a simple set of actions where only three actions are possible left, right and straight.

Q-learning Camera Formula 1

The training time with camera has a similar length of training than training with laser. This is because this camera approach is using a simplified approach where the training is done with the errors between the central line and the camera central direction.

Week Highlights

References

You May Also Enjoy