Coding period week 9

Week 9

This is the evaluation week of second month at Google Summer of Code, it is a little bit sad that there is just one month left of the program, I have been learning a lot and enjoying working at Jderobot.

The main tasks for this week are around behavior studio GUI and gym gazebo for q-learning with the formula 1 models. First I am going to update the models at the CustomRobots repository where models should use official gazebo plugins, instead of the previous custom ones from Jderobot base.

Once all models are uploaded to CustomRobots, those would be used by Behavior Studio within docker images and native installation. Gym Gazebo has been updated for ros-noetic but it is still working independently, it should be changed so it can work using the controls from Behavior Studio.

And finally some testing needs to be done once gym gazebo works along with behavior studio, first the q-learning with formula 1 laser should generate a model that can drive the formula 1 through the behavior studio GUI.

Custom Robots

Models for formula 1 have been uploaded to the custom repository in the noetic-devel branch, those models have already applied to the dockefile images for Behavior Studio.

Gym Gazebo

Added the agents inside the brains and also worlds used for reinforcement learning added to the GUI, so now is possible to use the GUI for choosing reinforcement learning brains and worlds.

Q-learning formula 1 laser

Found a bug that sometimes the laser went out of the surface and then the program crashes, so once I identified when this happens I restarted the training in a random position.

Unexpected Issues

Beside the problem at training, sometimes the GUI cannot handle properly the reinforcement learning training by itself, therefore some bugs need to be corrected in the integration and exceptions need to be handled to about crashes.

Week Highlights

• Issues

  • Add support for RL #37
  • [Models] ROS Noetic Support for F1 #9
  • [Models] ROS Noetic Support for F1 pose estimation plugin #10

• Pull Requests

  • Adding support for RL #62

• From now on I will be working directly on the main repositories as suggested by my mentor, so all changes would be visible for other members of the team.

References

• [1] Jderobot, BehaviorStudio (BehaviorSuite)