Coding period week 8

Week 8

Last week gym gazebo was upgraded to support ROS noetic, and worked on decoupling the formula plugins from Jderobot’s dependencies. Therefore, all plugins for formula 1 must be working and gym gazebo must be integrated to behavior studio.

Also it is time to try out q-learning with the formula 1 using a lidar as perception, once this is done it would be possible to test other more complex agents like the one with camera.

Q Learning

Before getting into deep q learning, we are going to make sure that models, plugins, world, and behavior studio work correctly in noetic, as a result a formula 1 with laser is being tested.

The lidar and camera versions of the formula 1 work partially with noetic.

Gym Gazebo and behavior Studio

Behavior Studio has a define architecture in order to work with deep learning models, but gym gazebo does not work the same way since gazebo and rviz are launched directly from the library when an environment is created.

Figure from the Behavior Studio documentation website .

Another challenge that should be done is to reuse current format of launching, changing models and environments using behavior studio architecture.

Some issues for next week

More tests need to be done with the laser q-learning, but within the behavior studio GUI since previous tests have been done externally. So the first steps would be to study which is the best way to make them work together.

Week Highlights

• Issues

  • Add support for RL #37
  • [Models] ROS Noetic Support for F1 #9
  • [Models] ROS Noetic Support for F1 pose estimation plugin #10

• Pull Requests

  • Adding support for RL #62

• From now on I will be working directly on the main repositories as suggested by my mentor, so all changes would be visible for other members of the team.

References

• [1] Jderobot, BehaviorStudio (BehaviorSuite)