Gym and Gazebo

2 minute read

Objectives

Getting familiar with the tools we will be using
Write some definitions about the tools.
Install the tools.
Reproduce an example.

Definitions

OpenAI Gym

OpenAI Gym is a framework to test and developed reinforcement learning algorithms, it includes many environments like the Atari games [1].

CarRacing

Gazebo

Is an open-source 3D robotics simulator, and it is integrated with ROS [2] .

Behaviour Studio

Is a tool that will benchmark different algorithms for autonomous driving using Jderobot tools [3].

Using Behaviour Studio, Gym and Gazebo

I used the installation guide from Ignacio Arranz’s master thesis repository. I only pasted here the installation commands, there are more for display and running environments in the Ignacio’s repository I mentioned before [5].

Installation

ROS Melodic and Gazebo 9.0 in Ubuntu 18.04

ROS wiki

sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'
sudo apt-key adv --keyserver 'hkp://keyserver.ubuntu.com:80' --recv-key C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
sudo apt update
sudo apt install ros-melodic-desktop-full
echo "source /opt/ros/melodic/setup.bash" >> ~/.bashrc
source ~/.bashrc

Gym-Gazebo

git clone https://github.com/RoboticsLabURJC/2019-tfm-ignacio-arranz
cd 2019-tfm-ignacio-arranz/gym-gazebo
virtualenv -p python3 gazebo-gym
source gazebo-gym/bin/activate
pip install -r requirements.txt
pip install -e .

Adding more packages needed

sudo apt-get install                     \
python-pip python3-vcstool python3-pyqt4 \
pyqt5-dev-tools                          \
libbluetooth-dev libspnav-dev            \
pyqt4-dev-tools libcwiid-dev             \
cmake gcc g++ qt4-qmake libqt4-dev       \
libusb-dev libftdi-dev                   \
python3-defusedxml python3-vcstool       \
ros-melodic-octomap-msgs                 \
ros-melodic-joy                          \
ros-melodic-geodesy                      \
ros-melodic-octomap-ros                  \
ros-melodic-control-toolbox              \
ros-melodic-pluginlib	                 \
ros-melodic-trajectory-msgs              \
ros-melodic-control-msgs                 \
ros-melodic-std-srvs 	                 \
ros-melodic-nodelet                      \
ros-melodic-urdf                         \
ros-melodic-rviz                         \
ros-melodic-kdl-conversions              \
ros-melodic-eigen-conversions            \
ros-melodic-tf2-sensor-msgs              \
ros-melodic-pcl-ros                      \
ros-melodic-navigation                   \
ros-melodic-sophus                       \
python-rviz

Problems encountered

Python

I got this error, because my tf2_ros was working with python2

ImportError: dynamic module does not define module export function (PyInit__tf2)

I tried many possible solutions like these.

Attempted solution

Then reading other solutions I found out that my problem might have been caused because I was using conda, so I did a clean docker installation and it worked!.

Environment variables

Loading the environment variables returned this message

bash formula1_setup.bash
cp: cannot stat '../assets/urdf/kobuki_nn_urdf/urdf/': No such file or directory                       
cp: cannot stat '../assets/meshes/lidar_lite_v2_withRay.dae': No such file or directory

I found the missing files in the gym-gazebo repository from erlerobot, so I copied to the Ignacio’s repository and the bash ran correctly now.

git clone https://github.com/erlerobot/gym-gazebo
cp -r gym-gazebo/gym_gazebo/envs/assets/meshes/ 2019-tfm-ignacio-arranz/gym-gazebo/gym-gazebo/envs/assets/
cp -r gym-gazebo/gym_gazebo/envs/assets/urdf/ 2019-tfm-ignacio-arranz/gym-gazebo/gym-gazebo/envs/assets/

Python 2 syntax

In order to run the code using python3 a few changes were made like changing xrange for range.

Environment variable not set

RLException: Invalid <arg> tag: environment variable 'GYM_GAZEBO_WORLD_CIRCUIT_F1' is not set.

Arg xml is <arg default="$(env GYM_GAZEBO_WORLD_CIRCUIT_F1)" name="world_file"/>
The traceback for the exception was written to the log file

export GYM_GAZEBO_WORLD_CIRCUIT_F1=/root/2019-tfm-ignacio-arranz/gym-gazebo/gym_gazebo/envs/assets/worlds/f1_1_simplecircuit.world

Running

Turtlebot example

cd gym_gazebo/envs/installation
bash setup_melodic.bash
bash turtlebot_setup.bash

cd agents/turtlebot
python3 circuit2_turtlebot_lidar_qlearn.py

Turtlebot

Formula 1 example

cd gym_gazebo/envs/installation
bash setup_melodic.bash
bash formula1_setup.bash

cd agents/f1
python f1_follow_line_camera.py

Formula1

Stopping the training

echo "alias killgazebogym='killall -9 rosout roslaunch rosmaster gzserver nodelet robot_state_publisher gzclient'" >> ~/.bashrc

Outcomes

I opened an issue for the missing files issue.

References

[1] Greg Brockman, et al. “OpenAI Gym” 2016.

[2] Andrew Howard, et al. “Gazebo” 2002 at the University of Southern California.

[3] Jderobot, BehaviorStudio (BehaviorSuite)

[4] Ignacio Arranz’s master thesis repository

[5] Gym-gazebo

[6] OpenCV with python3

Twitter LinkedIn