Preliminaries

I have been sick recently so this post briefs the work done from Aug 14 to Aug 27 in the span of two weeks. In the past two weeks, we revisited the challenge of improving our model’s obstacle avoidance. Our prior effort with a weighted L1 loss showed limited progress. The prevailing issue is the model fails to learn to use the brake at all. This week we would like to explore other approaches. One approach involves expanding the dataset with more braking scenarios. We hypothesize that a dataset enriched with varied braking examples might guide the model towards better braking responses. We also plan to experiment with combining the throttle and brake as one model output. At present, our model produces three separate outputs for throttle, steering, and braking, all ranging between [0, 1]. However, a practical observation is that increasing throttle often corresponds with decreased braking, and vice versa. By combining throttle and brake into a single output, we aim to improve the obstacle avoidance capabilities of our model.

Simultaneously, we’re progressing with our Behavior Metrics integration. A priority is to draft a PR for our recently developed traffic generation feature, inviting mentor feedback. Furthermore, if time allows, we hope to lay groundwork for loading our model within Behavior Metrics.

Objectives

  • Release posts for weeks 10 and 11
  • Expand dataset with more braking scenarios
  • Combine throttle and brake as one command
  • Draft PR for Behavior Metrics’ traffic generation feature

Execution

Dataset with Balanced Throttling & Braking

In week 8, we decided to trim the dataset by removing the stationary frames where the vehicle is just waiting in line at a red traffic light. The goal was to make the dataset more compact and consists of only “effective” data, which proved to be a wrong decison. As shown in the left image below, the vehicle is not using the brake (brake = 0) at all in most of the frames from the dataset. The right image shows that after we add more braking data, the amount of time the vehicle is braking vs not braking is roughly balanced.

Before (left) and after (right) adding more braking data

After including more braking data in the dataset, we evaluated the model by plotting out the predictions vs ground truth on the training set. As shown in the image blow, the model appears to have learned throttle and steer relatively well yet still struggles with brake. This corroborates with our observation that the model does loose throttle when there is obstacle ahead, but it often fails to utilize the brake. In the Evaluation Results section below, we conduct a more thorough evaluation on the performance of the model.

Combining Thorttle & Brake Control

Currently our model outputs three vehicle control commands: throttle, steer, and brake. Since the model has not been utilizing the brake after training, we decided to experiment on combining the throttle and brake into one command. Intuitively, the vehicle should either be braking or throttling and not both at the same time. We modify the model such that it outputs two values between [0, 1], one for throttle/brake, and one for steer. For the throttle/brake command, we threshold it at 0.5 such that if it’s above the threshold, the vehivle hits thorttle, and vice versa.

\(throttle = (combined\_control - 0.5) / 0.5 \quad \text{if combined_control >= 0.5}\) \(brake = (0.5 - combined\_control) / 0.5 \quad \text{if combined_control <0.5}\)

As shown in the image below, the model has learned brake well as most predictions are close to the ground truth. However, it still tends to over-use the throttle and under-utilize the brake.

Evaluation Results

We compare the following models:

  1. Unbalanced Dataset: model from our last iteration that was trained on an unbalanced dataset
  2. Balanced Dataset: same as 1 except the brake values in dataset is balanced
  3. Balanced Dataset + Additional Junction Data: same as 2 except with 50 additional episodes of the expert agent crossing a junction
  4. Balanced Dataset + Combined Control: combines throttle and brake into a single command
  5. Balanced Dataset + Combined Control + Additional Junction Data: same as 4 except with 50 additional episodes of the expert agent crossing a junction

From the evaluation results above, we make the following observations:

  • Compared to the model trained on the unbalanced dataset, the models trained on balanced dataset are better at obstacle avoidance in terms of decreased nubmer of collisions with other vehicles.
  • Despite the improved performance in obstacle avoidance, the overall success rate decreased drastically. Specifically, the new models frequently get stuck in a state and time out eventually. This might be because the model is being overly cautious and drives too slowly overall. The agent might be over-predicting the necessity to brake due to ambiguous situations or over-representing the braking action in the dataset.
  • Counterintuitively, the additional 50 episodes of “junction data” doesn’t improve the learning. Notably, the number of collisions increased significantly (model 3 verus 2, 5 verus 4). It’s possible that the behavior in the junctions is conflicting with behavior in other parts of the driving task. This can lead to situations where the agent is confused about which behavior to imitate.