In recent discussions, we realize that the need for a distance-aware method to enhance the precision of autonomous driving actions within the CARLA simulator. Our previous control models lack a mechanism for dynamically adjusting actions based on the actual distance to objects or specific targets. By introducing a distance-aware approach, control actions can be fine-tuned by calculating the difference between initial and final positions. This capability is crucial for maneuvers like U-turns, where precise control over trajectory and stopping distances significantly impacts performance.
In this week, LLMs are being explored for their potential to interpret natural language instructions, especially those that include ambiguous distance-related commands, such as “move a bit closer” or “stop at a safe distance.” By parsing these instructions, LLMs can generate actionable, distance-specific commands for the control module.
In terms of validation, a proof-of-concept for the distance measurement within a simplified CARLA scenario was developed. This demo will test the accuracy by measuring how well the system calculates and responds to varying distances in real-time.
In autonomous driving, instructions need to be clear, concise, and actionable. The new prompt system generates driving instructions by incorporating the specific action and distance parameters. Here’s how it works:
def generate_instruction_prompt(action):
"""
Generate a prompt for the OpenAI API to create a driving instruction for a given action.
"""
return f"""
You are generating driving instructions for an autonomous vehicle system. Each instruction should have a clear structure and follow these specific guidelines:
1. The instruction should contain the following fields:
- 'Instruction': A short, human-readable driving instruction that includes the action 'action'. The instruction should also specify the distance or condition.
- 'Action': This should be exactly one of the following: 'Right', 'Left', 'Straight', or 'LaneFollow'. For this instruction, it should be 'action'.
- 'Distance': A distance measurement in meters (e.g., '500 meters') or kilometers (e.g., '1.2 kilometers'). This should reflect how far the action applies or when the next instruction should be executed.
2. The 'Instruction' should be realistic and concise, similar to how a navigation system provides directions. Here are three detailed examples:
- Example 1:
- Instruction: "Turn right at the next intersection and continue for 500 meters."
- Action: "Right"
- Distance: "500 meters"
- Example 2:
- Instruction: "Proceed straight for 1.5 kilometers until you reach the gas station."
- Action: "Straight"
- Distance: "1.5 kilometers"
- Example 3:
- Instruction: "Follow the lane for 3 kilometers and exit at the next junction."
- Action: "LaneFollow"
- Distance: "3 kilometers"
3. Ensure the action 'action' is embedded naturally in the 'Instruction', and that it specifies when or where the action should occur.
4. The generated JSON output should have the following structure:
Instruction
Based on this structure, generate a driving instruction for the action 'action'.
"""
For example, an output could be:
{
"Instruction": "Follow the lane for 2 kilometers until you reach the gas station.",
"Action": "LaneFollow",
"Distance": "2 kilometers"
}
We validated smooth action transitions in Town01 using PID control for proof of concept. As shown in the following video.
The system includes a function to measure the distance traveled, which is essential for determining when to transition from one action to another. The function uses the Euclidean distance formula to calculate the distance between two points:
def find_dist(loc1, loc2):
return math.sqrt((loc2.x - loc1.x)**2 + (loc2.y - loc1.y)**2)
find_dist function computes the distance in CARLA’s 2D plane (X, Y) between the starting position (loc1) and the vehicle’s current position (loc2).dist_travelled by comparing the start and current locations, adjusting the vehicle’s controls based on whether the required distance has been reached.Example application:
start_location = vehicle.get_location()
current_location = vehicle.get_location()
dist_travelled = find_dist(start_location, current_location) # Calculate the distance traveled
In the drive_forward function, this logic ensures the vehicle halts upon reaching the specified distance:
if dist_travelled >= distance:
control = carla.VehicleControl(throttle=0.0, brake=1.0) # Stop the vehicle
vehicle.apply_control(control)
break
In the Town01 map, we tested whether the real-time distance measurement module was functioning correctly. A video demonstration is provided below.