Skip to content

Commit

Permalink
planning: add planning readme
Browse files Browse the repository at this point in the history
  • Loading branch information
lilu15 committed Sep 6, 2021
1 parent 0e9196d commit 9379c9c
Show file tree
Hide file tree
Showing 8 changed files with 24 additions and 259 deletions.
209 changes: 12 additions & 197 deletions modules/planning/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,204 +2,19 @@

## Introduction

Prior to the 5.5 release, Apollo uses the same configuration and parameters to
plan different driving scenarios. Although linear and easy to implement, this
approach was not flexible or scenario specific. As Apollo matures and takes on
different road conditions and driving use cases, we felt the need to move to a
more modular, scenario specific and wholistic approach for planning its
trajectory. In this approach, each driving use case is treated as a different
driving scenario. This is useful because an issue now reported in a particular
scenario can be fixed without affecting the working of other scenarios as
opposed to the previous versions, wherein an issue fix affected other driving
use cases as they were all treated as a single driving scenario.
Compared with previous versions, Apollo 7.0 adds a new dead end scenario, adds the "three-point turn", increases vehicle driving in and out ability, and expands the operation boundary of the urban road network. The "three-point turn" function is based on the open space planner framework and includes the following parts: scene conversion of dead ends, construction of open space ROI, and "three-point turn" trajectory planning.

Apollo 5.5, which focuses on curb-to-curb autonomous driving on urban roads,
introduced 2 new planning scenarios.
## Input
Localization, Perception, Prediction, HD Map (in modules/map/data), routing, task_manager.

Apollo 6.0 extended the technology to incorporate data-driven mothedologies to tackle trajectory planning problems with learning-based models, and introduced two new planning modes: E2E mode and Hybrid mode, by which the new capability of dynamically nudginng moving obstacles are demonstrated. In these two modes, a series of APIs is also defined where developers can generate their own training data and integrate their own models.
## Output
A collision-free and comfortable trajectory for control module to execute.

**Note:** The current development of E2E mode is in an early stage where the model is trained for the dynamic nudge scenario as a research demonstration purpose. The capability of the model is limited, and suboptimality is expected when it is tested with a wider selection of scenarios. E2E mode and Hybrid mode are not tested on real roads yet, but rather serve as a baseline to promote and encourage extensive research on learning based planning. All developers are welcome to collaborate with us by any means including algorithms, models and data.
### Planning Scenario Architecture
![](images/DeadEndScenario.png)

## E2E Mode
![](images/e2e_mode.png)
### How to Enable
- Change the configuration "learning_mode" in apollo/modules/planning/conf/planning_config.pb.txt to be "E2E_TEST" if Apollo is run in simulation or "E2E" on real vehicle
- Change the configuration "model_type" in apollo/modules/planning/conf/scenario/learning_model_sample_config.pb.txt to be either "CNN_LSTM" or "CNN" and adapt the following "cpu_model_file" and "gpu_model_file" file paths. "CNN_LSTM" is the preferred model for now.

### Model Inputs and Outputs
- Model input consists of a birdview image centered by vehicle pose and vehicle current velocity.
- Model output is planning trajectory

## Hybrid Mode
![](images/hybrid_mode.png)
### How to Enable
In configuration file,
```
./modules/planning/conf/planning_config.pb.txt
```
set learning_mode: `learning_mode: ` as `HYBRID` for road test or `HYBRID_TEST` for simulation.

### Parameters
The configurable parameters in hybrid model are listed in the configuration file
```
modules/planning/conf/scenario/lane_follow_hybrid_config.pb.txt
```
The parameter `path_reference_l_weight` is for adjusting hybrid model path output. A larger value of `path_reference_l_weight` means higher penalty of the difference between hybrid model path and learning model path in lateral direction.

## Apollo 5.5 vs E2E Mode vs Hybrid Mode

We demonstrate simulation results on a dynamic nudge scenario with Apollo 5.5, E2E mode, and Hybrid mode.

- Apollo 5.5

![](images/sim_rule.gif)

- E2E Mode

![](images/sim_e2e.gif)

- Hybrid Mode

![](images/sim_hybrid.gif)

## Driving Scenarios

There are 5 main driving scenarios that we will focus on Lane Follow, Intersection, Pull-Over, and the newly introduced Park-and-go and Emergency. Let's dive into them individually:

### Lane Follow - Default

As seen in the figure below, the lane-follow scenario, our default driving scenario, includes but is not limited to driving in a single lane (like cruising) or changing lane, following basic traffic convention or basic turning.

![](images/planning_default.png)

> Note: Side Pass
>
> > While the functionality of side pass still exists, it has now been made universal rather than limiting it to a type of scenario. The side-pass feature is incorporated as part of the path-bounds decider task. You can choose to turn it on or off by properly configuring the path-lane-borrow decider task. For example, if you want the vehicle to be agile, then turn side-pass on for all scenarios; if you feel it not safe to side-pass in intersections, then turn it off for those related scenarios.
### Intersection

The new intersection scenario includes STOP Signs, Traffic Lights and Bare Intersections which do not have either a light or a sign.

#### STOP Sign

There are two separate driving scenarios for STOP signs:

- **Unprotected**: In this scenario, the car is expected to navigate through a crossroad having a two-way STOP. Our ADC therefore has to creep through and gauge the crossroad's traffic density before continuing onto its path.

![](images/unprotected1.png)

- **Protected**: In this scenario, the car is expected to navigate through a crossroad having a four-way STOP. Our ADC will have to gauge the cars that come to a STOP before it and understand its position in the queue before moving.

![](images/protected.png)

In order to safely pass through a STOP sign, both protected and unprotected, the following steps are performed:

- Arriving at the STOP sign: Perceive all other cars or obstacles that are currently waiting at the other stop signs

- Come to a complete stop: Monitor to see if the cars that were previously stationary at other STOP signs have moved or not. It is essential that the cars that arrived before have all left

- Move forward slightly (Creep): Check to see if any other car is moving or in the case of unprotected stop, check to see if there are any oncoming vehicles on either side of the lane

- Safely move through the crossroad

> Note:
>
> > The team is working to add additional driving scenarios into our planner. One such example is handling Traffic Lights.
#### Traffic Light

In order to safely and smoothly pass through a traffic light, we created 3 driving scenarios:

- **Protected**: In this scenario, our ego car has to pass through an intersection with a clear traffic light indicator. A left arrow or right arrow in green for the corresponding turn.

- **Unprotected Left**: In this scenario, our ego car will have to make a left turn without a distinct light, meaning the car would need to yield to oncoming traffic. Just like in the unprotected STOP scenario, our ego car would have to creep to ensure that it is safe to cross the intersection before safely moving through the lane.

- **Unprotected Right**: In this scenario, our ego car is expected to make an unprotected right turn while yielding to oncoming traffic. Our ego car will need to creep slowly and gauge the traffic and then make a safe turn.

As discussed above, based on the three driving scenarios, the following 3 steps are performed:

- **Stop/Approach**: If a stop is required, our ego car will stop in front of traffic light stop line

- **Move forward slightly (Creep)**: Check to see if any other car is moving or in the case of unprotected turns, check to see if there are any oncoming vehicles on either side of the lane

- **Move**: Safely drive through the intersection

#### Bare Intersection

Bare intersection is a scenario designated to an intersection without either a STOP sign or a traffic light. In this scenario, the following steps are performed:

- **Approach**: Reach the intersection

- **Move forward slightly (Creep)**: Check to see if any other car is moving or in the case of unprotected stop, check to see if there are any oncoming vehicles on either side of the lane

- **Move**: Safely move through the intersection

### Park

The Apollo team is proud to introduce Open Space Planner, a new planning algorithm that can be used for several use cases especially the parking scenario. To learn more about Open Space Planner, please refer to [Open Space Planner Algorithm](https://github.com/ApolloAuto/apollo/blob/master/docs/specs/Open_Space_Planner.md)

#### Valet

The Valet scenario was designed to safely park your ego car in a targeted parking spot. There are 2 main stages that are performed in order to park your ego car:

1. **Approaching the Parking Spot**: In this stage, standard planning algorithms are used to approach the designated parking spot. The car will gradually cruise to a halt once it has found the right stopping point required in order to reverse into the parking spot as seen in the image below

![](images/parking2.png)

2. **Parking**: In this stage, Open Space Planner algorithm is used to generate a zig-zag trajectory which involves both forward and reverse driving (gear changes) in order to safely park the ego car. A sample trajectory generated for the scenario in the previous image, can be seen below:

![](images/parking1.png)

#### Pull Over

The Pull Over scenario was designed especially for maneuvering to the side of the road upon reaching your destination like for curb-side parallel parking. There are 3 main stages to accomplish the pull over scenario.

1. **Approaching**: In this stage, as there is no clear designated parking spot, the ego car simply approaches the side of the road where it seems feasible to park. Standard Planning algorithms are used for this stage. Once it comes to a halt, the second stage begins. An example of stage 1 can be seen in the image below:

![](images/pull_over.png)

2. **Retry Approach Parking**: In this stage the ego car adjusts itself to enter the parking spot. It is similar to the `Approach Parking Spot` case in the Valet scenario. An example of stage 2 can be seen in the image below:

![](images/pull_over1.png)

3. **Retry Parking**: This stage uses Open Space Planner to parallel park the vehicle. A zig-zag trajectory is generated to help the ego car park itself on the side of the road. A sample trajectory generated for the scenario in the previous image, can be seen below:

![](images/pull_over2.png)

There is a special case in the Pull Over scenario that does not need the Open Space Planner stage. This case occurs when there are no obstacles blocking the curb-side parking of the ego car. An example can be seen below. In such a case, the car simply approaches the parking spot and then enters the spot using standard planning.

![](images/pull_over3.png)

### Park-and-go

Apollo 5.5 which focuses on curb-to-curb driving introduces the Park-and-go scenario which was designed to handle curb side parking, planning a new trajectory to the next destination and then driving along that trajectory. This scenario is extremely useful in situations like curb-side delivery or passenger pickup or drop-off. This scenario combines Open Space Planner along with other traditional trajectory planners to ensure that the car not only parks safely but is also able to exit per the new trajectory.

![](images/park_and_go.png)

## Emergency

The Emergency scenario is another newly introduced scenario in Apollo 5.5, developed to ensure that the vehicle can be brought to a stop safely in the situation of a given trigger event (either via human input or due to a failure in one or more of the vehicle's hardware or software modules). This scenario is extremely crucial to urban driving as one often encounters several unforeseen situations on the road that requires the vehicle to come to a complete stop. There are two types of emergency stops that are performed using this scenario:

1. **Pull Over and Stop**: In this scenario, the autonomous car uses Open Space Planner to pull-over and then come to a full stop on the curb-side

2. **In Lane Stop**: In this scenario, the autonomous car comes to a complete stop in its lane. In this situation the car is well aware of its surrounding obstacles to prevent any rear-end collisions and ensure not just the safety of itself but also the vehicles around it.

![](images/emergency.png)

## Planning Module Architecture

In Apollo 5.5, the Planning module architecture has been modified to incorporate new curb-to-curb driving scenarios on urban roads. As seen in the figure below, we have 2 new complex scenarios Emergency and Park-and-go. In order to plan these scenarios effectively, we have 2 new Deciders - Path Reuse Decider and Speed Bound Decider and have updated existing deciders making the planning architecture robust and flexible enough to handle many different types of urban road driving scenarios.

Each driving scenario has its set of driving parameters that are unique to that scenario making it safer, efficient, easier to customize and debug and more flexible.

> Note:
>
> > If you wish to include your own driving scenarios, please refer to existing scenarios as a reference. We currently do not have a template for writing your own planning scenario.
![](images/architecture_5.5.png)


## Related Paper

1. [He R, Zhou J, Jiang S, Wang Y, Tao J, Song S, Hu J, Miao J, Luo Q. "TDR-OBCA: A Reliable Planner for Autonomous Driving in Free-Space Environment." *arXiv preprint arXiv:2009.11345.* ](https://arxiv.org/pdf/2009.11345.pdf)
2. [Zhou J, He R, Wang Y, Jiang S, Zhu Z, Hu J, Miao J, Luo Q. "DL-IAPS and PJSO: A Path/Speed Decoupled Trajectory Optimization and its Application in Autonomous Driving." *arXiv preprint arXiv:2009.11135.*](https://arxiv.org/pdf/2009.11135.pdf)
## Scenario Shows
![](images/EnterDeadEnd.png)
![](images/TuringAround1.png)
![](images/TuringAround2.png)
![](images/OutDeadEnd.png)
74 changes: 12 additions & 62 deletions modules/planning/README_cn.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,70 +2,20 @@

## 介绍

Apollo的之前版本,包含3.0都是用了相同的配置和参数规划不同的场景,这种方法虽然线性且实现简单,但不够灵活或用于特定场景。随着Apollo的成熟并承担不同的道路条件和驾驶用例,我们认为有必要采用更加模块化、适用于特定场景和整体的方法来规划其轨迹。
Apollo7.0相对之前的版本,本次新增了断头路场景,增加了"三点掉头"功能,增加了驶入驶出的能力,扩展了城市路网运营边界。"三点掉头"功能
基于open space planner框架,包含以下几个部分:断头路场景转换、开放空间ROI构建、掉头轨迹规划。

在这个方法中,每个驾驶用例都被当作不同的驾驶场景。这样做非常有用,因为与先前的版本相比,现在在特定场景中报告的问题可以在不影响其他场景的工作的情况下得到修复,其中问题修复影响其他驾驶用例,因为它们都被当作单个驾驶场景处理。
## 输入
输入信息:定位、感知、预测、高精地图、路由、任务管理模块。

## 驾驶场景

Apollo3.5主要聚焦在三个主要驾驶场景,即:
### 车道保持 - 默认

如下图所示,车道保持场景(我们的默认驾驶场景)包括但不限于在单车道(如巡航)或换道行驶,遵循基本的交通约定或基本转弯。

![](images/planning_default.png)


### Side Pass

在这种情况下,如果在自动驾驶车辆(ADC)的车道上有静态车辆或静态障碍物,并且车辆不能在不接触障碍物的情况下安全地通过车道,则执行以下策略:
- 检查邻近车道是否接近通行
- 如果无车辆,进行绕行,绕过当前车道进入邻道
- 一旦障碍物安全通过,回到原车道上

![](images/sidepass.png)


### 停止标识

停止标识有两种分离的驾驶场景:
1、未保护:在这种情况下,汽车预计会通过具有双向停车位的十字路口。因此,我们的ADC必须爬过并测量十字路口的交通密度,然后才能继续走上它的道路。

![](images/unprotected1.png)

2、受保护:在此场景中,汽车预期通过具有四向停车位的十字路口导航。我们的ADC将必须对在它之前停下来的汽车进行测量,并在移动之前了解它在队列中的位置。

![](images/protected.png)

为了安全地通过受保护和未受保护的停止标志,执行逻辑如下:
- 到达停车标志处:感知正等待在其他停车标志处的所有汽车或障碍物;
- 完全停止:Monitor检查之前停在其他停车标志处的汽车是否已经移动。以前到达的汽车必须全部离开;
- 稍微向前移动(爬行):检查是否有其他车辆正在移动或在无保护停车的情况下,检查车道两侧是否有迎面而来的车辆
- 安全通过十字路口

```
Note:
Apollo团队正在努力增加额外的驾驶场景到规划器。其中一个例子是处理交通灯。
```
## 输出
输出信息:控制模块可执行的顺滑无碰撞轨迹。

## Planning 模块架构
![](images/DeadEndScenario.png)

Apollo 3.5 Planning模块的架构已经改变,以反映我们针对不同驾驶场景的模块化方法。

如下图所示,在规划器中,是上面讨论的各个驾驶场景及其处理逻辑。

每个驾驶场景都有其独特的驾驶参数集,这些参数使该场景更安全、高效、更易于定制和调试,并且更加灵活。每个阶段可配置的,且被划分为 **任务**,并且可以通过编辑该场景的`config`文件来移动或创建每个任务。

部分重要特性包含如下:
- Apollo FSM(finite state machine):一个有限状态机,与高清地图确定车辆状态给定其位置和路线。
- Planning Dispatcher: 根据车辆的状态和其他相关信息,调用合适的Planner
- Planner:获取所需的上下文数据和其他信息,确定相应的车辆意图,执行该意图所需的规划任务并生成规划轨迹。它还将更新未来作业的上下文。
- Deciders & Optimizers :一组实现决策任务和各种优化的无状态库。优化器特别优化车辆的轨迹和速度。决策者是基于规则的分类决策者,他们建议何时换车道、何时停车、何时爬行(慢速行进)或爬行何时完成。
- 黄色框:这些框被包含在未来的场景和/或开发人员中,以便基于现实世界的驱动用例贡献他们自己的场景

```
Note:
如果您希望包括您自己的驾驶场景,请参考现有方案作为参考。我们目前没有用于编写您自己的规划场景的模板。
```

![](images/architecture.png)
## 场景展示
![](images/EnterDeadEnd.png)
![](images/TuringAround1.png)
![](images/TuringAround2.png)
![](images/OutDeadEnd.png)
Binary file added modules/planning/images/.DS_Store
Binary file not shown.
Binary file added modules/planning/images/DeadEndScenario.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added modules/planning/images/EnterDeadEnd.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added modules/planning/images/OutDeadEnd.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added modules/planning/images/TuringAround1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added modules/planning/images/TuringAround2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 9379c9c

Please sign in to comment.