Why is the code for testing autonomous driving different from the code for testing normal internet products?

I often hear this question recently: “How much testing does it take to get on the road?”

The code for testing autonomous driving is different from the code for testing normal internet products. As long as the code of the Internet product achieves the target function, it can be released. For example, mobile APP, as long as there is no obstacle for users to use it, it is a good code.

And driverless cars are different. There are problems in the code, not just a bug. Only a small fraction of the problems in the code are “known unknown”, which are expected problems. Most are “unknown unknown”, meaning unforeseen problems. If the problem is not discovered in time and brought to the road test, it will pose a threat to public safety.

Uber test car accident scene in 2018

In theory, the more thorough and careful the testing, the better. In reality, we often do not have enough time or resources to do all the tests, or the tools used for testing are not mature enough. Therefore, engineers often have to decide what trade-offs should be made under limited conditions.

In fact, the test code is just for two goals:

1. Find the underlying problem.

2. Effectively dig out the root cause of the problem.

For the first goal, we first look at whether the various levels of testing are comprehensive. Testing of autonomous driving is varied. First, engineers do their part in testing. From the first few engineers getting together for design review, to basic unit tests, to component-level tests, engineers must at least ensure that the few lines of code they write do not Something went wrong.

After basic testing is complete, the next step is to ensure that the code is compatible with other components. For example, engineers working on lidar models need to ensure that their code does not affect other sensors. At this time, you need to run the entire stack again, or hardware in the loop, and test other hardware systems together to see if there are compatibility problems, so as to achieve “continuous integration”. The specific method can refer to the V&V model.

There are also many ways to test. In addition to running code locally, the most important thing for autonomous driving is simulation. A powerful simulation platform can replace the drive test to a certain extent. Through simulation technology, not only can the existing driving data (log) be replayed, but also new scenarios can be created, and various parameters can be defined by themselves, so that the limited data can be used to its maximum effect in a short period of time.

After the simulation test, the code can be put on the car, tested in a closed course (closed course), and finally tested on the open road.

There are various ways of testing, but in general, the lower the level of testing, the lower the cost. If you wait until the road test to discover the problem, the cost is very high.

The reason is simple: the lower the level of testing, the easier it is to pinpoint the source of the problem. The higher the level of testing, the wider the parts involved. Once a problem is found, it will be difficult to troubleshoot.

Therefore, the underlying test design is particularly important. A test object can be a newly developed driving behavior or an improvement to an existing function. If it is an improvement to an existing function, it is necessary to quantify all the details into metrics. Once the metrics change, or “regression”, such as detecting a cyclist as a pedestrian, it is necessary to analyze its reason. So as to make every potential problem “rooted”.

If you are developing new driving functions, you can use the simulation platform to create the required scenarios, estimate possible problems, and then design corresponding indicators for each potential problem, so as to “prevent problems before they occur.”