STEMMER IMAGING

There is a hint of Formula 1 in the air watching the Formula Student Racing Team, members of the Munich University of Applied Sciences, diligently working together to develop and optimise their driverless racing car. Since 2005, students from various study programmes including automotive, mechanical and business engineering as well as computer science, design, business administration and many more, have been preparing for their future professional life in a fun, hands-on challenge.

“There are currently around 120 students and many of us are investing a great deal of time and effort in this project”, explains Timo Socher, a computer science student who has already been involved with the project for three years. In his role as CTO Driverless for the current season, he is responsible for all technical aspects of the driverless racing car that municHMotorsport is sending to the race in 2020. At the end of the project, which is organised completely independently, a six-figure sum will have been invested in this racer, which will navigate autonomously on a race track at speeds of up to 65 km/h.

Lap times and top speed are not the only criteria for the assessment of the cars in the battle for the winner's podium (see text box). However, the new developments have to prove their skills on the race track in three dynamic disciplines. The first one evaluates acceleration on a straight, 75-metre track from zero to a complete stop. In the second task, the so-called skidpad, the cars race on a well-known figure-eight circuit. The top discipline is the track drive: In this test, the cars have to complete ten laps on an unknown circuit that is up to 500 metres in length.

The course of all three disciplines is marked by blue and yellow traffic cones, which are placed at maximum distances of 5 metres apart, left and right along the boundary lines. "The autonomous vehicles follow these markings using the latest hardware and software technologies. They control acceleration, braking and steering movements of the racing cars to optimise the entire system for the best possible driving performance," explains Timo Socher.

 

Machine vision is one of the key components in our system and plays a decisive role in detecting the cones. As a sensor, it provides the subsequent evaluation systems with the basic data for the vehicle’s reactions. Hence, the quality of the environment and cone detection forms the basis for all other vehicle modules and is essential for the entire system’s stability and performance.

Timo Socher, CTO Driverless, municHMotorsport

In their current racing car, the Munich students have integrated two vision systems forming the base for the track detection. One vision system is attached to the safety bar and detects the cones at greater distances at a range of five to 20 metres. For the upcoming season, this range is to be extended to up to 30 metres in order to have more calculation time for the required measurements and to be able to react even more proactively. The second system consists of two Intel RealSense cameras mounted under the front nose of the racing car, each with overlapping angles of 80 degrees. They ensure the recording of the images to evaluate the boundary markers at close distances of between 1.5 and 8 metres.

The image data is transferred to the CPU located at the racer’s rear and processed by a Jetson Xavier embedded computer in combination with other sensor data. The system uses technologies such as the merged sensor data and visual odometry which calculates the vehicle’s position and orientation by analysing the recorded images. This technology is often used in industry as the basis for robot positioning –further proof of the close connection between the municHMotorsport project and real industrial applications.

A racing car should, of course, be as fast as possible. For this reason, the images from the camera systems must be quickly and efficiently processed in the CPU. The purpose of machine vision is to locate and classify the coloured cones in the colour images and to estimate their position in relation to the camera. Object recognition is hampered by a number of unpredictable factors such as weather and light conditions, or the actual state of the race track, which can deviate from the ideal state due to potholes, unevenness or the elevation profile. Also, the background may vary from race to race due to spectator stands and other objects.

In order to be optimally prepared for all the eventualities that may arise in terms of object recognition, the team uses state-of-the-art techniques, explains Socher: "There is a large number of algorithms and neural networks for object recognition and classification in colour images. In addition, there are of course, still more conventional imaging methods, such as edge detection using special colour filters. However, we chose a deep learning approach, which proved to be particularly robust to different weather and environmental conditions and therefore promised the best results for our purpose."

Timo Socher's colleagues took the required images with traffic cones in front of different backgrounds and environments under different conditions and further enriched the training data sets using data augmentation techniques. The main idea behind this approach: The more various training data is available, the better the accuracy of the trained models. Using data augmentation techniques, existing images are easily modified by software, for example, by adding random pixel values, softening images, slightly rotating them or changing contrasts to generate a larger number of training images. “In addition, we exchanged training data with other teams in the competition and used about 3,000 images from the so-called KITTI data set, which provides images of roads without traffic cones”, Timo Socher explains.

The entire training data set now includes several thousand images and serves as the basis for the simulations used by the student team to save time in optimising the autonomous racing car.

Initially, machine vision was unknown territory for many students of the Formula Student Racing team at the Munich University of Applied Sciences.

 

In order to gain more expertise, some of us attended training courses at the renowned STEMMER IMAGING European Imaging Academy (EIA). The training was particularly helpful for new project members to better understand the big picture and to quickly become familiar with the necessary machine vision basics.

Timo Socher, CTO Driverless, munichMotorsport

The European Imaging Academy offers a wide range of hands-on training courses, videos and events around machine vision. Beginners are introduced to the basics, while advanced users gain a more in-depth insight and useful tips and tricks to improve the efficiency of their solutions. Highly skilled experts conduct the training sessions in STEMMER IMAGING's customer centres throughout Europe. STEMMER IMAGING CTO Martin Kersting comments: “Formula Student combines theory with hands-on experience. Acquiring skills and key competencies such as interdisciplinary thinking, problem solving and business management know-how is excellently put into practice.”

The Munich University of Applied Sciences and the machine vision experts share a long-standing close relationship, the reason why STEMMER IMAGING has become a sponsor of municHMotorsport. "When developing the vision systems in our racing cars, we were able to benefit from many years of experience in the industry while being provided with powerful machine vision components such as 3D stereoscopic cameras", enthuses Timo Socher. These high-speed cameras are ideally suited for use as the eyes of autonomous racing cars due to their compact, robust design and great depth of field.

STEMMER IMAGING’s practical expertise also paid off when it came to transferring the image data recorded by both camera systems to the race car’s CPU: The cabling had to meet special requirements, as it was laid through the entire vehicle and partly alongside the power electronics. "For this reason, STEMMER IMAGING produced special EMC shielded cables for our racers, which was a great help to us".

For the coming racing season, the students are working flat out to develop a competitive vehicle that will represent their team, the Munich University of Applied Sciences and their sponsors in the best possible way. “We want to rank in the top three this year with our autonomous racer“, says Timo Socher, setting out the team’s goal. In any case, the team’s ambition and the machine vision systems used offer the best conditions for the winner’s podium.