Machine vision applications -  Successful examples from many industries


Successful examples from many industries

One of the best penalty-stopping goalkeepers

Numerous international football stars such as Messi, Neymar and others have already tried to beat this goalkeeper from the penalty spot – mostly without success. Standing between the goalposts is not a human being, but RoboKeeper – probably the world's fastest goalkeeper.

A whistle and the referee points to the penalty spot – this situation has been decisive in countless football matches and is not for the faint hearted. RoboKeeper, developed by the Fraunhofer Institute of Material Flow and Logistics (IML) in Dortmund, has no nerves and is usually the victor when football players of all levels attempt to shoot the ball past him into the goal.

The self-chosen boundary conditions of the project, which was started in 2005, differ from those of a real football match: The penalty taker shoots from around 9 metres from the goal instead of 11 metres and the goal measures 2.0 x 4.0 metres. It is guarded by RoboKeeper, a technical system designed to meet the highest requirements, as Thomas Albrecht, research scientist at Fraunhofer IML and RoboKeeper development project manager, explains: "The aim of the development was to stop the ball reliably, even when it is kicked with a speed of 100 km/h and a flat trajectory into the bottom left or right corner of the goal. That corresponds to the maximum distance for the goalkeeper, who is standing upright in the centre of the goal when the ball is kicked. The pure flight time of the ball at this speed is about 360 ms or a rather a little more, since the ball cannot be accelerated instantly from 0 to 100 km/h. During this time the probable trajectory of the ball and the position where it will reach the plane of the goal must be extrapolated and the complete movement of the goalkeeper, including all motor acceleration and braking procedures, must be completed in order to prevent a goal."

According to Albrecht the required accuracy lies within the range of a few centimetres, since it is not crucial for the goalkeeper figure to stop the ball centrally: "RoboKeeper only needs to meet the ball in such a way that it is deflected away and doesn't end up in the goal."

RoboKeeper requires only about 300 ms to reach the calculated goal-saving position – a shorter time even than a ball kicked hard.
Image source: 4attention

The penalty taker shoots from about 9 metres from the goal, which measures 2.0 x 4.0 metres.
Image source: 4attention

Fast image processing

In order to be able to satisfy these boundary conditions, the developers from Fraunhofer IML made use of a fast imaging system that serves as RoboKeeper's eyes. Two Allied Vision Prosilica GC655C colour area scan cameras with a Gigabit Ethernet interface and a resolution of 659 x 493 pixels with a Gigabit Ethernet interface and a resolution of 659 x 493 pixels, which are mounted above and to the sides of the goal, track the flight of the ball, which must differ in colour from its surroundings.

They each supply up to 90 frames per second, which are subsequently evaluated by imaging software developed by Fraunhofer IML that runs on a dual-core processor system from Kontron. Three images that capture the ball's movement towards the goal are analysed and the results are transmitted within the required time to the motor controller, which positions the goalkeeper to "save" it.

On the basis of these calculations and with the help of a high-performance gearbox, RoboKeeper only needs about 300 ms to reach the calculated goal-saving position – even quicker than a ball that’s been kicked hard. In doing so it accelerates around 20 times faster than a Formula 1 racing car and thus has the potential to reduce even experienced professional footballers to despair. So that the fun factor remains, however, the system can be set to seven different levels of difficulty and thus adapted to the player's performance level. Every participant can have a real chance of beating RoboKeeper.

"The main challenge is that when assembling the system outdoors the sunlight and therefore the illumination intensity of the scene vary a great deal, both over the course of the day and in the short term", says Timm Ulrich, one of the founders and managing director of the sports marketing agency 4attention, describing one of the decisive factors. This issue was overcome with camera systems that are equipped with an automatic iris controller and a motorised iris controlled by the video signal. The Goyo Optical CVO GAT23516AC lenses used in the system have a focal length of 3.5 mm, so the angle is wide enough to guarantee recognition of the ball from the penalty spot to the goal. Combining this lens and the Prosilica cameras was a crucial factor for the success of the entire system.

The perfect image processing setup for this application, consisting of the two cameras, the associated motorised lenses and the cables, was procured by Fraunhofer IML with the machine vision experts STEMMER IMAGING in Puchheim.

"Initially cameras from a competitor with a FireWire interface were installed in this system", Ulrich recalls. "However, they proved to be unsuitable for the harsh "on-the-road operation" due to mechanical and EMC problems. Market research into Gigabit cameras with motorised iris control then led us to the products from STEMMER IMAGING, which have proven to be ideal in the current setup."

Two Gigabit cameras are positioned above and to the sides of the goal and track the flight of the ball.
Image source: 4attention

On the road all over the world

The Cologne-based sports marketing agency 4attention has been using the RoboKeeper commercially since 2007 and hires out the 36 machines that it currently has all over the world, for example for the fringe programmes of sporting events. Fans can also try their luck on Red Square in Moscow during the 2018 World Cup. Apart from the system described, there is also a smaller version with a shortened shooting distance and smaller goal size as well as versions for field hockey, ice hockey and handball.

The challenges with these systems are different to those with the RoboKeeper, as Ulrich explains: "In the case of handball, the throwing distance is shorter, the ball speed is lower and the ball is somewhat smaller, but overall the conditions are comparable with the football system. The requirements are somewhat tougher with the field hockey version because the ball is much smaller. However, the ice hockey version has proven to be much more difficult for the vision system, because the puck is smaller and faster than a football or handball, in addition to which it is not spherical. Also, the position of the puck in flight is not usually stable – it rotates in the air. From the point of view of the cameras, therefore, the target object is changing continuously and has a varying size."

The developers at the Fraunhofer Institute IML successfully solved these higher requirements for the ice hockey system using cameras with a higher resolution and a faster frame rate, which were also supplied by STEMMER IMAGING. Instead of the Prosilica GC655C, two monochrome Prosilica GC780 Gigabit Ethernet cameras were used. These feature a higher resolution of 782 x 582 pixels and work with the same lens. If there were a world championship for penalty shootouts, the RoboKeeper system would certainly be the toughest challenge for the sportsmen and women in the sports mentioned.

Brief portrait

4attention4attention is a sports marketing agency. It was founded in 2002 and offers services to sport including live communication, sponsoring and entertainment. The marketing of the RoboKeepers has developed into the company's biggest building block.

Brief portrait

Fraunhofer IMLDas Fraunhofer IML in Dortmund is one of 66 institutes in the Fraunhofer-Gesellschaft and deals with all questions concerning logistics. Since automation technology is used in many research and development projects, the institute's employees have extensive know-how in the fields of fast (stereo) image processing, fast and high-precision drive technology and general control technology, which functions reliably even under difficult operating conditions.