Imaging systems for sports tracking
Machine vision technology does not only achieve extraordinary results in industrial applications, it is also very useful in other areas. The following article shows some examples from the sporting world.
Flight Club Darts’ recent opening at a new venue in Chicago indicates the growing popularity of social darts. This is the third venue, with two more, Manchester and London Victoria opening before the end of 2018. At the heart of the experience for customers is an automated darts scoring system, which is powered by a highly sophisticated 3D vision system. The vision system automatically tracks the darts, recording where on the dartboard that they land and immediately displaying the score, removing the need for a chalkboard and calculator!
With billions of calculations completed in just a fraction of a second, the system is so fast and seamless that players never have to wait for scores to be computed no matter how fast they can throw! Cheating is impossible as darts cannot be hand placed quickly enough to simulate the visual impact of a thrown dart.
Ball tracking technology
STEMMER IMAGING has been involved in ball tracking technology for many years, having worked closely with Hawk-Eye Innovations to deliver imaging technology for the decision review systems used internationally in sports such as cricket and tennis and for goal line technology in football. These types of system have been approved by many of the leading authorities in world sport and are very familiar to TV viewers, having become an essential part of sports broadcasting.
Multiple cameras located around the stadia produce individual images of the ball in motion. These are triangulated using sophisticated software to give an accurate 3D trajectory of the ball in real time. Combined with on screen graphics, the systems are used in tennis to review line decisions and in cricket to check leg before wicket (LBW) decisions as part of a player review system.
For goal line technology, the system is continuously monitoring images from 7 cameras focused on each goal. The images are processed to find the ball within the image and also identify areas, which are definitely not the ball. Control software combines the information from all cameras and is able to track the ball within the goal area. As soon as the system detects that the ball has crossed the goal line, it instantaneously sends a signal to the officials’ watch.
Real-time reaction to ball tracking
One of the most impressive examples of high-speed ball tracking comes in the form of RoboKeeper’ an automated goalkeeper. RoboKeeper was developed by the Fraunhofer Institute of Material Flow and Logistics in Dortmund, Germany and is marketed by 4attention, a sports marketing agency, as a visitor attraction at major events.
Visitors are invited to take a penalty from around 10 yards from the goal instead of the usual 12 yards against RoboKeeper who is guarding a goal measuring 2.0 x 4.0 metres. The flight of the ball is tracked using a camera system from STEMMER IMAGING consisting of two colour area scan cameras with a Gigabit Ethernet interface, which are mounted above and to the sides of the goal. Images at up to 90 frames per second are subsequently evaluated by imaging software developed by Fraunhofer IML.
If the ball were kicked at around 62 mph in a flat trajectory, it would take 360 ms to reach the bottom left or right corner of the goal. This position represents the maximum distance RoboKeeper has to move. Within this time, the probable trajectory of the ball and the position where it will reach the plane of the goal must be extrapolated and the complete movement of the goalkeeper, including all motor acceleration and braking procedures, must be completed in order to prevent a goal.
The system allows RoboKeeper to move to that position within 300 ms – making it rather difficult to score! To compensate for variations in lighting conditions for outside events, the camera systems are equipped with lenses with an automatic iris controller and a motorised iris controlled by the video signal. These lenses have a focal length of 3.5 mm, so the angle is wide enough to guarantee recognition of the ball from the penalty spot to the goal.
There is also an ice hockey version of RoboKeeper, which utilises cameras with a higher resolution and a faster frame rate. This allows for the fact that not only is the puck is smaller and faster than a football and not spherical, but it also rotates in the air during flight.
Sports coaching
Ball tracking is also an important part of sports coaching as well in officiating and is often used in conjunction with biomechanics. STEMMER IMAGING has worked closely with Quintic Consultancy Ltd during the development of the Quintic Ball Roll system for analysing putting performance in golf. Quintic is a specialist in Premier Sports Video Analysis Software, Sports Biomechanics & Performance Analysis Consultancy. The system utilises a high-speed camera (360 – 1080 frames per second) in combination with a custom-designed LED bar light to track the putter and golf ball throughout the impact zone of either a right handed or left handed player.
STEMMER IMAGING not only supplies many of the cameras used by Quintic, but has also worked closely with them to ensure successful integration of the cameras with the software. The system automatically measures over 45 parameters regarding the putter and ball, both graphically and numerically. The data is instantly displayed within the software for immediate analysis, review and intervention. It can be used for golf instruction, club fitting, club recommendation, coaching and custom putter fitting, and as a golf training aid for personal practice by professionals and amateur golfers alike.
The Quintic Ball Roll system v.3.4 captures images from a USB3 camera at 360 fps. Image Source: Quintic
Player tracking
Although ball tracking has an important role to play in professional sport, the tracking of players and other parameters during a football match is a key part of the overall analysis of the match for coaching and broadcast purposes. STEMMER IMAGING has been involved in the development of a player tracking system that has been used in professional football matches. Statistics on each player’s activities during a match can be captured live, using different camera configurations.
Between 2-10 cameras from different providers can be positioned around a stadium to achieve those statistical results. A combination of multicast imaging and intelligent image processing on multiple computers is used to monitor position coordinates for every player, the ball and the referee at all times during the game to allow calculation of players’ total running performance including average and maximum speeds, number and intensity of sprints and the distance covered.
Camera technology
In each sport, the challenge for the imaging system is similar – the choice of cameras, lenses and data transmission standards must meet the needs of the application and requires expert knowledge and understanding of the imaging requirements. The camera technology must be capable of delivering images at the required resolution and speed (frame rate) over the long distances from the camera positions to the control room for processing. In many cases, the ball is travelling at very high speeds. Cricket balls can be bowled at speeds approaching 100 mph, while tennis serves can exceed 160 mph.
In addition, as most of the sports are conducted outside, the cameras must be able to accommodate varying illumination conditions. The entire processing time for these systems must be short enough not to significantly disrupt the flow of play in a match. Typically the trajectory of the ball can be displayed within a few seconds of the call for a review. In the case of goal-line technology, the system automatically alerts the match officials if the ball crosses the goal line. The ball tracking systems provide millimetre positional accuracy. STEMMER IMAGING has proven expertise and experience to provide the camera and imaging technology to meet these challenging requirements.