Machine vision applications -  Successful examples from many industries

MACHINE VISION APPLICATIONS

Successful examples from many industries

Machine vision systems protect endangered birds

September 2019

On account of the increasingly strict wildlife conservation regulations, wind turbines have to be switched off when a protected species of bird approaches them. With the help of artificial intelligence and components from STEMMER IMAGING, a machine vision system from phil-vision banishes the danger of birds being struck by wind turbine blades and minimises expensive downtimes.

The wildlife conservation regulations for the construction and operation of wind turbines have been tightened enormously in the past years. For example, if bird species that are sensitive to wind power are present within a certain radius of wind turbines, the turbines generally have to be switched off for several days as soon as fieldwork such as ploughing, threshing or mowing takes place. During these periods the birds are at increased risk as they spend more time over these fields in the search for food.

Also, when protected birds nest close to existing turbines, these turbines may only be operated with restrictions. Therefore, whole-day shut-downs are often included in permits during the breeding period from March to August. However, every day on which a wind turbine doesn't produce electricity reduces its cost-effectiveness. Hence, despite the political will to expand the use of renewable energies compared to fossil fuels, the number of wind turbines being built has reduced sharply, in particular, due to the wildlife conservation regulations.

With the help of machine vision systems and artificial intelligence, this problem can now be solved in such a way that both the necessary cost-effectiveness for the wind farm operator and the needs of wildlife conservation are accounted for. In order to avoid long, expensive shut-down times in which there are often no animals in danger, Bürgerwindpark Hohenlohe GmbH is currently developing a system with the support of phil-vision GmbH with which large birds of prey can be recognised and localised with their flight paths tracked. The aim is to shut down wind turbines only when protected birds move within a certain distance of the turbines.

Together with phil-vision GmbH, Bürgerwindpark Hohenlohe GmbH is developing a system with which large birds of prey can be recognised and localised and their flight path tracked.
Image Source: phil-vision

Six industrial colour cameras with a resolution of 6 or 20 megapixels, which are fixed to the mast of a wind turbine in weatherproof protective enclosures, capture images in order to monitor the airspace around the wind turbine.
Image Source: phil-vision

Demanding task

According to phil-vision founder Gregor Philipiak, the recognition of birds and their classification for the subsequent decision as to whether the bird is a protected species is an extremely demanding task: "The fauna differs in every location of a wind turbine, so it's necessary to create individual protection concepts. With the first monitoring system, we initially elaborate on the specific requirements so that we can then design the target system to suit the individual turbine."

According to Philipiak the results so far have been very promising: "At present, we have about ten installed test systems." Such a system consists of six industrial colour cameras with a resolution of 6 or 20 megapixels, which are fixed to the mast of a wind turbine in weatherproof protective enclosures and capture images to monitor the complete airspace around the wind turbine."

To capture the entire surrounding area with a 360° all-round view, special wide-angle lenses were used so large fields of view can be surveyed.

The program tracks the flight paths of protected birds until they are no longer a threat to the operation of the wind turbine. Image Source: phil-vision

Hundreds of thousands of images

The evaluation of the captured images and the reliable recognition of endangered birds represent the actual skill in this application according to Philipiak: "Our system works on the basis of deep learning methods and for the learning phase it requires almost 400,000 images of birds of different types, at different distances and with various flight positions. Added to that there are about a further 100,000 images (per negative instance) such as insects, aircraft or helicopters as well as the captured images with the local special features."

From this large number of training images a powerful development system automatically creates a decision tree as well as a classifier, which serves as the basis for the differentiation in the subsequent usage phase.

"Through the use of deep learning methods we are thus creating an intelligent system that automatically detects the animals concerned against the most diverse backgrounds and under varying conditions and distinguishes them from possibly similar objects such as aircraft or flies", Philipiak explains.

For optimisation, the phil-vision system operates in two steps with different resolutions: First, all moving objects are detected with the help of 6-megapixel cameras. By switching to cameras with a 20-megapixel resolution, such objects can subsequently be identified with greater accuracy. The system thus determines whether a detected object is a bird at all and subsequently whether the animal belongs to a protected species.

The program tracks the flight paths of protected birds until they are no longer recognisable and thus no longer represent a potential threat to the operation of the wind turbine. If a protected bird gets closer to the wind turbine than the specified minimum distance, a corresponding signal is sent to the turbine's controller so that it can decelerate the turbine in good time.

 
For economic reasons, one of the priorities of a wind farm is to have as few false triggerings as possible. Through the use of powerful deep learning algorithms we have been able to fulfil this task better and better over the course of the programming and considerably reduce the number of false shut-downs.
Gregor Philipiak, Gründer, phil-vision

Ideas for the future

In its current version, the system has not yet reached the end of its possibilities, because the development team at phil-vision has further plans. One option for the future is the additional integration of a distance measurement, which is already being tested. This uses two cameras in stereo operation, as a result of which the desired depth information can be obtained. Prior calibration of all cameras ensures that manufacturing tolerances are eliminated and that the distance determined is correctly converted into metric data. This calibration is implemented with the aid of a drone, which flies to precisely defined points at different heights and distances. Via a marker, the image processing can subsequently detect the position of the drone in the captured image and store the defined distances for each position. The placing of the respective camera is determined so that precise data for approaching birds can be supplied as a result.

A further planned option is the distinction between the individual protected bird species by means of classification. Deep learning algorithms trained with large quantities of images of different birds are also used for this.

The creation of a database is planned for further evaluation. Collected values for bird occurrence at various locations are to be documented here. A web interface is to offer the possibility to retrieve and display the prepared data specifically for the respective wind farm or the individual wind turbine.

Experienced machine vision partner

phil-vision procures virtually all of the machine vision components required for this innovative system from STEMMER IMAGING. Above all Philipiak values the large selection of components available as well as the extensive know-how of the machine vision experts: "On account of the demanding application we had to test the most diverse camera and lens combinations during the development to find the optimum configuration. Our partner gave us outstanding support here. There were also special boundary conditions for the cables, which had to be resistant to UV light. STEMMER IMAGING manufactured them for us to the required specifications, which saved us a great deal of effort."

The PC technology for the systems also comes from STEMMER IMAGING. The software for evaluating the captured images was developed by phil-vision using the CVB Image Manager and the CVB Movie tool from the Common Vision Blox imaging library. The use of CVB and Open CV provides users with a high degree of flexibility so that, for example, other cameras can be used if necessary.

Wildlife conservation implemented cost-effectively

It seems successful deployment is not very far: In the first step a finished system for monitoring flight movements of protected bird species in the vicinity of wind turbines is to be used from spring 2020. In close collaboration with several biologists, protection concepts will then be drawn up on the basis of the respective (flight) behaviour of the birds, in accordance with which the machine vision systems will subsequently be designed and adapted to the specific boundary conditions. The goal should be to meet the wildlife conservation requirements and thus to be able to carry out the urgently expansion of renewable energies in the field of wind power.

Short portrait

phil-vision GmbHphil-vision GmbH has been offering solutions in all areas of machine vision since 2015. On the basis of its many years of technical know-how and with components from well-known manufacturers and distributors, the company has already developed a large number of systems that enable cost-effective solutions ranging from simple tasks with barcode and OCR recognition through to completeness checks, automated object recognition and high-precision 2D or 3D measuring systems.

CVB Image Manager
  • Platform for the development of image processing applications
  • Open programming environment
  • Simple to operate, combining flexibility with high performance
CVB Movie
  • Recording video sequences
  • Storage on hard disk
  • Use of installed multimedia codecs
STEMMER IMAGING

Puchheim, Germany

STEMMER IMAGING has been one of the leaders in the machine vision market since 1987. It is one of Europe's largest technology providers in this field. In 1997 STEMMER IMAGING presented Common Vision Blox (CVB), a powerful programming library for fast and reliable development and implementation of vision solutions, which has been deployed successfully throughout the world in more than 80,000 imaging applications in various industries.