High-end vision system for sorting bulk materials
Dust, temperature fluctuation, heavy objects, continuous load – vision systems used in mining operations need to withstand the toughest conditions. STEMMER IMAGING has developed such a system utilising the performance of FPGA components on frame grabbers. The technology employed is suited for sorting a host of different bulk materials.
The recently supplied vision system was developed for use in mining operations, where pieces of rock of varying sizes are transported at high speed along a 2 metre wide conveyor belt to the next processing station. The challenging task was to classify certain rocks during transport within a short space of time, to detect their geometric characteristics and their position on the conveyor belt, and to then discharge the found objects from the conveyor belt via ejectors.
In addition to the usual difficult conditions such as dust and changing light and temperature conditions, an aggravating factor was presented by the fact that the dark rock was difficult to distinguish from the dark colour of the conveyor belt. In addition, the desired objects had to be detected in a very short time to trigger the subsequent ejection mechanism in a timely and precise manner.
The image processing part of the sorting system was developed and tested by STEMMER IMAGING.
This extremely demanding task was solved by combining STEMMER IMAGING machine vision components. The optimal combination of these components was first checked and selected during a feasibility study by the Puchheim imaging experts.
Maximum performance required
Maximum illumination of the image capture zone was required: a Lux HD-2500/WH by manufacturers Hema, 250 cm wide and with water-cooled line illumination, was used here, which offered 40 Amperes power consumption at a voltage of 36 Volts resulting in an output of 1440 Watts as a special feature. High power supply units are required to operate this adjustable lighting bar.
Image recording is via a 3CMOS colour line camera, type LT400 CL-F with 4096 pixels by Danish manufacturer JAI. This is equipped with a CameraLink interface and can handle transmission of the captured data volumes up to 276 MByte/s in the CameraLink medium mode. To withstand the harsh environmental conditions prevailing at the location, this line camera was integrated in an extremely robust, double-walled protective camera housing with water-cooling and a double front window. This stable housing ensures that the camera will always operate in the required temperature range. A compressed air knife ensures that the window is kept free from dust and rock chips, so that the camera always has a clear view of the conveyor belt.
STEMMER IMAGING obtains such special housings from external suppliers and adapts them to the tasks in hand according to the specifications required. In this case, the Pucheim-based company handled the entire integration of all machine vision components as well as installation and connection of the camera.
FPGA for fast object recognition
In this application, the JAI camera records colour line images of the bulk material and passes the image data to a PC in which a MicroEnable IV – VD4-CL frame grabber by Silicon Software is used for the direct processing of the recorded images and classification of the objects. The core of this frame grabber is an FPGA component programmed with the FPGA development environment VisualApplets 2.0 by Silicon Software. This platform enables a more efficient and economic programming of FPGAs than conventional methods. Due to the optimised processing routines and the utilisation of FPGA resources, the system provides extremely fast object classification for this mining application with recognition times of less than 0.02 ms.
Classification of the objects includes characteristics such as the surface of the objects, their centre of gravity in both x and y direction, the surrounding rectangle, their orthogonal and diagonal circumference as well as the compactness of the individual pieces of rock. The calculation of the centre of gravity is required to detect the exact position of faulty objects and to enable subsequent ejection via actuators. The exact position, the timing and duration of actuator performance needs to be calculated to ensure reliable ejection of the faulty object.
In terms of image processing, sorting initially comprises the recording of an RGB camera signal at 10 Bits per level, followed by a white balance adjustment. These data are then converted into an HSI colour space to define the background and good parts in terms of colour. This results in three levels: good, unknown and background. This formed the basis for image segmentation using a Blob analysis to classify the detected objects according to their characteristics.
If a piece of rock is detected which needs to be ejected, then the frame grabber modulates a serial TTL trigger signal which is initially converted into a RS232 signal using a converter box specifically developed by STEMMER IMAGING for this purpose. A second converter box converts to RS485 format which allows a more reliable transmission of the data over longer distances and is also bus-capable. The processed fault signals are transmitted to two Adam 5000 E control units which each feature 128 outputs which can control the up to 256 connected actuators to eject the undesirable parts. The number of actuators really required by the customer for this application will be decided during commissioning. However, the system is most unlikely to be the limiting factor, as this part can be extended optionally up to 32768 = 128 * 28 outputs per daisy chain arrangement.
For all types of bulk materials
"The time required for the entire development of this high performance system was approximately five man-days", estimates the technical expert responsible for realising the application at STEMMER IMAGING. A major part of the project consisted in the programming of the FPGA with the aid of VisualApplets 2. Here the customer benefited from the experience of STEMMER IMAGING, which to-date is the only Silicon Software partner to meet the criteria for operating as a VisualApplets Design Center (VADC).
With its VisualApplets Competence Centres (VACC) and the higher level VADCs, Silicon Software has introduced a two-tier certification system for VisualApplets. This is also a quality statement to help the users of VisualApplets to find suitable and well-versed partners for their VisualApplets projects. A VisualApplets Competence Centre can conduct training and offer in-depth advice on this FPGA programming tool.
VisualApplets Design Centres go one step further: they can solve image processing requirements through FPGA programming with VisualApplets independently, which paid dividends in the application described here. It was also important for the customer to be able to develop the prepared source codes further themselves, which was allowed for in the design of the system by STEMMER IMAGING.
The incoming bulk material is recorded by a line camera and evaluated in less than 0.02 ms. Faulty objects are discharged via an ejection mechanism.
Illumination is via a 250 cm wide, water-cooled line lighting system with an output of 1440 Watts.
A robust, double-walled protective camera housing with water-cooling and a double front window with a compressed air knife protect the line camera.
The 3CMOS colour line camera LT-400 CL by JAI records the images and transfers them to the evaluating computer CameraLink.
The microEnable IV – VD4-CL frame grabber by Silicon Software processes the recorded images and classifies the objects. To this purpose it utilises an integrated FPGA component
Up to 256 actuators can be controlled for ejecting the undesired parts via two control units.
The system is presently being installed at the end customer's and is to commence operations shortly. It is however not limited to applications in mining: in principle this solution is suited for all applications where fast-moving bulk materials need to be classified, sorted or ejected. This also includes applications where other raw materials, granulates or perhaps foods need to be sorted with the aid of machine vision.