Ultra-high resolution for industrial imaging
Flat panel displays (FPDs) cover a growing number of technologies including OLEDs (organic light emitting diodes) and LCDs (liquid crystal displays). Each new generation of FPDs has increasing pixel densities requiring even higher resolution and sensitivity for quality inspection combined with pressure not to increase inspection times. Document scanning applications include continuous verification and/or quality inspection of numbered print and inspection of symbols and labels on web, sheet or single documents, as well as inspection of security features by checking the presence, position and integrity of applied features such as such as foil and hologram devices and base paper inserts like security threads.
Camera sensors with more and more pixels
These high resolution inspection applications generally utilise dedicated inspection equipment complete with fully integrated cameras, optics and illumination systems. The majority of imaging applications utilise area scan cameras and recent advances in CMOS and CCD technology have resulted in the availability of camera sensors with more and more pixels, providing ever increasing spatial resolution while maintaining or improving the camera frame rate. However, the number of available pixels is not the sole consideration. Mass-produced cameras sensors for smart phones typically can offer resolution of the order of 12 - 16 megapixels (Mpixels), yet the sensors themselves are very small and are very much cost driven. This means that they have very small pixels leading to poor light collection and poor signal-to-noise, which alone would preclude them from being used in industrial imaging applications. However they also have limited on chip exposure control resulting in a rolling shutter system where all the pixels are not exposed at the same time.
Requirements for industrial imaging cameras
While this might not be noticed in consumer ‘happy snapping’ applications, there are a host of other requirements for industrial imaging cameras in addition to the number of pixels. These include pixel size & shape, sensitivity, frame rate, full image exposure control and triggering characteristics, dynamic range, spectral response, image pre-processing, partial scanning, advanced multiple region readout and sequence control among others. In addition cameras are characterised by the interfacing standard that they employ to transfer data to the host computer for measurement and analysis (e.g. CameraLink, GigE Vision, USB3 Vision, CoaXPress, CameraLink HS etc). While to the novice the significant price difference between camera phones and machine vision high resolution cameras seems unfathomable, the significantly lower manufacturing quantities and larger sensor sizes for machine vision mean a completely different pricing model.
How many pixels are enough?
New industrial vision cameras come to market on a regular basis and some high resolution area scan cameras like the recently announced 604 Mpixel Vieworks VNP with Thermoelectric Peltier Cooling (TEC) and Pixel Shift Technology for resolutions from 14192 x 10640 up to 28384 x 21280 (4x pixel shift). While the JAI Spark SP-20000 is a 20 MPixel camera providing 5120 x 3840 pixels. But what if you have an application that requires more pixels?
Different solutions for applications requiring more pixels
One solution would be to use multiple cameras to produce a number of images that are stitched together, or to keep to a single camera and move either the sample or the camera to produce a number of images that can be stitched together to produce a composite image. A third option is to use pixel shift technology which will be discussed in detail in the next section. A different approach is the use of linescan cameras. Line scan cameras, in their simplest forms, use a sensor with a single line of pixels. Typical resolutions vary from 512 x 1 pixels up to 16384 x 1 pixels. Line scan cameras work by building up an ‘area image’ (typically in a frame grabber) from multiple lines by moving either the sample or the camera. As only the width of the resulting image is fixed, it not only allows a very high resolution image to be created but also means that the aspect ratio of the image can be chosen to match the particular sample. Typically area scan sensors have an aspect ratio of 4:3, 16:9 or 1:1, but line scan images are not limited to this. Even with a 4:3 ratio a 16k line scan camera would produce either a 358 Mpixel or 201 Mpixel image, depending on the orientation.
Considerations in the use of line scan cameras
Two key considerations in the use of line scan cameras (more about line scan technology) are movement and alignment. Relative movement between the sample and the camera is essential to allow the image to be produced and the rate of movement must be appropriate for the particular imaging requirement and must have no variation otherwise distortion will appear in the image. Similarly, alignment of the camera is crucial since If the line scan camera’s line sensor is not perpendicular to the direction of movement, the resulting image will contain skew.
A recent development using the line scan approach is the introduction of contact image sensors. These behave in the same way as fax machines and scanners but on an industrial basis with far higher resolutions, speeds and interfacing. The advantage is reduction in working distance, removal of lens distortion and easier set up.
Pixel shift technology
The use of pixel shift technology, however, significantly extends the resolution capability of area scan sensors. The Vieworks VNP-series industrial vision cameras with CoaxPress interface feature up to 151 Mpixel resolution (14192 x 10640 pixels) CMOS sensor as standard, but also have pixel shift technology which can provide an extended resolution of 604 million pixels (28348 × 21280) for ultra high resolution applications. The sensor is mounted on a precise piezoelectric crystal stage which allows the sensor to be nano-shifted by 1/3 or 1/2 of a pixel. This enables the standard resolution to be extended by 4 times to 604 million pixels.
The pixel shifting process
The pixel shifting process is shown in Figure 1. In this example, the sensor is shifted precisely by ½ pixel in the X and Y directions as shown, with the resulting image being a combination of the 4 individual images giving improved resolution (4 shot result image) in comparison with the standard output image (1 shot result image). Image combination is carried out in software on the processing PC. A similar procedure can be carried out by shifting the sensor by 1/3 pixel to produce 9 resultant images. The camera is available in both monochrome and colour versions and the pixel shifting process has additional benefits for colour imaging. The colour CCD camera uses Bayer Interpolation to produce colour images and unwanted artifacts can occur such as colour moiré or false color pixels.
Figure 1: Principle of pixel shifting technology
Ideally suited for low light/contrast industrial applications
Using pixel shifting, however, no colour artifacts or aliasing will occur and the colour resolution is optimized, as shown in Figure 2. The Vieworks VNP-series sensors can be thermo-electric Peltier (TEC) cooled to as much as 15 degrees below ambient for improved sensitivity. This provides extremely stable operating conditions, reducing noise and enabling exposure for a long period of time or at higher gain levels to increase camera sensitivity. This makes the cameras ideally suited to low light or low contrast level or non-uniform brightness industrial applications.
Improved colour resolution and larger field of view
The improvement in resolution is illustrated on a section of banknote in Figures 3 and 4. Figure 3 is recorded at the standard resolution settings, but the pattern is pixelated and unclear. Figure 4 shows the same region using pixel shift technology and the pattern is clearly resolved. In addition to the obvious benefits of improved resolution, at any given resolution setting, the pixel shift gives a larger field of view if fitted with the appropriate lens, which means it could avoid the need to stitch multiple images together or remove the need for multiple cameras, saving both time and money.
