A HIGH SPEED CAMERA SYSTEM BASED ON A IMAGE SENSOR IN STANDARD CMOS TECHNOLOGY

N. Stevanovic, M . Hillebrand, B. J. Hosticka, U. Iurgel, and A. Teuner

Fraunhofer Institute of Microelectronic Circuits and Systems FinkenstraSe 61 D-47057 Duisburg, Germany

ABSTRACT
In this contribution a novel camera system developed for high speed imaging will be presented. The core of the system consists of a CMOS image sensor manufactured in a 1 pm standard CMOS process. The special merit of the image sensor is the capability to acquire more than 1000 frameds using a global electronic shutter in each sensor cell. The image sensor itself is embedded in a sensor board that has been developed for digitizing the sensor's single serial analog output signal as well as for supplying the sensor with all control signals necessary for proper operation. Both the control signals and the digital output signal (RS-422) are generated on a FPGA-chip. Moreover, a graphical user interface adapted to the frame grabber card used (Matrox Pulsar) has been developed for the purposes of monitoring, high speed acquisition, and image processing.

2.

HARDWARE SYSTEM COMPONENTS

The realized camera consists of three hardware modules. The first module - the sensor module carries out the photo-electrical signal conversion at a high pixel clock. It delivers 10 or 12 bit decoded image information at the data rate of 22 Msamples/s. Great care was taken when designing the sensor board because this module processes both analog and digital signals.

-

1.

INTRODUCTION

The second module, a FPGA device, is programmed to generate the control signals for the sensor module and the interface card, which represents the last hardware module of the camera. The interface card converts the digital data into a RS-422 compatible signal and receives control signals from the host PC necessary for adjustment of the integration time. The block diagram of the high frame rate camera is illustrated in Figure 1.

As the degree of production automation is steadily increasing, omnipresent low-cost visual inspection is becoming mandatory for quality assurance. Thus, on-line monitoring of process lines should ideally allow instantaneous reactions to occurring problems, e. g. alarm triggering or removing defect particles during end-inspection. The classical optical-mechanical or electron-optical systems [ 11 for acquisition of images at high frame rates are not suited for these purposes because of their complexity and high costs. Nowadays semiconductor cameras using CCD image sensors offer an alternative solution, but they need a high intensity illumination, dissipate considerable power, and require complex peripheral electronics. Unlike conventional CCDs imagers, CMOS image sensors can be manufactured using standard CMOS processes. Thus, these imagers allow cointegration of readout and signal processing electronics. Also they offer other numerous advantages [2], e. g. low power dissipation and low cost manufacturing. Taking advantage of this approach we have developed a high frame rate camera system that is based on an image sensor with a resolution of 128 x 128 pixels. It has been realized in our in-house 1 pm standard CMOS technology and represents a major part of a sensor module of the camera. Beside the sensor module our camera system includes two other modules necessary for control signal generation and readout of acquired image data. The camera is connected by a RS-422 interface to a conventional frame grabber card which is slotted in a host PC. A graphical user interface called Speedgrub, implemented in C"/WindowsNT, enables performing all necessary operations, i. e. camera calibration, monitoring, high frame rate image acquisition.

Figure 1. Block diagram of the high frame rate camera.
The photo-electrical conversion on the sensor module is performed by the high frame rate CMOS sensor at a resolution of 128 x 128 pixels [3]. Using a global electronic shutter all pixels process the image at the same time slot so that no image smearing due to fast object motion can occur. The shutter time can be varied in order to enable adjustment of the photosensitivity to different illumination conditions. Moreover, an important feature of this CMOS sensor is the complete decoupling of the shutter time and the readout pixel clock. Due to this feature which is not available in CCD imagers it is now possible to use the camera not only for fast image acquisition but also for optical measurements such as triangulation. A schematic of the active pixel cell implemented in our image sensor is shown

0-7803-5471-0/99/$10.0001999IEEE

V- 148

Biasing as well as driving the control logic of the image sensor has been realized in the sensor module. The output image data are sequentially transferred via the frame grabber to the RAM of a host PC. As photosensitive element we use the photodiode formed by n-diffusiion and p-well. At the beginning of each frame both the reset and shutter signals must be set globally in each pixel to high. shutter is closed). 3. respectively. no smear effects and no blooming. The photocurrent caused by incident light discharges the storage capacitor Cs of each pixel. Figure 4: Chip microphotograph ensures a better driving capability and adjustment of the analog data signal at the proper operation point of the A D converter. Figure 4 shows the chip microphotograph of sensor integrated in a 1 pnn CMOS process. readout of the remaining voltage stored at Cs is started. The latter is essential in order to capture the imageis of fast objects at high speeds with sharp edges. Because of this high transfer V-149 . which Figure 5: Photograph of the realized sensor module. It cointegrates the most important functions necessary for an easy handling of the developed camera. The image sensor shows a linear response to illumination. The output voltage is available at a single serial analog output of the sensor.in the Figure 2. Its size is about 30 mm2. Photographs of the realized sensor module and the camera are shown in Figures. It is carried out row by row by activating the row select signal (see Figure 3 ) . e. In the pixel cell only NMOS transistors have been used in order to minimize the cell area. For the frame rate of 1000 framesk and 10 bit image data resolution a data transfer on the PCI bus of 3 2 MBytesls is needed. The signals that drive both the control logic of the image sensor and the A D converter are also generated on the FPGA chip. At the end of integration (i. Programming of the bias voltages (delivered by DIA converter in the same module) is performed using DIP switches in the interface module which are controlled using signals generated in a FPGA chip. The output signal of the sensor is connected to analog board electronics. GRAPHICAL USER INTERFACE AND CAMERA CALIBRATION In order to operate the high speed camera system a software package called Speedgrub has been developed that adapts to the Matrox Pulsar frame grabber card and the camera system. Figure 3: Timing diagram. 5 and 6. After the storage the capacitor Cs has been charged to VREF reset signal goes to low and the integration starts. T I p" Vntr I"n line - Figure 2: Schematic of the sensor cell.

..i of the pixel i with respect to the raw output voltage VSunsor.. denote the gain correction coefficient and Vof . 4. i(EDark) 9 Vo. The recording of the data can be triggered either manually from the Graphical User Interface (CUI) or electronically by interrupting a light barrier.. the offset correction of the pixel i. V.. e. the size of the RAM determines a maximum number of recorded images.. After recording a process of fast motion the program offers slow motion replay at a desired frame rate in the range of 1 to 100 frames/s. Considering the both nonuniformity types a calibrated voltage at the output Vcc. integration time of 1 O O p corresponds the rate of 1000 frameds). Vo(EBright) and VO. In order to find the best illumination conditions a global brightness adaptation can be performed by setting the integration time but it has to taken into account that the integration time affects the total frame rate (i. two reference images must be acquired . the data must be transmitted directly to the RAM of the host. ) and the other one of an uniformly bright scene ( EBrrskr From these data the coefficients for each pixel can be ). different informations of the operation mode such as magnification or contrast enhancement parameters are displayed at the CUI as shown in Figure 7. rate. For this purpose we have developed an algorithm that cancels the nonuniformity of the CMOS imager. Therefore.)are average values of the output voltage of all pixels as well as the output voltage of pixel i for both reference images at uniformly dark and at uniformly bright scenes respectively. EXPERIMENTAL RESULTS The high frame rate camera with a linear output characteristic shows a dynamic range of 58 dB and SNR of 54 dB.one of an uniformly dark scene ( E. It suppresses both offset and gain nonuniformity of the sensor. To obtain these coefficients.rr. The user interface allocates the wanted amount of memory as well as the desired trigger mode. The power consumption of the entire system is about 8 W at 5 V power : Figure 7 Graphical user interface of a Cff -program Speedgrub.. In order to obtain excellent image quality the camera must be calibrated. obtained as Figure 6: Photograph of the high frame rate camera. Additionally. it enables monitoring of scenes at 50 framesls.(EB. a of pixel i can be written as where Vo(EDark).150 .where G. Also..

R. The core of this system.1990. In this case. Fossum: "Active Pixel Sensors . disturbing effects like shadows or daylight variations). global brightness adaptation as well as cointegration of A/D converters. portable. 2 . V-151 . 6. [2] E. Figure 8: A lab scene recorded with camera at the rate of 100 frameds without (left image) and with presented calibration (right image). no blooming. for monitoring fast industrial manufacturing processes. Proceedings of ESSCIRC '98. 11. 316-319. This very broad operation range is very interesting especially for machine vision appliications. Besl: "Active. It is based on three hardware modules and it offers a compact. The integration time was set to approximately 100 11s. [4] P. 1: 127-152. Vol. Due to the linear characteristic to the incident light it is possible to reconstruct the distance to the objects within the acquired scene. J. Stevanovic. At the same time it offers a low-cost solution when compared to CCD based high speed camera systems. Additionally the capability of capturing the images at the very short integration times (down to several nanoseconds) qualifies the camera for the use for the 2D range estimation. Figure 8 shows two :. the camera is also suited for other applications like machine vision at programmable frame rates or range imaging using active pulsed illumination [4]. The last application results from the fact that the image sensor operates also beyond the visible spectrum in the near infrared range ( h= 750 nm . and low power solution for high speed video systems.Are CCD'S Dinosaurus". pp. Our future research will be directed towards high-speed CMOS cameras with higher resolution. Optical Range Imaging Sensors". an image sensor developed and realized in a 1 pm standard CMOS technology provides high speed imaging at low power consumption. Lately an optimization of a system has proven the capability of a camera system and CMOS-Sensor respectively to acquire the image of the strong illuminated scene at the integration time down to 30 ns. whereas the image sensor dissipates 280 mW. Beside the camera hardware a software package has been developed that enables an effective and comfortable operation of the camera. 5. at the lower frame rates it is possible to switch ithe shutter into "continuously on" in order to reduce clock feedthrough. alternate images with and without near-IR illumination within a short time interval of about 1 ms can be acquired for analysis of only the difference images. Machine Vision and Applications 1988. global electronic shutter etc. since the readout time of a frame is much smaller than the integration time at the rates of 100 framesls and less. pp. SUMMARY AND OUTLOOK A novel high speed camera system has been presented. pp. Huston: "High-speed Photography and Photonic Figure 9: Consecutive images of a bursting balloon recorded at the rate of 1030 framelsec (less than 1 ms between two images) using the calibrated camera. 900 nm). 1978. Moreover. NO. September 1998. E. Recording". the camera can be operated using a near infrared light.till images recorded before and after camera calibration described above under normal daylight conditions at a rate of 100 framesls. REFERENCES [ 11 A. All these features are not available in CCD cameras. Additionally. et al: "A High Frame Rate Imager in Standard CMOS Technology". SPIE.. due to the capability to set the integration time and the readout pixel clock independently. where the portion of the light and brightness of a scene respectively caused by time delay of reflected light is measured.. like no smear. higher dynamic range. Figure 9 shows four frames of a scene featuring a balloon burst. e. g. As the experiment illustrates clearly no smearing effects due to fast motion appear in the frames. 601-609. Journal of Physics E (Scientific Instruments). It fulfills all of the necessary requirements for a fast image acquisition. Proc. The sequence has been recorded at the rate of 1030 franiesls. The camera operates not only in a high speed mode up to 1030 framesls but also at lower frame rates down to 50 framesls. 7. [3] N.supply voltage.44. To suppress extraneous illumination distortions (e. g.