You are on page 1of 7

(IJCSIS) International Journal of Computer Science and Information Security

,
Vol. 8, No. 7, October 2010

1

An approach to a pseudo real-time image processing engine for
hyperspectral imaging

Sahar Sabbaghi Mahmouei* Prof.Dr.Shattri Mansor Abed Abedniya
Smart Technology and Robotics Programme Remote Sensing and GIS Programme, MBA Programme,
Institute of Advanced Technology (ITMA), Department of Civil Engineering, Faculty of Management (FOM),
Universiti Putra Malaysia, Serdang, Malaysia Universiti Putra Malaysia, Serdang, Malaysia Multimedia University, Malaysia
sabbaghi.sahar@gmail.com shattri@eng.upm.edu.my abed.abedniya@gmail.com

Abstract
Hyperspectral imaging provides an alternative way of increasing reflectance data such as extraction of various vegetation
the accuracy by adding another dimension: the spectral features. Satellite-based remote sensing provides a
wavelength. Recently, hyperspectral imaging is also finding its unique opportunity to obtain characteristics over large
way into many more applications, ranging from medical imaging areas, whereas airborne remote sensing provides remotely
in endoscopy for cancer detection to quality control in the sorting
of fruit and vegetables. But effective use of hyperspectral
sensed data over the medium scale, such as farms and
imaging requires an understanding of the nature and limitations small watersheds [4]. However, these studies largely
of the data and of various strategies for processing and depend on the availability of spectral images that are
interpreting it. Also, the breakthrough of this technology is usually quite expensive and need to be acquired by
limited by its cost, speed and complicated image interpretation. professional image providers. Ground based hyperspectral
We have therefore initiated work on designing real-time imaging has been used as a cheap tool to acquire remotely
hyperspectral image processing to tackle these problems by using sensed data from individual part of proposed area [4].
a combination of smart system design, and pseudo-real time In this paper, we propose an approach to pseudo
image processing software.  The main focus of this paper is the
real-time image processing engine for hyperspectral
development of a camera-based hyperspectral imaging system for
stationary remote sensing applications. The system consists of a
imaging to increase mission flexibility for environmental
high performance digital CCD camera, an intelligent processing planning, medical diagnostics, remote sensing, and natural
unit, an imaging spectrograph, an optional focal plane scanner resources applications. All processes in the
and a laptop computer equipped with a frame grabbing card.  In implementation of hyperspectral imagery and remote
addition, special software has been developed to synchronize sensing apply near real time image processing done at the
between the frame grabber (video capture card), and the digital spatial and numerical modeling laboratory (SNML) at the
camera with different image processing techniques for both University of Putra Malaysia. The main focus of this
digital and hyperspectral data. research is the development of a camera-based
hyperspectral imaging system for stationary remote
Keywords: Remote sensing, image processing, Real-Time, sensing applications.  Hyperspectral imaging provides an
frame grabber, hyperspectral, Hardware/Software Design.
alternative way of increasing the accuracy by adding
another dimension: the wavelength. Recently,
1. Introduction hyperspectral imaging is also finding its way into many
more applications, ranging from medical imaging in
Digital and Remote sensing image processing is nowadays endoscopy for cancer detection to quality control in the
a mature research area. Use of hyperspectral remote sorting of fruit and vegetables.
sensing in both research and operational applications has The impetus in performing this research was
been steadily increasing in the last decade. Hyperspectral given by existing snags and problems faced by workers in
imaging systems can capture imagery from tens to the field. So far, many of the image processing software
hundreds of narrow bands in the visible to infrared spectral available in the market do not process images in real time.
regions. These systems offer new opportunities for better The software has to download and read the images first
differentiation and estimation of biophysical attributes and and then prepare image-processing functionalities on them.
have the potential for identification of optimal bands In this paper, we attempt to show that it is possible to have
and/or band combinations for a variety of remote sensing pseudo-real image processing. This means that processing
applications [1-3],[11]. Different remote sensing is done on the fly: as soon as the camera captures the
applications have proven to be potential sources of image, the image processing algorithm comes into play
immediately in all embedded applications.

* Responsible author

201 http://sites.google.com/site/ijcsis/
ISSN 1947-5500
(IJCSIS) International Journal of Computer Science and Information Security,
Vol. 8, No. 7, October 2010

2

The Hyperspectral imaging system consists of systems allow greater resolution of data to be assimilated
four components: than do line scanner systems.
• A sensing component: a hyperspectral sensor
(high performance digital CCD camera now
known as the ImSpector manufactured by 3. Real-Time Hyperspectral Imaging System
SPECIM systems) for acquiring data or images. Requirement
• An optional focal plane scanner
• A video capture (frame grabber) card connected 3.1. The sensor
to the CPU from the camera helps in data capture.
The hyperspectral sensor used in this study was a ground
• Acer Extensa Notebook 4630z, which is a 2.0 based user-friendly line sensor ImSpector (V10) See
GHz Intel notebook, computer manufactured by (Fig.1). The new ImSpector Fast10 is a high intensity
Acer Inc., has been used as the CPU on the sensor imaging spectrograph, and makes spectral imaging
part. possible at hundreds and even up to 1500 images per
The remainder of this article is structured as second. ImSpector Fast10 imaging spectrograph
follows. Section 2 presents essential characteristics and provides[5]:
concepts in the scope of the work. Section 3 presents
System Requirement. We will describe developing
• high light throughput
software in section 4. Description of proposed method,
• superior image quality
design and a relative technique are discussed in section 5.
Section 6 shows the experimental result and discussions. • good spectral resolution of 15 nm
Section 7 presents the conclusion of this paper. • full VNIR spectrum of 400 - 1000 nm over a
narrow dimension, allowing short read out times
• maximum light intensity on the camera
2. Concepts and characteristics
pixels,allowing short integration times
In order to draw a clear picture of fundamental concepts • high speed acquisition in many low cost industrial
and characteristics of hyperspectral imaging, it is CCD and CMOS cameras
important to recap some key concepts and definitions
which are accepted by experts in this field. The ImSpector imaging spectrograph is a
component that can be combined with a broad range of
Real-time image processing: operating systems monochrome matrix cameras to form a spectral imaging
serve application requests nearly real-time. In the other device. Equipping the instrument with an objective lens
word manipulation of live images, typically within 50 to coupled with a monochrome area camera, converts
100 milliseconds, so the human user perceives them as ImSpector to a spectral line imaging camera. Operation is
instantaneous. based on the direct sight imaging spectrograph technology
Embedded systems: An embedded system is of the Spectral Imaging Ltd. (SPECIM), Oulu, Finland [6].
a computer system designed to perform one or a few ImSpector captures a line image of a target and disperses
dedicated functions often with real-time light from each line image pixel to spectrum. Each spectral
computing constraints. It is embedded as part of a image then contains line pixels in the spatial axis and
complete device often including hardware and mechanical spectral pixels in the spectral axis (Fig. 2) [4]. It is possible
parts. to acquire full spectral information for each line image
acquired from the target. Since ImSpector captures
Engine: The image processing engine, or image sequential images of the moving target (or the sensor itself
processor, is an important component of a digital moves), a 2D spectral image can be formed. This
camera and plays a vital role in creating the digital image. technology allows diverse opportunities to analyze the
The image processing engine comprises a combination of target accurately based on its spectral features.
hardware processors and software algorithms. The image
processor gathers the luminance and chrominance
information from the individual pixels and uses it to
compute/interpolate the correct color and brightness values
for each pixel.

Pushbroom: In remote sensing, an imaging device
consisting of a linear array of sensors (CCD camera)
which is swept across the area of observation. Pushbroom Fig. 1– hyperspectral sensor (ImSpector V10).

202 http://sites.google.com/site/ijcsis/
ISSN 1947-5500
(IJCSIS) International Journal of Computer Science and Information Security,
Vol. 8, No. 7, October 2010

3

3.1.1 Advantages of hyperspectral imaging system and projects a two-dimensional image profile (line image)
onto the CCD surface. This configuration allows image
Hyperspectral imaging is extremely advantageous in terms acquisition under stationary or laboratory settings [2], [10].
of its data, presenting the information in the spatial
direction which is useful for extracting information with 3.3 A video capture (frame grabber)
less loss of data. Some advantages of hyperspectral
imaging over conventional techniques such as: NIRS The FrameLink frame grabber is a TYPE II PC Card with
(Near-infrared spectroscopy), RGB, and hyperspectral both a Camera Link and Card bus interface. It provides the
imaging are shown in Table 1 [7, 8]. ability to capture digital video data from a ‘base
configuration’ Camera Link interface and transfer that data
Feature RGB NIRS MSI HSI to host memory via a Card bus (PCI) interface. The frame
imaging link is a professional state of the art PCMCIA card bus
digital video capture card, allowing user to display,
Spatial √ √ √ capture, store and preview mega pixel video image (up to
information 16 mega pixels) on the notebook computer [9]. The
Imperx FrameLink video capture card is as shown in
(Fig.3) below.
Spectral √ Limited
information √

Multi-
constituent Limited √ Limited

Sensitivity to
minor Limited
Components √
Fig.3 – The IMPERX FrameLink Fast CardBus video capture (frame
grabber) card. This picture has been taken from the official website of
Table.1 Advantages of hyperspectral imaging system Imperx Inc.

3.4 The computer system

The computer is an Intel Pentium III (800 MHz) processor
based system with 250 GB hard drive. The operating
system on the computer is Microsoft Windows XP. A PCI
interface board provided with the imaging system is
installed in a master PCI slot in the computer. The utility
software is installed in the computer for complete camera
control, image acquisition and applies image processing
technique. The Acer Notebook computer is as shown in
(Fig. 4) below.

Fig. 2 – The operating principles of ImSpector.

3.2 An optional focal plane scanner

The focal plane scanner performs line scanning across an Fig.4 – Different views of the Acer Extensa 4630z Notebook
input imaging area within the focal plane of the front lens computer. This has been used as the CPU on our hyperspectral imaging
and the spectrograph disperses each line into a spectrum system. These pictures have been obtained from Acer Inc.

203 http://sites.google.com/site/ijcsis/
ISSN 1947-5500
(IJCSIS) International Journal of Computer Science and Information Security,
Vol. 8, No. 7, October 2010

4

Furthermore, the functions embedded in the
3.5 Acquiring ground-based hyperspectral images popup menu endow the interface with many other image
processing and parameter extraction capabilities.
The ground-based hyperspectral line imaging systems is
shown in (Fig. 5). The hyperspectral sensor ImSpector
captures the scene. ImSpector captures a line image of the
scene and disperses it into a spectrum. By moving the
sensor up and down or left and right by means of a battery-
powered movable tripod base, the whole scene is captured.
The rate of image acquisition can be up to 30 fps, and data
can saved in an audio–video interleave (avi) file format.
The raw spectral data obtained by the sensor and the image
generated by applying a line pixel assembly algorithm to
the raw data in an image [4]. Each frame represents the
spectral data corresponding to a spatial line. The x-axis of
each frame is the spatial axis and the y-axis is the spectral
axis. Each frame is composed of 480 spectral lines, each
representing spectral data at a particular wavelength. In
order to facilitate comprehension of these spectral data, an Fig. 6 – Software interface
image is generated by applying a line pixel assembly
algorithm to every frame. Assembly of spectral lines with The Image Processing Toolbox provides a
an equivalent wavelength from all frames makes one comprehensive set of standard algorithms and graphical
image, and thus the procedure can generate a total of 480 tools for image processing, analysis, visualization, and
images, each displaying the scene captured with a different algorithm development. You can restore noisy or degraded
wavelength [6]. images, enhance images for improved intelligibility,
extract features, analyze shapes and textures, and register
two images. Most toolbox functions are written in the C++
language. A schematic diagram of the interface design and
its utilities is shown in (Fig. 6, 9).

4.1 Some key features of image acquisition toolbox

• Image enhancement, including linear and
nonlinear filtering, filter design, and automatic
contrast enhancement;
• Binarization filters (threshold, threshold with
carry, ordered dithering, Bayer dithering, Floyd-
Steinberg, Burkes;
Fig. 5 – Hyperspectral image acquisition system. • Color image processing, including color space
conversions and channel replacing, channel
filtering;
4. Developing the software • Spatial transformations and image registration,
including a graphical tool for control-point
Once the hyperspectral images are generated, they would selection;
appear as a stack of continuous images. Manipulation of • Fourier transformation (low pass and high pass
hyperspectral images and extraction of useful spectral filters;
information from these multidimensional data requires the • Mathematical morphology filters (erosion,
development of intelligent software. For this purpose, dilatation, opening, closing, hit & miss, thinning,
software with many high level computing and thickening);
visualization functions embedded in a number of useful • Edge detectors (homogeneity, difference, sobel,
toolboxes. (Fig.6). illustrates the main menu and its user canny);
interfaces for image processing and data extraction.
• Median filter, Adaptive smoothing, Conservative
smoothing;

204 http://sites.google.com/site/ijcsis/
ISSN 1947-5500
(IJCSIS) International Journal of Computer Science and Information Security,
Vol. 8, No. 7, October 2010

5

5. Methods
Preliminary image acquisition testing trials indicate that
In this technique only hyperspectral data for areas of this CCD camera-based hyperspectral imaging system has
interest captured. The principal component analysis potential for agricultural and natural resources
method is used to hyperspectral imaging systems shown in applications. (Fig.8) shows the architecture of the ground-
(Fig.8.) based hyperspectral line imaging systems based on
architecture at SMNL laboratory in UPM.
Based on the raw data, the software developed is
used to generate images for a three-dimensional, including
two spatial axes and one spectral axis that can be produced
in one of four ways: Fourier transform imaging, point-to-
point spectral scan in a spatial grid pattern, line by line
spatial scan i.e. the pushbroom technique, and wavelength
tuning with filters. The line by line spatial scan and
wavelength methods are more suitable. In the pushbroom
method spectral data acquire across the full spectral range
of single spatial lines consecutively to reconstruct the
hyper spectral tube.

The CCD camera provides 1280(h) x 1024(v)
pixel resolution and true 12-bit dynamic range. The
imaging spectrograph is attached to the camera via an Fig. 8 – architecture of the ground-based hyperspectral imaging
adapter to disperse radiation into a range of spectral bands. systems
The effective spectral range resulting from this integration
is from 400 nm to 1000 nm. Diffuse illumination of the
sample is made possible by a florescent-halogen or LED 6. Experimental result and discussion
source [6]. The light reflected from the target enters the
objective lens and then spread into its component
In order to test the sensor design concept and to integrate
wavelengths as shown in (Fig7).
software design, we simulate a realistic scene. The Digital
Imaging and hyperspectral software, developed at Institute
The optional focal plane scanner can be attached
of Advance Technology (ITMA) in Malaysia. By scene
to the front of the spectrograph via another adapter for
simulation and sensor modeling, we hope to reduce the
stationary image acquisition. The camera and the frame
cost and development time in new sensor designs, together
grabbing card are connected via a double coaxial cable,
with the support of the algorithm and the techniques
and the utility software allows for complete camera control
method. The image processing algorithms are designed
and image acquisition. The imaging system captures one
only to demonstrate the idea of effectively capturing
line image for all the bands at a time and the focal plane
hyperspectral data. Needless to say, more sophisticated
scanner serves as a mobile platform to carry out
algorithms need to be developed for more challenging
pushbroom scanning in the along direction.
tasks.
After software executed, the main window will
appear. The main window provides the primary area for
viewing real-time images received from the camera. When
image viewing is active, pull-down menu with two options
reveals: ‘Player’ and ‘Exit’. Player button will toggle
between ‘Start Grab’ and ‘Stop Grab’ every time the user
clicks on it. By clicking on ‘Start Grab’ enables the engine
and causes the main window to display live images
received from the camera. Clicking on ‘Stop Grab’
disables the engine and causes the display to freeze. When
recording images to disk, Image Format option selects the
format, ‘BMP’, ‘JPEG’ or ‘TIFF’ that the image will be
saved in. Selecting ‘JPEG’ activates a compression slider.
‘Best Quality’ provides the least compression while
‘Smallest File’ provides the most compression.
Fig 7– A Scheme diagram the of current Hyperspectral imaging system

205 http://sites.google.com/site/ijcsis/
ISSN 1947-5500
(IJCSIS) International Journal of Computer Science and Information Security,
Vol. 8, No. 7, October 2010

6

In order to evaluate the system and simulate a
realistic scene we pluck the leaf from the tree in the fields
near our campus and around University of Putra Malaysia.
In hyperspectral imaging system design, different portion
of bandwidth can be selected and determined by analyzing
model spectral profile combined to a single image profile
and a binary decision was made using a threshold found by
experience. Thus, object can be demonstrated in real time.
In (Fig .9) we show a single image snapshot captured, and
the result was combined to produce a co-registered
composite image. After capture the scene raw data save as
‘JPEG’ format then we apply some image processing
technique in order to assess our software.
Fig.10 – Apply Thresholding Filter
Here we select histogram filter to determine the overall
intensity of the image that is suitable for our inspection The hyperspectral line sensor captures the raw
task. Based on the histogram data, you can adjust your reflectance data in the approach that illustrated above. As
image acquisition conditions to acquire higher quality this is a ground-based system, the cost is much lower than
images see (Fig.9). for airborne- or satellite-based remotely sensed data. The
nominal spectral resolution of 1.5–2nm within the
wavelength range of 400–1000nm is sufficient for most
application studies. The software developed serves a
pivotal role in dealing with the spectral data that are
captured. It can generate images from the raw spectral data
in an audio–video interleave format or image format.
Useful image analysis algorithms are included, such as;
Thresholing and other functions determine whether an
image meets certain criteria for inclusion in an analysis.

7. Conclusions

This paper reviews the recent developments in ground-
based hyperspectral imaging system for acquisition of
Fig.9 – captures the scene and apply Histogram Filter
reflectance data that is useful for many real-life
applications such as; environmental planning and natural
Another filter that applied for evaluate our system work is
resources applications. The hyperspectral imaging
thresholding (Fig. 10). Objective of thresholding filter is
technique described in this article provides a new
converting the image into binary objects. Thresholding is
opportunity for determining the optical properties and
the simplest method of image segmentation. From
quality of product such as food and agricultural products.
a grayscale image, so we could apply the basic
morphology processes, Image analysis capability can also
Compared to other techniques, the hyperspectral
be expanded to include other types of analytical techniques
imaging technique is simpler, faster and easier to use, and
for a particular image analysis purpose.
more importantly it is capable of determining optical
properties over a broad spectral range simultaneously. The
technique also is useful for measuring the optical
properties of turbid food and agricultural products.
Moreover the hyperspectral imaging technique is
potentially useful in assessing, sorting, and grading fruit
quality.

206 http://sites.google.com/site/ijcsis/
ISSN 1947-5500
(IJCSIS) International Journal of Computer Science and Information Security,
Vol. 8, No. 7, October 2010

7

References
[1] D.Tuia, and G.Camps-Valls, "Recent advances in remote
sensing image processing", IEEE International Conference
on Image Processing (ICIP), 2009 , 3705 – 3708. AUTHORS PROFILE
[2] H. James Everitt, and R. Michael Davis , and Chenghai
Yang, "A CCD Camera-based Hyperspectral Imaging System for
Stationary and Airborne Applications" , Geocarto International,
Vol. 18, No. 2, June 2003. Sahar Sabbaghi Mahmouei is currently
doing her Master degree in Institute of
[3] H. James Everitt, Chenghai Yang, Joe M. Bradford and Dale Advanced Technology and Research
Murden, Airborne Hyperspectral Imagery and Yield Monitor (ITMA), Universiti Putra Malaysia, UPM.
Data for Mapping Cotton Yield Variability, Volume 5, Number Sahar has received her B.Sc in software
5, 445-461,2004. computer engineering field in 2006 from Iran
Azad University. Her research interest
includes image processing, machine vision,
[4] Xujun Yea, Kenshi Sakaib, Hiroshi Okamotoc, Leroy O. artificial Intelligence and e-commerce.
Garcianod," A ground-based hyperspectral imaging system for
characterizing vegetation spectral features",Elsevier Science
Publishers B. V, Volume 63, NO 1, 13-21 , August 2008.

[5] Official website of Spectral Imaging Ltd, Finland.
SPECIM: http://www.specim.fi/

[6] Users Manual for Imspector spectrograph Ver.2.0 from
SPECIM website.

[7] A. A., O'Donnell, C. P., Cullen, P. J., Downey, G., and Frias,
J. M. 2007. "Hyperspectral Imaging - an Emerging Process
Analytical Tool for Food Quality and Safety Control",Volume 18,
Issue 12, 2007, 18(12); 590-598.

[8] Osama M. Ben Saaed, Abdul Rashid Mohamed Shariff,
Helmi Zulhaidi Mohd Shafri, Ahmad Rodzi Mahmud,Meftah
Salem M Alfatni, "Hyperspectral Technique System for Fruit
Quality Determin ", Map Asia 2010 and ISG 2010 Conference,
July2010.

[9] Official website of Imperx Inc,USA .
FrameLink: http://www.imperx.com/

[10] C. Mao., "Hyperspectral imaging systems with digital CCD
cameras for both airborne and laboratory application", 17th
Biennial Workshop on Videography and Color Photography in
Resource Assessment, American Society for Photogrammetry
and Remote sensing, Bethesda, MD. pp. 31-40,1999.

[11] C. Mao., " Hyperspectral focal plane scanning-an innovative
approach to airborne and laboratory pushbroom hyperspectral
imaging. Proc. 2nd International Conference on Geospatial
Information in Agriculture and Forestry", ERIM International,
Inc.,Ann Arbor, MI. Vol. 1, pp. 424-428, 2000.

207 http://sites.google.com/site/ijcsis/
ISSN 1947-5500