Research on Image processing

CONCEPTS OF MULTIMEDIA PROCESSING & TRANSMISSION-CSIT 402

Group 17

Concepts Overview
We will explore the fundamentals, historical context, techniques, applications, challenges, and
future prospects of multimedia image processing.
Introduction and Fundamentals

 What's an image?
 A simple image model
 Fundamental steps in image processing
 Elements of digital image processing systems
 Applications of image processing
 Benefits of image processing

Image
An image refers to a 2D light intensity function f(x,y), where (x,y) denote spatial coordinates
and the value of f at any point (x,y) is proportional to the brightness or gray levels of the image
at that point.
Before we jump into image processing, we need to first understand what exactly constitutes an
image. An image is represented by its dimensions (height and width) based on the number of
pixels. For example, if the dimensions of an image are 500 x 400 (width x height), the total
number of pixels in the image is 200000.
This pixel is a point on the image that takes on a specific shade, opacity or color. It is usually
represented in one of the following:

 Grayscale - A pixel is an integer with a value between 0 to 255 (0 is completely black

and 255 is completely white).
 RGB - A pixel is made up of 3 integers between 0 to 255 (the integers represent the
intensity of red, green, and blue).
  RGBA - It is an extension of RGB with an added alpha field, which represents the
opacity of the image.

Difference Between Analog and Digital Images

Analog Image:
- An analog image is a continuous representation of visual information.
- It's captured directly from the real world using devices like cameras and scanners.
- Analog images have an infinite range of colors and tones.
- The information in an analog image is stored in a continuous manner, like a film negative.
- Analog images are susceptible to degradation and noise due to external factors.

Digital Image:

• A digital image is an image f(x,y) that has been discretized both in spatial coordinates and
brightness.
That is a discrete representation of visual information.
• The elements of such a digital array are called image elements or pixels.
Image processing requires fixed sequences of operations that are performed at each pixel of an
image. The image processor performs the first sequence of operations on the image, pixel by
pixel. Once this is fully done, it will begin to perform the second operation, and so on. The
output value of these operations can be computed at any pixel of the image.

- It's obtained by converting analog information into digital data using devices like digital
cameras and scanners.
- Digital images have a finite range of colors and tones, determined by the bit depth.
- The information in a digital image is stored in discrete values, usually as pixels with specific
color or intensity values.
- Digital images are more resistant to degradation and noise, as they can be stored and
transmitted in a more controlled manner.
Emergence of Television and Photography
Early television systems and photographic techniques laid the foundation for modern image
processing. Techniques like darkroom manipulation were precursors to digital image editing.

Example: George Carey's 1858 photograph "First Photograph Ever Taken" marked the early
use of image manipulation to remove people from the scene.

Pioneering Analog Image Manipulation Techniques

In the mid-20th century, analog methods like dodging and burning were used to enhance
photographs. These methods influenced digital image processing techniques.
Laney, R.C. (1997). "Computational Image Analysis". IEEE Computer Society Press.
2.2 Transition to Digital Imaging
Advent of Digital Cameras and CCD Sensors
The introduction of CCD sensors revolutionized image capture. Digital cameras replaced film,
enabling instant viewing and manipulation of images.

Example: Sony's Mavica (1981) was one of the first digital cameras, capturing images on a
floppy disk.
Russell Kirsch's First Digital Image
Russell Kirsch's 1957 176x176 pixel digital image of his son paved the way for digital imaging.
This marked the beginning of pixel-based digital representations.

Image Processing

Multimedia Image Processing is the field that encompasses the manipulation, analysis, and
enhancement of visual data, ranging from photographs and videos to animations and virtual
reality content. It is driven by the goal of improving visual quality, extracting meaningful
information, and optimizing multimedia experiences.
There are basically 2 main forms of image processing. That is Analogue image processing and
Digital image processing.

Analog Image Processing

Analog image processing involves manipulating continuous visual data directly. This was
common in the early days of photography and film, where techniques like dodging, burning, and
color tinting were used in darkrooms to modify images. These methods were often manual and
required physical alterations to the original images.

Digital Image Processing

Digital Image processing is the process of transforming an image into a digital form and
performing certain operations to get some useful information from it. The image processing
system usually treats all images as 2D signals when applying certain predetermined signal
processing methods.
It involves applying various algorithms and techniques to manipulate discrete digital images. It
leverages the computational power of computers to enhance, restore, analyze, and transform
images. Digital image processing offers more precise control over image manipulation, as it
operates on discrete pixel values. It encompasses a wide range of operations, including noise
reduction, edge detection, image enhancement, and more.

The key difference between analog and digital images lies in their continuous vs. discrete nature
and the methods used for processing. Analog image processing involves direct physical
manipulation of continuous images, while digital image processing involves algorithmic
manipulation of discrete digital images using computers and software.

Digital Image Processing- Unveiling Visual Insights

Digital Image Processing plays a pivotal role in various applications such as medical imaging,
entertainment, surveillance, and remote sensing. Its techniques enable us to reveal hidden details,
detect anomalies, and create visually compelling content.
Fundamental Image Processing Steps

Image Acquisition
Image acquisition is the first step in image processing. This step is also known as preprocessing
in image processing. It involves retrieving the image from a source, usually a hardware-based
source.

Image Enhancement
Image enhancement is the process of bringing out and highlighting certain features of interest in
an image that has been obscured. This can involve changing the brightness, contrast, etc.

Image Restoration
Image restoration is the process of improving the appearance of an image. However, unlike
image enhancement, image restoration is done using certain mathematical or probabilistic
models.

Color Image Processing

Color image processing includes a number of color modeling techniques in a digital domain.
This step has gained prominence due to the significant use of digital images over the internet.

Wavelets and Multiresolution Processing

Wavelets are used to represent images in various degrees of resolution. The images are
subdivided into wavelets or smaller regions for data compression and for pyramidal
representation.

Compression
Compression is a process used to reduce the storage required to save an image or the bandwidth
required to transmit it. This is done particularly when the image is for use on the Internet.

Morphological Processing
Morphological processing is a set of processing operations for morphing images based on their
shapes.

Segmentation
Segmentation is one of the most difficult steps of image processing. It involves partitioning an
image into its constituent parts or objects. 

Representation and Description

After an image is segmented into regions in the segmentation process, each region is represented
and described in a form suitable for further computer processing. Representation deals with the
image’s characteristics and regional properties. Description deals with extracting quantitative
information that helps differentiate one class of objects from the other.

Recognition
Recognition assigns a label to an object based on its description.

Components of Image Processing

Computer
A general-purpose computer, which may be anything from a PC to a supercomputer, is used in
an image processing system. Sometimes, specifically built computers are utilized in specialized
applications to reach a specified degree of performance.

Hardware for Specialized Image Processing

It comprises the digitizer and hardware that can carry out basic operations, including an
Arithmetic Logic Unit (ALU), which can carry out simultaneous arithmetic and logical
operations on whole pictures.

Massive Storing
In applications involving image processing, the skill is essential. The three main types of digital
storage for image processing applications are as follows: Three types of storage exist (1) short-
term storage, (2) online storage for quick recall (3) archive storage, which is characterized by
rare access.

Camera Sensors
It alludes to perception. The image sensor's primary function is to collect incoming light,
transform it into an electrical signal, measure that signal, and then output it to supporting
electronics. It consists of a two-dimensional array of light-sensitive components that convert
photons into electrons. Images are captured by equipment like digital cameras using image
sensors like CCD and CMOS. Two components are often needed on image sensors to collect
digital pictures. The first is an actual tool (sensor) that can detect the energy emitted by the
object we want to turn into an image. The second is a digitizer, which transforms a physical
sensing device's output into digital form.

Image Display
The pictures are shown.

Software
The image processing software comprises specialized modules that carry out particular functions.

Hardcopy Equipment
Laser printers, film cameras, heat-sensitive equipment, inkjet printers, and digital equipment like
optical and CDROM discs are just a few examples of the instruments used to record pictures.

Networking
To send visual data through a networked computer, it is a necessary component. The most
important factor in picture transmission is bandwidth since image processing applications require
vast amounts of data.
Blind Deconvolution in Image Processing
Blind image deconvolution is the challenge of recovering a clear picture from a blurry and noisy
one without knowing precisely how the image was blurred, as obtained by an ideal pinhole
camera. The unidentified blurring operation might be brought on by defocus, camera movement,
scene motion, or other optical defects. A trade-off between exposure duration and aperture
setting is necessary for proper photography exposure. The photographer might utilize a big
aperture or a lengthy exposure period when the lighting is bad. The first option produces motion
blur when the camera moves concerning objects in the scene while the exposure takes place.
When using the second option, things farther from the focal plane become out-of-focus blurred.
This may lead to blind deconvolution in image processing.
Importance of Phase in Image Processing
Phase, in a nutshell, contains information about the positions of features. Phase-only and
magnitude-only photos cannot be combined to produce the original. To obtain the original,
multiply them in the Fourier domain and reverse the transformation. A repeated waveform's
phase describes the position or timing of a particular point within a wave cycle. Instead of the
actual absolute phases of the signals, the phase difference between waves usually matters.

Ringing Effect in Image Processing

The unpleasant ringing effect often referred to as the Gibbs phenomenon in mathematical
approaches to image processing, is an artifact that appears as rippling ripples close to sharp edges
in photos and videos. The loss or distortion of high-frequency information in the image results in
this effect.

• An image processing system in the form of a knowledge database.

.9
Color processing
• Basics of color
• Color models in images
• Color models in video

Basics of color
(a) Light and spectra
• Color is the perceptual result of light in the
visible region of the spectrum, having in the
region of 400nm to 700nm, incident upon the
retina.
• Visible Light is a form of electromagnetic
energy consisting of a spectrum of frequencies
having wavelengths range from about 400nm for
violet light to about 700nm for red light.
• Most light we see is a combination of many
wavelengths.
(b) Primaries
• Any color can be matched by proper proportions
of three component colors called primaries.
• The most common primaries are red, blue and
green.

• The following terms are used to define color

light:
1. Brightness or Luminance: This is the amount
of light received by the eye regardless of
color.
2. Hue: This is the predominant spectral color in
the light.
3. Saturation: This indicates the spectral purity
of the color in the light.

• In 1931, the CIE adopted a set of nonphysical

primaries, X, Y and Z.
Color models in images:
• A color image is a 2D array of (R,G,B) integer
triplets.
• CRT displays have 3 phosphors (RGB) which
produce a combination of wavelengths when excited
with electrons.
• CMY model, which uses Cyan, Magenta and
Yellow primaries, is mostly used in printing devices
where the color pigments on the paper absorb
certain colors.

• Sometimes, an alternative CMYK model (K stands

for black) is used in color printing to produce a
darker black than simply mixing CMY.

Color models in video

• YIQ and YUV are the 2 commonly used color
models in video.
(a). YIQ Model
• YIQ is used in color TV broadcasting, which is
downward compatible with B/W TV where only
Y is used.

• Y (luminance) is the CIE Y primary.

(b). YUV (YCbCr) model
• They are initially for PAL analog video, but it's
now used in CCIR 601 standard for digital
video.
• Y = 0.299R+0.587G+0.114B
Cb = B-Y
Cr = R-Y
• Y = 0.299R+0.587G+0.114B

U = 0.565(B-Y)
V = 0.713(R-Y)

(c). Chroma subsampling

Horizontally subsampled color signals

by a factor of 2.
• 4:1:1 Horizontally subsampled by a factor of
4
• 4:2:0 Subsampled in both the horizontal and
vertical axes by a factor of 2 between
pixels as shown in the figure.
• 4:1:1 and 4:2:0 are mostly used in JPEG and
MPEG.

A simple image model:

• To be suitable for computer processing, an image f(x,y) must be digitalized both spatially and
in amplitude.
• Digitization of the spatial coordinates (x,y) is called image sampling. • Amplitude digitization
is called gray-level quantization.
Elements of digital image processing systems:
• The basic operations performed in a digital image
processing systems include (1) acquisition, (2)
storage, (3) processing, (4) communication and (5)
display.

Applications of image processing:

Understanding the Diversity of Applications
From medical diagnoses to virtual reality experiences, multimedia image processing plays a
pivotal role in diverse fields.

• Interest in digital image processing methods stems from 2 principal application areas:
(1) improvement of pictorial information for human interpretation, and

(2) (2) processing of scene data for autonomous machine perception. • In the second application
area, interest focuses on procedures for extracting from an image information in a form
suitable for computer processing.
(3) • Examples include automatic character recognition,
(4) industrial machine vision for product assembly and inspection,
(5) military recognizance,
(6) automatic processing of fingerprints etc.

Enhancing Visual Quality: Noise Reduction

Image noise, like grain in low-light photographs, degrades quality. Denoising techniques, such as
Gaussian filters, remove noise while preserving essential details.

Example: In medical imaging, reducing noise in X-rays enhances the visibility of subtle
abnormalities.

Object Identification: Facial Recognition

Facial recognition algorithms identify individuals based on facial features. These algorithms
have applications in security, access control, and social media tagging.

Example: Apple's Face ID uses facial recognition to unlock iPhones securely.

Real-world Examples and Their Impact

In augmented reality (AR), image processing aligns virtual objects with the real world. AR apps
like Pokémon GO overlay virtual creatures onto the physical environment, enhancing user
experiences.

3.1 Types of Image Processing

Preprocessing: Noise Reduction

Noise reduction techniques, like median filtering, remove unwanted elements, enhancing the
image's clarity.

Example: Astronomical images often require noise reduction to unveil distant celestial objects.
Enhancement: Contrast Adjustment
Contrast enhancement techniques modify pixel intensities to improve visual perception of
details.

Example: Enhancing satellite images helps detect changes in land use or deforestation.

Restoration: Image Deblurring

Image deblurring techniques recover details from blurred images caused by motion or defocus.

Example: Medical scans can be deblurred to make subtle structures visible.

Compression: JPEG vs. PNG

Image compression techniques like JPEG and PNG reduce file size while preserving image
quality.

Example: JPEG is commonly used for photographs on websites due to its balance between
quality and size.

Transformation: Fourier Transforms

Fourier transforms convert images into frequency domains, useful for analyzing periodic
patterns.

Example: Fourier analysis helps identify specific frequencies in medical EEG signals.

Recognition: Object Detection

Object detection techniques locate and classify objects within images.

Example: Self-driving cars use object detection to identify pedestrians and other vehicles.

3.2 Image Filtering and Segmentation

Filtering Techniques: Sobel Operator
The Sobel operator enhances edges in images, which is crucial for feature extraction.

Example: Edge detection in medical images aids in identifying tumors.

Image Segmentation Algorithms: Watershed Algorithm

The watershed algorithm segments images into distinct regions based on local intensity
gradients.

Example: Cell segmentation in microscopic images assists in studying cell behavior.

4.1 Image Filtering and Enhancement

Edge Detection: Canny Edge Detector

The Canny edge detection algorithm identifies edges in images by locating areas of rapid
intensity changes.

Example: Edge detection helps detect road lanes in self-driving cars.

Histogram Equalization: Improving Contrast

Histogram equalization redistributes pixel intensities, enhancing image contrast.

Example: Improving the contrast of an X-ray can reveal hidden fractures.

4.2 Image Restoration and Compression

Create an enhanced image from the original image
Deconvolution: Image Deblurring
Deconvolution algorithms restore details lost due to blurring, such as in astronomical images.

Example: Restoring blurred medical images improves diagnosis accuracy.

Lossy vs. Lossless Compression: Trade-offs and Applications
Lossy compression sacrifices some image details for higher compression ratios, while lossless
compression retains all information.

Example: Lossy compression is suitable for web images, while lossless compression is
preferred for medical images.

4.3 Pattern Recognition and Object Detection

Pattern recognition measures the various patterns around the objects in the image
Haar Cascades: Face Detection
Haar cascades are used for real-time face detection by identifying features like eyes and noses.

Example: Face detection is employed in security systems for access control.

Convolutional Neural Networks (CNNs) for Object Recognition

CNNs learn features from images and classify objects with high accuracy.
Example: CNNs are used in autonomous vehicles to identify pedestrians and road signs.

Applications of Image Processing

Medical Image Retrieval-- MRI and CT Scan Analysis
Image processing enhances medical images, aiding doctors in diagnosing conditions.
Example: Image processing assists in identifying tumors in MRI scans.
Image processing has been extensively used in medical research and has enabled more efficient
and accurate treatment plans. For example, it can be used for the early detection of breast cancer
using a sophisticated nodule detection algorithm in breast scans. Since medical usage calls for
highly trained image processors, these applications require significant implementation and
evaluation before they can be accepted for use.
Traffic Sensing Technologies
In the case of traffic sensors, we use a video image processing system or VIPS. This consists of
a) an image capturing system b) a telecommunication system and c) an image processing system.
When capturing video, a VIPS has several detection zones which output an “on” signal whenever
a vehicle enters the zone, and then output an “off” signal whenever the vehicle exits the detection
zone. These detection zones can be set up for multiple lanes and can be used to sense the traffic
in a particular station.

Left - normal traffic image | Right - a VIPS image with detection zones

Besides this, it can auto record the license plate of the vehicle, distinguish the type of vehicle,
monitor the speed of the driver on the highway and lots more.

Image Reconstruction
Image processing can be used to recover and fill in the missing or corrupt parts of an image. This
involves using image processing systems that have been trained extensively with existing photo
datasets to create newer versions of old and damaged photos.
 Fig: Reconstructing damaged images using image processing

Face Detection
One of the most common applications of image processing that we use today is face detection. It
follows deep learning algorithms where the machine is first trained with the specific features of
human faces, such as the shape of the face, the distance between the eyes, etc. After teaching the
machine these human face features, it will start to accept all objects in an image that resemble a
human face. Face detection is a vital tool used in security, biometrics and even filters available
on most social media apps these days.

Entertainment and Media

Visual Effects in Movies and Games
Image processing techniques contribute to CGI effects, bringing imaginary worlds to life.

Example: CGI enhances the realism of creatures in movies like "Avatar."

Surveillance and Security

Video Analytics and Anomaly Detection
Image processing enables automated surveillance systems to detect unusual behavior.

Example: Anomaly detection can alert security personnel to suspicious activities.

Remote Sensing and GIS

Satellite Imagery Analysis for Environmental Monitoring

Image processing aids in monitoring deforestation, urban growth, and natural disasters.

Example: Identifying changes in land use patterns using satellite imagery.

Industrial Inspection and Quality Control

Defect Detection in Manufacturing

Image processing systems identify defects in products, ensuring quality control.

Example: Inspecting semiconductor wafers for manufacturing defects.

Benefits of Image Processing

The implementation of image processing techniques has had a massive impact on many tech
organizations. Here are some of the most useful benefits of image processing, regardless of the
field of operation:

 The digital image can be made available in any desired format (improved image, X-
Ray, photo negative, etc)
 It helps to improve images for human interpretation
 Information can be processed and extracted from images for machine interpretation
 The pixels in the image can be manipulated to any desired density and contrast
  Images can be stored and retrieved easily
 It allows for easy electronic transmission of images to third-party providers

Challenges and Future Directions

Handling Big Data

Real-time Processing of Massive Image Streams
Big data challenges arise in processing real-time data from multiple sources like surveillance
cameras.
Example: Traffic management systems analyze real-time camera feeds to optimize traffic flow.

Real-time Image Processing

Real-time Object Tracking
Real-time image processing is crucial for tracking moving objects like vehicles and pedestrians.

Example: Autonomous vehicles use real-time image processing for safe navigation.

Integration of AI and Machine Learning

Deep Learning for Image Classification

Deep learning models like CNNs and GANs enable accurate image classification and synthesis.

Example: GANs are used for generating realistic images, like in the "This Person Does Not
Exist" project.

7.2 Future Prospects and Research Areas

Emerging Technologies:
Virtual Reality, Augmented Reality
Image processing will be integral to shaping the immersive experiences of virtual and augmented
reality applications.
Ethical Considerations in Image Manipulation and AI Integration
As AI-driven image synthesis advances, ethical concerns regarding fake media and privacy
become increasingly important.

Acknowledgments
Special thanks to the pioneers in the field of multimedia image processing and the researchers
who continue to advance this fascinating discipline.

This concludes the concept on multimedia image processing. We've delved into the foundational
concepts, historical developments, diverse applications, challenges, and exciting future directions
of this dynamic field. As technology continues to evolve, the role of image processing in
reshaping how we perceive and interact with visual information remains ever more critical.

Reference:
1. Gonzalez, R.C., & Woods, R.E. (2018). "Digital Image Processing". Pearson Education.
2. Szeliski, R. (2010). "Computer Vision: Algorithms and Applications". Springer.
3. Pratt, W.K. (2001). "Digital Image Processing". John Wiley & Sons.
4. OpenCV Documentation: [https://docs.opencv.org/](https://docs.opencv.org/)
5. IEEE Xplore: [https://ieeexplore.ieee.org/](https://ieeexplore.ieee.org/)
6. Goodfellow, I., Bengio, Y., & Courville, A. (2016). "Deep Learning". MIT Press.
7. Coursera - Digital Image Processing: [https://www.coursera.org/](https://www.coursera.org/)
8. Watt, A., & Policarpo, F. (2016). "3D Computer Graphics". Addison-Wesley Professional.
9. Vincent, L., & Soille, P. (1991). "Watersheds in Digital Spaces: An Efficient Algorithm Based
on Immersion Simulations". IEEE Transactions on Pattern Analysis and Machine Intelligence.
10. Kirsch, R.A. (1957). "A Program for Aligning Pictures on a Digital Computer". Journal of
the ACM.
11. Pratt, W.K. (2001). "Digital Image Processing". John Wiley & Sons.
12. Szeliski, R. (2010). "Computer Vision: Algorithms and Applications". Springer.
13. Gonzalez, R.C., & Woods, R.E. (2018). "Digital Image Processing". Pearson Education.
14. Farid, H. (2020). "Photo Forensics". MIT Press.
15. Canny, J. (1986). "A Computational Approach to Edge Detection". IEEE Transactions on
Pattern Analysis and Machine Intelligence.
16. Viola, P., & Jones, M. (2001). "Rapid Object Detection using a Boosted Cascade of Simple
Features". CVPR.
17. LeCun, Y., Bengio, Y., & Hinton, G. (2015). "Deep Learning". Nature.
18. Pitas, I. (2000). "Digital Image Processing Algorithms and Applications". John Wiley &
Sons.