DIGITAL IMAGE PROCESSING

WORKSHOP
DAY 2
 Topics to be covered
• Image Thresholding
• Image Enhancement techniques
-> Filtering
1. Smoothing
2. Sharpening
• Edge detection
IMAGE THRESHOLDING
 Thresholding
• Image thresholding is a simple, yet effective, way of partitioning an
image into a foreground and background.
• Simple Thresholding:
– If pixel value is greater than a threshold value, it is assigned one value (may be
white), else it is assigned another value (may be black).
– The function used is cv.threshold.
– First argument is the source image, which should be a grayscale image.
– Second argument is the threshold value which is used to classify the pixel
values.
– Third argument is the maxVal which represents the value to be given if pixel
value is more than (sometimes less than) the threshold value.
– Different types are:
• cv.THRESH_BINARY
• cv.THRESH_BINARY_INV
• cv.THRESH_TRUNC
• cv.THRESH_TOZERO
• cv.THRESH_TOZERO_INV
 Thresholding
import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt
img = cv.imread('gradient.png',0)
ret,thresh1 = cv.threshold(img,127,255,cv.THRESH_BINARY)

titles = [‘ Original_Img ',‘ BINARY ']

images = [img, thresh1]
for i in xrange(2):
plt.subplot(2,3,i+1),plt.imshow(images[i],‘ gray ’)
plt.title(titles[i])
plt.xticks([]),plt.yticks([]) Different types of Simple Thresholding

plt.show()
 Thresholding
• Adaptive Thresholding:
– In simple thresholding, we used a global value as threshold value. But
image might have different lighting conditions in different areas.
– In that case, we go for adaptive thresholding. We calculate the threshold
for a small regions of the image. So we get different thresholds for
different regions of the same image.
– It has three ‘special’ input parameters and only one output argument:
• Adaptive Method - It decides how thresholding value is calculated.
– cv.ADAPTIVE_THRESH_MEAN_C : threshold value is the mean of neighbourhood area.
– cv.ADAPTIVE_THRESH_GAUSSIAN_C : threshold value is the weighted sum of
neighbourhood values where weights are a gaussian window.
• Block Size - It decides the size of neighbourhood area.
• C - It is just a constant which is subtracted from the mean or weighted mean
calculated.
Thresholding

Different types of Adaptive

Thresholding
IMAGE ENHANCEMENT TECHNIQUES
Image Enhancement Techniques
 IMAGE FILTERING
• Filtering is a technique for modifying or enhancing an image.
• It is a neighbourhood operation, in which the value of any given pixel in
the output image is determined by applying some algorithm to the
values of the pixels in the neighbourhood of the corresponding input
pixel.
• Image Filtering is basically of two types, Linear Filtering and Non-Linear
Filtering.
• Image processing operations implemented with filtering include
Smoothing, Sharpening and Edge enhancement
 Convolution
• The heart of the image filtering is the convolution operation.
• It consists of an input image and a small 2D array or matrix
called kernel or mask.
• The convolution is performed by sliding the kernel over the
image, generally starting at the top left corner, so as to move
the kernel through all the positions where the kernel fits
entirely within the boundaries of the image.
In the above example, there is an input image and a kernel, convolution is performed by sliding
the kernel over the entire image.

Value of the resultant operation is assigned to the upper left corner pixel so if we are just
considering a weighted sum between kernel and input image then kernel operation applied on
the I57 pixel of the input image will look like the one given below:
• Filters could be either smoothing, or sharpening.
• Smoothing filters are used in blurring.
• Sharpening filters are used in edge detection.
INTRODUCTION TO SMOOTHING
• Smoothing, also called blurring, is a simple and frequently used
image processing operation. It is usually done to reduce noise or
other fine-scale structures/rapid phenomena.
• Image blurring is achieved by convolving the image with a low-pass
filter kernel. It is useful for removing noise. It actually removes high
frequency content (e.g: noise, edges) from the image resulting in
edges being blurred when this is filter is applied. (Well, there are
blurring techniques which do not blur edges).
 Spatial Domain Filtering
• Spatial domain filtering means applying some operation on the
neighbourhood pixels and assigning the result to the pixel
whose neighbourhood was considered.
• Some neighborhood operations work with the values of the
image pixels in the neighborhood and the corresponding
values of a sub image that has the same dimensions as the
neighborhood.
• The sub image is called a filter, mask, kernel, template, or
window, with the first three terms being the most prevalent
terminology.
• The sub image or filter is just a 2D matrix which is hovered
over the whole image pixel by pixel, an operation between
pixel values and matrix values occurs and result is assigned to
the pixel at the centre of that matrix (generally).
• The values in a filter sub-image are referred to as coefficients,
rather than pixels.
 Linear Spatial Filtering
• Linear filtering is filtering in which the value of an output pixel
is a linear combination of the values of the pixels in the input
pixel's neighborhood. As in the example of convolution.
• That is, the spatial operation performed by the filter is a linear
combination of the pixel’s neighbourhood.
 Frequency components in an Image
• “Frequency” means the rate of change of intensity per pixel.
Let’s say you have some region in your image that changes
from white to black. If it takes many pixels to undergo that
change, it’s low frequency. The fewer the pixels it takes to
represent that intensity variation, the higher the frequency.

Another way to think of it is to take a line across your image

and plot the intensity values (just in your head) to get a feel for
the frequency.
• Example:
if you take a line across this:

• The intensity variation follows a sinusoid:

• Alternatively, if you take a line across your image and get
something like this:

• These edges represent pretty high frequency features; the pixel

intensity changes from low to high across 2 pixels only. In fact,
this is the highest frequency possible in any image: a 2 pixel
transition (also called the Nyquist frequency).
• Sharp contrast edges are high frequencies, smooth gradients
are low frequencies.
• A flat, single color image would have the lowest frequency. An
image with a black and white checkerboard pattern at the pixel
level would have the highest frequency.
 Smoothing Linear Filters
• These are low-pass filters. They reject the high frequency
components in the image i.e. sharp edges and pixel variations
which occur due to the presence of noise in the image.
• The output (response) of a smoothing, linear spatial filter is the
average of the pixels contained in the neighborhood of the filter
mask.
• Also called averaging filters.
• Primary objective is removal of noise and smoothing of false
contours that result from using an insufficient number of gray
levels.
• Standard Average Smoothing filter:

• Weighted average smoothing filter:

• This is also called Gaussian Blur.
 Gaussian Blur
• Gaussian blur/smoothing is the most commonly used smoothing
technique to eliminate noises in images and videos. In this
technique, an image should be convolved with a Gaussian kernel to
produce the smoothed image.

• You may define the size of the kernel according to your

requirement. But the standard deviation of the Gaussian
distribution in X and Y direction should be chosen carefully
considering the size of the kernel such that the edges of the kernel
is close to zero.
 Gaussian blur
• A 3 x 3 and a 5 x 5 Gaussian kernel:

3 x 3 Gaussian Kernel 5 x 5 Gaussian Kernel

• You have to choose a right size of the kernel to define the

neighbourhood of each pixel. If it is too large, small features of the
image may be disappeared and the image will look blurred. If it is too
small, you cannot eliminate noises in the image.
Code for Gaussian Blur
 Gaussian blur applied on Images

Original Gaussian blurred

 Non- Linear Spatial Filtering
• Output of filter assigned to the pixel is not a linear combination of the
neighborhood pixel values of the pixel under consideration.
• The mask is used to obtain the neighboring pixel values, and then ordering
mechanisms produce the output pixel.
• An example:
Median Filter: replaces the value of a pixel by the median (50th percentile) of the
gray levels in the neighborhood of that pixel.
• The original value of the pixel is included in the computation of the median.
• Effective in the presence of impulse noise, also called salt-and-pepper noise.
• Similarly we can have max (100th percentile) and min (0th percentile) filtering.
 MEDIAN BLUR
• Here, the function cv2.medianBlur() takes median of all the
pixels under kernel area and central element is replaced with
this median value. Its kernel size should be a positive odd
integer.
 Code for Median Blur
import cv2
import numpy as np
cap = cv2.VideoCapture(0)
while (1):
_ , frame = cap.read ()
hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
lower_red = np.array ([30,150,50])
upper_red = np.array ([255,255,180])
mask = cv2.inRange ( hsv, lower_red, upper_red )
res = cv2.bitwise_and ( frame, frame, mask= mask )
kernel = np.ones (( 15,15 ), np.float 32) / 225
median = cv2.MedianBlur ( res,15 )
cv2.imshow( ‘ Original ’, frame )
cv2.imshow( ‘Median Blur ’ , median )
k = cv2.waitKey(5) & 0xFF
if k == 27:
break

cv2.destroyAllWindows()
cap.release ()
 Median blur applied on Images

In this demo, We added a 50% noise MEDIAN FILTER APPLICATION

to our original image and applied
median blur.
Comparisons
 Frequency Domain Filtering
• 1D and 2DFourier Transform
• 1D Discrete Fourier Transform
• 2D Discrete Fourier Transform
• Low pass filtering  Smoothing
• High pass filtering  Sharpening
 Gaussian Blur in Frequency Domain
• Since the Fourier transform of a Gaussian is another Gaussian,
applying a Gaussian blur has the effect of reducing the image's
high-frequency components.
• A Gaussian blur is thus a low pass filter.
• Gaussian function in 1D:

• Gaussian function in 2D:

• Gaussian blur in frequency domain and correspondingly in
spatial domain:
Introduction to sharpening
• Image sharpening (high pass filtering) is defined as enhancing the
high-frequency components of an image so that the edges and fine
details in it get highlighted thereby improving the visual quality.
• Image sharpening is widely used in printing and photographic
industries for increasing the local contrast and sharpening the
images.
• Image sharpening consists of adding to the original image a signal
that is proportional to a high-pass filtered version of the original
image.
• The sharpening operation can be represented by:
• Where xi,j is the original pixel value at the coordinate (i,j). F(.) is the high-
pass filter, λ is a tuning parameter greater than or equal zero, and Si,j is the
sharpened pixel at the coordinate .
• Increasing the value of λ yields a more sharpened image. The key point in
the effective sharpening process lies in the choice of the high-pass filtering
operation.
• Traditionally, linear filters have been used to implement the
high-pass filter, however, linear techniques can lead to unacceptable
results if the original image is corrupted with noise.
• A tradeoff between noise attenuation and edge highlighting can be
obtained if a weighted median filter with appropriated weights is used.
• To illustrate this, a weighted median filter is applied to a gray-scale image
where the following filter mask is used.
• The filtered image multiplied with some λ factor is then added
to the original image to get the final
sharpened image
• Alternate method is to apply a sharpening kernel as given
below:
# Create our shapening kernel, it must equal to one eventually

kernel_sharpening = np.array([[-1,-1,-1],
[-1, 9,-1],
[-1,-1,-1]])

# applying the sharpening kernel to the input image & displaying it.

sharpened = cv2.filter2D(image, -1, kernel_sharpening)cv2.imshow('Image Sharpening', sharpened)

cv2.imshow('Image Sharpening', sharpened)

INTRODUCTION TO EDGE DETECTION
 What is Edge Detection
• What are edges
– We can also say that sudden changes of discontinuities in an image are called as edges.
Significant transitions in an image are called as edges
• Types of edges
Generally edges are of three types:
– Horizontal edges
– Vertical Edges
– Diagonal Edges
• Why detect edges Horizontal edges Vertical edges
Most of the shape information of an image is enclosed in edges. So first we detect these
edges in an image and by using these filters and then by enhancing those areas of image
which contains edges, sharpness of the image will increase and image will become clearer.
 What is Edge Detection
• Edge detection is an image processing technique for finding
the boundaries of objects within images.
• It works by detecting discontinuities in brightness. The points
at which image brightness changes sharply are typically
organized into a set of curved line segments termed edges.
• Common edge detection algorithms include Sobel, Canny,
Prewitt, Robinson Compass, Krisch Compass, Laplacian and
fuzzy logic methods.
 Relating Edges and Derivatives
• The lower part is the 1-D image. The upper part is the intensity of each pixel of
the 1-D image plotted as a graph. Blacks have a low intensity, so the graph curve
is low. It reaches full height at the white end of the image. Note that the center
of the curve has a steep slope meaning you've got an edge.

• Looking for these peaks is exactly what gradient based edge detection methods
do. You’re not interested in what the actual colors are.
 Relating Edges and Derivatives
• There has to be a certain threshold above which an edge is
classified as a peak else it must be considered part of noise.
• On the left (where the curve is rising), the slope
is positive. On the right, the slope is negative.
• So there must exist a point where there is a
zero crossing. That point is the edge's location.
Edge detectors that are based on this idea
are called Laplacian edge detectors.
• Now, all of this is for 1-D images and holds for 2-D images as well.
Another thing is - these are based on continuous images. But in
reality, that is never the case. So we'll have to approximate these
derivatives based on the pixelated data that we do have. This is
done with the help of convolutions.
• When taking the derivative of an image, you’re actually taking
what’s called a discrete derivative, and it’s more of an
approximation of the derivative. One simple example is that you
can take the derivative in the x-direction at pixel x1 by taking the
difference between the pixel values to the left and right of your
pixel (x0 and x2).
• It’s easiest to see how the image derivative is useful in locating
edges. The derivative of a function tells you the rate of change, and
an edge constitutes a high rate of change.
• The derivative of an image has two dimensions. We can take the
derivative in the x direction and in the y direction, and together
these make up the “gradient vector”:
• Going back to edge detection, the gradient direction gives you the normal to the
edge (it’s perpendicular to the edge), and the gradient magnitude gives you the
strength of the edge.
• To compute the derivative with respect to x at a given pixel, it sounds like in
practice, to reduce the affects of noise, we:
-> Use the “central difference”
-> Average the derivative of the pixel with that of the row above and row below:
• Apply the mask
• Again, we could just use the middle row for fx and the middle
column for fy, but we include the surrounding pixels to help reduce
the affect of noise.
• We can’t apply the mask to the border pixels so we don’t, the
derivative at those pixels is set to 0.

• Adding the effects of x and y-direction gradients, we get the overall

edges present in the images: or simply
where Gx and Gy are gradients in x and y direction.
 Sharpening Spatial Filters
• The principal objective of sharpening is to highlight fine detail in an image
or to enhance detail that has been blurred.
• Sharpening is accomplished by spatial differentiation.
• Image differentiation enhances edges and other discontinuities (such as
noise) and deemphasizes areas with slowly varying gray-level values.
• A basic definition of the first-order derivative of a one-dimensional
function f(x) is the difference

• Similarly, we define a second-order derivative as the difference

 Second order derivative - Laplacian Filter
• Laplacian function:
• X Direction:

• Y Direction:

• Summing the two:

 Laplacian Edge Detector
• Unlike the Sobel edge detector, the Laplacian edge detector uses
only one kernel. It calculates second order derivatives in a single
pass. Here's the kernel used for it:

• You can use either one of these. Or if you want a better

approximation, you can create a 5x5 kernel (it has a 24 at the
center and everything else is -1).
 Laplacian Edge Detector

• One serious drawback - because we're working with second order

derivatives, the laplacian edge detector is extremely sensitive to
noise. We have to reduce noise - maybe using the Gaussian blur.
• Laplacians are computationally faster to calculate (only one kernel
vs two kernels) and sometimes produce exceptional results.
Laplacian Edge Detector
 First order derivative
• For a 3X3 mask, gradient is

• Sobel Edge Detector: The Sobel operator performs a 2-D spatial

gradient measurement on an image and so emphasizes regions of high
frequency that correspond to edges.
• The sobel operator consists of a pair of 3×3 convolution kernels. One
kernel is simply the other rotated by 90°.
 Sobel Edge Detector
• The Sobel operator performs a 2-D spatial gradient measurement
on an image and so emphasizes regions of high frequency that
correspond to edges.
• Typically it is used to find the approximate absolute gradient
magnitude at each point in an input grayscale image.
• The operator consists of a pair of 3×3 convolution kernels. One
kernel is simply the other rotated by 90°.
 Sobel Edge Detector
• These kernels are designed to respond maximally to edges
running vertically and horizontally relative to the pixel grid, one
kernel for each of the two perpendicular orientations.
• The kernels can be applied separately to the input image, to
produce separate measurements of the gradient component in
each orientation (call these Gx and Gy).
• These can then be combined together to find the absolute
magnitude of the gradient at each point and the orientation of
that gradient. The gradient magnitude is given by:
 Sobel Edge Detector
• Typically, an approximate magnitude is computed using:

• The angle of orientation of the edge (relative to the pixel

grid) giving rise to the spatial gradient is given by:

Results of sobel edge detection on (a)original image: (b)vertical edges, (c)horizontal edges
 Sobel Edge Detector
import cv2
import numpy as np

img = cv2.imread (‘image.bmp’, cv2.IMREAD_GRAYSCALE )

rows, cols = img.shape

sobel_horizontal = cv2.Sobel( img, cv2. CV_64F, 1, 0, ksize=5 )

sobel_vertical = cv2.Sobel( img, cv2. CV_64F, 1, 0, ksize=5 )

cv2.imshow(‘Original’, img )
cv2.imshow(‘Sobel horizontal’, sobel_horizontal )
cv2.imshow(‘Sobel vertical’, sobel_vertical )

cv2.waitKey (0)
Sobel and laplacian filter overview
CANNY EDGE DETECTION
 Canny Edge Detector
• Overview
– The canny edge detector is a multistage edge detection algorithm.
The steps are:
• Preprocessing
• Calculating gradients
• Nonmaximum suppression
– Thresholding with hysterysis
– The two key parameters of the algorithm are - an upper threshold
and a lower threshold. The upper threshold is used to mark edges
that are definitely edges. The lower threshold is to find faint pixels
that are actually a part of an edge.
• Preprocessing

– Edge detectors are are prone to noise. A bit of smoothing with a

Gaussian blur helps. From what I've seen, software packages don't do
this step automatically, you need to do this yourself.
– Usually, a 5x5 Gaussian filter with standard deviation = 1.4 is used for
this purpose.
• Calculating gradients
– Next, gradient magnitudes and directions are calculated at every single
point in the image. The magnitude of the gradient at a point determines if
it possibly lies on an edge or not. A high gradient magnitude means the
colors are changing rapidly - implying an edge. A low gradient implies no
substantial changes. So it's not an edge.
– The direction of the gradient shows how the edge is oriented.
– To calculate these, the standard sobel edge detector is used.
– The magnitude of gradient is. .

– The direction of gradient is.

– Here, Gx and Gy are the X and Y derivatives at the point being considered.
– Once we have the gradient magnitudes and orientations, we can get
started with the actual edge detection.
• Non-maximum suppression
– If a pixel is not a maximum, it is suppressed.
– To do this, you iterate over all pixels.
– Let's say you're at the grey pixel. There are only four possible edges
possible - it could either go from north to south (the green
neighbours), from east to west (the blue ones) or one of the
diagonals (the yellow or red neighbours).
– So using the current pixel's gradient direction, you try and estimate
where the edge is going.
• Thresholding with Hysteresis
– This stage decides which are all edges are really edges and which are not. For this,
we need two threshold values, minVal and maxVal.
– Any edges with intensity gradient more than maxVal are sure to be edges and those
below minVal are sure to be non-edges, so discarded.
– Those who lie between these two thresholds are classified edges or non-edges based
on their connectivity. If they are connected to “sure-edge” pixels, they are
considered to be part of edges. Otherwise, they are also discarded.

• The edge A is above the maxVal, so considered as

“sure-edge”.
• Although edge C is below maxVal, it is connected to
edge A, so that also considered as valid edge and
we get that full curve.
• But edge B, although it is above minVal and is in
same region as that of edge C, it is not connected to
any “sure-edge”, so that is discarded.
 Canny Edge Detector
 OpenCV puts all the above in single function, cv2.Canny().
 First argument is our input image.
 Second and third arguments are our minVal and maxVal respectively.
 Four argument is aperture_size. It is the size of Sobel kernel used for find
image gradients. By default it is 3.
 Last argument is L2gradient which specifies the equation for finding
gradient magnitude. If it is True, it uses the equation mentioned above
which is more accurate, otherwise it uses this function: . By default, it is
False.
Canny Edge Detector