MECHATRONICS

MODULE –vI
 SYLLABUS
Module –VI

Robotic vision system - Image acquisition: Vidicon, charge coupled device (CCD)
and charge injection device (CID) cameras. Image processing techniques:
histogram processing: sliding, stretching, equalization and thresholding.

Case studies of Mechatronics systems: Automatic camera, bar code reader, pick
and place robot, automatic car park barrier system, automobile engine
management system.
 ROBOTIC VISION
• Robot vision may be defined as the process of extracting, characterizing, and
interpreting information from images of a three dimensional world. Four
steps to General Purpose Robot Vision are
 Image formation
 Processing of image in a form suitable for analysis by computer
 Defining and analyzing the characteristics of image
 Interpretation of image and decision making.
• The image of the object is sensed using any image sensing devices like camera.
This image is send to memory as frames .After acquiring the image it is
processed in computer to improve the quality which includes adjusting of
brightness, contrast, sharpness, etc. Final processed image is taken to output
device.
 IMAGE ACQUISITION

• Image acquisition is the creation of digital images , typically from a

physical scene.
• The first link in the vision chain is the camera . It plays the role of robotic
eye or the sensor. This is the imaging component or the non contact or
remote sensor.
• The visual information is converted into electric signals in the camera and
when Sampled Spatially and quantized, these signals give a digital image in
real time by a process called digitizing. The robotic vision cameras are
essentially optoelectronic transducers which convert optical input signal to
electrical output signal . They fall in the domain of TV cameras. There is a
variety of camera technologies available for imaging.

E.g.:- CCD, CID

 VIDICON TUBE

• The basic structure of the vidicon camera tube is shown in the figure. The
optical image is formed on the glass faceplate coated with a thin
photosensitive layer composed of a large number of tiny photo resistive
elements. The resistance of the element decreases with increasing
illumination. Once the image forms on the faceplate, a charge is
accumulated, which is function of the intensity of the impinging the light
over a specified time, from which an electrical video signal is derived.
• The charge built up is read by scanning the photo sensitive layer by
focused electron beam produced by electron gun at the rear of the tube.
The scanning is controlled by a deflection coil mounted along the length of
the tube. The electron beam is made to scan the entire surface, typically ,
30 times per second per line
WORKING
• This camera tube based on the photo conductive properties of semiconductors i.e., decrease
in resistance with the amount of incident light. The tube is shown in figure. It consists of –
• (a). Signal Plate: Which is a conducting metallic film very thin so as to be transparent. The
side of this film facing cathode is coated with a very thin layer of photoconductive material
(amorphous selenium). This side is scanned by electron beam. The optical image is focused
on the other side of this film.

• (b).Scanning System: The electron beam for scanning is formed by the

combination of cathode, control grid-1, accelerating grid-2 and anode grid-3. The
focusing coil produces an axial field which focuses the beam on the film. Vertical
and horizontal deflection of the beam, so as to the scan the whole film, is
accomplished by passing saw-tooth current waves through deflecting coils which
thus produce transverse horizontal and vertical magnetic fields respectively. The
alignment coils are for initial adjustment of the direction of electron beam.
• Operation: When the scanning beam passes over the photo conductive
material of the signal plate, it deposits electrons so that the potential of this
side of plate is reduced to that of the cathode. But the otherside of the film
(plate) is still at its original potential. Consequently a potential difference
across a given point the photoconductive material is created. It is
approximately 30 V. Before the next scanning (which may be done after an
interval of 1/50 or 1/25 sec.) the charge leaks through photoconductive
material at a rate determined by the conductivity of the material which, in
turn depends upon the amount of incident light.

• White portions of the object will project more light on the film and make it
more conductive. This charge leaked to photoconductive side of the film
will vary according to illumination of the object. As a result, potential at
every point on the photoconductive side will vary. Now the electron beam
again starts scanning the photoconductive side of the film but this time the
charge deposited by the beam in order to reduce its potential towards zero
(cathode potential) will vary with time. Therefore current through RL (and
hence the output voltage) will follow the changes in potential difference
between two surfaces of the film and hence follows the variations of light
intensity of successive points in the optical image.
 CHARGE COUPLED DEVICE (CCD)

• The charge-coupled device falls in the category of solid state

semiconductor devices . A monolithic array of closely spaced metal oxide
semi conductor forms the photo sensitive layer. A simplified construction
of CCD is shown in the figure
• The light is absorbed on the photoconductive substrate and charge
accumulates around the isolated wells under control of electrodes. Each
isolated well represents a pixel. Charges are accumulated for the time
when it takes to complete a single image scan. The charge built up is
proportional to the intensity of the image. lf the intensity of the charge is
of high then the image is of high quality. lf the charge intensity is less, then
the image is of less quality. When there is no light source we can extract
the image using a microprocessor.
 Charge Injection Device

• A charge injection device uses an image sensor in which the image points
are accessed with reference to their horizontal and vertical coordinates.
• After the camera is initiated, the camera's lens focuses reflected light
through the camera aperture, light filters and onto the electronic image
sensor. The image sensor is arranged in a grid pattern, with each individual
square called a pixel. The image sensor cannot determine the color of light
recorded, only the intensity. Color is typically identified by the use of a
color filter which will only allow one color of light from the visible
spectrum into each pixel.
• Each pixel of the CID image sensor is arranged over two intersecting
capacitors. After the intensity of the light is read, it is stored on the sensor
until the pixel is ground or 'injected' into the substrate. Since the image
stays on the sensor, a displacement current equal to the signal on the
sensor is created. The displacement current is read when the charge is
shifted between capacitors, and is then amplified and converted to a
voltage. This voltage is the camera output which is created as a video
signal.
• Since the image is still on the CID image sensor, this image is non-
destructive and allows for an accurate exposure, even in low-light
scenarios. If the charge injection is interrupted, a CID can offer an
excellent time-lapse exposure. Blooming and smearing is greatly reduced
in CID cameras because the charge is held in the pixel, and there are no
opaque areas between pixels. This makes CID sensors apt for pattern
recognition, target tracking, image processing, and precision
measurement.
 IMAGE PROCESSING
• An image is an array or a matrix of square pixels (picture elements)
arranged in columns and rows. Images are obtained using various image
acquisition devices like cameras. After acquiring these images they are
stored in computer memory as digitized image. These acquired images will
be having errors during their acquisition. lf we remove these errors we will
get clear images. The removal of errors for making the image more bright
and clear is done by image processing.
• Image processing is a method to convert an image into digital form and
Perform some operations on it, in order to get an enhanced image or to
extract some useful information from it. lt is a type of Signal dispensation in
which input is image, like video frame or photograph and output may be image
or characteristics associated with that image. Usually Image Processing system
includes treating images as two dimensional signals while applying already set
signal processing methods to them. There are two processing objectives:
– To extract the spatial parameter of the objects
– To recognize the objects in the image
• The various image processing techniques includes connectivity method,
Thresholding, histogram analysis etc.
 THRESHOLDING TECHNIQUE

• Thresholding is the simplest method of image segmentation. Thresholding

can be used to create binary .mages (black and white images) from gray
scale images.
• In an imaging process there is an object and surroundings. The
surroundings and object is identified by applying some thrush hold
values. Generally the thrush hold value of an object is higher than that
of surroundings. Then only we can highlight the object. If the threshold
values applied to the surroundings is higher, then the surroundings get
highlighted to the image.
• The input to a thresholding operation is typically a grayscale or color
image. ln the simplest implementation, the output is a binary image
representing the segmentation. Black pixels correspond to background
and white pixels correspond to foreground (or vice versa). ln simple
implementations, the segmentation is determined by a single parameter
known as the intensity threshold. In a single pass, each pixel in the image
is compared with this threshold. lf the pixel’s intensity is higher than the
threshold the pixel is set to stay within the output. if it is less than the
threshold it is set to black
• Figure shows some typical histograms along with suitable choices of threshold. Fig
(A) shows a classic bi-modal intensity distribution. This image can be successfully
segmented using a single threshold T1 .Figure (B) is slightly more complicated.
Here we suppose the central peak represents the objects we are interested in and
so threshold segmentation requires two thresholds: T1 and T2. ln figure (C), the
two peaks of a bi-modal distribution have run together and so it is almost certainly
not possible to successfully segment this image using a single global threshold. ln
more sophisticated implementations, multiple thresholds can be specified, so that
a band of intensity values can be set to white while everything else is set to black.
• For color or multi-spectral images, it may be possible to set different
thresholds for each color channel and select just those pixels within a
specified cuboid in RGB space. This is called dynamic loading. ln dynamic
loading we apply different thresholding values for different parts for a
picture Another common variant is to set to black all those pixels
corresponding to background, but leave foreground pixels at their original
colour/intensity (as opposed to forcing them to white), so that
information is not lost.
 HISTOGRAM ANALISIS
• Contrast : It is the difference between maximum and minimum pixel
intensity. There are two methods of enhancing contrast. The first one is
called Histogram stretching and the second one is called Histogram
equalization.

• Digital images can be broadly classified into two.

Low contrast image
High contrast image

• In a low contrast image the pixel clusters in a two specific gray scale
values. This reduces the clarity. But in order to obtain high contrast image
the pixels should be distributed uniformly across all over to get high
contrast image. This is done through histogram analysis
 Histogram
• A histogram is a graph that shows frequency of anything. Usually
histogram has bars that represent frequency of occurring of data in the
whole data set. A Histogram has two axes the x axis contains event
whose frequency you have to count and y axis contains frequency. The
different heights of bar shows different frequency of occurrence data.
• An image Histogram is a type of histogram that acts as a graphical
representation of the lightness/color distribution in a digital image. It
plots the number of pixels for each value. Histogram analysis is used to
improve the contrast and brightness of the image.
 Histogram sliding
• ln histogram sliding, we just shift a complete histogram rightwards or
leftwards. Due to shifting or sliding of histogram towards right or left, a
clear change can be seen in the image. A histogram of an image is shown
below. For increasing brightness using histogram sliding:
• On the y axis of this histogram is the frequency or count. And on the x axis, we
have gray level values. As you can see from the above histogram, that those gray
level intensities whose count is more than 700, lies in the first half portion, means
towards blacker portion. That's why we got an image that is a bit darker.
• ln order to bright it, we will slide its histogram towards right, or towards whiter
portion. In order to do we need to add at least a value of 50 to this image. Because
we can see from the histogram above, that this image also have 0 pixel intensities,
that are pure black. So if we add 0 to 50, we will shift all the values lies at 0
intensity to 50 intensity and all the rest of the values will be shifted accordingly.
• For decreasing brightness using histogram sliding: We have to decrease
brightness of this new image to such an extent that the old image looks
brighter; for that we need to subtract some value all the matrix of the new
image
 Histogram stretching
• A histogram of an image is shown below . ln order to do histogram
stretching first we need to calculate contrast from this image
• Now we will increase the contrast of the image. The formula for stretching
the histogram of the image to increase the contrast is:

• The formula requires finding the minimum and maximum pixel intensity
multiply by levels of gray. in our case the image is 8 bpp, (8 bits per pixel is
selected) so levels of gray are 256. The minimum value is 0 and the
maximum value is 225. So the formula in our case is

• Where f(x , y) denotes the value of each pixel intensity. For each f(x ,y ) in
an image, we will calculate a value using this formula. After doing this we
will be able to enhance our contrast by adding more values.
• The following image appears more contrast after applying histogram stretching.
The stretched histogram of image is shown below. The histogram is now stretched
or expanded.
 Histogram Equalization

• Histogram equalization is used to enhance contrast. lt is not necessary a

contrast will always be increase in this. There may be some cases where
Histogram equalization can be worse. ln those cases the contrast is
decreased. The histogram of an image has been shown above.
• Now we will perform histogram equalization to it. First we have to
calculate the PMF (probability mass function) of all the pixels in this
image, after that calculate CDF (cumulative distributive function). Then in
this step you will multiply the CDF value with (Gray levels (minus) 1).
Considering we have an 8 bpp image. Then number of levels we have are
8. And 1 subtracts 8 is 7. so we multiply CDF by 7. After multiplying we will
get values as shown below.
 TRADITIONAL AND MECHATRONIC
DESIGN APPROACH
• Engineering design is a complex process which involves interaction between many skills
and discipline.
• In traditional design, the components are designed through mechanical, hydraulic or
pneumatic components and principles. But in mechatronics approach, mechanical,
electronics, computer technology and control engineering principles are included to
design a system.

• Eg. In traditional design of the temperature control for a central AC system involves a
bimetallic thermostat in a closed loop control system. The basic principle behind this
system is that the bending of the bimetallic strip changes as the temperature change and is
used to operate an ON/OFF switch for the temperature control of the AC system. The
bimetallic thermostat is less sensitive compared since the temperature is not accurately
controlled and it is not suitable for having different temperature at different time of the
day because it is very difficult to achieve.
• The same system can be modified by mechatronics approach, which uses a
microprocessor controlled thermocouple as the sensor. Such a system has may advantages
over traditional system. The microprocessor controlled thermo diode system can
overcome above said difficulties and is giving precision and programmed control. This
system is much more flexible.
SL NO. TRADITIONAL APPROACH MECHATRONICS APPROACH
1 Bulky system Compact
It is a complex process involving It is the basic integration of various
2 interactions between many skills and emerging technologies with mechanical
disciplines engineering.
The control is accomplished manually A microprocessor is used as a controller
3
by programming it
Complex mechanisms Simplified mechanisms that can be
4
transferred to software through programs
5 Non adjustable movements Programmed movements
6 Constant speed drives Variable speed drives
7 Mechanical synchronization Electronic synchronization
Accuracy determined by tolerance of Accuracy achieved by feedback – more
8
mechanism – less accurate accurate
9 Flexibility is less Flexibility is more
It consists more components and moving It involves less components and moving
10
parts parts
11 Less cost High cost
CASE STUDIES OF MECHATRONICS
SYSTEMS
The following case studies of mechatronic systems are explained :

1. Pick and place robot

2. Automatic car park system
3. Engine management system
4. Automatic camera
5. Bar code reader
 PICK AND PLACE ROBOTS
• Pick and place robots are commonly used for
automation processes.
• They are employed to pick an object and place it in
another place according to the process requirement.
• The basic form of a pick and place robot is shown in
the figure below. It consists of a column base
structure, a movable arm and a gripper mechanism.
• The robot has three axes and about these three axes
only motion occurs. The following movements are
required for this robot.
1. Clockwise and Anti-clockwise rotation of the robot
unit on its base
2. Horizontal linear movement of the arm to extend or
contraction
3. Up and down movement of the arm
4. Open or close movement of the gripper
• The above movements are accomplished by the use of pneumatic cylinders
operated by solenoid controlled values with limit switches.
• The limit switches are used to indicate when a motion is completed. Thus, a
clockwise rotation of the robot unit can be obtained from a piston and cylinder
arrangement during its extension and that of counter clockwise during its
retraction,
• Likewise, the upward and downward movement of the arm can be obtained from
a piston and cylinder arrangement during the extension and retraction of a piston
respectively.
• Similarly, the gripper can be opened or closed by the piston in a linear cylinder
during its extension is shown in figure.
 AUTOMATIC CAR PARK BARRIER SYSTEM
• The main requirement of the system is that, the in-barrier is to be opened to
allow the car inside if correct money (coin) is inserted in the collection box and
the out barrier is to be opened to allow the car outside, if the car is detected at
the car park side of the barrier. Also, a mechanism to lift and lower it
automatically.

• When the current flows through the solenoid A & the piston in the cylinder
extends to move upward and causes the barrier to rotate about its pivot and thus
the barrier raises to allow the car inside. When the current flows through the
solenoid A ceases, the spring on the solenoid valve makes the contacts to open
and thus moves the valve back to its original position.

• When the current flows through solenoid B, the piston in the cylinder moves
downward end causes the barrier to get down.
• Limit switches are used to detect when the barrier is down and also when fully
up, which can be controlled by a PLC.
 ENGINE MANAGEMENT SYSTEM
• Engine management system is now-a-days, used in many of the modem cars,
which includes many electronic control systems such as microcontrollers for the
control of various engine factors.

• The main objective of the system is to ensure that the engine is operated at its
optimum settings, managing the ignition and fuelling requirements of the
engine,
• The power and speed of the engine can by varying the ignition timing and the
Air fuel mixture.

• The system consists of number of sensor for observing vehicle speed, Engine
temperature, oil and fuel pressure, air flow etc. These sensors supplies input
signals to the microprocessor after suitable signal conditioning and provides
output signals via drivers to actuate corresponding actuators.

• [Actuators - Actuators are devices used for producing the required output .
They include solenoid valves , electromagnetic components , stepper motors,
indication lamps etc.]
• The engine speed sensor is an inductive sensor while temperature sensor is
usually a thermistor. The air flow rate sensor may be a hot wire anemometer; as
the air passes over a heated wire it will be cooled and the amount of cooling
depends upon mass flow rate.

• To control the ignition delay, the crank shaft drives a distribution which makes
electrical contacts for each spark plug in turn and a timing wheel. This timing
wheel generates pulses - to indicate the crankshaft position. The microprocessor
then adjusts the timing at which high voltage pulses are sent to the distributor so
that they occur at right moments of time (firing order).

• To control the amount of air-fuel mixture entering into a cylinder during the
suction stroke, the microprocessor varies the time for which a solenoid is
activated at the inlet valve on the basis of inputs received by the engine
temperature and the throttle position.

• The amount of fuel to be injected into the air stream can be determined on input
from a sensor for the mass rate of air, or computed from other measurements.
The microprocessor then gives as output to control of fuel inject valve.
 AUTOMATIC CAMERA
• The modern camera is likely to have automatic focusing and exposure.
• When the switch is operated to activate the system and the camera pointed at the
object being photographed, the microprocessor takes the input from the range
sensor and sends an output to the lens position drive to move the lens to achieve
focusing.
• The lens position is fed back to the microprocessor so that the feedback signal
can be used to modify the lens position with the inputs from the range sensor.
• The light sensor gives an input to the microprocessor which then gives an output
to determine, if the photographer has selected the shutter controlled rather than
aperture controlled mode ie the time for which the shutter will be opened.
• When the photograph has been taken, the microprocessor gives an output to the
motor drive to advance the film ready for the next photograph.
• The programming of the microprocessor consists of logic decisions with the
input and output signals either being low or high to give on-off states.
• A few steps of the program for the automatic camera might be of the form:
 BAR CODE READER
• A barcode reader (or barcode scanner) is an electronic device that can read
and output printed barcodes to a computer.
• Like a flatbed scanner, it consists of a light source, a lens and a light sensor
translating optical impulses into electrical ones. Additionally, nearly all barcode
readers contain decoder circuitry analyzing the barcode's image data provided
by the sensor and sending the barcode's content to the scanner's output port.
• A product purchased from the market generally bears coded black and white
bars on the product packet.
• This coded bar, that is, a barcode, is read by exposing it to a laser light beam
from a hand-held wand, which is held over the product.
• By reading the barcode, the output of the computer produces the name of the
product and its cost, hence, a barcode is used to indicate the nature of the
product and its cost.
• A barcode is composed of a series of black and white lines on the packaging;
sometimes, it also bears the code of the company, that is, a product of LG, Tata,
Bata, and any other company.
• The following two schemes are used in barcoding: