AUTOMATED TESTING OF VEHICLE

INSTRUMENT CLUSTER BASED ON COMPUTER

VISION

Tan Wei Ren, Wan Shahmisufi bin Wan Jamaludin, Kueh Ying Lin, Muhammad Na-
siruddin Mahyuddin, and Bakhtiar Affendi Bin Rosdi

School of Electrical & Electronic Engineering Campus, Universiti Sains Malaysia, 14300 Ni-
bong Tebal, Pulau Pinang
lanson1991@hotmail.com, wanshahmisufi90@gmail.com,
kuehyinglin93@gmail.com, nasiruddin@usm.my, eebakhtiar@usm.my

Abstract. With advancement of technologies, instrument cluster is become more

complex. Conventional manual testing and validation is difficult to cover all the
test cases to provide flawless delivery of product within restricted development
timescale. As for now, we are proposing to use computer vision system in auto-
motive manufacturing for automated design validation testing process. The pur-
pose of the inclusion of computer vision system is to replace the conventional
design validation testing process which is time consuming and extremely labor
intensive. The speedometer, tachometer, fuel gauge and temperature gauge are
inspected by comparing the accuracy of the pointer detected using the developed
algorithms with the pointer position displayed on the meter. Besides that, the sig-
nal indicators status can be inspected using pixel intensity test. Under the assump-
tion of controlled light surrounding and fixed position of the camera, pixel inten-
sity can produce accurate results. The deviation error of needle gauge test and
signal indicator test are 2.3%, those error occur while there is some noises influ-
ence the threshold value. Besides that, processing time of computer vision is
within 0.5 seconds which is quite efficiency in testing process. In conclusion, the
machine vision system is able to help for spectating the automated instrument
cluster testing process.

Keywords: Needle gauge test, white pixel intensity test, automated instrument
cluster test (AIC).

1 Introduction

The instrument cluster is generally located directly above the steering wheel in every
vehicle. It is used to display important vehicle operation information to the driver such
as vehicle speed, fuel level, temperature of engine and status of various vehicular sys-
tems. A clear and accurate presentation of the vehicle condition is very important at any
given point of time to ensure the safety and comfort of the driver. The information are
usually in the form of needle gauges, digital readouts and lighted symbols which are
centrally in the primary viewing range of the driver.
2

Nowadays, most of the vehicle instrument cluster consist mainly of needle gauges
driven by stepper motors together with all of the warning lights that are built into a
single module [1]. In more luxurious car, more advance instrument clusters are used
which have some sort of digital multi-display which can be changed by inputs from the
driver to show different information about the vehicle and allow the driver to change
various settings.
The inclusion of machine vision in automotive manufacturing processes in the pre-
vious years as stated in [2] and [3] has been a revolutionary process that are able to
reduce inspection time as well as labor cost during the process of testing and validation.
The importance of including machine vision system in manufacturing process is that
the development cycle of a product can be reduced and the conventional manual testing
process which is less efficient will be replaced with highly efficient machine vision
system.
In [4-7], automation for the instrument cluster testing and validation using machine
vision had been developed using industry camera which has image processing software
that used library vision tools.
From existing testing, industry is using semi auto testing in the instrument cluster
which tester is needed to control the signal sending to cluster and spectating the result
by using human eyes. This may need a longer time for complete testing a cluster and
human error may occur.
In this paper, we proposed needle gauge test and signal indicator test to visual output
of instrument cluster. In our developed algorithm, we calculate the angle of the needle
to compute the result of needle gauge test and check whether the signal indicator light
is on or off in signal indicator test. With the proposed algorithm for needle gauge test,
pointer detection technique will get the angle of the needle and come out the result
based on the angle. Besides that, pixel intensity test will check the white pixel intensity
percentage in particular signal indicator light area. If the percentage of the white pixel
is exceed constant value then the algorithm will determine that the signal indicator light
is “ON”.
This paper consists of four main sections. Second section will explain about the al-
gorithm of computer vision which divided in two categories which are needle gauge
test and signal indicator test. Third section will show the result of this research. Lastly,
forth section will finalize this research.

2 Proposed algorithm for computer vision

In this paper, computer vision is used to capture and process the image of the Auto-
mated Instrument Cluster. Needle gauge test and signal indicator test are two main tests
in AIC test algorithm.

2.1 Needle Gauge Test

Needle gauge test is to test on speedometer, tachometer, fuel gauge and engine temper-
ature gauge which the output is mainly showing by the needle’s location. In this test,
 3

the system will compute the vehicle’s speed in kilometer per hour, km/h which show
on speedometer, the engine revolution per minutes (RPM) in tachometer, fuel level in
fuel gauge and engine temperature level in engine temperature gauge by getting the
angle of the needles.
The flow is start by capturing and calibrating the cluster’s image to make sure the
image is align correctly. Secondly, it will crop out the section of gauge which is going
to test. In [8] image thinning is used for analog measurement, while in this system im-
age thinning is used to reduce the noises on the blob and error when detecting the line.
Then, the cropped image is converted in to binary image to prevent noise for image
processing. After that, Hough transform line detection algorithm is applied to determine
the position of the pointer. Lastly, it will calculate the angle of the pointer and compute
the result of the particular gauge test (KM/H, RPM, Fuel level, Temperature level).
Needle gauge test will calculate based on formula given in Eq. (1):
Result = Constant * Angle. (1)
Constant in needle gauge test is determined by formula given in Eq. (2):
𝐷𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒 𝑏𝑒𝑡𝑤𝑒𝑒𝑛 2 𝑚𝑒𝑎𝑠𝑢𝑟𝑒𝑚𝑒𝑛𝑡(𝑐𝑢𝑟𝑟𝑒𝑛𝑡 𝑠𝑝𝑒𝑒𝑑−𝑟𝑒𝑓𝑒𝑟𝑒𝑛𝑐𝑒 𝑠𝑝𝑒𝑒𝑑(20))
𝐶𝑜𝑛𝑠𝑡𝑎𝑛𝑡 = (2)
𝐴𝑛𝑔𝑙𝑒 𝑏𝑒𝑡𝑤𝑒𝑒𝑛 2 𝑝𝑜𝑖𝑛𝑡𝑒𝑟 𝑝𝑜𝑠𝑖𝑡𝑖𝑜𝑛(𝑐𝑢𝑟𝑟𝑒𝑛𝑡 𝑎𝑛𝑔𝑙𝑒−𝑟𝑒𝑓𝑒𝑟𝑒𝑛𝑐𝑒 𝑎𝑛𝑔𝑙𝑒)

The algorithm to measure all the gauge are the same, the only difference is the area to
be cropped.
Fig. 1 shows the process of needle gauge test of speedometer at 80km/h, (a) is showing
the image that cropped out from whole cluster’s image. Then (b) is showing the cropped
image is been converted into binary image. After that, image thinning is applied on the
binary image. Follow by Hough transform is used to detect the needle in the image.
Lastly, system will calculate the angle between the reference line (20/km/h) and the red
line, based on the angle difference between two lines, system will compute the result
of current speed as in (e)

(a) Cropped Image (b) Binary Image

(e) angle calculation

(c) Image Thinning (d) Hough Transform

Fig. 1. Image of speedometer while perform needle gauge test.

4

2.2 Signal Indicator Test

The purpose of the inspection on signal indicator is to check whether the status of signal
indicator light in cluster is on or off. The method used to check the status of the signal
indicator is to apply the pixel intensity test at specific location. The pixel intensity test
is to identify the number of white pixels in an area of a specific location. Since the
purpose of the testing is to check the status on or off of a signal indicator, this test can
be executed easily by setting the location of the signal indicator. The location of all the
signal indicator have to be identified and stored in the database before performing any
test.
This algorithm start by determining which signal indicator needed to test and select
the position of the particular signal indicator. Secondly, it will crop out the signal indi-
cator area for further processing. After that, it will apply the cropped area to become
binary which the image will become black and white. Next, it will obtain how many
pixel in the cropped image. Lastly, it will count white pixel in the image and determine
whether the signal indicator light is turning on or off.
In this test, white pixel intensity technique is used, it will determine the percentage
of white pixel in the cropped area. The percentage is calculated with formula in Eq. (3).
𝑇𝑜𝑡𝑎𝑙 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 𝑤ℎ𝑖𝑡𝑒 𝑝𝑖𝑥𝑒𝑙
𝑊ℎ𝑖𝑡𝑒 𝑃𝑖𝑥𝑒𝑙 𝑃𝑒𝑟𝑐𝑒𝑛𝑡𝑎𝑔𝑒 = × 100% (3)
𝑇𝑜𝑡𝑎𝑙 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 𝑐𝑟𝑜𝑝𝑝𝑒𝑑 𝑖𝑚𝑎𝑔𝑒 𝑝𝑖𝑥𝑒𝑙
White pixel intensity test will crop the image of the signal indicator light that going to
test and get the pixel of the cropped image. Then it will get the total white pixel in the
cropped image. After these two values are determined, then it will calculate the per-
centage based on the formula in Eq. (3) to obtain the white pixel percentage.
Based on fig. 2, black color and white color pixel is shown clearly, white pixel intensity
test will calculate how many white pixel in the cropped image of signal indicator and
divide the total pixel in the image to come out the result of white pixel percentage.
Based on the Eq. (3) total value of whole image in fig. 2(c) will be 208, total value of
white pixel in the image will be 76. So the white pixel percentage = 76/208, equal to
36.54%. So system will determine this test as the left signal is “ON”.

(a) Cropped Image of (b) Binary Image of

left signal left signal
(c) Sample of Binary Image in
pixel view
Fig. 2. Image of left signal while perform white pixel intensity test
 5

3 Result

Needle gauge test is test that used to measure the needle’s angle for speedometer, ta-
chometer, fuel gauge and temperature gauge. Based on the angle of the needle, those
result of gauge will be calculated and shown. In needle gauge test, we took 50 photos
of each gauge to test the algorithm. Those 50 photos is to test the deviation error occurs
in this algorithm. After test the algorithm and the algorithm is functioning well, we can
proceed to real time testing of the instrument cluster by using the algorithm.
In fig. 3, speed log file is a log file that show the result of speed after calculation from
the speedometer. In fig. 3, showing the image that captured in live (left side is first test
and second image is last test in the log file) and stored in particular folder and captured
speedometer image. Besides that, speeds is the speed that calculated based on the im-
age. Then the speeds in Txt is the expected speed that given by the test instruction in
text file format. Furthermore, status is to check that the speeds and speeds in Txt is
matched with stated tolerances.
Log file is generated once the testing is ongoing, all the result will be stored for re-
view after the testing. All needle gauge test no matter tachometer, fuel gauge, temper-
ature gauge will be same on the display but only background calculation is different.

Fig. 3. Speed log file and speedometer image

White pixel intensity test is the method that used to check the status of the signal
indicators light is on or off. In white pixel intensity test, we took another 45 photos to
test the algorithm. That is 15 signal indicator in cluster, 2 photo for each signal indicator
in dark condition, and 1 photo for each signal indicator in light condition.
Based on Table 1, the lowest white pixel intensity percentage 17.9205% while the high-
est white pixel intensity percentage is 45.3448%. The white pixel intensity in an image
depends on the size of the signal indicator and the image which is converted into binary
image. The lower the threshold value, the higher the white pixel intensity for a signal
indicator that are switched on. The minimum white pixel intensity percentage set to
identify a turned on signal indicator is 10%. This 10% is allocated to prevent for the
noise or reflection on the vehicle instrument cluster. White pixel percentage of every
signal is different is because of the signal indicator light’s size is different.

Table 1. White Pixel Intensity test result

Signal Indicator White Pixel Intensity Percentage (%)
6

Disk Brake 45.3448

Upper Beam 33.8003
Left 27.1223
Engine Oil 1 17.9205
Right 25.1419

4 Conclusion

In this research, the proposed algorithms are solving the inspection problem in auto-
mated instrument cluster test. Based on the result stated above, the proposed algorithms
can produce accurate results under a controlled environment with fix position of the
instrument cluster and surrounding lighting. Angle calculation is chosen and the pixel
intensity test is favored because it can get an accurate result of 2.3% and the processing
time requires less than 0.5s which is highly efficient.
In conclusion, the proposed algorithms for the automated testing are able to produce
desired result and can be implemented and replace the conventional manual testing in
the manufacturing line in the future.

References
1. Automotive Electronics - Instrument Clusters. Available: http://www.cvel.clem-
son.edu/auto/systems/instrument_cluster.html. 2016/05/12
2. P. Hage and B. Jones, "Machine vision-based quality control systems for the automotive
industry," Assembly Automation, vol. 15, pp. 32-34, 1995
3. S. Quan, X. Ning, and C. Yifan, "Development of an automatic optical measurement system
for automotive part surface inspection," in Proceedings, 2005 IEEE/ASME International
Conference on Advanced Intelligent Mechatrocnis., 2005, pp. 1557-1562.
4. S. Das and P. Das, "Competitiveness and its impact on research and development in indian
automobile industry," International Journal of Management and Transformation, vol. 5, pp.
79-89, 2011.
5. P. Dr.B.Sharmila and P. S. P.Venkadesan, "Automation for instrument cluster panel testing
using machine vision and NI-PXI hardware-in-the-loop," International Journal of Advanced
Research in Electrical, Electronics and Instrumentation Engineering, vol. 4, 2015.
6. G. Tsampardoukas and A. Mouzakitis, "Hardware-in-the-loop visual display validation uti-
lising vision system," in Control 2010, UKACC International Conference on, 2010, pp. 1-5.
7. H. Yingping, A. Mouzakitis, R. McMurran, G. Dhadyalla, and R. P. Jones, "Design valida-
tion testing of vehicle instrument cluster using machine vision and hardware-in-the-loop,"
in Vehicular Electronics and Safety, 2008. ICVES 2008. IEEE International Conference on,
2008, pp. 265-270.
8. J. Zhang, Y. Wang, and F. Lin, "Automatic reading recognition system for analog measuring
instruments base on digital image processing," Journal of Applied Sciences, vol. 13, p. 2562,
2013.