Available online at www.sciencedirect.

com

ScienceDirect
Available online at www.sciencedirect.com

ScienceDirect
Procedia Manufacturing 00 (2019) 000–000
www.elsevier.com/locate/procedia

Procedia Manufacturing 42 (2020) 337–343

International Conference on Industry 4.0 and Smart Manufacturing (ISM 2019)

The Use of Marker-Based Augmented Reality in Space Measurement

Salin Boonbrahma*, Poonpong Boonbrahma, and Charlee Kaewrata
aWalailak
Walailak University, Nakorn si Thammarat 80160, Thailand

* Corresponding author. Tel.: +66-75-672257; fax: +66-75-672205. E-mail address: salil.boonbrahm@gmail.com

Abstract

There are many applications on using AR in supporting manufacturing and related areas ranging from designing, simulation, testing, repairing,
and maintenance. The objective of this research is to find the algorithm for measuring the size and find the shape of the room for facility
management using marker-based AR technology. The applications program was designed and developed using the Unity 3D game engine and
OpenCV library. It was designed to be used by many devices running on various operating systems such as Windows, OSX, iOS, and Android.
Test of the concept was done in three situations, i.e., three rooms with different sizes. The first room is the smallest one with the size = 25.245
m3. The second room is the biggest one with the size of 117.35 m3 and the third one with the size = 56.1 m3. The results calculating from the
distances of the markers show that the sizes of the space obtaining from the application are different from the manual measure less than 3%,
which means that the system can provide high accuracy in measurement.

© 2020The
© 2020 TheAuthors.
Authors. Published
Published by Elsevier
by Elsevier B.V.B.V.
This
This isisan
anopen
openaccess
access article
article under
under the CCthe BY-NC-ND
CC BY-NC-ND licenselicense (http://creativecommons.org/licenses/by-nc-nd/4.0/)
(http://creativecommons.org/licenses/by-nc-nd/4.0/)
Peer-review underresponsibility
Peer-review under responsibilityofofthethe scientific
scientific committee
committee of the
of the International
International Conference
Conference on Industry
on Industry 4.0Smart
4.0 and and Smart Manufacturing.
Manufacturing.

Keywords: augmented reality; marker-based AR, space measurement;

1. Introduction usually, the volume or shape of the room or space is required,

so it is not practical.
In manufacturing, controlling the environment in the In the last few years, there have been some significant signs
working area required a precise volume of the space. For of progress in the field of information technology, especially
example, to control the room temperature, not only the in Virtual Reality (VR) and Augmented Reality (AR). VR is
emission of heat from the machine and the number of people the technology that let the user wholly immersed into the
in the room are needed but the exact volume of the room is virtual world while AR integrates virtual world generated by
also required. In logistics, to keep on top of freight costs, digital computer with the real environment. Even though AR
volume and weight measurement of goods must be done is considered as the subset of VR, but in terms of applications,
quickly and accurately. Also, the space available in the they are significantly different. Both AR and VR can be
container is required in order to calculate how much more the applied in many fields, but manufacturing and related
product or goods can be added to fill up the available space. industries had taken this technology seriously. Since AR deals
Depending on the reason for measuring the room, different with both the virtual and real word, it is suitable for space
measurements need to be taken. Usually, making the space management, which required the simulation of calculated
measurement by hand is the common method, but it is a shape and size of the virtual space projected on top of the real
tedious work that required lots of work and time, especially if world.
the space is big. Another way to do this is by using tools like a There are two categories of AR i.e., marker-based AR and
laser measuring devices, which can give more accurate results markerless AR. The differences between these two rely on the
and save time. Both methods will give only the distance, but information source for the position of the digital content
2351-9789 © 2020 The Authors. Published by Elsevier B.V.
This is an©open
2351-9789 access
2020 The article
Authors. under by
Published theElsevier
CC BY-NC-ND
B.V. license (http://creativecommons.org/licenses/by-nc-nd/4.0/)
Peer-review
This is an openunder
access responsibility
article under the of
CCthe scientificlicense
BY-NC-ND committee of the International Conference on Industry 4.0 and Smart Manufacturing.
(http://creativecommons.org/licenses/by-nc-nd/4.0/)
10.1016/j.promfg.2020.02.081
Peer-review under responsibility of the scientific committee of the International Conference on Industry 4.0 and Smart Manufacturing.
338 Salin Boonbrahm et al. / Procedia Manufacturing 42 (2020) 337–343
2 Author name / Procedia Manufacturing 00 (2019) 000–000
within the user’s view. In the case of marker-based AR, the
system needs to know where the user is and the position that
the user wants to focus. One solution is to add a detectable
pre-defined sign or “marker” in the environment and use
computer vision techniques to detect it, which is called
“tracking.” The marker can be in the form of a 2D picture, QR
code, or 3D object. The function of the marker is to identify
the position of the virtual objects to be displayed. In the case
of the markerless AR, the user can place the virtual object at
any position within the range of the camera view using a
combination of GPS and the compass sensor on the
smartphone as we have seen in the case of Pokémon Go [1].
Today, there is a new platform for markerless AR applications
such as ARKit for iOS device and ARCore for Android
device, which simplify an AR app development. These two
platforms use data from a device’s motion sensor and camera
to identify spatial movement across six axes. This allows
virtual objects to remain accurately positioned in the real
world. With plane detection function, virtual items can be
placed onto surfaces in a realistic way [2]. These two
platforms seemed to be successful in term of entertainment
perspective but for space measurement, they are still far from
perfect [3]. The problems lie on the accuracy on determining
where the floor is. Technically the system can detect non-
uniform color floor but not uniform white floor, so it cannot
place the virtual model on the spot assigned. Since managing
the space required the distance from the wall to wall,
anchoring the position on each end is necessary so marker-
based AR is the suitable one.
The AR measuring systems, not only can measure distance
but also generate the shape of the full dimension of the room
or any close space. The system can also measuring and
generate individual shape object such as parcel or pellet,
quickly and precisely making it perfect for warehouse
management too. In this research, AR measuring technology,
which is three-dimensional, non-contact measurement were
used for supporting space management in the manufacturing
and logistic platform.
2. Literature review
Various areas of research on using marker-based AR
applications in manufacturing had been done in the past few
years [4]. Michalos G et al. developed an AR-based
application to support human-robot interactive cooperation in
an automotive environment. Their objectives are to enhance
human safety and increase their productivity in an industrial
environment, where robots co-exist with humans. The results
indicate that the approach can significantly enhance the
operator’s working conditions and their integration in the
assembly process [5]. Aziz FA et al. [6] had developed an AR
platform to assist the automotive industry technician in the
inspection and maintenance process. After evaluation of the
effectiveness of the new implementing AR system compared
to traditional manual sheets inspection, the results indicate
that the AR system can reduce the processing time by 10.72
percent. Lee S. and Akin O., who worked on developing AR-
based equipment Operations and Maintenance (O&M)
fieldwork, reported that O&M field workers could save over
50% of the time on average in locating the operation
destinations with the navigation function of their AR-based
fieldwork facilitator than with conventional methods [7].
Research on using AR for space measurement starts to grow
due to the advancement of mobile hardware and application
software. Daponte et al. [8] presented the survey of
applications using the AR as the user interface of the
measurement system. The future trends for researches
oriented on the development of calibration procedures for
tracking systems are also discussed. Bergquist R. et al. [9]
applied the ARKit device to measure surface areas by placing
anchors to mark an area to be measured. Even though the AR
tool does not provide the same precision as manual
measurements, but is still reasonably within boundaries if an
estimation is acceptable. Whitlock M. et al. [10] explores AR
interactions at a distance, measuring how applications may
support fluid, efficient, and intuitive interactive experiences in
room-scale augmented reality.
Many software companies also start developing the
application program for turning mobile devices into a
measuring system [11, 12].
Usually, the research on AR application will emphasis on
tracking the marker’s position and then display the virtual
information at the location assigned. Tracking is a technique
that assists the camera in finding the orientation and position
of the marker concerning the real world. In this research,
instead of using the information to display virtual objects, the
information of the markers, i.e., their position (x, y, z) and
their rotation axis, was used for calculating the space and
shape of the room or container. With this concept, the facility
management is more effective.
3. Methodologies
The objective of this research is to find the algorithm for
measuring the size and find the shape of the room for facility
management using marker-based AR technology. This
process will involve designing the marker, tracking them, and
use the information acquired from them to calculate and
generate the shape and size of the room. Usually, there are
many types of marker available but selecting the suitable
marker types depending on the system requirements [13]. In
this case, the ArUco marker was chosen since they are
suitable for our requirements in terms of reliability,
robustness, and versatility. Besides that, their recognition is
speedy, with a low level of ambiguity in indoor environments.
Also, the marker has been included within the last stable
version of OpenCV [14]. OpenCV (Open source computer
vision) is a library of programming functions mainly aimed at
real-time computer vision. Usually, an ArUco marker is
represented by the black and white square pattern of 7×7
block. The black border has a thickness of 1 block, so the
internal pattern is a sub-matrix is 6×6 blocks. When tracking,
this pattern will represent the type of the marker. And the
arrangement of the inner matrix box will represent the
location and the orientation of the camera in real-time.
 Salin Boonbrahm et al. / Procedia Manufacturing 42 (2020) 337–343
Author name / Procedia Manufacturing 00 (2019) 000–000
3.1. Tracking the marker
Step in tracking the marker can be divided into three steps:
1. Detection of Marker Candidates: In this stage,
information from the camera will be analyzed to check
whether there is a square marker or not using adaptive
threshold technique and then filtered other information
out except the marker from the picture using contour
filtering.
2. Marker Analysis: In this step, the Otsu threshold
technique was used for separating black and white bit.
After calculation, the marker can identify whether its
ID is in the database or not.
3. Pose estimation technique: The technique used to
estimate the position of the marker by combining the
coordinate of the maker in the real world and the
virtual world. The process will give the position and
rotation of the marker.
Fig. 1 shows the output from the process and Fig. 2 shows
the marker with the center of the pivot, which will detach with
the marker even though it is moved or rotated.
Fig. 1. (a) Picture from camera; (b) after adaptive threshold and contour
filtering; (c) pose estimation.
Fig. 2. Simulation of the center of pivot on the rotated marker.
3.2. Construction of references
Since the position (x, y, z) of the marker is in virtual space,
so there is no reference point, then the distance cannot be
converted into real space. In order to solve this problem, we
decided to use the size of the marker, which we set to be
10x10 cm. to be a frame of reference.
339
3
3.3. Data collection
Since we want to find the distance between two positions,
i.e., two walls of the room or two sides of the same wall, then
the markers of the two positions must be collected or capture
on the same picture. It will confirm that both positions are in
the same frame of reference. In each capture, data acquired
from the marker such as ID, position, and rotation angle will
be kept in the database for further analysis. In the case of a
small room where the two markers from the opposite wall can
be captured at the same time, then data from both markers can
be kept in the database directly without any further
modification. If the room is too big, then we have to place
another marker in between. In that case, we have to capture
two times or more. First, capture the first two markers and
then capture another two with one marker from the first
capture must be in the frame. The common one will be used
as the reference or connector between the two captures.
After finished collecting all the data from the markers, the
system will put them into application database in the form of
data set such as:
Room = {{Left, Right}, {Left, Back}, {Left, Front}, {Left,
Floor}, {Floor, Ceiling}}
Each subset means one set of data from one capture. The
dataset from the common marker will be used as a connector
to another set of data.
3.4. Connecting two sets of markers
Position of the marker in virtual space (x, y, z) can be
represented in the matrix form as follow:
(1)
To connect two sets of captures, we have to map the
common marker into the same position and same rotation
angle. Since they are the same marker, then the data set will
change to:
Room 1 = {Left, Right, Back, Front, Floor, Ceiling}
Besides mapping to the same position, the rotation angle of
the two needs to be the same as well. Rotation matrix of the
common or connector marker is as follow:
(2)
(3)
340 Salin Boonbrahm et al. / Procedia Manufacturing 42 (2020) 337–343
4 Author name / Procedia Manufacturing 00 (2019) 000–000

(4)

where i is the index of the subset and i – 1 is the previous

index.

3.5. Measuring the distance between markers

The marker on each wall may not be in the same height or

the middle of the wall, to make a precise measurement, only
the position on one plane (vector) will be used for calculation.
For example:

(5)

where
W = width of the room,
XL = position of the left side marker in x axis, and
XR = position of the right side marker in x axis.

(6)

where
L = length of the room,
Fig. 3. Block diagram of the conceptual framework.
ZF = position of the front side marker in z axis, and
ZB = position of the back side marker in z axis.
4. Experiment setup
(7)
In this research, the applications program was designed
and developed using the Unity 3D game engine and OpenCV
where
library. It was designed to be used by many devices running
H = height of the room,
on various operating systems such as Windows, OSX, iOS,
YC = position of the ceiling side marker in y axis, and
and Android. Unity 3D is a cross-platform game engine which
YFl = position of the floor side marker in y axis.
can be used to create 3D, 2D, VR, and AR, as well as
simulations.
From the width, length and height, then simulation of the
room shape can be done easily.
4.1. Testing the prototype
3.6. Conceptual framework
For the process of testing, we are running the experiment
in three situations, i.e., three rooms with different sizes. The
The conceptual framework of this research is shown in Fig.
first room is the smallest one with the size = 25.245 m 3. The
3. From the conceptual framework, to find the distance
second room is the biggest one with the size of 117.35 m3 and
between two points, the camera must capture at least two
the third one with the size = 56.1 m3.
markers at a time. After checking whether they are markers,
then the system will check whether the pivot points are in the
4.2. Experiment on the real environment
same direction, which means that they are on the opposite side
of the wall. If so, then the system will calculate the angle of
Since the first room is quite small (width is only 1.8
the walls and the distance between the two walls. If the two
metre), the three markers can be captured at the same time
pivot points are perpendicular to each other, then another two
which means that the width of the room can be calculated
markers must be captured, and the distance must be combined
without further modification. Fig. 4 shows that the three can
with the first set to represent the distance between the walls.
be captured at the same time.
For the length, since the room has a long length compared
to the width, so it is impossible to use the mobile devices to
capture the markers on both side. In this case, we have to use
the connected marker as a bridge between two markers on
opposite sides. The example of the connected marker is the
marker on the floor in Fig. 4.
 Salin Boonbrahm et al. / Procedia Manufacturing 42 (2020) 337–343 341
Author name / Procedia Manufacturing 00 (2019) 000–000 5

Fig. 7. Simulation of the first room with size indication.

Fig. 4. Markers captured in room #1.
After getting the position and the pivot axis of each marker
as shown in Fig. 5, the width, length, and height of the room
can be calculated from the equation (5), (6), and (7). The
model of each side of the room is shown in Fig. 6.
With the same process, the shape and size of the second
room and third room can be calculated and simulated as in
Fig. 6 and Fig.7 respectively. Fig 8 shows the comparison of
the 3D of these three rooms.

Fig. 8. Simulation of the second room with size indication.

Fig. 5. Markers captured with pivot axis.

Fig. 9. Simulation of the third room with size indication.

Fig. 6. Simulation of individual side of the room

Also, the shape and size of each room can be seen in 3D

model as shown in Fig. 7 to Fig. 9.

Fig. 10. The graphical image of the three rooms.

342 Salin Boonbrahm et al. / Procedia Manufacturing 42 (2020) 337–343
6 Author name / Procedia Manufacturing 00 (2019) 000–000

5. Results and discussion

The result from each experiment was then compared with

the manual measurement, as shown in Table 1-3.

Table 1. The comparison of the first room.

Measurement of Manual Calculated % different
measurement from the
(cm) system (cm)
The width 180 181 0.55%
The length 550 554 0.72 Fig. 12. Simulation from the measurement showing the part of the room that
was occupied.
The height 255 252 1.17

With this result, it can be concluded that the marker-based

AR measuring system can be applied in managing space in
Table 2. The comparison of the second room.
the manufacturing plant, warehouse, or even the container.
Measurement of Manual Calculated % different
measurement from the
(cm) system (cm)
6. Conclusion
The width 590 598 1.35
The possibility of using marker-based AR technology in
The length 780 782 0.25 measuring the size and shape of the room was explored in this
The height 255 254 0.39 research. The objective is to use this technology to manage
the usage of the rooms in the factories or manufacturing
facilities such as warehouses or containers. Results from the
Table 3. The comparison of the third room. measurement in three rooms with different size confirmed that
Measurement of Manual Calculated % different the algorithm for measuring the distance using marker-based
measurement from the AR could give high precision in measurement. This technique
(cm) system (cm) can be applied in other purposes, such as calculating the
The width 415 421 1.44 volume occupied by the products or goods in the warehouse.
The length 520 525 0.96
The height 260 266 2.30
References

[1] Paladini M. 3 Different types of AR explained: Marker-based, Markerless

The results calculating from the distances of the markers
provided high accuracy. Table 1-3 shows that the sizes of all
three rooms obtained from the system are different from the
manual measurement of less than 3%. This result confirmed
that marker-based AR measurement could be used in
managing space effectively.
Since the same technique can be used for measuring small
items such as parcels or pellets, so an estimation of spaces
required by those items or space available after putting those
items into the warehouse or container can be calculated.
Fig.11 shows the space occupied by many boxes. By using a
marker-based AR measuring technique, the space occupied by
the boxed and the space available for more boxes can be seen
in Fig. 12.
Fig. 11. The boxes occupied parts of the room.
& Lovation, retrived from https://www.blippar.com/blog/2018/08/
14/marker-based-markerless-or-location-based-ar-different-types-of-ar.
[2] Tatyana Shavel. ARCore vs. ARKit: Which Is Better for Building
Augmented Reality Apps?. Retrieved from:
https://www.iflexion.com/blog/arcore-vs-arkit, 2018.
[3] Yao A. A workaround for the limitations of ARKit 2. Retrieved from
https://ttt.studio/blog/a-workaround-for-the-limitations-of-arkit-2/, 2019.
[4] Ong SK,Yuan ML, Nee AYC. Augmented reality applications in
manufacturing: a survey, International Journal of Production Research
2008; 46: 2707-2742.
[5] Michalosa G, Karagiannisa P, Makrisa S, Tokçalarb O, Chryssolourisa G.
Augmented reality (AR) applications for supporting human-robot
interactive cooperation, Procedia CIRP 41 (2016):370 – 375.
[6] Aziz FA, Alostad E, Sulaiman S, Ahmad KA. Augmented Reality Marker
Based to Aid Inspection and Maintenance Process in Automotive
Industry, International Journal of Engineering and Advanced Technology
(IJEAT) 2019; 8, 3S:417-421.
[7] Lee S, Akin O. Augmented reality-based computational fieldwork support
for equipment operations and maintenance, Automation in Construction
2011, 20(4):338–352.
[8] Daponte P, Devito L, Picariello F, Riccio M. 2014. State of the art and
future developments of the Augmented Reality for measurement
applications. Measurement Journal, Vol. 57, pp. 53–70.
[9] Bergquist R, Stenbeck N. Using Augmented Reality to Measure Vertical
Surfaces, Bachelor thesis, Linköping University, 2018.
[10] Whitlock M, Hanner E, Brubaker JR, Kane S, Szafir DA. Interacting
with Distant Objects in Augmented Reality. Proceeding of IEEE Virtual
Reality 2018, DOI: 10.1109/VR.2018.8446381.
[11] Ro L. AR Measure turns your phone into a virtual measuring tape.
Retrieved from: https://www.curbed.com/2017/6/29/15894556/ar-
measure-app-augmented-reality-ruler-measuring-tape-ios, 2018.
 Salin Boonbrahm et al. / Procedia Manufacturing 42 (2020) 337–343 343
Author name / Procedia Manufacturing 00 (2019) 000–000 7

[12] Marsden R. How measuring apps became a reality. Retrieved from:

https://www.thenational.ae/arts-culture/how-measuring-apps-became-a-
reality-1.745876, 2018.
[13] Siltanen S. Theory and applications of marker-based augmented reality.
Julkaisija-Utgivare; 2012.
[14] Avola D, Cinque L, Foresti GL, Mercuri C, Pannone D. A Practical
Framework for the Development of Augmented Reality Applications by
using ArUco Markers, International Conference on Pattern Recognition
Applications and Methods 2016: 645-654.