Available online

Available atonline
www.sciencedirect.com
online
Available at www.sciencedirect.com
at www.sciencedirect.com
Available online at www.sciencedirect.com

ScienceDirect
ScienceDirect
Procedia CIRPCIRP
Procedia 00 CIRP
Procedia (2017) 000–000
106 (2022)
00 (2022) 168–173
000–000 www.elsevier.com/locate/procedia
Procedia CIRP 00 (2022) 000–000 www.elsevier.com/locate/procedia
www.elsevier.com/locate/procedia

9th CIRP Conference on Assembly Technology and Systems

9th CIRP Conference on Assembly Technology and Systems
Methodology to integrate ergonomics information in contextualized digital
Methodology to integrate ergonomics
28th CIRP Design
information
Conference,
in contextualized digital
May 2018, Nantes, France
work instructions
work instructions
AArno
new methodology
Claeysa,b,
to analyze
, Steven Hoedt a,b
, Corentinthe
Arno Claeysa,b,, Steven Hoedta,b , Corentin
functional
Domken
Domken
c
and physical
c , El-Houssaine
, El-Houssaine Aghezzaf
architecture
Aghezzafa,b
a,b
, Dieter Claeys
of,,
, Dieter Claeysa,b
a,b
a,b
existing products for an assembly Johannes Cottyna,bproduct family identification
oriented
a Department
Johannes Cottyn
of Industrial Systems Engineering and Product Design, Ghent University, Technologiepark 903, 9052 Gent-Zwijnaarde, Belgium
a Department
of Industrial Systems Engineering andEngineering
Product Design, Ghent University,
b Industrial Systems (ISyE), Flanders Make, Technologiepark 903, 9052 Gent-Zwijnaarde, Belgium
www.FlandersMake.be
Paul Stief *, Jean-Yves Dantan, Alain Etienne, Ali Siadat
b Industrial Systems Engineering (ISyE), Flanders Make, www.FlandersMake.be
c DecisionS Core Lab, Flanders Make, Oude Dietersebaan 133, 3920 Lommel, Belgium
c DecisionS Core Lab, Flanders Make, Oude Dietersebaan 133, 3920 Lommel, Belgium
* Corresponding
Écoleauthor. Tel.: Supérieure
Nationale d’Arts et
+32-56-24-12-21. E-mail address:
Métiers, Arts etarno.claeys@ugent.be
Métiers ParisTech, LCFC EA 4495, 4 Rue Augustin Fresnel, Metz 57078, France
* Corresponding author. Tel.: +32-56-24-12-21. E-mail address: arno.claeys@ugent.be

* Corresponding author. Tel.: +33 3 87 37 54 30; E-mail address: paul.stief@ensam.eu

Abstract
Abstract
Musculoskeletal disorders (MSDs), often caused by working in non-ergonomic circumstances, are the most important category of work-related
Musculoskeletal
diseases. Different
Abstract disorders
methods(MSDs),
exist foroften causedergonomics
examining by workingthrough in non-ergonomic
a variety ofcircumstances,
postural evaluationare the most important
techniques category
(e.g. Rapid EntireofBody
work-related
Assess-
diseases. Different
ment (REBA)), methods
which exist for
are mostly examining
performed ergonomics
through manual through a varietyobservations.
survey-based of postural evaluation techniques
Consequently, (e.g. ergonomics
measuring Rapid EntireofBody Assess-
operators is
ment (REBA)),
time-consuming which
and are
happens mostly
only performed
sporadically, through
often manual
after survey-based
problems occur. observations.
Furthermore, Consequently,
as an ergonomist
In today’s business environment, the trend towards more product variety and customization is unbroken. Due to this development, the need of measuring
must ergonomics
assign a score to of
the operators
performed is
time-consuming
activity,
agile andthe and happens
ergonomics
reconfigurable only sporadically,
evaluation
production issystems
subjective. often
Theseafter
emerged problems
problems
to cope occur.
canvarious
with be Furthermore,
resolved and as
by continuous
products an and
ergonomist
product automated
families. must assignand
Toergonomic
design a score
load to theproduction
monitoring
optimize performed
meth-
activity,
ods that as
systems the ergonomics
directly
well asprovide evaluation
feedback
to choose is subjective.
to
the optimal operators.
product These
In orderproblems
matches, to monitor
productcanthebe operator
resolved
analysis by continuous
without
methods and Indeed,
areinterfering
needed. automated atergonomic
the taskmost hand, load monitoring
a vision-based
of the known methods meth-
ergonomics
aim to
ods that directly
monitoring systemprovide
is feedbackHere,
developed. to operators.
ergonomic In order
featuresto monitor
(e.g. jointtheangles)
operatorare without
estimated interfering
based onthe task at video
multiple hand, streams,
a vision-based
which ergonomics
are used to
analyze a product or one product family on the physical level. Different product families, however, may differ largely in terms of the number and
monitoring
calculate an system
objectiveis developed.
ergonomic Here, using
score ergonomic
a features
standard (e.g. jointevaluation
ergonomics angles) are estimated
technique. based oncognitive
Although multipleoperator
video streams,
support which
systems are(i.e.
used to
dig-
nature of components. This fact impedes an efficient comparison and choice of appropriate product family combinations for the production
calculate
ital work an objective ergonomic
instructions) for manual score using environments
assembly a standard ergonomics
are widely evaluation
used, theytechnique.
do Although
typically not cognitive
include operator support
ergonomics systems
information. The (i.e.
goaldig-
of
system. A new methodology is proposed to analyze existing products in view of their functional and physical architecture. The aim is to cluster
ital work instructions)
the methodology for manual
presented in this assembly
work is to environments
a) formulate orare widely
update used,work
digital they instructions,
do typically b) notprovide
includethe
ergonomics
operator information.
feedback itsThe goal of
onreconfigurable
ergonomic
these products in new assembly oriented product families for the optimization of existing assembly lines and the creation of future
the methodology
soundness via presented
work in thisand
instructions work c) is to a) formulate
generate ergonomic or risk
update digital
level work
reports in instructions,
an automated b)way
provide theonoperator
based the feedback
outcomes of on
the its ergonomic
vision-based
assembly systems. Based on Datum Flow Chain, the physical structure of the products is analyzed. Functional subassemblies are identified, and
soundness
ergonomicsvia work instructions
evaluator. Making the and c) generate
ergonomic ergonomic
guidelines risk level reports
context-aware ensures in that
an automated
interventionswayoccur
basedaton
thethe outcomes
right time. Theof the vision-based
methodology is
a functional analysis is performed. Moreover, a hybrid functional and physical architecture graph (HyFPAG) is the output which depicts the
ergonomics
integrated inevaluator.
a framework Making the ergonomic
for context-aware guidelines
work context-aware
instructions ensures
and is validated viathat interventions
a proof occur
of concept, at the
based on anright time.
actual The methodology
industrial case. Results is
similarity
integrated between productfor
in a framework families by providing
context-aware design support
work instructions and to both, production
is validated via a proof system planners
of concept, andon
based product designers.
an actual industrialAncase.
illustrative
Results
show that context-awareness enhances user acceptance and ergonomics of the operator.
example
show thatofcontext-awareness
a nail-clipper is used to explain
enhances user the proposed
acceptance andmethodology.
ergonomics of Anthe
industrial
operator.case study on two product families of steering columns of
thyssenkrupp Presta France is then carried out to give a first industrial evaluation of the proposed approach.
©© 2022The
©2017 TheAuthors.
Authors.Published
Publishedby byElsevier
ElsevierB.V.Ltd.
© 2022
This
This is
is The
an
an Authors.
open
open access
access Published
article
article by Elsevier
under
under the
the CC
CC Ltd.
BY-NC-ND
BY-NC-ND license
license (https://creativecommons.org/licenses/by-nc-nd/4.0)
(http://creativecommons.org/licenses/by-nc-nd/4.0/)
Peer-review
This is an openunder responsibility
access of the scientific
article under CC BY-NC-ND committeelicenseof the 28th CIRP Design Conference 2018.
(http://creativecommons.org/licenses/by-nc-nd/4.0/)
Peer-review
Peer-review under responsibility
responsibility ofofthe
thescientific
scientific committee
committee ofofthethe
9 th
9thCIRP
CIRP Conference
Conference ononAssembly
AssemblyTechnology
Technologyandand
Systems
Systems.
Peer-review under responsibility of the scientific committee of the 9th CIRP Conference on Assembly Technology and Systems.
Keywords:Assembly;
Keywords: Design manufacturing;
Human-centered method; FamilyErgonomics;
identificationOperator Support Systems; Manual Assembly Systems
Keywords: Human-centered manufacturing; Ergonomics; Operator Support Systems; Manual Assembly Systems

1. Introduction longed statistic body posture, etc. Moreover, it is shown that

1.1.Introduction
Introduction of the product
longed statisticrange
body and characteristics
posture, etc. manufactured
Moreover, and/or
poor ergonomics during manual assembly hasita isdirect
shown that
impact
assembled
poor in
ergonomics this
on product quality [4]. system.
during In this
manual context,
assembly the main
has a challenge
direct impact in
Musculoskeletal disorders (MSDs) are accounted for 53% of
Due to the fast
Musculoskeletal development
disorders (MSDs) areinaccounted
the domainfor 53% of
of modelling
on product and analysis
quality [4]. is now not only
In most cases, the immediate work environment conditionto cope with single
all work-related diseases in the European Union (EU) [1] and
communication
all remain the and
thuswork-related mostancommon
diseases ongoing
in the Europeantrend of
work-related digitization
Union [1] and
(EU)problem.
health and products,
hasIna most a limited
large cases,
impact.theproduct range work
immediate
Nevertheless, or
in existing product
theenvironment
context families,
condition
of high vari-
digitalization,
thus remain manufacturing
the most common enterprises
work-related
According to the report of the European Agency for Safety and are facing
health important
problem. but
has also
a to
large be able
impact. to analyze and
Nevertheless, to compare
in
ance, low volume (HVLV) assembly systems, especially the products
context of to
high define
vari-
for
challenges
According in
to today’s
the report market
of the environments:
European
Health at Work [2], MSDs are responsible for more than half Agency a
for continuing
Safety and new
ance, product
low families.
volume It
(HVLV) can be observed
assembly that
systems,
large machinery, ergonomic workstation design is not trivial. classical existing
especially for
tendency
Health attowards
Work reduction
[2], MSDs of
are product
responsibledevelopment
of all absences from work in the EU, which results in an esti- for more times
than and
half product
large families
machinery, are regrouped
ergonomic in function
workstation of clients
design
Several methods exist for examining work ergonomics through is or
notfeatures.
trivial.
shortened
of costproduct
all absences
mated in thefromlifecycles.
regionwork Inthe
ofin2% addition,
ofEU,the whichthere
gross is an increasing
results
domestic inproduct.
an esti- However,
Several assembly
a varietymethods
of postural oriented
exist evaluationproduct
for examining families
techniques, are hardly
work ergonomics
such toRapid
find.
as thethrough
demand
mated of
cost customization,
in the region ofbeing
2% ofat the
the same
gross
The highest rate of complaints was recorded among industrial time
domestic in a global
product. a On
varietythe product
of posturalfamily level,
evaluation products
techniques,
Entire Body Assessment (REBA) [5]. Most of these evalua- differ
such mainly
as the in two
Rapid
competition
The highest with
rate ofcompetitors
complaints all
was over the
recorded
operators and manual assemblers [3]. Work-related MSDs refer world.
among This trend,
industrial main
Entire characteristics:
Body (i)
Assessment the number
(REBA) of
[5].components
Most
tion methods are manual survey-based observation techniques. of and
these (ii)
evalua-the
which
operators is inducing
and manual the development
assemblers [3]. from macro
Work-related
to reduction of the human functionality or damage of human MSDsto micro
refer type
tion of components
methods are (e.g.
manual mechanical,
survey-based electrical,
observation
These paper-based ergonomic risk assessment tools are very electronical).
techniques.
markets,
to
bodyreductionresults
of the
structures. in human
These diminished lot are
functionality
impairments sizes ordue
mainly to augmenting
damage
caused ofbyhuman
repet- Classical
These
time consuming methodologies
paper-based andergonomic
thereforeconsidering
work mainly
risk assessment
cells single
are often products
toolsonly are anal-
very
product
body varieties
structures. (high-volume
These impairments to low-volume
are mainly
itive manual activities, lifting and transporting heavy loads, pro- production)
caused by [1].
repet- or
ysed after problems occur. Moreover, the assessment of anthe
timesolitary,
consuming already
and existing
therefore product
work families
cells are often analyze
only anal-
er-
To cope
itive manualwithactivities,
this augmenting
lifting and variety as wellheavy
transporting as to loads,
be ablepro-to product
ysed after structure
problems on aoccur.
physical level (components
Moreover, the assessment level)of which
an er-
2212-8271 © 2022 The Authors. Published by Elsevier Ltd.
identify
2212-8271 possible
© 2022
This is an open
optimization
Thearticle
access Authors. Published
under
potentials in the existing
by Elsevier Ltd.
the CC BY-NC-ND
causes difficulties regarding an efficient definition and
license (http://creativecommons.org/licenses/by-nc-nd/4.0/)
production
This is an system,
open access it is
article
Peer-review under responsibility ofimportant
under the CC to have
BY-NC-ND a precise
license knowledge comparison ofTechnology
differentandproduct
(http://creativecommons.org/licenses/by-nc-nd/4.0/)
scientific committee of the 9th CIRP Conference on Assembly Systems. families. Addressing this
Peer-review under responsibility of the scientific committee of the 9th CIRP Conference on Assembly Technology and Systems.
2212-8271 © 2022 The Authors. Published by Elsevier Ltd.
This is an©open
2212-8271 2017access article Published
The Authors. under theby CC BY-NC-ND
Elsevier B.V. license (https://creativecommons.org/licenses/by-nc-nd/4.0)
Peer-reviewunder
Peer-review under responsibility
responsibility of the
of the scientific
scientific committee
committee of theof28th
the CIRP
9 th CIRP
DesignConference
Conferenceon2018.
Assembly Technology and Systems
10.1016/j.procir.2022.02.173
 Arno Claeys et al. / Procedia CIRP 106 (2022) 168–173
A. Claeys et al. / Procedia CIRP 00 (2022) 000–000
gonomic score is subjective, as the performed activities are
scored by the observation of only one person.
Given the trend towards flexible assembly systems [6], ini-
tiated by the demand for mass customized products, a static
ergonomic risk analysis of a work cell no longer reflects the
true ergonomic load of an operator due to the rapid change of
tasks. Next to the work environment, the ergonomic workload
is mainly influenced by the operator’s own actions. Therefore,
it is of importance to encourage the operator to perform work
using the correct ergonomic techniques.
In literature, different approaches for continuous ergonomic
load monitoring are described [7, 8, 9]. In the scope of indus-
trial applications, these principles are mainly used for physical
human-robot applications [10]. Nevertheless, it is not known
how this can be done over prolonged time periods without af-
fecting or distracting the operator. Furthermore, no methods on
how ergonomic feedback effectively can be provided towards
operators on the shopfloor is found. While digital work instruc-
tions (DWIs) are available to offer cognitive support towards
operators within HVLV assembly environments [11], there is
currently no integration to inform the operator on his ergonomic
soundness.
This work provides a methodology for integrating er-
gonomics information into assembly information to provide
feedback on ergonomic soundness of the task execution via
DWIs. The remainder of the paper is organized as follows.
In Section 2, a video-based approach for monitoring the er-
gonomic surroundings of an operator is proposed. Next, the
integration of ergonomics information within assembly infor-
mation is described in Section 3. First validation tests for the
proposed methodology are conducted and described in Section
4. Finally, the conclusions and an outlook is to be found in Sec-
tion 5.
2. Ergonomics monitor
To warrant objective and continuous ergonomic load mon-
itoring, a vision-based ergonomics monitor is developed. The
set-up consists of low-cost, low intrusive sensing solutions that
allow for measurements over prolonged time periods without
affecting or distracting the operator. Based on multiple video
streams, recorded at an assembly work cell, the operator’s pos-
ture, and finally a real-time ergonomic score, is calculated. Fig-
ure 1 gives an overview of all analysis steps within the er-
gonomics monitor. First, a convolutional neural network (CNN)
is used to detect body key-points in a frame image. Next, key
points coming from the different video streams are intersected
in space to determine the three-dimensional (3D) position of
the body joints. Finally, based on a standard ergonomic norm
(REBA), an ergonomic score is computed via the joint angles.
The algorithms described below are written in Python and make
use of the Pytorch library for CNN processing and OpenCV for
image processing.
2
169
2
2.1. Key-points detection
Via the HigherHRNet CNN [12], pixel positions for the dif-
ferent key-points are retrieved. The key-points of interest are:
nose, eye (left/right), shoulder (left/right), elbow (left/right),
wrist (left/right), hip (left/right), knee (left/right) and ankle
(left/right). This network follows the bottom up approach.
2.2. 3D inference
To retrieve accurate projective points, all key-point positions
are corrected via calibration data. Based on the camera posi-
tions, a ray (i.e. 3D line) is casted for each point through space.
The intersections of these rays are used to calculate a 3D point
per key-point. As there is some error margin on the 2D points,
it is possible that the rays from the different viewpoints do not
intersect exactly in space. For this reason, an approximate solu-
tion is computed by minimizing the square of the distance to the
rays. In this way, a three-dimensional skeleton can be inferred
for an individual. When more than one individual is detected in
the images, a heuristic algorithm groups the views of the differ-
ent individuals by minimizing the projection error on the com-
puted key-point positions [13]. To be able to compute the po-
sition of a joint, at least two camera views are required. When
occlusions occur, the maximum of the available rays are used.
If less than two rays are available, the joint status considered as
unknown and will be ignored for the rest of the processing.
2.3. Ergonomics assessment
Based on the 3D key-points, obtained in the previous step,
all the body joint angles are computed. First, the sternum and
pelvis locations are estimated based on the centre of the rays
from the shoulders and hips, respectively. The back flexion
is represented by the angle between the vertical axis and the
spine. Arm and leg torsions can only be computed if there are
enough flexions in the elbows and knees. Therefore, wrist and
ankle angles cannot be inferred reliably within this system and
will therefore be ignored. To calculate the angles of the upper
body, a standard frame is attached to the sternum. This frame is
aligned with the spine axis and the shoulders. The head frame
is registered from the nose, eyes, and ears positions. The neck
flexion, rotation and lateral flexion are derived from the chained
rotations between the sternum and the head frame. The arm
frames are created via the wrist, elbow, and shoulder joint po-
sitions. Via these frames, arm rotation, flexion, and adduction
angles are calculated. In the same way, hip flexion, adduction
and rotations are computed using the ankle, knee and hip joint
positions. Elbow and knee flexion are computed from the angle
between the upper and lower arms and legs respectively. Simi-
lar to the sternum frame, a hip frame is attached to the pelvis.
The back torsion is calculated by the chain rotation between
the sternum frame and the hip frame. Based on the calculated
joint angles, a partial REBA analysis is performed. Two groups
are scored: group A (trunk, neck, and legs) and group B (up-
per arms, lower arms, and wrists). As indicated above, the wrist
joint is ignored in the proposed ergonomics monitor. For each
170 Arno Claeys et al. / Procedia CIRP 106 (2022) 168–173
A. Claeys et al. / Procedia CIRP 00 (2022) 000–000 3

Fig. 1. Ergonomics monitor video processing steps

group, a score is obtained by using the corresponding REBA ta-

ble. Next to joint angles, additional parameters (e.g. force/load,
coupling, and activity score) can be set via an interface. A final
REBA score is obtained by further processing the scores from
both group A and B, and by adding an activity score. In this
way, a REBA score is automatically calculated per video frame
for each detected individual.

3. Ergonomics information in contextualized digital work

instructions

During previous research, a middleware framework for

context-aware operator support within HVLV assembly envi-
ronments is developed [14, 15]. This framework, illustrated
in Figure 2, consists of four domain ontologies for creating
an abstract structured representation of assembly information
to enable the integration of algorithms for optimization, au-
thoring and context-awareness of digital work instructions. In
the context of this work, an ergonomics module is added to
the framework to integrate information coming from the er-
gonomics monitor, described above, and to ensure the predeter-
mined functionalities for optimal ergonomics monitoring and
feedback.
The assembly information model structures manufactur-
ing data (e.g. resource information, manufacturing operations,
manufacturing performance, etc.) based on the industrial stan-
dard ISA-95 (IEC 62264, Enterprise-control system integra-
tion). Existing business and manufacturing data are trans-
lated to the standardized models through an assembly informa-
tion configurator. Business to manufacturing markup language
(B2MML) provides an ISA-95 based method to exchange this
type of data. As illustrated in Figure 3, three parts of the as-
sembly information model are used in this work. The task ob-
ject contains all information to assemble a predefined product
via parameters, resource (personnel, equipment, and material)
specifications and a workflow specification. In task parame-
ters, additional information such as configurations or settings
(e.g. torque value) are stored. Via resource specifications, the
required resources are specified along with a quantity setting.
A workflow gives detailed steps on how to perform an assem-
bly operation, and may have linked DWIs. An order contains all
scheduling related information for a product order. Each order
refers to one task and specifies order dependent information.
3
Fig. 2. Middleware framework for context-aware operator support
When an order is executed, a report is generated, which con-
tains all performance related information such as timestamps,
parameter data (e.g. sensor feedback) and actual consumed re-
sources. Each report may correspond to an order, a task or a
workflow step.
DWIs are typically modelled as flat and unstructured docu-
ments (e.g. pdf-files, images, etc.), which causes a number of
drawbacks. Therefore, an assembly instruction model is devel-
oped to store DWIs in a structured and modular way. The main
principle is to split instructions into components which are de-
fined by an instruction component type (e.g. text, image, 3D
drawing, etc.). Furthermore, individual components can be en-
riched with contextual data which facilitates personalization of
DWIs.
To contextualize assembly information, a generic context
model is fundamental. A categorization of context types will
help the application designer to uncover the most useful pieces
of context for a certain application. The primary context types
for characterizing the situation of a particular entity are loca-
tion, identity, activity, and time [16]. These context types not
only answer the questions of who, what, when, and where, but
also act as indices into other sources of contextual information.
 Arno Claeys et al. / Procedia CIRP 106 (2022) 168–173
A. Claeys et al. / Procedia CIRP 00 (2022) 000–000
Fig. 3. Integration of ergonomics information within assembly information
Thus, primary pieces of context for one or more entities can
be used to derive secondary context for that same entity. This
principle is used to derive assembly related context form both
historical data (e.g. manufacturing data) and sensor data. Possi-
ble types of secondary assembly context categories include ex-
perience, condition, requirements and, within the scope of this
work: ergonomics.
Based on the ergonomic scores obtained from the er-
gonomics monitor, the ergonomics module performs several ac-
tions as illustrated in Figure 3. Below, a detailed description of
these different actions is given.
3.1. Automated risk level reports
An ergonomic risk level report (Figure 3, a) is an overview
of all monitored ergonomic threats for a certain operation. As
stated before, a report is generated for each step within a task.
The output from the ergonomics monitor is normalized and
logged per second and is added to the report. In the scope of
this work, the REBA scores per joint (e.g. legs, shoulders, etc.)
and per group (A, B and C) are stored. Per step, an ergonomic
score is determined by taking the maximum moving average
score. Hence, short impulses are filtered and idle recordings are
eliminated. Based on these reports, flags can be placed on tasks
when the ergonomic score exceeds a given threshold, or when
the score is remarkably higher compared to similar tasks. In
this way, non-ergonomic situations can be detected in an earlier
stage (i.e. before injuries occur). Flagged tasks can be evaluated
by an ergonomist and ergonomics parameters (e.g. ergonomic
risk, REBA threshold) can be updated for that task. Based on
these parameters, decisions can be made to optimize the work-
station, and the ergonomic performance of an operator can be
monitored more accurately. Moreover, based on individualized
operator reports (Figure 3, b), operators can be reallocated in
order to balance the ergonomic load [17].
3.2. Formulate and update work instructions
After ergonomics analysis, DWIs may require updates (Fig-
ure 3, c). Instruction content can be edited by adding, removing
4
171
4
or correcting instruction components. This process can be par-
tially automated by suggesting instruction components based
on context. Based on the ergonomics parameters, obtained from
the ergonomics monitor, suggestions are made for adding rele-
vant instruction components (e.g. an awareness icon for lifting
operations) that inform the operator to use the most optimal
way for executing the task ergonomically. In some cases, more
detailed information in the form of a guideline is required to
inform the operator about the correct ergonomic method. Here,
a procedure is updated by adding a guideline instruction to the
workflow.
3.3. Feedback on ergonomic soundness
By making ergonomics information context-aware, content
filtering can be applied. Based on historical performance data,
stored in the reports, decisions can be made to show or hide
a certain instruction. For example, when an operator starts
falling back into old (bad) habits, an ergonomic guideline can
be shown at the next execution of the task. Decision rules
may be implemented in a static (e.g. periodic, ergonomic er-
rors, etc.) or dynamic way (e.g. recommender system based
on operator feedback). Allowing the operators to give feedback
on the instructions is highly necessary when applying context-
awareness, as this may be used to refine the context filter rules.
4. Validation
The methodology proposed in this work is implemented in a
proof of concept based on an industrial use case. The tasks used
during the validation process are originated from the assembly
of a pneumatic air compressor. The ergonomic risks identified
within the tasks are: lifting heavy objects (> 8kg), rotation and
bad posture. The use case assembly work cell is replicated in
a controlled lab environment equipped with a set of sensors,
including four industrial cameras (Allied Vision Manta G201C,
1/1.8” 4.4 11mm Varifocal Lens) connected to a computer with
Cuda capabilities (Nvidia RTX 3070 GPU).
Figure 4 gives a schematic overview of the proof of concept
system architecture. The central part is the Assembly Knowl-
172 Arno Claeys et al. / Procedia CIRP 106 (2022) 168–173
A. Claeys et al. / Procedia CIRP 00 (2022) 000–000
Fig. 4. Proof of concept system architecture
edge Store (AKS), containing the ontology schemas described
in Section 3. Assembly information, DWIs and ergonomic
guidelines are managed in a cloud based DWI platform1 . Dur-
ing task execution, timestamps and explicit operator feedback
are logged and stored in a report that corresponds to a product
order. Via an application programming interface (API), all rel-
evant manufacturing data (task, scheduling, and performance
data) are exported. Via the assembly information configurator
(Figure 2), the DWI platform data model is transformed to ISA-
95 models and transmitted as B2MML messages. In this way,
all manufacturing data coming from the DWI platform are in-
tegrated in the AKS. When new information is available, the
DWI platform pushes it to the AKS.
Via an ergonomics configurator, both assembly and DWI
data are enriched with ergonomics information. For the proof
of concept, ergonomic risks (e.g. lifting), ergonomics param-
eters and a REBA threshold are assigned manually to a task
step. The threshold scores are determined by comparing er-
gonomic scores coming from the ergonomics monitor to eval-
uations done by an ergonomist. Based on this information, the
system determines if an operator’s execution could be ergonom-
ically improved.
The ergonomics monitor calculates a real-time REBA score
based on the postural body angles, and the ergonomics parame-
ters (force/load, coupling, and activity score) obtained from the
AKS for the active task step. The ergonomic measurements are
returned to the AKS and are linked to performance data orig-
inated from the DWI platform. In this way, all measurements
are linked to an identifiable task step. After each task iteration,
the ergonomic scores are evaluated per ergonomic risk, and
stored as a relation between the person and the corresponding
ergonomic risk. When the evaluated score exceeds the thresh-
old, the evaluation is flagged.
Ergonomic guidelines are equipped with context filter rules,
that check if a context status for a defined target is met. Here,
1 https://www.azumuta.com
5
5
the target is the ergonomic risk that corresponds to the guide-
line, and the context status is the ergonomic error flag. Thus,
if an ergonomic error is made for the ergonomic risk that cor-
responds to the guideline, then the guideline is considered as
relevant. The DWI platform is configured to communicate with
the AKS via WebSocket. Based on the context filter rule, a con-
text request is send to the AKS. Based on the relation between
the person (active user of the DWI platform) and the ergonomic
risk, a relevance score is returned that determines if a guideline
should be presented.
The proposed proof of concept is validated via an experi-
mental approach. Fifteen participants (engineering students and
academic personnel) performed ten assemblies each. The par-
ticipants were divided into three groups. The first group (sce-
nario A) only received assembly related information and was
thus unaware of ergonomic soundness. The second group (sce-
nario B) received static ergonomic guidelines as a part of the
assembly instructions. Here, the operator received the same in-
formation for each iteration. For the last group (scenario C),
ergonomics information was contextualized meaning that in-
structions are filtered based on the participant’s ergonomic per-
formance.
Table 1. Average REBA scores per ergonomic risk per scenario
Ergonomic risk Scenario Av. REBA
A 4,39
Lifting B 4,35
C 4,17
A 3,23
Posture B 2,67
C 3,04
A 3,23
Rotation B 3,35
C 3,10
In Table 1, the average REBA scores, recorded during the
experiment runs, are summarized per ergonomic risk and per
 Arno Claeys et al. / Procedia CIRP 106 (2022) 168–173
A. Claeys et al. / Procedia CIRP 00 (2022) 000–000
scenario. Here, a positive evolution is visible when an opera-
tor receives ergonomic guidelines (scenario B and C). Never-
theless, when guidelines are delivered too frequently (scenario
B), the operator ignores the guidelines. Consequently, the op-
erator falls back into old habits which has a negative impact
on the ergonomic score. Based on explicit operator feedback,
a negative rating was visible for scenario B while for scenario
C, operators were more satisfied with the received information.
Although this proof of concept is validated over a short period
with a limited number of participants, the positive impact of
context-aware ergonomic guidelines is visible.
5. Conclusions and further research
This work proposes a methodology for integrating er-
gonomics information in contextualized digital work instruc-
tions. To monitor the operator without interfering the task
at hand, a vision-based ergonomics monitoring system is de-
veloped. Via vision techniques, ergonomic features are esti-
mated and are used to calculate an objective ergonomic score
based on the REBA evaluation technique. Based on first tests,
the obtained scores are found accurate. Next, a methodol-
ogy to integrate ergonomics information within context-aware
operator support systems is proposed. The methodology en-
ables the automatic generation of ergonomic risk level reports,
through which shopfloor operators can be informed with their
ergonomic soundness and DWIs can be updated or reformu-
lated based on ergonomic performances. To validate the pro-
posed methodology, a proof of concept, based on an actual in-
dustrial case is built in a controlled lab environment. A first
experimental test shows that context-aware ergonomics infor-
mation and guidelines have a positive impact on the operator’s
motivation to perform actions in an ergonomically correct way.
While these first results are promising, there are some limita-
tions that should be marked. The scores returned by the er-
gonomics monitor were only calculated via the REBA evalu-
ation method, which is not the ideal assessment tool for all er-
gonomic situations (e.g. lifting). During further work, more er-
gonomics evaluation methods will be added to the ergonomics
monitor. Also, it is hard to interpret the ergonomic scores with-
out manual intervention. This could be resolved by applying
machine learning algorithms that can derive additional context
and add it to the ergonomic reports (e.g. the intention of an op-
erator). The tests were performed over a short period of time
and with a limited amount of participants in a controlled lab en-
vironment. As a next step, the proposed methodology should be
tested for a longer period in a real industrial setting. Based on
these results, steps could be taken to configure a recommender
system based on the ergonomic performance of an operator.
Acknowledgements
This research is supported by VLAIO (Flanders Innovation
& Entrepreneurship) and Flanders Make, the strategic centre for
the manufacturing industry in Flanders, within the framework
6
173
6
of the academia-industry collective project Ergo-EyeHand, Er-
gonomic Monitoring and Improvement (HBC.2018.0250).
References
[1] S. Bevan, “Economic impact of musculoskeletal disorders (MSDs) on work
in Europe,” Best Practice and Research: Clinical Rheumatology, vol. 29,
no. 3, pp. 356–373, 2015.
[2] J. De Kok, P. Vroonhof, J. Snijders, G. Roullis, M. Clarke, K. Peereboom,
P. van. Dorst, and I. Isusi, Work-related musculoskeletal disorders : preva-
lence, costs and demographics in the EU. 2019.
[3] J. Grobelny and R. Michalski, “Preventing work-related musculoskele-
tal disorders in manufacturing by digital human modeling,” International
Journal of Environmental Research and Public Health, vol. 17, no. 22,
pp. 1–19, 2020.
[4] A.-C. Falck, R. Örtengren, and D. Högberg, “The impact of poor assem-
bly ergonomics on product quality: A cost–benefit analysis in car manu-
facturing,” Human Factors and Ergonomics in Manufacturing & Service
Industries, vol. 20, no. 1, pp. 24–41, 2010.
[5] L. McAtamney and S. Hignett, “Rapid Entire Body Assessment,” Hand-
book of Human Factors and Ergonomics Methods, vol. 31, pp. 8–1–8–11,
2004.
[6] L. V. D. Ginste, A. D. Cock, A. V. Alboom, Y. Singh, E.-H. Aghezzaf, and
J. Cottyn, “A formal skill model facilitating the design and operation of
flexible assembly workstations,” in IFIP International Conference on Ad-
vances in Production Management Systems, pp. 108–116, Springer, 2021.
[7] U. Jayaram, S. Jayaram, I. Shaikh, Y. Kim, and C. Palmer, “Introducing
quantitative analysis methods into virtual environments for real-time and
continuous ergonomic evaluations,” Computers in industry, vol. 57, no. 3,
pp. 283–296, 2006.
[8] A. Yazdani, R. S. Novin, A. Merryweather, and T. Hermans, “Ergonomi-
cally intelligent physical human-robot interaction: Postural estimation, as-
sessment, and optimization,” arXiv preprint arXiv:2108.05971, 2021.
[9] M. Peruzzini, F. Grandi, and M. Pellicciari, “Exploring the potential of op-
erator 4.0 interface and monitoring,” Computers & Industrial Engineering,
vol. 139, p. 105600, 2020.
[10] L. Gualtieri, I. Palomba, E. J. Wehrle, and R. Vidoni, “The opportunities
and challenges of sme manufacturing automation: safety and ergonomics
in human–robot collaboration,” in Industry 4.0 for SMEs, pp. 105–144, Pal-
grave Macmillan, Cham, 2020.
[11] S. Makris, G. Pintzos, L. Rentzos, and G. Chryssolouris, “Assembly sup-
port using ar technology based on automatic sequence generation,” CIRP
Annals, vol. 62, no. 1, pp. 9–12, 2013.
[12] B. Cheng, B. Xiao, J. Wang, H. Shi, T. S. Huang, and L. Zhang, “Higherhr-
Net: Scale-aware representation learning for bottom-up human pose esti-
mation,” Proceedings of the IEEE Computer Society Conference on Com-
puter Vision and Pattern Recognition, pp. 5385–5394, 2020.
[13] S. Dehaeck, C. Domken, T. Bey-Temsamani, and G. Abedrabbo, “A strong
geometric baseline for cross-view matching of multi-person 3d pose esti-
mation from multi-view images (in press),” ICIAP 2021 Conference Pro-
ceedings, 2021, 2021.
[14] A. Claeys, S. Hoedt, H. Van Landeghem, and J. Cottyn, “Generic Model for
Managing Context-Aware Assembly Instructions,” IFAC-PapersOnLine,
vol. 49, no. 12, pp. 1181–1186, 2016.
[15] A. Claeys, S. Hoedt, M. Schamp, H. V. Landeghem, and J. Cottyn, “Onto-
logical Model for Managing Context-aware Assembly Instructions,” IFAC-
PapersOnLine, vol. 51, no. 11, pp. 176–181, 2018.
[16] A. Dey, G. Abowd, and D. Salber, “A Conceptual Framework and a Toolkit
for Supporting the Rapid Prototyping of Context-Aware Applications,”
Human-Computer Interaction, vol. 16, no. 2, pp. 97–166, 2001.
[17] G. Michalos, S. Makris, L. Rentzos, and G. Chryssolouris, “Dynamic job
rotation for workload balancing in human based assembly systems,” CIRP
Journal of Manufacturing Science and Technology, vol. 2, no. 3, pp. 153–
160, 2010.