Reengineering an Industrial HMI: Approach,
Objectives, and Challenges
Bernhard Dorninger∗ , Michael Moser∗ , and Albin Kern†
∗ Software Competence Center Hagenberg, Austria
{firstname.lastname}@scch.at
† ENGEL Austria GmbH, Austria
albin.kern@engel.at
Abstract—Human Machine Interfaces (HMI) play a pivotal
role in operating industrial machines. Depending on the extension
of a manufacturers domain and the range of its machines as well
as the possible options and variants, the ensuing HMI component
repository may become substantially large, resulting in significant
maintenance requirements and subsequent cost. A combination
of cost pressure and other factors, such as significant change
of requirements, may then call for a substantial reengineering.
A viable alternative to manually reengineering the whole HMI
framework might be the use of (semi)-automated reengineering
techniques for suitable parts. We describe such a model based
reengineering procedure relying on static analysis of the existing
source code for suited aspects of a large HMI framework. We will
sketch our overall approach including the objectives and highlight
some important challenges of transforming HMI component
information extracted from source code into a representation
developed for the completely redesigned HMI infrastructure in
the light of an existing product assembly and configuration
process at a large machinery manufacturer.
I. I NTRODUCTION
Reengineering of large software systems is a complicated
and usually labor-intensive task, with the reasons for reengi-
neering a productive software system being manifold. In [1],
Parnas clusters reasons for software becoming obsolete and
sketches possible strategies to deal with the inevitable process
of software aging. These reasons may include end-of-life of
key technologies, new requirements difficult or impossible
to fulfill with operational technologies, or just exuberant
maintenance cost.
Once the decision to reengineer a software system has
been made the extent of reengineering has to be determined.
A complete redesign from scratch is not always possible or
(economically) reasonable, but on the other hand quite often
inevitable due to structural degrading of a software system,
often referred to as design erosion[2], which is quite frequently
caused by maintenance [3]. Another interesting question arises
deciding upon manual or (semi)automated reengineering of
specific parts of the software, e.g. by applying model based
techniques. However, many parts of a system may be so
complex a cost efficient model based reengineering is not
feasible [4], [5] .
All of this applies to HMIs, too. Additional typical reasons
to initiate a major HMI reengineering include the introduction
of new and contemporary interaction concepts and/or provid-
ing a fresh and modern look and feel (L&F).
978-1-5386-4969-5/18/$31.00
c 2018 IEEE 547
A further interesting aspect concerns the flexibility of the
HMI in the context of product assembly. Particularly in series
production, HMIs are not a one-off effort, but are usually
assembled from a number of reusable components. Having
a high number of machine types with a lot of options and
variants demands a certain degree of flexibility. Maintenance
for the sake of ad-hoc flexibilizing then often thwarts the
existing architecture - frequently out of incomprehension of
the originally intended design. Parnas called this ”Ignorant
Surgery” , having an even more negative impact when follow-
ing a minimal effort strategy in maintenance [2]. Another in-
teresting aspect is the HMI part of industrial machine software
having become more and more an essential - because visible
- asset in recent years. With the rapidly evolving technologies
and design principles of HMIs - at least compared to the
evolution of Programmable Logic Control (PLC) development
standards like IEC-61131, it has become more likely to subject
the HMI to reengineering than the PLC software. This imposes
a somewhat strong requirement in such a way that it shall
become more amenable to recurring reengineering or evolution
(”Design for success” [1]).
The remainder of the paper is organized as follows: In
Section II, we provide an introduction to the industrial context
of our project and its most important objectives and require-
ments, plus its connection with the current HMI assembly and
configuration process. In Section III, we outline our proposed
reengineering approach in the context of a proposed adaption
of the assembly process. In Section IV, we highlight some
important challenges and how we plan to address or how we
have addressed them Section V briefly cites related work and
Section VI provides a short conclusion to this paper.
II. C ONTEXT, M OTIVATION , AND O BJECTIVES
ENGEL is a large machine manufacturer, who offers a
broad portfolio of machines tailorable to specific customer
needs, where each machine can be equipped and configured
with numerous different options. Regarding the engineering
process, the company covers all relevant fields and disciplines
from mechanical construction up to development of control
system and HMI software. The productive version of the
latter is still based on a proprietary Java UI library, which
is long past its end-of-life date and thus no more maintained.
Despite this, the current HMI framework is still maintained
SANER 2018, Campobasso, Italy
Industry Track
and evolving. To address the growing concerns especially with
maintenance, it has been decided to completely reengineer the
HMI software based on JavaFX [6]. This decision has also
been backed by the fact of evolution in the connectivity to the
control system, as OPC-UA [7] is set to play a key role here
in the future.
In short, the key motivational factors for reengineering the
HMI are:
• End-of-life of base HMI library - including unclear
licensing issues
• Inflexible reflection of product variants and options in the
HMI
• Antiquated L&F of old HMI, outdated interaction con-
cepts
• Tight coupling of HMI to PLC system impeding main-
tainability and testability
A. Characteristics of the Legacy HMI
Apart from infrastructural components, the HMI consists
of a high number of screens, with each screen covering a
certain aspect of a machine’s components or production related
tasks. Screens are usually made up of de-facto standardized UI
components in terms of L&F and layout behavior, but can also
be a highly specific intra-HMI application. The former are of
interest in our reengineering, whilst the latter are considered
to be too complex to be processed automatically.
Model based screens consist of so called ”stripes”, which
in turn consist of one or more so called ”base widgets” (e.g.
an component made up of a descriptive label, a number input
field and a label for the technical unit) - typically arranged in
rows. The building blocks for these base widgets can be more
simple base widgets or can be standard widgets provided by
the underlying UI library. Base Widgets display process data
of a machine (PLC variables), e.g. a certain state, selection or
actual value and sometimes also allow alterations of reference
values or set-points.
Where the composition of screens from stripes is already
achieved via a model, stripes and base widgets are manually
coded. In addition, stripe code may contain hardcoded options,
typically conditional code blocks creating additional rows in
the stripe. Conditions typically check the presence or even the
value of one or more PLC variables on the machine.
Figure 1 shows a simplified example of a screen consisting
of two stripes. The first strip consists of a single line using a
”double input field ” base widget. The second stripe consists of
more lines using different types of base widgets. The callout
shows one of the base widgets, which features a description
label, an input field, a label for the physical unit and an
indicative mark, e.g. for signaling the value exceeding a limit.
B. The HMI in the Product Assembly Process
As mentioned, XYZ builds machines allowing different op-
tions and variants from a broad range of de-facto standardized
core products. This variability also is reflected in the HMI. As
of now, the set of screens a machine HMI provides, is decided
at engineering time. This also includes the content of a screen,
548
Fig. 1. Sample of a HMI screen
i.e. which stripes a screen contains. Screen assembly and
configuration is already automation-supported by the current
product assembly and configuration process. The result is
a screen model, which is processed by the machine HMI
application at runtime. However, as mentioned in Section II-A,
the current process does not fully extend this approach to the
level of stripes - as the stripes themselves are hardcoded.
Stripe options and variants may be present in three types:
OT-1 Hardcoded options in stripes: Typically, stripe code
adds additional widget rows shielded by conditional ex-
pressions.
OT-2 Subclass variants: the stripe code defining the content
exists in the form of an abstract class with subclasses
injecting a hardcoded PLC variable name prefix or some
other parameters at construction time.
OT-3 Project Clones: a whole stripe project is copied and
slightly altered at various spots.
The first form of variability is attended to at runtime -
upon starting the HMI. During HMI initialization, the stripe
code checks the presence of PLC variables or its value to
decide upon the widgets to be created and displayed. The other
two forms are covered at engineering time by the assembly
and configuration process, which has to select the correct
stripe code and resource artifacts, depending on the machine
configuration.
C. Reengineering Objectives
An overall objective of the new HMI framework is allowing
the definition of the stripe contents and options with models.
Thus, the screen meta model of the existing HMI framework
has to be extended to not only allow the specification screens
and the contained stripes, but also to enable the specification
of a stripes content in a straightforward manner. Building new
stripes and assembling them to screens - manually as well
as in the context of the established automated assembly and
configuration process - shall be supported in a comprehensible
way.
On the other hand, reuse of stripes from the legacy HMI
repository shall be supported, too. Consequently, we identify
five main objectives, which dominate our efforts of stripe
legacy code:
1) Extract the HMI containment tree of each stripe to a im-
plementation technology agnostic meta model (Abstract
User Interface - AUI)
2) Preserve stripe component attributes, especially the bound
PLC variables
3) Preserve conditional variants and options
4) Provide repeatable generation of target artifacts, partic-
ularly screen model fragments for stripe content and
binding information for PLC variables.
5) Resulting representation of legacy stripe reengineering
must fit the existing automated HMI assembly and con-
figuration process
III. A PPROACH
Before elaborating on our approach of re-engineering the
stripe code, we will sketch our approach of enhancing the
existing HMI assembly and configuration process.
A. Enhancing the HMI Assembly Process
Fig. 2. HMI assembly process (simplified)
Figure 2 shows the intended process for simplification
we focus on the aspects of assembling a stripe and not the
full HMI. The cornerstones of this process enhancements is
the decomposition of stripes to fragments plus extracting the
information concerning their combination (Option Info, OI)
with respect to machine type and available PLC variables. We
identify five steps for a single stripe model to be created.
1) Query Option Info
2) Find relevant PLC variable definitions
3) Determine required fragments
4) Gather stripe fragments from Repository
5) Combine Fragments and merge result model
Stripe fragments as well as OI reside in a database. Assem-
bling a single stripe starts by querying the OI for this specific
stripe providing the machine type and the screen the stripe will
be part of (1). The next step (2) looks up the PLC variables
needed for evaluating the conditions specified by the OI. A
machine PLC project produces files containing the PLC system
variables (SV) of each machine component. During this step,
the SV files relevant for a stripe are searched for the presence
of the necessary variables. In step (3), the set of relevant stripe
549
fragment references is determined by evaluating the conditions
from the OI. The references are used to fetch the respective
fragments from the stripe fragment repository in step (4). In
the final step (5), these fragments are combined and the result
is merged into the machine screen model. These steps have to
be repeated for each stripe in a screen.
B. Reengineering of Legacy Stripes
In the light of this intended process (enhancement), we
take the following 4 step approach depicted in figure 3 to re-
engineering, which relies on static analysis of the stripe source
code
Fig. 3. Approach
1) Candidate Analysis: The first step has been a predomi-
nantly search-based identification of the relevant base widgets
used by the stripes and the relevant methods of these widgets
being called during initialization. In addition we examined the
stripe source code regarding structural peculiarities which may
originate in e.g. the programmer’s coding habits to get a rough
idea of the language features the analyzer software needs to
support.
2) Parsing Stripe Source: For parsing and generating the
AST of a stripe, we employed a standard Java Parser provided
by the Eclipse IDE (JDT). The choice for this parser was
natural due to Eclipse being the established development tool
for HMI components at ENGEL.
3) AST-Based Analysis: The AST ensuing from parsing
provides the basis for the analysis step, which accomplishes
the extraction of the required information to the AUI. The
AUI acts as an intermediate entity for decoupling analysis and
the following processing step. Information extraction works
by partial interpretation of the method source code along the
creation call path of the analyzed stripe.
4) Processing Analysis Models: After the analysis the stripe
AUI instances are subjected to three processing tasks, which
shall produce the information sketched in III-A.
a) Stripe AUI Decomposition: The stripe AUI instances
gained from the analysis are decomposed into fragments
with each none-empty condition branch in a stripe be-
coming a distinct fragment in context of that stripe. This
includes PLC relevant information such as a widgets PLC
variable attributes as well as image references and text
keys, which are evaluated to the concrete images/texts
at runtime. The resulting fragments are stored into the
fragment repository.
b) Condition to Fragment Mapping: In legacy stripe code,
the addition of a widget to a stripe may be coded
within the block of a conditional branch. In the AUI
 from the analysis, this fact is memorized by mapping
the conditions to the partial sub-tree of a container. In
this step, this mapping information is externalized into a
separate index record in the repository, becoming the key
part of the OI in the stripe fragment repository.
c) L&F Attribute Management: An additional task deals with
the HMI widget attributes not directly relevant for the
content of a stripe, but rather its appearance (e.g. color
and font attributes). JavaFX follows a different paradigm
with L&F attributes than legacy Java UI frameworks
like AWT or Swing. While the latter only offer the
direct setting of attributes via API calls, JavaFX strongly
encourages such attributes to be set via one or more
centralized stylesheets (a reduced variant of W3C CSS
[8]). Thus, at the moment this task serves merely to
collect used colors and fonts as a means of verification
for the HMI designers (e.g. for detecting special color
definitions).
IV. C HALLENGES
The development of the new HMI framework as well as the
reengineering of the legacy screens and stripes is still work in
progress. There are quite a number of challenges concerning
all aspects of the reengineering, which have to be addressed -
with some of them not solved yet. Due to space reasons we
cannot describe all of them; instead we will pick some of the
more important.
A. Parser / Analyzer
1) Evolving Codebase: The stripe legacy codebase, which
is the subject of reengineering is still the basis for productive
HMI releases. This implies ongoing maintenance and even
the development of additional stripes, which of course may
contain options and variants. While having a manageable
number of newly introduced components is not a challenge
per se, the issue here is the changing coding style. With the
evolution of the Java language, certain new features are used
by some of the programmers and must thus be addressed by
the stripe analyzer. For instance, at the start of the project, none
of the stripe components used Java Generics, but their use in
stripe development is increasing, so the AST analyzer has to
be adapted accordingly. Other troubles are brought by making
heavy use of collections or other Java API classes, which
require considerable effort to the necessary interpretation of
the legacy code AST. This is an issue as described in [9],
albeit in a different context.
In this case it is important to weigh up cost and benefit of
automation support. For instance, during the candidate analysis
it was detected that the stripe codebase contains more than over
400 instances of for loops, while there was only one instance
of a do-while loop. It is out of question the analyzer ought
to support interpretation of for loops, but supporting do-while
does not seem reasonable.
2) Delimiting Static From Runtime Options: Static options
are options hardcoded in stripe code, whose evaluation is
feasible with information present at engineering time (e.g. the
550
presence of a PLC variable). In contrast, stripes may contain
code computing their content from PLC values (e.g. the num-
ber of input fields depending on a number of installed machine
components) at runtime. Static options may be externalized as
described in III-A, while the latter must be retained in the
target model. Analysis needs to be able to delimit these two
forms.
3) Correctness of the Analysis: The analyzer processing the
legacy stripe code of course has to be tested. Considering the
sheer amount of stripe components, it is neither possible nor
rational to provide a test case for each of them. We try to deal
with this issue by providing specifically coded pseudostripes,
each representing at least one concrete problem pattern (e.g.
a pseudostripe for testing a correct unrolling of for loops).
B. Transformation and Target Model
1) Eliminating Code Clones: Code clones are described
to have the potential of causing significant trouble in soft-
ware systems, especially in maintenance. The current stripe
repository and the HMI assembly process employ altered
code duplicates for variability, resembling type-2 and type-
3 code clones as described by Koschke [10]. OT-2 options
from Section II-B are a consequence of the tight coupling
of stripe code with the underlying PLC API, as there is no
flexible way to attach PLC variables to a stripe or its contained
widgets. However, this option type can be dealt with by our
reengineering approach, as all necessary information resides
in the stripe code. We have to make sure to gather the correct
PLC variable names for each stripe subclass.
In contrast, processing project clones as described in Section
II-B, OT-3 lacks the necessary information concerning the
stripe project selection criteria, which is buried in the product
configuration system. In addition, alterations in cloned projects
are not confined to the source code, but may also affect other
artifacts such as resources. This is an issue, which has yet to
be discussed.
2) Target Model Sustainability vs Information Gap: A very
interesting question is whether the target screen model is
capable of representing the information gained from the stripe
analysis. The new screen meta model is being developed
primarily following the requirements of the new-style HMI
applications, currently proposing a very simple row-wise ar-
rangement of base widgets. This poses no problem as long as
a legacy stripe conforms to this scheme. But as it turned out,
there is a significant amount of stripes not adhering to this
practice, instead introducing additional UI containers with ad-
hoc coded component arrangement (e.g. proprietary layouts)
resulting in a stripe AUI exhibiting a deep content structure not
matching the intended flat structure in the new screen model.
Thus, our first attempt to neglect layout information proved
to be too simplistic, as these cases imply deep structure and
layout information needs to be considered or even retained.
One challenge here is to distinguish between irrelevant -
which may be omitted - and significant structural and layout
information. Significant information has either to be dealt with
by extending the newly proposed screen meta model in such
a way that it allows more complex content specifications or
by representing ”problematic” parts of a legacy stripe with a
specific base widget. However, we have to observe carefully
not to allow the collection of base widgets to grow too
much creating the same problems which are manifest in the
legacy stripe repository. The work of Sinha and Karim [11]
on compacting layout trees might provide some inspiration
here. Furthermore -as mentioned in Section IV-A2 - runtime
options need to be preserved in a stripe fragment and cannot
be externalized to the assembly process OI. We have to make
sure that the target screen model is capable of expressing such
dynamic constructs.
On the other hand, we must stay put for an easily compre-
hensible screen model.
3) Target Fragmentation: Considering the proposed HMI
assembly process, it is obvious that a stripe usually will not
be present as a whole except if it has no options or variants.
It is represented by a number of stripe model fragments,
which will be combined depending on the machine type and
its configuration. This fact implies a few consequences and
reveals some pros and cons.
Expectedly, the stripe repository will become extensive, but
this would also be the case if not relying on a fragmenting ap-
proach. However, fragmented components might be considered
a bit of a downside. A good tool support will be the key here
to allow the simulation of machine configurations resulting in
the generation of the appropriate stripe definitions. As there is
no stripe code anymore, testing and verification is transferred
to the assembly and configuration process: Does a stripe
fulfill all expectations given a specific engineering context (i.e.
concrete machine setting)? Are all expected widgets present
with the correct type? Referring the correct data points (PLC
variables)? Regarding the correct look of a stripe, there will be
a generic renderer, who will interpret the given screen models
and builds the HMI at machine runtime. The same renderer
can be used to preview resulting stripes or full screens during
engineering and lay the base for (visual) testing.
We expect a substantial benefit arising with this approach
by reducing the number of conditions to check at runtime
considerably. A concrete screen model deployed on a specific
machine will induce little or no static condition checking at all,
as the model is built from the stripe fragments at engineering
time for that specific machine. As each runtime check for PLC
variable existence or even its value consumes a fair amount
of time, we hope for considerably lower HMI startup times.
Another advantage is that the approach of decoupling stripe
content and PLC variable references unlocks the potential of
getting rid of code clones, as each stripe fragment will exist
only once in the repository.
V. R ELATED W ORK
There are certainly yet a lot of efforts in reengineering or
migrating a HMI. An early work employing reverse engineer-
ing and model based reengineering is described by Merlo
et.al [12]. Staiger [13] describes the reengineering of a QT
application, amplifying the recovery of content structure as
551
well as handling implementation specific event processing.
Wolff and Forbrig [14] propose an approach allowing altering
reverse engineered HMIs within an UI editor before generating
the target. A significant work is from Ramon et.al [15]
highlighting the relevance of structural and layout information
and how to retrieve it from source code.
VI. C ONCLUSION
In this paper, we have described our ongoing work of
reengineering a considerably large source code collection of
HMI components (so-called stripes) of a machine visualization
framework. The stripe reengineering is part of a much more
extensive project aiming to provide a completely new base for
machine visualization applications. The currently hardcoded
stripes shall be replaced by model based definitions, which
are part of a machines screen and desktop model. We have
described a possible adaption of the current HMI assembly
process and have provided an overview of our reengineering
approach. Furthermore we described and discussed some of
the important challenges we have come across and are still
dealing with.
ACKNOWLEDGMENT
The research reported in this paper has been supported by the Austrian
Ministry for Transport, Innovation and Technology, the Federal Ministry for
Digital and Economic Affairs, and the Province of Upper Austria within the
frame of the COMET center SCCH.
R EFERENCES
[1] D. L. Parnas, “Software Aging,” in ICSE ’94: Proceedings of the 16th
international conference on Software engineering. Los Alamitos, CA,
USA: IEEE Computer Society Press, 1994, pp. 279–287.
[2] J. van Gurp and J. Bosch, “Design erosion: problems and causes,”
Journal of Systems and Software, vol. 61, no. 2, pp. 105 – 119, 2002.
[3] K. Bennett, “Legacy systems: coping with success,” IEEE Software,
vol. 12, no. 1, pp. 19–23, Jan 1995.
[4] I. Sommerville et al., Software engineering. Addison-wesley, 2007,
ch. 28.
[5] H. M. Sneed, “Planning the reengineering of legacy systems,” IEEE
software, vol. 12, no. 1, pp. 24–34, 1995.
[6] (2014) Javafx overview. Oracle. Last visited: 20180110. [Online].
Available: https://docs.oracle.com/javase/8/javafx/get-started-tutorial/
jfx-overview.htm
[7] U. Enste and W. Mahnke, “OPC Unified Architecture,” at-
Automatisierungstechnik, vol. 59, no. 7, pp. 397–404, 2011.
[8] B. Bos and H. W. Lie, “Cascading style sheets (CSS1) level
1 specification,” W3C, W3C Recommendation, Apr. 2008,
http://www.w3.org/TR/2008/REC-CSS1-20080411/.
[9] H. M. Sneed, “The drawbacks of model-driven software evolution,”
IEEE CSMR, vol. 7, 2007.
[10] R. Koschke, Identifying and Removing Software Clones. Berlin,
Heidelberg: Springer Berlin Heidelberg, 2008, pp. 15–36.
[11] N. Sinha and R. Karim, “Compiling Mockups to Flexible UIs,” in
Proceedings of the 2013 9th Joint Meeting on Foundations of Software
Engineering, ser. ESEC/FSE 2013. New York, NY, USA: ACM, 2013,
pp. 312–322.
[12] E. Merlo, P.-Y. Gagné, J.-F. Girard, K. Kontogiannis, L. Hendren,
P. Panangaden, and R. De Mori, “Reengineering User Interfaces,” IEEE
Software, vol. 12, no. 1, pp. 64–73, 1995.
[13] S. Staiger, “Reverse engineering of graphical user interfaces using static
analyses,” in Reverse Engineering, 2007. WCRE 2007. 14th Working
Conference on, Oct 2007, pp. 189–198.
[14] A. Wolff and P. Forbig, “Model based reengineering of user interfaces,”
in Model Driven Development of Advanced User Interfaces, 2007.
[15] Ó. S. Ramón, J. S. Cuadrado, and J. G. Molina, “Model-driven reverse
engineering of legacy graphical user interfaces,” Automated Software
Engineering, vol. 21, no. 2, pp. 147–186, 2014.