8th International Conference on Through-Life Engineering Service

Case
8th Western Reserve
International University,
Conference Cleveland,
Available OH,
on Through-Life USA.
at October
Engineering
online 27-29, 2019
Service
www.sciencedirect.com
Case Western Reserve University, Cleveland,
Available OH, USA.
online at October 27-29, 2019
www.sciencedirect.com
ScienceDirect
Available online at www.sciencedirect.com
ScienceDirect
Procedia Manufacturing 00 (2019) 000–000

ScienceDirect
Procedia Manufacturing 00 (2019) 000–000 www.elsevier.com/locate/procedia
www.elsevier.com/locate/procedia
Procedia Manufacturing 49 (2020) 206–210

8th International Conference on Through-Life Engineering Service – TESConf 2019, October 27 – 29, 2019
8th International Conference on Through-Life Engineering Service – TESConf 2019, October 27 – 29, 2019
Service Data Quality Management Framework to Enable Through-life
Service Data Quality Management Framework
Engineering Services to Enable Through-life
Engineering Services
Fernanda Cameraa, John Ahmet Erkoyuncua*, Steve Wildingb
Fernanda Cameraa, John Ahmet Erkoyuncua*, Steve Wildingb
a
Through-life Engineering Services Centre, Cranfield University, Cranfield, MK43 0AL, UK
a bBABCOCK International, Devonport Royal Dockyard, Plymouth, Devon, PL1 4SG, UK
Through-life Engineering Services Centre, Cranfield University, Cranfield, MK43 0AL, UK
b
BABCOCK International, Devonport Royal Dockyard, Plymouth, Devon, PL1 4SG, UK
* Corresponding author. Tel.: +44 (0) 1234 754717. E-mail address: j.a.erkoyuncu@cranfield.ac.uk
* Corresponding author. Tel.: +44 (0) 1234 754717. E-mail address: j.a.erkoyuncu@cranfield.ac.uk

Abstract
Abstract
With the growth of the product-service system (PSS) in recent years, how to better manage the service data to improve the informed decision-
making
With thecapability
growth ofhasthebecome an on-going
product-service aim(PSS)
system amonginthe Through-life
recent years, how Engineering Services
to better manage the(TES) firms.
service data This scenario
to improve thehas led managers,
informed decision-
more
making andcapability
more, to has
turnbecome
their attention to theaim
an on-going quality of data
among and information
the Through-life created, gathered
Engineering Servicesand used
(TES) within
firms. Thisthescenario
company. hasEncouraged by this
led managers,
background,
more and more,a service
to turndata
theirquality framework
attention has been
to the quality developed
of data aiming tocreated,
and information provide gathered
companies andwith
useda set of methods
within and tools
the company. to prioritise
Encouraged by this
relevant service
background, data and
a service dataassess its framework
quality quality levels.
hasThe
beenprocess involves
developed aimingfour
to main steps
provide that go through:
companies (1)ofMapping
with a set methodsoutandimportant data for
tools to prioritise
Through-life
relevant Support
service available
data and assess within the company
its quality and
levels. The its internal
process and four
involves external
mainflows;
steps (2)
thatApplication
go through:of(1)a Mapping
multiple criteria decision-making
out important data for
technique to prioritise
Through-life the relevant
Support available datathe
within setcompany
considering
andits
itscosts for being
internal collected
and external and (2)
flows; maintained, business
Application impact, criteria
of a multiple frequency of use and ease
decision-making
of obtainability;
technique (3) Quality
to prioritise assessment
the relevant of the
data set prioritised
considering itsdataset, based
costs for beingoncollected
a capability
and maturity model;
maintained, (4) Defining
business impact, strategies
frequencyto ofaddress
use anddata
ease
quality
of issues. Validation
obtainability; (3) Quality onassessment
an industrialofcase study demonstrates
the prioritised potential
dataset, based on a benefits of maturity
capability the process and further
model; work opportunities.
(4) Defining strategies to address data
quality issues. Validation on an industrial case study demonstrates potential benefits of the process and further work opportunities.
© 2019 The Authors. Published by Elsevier B.V.
© 2020
This is an
© 2019 The
TheopenAuthors. Published
accessPublished
Authors. article under by
the Elsevier
B.V.B.V. license(http://creativecommons.org/licenses/by-nc-nd/4.0/)
CC BY-NC-ND
by Elsevier
This
This isisan
anopen
Peer-review open
underaccess article
responsibility
access article under the BY-NC-ND
CC BY-NC-ND
of the scientific
under CC committee oflicense (http://creativecommons.org/licenses/by-nc-nd/4.0/)
the 8th International Conference on Through-Life Engineering Service –
license(http://creativecommons.org/licenses/by-nc-nd/4.0/)
Peer-review
TESConf
Peer-review 2019.under
under responsibility
responsibility ofscientific
of the the scientific committee
committee of the 8thofInternational
the 8th International
Conference onConference
Through-Life on Engineering
Through-Life Engineering
Service –
Service
TESConf–TESConf
2019. 2019.
Keywords: Data quality management; Through-life service; Service data
Keywords: Data quality management; Through-life service; Service data

1. Introduction quality management framework was developed aiming to

1. Introduction quality
provide management
TES companies framework
with a setwasof developed
methods and aiming
tools to
According to Redding and Roy [1], Through-life provide
prioritiseTES companies
relevant data for with a set of methods
decision-making, assess and improve
tools to
AccordingServices
Engineering to Redding
(TES) is:and Roy of[1],
“a result Through-life
the application of prioritise
it’s relevant
quality levels.data forstudy
This decision-making, assessthe
plans to answer andfollowing
improve
Engineering Services
explicit and tacit (TES)
‘service is: “a result
knowledge’ of the application
supported by the use of it’s quality (1)
questions: levels.
What This
is study plans to answer
the important data forthethrough-life
following
explicit and tacit
monitoring, ‘serviceprognostic
diagnostic, knowledge’ supported by
technologies andthe use of
decision questions:(2)(1)How
support? What is the
do we important
prioritise data?data for through-life
(3) What is the data
monitoring, diagnostic,
support systems whilst the prognostic
product istechnologies
in use, and and decision
maintenance support?level
quality (2) for
How thedoprioritised
we prioritise
data data? (3)What
set? (4) Whatcan is be
themade
data
support systems
and repair andwhilst the product
overhaul (MRO) is infunctions
use, and maintenance
to mitigate quality
to leveldata
improve for quality
the prioritised
issues? data set? (4) What can be made
and repair and
degradation, overhaul
restore (MRO) functionality,
‘as design’ functions to maximise
mitigate to improve data describes
This paper quality issues?
the framework, and its potential
degradation, restorethus
product availability, ‘asreducing
design’ whole
functionality, maximise
life operating cost”. This paper
application in onedescribes the leading
of the UK’s framework,
defenceandcompanies
its potential
as a
product
Althoughavailability,
there are athus reducing
great variety whole life operating
of researches on thecost”.
data application
use case. inTheonesponsor’s
of the UK’s leading defence
organisation companies asfora
is responsible
Although
management there are ait great
topic, variety
has been of researches
identified a lackonofthe data
studies use case. supporting
managing, The sponsor’s organisation
and operating is responsible
complex equipment and for
management topic, ittohas
combining processes been the
support identified a lack ofof relevant
identification studies managing, supporting
infrastructure, providing andengineering,
operating complex
technicalequipment
and trainingand
combining processes to support
data for decision-making and datathe quality
identification of relevant
assessment in the infrastructure,
services aroundproviding
the world. engineering,
In the UK,technical
it is one and
of thetraining
main
data forofdecision-making
context TES. In order toand data this
address quality assessment
matter, a serviceindata
the services around
maintenance the providers
service world. In forthethe
UK, it isNavy.
Royal one of the main
context of TES. In order to address this matter, a service data maintenance service providers for the Royal Navy.
2351-9789 © 2020 The Authors. Published by Elsevier B.V.
This is an©open
2351-9789 access
2019 The article
Authors. under by
Published theElsevier
CC BY-NC-ND
B.V. license (http://creativecommons.org/licenses/by-nc-nd/4.0/)
Peer-review
This is an open
2351-9789 under
© 2019 Theresponsibility
access article under
Authors. the of
CCthe
Published scientific
BY-NC-ND
by Elsevier committee of the 8th International Conference on Through-Life Engineering Service –
license(http://creativecommons.org/licenses/by-nc-nd/4.0/)
B.V.
TESConf
Peer-review
This 2019.
under
is an open responsibility
access of the
article under the scientific
CC BY-NC-NDcommittee of the 8th International Conference on Through-Life Engineering Service – TESConf 2019.
license(http://creativecommons.org/licenses/by-nc-nd/4.0/)
10.1016/j.promfg.2020.07.020
Peer-review under responsibility of the scientific committee of the 8th International Conference on Through-Life Engineering Service – TESConf 2019.
 Fernanda Camera et al. / Procedia Manufacturing 49 (2020) 206–210
2 Camera et al./ Procedia Manufacturing 00 (2019) 000–000
During over three hours of interviews with a Through-life
Support Manager and an Operational Support Manager at the
sponsor’s company, it was possible to observe that the
organisation is not solely responsible for entering data into
many of the systems they have to handle in a daily basis and, as
such, they have little or no real control over them. Therefore,
this scenario leads to some challenges related to data
management pointed out as follows:
• Isolation of information: there are “islands” of
data/information in different systems and relational
databases, also in different formats.
• Completeness and accuracy issues: Each system that holds
data has varying levels of completeness and accuracy; the
most accurate may only be 70% complete at any given time.
• Poor system’s usability: There are some good data
management maintenance systems, designed to capture
right data, however they are sometimes considered difficult
to navigate through and use.
• Awareness: Team in charge of data entry sometimes is not
aware on how data is going to be used later.
• Regulations: The International Traffic in Arms Regulations
(ITAR) and Export Administration Regulations (EAR) are
both derived from the United States export control laws and
they affect defence companies that deal with weapons,
munitions, and military items related. It is not possible to
store ITAR sensitive information in certain systems that do
not meet some specific data control requirements.
2. Literature review
A current major challenges for maintenance service
providers is managing the ever-increasing information data
flow and system complexity [2]. According to Murthy et al. [3]
effective maintenance management requires data gathering and
analysis for decision-making. Similarly, Mamrot et al. [4] point
out that the Maintenance, Repair and Overhaul (MRO)
activities can be planned in a more efficient way based on data
collected during the product use.
2.1. Data quality
Gitzel et al. [5] have pointed out that manufacturers and
maintenance service companies need to better understand the
failure behaviour of their machinery for different purposes and
complements that there are two main sources of inaccuracy in
modelling the failure behaviour: poor fit between the estimated
distribution and the recorded points of failure, and data quality
problems. In order to address the latter, they have proposed a
data quality dashboard for reliability data, which identifies data
quality issues and gives recommendation on how to fix the
problems.
Poor or imprecise data may influence the decision
erroneously [6]. That is why data quality problems increase the
complexity of service data management. As reported by Tsang
et al. [7], some quality issues that often arise include: the
equipment’s working age not been recorded, incompleteness or
absence of records on the type of maintenance operation
performed, transcription mistake, and truncated values
207
acquired from sensors. The same paper also notes that, in order
to prevent those problems, employees in charge of data capture,
typically tradespersons, must be trained and educated
meticulously. Additionally, automating data capture and data
logging processes can be helpful. Data quality is a
multidimensional concept and, according to Borek et al. [8],
“data and information quality is defined as the fitness for use
of data and information. It is strongly dependent on the user
and the context of usage”.
As it will be possible to notice, there is no consensus on a
specific set of dimensions [9]; that is why Borek et al. [8]
recommend that each company should select the most
appropriate data quality dimensions according to its specific
goals. DAMA UK [10] complements that the dimensions
selection should also take into consideration the business
context, organisation’s requirements, and levels of risk, among
others. The most cited dimensions of data quality are,
Completeness, Relevancy (variations: Meaningfulness,
Useful), Accessibility (variation: Availability), Free-of-Error
(variations: Accuracy, Correctness, Reliability), Consistent
Representation (variations: Unambiguous, Consistency,
Format), and Timeliness (variations: Datedness, Currency).
2.2. Data quality assessment and research gaps
A lot of studies have discussed the data quality assessment
subject. Jones-Farmer et al. [11], for instance, have proposed
the use of control charting methods as a tool for data quality
monitoring and improvement. Bergdahl et al. [12] have
developed a handbook to facilitate a systematic implementation
of data quality assessment in the European Statistical System.
It discusses important assessment methods, as quality reports
and indicators. Regarding the self-assessment, the study
proposes a general process. Burns et al. [13] have implemented
a data quality assessment program focused on the U.S.
Department of Transportation data systems. The framework is
structured based on three main components: i) quality criteria,
defining the target characteristics for the system; ii) data system
metadata, information regarding the existing data and
processes necessary to review the data systems; and iii) rating
questions, linking the metadata to the quality criteria.
Despite the existence of extensive literature regarding data
management and data quality in general, there is very little
published researches that discuss data quality issues and data
management in the Through-life Engineering Services or
industrial maintenance contexts. When reviewing the state of
the art on data quality and data management, some specific
gaps could be identified and would require further research:
• It was not possible to identify among the data quality
assessment literature, maturity level statements specific for
the most common data quality dimensions.
• Similarly, there are no studies that debate the most suitable
data quality dimensions by industry, and particularly in the
context of TES.
• A lack of studies combining processes to support the
prioritisation of relevant data for decision-making and data
quality assessment in the context of TES.
208 Fernanda Camera et al. / Procedia Manufacturing 49 (2020) 206–210
Camera et al/ Procedia Manufacturing 00 (2019) 000–000
3. TES Data Quality Management Framework
This section aims to describe the development of the
Through-life Engineering Services Data Quality (TESDQ),
detailing its four main steps, as illustrated in Fig. 1.
Fig. 1. TESDQ Framework
3.1. What is considered important data for through-life
support?
Based on the analysis of a data flow document provided by
the sponsor and some interviews, it was identified that, from
their perspective, a good Through-life support is comprised of:
• Interaction with all customers and other stakeholders
• Qualified and Experienced Engineers and Support
Team
• Post design support
• Flexible approach to contracting
• Configuration and asset management
• Engineering support
• Inventory management
• Data gathering
• Correct facilities
• Transparent and intelligent data dissemination
• Good understanding of equipment being supported
Fig. 2. Sponsor’s simplified relevant data sample to better
manage through-life support
Each of these perspectives handles with various types of
data and documentation, which are provided from different
sources and data management systems. As a use case for the
3
next step of the framework (data prioritisation), the data
gathering perspective was selected, as illustrated in Fig. 2.
3.2. How do we prioritise the data?
The Analytical Hierarchy Process (AHP) was the selected
technique for prioritizing data requirements. AHP decomposes
the decision problem into a hierarchy of sub-problems (criteria
and sub-criteria). Once the hierarchy is built, the users start to
perform the pairwise comparison, i.e. comparing the elements
to one another at a time [14], according to a specific scale. Fig.
3. provides an overview of the comparison in the TES context.
The criteria considered includes: 1) Cost of collecting &
maintaining data – Cost is usually one of the main important
criteria in a decision-making process, 2) Business impact – The
degree to which the type of data is important for the business
and decision-making, 3) Frequency of use and 4) Ease of
obtain. There is further quantitative metrics to compare the sub-
criteria.
Fig. 3. Main criteria and sub-criteria sets for prioritising data
3.3. What is the data quality level for the prioritised dataset?
This section aims to describe the development of a
performance measurement tool for data quality, including the
quality issues considered, the descriptions of the five-level
maturity scale, the tool functioning, scoring method and
visualisation of the results. The eight dimensions (Fig. 4.)
covered in the Data Quality Assessment Tool have been
identified from:
• The most cited dimensions in the reviewed literature, which
demonstrates the importance of the key issues.
• Specific challenges related to the sponsor’s environment
and the defence industry.
Fig. 4. Data quality dimensions
 Fernanda Camera et al. / Procedia Manufacturing 49 (2020) 206–210
4 Camera et al./ Procedia Manufacturing 00 (2019) 000–000
3.4. What can be done to improve data quality issues?
This section debates the final step of the framework:
strategies to improve data quality. According to Batini et al.
[15], in general, data quality methodologies address two types
of improvement strategies: data-driven and process-driven. The
first one aims to improve data quality by directly amending the
value of data, while the process-driven initiatives redesign the
process that creates or modifies data. Among the data-driven
techniques discussed by the authors, some examples are: error
identification and correction; data standardisation or
normalisation, which complements or replace non-standard
values; and data integration, which provides a unified view of
the data provided by multiple sources. The process-driven
techniques include process control, which inserts verification
and control actions in the data production process; and process
redesign, which redesign processes in order to remove the root
causes of quality problems.
Based on the insights identified in the literature, as the
examples discussed above, a list of improvement strategies was
created, including some recommendations from the author,
under four categories of strategies: data-driven, process-driven,
education & training, and management. After performing the
assessment, companies would have a “menu” of improvement
strategies to select the suitable ones through a checkbox.
4. Case study: navy based data quality management
4.1. Overview of validation
The Data Quality Assessment framework was applied in a
research environment of a defence service provider company
in the UK, focusing the “Defects and incidents” type of data.
Once the top ranked data was known, the next step was to
perform the assessment focusing on the three main systems that
provide this type of data within the organisation: UMMS – Unit
Maintenance Management System, ODMS – Operational
Defects Management System, and SharePoint, used to record
data regarding DRACAS methodology. The author moderated
the sessions, in order to ensure that the statements and
methodology were fully understood. The assessments were
performed separately by a Through-life Support Manager with
23 years of professional experience with the Royal Navy, and
by an Operational Support and BID Manager with 34 years of
engineering experience. Due to confidentiality constraints, the
assessment’s results will not be presented in detail.
4.2. Framework Validation
The dimensions’ overall result for the AS-IS and TO-BE
status is presented in a spider diagram, as illustrated in Fig. 5.
As it is possible to observe, Accuracy and Completeness
(scored 2) continue to be the two dimensions that need more
improvement in comparison to the other dimensions. Overall,
evaluating the defects and incidents data available in this
specific data management system is presented in Fig. 5. The
figure illustrates a clear focus on areas to improve by
comparing the current and future states. The framework offers
a structured way to realise where data requirements are within
209
the TES context, and how to prioritise improvement needs for
data management. Although, the data input is qualitative, it
offers a set of metrics that can be evaluated across key
stakeholders.
Fig. 5. Results visualisation for one of the sponsor's systems
During the last industrial validation session that took place
in the sponsor’s facilities, two professionals from the Through-
life Support team were asked to fill three validation
questionnaires: the first one focusing the Analytical Hierarchy
Process method, and the criteria sets defined; the second
covering in details the assessment tool, the dimensions
selected, the maturity levels statements created, as well as the
method itself; and the last one focusing the whole framework,
its effectiveness, relevancy and potential. The qualitative
validation involved scoring criteria from 1 to 5, representing
the scale (1) very poor/strongly disagree/very difficult, (2)
poor/disagree/difficult/, (3) satisfactory/neutral (4)
good/agree/easy, and (5) very good/strongly agree/very easy.
The results are detailed in Table 1. The respondent 1 (R1) is a
Through-life Support Manager with 23 years of professional
experience with the Royal Navy, and the respondent 2 (R2) is
an Operational Support and BID Manager with 34 years of
engineering experience.
The four-stages framework could be successfully applied as
a use case in a TES provider company in the defence industry,
validating the process’ concept and its effectiveness for
improving data quality issues. If there were more time, it would
have been very interesting to also validate it in different TES
companies and compare the results. The maturity model has
demonstrated to be an effective technique for the data quality
(DQ) assessment and could also be applied in different kind of
assessments within an organisation. The construction of the
maturity level statements was challenging, however, having
started its development from the “big picture”, with the macro-
level definition from Gartner’s Institute, was really helpful to
guide the development of the specific statements for each DQ
dimension.
Two new data quality dimensions were also introduced to
literature: Ease of Input and Security & Export Controls.
Analysing some challenges faced by the sponsors regarding the
usability of one data management system in particular, it was
identified the need for developing the “Ease of Input”
dimension into the assessment. Some difficulties related to data
entry have indicated that it can lead to other two issues:
completeness and accuracy, working as one of the “root-
causes” of these two recurrent problems. Thus, perform the
assessment also looking at the ease of input dimension, as well
as from the accessibility perspective, can help to identify
210 Fernanda Camera et al. / Procedia Manufacturing 49 (2020) 206–210
Camera et al/ Procedia Manufacturing 00 (2019) 000–000 5

quicker usability issues in some systems and address them every three months. The important is that, if systematically
earlier. Although Security is a dimension already cited by applied, framework can work very well as a historical
different authors, a distinctive perspective related to the ITAR performance instrument, providing KPIs and enabling the
and EAR regulations was incorporated into the dimension. The management team to monitor their progress on a regular basis.
export control regulations are particularly critical for the Some future work opportunities have been identified in
military environment and this dimension evaluates if the data order to enhance data quality and improve the developed
security procedures are also meeting the compliances. framework. Automated data collection can help to improve
completeness and accuracy issues and an augmented reality
Table 1. Qualitative validation feedback tablet could be a good solution to address this matter. The
maturity model assessment could be expanded in order to
Questions R1 R2 Total incorporate new perspectives, for instance data governance,
issues related to accountability for data quality, metadata
Ease of understanding of the management, policies and procedures, among others.
4 3 3.5
AHP method.
The Capability Maturity Acknowledgements
Model was an appropriate 4 4 4
method for the assessment.
This research is supported by the Through-life Engineering
Effectiveness of the Maturity
4 4 4 Services Centre at Cranfield University. The authors would like
Model assessment.
to acknowledge BABCOCK International for the support.
Business representativeness
of the DQ dimensions 4 4 4
References
selected.
Clarity and objectivity of the [1] Redding L and Roy R. Through-Life Engineering Services: Motivation,
4 4 4
DQ dimensions' definitions. Theory, And Practice, n.p.: Cham: Springer, 2015, p. 52-60.
The framework is not [2] Metso L, Marttonen S, Thenent N, Newnes L. Adapting the SHEL model
complex and the stages are 4 4 4 in investigating industrial maintenance, JQME, 2016; 22(1): 62-80.
simple to follow. [3] Murthy D, Karim M, Ahmadi A. Data management in maintenance
outsourcing’, RESS, 2015; 142: 100-110.
The systematically use of the [4] Mamrot M, Nicklas J, Schlüter N, Winzer P, Lindner A. Abramovici M.
framework can improve data 4 5 4.5 Concept for a Sustainable Industrial Product Service Systems based on
quality. Field Data, 13th Global Conference on Sustainable Manufacturing: GCSM
The systematically use of the 2015. Binh Du'o'ng New City, Viet Nam, 16-18 September 2015. Vol. 40:
framework can improve data 4 5 4.5 687-692.
[5] Gitzel R., Turrin S. Maczey S. A Data Quality Dashboard for Reliability
management processes.
Data, Proceedings – 17th IEEE Conference on Business Informatics, CBI
2015. Lisbon, Portugal, 13-16 July 2015. IEEE, art. no. 7264720; 1: 90-97.
5. Concluding remarks [6] Tretten P, Karim R. Enhancing the usability of maintenance data
management systems, JQME, 2014; 20(3): 290-303.
The work presents an innovative approach incorporated into [7] Tsang AHC, Yeung WK, Jardine AKS, Leung BPK. Data management for
CBM optimization, JQME, 2006; 12(1): 37-51.
the Analytical Hierarchy Process technique for data [8] Borek A, Parlikad AK, Webb J. Woodall P. Total information risk
prioritisation, and introduces two new data quality dimensions management – maximizing the value of data and information assets.
to literature: ease of input and security & export controls. The Morgan Kaufmann. 2014, p. 32-48.
industrial and academic validations have shown that: [9] Wand Y, Wang R. Anchoring data quality dimensions in ontological
• The framework is easy to understand and its stages are foundations, Communications of the ACM. 1996; 39 (11): 86-95.
[10] DAMA UK. The six primary dimensions for data quality assessment.
simple to follow. 2013. Available at: https://www.em360tech.com/wp-
• If applied systematically, the framework can improve content/files_mf/1407250286
data quality levels and data management practices. DAMAUKDQDimensionsWhitePaperR37.pdf (Accessed: 17 June 2019).
• In addition to the defence sector, the process can be [11] Jones-Farmer LA, Ezell JD, Hazen BT. Applying control chart methods
adopted by different industries. to enhance data quality. Technometrics. 2014; 56(1): 29-41.
[12] Bergdahl M, Ehling M, Elvers E, Földesi E, Körner T, Kron A, Lohauß P,
• The framework has a relevant contribution to the data Mag K, Morais V, Nimmergut A, Sæbø HV. 2007. Handbook on data
quality research topic. quality assessment methods and tools. Eurostat.
The authors believes that TES companies should perform [13] Burns EM, MacDonald PO, Champaneri A. Data quality assessment
the data quality assessment in a strategic/corporate level at least methodology: a framework. Proceedings of the Survey Research Methods
once a year, since this is a reasonable length of time to Section, 2002. American Statistical Association. Available at:
http://www.amstat.org/sections/srms/Proceedings/y2002/Files/JSM2002-
implement the improvement strategies and measure its 000347.pdf (Accessed: 12 June 2016).
outcomes. The industrial sponsors have also acknowledged this [14] Majumder M. Impact of urbanization on water shortage in face of climatic
periodicity. From the operational perspective, maybe the aberrations. Springer. 2015 ISBN: 978-981-4560-72-6.
frequency could be reduced to a weekly basis, or every 15 days, [15] Batini C, Cappiello C, Francalanci C, Maurino A. Methodologies for data
for instance to measure the data quality status during a bidding quality assessment and improvement, ACM Computing Surveys.
2009;41(3): 1-52.
process. From the tactical perspective, for example considering
a department’s view, the assessment could maybe be performed