Water Resources Management (2019) 33:4939–4953

https://doi.org/10.1007/s11269-019-02429-y

Assessment of a Web-Based Water Information System

Performance in the Context
of Groundwater Governance

Nuno Barreiras 1 1
& Francisco Nunes Correia & Rafaela Saldanha Matos
2

Received: 8 March 2019 / Accepted: 17 November 2019/

Published online: 30 November 2019
# Springer Nature B.V. 2019

Abstract
Portugal recently faced a severe financial crisis that led to the suspension of funding for
water monitoring programmes of the Portuguese National Water Resources Information
System (SNIRH). This caused a severe impact as SNIRH and shared databases are
considered key mechanisms for sharing water basin, country and international policy
and data. The available information and data quickly become outdated as well as
incomplete and compromised, due to these financial cuts with evident impact in the
operational and analytical mechanisms. Therefore, an assessment of these impacts and the
identification of gaps and areas needing development and improvement, according to the
fundamentals of OECD principles on water governance was strictly necessary, in order to
comply with the substantial role of a national water information system (WIS). This study
provided the first performance assessment of a WIS based on the refinement of the
updated DeLone and McLean information system success, in the context of groundwater
governance. The model consists of five analytical dimensions and the quantifica-
tion of their relationships was made through the application of structural equa-
tions modelling and multiple correspondence analysis. The data for the model was
collected through a questionnaire held at a national level. The proposed model
proved to produce good reliability estimates and the findings of this research
provide important input and implications for the water information systems, and
for groundwater management and governance in particular. The paper ends with a
discussion on the limitations and areas of improvement that could be addressed in
future developments.

Keywords Groundwater management . Information system . Performance . Governance . Success

model

* Nuno Barreiras
nuno.barreiras@tecnico.ulisboa.pt; https://orcid.org/0000–0001–9025–0625

Extended author information available on the last page of the article

4940 Barreiras N. et al.

1 Introduction

Countries with longer traditions of groundwater management and political vision and well-
resourced agencies in the public sector are better placed to manage groundwater resources.
Nevertheless, the phenomenon of lack of monitoring data is in many countries a major
difficulty to an effective groundwater management and governance. In Portugal, the
National Water Resources Information System (SNIRH) was developed by the former Water
Institute (INAG) and presented publicly in 1996. SNIRH has become essential to the various
stakeholders in the sector for providing reliable, continuous, low-cost, readily available and
easily accessible information. This have led to major progress and improvements in integrated
water management.
The creation of INAG was also critical for the improvement of SNIRH due to goals defined
in its Organic Law. Despite the fact that the EU Water Framework Directive (WFD) (EC 2000)
was approved in 2000, the particularities and specificities of monitoring aspects had been
discussed for a long time before, therefore the restructuring of the monitoring networks was
considered to be a successful case. The WFD had defined some objectives to be implemented
by 2006, one of which specifically on the intention to implement new monitoring and data
sharing programmes. Consequently, INAG actively participated with working groups aimed at
ensuring the accomplishment of those requisites throughout Europe. Along the investment and
implementation of a wider structured monitoring network (circa 1999), other areas once
independent had been integrated. At this point, the river basin plans and the national water
plan had been approved and several action plans emerged, both for development and im-
provement of each management area regulated by community directives. These areas included
the inland surface waters, transitional waters, coastal waters and groundwater. Data series and
information became vital for the improvement of monitoring and for the development of these
action plans. For that reason, became an open database of hydro-meteorological and water
quality data, both for surface and groundwater, under the Ministry of Environment, presently
accessible through the web portal (https://snirh.apambiente.pt). After the merging of INAG
with the Portuguese Environment Agency (APA) in 2012, SNIRH is managed by this later
agency under the Ministry of Environment.
Portugal faced a severe financial crisis that led the country to a three-year economic
adjustment programme in order to cope with the 2010–14 Portuguese financial crisis with a
major impact in the funding to water monitoring programmes of SNIRH, namely the main-
tenance of monitoring stations. This major event, naturally, caused the degradation of data
without apparent reaction from the governing institution. The financial cuts and the reduction
of specialized and technical staff in an organization like APA were revealed the obvious
reasons for the subsequent service flaws in some functions of SNIRH. The alleged limitations
of APA, strongly induced by political decisions and adaptations to the economic conditions,
are known today to be the cause of: 1) the severe deterioration of data and monitoring
networks and; 2) the disappearing of the inherited technical know-how by dismissing technical
human resources, which were the fruit of several years of capacitation, development and
accumulated experience. Taking into account SNIRH as a web-based water information
system (WIS), with data and information on groundwater, and the aforementioned background
for the present research, it is timely and opportune to look at the OECD’s principles on water
governance (OECD 2018) which defends that it is necessary to produce, update, and share
timely, consistent, comparable and policy-relevant water and water-related data and informa-
tion, and use it to guide, assess and improve water policy. Therefore, an assessment of SNIRH
Assessment of a Web-Based Water Information System Performance in the... 4941

as well as the identification of gaps and areas needing development and improvement was
strictly necessary, in order to comply with the substantial role of a national WIS. This study
aims at providing a novel performance assessment of a WIS in the context of groundwater
governance, for the collective groundwater management in a sustainable and efficient way.

1.1 Methodologies

The methodology adopted to assess SNIRH effectiveness and performance in the context of
groundwater governance is based in DeLone and McLean information system success
(DeLone and McLean 2003). The application of this model required 3 stages of development:
1) the refinement of DeLone and McLean information system success model for a web-based
WIS performance, 2) the development of a research design applicable to the model, 3) the
analysis and quantification of relationships among the model variables through the application
of structural equation modelling (SEM) and multiple correspondence analysis (MCA).

1.2 The Refinement of the Updated DeLone and McLean Information System Success
Model

There is a quite vast research on information systems (IS) success which includes variables as user
satisfaction, task-technology fit, user involvement and participation (Doll and Torkzadeh 1988;
Galletta and Lederer 1989; Igbaria and Nachman 1990). In order to integrate concepts and
findings, as well as to organize and introduce taxonomy based on the extensive literature on the
topic, DeLone and McLean (1992) presented their first version of the IS Success Model, that
includes six interrelated and interdependent categories: system quality, information quality, use,
user satisfaction, individual impact, and organizational impact. This model quickly become one of
the dominant evaluation frameworks in IS research, in part due to its comprehensibility, simplicity
(Urbach et al. 2009) and adaptability. Following the string of events and based on the evaluation
of the multiple contributions for its improvement, DeLone and McLean proposed an updated IS
success model in 2003 (DeLone and McLean 2003) (Fig. 1). The updated version of the model
has received great appreciation in the IS community, and most of its propositions explaining the
success of an IS are actually supported (Petter et al. 2008; Urbach and Müller 2012).
Looking at its dimensions and the relations between them, the model can be interpreted as a
system that is evaluated in terms of information, system, and service quality. These character-
istics are going to affect subsequent use or intention to use and user satisfaction. Certain

Fig. 1 Updated DeLone and McLean IS Success Model (DeLone and McLean 2003)
4942 Barreiras N. et al.

benefits will be achieved by using the system. The net benefits will influence in a positive or
negative way the user satisfaction and the further use of the IS.
There is an evident scarcity of research on the IS success in areas such as web-based WIS,
or even in the broader topic of water governance. Therefore, the proposed model aims at the
application and refinement of the updated Delone and Mclean IS Success model (2003) to
assess the performance of a web-based WIS in the context of groundwater governance. In
order to establish the best adaptation of the model to the context, the following considerations
were made:
1. The service quality dimension represents the customer support quality from the IS and IT
department personnel, such as, training, hotline, or helpdesk. For the present context, and in
agreement with Seddon (1997), this dimension is not considered to be an important quality
measure of a system.
2. The dimension use/intention to use represents the manner in which an information
system is utilized by its users. Thus, due to the characteristics of using a web-based WIS, it
is proposed to use the dimension system use (utility and suitability) as a way to define the
measurement in terms of reporting/appropriateness, accessibility, public participation and
society, also supported by Eldrandaly et al. (2015) and Nedovic-Budic (1999).
3. Net benefit refers to the user-perceived net benefit assessment toward using a specific
context (Wang and Liao 2008), in this case web-based WIS in groundwater governance. Users
(as taxpayers) may feel that they are not getting proper benefit for their money, or they would
like to see a better benefit in terms of cost/time savings, or through decision making processes.
Therefore, perceived net benefit seems to be an important success measure, with focus on the
evaluation criteria at the level of understanding, efficiency and effectiveness (Nedovic-Budic
1999; Wang and Liao 2008; Eldrandaly et al. 2015).
4. The perceived net benefits are dependent on the user satisfaction and system use only, so
the feedback links from perceived net benefits to both system use and user satisfaction were
excluded. To avoid complexity and because the user satisfaction is dependent on the system
use it is assumed that there is no feedback link between user satisfaction to system use.
Therefore, the dimensions considered for the proposed model are system quality, informa-
tion quality, system use (utility and suitability), user satisfaction and perceived net benefits, as
shown in Fig. 2. The arrows between success dimensions represent the temporal (process) and
causal relationships. To study the interrelationships among those dimensions and how they

Fig. 2 Respecified DeLone and McLean model to assess the performance of a web-based WIS, in the context of
Groundwater Governance
Assessment of a Web-Based Water Information System Performance in the... 4943

control each other, the proposed model suggests the following seven hypothesis (Fig. 2), in the
groundwater governance context:
H1: Information quality will positively affect system use;H2: Information quality will
positively affect user satisfaction;H3: System quality will positively affect system use;H4:
System quality will positively affect user satisfaction;H5: System use will positively affect user
satisfaction;H6: System use will positively affect perceived net benefits;H7: User satisfaction
will positively affect perceived net benefits.

1.3 Model Constructs and Hypothesis

Information quality (IQ): this dimension measures the quality of information provided to the
users, related to groundwater, in the form of raw and processed monitoring data, maps, tables,
charts, reports and other types of provided information. It is measured by completeness,
relevance, timeliness, reliability and scale.
System quality (SQ): measures the success of the technical aspects of the web-based WIS.
The system quality was measured by response time, operability and ease of use.
System use (utility and suitability) (U): in this context, the utility and the suitability to
provide adequate and consistent data, for every kind of users, from the technical and non-
technical point of view, are relevant to measure system use, as well as fostering public
participation and values of transparency, trust and comparability of data and information.
These measures are within the evaluation criteria of reporting/appropriateness, accessibility,
public participation and society.
User satisfaction (GS): As some researchers simplified the various quality dimensions of
either instruments into a single item to measure global satisfaction with an information system
(Rai et al. 2002) parsimony has been applied. According to some authors (Bergkvist and
Rossiter 2007; Diamantopoulos et al. 2012; Bergkvist 2015) a 1-measure questionnaire for one
dimension can be as effective as a multi-measure. For these reasons, the user satisfaction was
measured by one single measure: the overall satisfaction item.
Perceived net benefits (PNB): The most common measure at the individual level is the perceived
usefulness. Segars and Grover (1993) used job performance and effectiveness to measure a separate
construct called effectiveness. The three-factor construct composed by perceived ease of use,
perceived usefulness and effectiveness, results in a relatively strong fit (Petter et al. 2008).
Torkzadeh and Doll (1999) have created an instrument to measure different aspects – task
productivity, task innovation, customer satisfaction, and management control. In this case, under-
standing (that leads to task productivity), efficiency and effectiveness are the adopted criteria.
The measurement items selected for the constructs were mainly adapted from previous
studies to ensure content validity. Each item was adapted to specifically reference a web-based
WIS in the groundwater governance context (Table 1).

1.4 Research Design

In order to collect data to test and validate the proposed model a questionnaire was prepared
and sent to SNIRH users. The questionnaire included 1) characteristics of respondents, 2) four
categories of questions that correspond directly to the success dimensions of the model
constructs, and 3) a free text comment field. Likert scales (1~5), ranging from “strongly
disagree” to “strongly agree” were used for all questions plus an “I don’t know” field (null
answer) representing the inability of the respondent to answer the questions. This Null point
 4944

Table 1 Measures of the model constructs

Success Dimension Evaluation Criteria Measurement Items Code References

System Quality Response Time The data is provided quickly SQ1

Operability The methods used to provide data are adequate SQ2
Ease of use The WIS is user-friendly and easy to learn SQ3
Information Quality Completeness The data provided is sufficient and consistent IQ1 Doll and Torkzadeh 1988; Clapp et al. 1989;
Relevance The data meet your needs IQ2 Brown 1996; Nedovic-Budic 1999; Obermeyer
Timeliness The WIS provides updated data IQ3 and Pinto 2008; Wang and Liao 2008;
Reliability The WIS provides reliable data IQ4 Eldrandaly et al. 2015
Scale The data is adjusted and adequate to the aquifer scale IQ5
System Use Reporting/ Appropriateness The type and frequency of reporting is adequate U1 Franz and Robey 1986; Worrall 1994;
(Utility and Accessibility The data is accessible to different kind and levels of U2 Nedovic-Budic 1999; Eldrandaly et al. 2015
Suitability) users and stakeholders
Public Participation/ Society The WIS is used to foster public participation U3
Society The WIS improves transparency, trust and comparability U4
of data and information
User Satisfaction Overall Satisfaction Global satisfaction GS1
Seddon and Yip 1992; Rai et al. 2002; Almutairi
and Subramanian 2005; Jarupathirun and
Zahedi 2007; Gable et al. 2008; Eldrandaly et al. 2015
Perceived Net Benefits Understanding The WIS enhances a better understanding of the problems NB1 Nedovic-Budic 1999; Wang and Liao 2008;
Efficiency The WIS provides improvement and execution of work NB2 Eldrandaly et al. 2015
Effectiveness The WIS contributes to reducing risk in the decision NB3
making process
Barreiras N. et al.
Assessment of a Web-Based Water Information System Performance in the... 4945

Fig. 3 Characteristics of the questionnaire respondents

filters the respondents familiarized with the studied object from the ones that are not familiar-
ized with the object. Before the questionnaire become published some interviews with
hydrogeology specialists and experienced users/developers of WIS were conducted in order
to follow up a testing of the measures to validate the measurement items. The questionnaire
was published online and held nationally throughout the main Portuguese stakeholders on the
subject and clearly related to groundwater. At total, 122 answers were received. Respondents
with more than 30% of null answers were rejected from the sample. The remaining 111
answers were transformed into a 5-point scale, converting each null answer into a mid-term
answer of the 5-point Likert scale (“Neither agree nor disagree”).
These statistical data indicates that there is a good sample distribution in terms of gender,
age, users’ activity and experience with SNIRH (Fig. 3).

2 Results

The collected data is shown in Fig. 4. The data shows that the representativeness of option 1
(Strongly disagree/Totally dissatisfied) is between 1% and 17%. IQ1 and IQ3 are the measured
items with the biggest expression with disagreeing answers, regarding completeness of data
and timeliness, respectively. The same two items also resulted in a higher number of answers
4946 Barreiras N. et al.

Fig. 4 Answers to the questionnaire by measured item

with the option 2 (Partially disagree/Dissatisfied), about 34% of all the answers. According to
this sample, the timeframe and socio-economic background the questionnaire was answered,
the information quality dimension of SNIRH faces serious challenges in terms of completeness
and timeliness of the data/information available.
The items SQ1 (response time), SQ2 (operability), SQ3 (ease of use), IQ2 (relevance), IQ5
(scale), U1 (reporting/appropriateness), U2 (accessibility) also show that the sum of the
options 1 and option 2 are above 20 answers out of 111. For the measures IQ1, IQ3 e U1,
the sum of disagreeing answers can represent up to 31%.
There is a large percentage of answers with option 3 (Neither agree nor disagree) for the
items IQ5 and U1, representing 40% and 31% respectively. For IQ5, this means that either
users are not totally aware of what is the scale (of the aquifer) or they don’t have a solid
opinion on this criteria. The same happens to U1, where the evaluation criteria were the type of
reporting and its appropriateness. Therefore, can be said that the respondent’s answers about
the scale and reporting/appropriateness of the data is generally smooth and probably due to a
phenomenon of unfamiliarity with the usual data requirements for the several applications.
Regarding the options 4 (Partially agree/Satisfied) and 5 (Strongly agree/Totally satisfied)
there are some measured items with quite favourable answers. This is the case of SQ1, SQ2,
SQ3, IQ2, IQ4 (reliability), U2, U3 (public participation/society), U4 (society), NB1 (under-
standing), NB2 (efficiency), NB3 (effectiveness) and GS (overall satisfaction). The highlights
goes for IQ4 which measures the reliability of the data and information and NB1 that evaluates
the perceived net benefit to provide a better understanding of the problems to be solved. These
two options alone (4 and 5) gathered 78% of the answers of the item IQ4 and 82% for NB1.
Regarding GS, the results show that despite the good levels of agreement in most items the
general satisfaction is attenuated by some item yet to be identified, which has probably a
greater weight in the respondents’ satisfaction. That fact is seen in the lowest percentage of the
option 5 (Strongly agree/Totally satisfied) of the GS item, holding only 5% of the answers. The
item with the highest number of answers with option 5 is NB1 (44%). From a general
perspective, and according to the data it could be said that the success dimension perceived
net benefits gathers the most favourable answers, while information quality gathers the most
unfavourable answers.
Assessment of a Web-Based Water Information System Performance in the... 4947

The questionnaire ends with a free-text comment field in which the user can freely
express himself about any matter concerning SNIRH. This way the respondent was able
to highlight any other aspect or characteristic that was not contemplated in the question-
naire. This allowed to cataloguing all the themes raised by the respondents and to group
them into categories: financing, interface, data/information, transparency and society,
improvement. Some respondents appointed the theme ‘lack of financing’ one of the
major problems of SNIRH that, consequently affected several aspects related to one the
main function of this information system: the generation, processing and sharing of data
and information. This category alone carries about 44% of the total comments. The
category Interface also showed that the respondents are sensitive to technology, acces-
sibility and download methods of data and information. The interface is outdated and
needs intervention for the modernization of the web portal, namely in the section where
the data is searchable and exportable.

3 Analysis

3.1 Reliability Analysis

In order to obtain quantifiable results from the empirical respecified DeLone and
McLean model, a structural equation model (SEM) was applied. A major component
of the confirmatory stage of the proposed SEM is the reliability analysis of the
observed variables, which for this case are the measurement items. The reliability
was measured using Cronbach’s alpha. A Cronbach’s alpha coefficient exceeding 0.7
represents a the recommendable threshold for the internal consistency (Hair et al.
2006; George and Mallery 2009). The coefficient for the constructs of the model are
displayed on Table 2. The Cronbach’s alpha for information quality, system quality
and perceived bet benefits are ranging from 0.773 to 0.865 except for system use
(0.640). Nevertheless, a summary of a reliability analysis for all the items results in a
coefficient of 0.881.

Table 2 Cronbach’s Alpha reliability analysis

Construct (Cronbach’s Alpha) Items Item-total correlations

Information Quality (0.785) IQ1 0.665

IQ2 0.672
IQ3 0.567
IQ4 0.437
IQ5 0.476
System Quality (0.773) SQ1 0.517
SQ2 0.660
SQ3 0.666
System Use (0.640) U1 0.400
U2 0.527
U3 0.276
U4 0.504
Perceived Net Benefits (0.865) NB1 0.701
NB2 0.817
NB3 0.719
4948 Barreiras N. et al.

Table 3 Model-fit indexes

Fit Indices Acceptable Threshold Levels Structural

Model

x2 Low X2 relative to degrees of freedom with 1.066

an insignificant p value (p > 0.05)
Degrees of freedom (df) 2
x2/df Values less than 2 (Ullman 2001) 0.533
Goodness of Fit Index (GFI) Values greater than 0.95 (Miles and Shevlin 1998) 0.996
Adjusted Goodness of Fit Index (AGFI) Values greater than 0.95 (Hooper et al. 2008) 0.971
Normed Fit Index (NFI) Values greater than 0.95 (Hu and Bentler 1999) 0.989
Comparative Fit Index (CFI) Values greater than 0.95 (Hu and Bentler 1999) 0.997
Root Mean Square Residual (RMSR) Values less than 0.05 (Byrne 2010) 0.011
Root Mean Square Error of Values less than 0.07 (Steiger 2007) 0.002
Approximation (RMSEA)

3.2 Model Analysis

To resume the model analysis and turn empirical findings and hypothesis into quantifiable
measures, the SEM technique was applied using LISREL 9.30 (Jöreskog and Sörbom 2017).
The model-fit measures used to assess the model’s overall are presented in Table 3. This group
includes the most common indexes for the assessment of the model’s overall goodness of fit
(Hu and Bentler 1999; Schreiber et al. 2006). All the model-fit indexes exceeded their
respective common acceptance levels suggested by previous research (Miles and Shevlin
1998; Hu and Bentler 1999; Steiger 2007), thus demonstrating that the measurement model
exhibited a good fit with the data collected.
This way, the properties of the causal path, including the path coefficient (γ), p-values and
variance (R2) explained for each equation in the hypothesized model, are presented in Fig. 5. As
expected, information quality (IQ) had a significant influence in both system use (U) and global
satisfaction (GS). Thus, H1a and H1b were supported (with path coefficients of γ = 0.26 and
γ = 0.29, respectively). System quality (SQ) had a significant impact in both global satisfaction
(GS) and system use (U). H2a and H2b were supported (with path coefficients of γ = 0.14 and

Fig. 5 Hypotheses testing results

Assessment of a Web-Based Water Information System Performance in the... 4949

Table 4 The direct, indirect and total effect of dominants on Perceived Net Benefits

Items Direct effect Indirect effect Total effect

U GS PNB U GS PNB U GS PNB

SQ 0.14 0.30 0.03 0.12 0.14 0.34 0.12

IQ 0.26 0.29 0.07 0.14 0.26 0.35 0.14
U 0.25 0.14 0.25 0.21
GS 0.31 0.31

γ = 0.30, respectively). Consequently, IQ exhibited a stronger effect than SQ in influencing U

and GS. In addition, U had a significant influence in both GS and in PNB. H3a and H3b were
supported (with path coefficients of γ = 0.25 and γ = 0.14). Finally, GS appeared to be a
significant determinant of PNB. H4 was also supported (path coefficient γ = 0.31).
Altogether, this model accounted for 74% of the variance in PNB with GS exerting the stronger
direct effect. Thus, 33% of the variance in GS was explained by IQ, SQ, and U while 12% of the
variance in U was explained by IQ and SQ. The direct and total effect of GS on PNB was 0.31.
However, the direct and total effects of U on PNB were 0.14 and 0.21, respectively. Among the
two quality-related constructs, IQ had the strongest total effect in PNB.
The direct, indirect, and total effects of information quality, system quality, system use, and
global satisfaction in perceived net benefit were summarized in Table 4. At this point, could be
concluded that this model is a reasonably good means of data description.

3.3 Multiple Correspondence Analysis

As discussed previously in this chapter, one of the major concerns regarding the application of
this model is the analysis and interpretation of the relationships between the various model
dimensions or even measures. Therefore, multiple correspondence analysis (MCA) was
applied to the questionnaire results in order to map the relationships between the measurement
items/dimensions and the characteristics of the respondent (Fig. 3). The Likert scale used for
the questionnaire answers was rearranged into 3 levels: 1 corresponding to the first two Likert
levels (1–2); 2 corresponding to the middle value as a neutral answer; 3 that assigns to
agreement answers of the Likert scale (4–5).
The dimensions (categories) of the measures (qualitative variables) were coded and further
processed by MCA. The corresponding scree plot is based on the eleven non null eigenvalues
and on the corresponding percentage of the adjusted inertia of 74.23% for F2 (Table 5).

Table 5 Eigenvalues and percentages of inertia

F1 F2 F3 F4 F5 F6 F7 F8 F9 F10 F11

Eigenvalue 0.339 0.208 0.146 0.130 0.098 0.093 0.086 0.081 0.070 0.066 0.064
Inertia (%) 16.962 10.407 7.304 6.487 4.891 4.658 4.322 4.048 3.499 3.294 3.203
Cumulative % 16.962 27.370 34.674 41.161 46.052 50.709 55.031 59.078 62.578 65.872 69.075
Adjusted 0.087 0.024 0.008 0.005 0.001 0.001 0.001 0.000 0.000 0.000 0.000
Inertia
Adjusted 58.128 16.101 5.302 3.432 0.946 0.713 0.435 0.259 0.043 0.009 0.002
Inertia (%)
Cumulative % 58.128 74.229 79.531 82.963 83.909 84.623 85.057 85.316 85.358 85.367 85.369
4950 Barreiras N. et al.

Symmetric Variable Plot

(axes F1 and F2: 74.23%)
1.0

IQ4-1

IQ5-1 NB1-1 NB2-1

E-3 U4-1
0.5
U1-3 SQ2-3 SQ3-3 IQ3-1 U1-1 IQ2-1
IQ5-3 U2-3 H-3 U3-1 GS1-1 NB3-1
GS1-3 NB2-3 U3-3
IQ1-3 NB3-3 SQ1-1
A-4
IQ2-3 U4-3 SQ1-3 E-4 IQ1-1 U2-1
0.0 A-2 IQ4-3 E-2
NB1-3 A-1
IQ3-3
A-3 H-2 SQ2-1
F2 (16.10%)

IQ4-2
H-1
-0.5 U3-2 IQ5-2 U2-2 SQ3-1 Characteriscs of respondents:
- Acvity
U1-2 NB3-2 U4-2
E-1 NB1-2 - Educaon
SQ1-2
-1.0 - Experience
SQ2-2
IQ3-2 IQ1-2
NB2-2 Answer:
GS1-2
- Agreeing
-1.5 - Neutral
- Disagreeing
SQ3-2 IQ2-2
-2.0
-1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5
F1 (58.13%)

Fig. 6 Symmetric Variable Plot for axes F1 and F2: 74.23%

The Fig. 6 shows the correspondence map where both the characteristics and the answers to
the questionnaire are displayed on the first two axes. The question codes are the same listed in
Table 1 and the numbers next to it refer to the rearranged Likert scale (1–3). According to the
plot, the group of disagreeing answers is explained by F1 (58.13%) on the positive quadrants,
but apparently does not have a clear relation with any user characteristic. The neutral answers
seems to be related to the first level of each user characteristic, namely students with low
experience with SNIRH and lower academic qualifications. In regard of the education level,
could be assumed that E-1 (lowest level of education) is associated to neutral answers,
probably derived from the inability of the respondents to answer what was asked on each
question. The agreeing answers are clustered on the second quadrant of the graph and it seems
that they have some relation to the activities 2 and 3 (A-2 and A-3), private and public sectors
respectively. There is no clear relation between the level of experience of the respondent and
these answers.
The correlation matrix (Pearson, r) shows the best correlated answers according by
evaluation criteria of the questionnaire measures. It seems that the pairs of best correlated
measures (correlation above 0.5) are mostly within the same dimension (PNB, IQ and SQ).
This might indicate that the respondents tend to assume the same values for different measures
within the same dimension or, on the other hand, they might understand that these measures
complement the others of the same dimension. Analysing, for example, the dimension PNB,
which shows good correlation between the measures NB1-NB2 (r = 0.699) and NB1-NB3 (r =
0.584), can indicate a complementary and consequential relation: 1) a better understanding of
the problems influences the improvement and execution of work, and consequently, 2)
contributes to reduced risk in the decision making process. The same goes for the related
criteria completeness (IQ1) and relevance (IQ2) (r = 0.681) of the information quality provided
Assessment of a Web-Based Water Information System Performance in the... 4951

by SNIRH, which means that the provided data being sufficient and consistent will meet the
user needs. For the case of operability (SQ2) and ease of use (SQ3) (r = 0.622) regarding the
system quality can be stated that the methods implemented by SNIRH to providing
data are adequate and aids the user to feel comfortable with the system. The aspects
that do most influence users regarding the overall satisfaction (GS1) seems to be the
ease of use (SQ3) (r = 0.444), the completeness (SQ1) (r = 0.438) and the scale (IQ5)
(r = 0.420). It also seems to be quite well related to the understanding (NB1) (r =
0.410) and the efficiency (NB2) (r = 0.490).

4 Discussion and Conclusions

This study presented a novel and an effective methodology to assess the effectiveness and
performance of SNIRH, a web-based WIS in the context of groundwater governance. The
method is based on the refinement of the updated DeLone and McLean IS success model
(2003) and in previous studies. The results indicated that Information quality, system quality,
system use, global satisfaction and perceived net benefits were valid dimensions to measure
WIS performance, along the 16 items presented. The proposed model proved to produce good
reliability estimates thus demonstrated that the measurement model exhibited a good fit with
the data collected. The empirical data showed that information quality had a significant
influence in both system use and global satisfaction. System quality had a significant impact
in both global satisfaction and system use. Information quality exhibited a stronger effect than
system quality in influencing system use and global satisfaction. System use had a significant
influence in both global satisfaction and in perceived net benefit. And finally, global satisfac-
tion appeared to be a significant dimension for perceived net benefit. From these results it is
observable that users tend to agree in the perceived net benefits of SNIRH, mainly on
efficiency (SNIRH provides improvement and execution of work) and effectiveness (contrib-
utes to reducing risk in the decision making process). There is a general manifestation of
satisfaction with SNIRH except for information quality. As one the most influencing dimen-
sions, it shows that some areas face serious challenges, namely in the areas of completeness
and timeliness. This could be explained by 1) the timing of the survey, shortly after the
suspension of the monitoring operations, and 2) the discontinuing of long data series or/and the
long observation lags that were practiced. At the meanwhile, in Portugal, the revision of the
river basin management plans (RBMP), from 2009, was in progress and the degradation of
data did not give as much input as it would be desirable. The re-evaluation of the quality and
quantity status of groundwater bodies was therefore affected. Nevertheless, it is observed that
users tend to respond positively on SNIRH reliability and transparency. In fact, perhaps due to
the timing and aftermath of the financial crisis, the survey is considered to be quite successful.
It gathered over 120 answers in Portugal, for a specific technical field as groundwater. This
could be interpreted as the sense of responsibility of the users in contributing to this study and
assessment. Regarding the activity sector it is observed that the education/research sector is the
most participative in contrast with the private sector. This could be explained by the dissem-
ination means or by a reason that was not identified. This point requires further research that
could study more effectively how, when and what for the private sector uses data from SNIRH.
As it is known the private sector, mostly the times, generate data at works of smaller scale that
requires local data. If possible, could be very useful to integrate data from the private sector in
SNIRH, as a complement to the existing data at a larger scale.
4952 Barreiras N. et al.

This assessment is considered to be an important first step to the improvement and

achievement of a model for measuring the performance of a web-based WIS, in the context
of groundwater governance. For this reason cannot be assumed as definitive model as it is only
based in a single study. Requires to be revisited, improved and extended to future events, and
validation of measurement is necessary for different properties and over a variety of contexts
and situations.

Acknowledgements This research was funded by Fundação para a Ciência e Tecnologia in the scope of the
PhD programme H2Doc with the Ref. number PD/BD/105969/2014.

Compliance with Ethical Standards

Conflict of Interest The authors declare no conflict of interest.

References

Almutairi H, Subramanian GH (2005) An empirical application of the DeLone and McLean model in the Kuwaiti
private sector. J Comput Inf Syst 45:113–122. https://doi.org/10.1080/08874417.2005.11645849
Bergkvist L (2015) Appropriate use of single-item measures is here to stay. Mark Lett 26:245–255. https://doi.
org/10.1007/s11002-014-9325-y
Bergkvist L, Rossiter JR (2007) The predictive validity of multiple-item versus single-item measures of the same
constructs. J Mark Res 44:175–184. https://doi.org/10.1509/jmkr.44.2.175
Brown MM (1996) An empirical assessment of the hurdles to geographic information system success in local
government. State Local Gov Rev 28:193–204
Byrne BM (2010) Structural equation modeling with AMOS. Basic concepts, applications, and programming.
2nd ed. Routledge, Taylor & Francis Group, New York
Clapp JL, McLaughlin JD, Sullivan JG, Vonderohe AP (1989) Toward a method for the evaluation of
multipurpose land information systems. J Urban Reg Inf Syst Assoc 1:39–45
DeLone WH, McLean ER (1992) Information systems success: the quest for the dependent variable. Inf Syst Res
3:60–95. https://doi.org/10.1287/isre.3.1.60
DeLone WH, McLean ER (2003) The DeLone and McLean model of information systems success: a ten-year
update. J Manag Inf Syst 19:9–30. https://doi.org/10.1080/07421222.2003.11045748
Diamantopoulos A, Sarstedt M, Fuchs C et al (2012) Guidelines for choosing between multi-item and single-item
scales for construct measurement: a predictive validity perspective. J Acad Mark Sci 40:434–449. https://doi.
org/10.1007/s11747-011-0300-3
Doll WJ, Torkzadeh G (1988) The measurement of end-user computing satisfaction. MIS Q 12:259–274.
https://doi.org/10.2307/248851
EC (2000) European Comission, Directive 2000/60/EC of the European Parliament and of the Council of
October 23 2000 establishing a framework for community action in the field of water policy. 1–73
Eldrandaly KA, Naguib SM, Hassan MM (2015) A model for measuring geographic information systems
success. J Geogr Inf Syst 7:328–347. https://doi.org/10.4236/jgis.2015.74026
Franz CR, Robey D (1986) Organizational context, user involvement, and the usefulness of information systems.
Decis Sci 17:329–356. https://doi.org/10.1111/j.1540-5915.1986.tb00230.x
Gable G, Sedera D, Chan T (2008) Re-conceptualizing information system success: the IS-impact measurement
model. J Assoc Inf Syst 9:. doi: https://doi.org/10.17705/1jais.00164
Galletta DF, Lederer AL (1989) Some cautions on the measurement of user information satisfaction*. Decis Sci
20:419–434. https://doi.org/10.1111/j.1540-5915.1989.tb01558.x
George D, Mallery P (2009) SPSS for windows step by step: a simple study guide and reference, 17.0 update,
10th edn. Allyn & bacon, Inc., Needham Heights, MA, USA
Hair JF, Black B, Babin B et al (2006) Multivariate data analysis, 6th edn. Pearson Prentice Hall, Upper Saddle River
Hooper D, Coughlan J, Mullen MR (2008) Structural equation Modelling: guidelines for determining model fit.
Electron J Bus Res Methods 6:53–60. https://doi.org/10.1037/1082-989X.12.1.58
Hu L, Bentler PM (1999) Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria
versus new alternatives. Struct Equ Model A Multidiscip J 6:1–55. https://doi.org/10.1080
/10705519909540118
Assessment of a Web-Based Water Information System Performance in the... 4953

Igbaria M, Nachman SA (1990) Correlates of user satisfaction with end user computing: an exploratory study. Inf
Manag 19:73–82. https://doi.org/10.1016/0378-7206(90)90017-C
Jarupathirun S, Zahedi FM (2007) Exploring the influence of perceptual factors in the success of web-based
spatial DSS. Decis Support Syst 43:933–951. https://doi.org/10.1016/j.dss.2005.05.024
Jöreskog K, Sörbom D (2017) LISREL 9.30 student edition
Miles JNV, Shevlin M (1998) Effects of sample size, model specification and factor loadings on the GFI in
confirmatory factor analysis. Pers Individ Dif 25:85–90. https://doi.org/10.1016/S0191-8869(98)00055-5
Nedovic-Budic Z (1999) Evaluating the effects of GIS technology : review of methods. J Plan Lit 13:284–295.
https://doi.org/10.1177/08854129922092405
Obermeyer NJ, Pinto JK (2008) Managing geographic information systems. The Guilford Press, New York
OECD (2018) Implementing the OECD principles on water governance: Indicator framework and evolving
practices. OECD Publishing, Paris, OECD Studies on Water
Petter S, DeLone W, McLean E (2008) Measuring information systems success: models, dimensions, measures,
and interrelationships. Eur J Inf Syst 17:236–263. https://doi.org/10.1057/ejis.2008.15
Rai A, Lang SS, Welker RB (2002) Assessing the validity of IS success models: an empirical test and theoretical
analysis. Inf Syst Res 13:50–69. https://doi.org/10.1287/isre.13.1.50.96
Schreiber JB, Nora A, Stage FK et al (2006) Reportig structural equation modeling and confirmatory factor
analysis results : a review. J Educ Res 99:232–338. https://doi.org/10.3200/JOER.99.6.323-338
Seddon PB (1997) A Respecification and extension of the DeLone and McLean model of IS success. Info Sys
Res 8:240–253. https://doi.org/10.1287/isre.8.3.240
Seddon PB, Yip SK (1992) An empirical evaluation of user information satisfaction (UIS) measures for use with
general ledger accounting software. J Inf Syst 6:75–92
Segars A, Grover V (1993) Re-examining perceived ease of use and usefulness: a confirmatory factor analysis.
MIS Q 17:517–525. https://doi.org/10.2307/249590
Steiger JH (2007) Understanding the limitations of global fit assessment in structural equation modeling. Pers
Individ Dif 42:893–898. https://doi.org/10.1016/j.paid.2006.09.017
Torkzadeh G, Doll WJ (1999) The development of a tool for measuring the perceived impact of information
technology on work. Omega 27:327–339
Ullman J (2001) Structural equation modeling. In: Tabachnick BG, Fidell LS (eds) Using multivariate statistics,
4th edn. Allyn & Bacon, Needham Heights, pp 653–771
Urbach N, Müller B (2012) The updated DeLone and McLean model of information systems success BT -
information systems theory: explaining and predicting our digital society, Vol. 1. In: Dwivedi YK, Wade
MR, Schneberger SL (eds) Springer New York. NY, New York, pp 1–18
Urbach N, Smolnik S, Riempp G (2009) The state of research on information systems success. Bus Inf Syst Eng
1:315–325. https://doi.org/10.1007/s12599-009-0059-y
Wang YS, Liao YW (2008) Assessing eGovernment systems success: a validation of the DeLone and McLean
model of information systems success. Gov Inf Q 25:717–733. https://doi.org/10.1016/j.giq.2007.06.002
Worrall L (1994) The role of GIS-based spatial analysis in strategic management in local government. Comput
Environ Urban Syst 18:323–332. https://doi.org/10.1016/0198-9715(94)90014-0

Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and
institutional affiliations.

Affiliations

Nuno Barreiras 1 & Francisco Nunes Correia 1 & Rafaela Saldanha Matos 2
1
CERIS, Instituto Superior Técnico, Universidade de Lisboa, Av. Rovisco Pais, 1049-001 Lisboa, Portugal
2
LNEC, Av. do Brasil 101, 1700-066 Lisboa, Portugal