DATA QUALITY AND ITS ROLE IN DECISION MAKING

TONY GILL
GradCertPubSecMgmt, GAICD

Department of Lands and Planning GPO Box 1680, Darwin NT 0801 Tel: +61 8 8995 5317 Email: tony.gill@nt.gov.au Abstract
As more organisations, government and non-government, embrace spatial information and technologies, greater attention needs to be placed on the role and importance of authoritative data sources. The NT Government’s Spatially Enabling Government (SEG) initiative will further drive the discussion within Government. However, awareness and the significance of the issue must be brought into the public domain. This paper highlights the need for greater focus on the role of authoritative spatial datasets in decision making by government, industry and the public and argues the need for data custodians of authoritative spatial data sources to place greater attention on transparent quality assurance practices. Questions posed by this paper and their implications include; what is the role of quality assurance in the creation and maintenance of spatial data, should we be demanding it of spatial data suppliers, what value will it add and what are the risks in not achieving quality assurance certification? With the emergence of a range of alternative spatial data sources, the case for a dataset certification regime is put forward as a mechanism whereby custodians of authoritative spatial data sources can stand out in the growing information market place.

Spatially Enabling Government Spatially Enabling Government (SEG) is a Cabinet endorsed initiative of the NT Government with the objective of providing a whole-of-government (or as equated by Althaus, C & Tiernan (2005) as joined-up-government) approach for the enhancement of Government services through the provision of underlying spatial information and associated services. SEG includes a range of activities, not all fully funded, including; • • • • • Establishing location and street addressing services Improved coverage, currency and quality of imagery and mapping Provision of satellite-based positioning infrastructure and facilities Assistance to agencies to spatially enable their information and business operations, and Provision of a governance framework for the realisation of SEG benefits

1

The vision of SEG is to generate significant opportunities to deliver integrated services to the community, but to also become an enabling component of Joined-up Government. The NTG’s ICT Strategic Intent 2010-15 document (NTG, 2009) highlighted the significance of information in the decision making process when it stated “Decisions are based on information and knowledge” and as a consequence “information must be easy to access securely, accurate, reliable, timely and usable”. Without achieving these attributes, there is a risk in policy and operational decisions being made that are not only flawed and costly, but could put lives at risk. Government, industry and the community are in a spatially or location enabling phase. There is recognition that everything happens somewhere and as Tony Maber (MDS, 2011) states “Organisations not using the location element of retained or acquired business information are missing out on a valuable ingredient when making decisions”. Maber claims “A picture paints a thousand words, but a map tells the whole story!” (MDS, 2011). However, explorer Captain James Cook, found time and again that maps of his time were created based on myth, misinformation or just plain poor information rather than fact or carefully observed measurement (Film Australia, 2007). Without Captain Cook correcting the maps of the day many decisions of the time may have had disastrous consequences for mariners and business men. In a whole-of-government approach, solving complex issues like Closing The Gap of indigenous disadvantage require sound sources of information that span agencies and levels of government. Key to this is ensuring we start from a sound base of good quality authoritative information. Origins of quality and its links to spatial information Dr. Joseph M. Juran’s book, Managerial Breakthrough in 1964 led to quality management and improvement systems like Lean and Six Sigma (Juran, 2011). The American Society for Quality (ASQ) was formed in 1946 as the American Society for Quality Control and now self-describes itself as the “champion of the quality movement” (ASQ, 2011), of which Juran was made an Honorary Member in 1968. While ISO 9000 has its roots from British standards (Wikipedia, 2011b), in the early 1980’s in the USA, Government and industry, saw the need to focus on quality as part of doing business in an expanding competitive global marketplace (NIST, 2010) and enacted legislation, The Malcolm Baldrige National Quality Improvement Act of 1987 that established the Malcolm Baldrige National Quality Award “to promote awareness of performance excellence as an important element in competitiveness … that would help U.S. companies achieve worldclass quality” (NIST, 2010). In 2001, the U.S. National Institute of Standards and Technology (NIST) commissioned an evaluation of the net benefits of the Baldrige Program and extrapolated entire economy benefits of USD $24.65 billion (year 2000 present value) based on USD $2.17 billion (year 2000 present value) for members of the ASQ (NIST, 2001). The evaluation has

2

demonstrated linkages between the implementation of quality initiatives to the organisation and the nation. The ISO 19000 series of standards for Geographic Information contains a range of standards that relate directly to the field of spatial data quality. This includes, among others, ISO 19113:2004 (Quality Principles), ISO 19114:2005 (Quality Evaluation Procedures), ISO 19138:2008 (Data Quality Measures), ISO 19115:2005 (Metadata). This represents a relatively comprehensive set of standards for the management of spatial information and the recording and measurement of its quality at the dataset/product level. The draft ISO 19158 (Quality Assurance of Data Supply) standard is currently being worked on and is expected to be published in April/May 2011 (Body, 2011). It recognises the role of ISO 9000:2005 as establishing the principles of quality management and other ISO 19000 series standards for the data quality reporting and evaluation. The Open Geospatial Consortium (OGC) is a consensus-based organisation of industry, government agencies and universities with the aim of developing interface standards to support interoperable vendor independent geo-enabled solutions (OGC, 2011). OGC have recognised the need to ensure that these interoperable systems are capable of exchanging information on data quality and have formed a Data Quality Working Group (OGC, 2011b). What defines quality? The Government of British Columbia defines data quality as being “The state of completeness, validity, consistency, timeliness and accuracy that makes data appropriate for a specific use” (Wikipedia 2011) and aligns well with the NTG ICT Strategic Intent. The standard on Geographic Information – Quality Principles (AS/NZS ISO 19113:2004) raised a key issue that challenges the specific use qualifier, when it acknowledges the increasing tendency for geographic datasets to be “shared, integrated and used for purposes other than their producer’s intended ones”. As such, quality becomes a subjective assessment by the consumer of the data as to its fitness for use for a particular purpose that may be outside the original intended use of the data by the producer. The significance of Authoritativeness in quality Authoritative is defined as “arising or originating from a figure of authority” or “highly accurate or definitive” (Wikitionary, 2011) and authoritativeness is defined as “the quality of possessing authority” or “the quality of trustworthiness and reliability” (Wikitionary, 2011b). All of these descriptors could easily be applied to cadastral datasets produced and overseen by the respective Surveyors-General in each State and Territory jurisdiction of Australia. This comes to being because of the Torrens Title System whereby the Government, through the Registrars-General and Surveyors-General guarantee indefeasibility of title. This provides a high level of confidence to the public, financial institutions and industry in the accuracy and

3

reliability of the cadastre. That is not to say that the cadastre does not and will not be “inaccurate” relative to other measures at times and the consequences that government must deal with in these circumstances. However, it is not always as clear for many other datasets. Take for instance topographic data. What is it that makes one topographic dataset more authoritative than another? Both may well have observed the XYZ of a specific physical feature on the earth’s surface, but have recorded different values both within acceptance tolerances of the techniques employed. Contours produced from a DEM of LIDAR points with 1.0m spacing may well have individual Z accuracies of +/- 0.15m will not concur with contours produced from a more sparsely populated DEM with observed breaklines from photogrammetry with Z accuracies of +/- 0.30m. Which dataset is more accurate is open for debate as it really depends on the intended use and whether the advantages and limitations of one technique override the other. Trust – Is it real or imagined? David Schell, then Chairman and CEO of Open Geospatial Consortium, stated “there is a tremendous need to pay attention to liabilities, that is the risks associated with providing data and services that may figure prominently in important decisions affecting lives and property” and that “digital spatial information is a matter of public trust and we need to recognise this” (GIM, 2008). Trustworthiness and reliability are at the heart of authoritativeness and for many government produced datasets this is assumed by the community of users. Unlike private enterprise, Governments are enduring entities and as Schell points out, liability must be managed and as a result a level of trust is inherently placed in government produced datasets. This attention to liabilities makes government slow at responding to user requirements or paradigm shifts compared to their private enterprise counterparts. Private enterprise however, trades in its “Reputation Capital” (Van der Vlugt, 2011). The purpose of a private enterprise organisation is to be profitable to its owners and not necessarily to be enduring, although company structures enable it to be so. For private enterprises whose stock in trade is services, be it professional or information, reputation is everything. It is the fundamental asset to the organisation and is most recognisable through goodwill on a company’s balance sheet. A change in reputation has direct linkages back to the underlying value of the enterprise. The goodwill associated with a company makes it attractive to other companies and individuals and provides an exit strategy for its owners. Therefore, maintaining the reputation capital is vital to the owner’s interests. Rebuilding a damaged reputation is costly and sometimes terminal to the organisation such as the case of Pan Pharmaceuticals Limited’s eventual collapse as a consequence of the Therapeutic Goods Administration’s suspension of its license “because of serious concerns about the quality and safety of products manufactured by the company” (TGA, 2003). Elements of decision making Kohl (2010), defines the decision making process as having six steps;

4

1. Identify and define the problem 2. Gather and analyse the information 3. Develop alternative solutions 4. Choose the best alternative 5. Take Action 6. Evaluate the decision. Importantly, Kohl emphasises the critical need to “have accurate information to solve issues” in the decision making process and that if you “short-change this part of the process, you can create unnecessary delays and unintended results”. What is the role of quality assurance in the creation and maintenance of spatial data? The substitution of unauthoritative or unknown quality data can lead to poor decisions that at best may be an inconvenience and at worst life threatening to those relying on the subsequent decision. In the case of the Gretley Mine Disaster (DPI, 1998), his Honour Acting Judge Staunton, in his Summary of Findings (DPI, 1998b, pg. 41), identified a root cause of the water in-rush that killed 4 miners as a failure by the Department to properly interpret a plan in compiling two other plans, which were subsequently relied upon by others in the compilation of other documents such as the mine plan. The Government’s response to the inquiry, draws on evidence from the inquiry “that the surveyors who worked at the Gretley mine implicitly believed the plans of the old workings were accurate because they were held by the Department” (DPI, 1998c, pg. 3), even though the Department maintained the view it was “solely the repository for records”. While these tragic consequences should be few and far between, it highlights the significance of quality assurance practices in the management of information as well as understanding the lineage of information that is compiled from other sources that are subsequently relied upon. Should we be demanding quality assurance of spatial data suppliers? NT Government tenders request tenderers to indicate if they have a documented Performance Plan and on which standard is it based (e.g. ISO 9000:2000, or other). The purpose behind this is to understand if the tenderer has sufficient processes in place to ensure that the requested product or service is delivered to specification and where it is not then there are processes in place to correct the defect and minimise its future occurrence, thus reducing the potential risk to the acquirer of the products and services. In comparison, Government in it liability management modus operandi, attempts to use disclaimers to absolve itself from anything but a duty of care through statements like “does not warrant this product or any part of it is correct or

5

complete and will not be liable for any loss, damage or injury suffered by any person as a result of its inaccuracy or incompleteness” (DLP, 2011). If quality assurance practices are something government demands of its suppliers, why is it that Government cannot embrace the same standards when it is a provider of spatial product and services? What value will it add and what are the risks in not achieving quality assurance certification? Public trust and reputation capital are crucial elements in an end-users ability to do their day-to-day work without regard to the underlying soundness of the information on which they rely to make decisions. The Gretley mine disaster brings home how fragile and false this blind trust can be. A functioning and effective quality management system is both an insurance policy on an organisation’s reputation capital and as evaluated by NIST adds value to the organisation. When procuring products and services, government aims to get value for money. This is not a simple equation of cost versus time, but recognition that quality is part of the equation. An organisation that has effective quality management systems is more than likely able to contain the cost and time elements while producing a quality outcome. Google embraces User Generated Content (Crowd Sourcing) 9 November 2010 was a significant date in the spatial world as this is when Google announced (Google, 2010) they were moving from the widely accepted authoritative PSMA Australia (PSMA) data source for cadastral and addressing information. The PSMA data is aggregated on a quarterly basis from all of the jurisdictions in Australia, Australia Post and the Australian Electoral Commission to produce CADLite™ and GNAF™ that is released to value added resellers and on-licensed to organisations such as Google for use in a range of web-based map and location services. Google have licensed a one-off copy of non-authoritative datasets from the Telstra subsidiary Sensis (SpatialSource, 2011). Interestingly, numerous places in the Northern Territory can be found to have inaccuracies in Google Maps, some of which end-users would not even know were problematic. Examples include non-current parcels in Tennant Creek (Google 2011b) to a non-authoritative cadastre misaligning with road centrelines (Google, 2011c). It is questionable how many users will be even bothered to report the problem using the “Report a problem” link that requires the user to go through a registration process and for Google to fix it. This is a very interesting development for Google. They have built a reputation of trust in delivering the search results that their user base expects within the first page. When their user base starts to question the accuracy (trustworthiness) of their map data, then Google runs the risk of taking a hit on their reputation capital. As most internet based businesses rely on visitation rates, as the user base declines, so does the market value and revenue generation capacity of the organisation.

6

In 2007, Edelman questioned the end of MySpace (Edelman, 2007). In 2011 MySpace announced it was closing operations in Australia, UK and Germany as it faces stiff competition for its user base from companies like Facebook (TechGeek, 2011). Edelman made an interesting point that MySpace had “built on an unsustainable foundation. They’ve made the classic gamble that shortterm gain will trump long-term stability … MySpace is headed for a big, clumsy fall”. As Edwin Haverkamp of MapData Sciences, explains “spatial elements in data are the glue that binds information together … You need good quality and frequently updated mapping data to make critical business decisions and every day planning” (MDS, 2011). Time will tell whether Google are able to persist with User Generated Content or whether they return to using an authoritative and regularly maintained source of data that their user base can be confident in using to base decisions on. Authoritative data sources – standing out from the crowd The question remains. How do authoritative data sources stand out among the ever crowded marketplace for spatial information? As more and more web-based mapping services (e.g. Google Maps, Microsoft Bing Maps) and various mashups appear, it is becoming harder for the end user to get a sense of the quality of the underlying data. ISO 9000 Certification is proudly emblazoned across company letterheads and documents for a reason. It demonstrates to it clients, current and prospective, as wells as its competitors, that it takes quality seriously and that its makes business sense. The time is rapidly approaching whereby spatial data suppliers achieve ISO 9000 certification as well as full compliance with the ISO 19000 standards relating to quality to not only insure their reputation capital but also as a mechanism to manage their liabilities. There is even a case for the spatial data suppliers to embrace a certification regime that is able to recognise the authoritativeness of a dataset as one of the qualities that define it. Once this is achieved, the OGC Data Quality Working Group can then provide a mechanism by which online mapping and mashups can signify the authoritativeness of the various datasets being displayed in a similar way that web page certificates are issued and are able to be queried by the end-user. Conclusion Data quality is a key component of the decision making process. Without understanding data quality poor, costly and sometimes tragic consequences can result. Organisations that do not embrace a quality management system are likely to suffer from a loss of reputation and as a result may not survive in the longer-term. Users need to know whether they can trust the spatial information they are using and a mechanism needs to be found that provides them with a level of confidence that decisions they are about to make using spatial information are foundered on good quality reliable data.

7

References Althaus, C & Tiernan, A 2005, Managing Out: The Public Sector in the Community, PSM Program 2005, Australian Public Service Commission, Canberra ASQ 2011, Who We Are, American Society for Quality, viewed 27 Jan 2011, <http://asq.org/about-asq/who-we-are/index.html> Body, C 2011, Geospatial Standards Architect and Chair IT-004 Geographic Information Committee, Standards Australia, pers. comm., Jan 2011 DLP 2011, Standard Map Product Disclaimer, Land Information Division, Department of Lands and Planning, Northern Territory Government DPI 1998, Mine Safety - Gretley Inquiry Summary, Department of Primary Industry, New South Wales Government, viewed 10 Jan 2011, <http://www.dpi.nsw.gov.au/__data/assets/pdf_file/0004/87160/GretleyInquiry-summary.pdf DPI 1998b, Mine Safety - Gretley Inquiry, Department of Primary Industry, New South Wales Government, viewed 10 Jan 2011, <http://www.dpi.nsw.gov.au/minerals/safety/mine-safety-initiatives/gretleyinquiry> DPI 1998c, Mine Safety – Government Response, Department of Primary Industry, New South Wales Government, viewed 10 Jan 2011, <http://www.dpi.nsw.gov.au/__data/assets/pdf_file/0003/87159/GretleyInquiry-Govt-response.pdf> Edelman, D 2007, The End of MySpace, viewed 2 Feb 2011, <http://www.davidlouisedelman.com/technology/end-of-myspace/> Film Australia 2007, Captain Cook: Obsession and Discovery, Film Australia GIM 2008, A New Age II, GIM International, February 2008 Google 2010, Changing World, Changing Maps, Google, viewed 4 Jan 2011, <http://google-latlong.blogspot.com/2010/11/changing-world-changingmaps.html> Google 2010b, Google Maps – Tennant Creek, Google, viewed 2 Feb 2011, < http://maps.google.com.au/maps?f=q&source=s_q&hl=en&geocode=&q=t ennant+creek&sll=25.335448,135.745076&sspn=65.081593,60.029297&ie=UTF8&hq=&hne ar=Tennant+Creek+Northern+Territory&ll=19.654935,134.196432&spn=0.017318,0.014656&z=16> Google 2010c, Google Maps – Katherine, Google, viewed 2 Feb 2011, < http://maps.google.com.au/maps?f=q&source=s_q&hl=en&geocode=&q=k atherine&aq=&sll=14.231523,129.522028&sspn=0.071299,0.058622&ie=UTF8&hq=&hnear= Katherine,+Northern+Territory&ll=14.468796,132.292452&spn=0.002226,0.001832&z=19>

8

Juran 2011, About Juran – Our Founder, Juran Institute, viewed 27 Jan 2011, <http://www.juran.com/about_juran_institute_our_founder.html> Kohl, Tammy A.S. 2010, The Principles of Decision Making, viewed 31 Jan 2011, <http://www.goodezines.com/ezinearticles/the-principles-ofdecision-making/> MDS 2011, What’s Happening in Mapping – Issue #48, MapData Sciences, viewed 31 Jan 2011, <http://www.mapds.com.au/Newsletters/newsletter/1101/1101.asp> NIST 2001, Economic Evaluation of the Baldrige National Quality Program, National Institute of Standards and Technology, viewed 27 Jan 2011, <http://www.nist.gov/director/planning/upload/report01-3.pdf> NIST 2010, Baldrige FAQs, Baldrige Performance Excellence Program, Frequently Asked Questions, National Institute of Standards and Technology, viewed 27 Jan 2011, <http://www.nist.gov/baldrige/about/baldrige_faqs.cfm> NIST 2010b, The Malcolm Baldrige National Quality Improvement Act of 1987 – Public Law 100-107, National Institute of Standards and Technology, viewed 31 Jan 2011, <http://www.nist.gov/baldrige/about/improvement_act.cfm> NTG 2009, NTG Information Communication Technology – Strategic Intent 2010-2015, June 2009, Version 0.7, ICT Policy and Strategy Branch, Department of Business and Employment, Northern Territory Government, viewed 27 Jan 2011, <http://www.territory2030.nt.gov.au/doc/feedback/45NTGIMC_Attachment 1.pdf> OGC 2011, About OGC, Open Geospatial Consortium, viewed 1 Feb 2011, <http://www.opengeospatial.org/ogc> OGC 2011b, Data Quality Working Group, Open Geospatial Consortium, viewed 1 Feb 2011, <http://www.opengeospatial.org/projects/groups/dqwg> SpatialSource 2011, Sensis comments on Google Development, SpatialSource, viewed 2 Feb 2011, <http://www.spatialsource.com.au/2011/01/25/article/Sensis-commentson-Google-development/YLZVBBGXGV> TechGeek 2011, MySpace to close operations in Australia, axes 500 jobs, TechGeek.com.au, viewed 2 Feb 2011, <http://techgeek.com.au/2011/01/12/myspace-to-close-operations-inaustralia-axes-500-jobs/> TGA 2003, Pan Pharmaceuticals Limited – Regulatory Action and Product Recall Information, Therapeutic Goods Administration, Department of Health and Aging, Australian Government, viewed 31 Jan 2011, <http://www.tga.gov.au/recalls/2003/pan.htm>

9

Van de Vlugt, Maurits, 2011, LinkedIn Poll – How do you know if a data source you are using is authoritative?, LinkedIn Poll Comment, <http://polls.linkedin.com/vote/116805/pfdby> Wikipedia 2011, Data Quality, viewed 27 Jan 2011, <http://en.wikipedia.org/wiki/Data_quality> Wikipedia 2011b, ISO9000, viewed 31 Jan 2011, <http://en.wikipedia.org/wiki/ISO_9000> Wikitionary 2011, Authoritative, viewed 27 Jan 2011, <http://en.wiktionary.org/wiki/authoritative> Wikitionary 2011b, Authoritativeness, viewed 27 Jan 2011, <http://en.wiktionary.org/wiki/authoritativeness>

10

Sign up to vote on this title
UsefulNot useful

Master Your Semester with Scribd & The New York Times

Special offer for students: Only $4.99/month.

Master Your Semester with a Special Offer from Scribd & The New York Times

Cancel anytime.