SDI Standards.

You have seen in the previous section that the SDI components are wrapped in, or controlled
by, international standards.
Why is this control necessary? (Do you think they are necessary?)
We know that when we travel to other continents it is often necessary to carry electrical plug
adapters:

Electrical plug adapter (for Italy).

Imagine if every company that manufactured electrical equipment created its own type of plug!
Imagine needing special adapters for Sony, for Philips, for IBM, for Dell, etc. The world would be
a crazy place (even crazier than now). To prevent this potential chaos, nations and even entire
continents have created agreements to standardize the electrical plug configuration. (Why
have we not all agreed on one type of plug? Politics as usual...)
These international standards are official (de jure) standards, meaning that national standards
organizations meet and formally decide them.
Something similar happened in the PC computer world about 20 years ago (with the introduction
of Windows95): so-called plug-and-play standards (also called PnP, or jokingly "plug-and-pray").
These non-official (de facto) industry-led standards set the basic rules for allowing any
hardware peripheral (say, a hard drive or a DVD reader/writer) from any manufacturer, to be
installed in any* PC from any manufacturer. (*any PC with an operating system and BIOS
software that also recognizes the standard rules)
These simple rules revolutionized the PC industry and caused prices to drop (or rather, they
have not increased in the past 15 years: you get better and faster computers for the same $1000
investment). This is because now a hardware vendor is unable to trap the user into purchasing
only its peripherals. Only a decade ago, an IBM laptop had almost 100% IBM-branded
components, and many perhaps would not function in a Dell laptop. In the GIS world, Intergraph
Corporation (INGR) sold a complete Intergraph-branded package: INGR GIS software running on
an INGR operating system (a special version of Unix), running on INGR computers using INGR
microprocessors. This practice is no longer possible (or is less common) due to the market
pressures of PnP and similar standards. The ability to mix and choose components from
multiple vendors, provides more flexibility to users and increases competition in the marketplace.
In both cases (electrical plugs and PnP) the standards in question are interface standards. The
electrical plug does not affect the performance of the radio or toaster it is connected to, in the
same way that PnP standards do not affect the performance of DVD writers. They do,
however facilitate the connections in a wide range of possible situations.
To properly understand SDI standards it is important to have a good definition of interface: a
boundary across which two systems communicate. This means that an interface is not the
connection, per se, but rather it is the connection environment: the rules for connection.
Like water, gas, and transportation infrastructures, SDIs also are controlled and guided by
international standards, some de facto industry agreements and other more formal, de jure rules.
In cars, steering wheels and brake pedals all work the same way, as they are
the interface between car and driver. 
In this lesson we will point you to several documents that will explain the relevant standards
that affect SDIs. Some are quite long and complicated texts, so we will try to indicate which are
the most relevant parts: this course is too short to be able to get into all the details.

SDI Standards organizations and examples

Specific SDI-related standards organizations.
The geographic information field is full of technical acronyms and jargon, and acronyms can be
confusing. Many people in the GIS field still do not know what SDI means. Worse, if you search
Google on "SDI" you will see that it can mean many things to many people in many fields. The
relevant SDI standards are also nearly all acronyms-- WMS, GML, WFS, WCS, CS-W-- or else
numbers: ISO19115, 19139, 19119, ...
Here we will look at some of the foundation standards, and link you to the full documents
(where possible) so that you get an idea of the level of detail. This detail normally is not useful
unless you are really planning to implement an SDI component.
First, a brief explanation of the two main sources of SDI standards since the late
1990s: OGC and ISO/TC211.
Open Geospatial Consortium (OGC)
OGC is a consensus membership organization (a club), meaning that it has a very small
infrastructure (about 15 employees including 4 secretaries) but more importantly consists of
member companies, government organizations, and universities. These members (especially
the PC, or Program Committee, board of member representatives) make almost all of the
decisions in OGC: the OGC employees are mostly facilitators. The decisions are consensus-
based, which means that the members reach minimum working agreements about interface
specifications. The process is unofficially called "coopetition" (cooperation+competition):
competitors in the GIS/SDI market sit side by side and collaborate in the search for interoperable
interfaces to facilitate growth in their common marketplace. This is GIS software industry's
equivalent to plug-and-play.
OGC consensus agreements (and resulting specifications) are not perfect for any one member,
however they are judged as being "good enough" for the majority of members to be able to agree
and to move ahead as an industry. OGC was created in 1994 to help the GIS industry escape
from it's small island among the wide ocean of Information Technologies. It is still struggling to
stay relevant among the larger IT world, which keeps advancing.
The OGC consensus process determines what are the problems to be solved by standardization,
and then proposes and votes on OpenGIS interface specifications that help in these problem
areas.
More OGC history and background details at: http://www.opengeospatial.org/ogc/history
The "long history" of OGC is also available at that page. Some people might be surprised to learn
that OGC grew out of the GRASS (open source raster GIS) user community. Also the OGC
founder sold UNIX workstations, and in that world there were UNIX interoparability problems
which slowed business growth.
OGC is a de facto standards organization. In fact, for many years OGC did not produce
standards per se; it produced industry specifications. (This is similar to W3C--the so-called
web standards organization-- that also does not produce standards but, rather, recommendations
and specifications.)  But now OGC calls them standards. Not being an official government
organization they can call them whatever the industry agrees to call them.
OGC specifications are often good enough for software implementation by businesses, however
at times they are judged as not being official, and therefore they cannot be required in some
public sector tenders and project requirements. To facilitate the uptake (application) of OGC
specifications, in 1999 OGC signed a formal working agreement (Class-A liaison status) with
the de jure standards organization ISO, specifically with the ISO Technical Committee 211
("geographic information"). Under this agreement some of the more stable and widely adopted
OGC specifications have become (or are becoming) ISO international standards. We will see
examples below.
Why become an OGC member? Many of the specification documents (standards) are
published at the public OGC website, to be freely exploited by members and non-members alike.
Members, however, are provided access to a much larger intranet (Member Portal) containing
the draft documents as they move through the editing and voting process, and also have access
to OGC presentations, internal notes, etc. This means that when a version 1.0 specification
appears on the public web site, members have had between 6 and 18 months to prepare
themselves. Many members implement specifications even before they appear in public.
Universities like UJI become members to learn what the industry is thinking and planning, and
also to influence their thinking (in a small way). This increased access, and voting priveledge to
be able to "steer the process" comes at a cost: membership fees. You will see at the
membership web page, that the higher levels of membership are negotiated with OGC (the
Strategic level is something like $250.000 per year, not necessarily cash: often the time of
technical employees is donated to the OGC), and then the lower levels have decreasing levels of
cost and priveledges. In fact, for a small business the highest cost associated with being an
active member is not the membership fee, but rather travel cost for sending employees to attend
the frequent meetings in USA and Europe (occasionally Asia). It is difficult to properly exploit the
OGC process unless you attend the meetings in person.
Universities and local government organisations pay very little, US$500 per year, to get a similar
level of access (however no vote).
Finally, it probably goes without saying, that participation in OGC provides interesting business
and research networking (person to person) opportunities.

ISO Technical Committee 211

ISO/TC211 is the international de jure GI standards organization. 
Entering the TC211 "Program of Work" you will encounter a long list of dozens of standards
documents, meeting reports, etc., however you will be unable to access most of them, unless
you have the password.    You will see a huge collection of standard numbers: the so-
called 19000 family.
These are internal documents for the use of ISO/TC211 representatives: each member nation
has one or more representatives (in Spain it is normally a representative from the national
standards body AENOR). If you need to get access to an ISO international standard document
there are essentially two solutions:

1. contact your national representative and ask to have a look (this is appropriate only in
cases where you might be working on a related standards committee or perhaps on an
OGC-related project);
2. visit the ISO website (or in Spain the AENOR website) and purchase a PDF or paper
copy of the document. Note that only final standards are sold, not draft versions.
3. here you can have a look at one of the documents, so that you get an idea of their format
and level of detail. It is not a standard per se, rather a report on a future standard on
 ontologies. (Ontologies help to define common terminology so that, for example, there is
agreement on the meaning and measurement units of slope or intersection or windspeed;
this is needed when GISs talk to each other and share data.)

National representatives make decisions within ISO/TC211, however there are surprisingly few
nations with status of participating (voting) members: only 32...including Spain and Ecuador.
Other nations and partner organizations are observers and do not vote on ISO/TC211 standards.
If we take a look, again, at the long list of 19000 standards, we will find several completed or in
draft stage that are essentially equal to (or de jure approved versions of) OGC specifications.
Examples include:

 19125 Simple Features access

 19128 Web Map Server interface (WMS)
 19136 Geographic Markup Language
 19142 Web Feature Service interface (WFS)

On the Overview web page you will find that many of the standards are hyperlinked, and they
provide a summary of what each standard's goal is.
In addition to the 4 implementation standards mentioned above, other ISO standards that have
evolved from earlier OGC Abstract Specifications. These are more conceptual or informative
specifications, such as for example the description of "feature geometry" (Abstract Specification
topic 1).
This process, converting industry-led specifications to de jure international standards, is
facilitated by the fact that many of the members of both groups (OGC and ISO/TC211) are the
same people. It is normally easy to reach agreement with yourself. 

What about CEN, INSPIRE, FGDC,...?

There have been many other regional or thematic standards organizations at work since the
1980s, including the European CEN Technical Committee 287, the US Federal Geographic Data
Committee (FGDC), and more recently the European INSPIRE initiative. Each has a long history
and the interrelationships are sometimes complex, but to summarize let us say that the
standards of ISO/TC211 are subsuming (replacing) the work of the other organizations (or rather,
in many cases the other organizations are simply adopting ISO standards with local
modifications). CEN Technical Committee 287 has recently been reactivated (after several years
dormant) with the goal to essentially Europeanize some of the ISO standards. Converting these
standards to CEN standards allows them be translated into several European languages (CEN
pays for the translation).

OGC published in 2005 a nice review paper that talks about the relations among these various
standards initiatives and groups. INSPIRE is adopting many ISO+CEN standards, and will be
one of the topics covered in the upcoming lesson on Institutional aspects. Interestingly, 12 years
later, OGC and W3C are still working to try to agree on standards for geospatial data on the
web! 

So, what do these standards acronyms and numbers mean to people who are interested
in building part of an SDI?
Let us recall the 3-tier SDI architecture: SDI clients (top) seeking geodata from data servers
(bottom) with the assistance of geoprocessing services (in the middle) including the most critical
service...the metadata catalogue service.
Let us also recall from the SDI definitions in Lesson 1, that SDI is necessarily
a collaboration among stakeholders ("implicados" in Spanish, actors in the SDI)...somewhat
similar to a distributed corporate GIS. In certain situations a single institution will provide all these
components, and in some cases even with software purchased from the same single vendor. But
in theory having access to components from multiple providers allows more flexibility for making
changes. The information technology (IT) field is characterized not by stability but rather by
constant change, and so will be the future of SDIs. We have already seen how technology has
changed over the first 2 decades of national SDIs: vendor-specific software and national (or
regional) data formats have given way to open standards-based software interfaces, and to more
or less standardized data formats (or families of formats, profiles), and then, more recently there
has been a trend back toward single vendor solutions or house-built open source solutions.
Our geodata servers at the bottom of the 3-tier stack need to be able to receive and understand
requests (queries, petitions) from a wide variety of possible clients and intermediate services.
That is, geoservers need to adopt the server-half of the relevant interface specifications.
Clients and intermediate services need to adopt the other half of the same interface
specifications: 

-----
WMS
Let us take a closer look at the specific case of the Web Map Service (WMS) interface.
Remember that in the last lesson you called WMS services to access climate data of Catalunya.
But what was happening to make it work?
In the years between the invention of the WWW (about 1993) and 1999, many map server
applications appeared on the market. Unfortunately almost all of them created their own special
data formats, required their own special viewer software (usually plug-ins for web browsers), and
were optimised to work only with clients from the same vendor. Each client and server were on
their own island. In short: chaos in the marketplace: a complete lack of interoperability. But
because OGC was created to improve interoperability in the GI marketplace, it quickly noticed
this chaos and changed its focus, from local processing specifications (such as Simple Features
for COM, SQL, etc.) to web service specifications.
In 1999 OGC created its first "testbed" (pilot testing implementation): Web Mapping Testbed 1.
This consisted of a simulated scenario of a hurricane approaching the gulf coast of the USA
(near Mississippi); in order to provide useful evacuation information, multi-vendor GI clients
would be used to access geodata GI servers, also from multiple vendors.

The secret to making the various connections happen: adherence to a newly-created interface
specification....the birth of the WMS specification.

The Web Map Testbed (WMT-1) clearly demonstrated that client-server interoperability was
possible. This left vendors with two options: become interoperable or remain closed. Most
vendors chose the first option, however a long list of minor technical details (small differences in
syntax within the implementations) have been responsible for all sorts of minor incompatibilities
between multiple vendors....small fixes are almost always necessary. (In the end, SDIs are more
complex than electrical plugs, and true interoperability is normally reached by using smart
consultants who understand the standards and their flaws.)

WMS Functions.

The question "what functionality does a Web Map Server have?" is a tricky one. We must be
very careful to clearly separate the types of requests the server recognizes from clients, from
its internal functionality (how it produces and serves maps).

If you study the WMS specification document, you will find, among many other extraneous
details, that the interface specifies only 3 methods or requests:

 getCapabilities: tell me (client) what the server can do

 getMap: send me (client) the map specified in the request
 getFeatureInfo: tell me (client) more about this <mouse click> feature; this is
optional

The many details in the specification document are to provide implementers (programmers) with
more options, to be able to customise their server, and to support (or not) certain optional
functionality.

getCapabilities: the client sends a request with the WMS version it is expecting, and the server
replies with descriptive information about who built it, what formats it supports, names of layers
provided, and spatial reference systems supported. If you send (click on the link)
a getCapabilities request to the British Geological Survey's WMS, you will see its response. If
you program a client app that wants to retrieve map data from a WMS, first your app needs to
ask the server "hey, what's available?".
Then, if your app finds the data sources it wants, it sends a getmap request.

getMap: to see an example from another server, just click here to send a getmap request to an
environmental server and get in return a map of the cryosphere over the Arctic. This is the
request you sent:

http://nsidc.org/cgi-bin/atlas_north?
service=WMS&
version =1.1.1&
request =GetMap&
srs=EPSG:32661&
format=image/gif&
width=1000&
height=1000&
bbox=-2700000,-2700000,6700000,6700000&
layers=sea_ice_extent_01,land,snow_extent_01,permafrost_extent,country_borders,treeline,nort
h_pole_geographic,arctic_circle,country_labels,geographic_features_sea 

The request looks complicated but it is rather easy to interpret the main parameters: it asks for
several map layers (sea_ice_extent, land, snow, etc.) covering a specific geographic area (BBOX
in latitude and longitude coords), in spatial reference system 32661 and specifies that they
should be returned as a GIF image of 1000x1000 pixels.

The getMap operation causes the server to locate the appropriate geodata from the data store
(Oracle, files, etc.), to rasterize it (if it is in vector format) and then to return the bitmap file
(JPEG, GIF, PNG) to the client for visualization. Remember that WMS does not deliver
geodata per se, it delivers images (a picture) of the internal geodata. With each new zoom or
pan, a new image is sent. These images are rather lightweight, and so performance is normally
fine (depends on many server-side and network factors). The full map view is divided into tiles, or
small pieces of the overall image.

To check to see that the server really is building and sending a new map each time, just edit the
long URL at the top of the map window in the browser, and change the 1000 by 1000
dimensions to 500 by 500, or change slightly the BBOX coordinates, and then refresh the screen.
You should see the diffeence in the new map produced by the WMS server.

This GetMap query should work on any GIS client or SDI viewer that claims to support the WMS
standard. In this example, a GetMap request is hidden inside the ArcGIS Online system, but it
still accesses the radar data (updated every minutes) via an internal WMS request.

Summarizing, the standards and specification documents of ISO/TC211 and OGC can act as
guides, to help technically-capable people to build and connect diverse SDI components in a way
that maximizes interoperability. The key is the interface concept: allow unlimited innovation
within a component, but at the same time constrain (control) its interaction with other
components by speaking a lingua franca.

We have seen only the WMS details here, but be assured that the Web Feature Server, Web
Coverage Server, etc. work on similar principles. The WFS is quite a bit more complicated
(beyond the scope of this short course) because it deals with query filters, and returns geodata in
GML format. Unless you are familiar with XML concepts (especially namespaces) then it is
difficult to appreciate what WFS/GML offers.

In lesson 5 we will look again at the catalogue services, which are a bit different from
WMS/WFS however they do follow the same principal of connecting diverse servers to diverse
clients thanks to interface standards.

Standards Overview
SDI standards overview.
Now that we have frightened you (or bored you to sleep) with some of the specific standards
documents, here are a couple of general documents showing ordinary developers (not GIS
nerds) how these standards can be useful.

 Eclipse Newsletter, article by Raj Singh.

 Overview on open standards by Esri, which of course supports its own internal formats
but also the de facto open standards as well.

Summarizing again: standards are a way to control, to some extent, the innovative
developments in any field, so that the multiple developments do not all go off in their own
directions.
Another critical message that we hope you have captured from this lesson, is that although the
standardization organizations have edited many, many documents pertaining to GI, in the SDI
field there are really quite few standards to pay attention to. Nearly all of these SDI
standards are interface standards: standards to facilitate (again) collaboration between
different stakeholders or responsible parties within the SDI.
On the non-technology side, we should just briefly mention that agreed definitions of SDI, and
agreed minimum necessary levels of collaboration, of data sharing, of openness, are also a
more intangible part of SDI standards. These ideas are spread not through standards
documents per se, but rather through education (capacity building) activities: seminars,
courses, e-mail discussion lists,...
Now it is time for you to answer a few (3) questions, as a control that you have read the
previous 3 pages and have downloaded and looked over the hyperlinked documents....for
example the ISO standards list.