Introduction to GDAL/OGR

Johannes van der Kwast

UNESCO-IHE Institute for Water Education

E-mail: j.vanderkwast@unesco-ihe.org

Version 3.1, September 2015

OpenCourseWare

1
Contents
1 Introduction ............................................................................................................................................. 3
1.1 Learning objectives .......................................................................................................................... 3
2 Prerequisites ............................................................................................................................................ 3
2.1 Installation ....................................................................................................................................... 3
3 Using the command line .......................................................................................................................... 4
3.1 Opening a command prompt ........................................................................................................... 4
3.2 Viewing the contents of a folder...................................................................................................... 5
3.3 Changing directories ........................................................................................................................ 6
3.4 Creating files .................................................................................................................................... 8
3.5 Copy, move, rename and delete files ............................................................................................ 11
3.6 Command history ........................................................................................................................... 12
4 GDAL - The Geospatial Data Abstraction Library ................................................................................... 13
5 Retrieving information from GIS data .................................................................................................... 14
5.1 Raster data ..................................................................................................................................... 14
5.2 Vector data..................................................................................................................................... 14
6 Reprojecting files ................................................................................................................................... 15
6.1 Introduction ................................................................................................................................... 15
6.2 Raster data ..................................................................................................................................... 15
6.3 Vector data..................................................................................................................................... 15
7 Change raster properties ....................................................................................................................... 16
7.1 Resize an image.............................................................................................................................. 16
7.2 Change raster format ..................................................................................................................... 16
8 Spatial queries of vectors ....................................................................................................................... 17
9 Convert Comma Separated Files ............................................................................................................ 18
10 Batch conversion .................................................................................................................................... 20

2
1 Introduction
During these exercises you will get familiar with the basics of the Geodata Abstraction Library (GDAL).

The first part of these exercises introduce you to the DOS command prompt. If you are already familiar
with DOS commands, you can skip chapter 3 and start immediately with chapter 4.

These exercises are provided as OpenCourseWare and come without support. For support, please
register to our related short courses that can be found at http://www.unesco-ihe.org/shortcourses.

1.1 Learning objectives

After these exercises you will be able to:

• Use the DOS prompt

• Understand files and directories and their relative and absolute paths
• Retrieve information (metadata) from raster and vector data
• Convert raster and vector data formats
• Reproject raster and vector data formats
• Change raster properties
• Perform spatial queries on vector layers
• Convert comma separated files
• Convert spatial data to Google KML.

2 Prerequisites

2.1 Installation

For these exercises GDAL needs to be installed, preferably using the OSGEO4W distribution package.

The GDAL website: http://www.gdal.org

The OSGEO4W website: http://trac.osgeo.org/osgeo4w/

3
3 Using the command line
During the excersises we are going to use the Command Prompt. If you already know how to use the the
Command Prompt (aka DOS prompt), you can skip this section.

3.1 Opening a command prompt

To open a command prompt you can go do:

Start → All Programs → OSGeo4W →OSGeo4W Shell

Now you will see the Command Prompt on your screen:

4
 C:\> is called the command prompt. It shows your current working directory (or folder). In this case it

means that you are at drive C (which is called the root). You cannot delete or edit the command

prompt. You'll have to type the commands after the > of the command prompt.

 How does your command prompt look like? Give drive name, directory and (sub)directories. It
may differ from the example in the screenshot.

3.2 Viewing the contents of a folder

In this section, you will view the contents of a directory by using the dir command. The dir
command stands for "directory."

 Type the following command after the prompt: dir

After typing a command press the <Enter> key to execute.

You will see something similar to this:

This is called a directory list. A directory list is a list of all the files and subdirectories that a directory
contains. If you type dir /? you will see all options of the dir command. For example, if you type
dir /w you will see the directory listing in columns.

5
3.3 Changing directories
Look at the list on your screen. All the names that have <DIR> beside them are directories (or folders).
The others are files. The size of the files is also displayed. You can see a list of the files in another directory
by changing to that directory, and then using the dir command again.

 Type the following commands:

md John
md Peter

These commands create subdirectories called John and Peter. Check if the directory has been created
by listing the directories.

 Type:
dir

 Also try:
dir /AD

that only lists the directories

 Now to go to the newly created directory "John" by typing

cd John

cd means "change directory". You will see that the command prompt changes to C:\...\John>

6
DOS is not case sensitive (it interprets upper and lowercase in a similar way), so you could also have
typed cd john.

 List the contents of the directory John by typing

dir

Now you'll see that the directory is not empty. There are to directories, namely . and ..

Actually, the directory is empty, but it also shows the relative paths:

. means "current directory"

.. means "mother directory (or one directory up)"

 Type:
cd ..

 Check the command prompt: we're back in the directory where we started the exercise. Now

type :

cd .

 What is the result of this command? Explain why.

We can go back to directory John using the relative path: cd

.\John

Note that this is the same as cd John.

7
Or the absolute path (in my case): cd C:\Users\jkw\John

If we want to go to the root (drive C) we can do it using the absolute path:

cd \

or the relative path:

cd ..\..\..

 Go back to the directory called John. You can choose if you want to use the relative or absolute
path.
 Type cd ..\Peter
Look at the prompt and explain what this command did.

Both ways of changing directories are equivalent. Why bother than? Well, if you move your files that are
organized in directories and subdirecties to another drive (e.g. drive D), than your absolute paths will not
work, while your relative paths will still refer to the same locations. This is specifically important when
writing scripts, which we'll do later.

 Now we're going to delete the directory called Peter. First move one directory up, type:
cd ..
than type:
rd Peter

rd means "remove directory".

 List the directory to check if Peter has been removed.

 If you want to change to another drive, e.g. drive D, you use this

command:

d:

 Now go back to c by typing:

c:

You'll be back in the last directory accessed on the C drive.

 Move back to the directory where you created John before continuing with the next exercise.

3.4 Creating files

Of course you can create files using your Windows software, but since you're learning the command
prompt you'll learn how to create a simple ASCII text file.

8
  Type:

copy con listdirectories.bat

 Type:

dir /AD

 Type

<CTRL>-<Z>

(keep the control button pressed and type z)

Your screen looks like this:

 Check if the file is in the directory list. What is the size of the file? Now display the contents of
the file on the screen. Type:

type listdirectories.bat

 Now type:
listdirectories

Congratulations! You have created your first batch file. Batch files are scripts that can be used to
execute commands in batch. In this case the file executed dir /AD

Because batch files always have the file extension .bat, the computer knows that this is a batch file
and will execute the commands in the file. We'll come back to this later.

 Let's create another file. Type:

dir > list.txt

This command will not show the result of the dir command on the screen, but saves it to an ASCII
file called list.txt. So we can use > after a command to save its results to a file instead of
printing it to the screen.

 Check the contents of the file:

type list.txt

9
 If the list is larger than the Command Prompt screen, you have to scroll. With large text files it is
easier to use:

type list.txt | more

The result of list.txt is given to the more command which displays the results in pages as big as your
window. Press <ENTER> to see the next line. Press <SPACE BAR> to see the next page. Press
<CTRL-C> to stop. You can use this last key combination to stop any command if it isn't doing what
you like, it crashed or it takes too long.

 Type:
more

Nothing happens, because the more command expects input from another command. So you can
wait forever. In this case you can stop the execution of the more command by using <CTRL-C>.

 Now try this:

type listdirectories.bat >> list.txt

 Display the result:

type list.txt | more

 What happened?

In summary:

• > saves the result of a command to a new file. If the file on the right hand of > already exists,
it will be overwritten.

• >> appends the result of a command to an existing ASCII file.

• | uses the result of the command on the left hand side in the command on the right hand side

of the |. This is called a pipe.

• Use <CTRL-C> to stop the execution of a command.

We can also use these operators to create so called lock files. These are used in scripting. The script will
check if a certain file exists. If it exists, the program will wait, if it is removed, the program continues.
Therefore, these files can be empty. You can create a lockfile with: type NUL > lockfile.txt

10
3.5 Copy, move, rename and delete files
 Now we can copy the file list.txt to a new file by typing:

copy list.txt newlist.txt

 We can also copy list.txt to subdirectory John:

copy list.txt john\newlist.txt

 Now we move newlist.txt to the directory John:

move newlist.txt John

 Go to the subdirectory John and check the directory listing.

 Because it is a bit confusing to have a copy of list.txt in subdirectory John, we're going to rename
it:

rename list.txt listjohn.txt

 Check the directory listing again.

 Save the directory listing to a file called dirjohn.lst

Now I'll introduce so called wildcards.

 Type dir
 Type
dir *.txt
dir *john.*
dir *.??t
dir *.?xt
Explain the function of * and ?

 We can also append textfiles:

copy dirjohn.lst+listjohn.txt listappend.txt

 Because we made copies, we want to remove duplicate file. Type:

del newlist.txt

You can now remove all files by typing:

(Make sure you are in the right folder!!)

del *.* (or simply: del .)

11
 Now remove the directory John.

3.6 Command history

Sometimes you often use the same command. There are several tricks to type them more efficiently.

 Type <F3>

This repeats the last command used.

 Clear the command prompt and press the right arrow button several times. You can see that the
characters of the previous command are repeated. If you press the up and down buttons, you
can browse through the previously used command.

When you use <TAB> while typing a path or a filename, it will try to automatically complete it.

 With the doskey command we can do even more. Type

doskey /h

This prints all the commands you typed during this session to the screen.

If you close the command prompt, the command history will be lost.

 Save the command history to a text file using one of the commands previously learned.

In this way you can edit the command history file in e.g. notepad. If you remove all the wrong commands
and you save the file with the .bat extension, you can execute all of the commands in batch.

 Try this for a few of the commands you have learned so far.

You can close a command prompt either by clicking on the cross in the corner, typing exit and pressing
enter, or choosing Close when right clicking the Command Prompt icon on the task bar.

12
4 GDAL - The Geospatial Data Abstraction Library
GDAL is a translator library for raster geospatial data formats that is released under an X/MIT style Open
Source license by the Open Source Geospatial Foundation. As a library, it presents a single abstract data
model to the calling application for all supported formats. It also comes with a variety of useful
commandline utilities for data translation and processing. See for more info: http://www.gdal.org

 Download the GDAL using the OSGeo4W installer: http://trac.osgeo.org/osgeo4w/

 Download the dataset for this exercise and save it to your harddrive e.g. D:\gdalExercises

The folder contains freely available GIS data:

roads.shp: road map from OpenStreetMap (http://openstreetmap.org)

srtm_37_02.tif: tile of a Digital Elevation Model (DEM) from the Shuttle Radar
Topography Mission (SRTM) (http://www2.jpl.nasa.gov/srtm/)

gem_2011_gn1.shp: borders of Dutch communities, freely available from CBS (Statistics

Netherlands) and Kadaster (http://www.cbs.nl).

Before continuing, make sure that you are in the right folder.

 Open the OSGeo4W Shell (Start → All Programs → OSGeo4W → OSGeo4W Shell)

 Change directory to
D:\gdalExercises

13
5 Retrieving information from GIS data

5.1 Raster data

One of the easiest and most useful commands in GDAL is gdalinfo. When given an image as an
argument, it retrieves and prints all relevant information that is known about the file. This is especially
useful if the image contains additional tag data, as is the case with TIF files. When working with satellite
imagery, this is an extremely useful way of keeping track of the images location in long./lat. coordinates
as well as the image projection.

 Execute the following command:

gdalinfo srtm_37_02.tif
 What is the size of the image?
 What is the coordinate system? What is the EPSG code?

5.2 Vector data

Sometimes similar information is needed from a vector image, for this there is ogrinfo.

 Execute the following commands:

ogrinfo -al roads.shp | more
ogrinfo -al gem_2011_gn1.shp | more
 What are the coordinate systems of these shapefiles?
 Lookup the EPSG codes of both shapefiles at http://spatialreference.org. You'll need this later.

There exists a useful online tool to convert .prj format (as included by ESRI files) to EPSG code:
http://prj2epsg.org

14
6 Reprojecting files
6.1 Introduction
In this exercise we want to make a map of the community of Delft with the main roads and the relief.
Because the datasets are in different formats we have to reproject them to a common coordinate
system. Here we reproject all datasets to the Dutch Amersfoort/RD New projection.

6.2 Raster data

GDAL has the capability to change a raster coordinate system using the following syntax:

gdalwarp -t_srs EPSG:... <input> <output>

The -t_srs argument specifies the target coordinate system. If the source coordinate system is
unknown it must be specified with the -s_srs argument. EPSG:... specifies the EPSG code of the
projection.

We are now going to reproject a Digital Elevation Model (DEM) acquired by the Shuttle Radar
Topography Mission (SRTM). You can use this website to download DEM's for your area of interest:
http://srtm.csi.cgiar.org/index.asp

In order to reproject the DEM from WGS-84 lat/lon to Amersfoort/RD New we use this command:

gdalwarp -t_srs EPSG:XXXXX srtm_37_02.tif dem_rd.tif

 Replace XXXXX with the proper EPSG code for Amersfoort/RD New (see one of your previous
answers using ogrinfo).
 Execute the command and visualize the result in QGIS.

6.3 Vector data

For vector data again ogr is used to convert the OpenStreetMaps road data to the Amersfoort/RD New
projection.

 Execute:
ogr2ogr -t_srs EPSG:XXXXX roadsreprojected.shp roads.shp
Replace XXXXX with the proper EPSG code.

Note that with ogr the output filename is typed before the input filename! With gdal and most other
tools this is different.

15
7 Change raster properties
7.1 Resize an image
The gdal_translate command gives us several options to modify raster images.
gdal_translate can be used simply to change the size of an image using the -outsize
parameter, which takes two integer values as the xsize and ysize respectively, or two percentage
values to scale the image. The syntax is:
gdal_translate -outsize newxsize newysize inputFile outputFile

 Let's apply it to our DEM:

gdal_translate -outsize 15% 15% dem_rd.tif resized.tif
 Visualize the original image and the resized image in QGIS.
Explain what happened.

7.2 Change raster format

gdal_translate's primary function is to change between image formats. Resizing and changing the
image format can also be combined into one step by using both the -outsize and -of parameters.
The basic syntax is:

gdal_translate -of FORMAT inputFile outputFile

All supported formats can be found here: http://gdal.org/formats_list.html

Now we are going to convert the DEM from geoTiff to PCRaster format. PCRaster is open source
software for spatial dynamic modelling and has its own GIS format (http://pcraster.geo.uu.nl).

 Execute:
gdal_translate -of PCRaster -ot Float64 dem_rd.tif dem.map
 Visualize the result in QGIS

The -ot argument is needed to specifiy that it is continuous data, so PCRaster will interpret the map as
a scalar (continuous) data layer. Use -ot Int32 to convert to integer (thematic) maps. Use
-ot Byte to convert to a boolean PCRaster map.

16
8 Spatial queries of vectors
For our map of Delft we want to do the following GIS analysis:

 Select the community of Delft from the community map and save it into a new shapefile;
 Intersect the community boundaries of Delft with the road map of the Netherlands.

We can use a spatial query to select a feature from a vector map.

 What is the attribute in the community map containing the names of the communities? You can
use either ogrinfo or QGIS to answer this question.

 Execute the following command:

ogr2ogr -f "ESRI Shapefile" -where GM_NAAM='Delft'
-a_srs EPSG:28992 delft.shp gem_2011_gn1.shp

This will save the feature with GM_NAAM Delft to a new file called delft.shp. The argument -
a_srs EPSG:28992 is used to assign the Amersfoort/RD New projection to the output file. The
argument -f defines the output format.

 Now open in QGIS the reprojected DEM (dem_rd.tif), the reprojected road map
(roadsreprojected.shp) and the community of Delft (delft.shp).

17
9 Convert Comma Separated Files
Sometimes you want to reproject coordinates in an ASCII file, e.g. which has been saved in a
spreadsheet program. Here we will convert the coordinates in a comma separated ASCII file
(locations.csv) to a new ASCII file (locations_reprojected.csv).

 View the contents of locations.csv using notepad

 First, we have to create a virtual data source by creating an XML control file. On your windows
computer open notepad and type the XML code in the screenshot below. Use indentations of
three spaces.

 Save the file as "locations.vrt" in the gdalExercises folder.

Some explanation about the XML file:

 <OGRVRTLayer name="locations"> should correspond with the

<SrcDataSource>locations.csv</SrcDataSource>
 <GeometryField encoding="PointFromColumns" x="lon" y="lat"/>
indicates the columns with the coordinates that you want to convert.

 Execute the following command:

ogr2ogr -t_srs EPSG:28992 -f "CSV" locations_reprojected
locations.vrt -lco GEOMETRY=AS_XY

In this example locations.csv with lat/lon WGS-84 coordinates is converted to

locations_projected.csv with Amersfoort/RD New projection. Note that the file is saved in the
folder with the same name as the output file.

 Use notepad to check locations_reprojected.csv. What is saved in each column?

 In the same way we can convert the comma separated file to a shapefile. Execute:
ogr2ogr -f "ESRI Shapefile" locations.shp locations.vrt
 Visualize the shapefile in QGIS by plotting the locations over the DEM, road map and Delft
community border. Make a nice map.

 We can also convert the locations to Google Earth KML format. Type:
ogr2ogr -f KML locations.kml locations.vrt locations
18
 Note that you don't need to specify an output projection (-t_srs), because for KML this is
always WGS 84 (EPSG:4326).

 Visualize the results in Google Earth in Windows. If you don't have Google Earth yet, you can
download it from: http://www.google.com/earth/index.html

 Double click the file and it opens in Google Earth.

 What are the objects in our CSV file?

19
10 Batch conversion
Desktop GIS programmes are very useful for GIS operations, but are hard to use if we have to repeat the
same task for many GIS layers. Then scripting can be a solution.

Here we have an example dataset from a land-use model of Dublin. The data are in IDRISI raster format,
with one layer for each year between 1990 and 2030. Our task is to convert all layers to .tif format.

 Unzip landuse.zip to a folder called "landuse".

 Open the command prompt and "cd" to this folder.
 Make a batch file with the following commands:

Try to understand the code. This is a for loop that loops over all *.rst files in the folder. %%f is the
variable that contains the filename of each file. With echo we can print something to the screen.
Here we print %%~nf , which is the part of the filename before the dot that separates it from the
extension. Then we use the gdal_translate command with output format GeoTiff. At the end of
the line we add the .tif extension to the filename.

 Now you can execute the batch file and check the results.

20