1
Toponym Recognition in Historical Maps with
Area/Linear Feature Alignment
Toby Baratta and Bo Wang
Grinnell College, Grinnell, Iowa, USA
I. I NTRODUCTION
Maps reflect the geography of our Earth. Historical maps
display the changes in the natural surroundings, the
movement of people and the alteration of the Earth
surface that humans have accomplished. Historical maps
in digitalized forms are becoming more accessible to
scholars and the general public, however, the informa-
tion in these maps remains largely unsearchable. Very
limited number of historical maps are annotated with
geographical coordinates obtained by modern satellite
technologies and there is a good reason for this scarcity:
manually verifying and annotating toponyms (place
names) on historical maps is a tedious task. Unlike other
text files such as books, the location of strings on a map
is restrained by geography and cartographic methods,
thus highly unpredictable. Non-text elements, such as
boundary lines, roads, natural geographic features, cre-
ate significant noise for the traditional OCR techniques.
Yet a well-annotated and search-friendly database of
historical maps will save researchers hours of visually
scrutinizing old maps for a place name and direct them
to the geographic information of the certain place. In
order to build this database, we aim to create a program
which automatically extract and accurately recognize
toponyms on historical maps. Weinman’s work (2013)
on historical maps successfully improves single-point to-
ponym recognition by gazetteer alignment. In this paper,
we discuss our work of integrating area/linear features
into the existing system in order to expand the range
of geographic features that we can accurately recognize.
Our research question is whether incorporating shape
coordinates reduces the recognition error. Geospatial
vector data which specifies the shape and location of
lakes and rivers are added to the gazetteer. And we use
a modified joint probability model to calculate the proba-
bilities of possible toponyms with alignment between the
maps and the new gazetteer which includes area/linear
shape coordinates.
Part II review the related works and the cartographic
backgrounds of our research. Part III specifies the sys-
tem, task, and environment of our algorithm. Part IV
describes our experiments and analysis. Part V concludes
our project and leads to future research.
II. BACKGROUND
A. Related Works
Plentiful researches have been dedicated to automatic
text recognition and information integration on maps.
Gelernter, et al. aims to enrich the gazetteer with user-
geocoded data collected online and their research pro-
vides insight on correctly matching geographic features
and integrating lake/area shape information with the
original gazetteer. Chen et al. uses road intersection
points to conflate different geospatial datasets from var-
ious sources in order to identify the area which the
certain map section covers. However, these projects do
not concern historical maps and the integrated geo-
graphic information was not used to optimize toponym
recognition.
Weinman (2013) poposed a joint probability model
which recalculates the likelihood of each toponym can-
didate for all the strings found in the map, based on
the alignment between the map coordinates and the
projection of the global georectification. [?] Our project
follows the same idea, however the probability model is
modified to include non-point features.
Weinman’s paper from 2013 relates the most closely
to what we have done, as what we worked on mostly
was just an expanded version of his works. We added
onto his works by adding in the area features, updating
the gazetteer to the 2014 version, and working with
Weinman to devise a probability model to deal with area
and linear features.
B. Cartography
In order to recalculate the possibilities of toponym candi-
dates, we need to project the known geographic features
on the historical maps and measure the accordance
between the projection and the string location on the
map. As stated in Weinman (2013), the specific projection
of a given historical map is often missing, therefore
we choose the same projection family. For the sake of
controlling and comparison, we continue to use the
Hassler polyconic projection. [?]
Another issue to concern is the varying datums of the
Geographic Coordinate System (GCS) used by different
sources of data. A GCS is a system based on a datum
2
to define locations on the surface of the Earth. A da-
tum is defined by a reference ellipsoid which models
the shape of the Earth and the its relative position to
the center. All sources of data in our project use GCS
North American 1983 based on the NAD83 datum with
the GRS80 ellipsoid, therefore there is no need for us
to consider the possible errors resulted from different
reference systems. [?] The problem we face is that Matlab
does not specifically define any built-in GCS. Though
it does include various options of reference ellipsoids:
“grs80”, “wgs84”, etc. We choose the “grs80” ellipsoid
since NAD83 is based on this model. We assume that the
reference ellipsoid have properties which accord with the
NAD83 datum. Since Matlab uses longitude and latitude
to express location on the Earth, we also assume that
it uses the most common geocentric coordinate system.
Since we are not working on extremely high level of
accuracy, the assumptions above serve our purpose well.
C. Data
1. Maps
In order to compare our results with Weinman’s results,
we continue to use the manually annotated maps for
our experiments from the David Rumsey Historical Map
Collection. [1] The annotations are checked and im-
proved for our research with higher accuracy and more
complete coverage. From prior experiments, training and
testing data was already separated within this collection.
Then, we manually went through the testing database to
find maps that had a comparably large number of lakes
on them. These maps vary considerably in the accuracy
of the alignment, distribution of different lake shapes,
thus we think that this relatively small sample will give
us enough data to analyze the efficiency of our system.
[?] stated in his paper, which remains true throughout
our experiment:
For our experiments, we have annotated 12
maps from the David Rumsey collection [1],
with each map’s approximate region noted (i.e.,
the primary state or counties and any sur-
rounding states having toponyms represented).
A simple bound on coordinate ranges, as given
by some online map collections, would also
suffice to restrict the gazetteer ultimately used
for recognition. Annotations in a simple XML
format include tight piecewise linear baseline
guides, text height and ground truth string for
each toponym. 4 In addition to the regional map
shown in Figure 1, many are railroad and public
survey maps. The average text height is 25 ± 16
pixels.
Most maps in our collection are on the scale of one
single state; a few includes several states or only part
of one state. Notes on which states appear on a certain
map is used to restrict the gazetteer to only the states in
question. Names of lakes and rivers are annotated but
the corresponding map coordintes of these area/linear
features are only the primary points.
2. Gazetteer
In our experiment, we continue to use the Geographic
Names Information System (GNIS) from the U.S. Board
on Geographic Names (USBGN) as our authoritative
gazetteer. For our purpose, in addition to the categories
of Civil and Populated Place, we also include the cate-
gories of Lake, Swamp, Reservoir and Bay. The gazetteer
was just updated in 2014 and for our gazetteer creation
and posterior ranking we use that gazetteer. However,
our parse scores in all cases were done using the 2010
gazetteer dataset. We feel that if given more time to run
our tests with 2014 parses, we might see a slightly im-
proved difference in our resuts although further testing
would be needed to prove this.
However, the GNIS only contain primary single-point
coordinates for area/linear features. In order to an-
swer our research question, we need to link the data
of coordinates of the area/linear to the gazetteer. We
searched the open resources of geospatial vector data of
lakes/rivers on the Internet and found the following two
databases: The National Atlas [?] shapefiles of lakes and
the National Oceanic and Atmospheric Administration
[?] shapefiles of rivers. Among a number of different
resources, we choose these two databases because they
cover the largest number of water bodies, include suffi-
cient geographic information on the water bodies and
the shapefiles are easy to work with in Matlab. The
TNA and the NOAA datasets contains feature shape
information but they do not contain historical names of
lakes and rivers. Since we are working with historical
maps, it is important that we link the GNIS gazetteer and
the shapefile databases. We compare the edit distance
between the names of the lake/river in GNIS and in
TNA/NOAA as well as the geographic distance between
certain points on the features to match the water bodies.
Details are specified in [?]. With the linking algorithm,
we successfully matched 9531 lake shapefiles to the
gazetteer.
III. A PPROACH
A. Algorithm
1. System
Our probability model is based on the probability model
in Weinman (2013), which is modified to include the
area features. It is a Bayesian network. The difference
from the model before is that we include toponyms of
lakes here and thus integrate the area shapefiles into the
model.
The key variables - strings, points and alignment -
are retained in the model. Toponyms are assigned in
different branches of the model based on its category
in our gazetteer. Instead of using RANSAC in search for
an alignment to maximize the probability of a toponym,
we use the ground truth alignment obtained by manual

annotations of the maps. The probability model applies
to each word image and the corresponding image co-
ordinates found on a map. A flow chart explaining our
process can be seen below.
Figure 1. Flow Chart of our Experimental System
Given a map image of a string, the prior probability
of the recognition candidates P (y|W ) is given by the
OCR. Te initial steps of processing the prior probabilities
remains the same as in Weinman (2013). We assign
an outlier probability to non-geographical text such as
legends. The bias term γ (0 < γ < 1) is decided for each
map based on the alignment. The probability for inliers
is still a simple ratio with no bias. We also consider no
bias among the points where the geographic feature is
labeled.
In order to maximize the probability of the correct to-
ponym match, we recalculate the probabilities by finding
an alignment between the map and the real-life geo-
graphical data which we retrieve from existing database.
With the alignment, we calculate the inferred locations
of the toponyms on the map and further adjust the
likelihood of the toponyms by their distance from label
point p. The former model uses a Gaussian probability
to include the area around single point features by
considering a certain value of standard deviation (σ).
([?], 4)
The new new modified model assigns a uniform
likelihood within the shape of the lake and a normal
(gaussian) probability distribution around the lake with
a certain scale factor. The functions regarding this specif-
ically are found within [?].
2. Task
The initial input for the whole system is images of strings
on a map, which is not controlled by us, but by the
future users of our system. It should be clarified that this
image input is different from the information included in
a certain map. What the users decide is which particular
part of certain maps that they want to learn about. The
input for our part of the system is the prior probability
generated by the OCR. The OCR system gives a list of
possible recognition hypotheses for each string detected.
In later steps we only consider the hypotheses on this
list. With sufficient training data, the OCR parses the
strings with a geolexicon and a regular dictionary. The
3
geolexicon we use is also GNIS. Whether to include the
full geolexicon or the partial geolexicon with only civil
and populated places is a factor that could influence the
results of our experiment. We address this factor later in
experiments. An example of the prior probabilites would
be where if given an image with the string “Grinnell”,
the prior probabilities (parse scores) would be “Grinnell -
43%”, “Cornell - 52%”, and “Corn hell - 5%”. That would
be the pure output of the OCR model.
3. Environment
The environment is made up of the variables not con-
trolled in the system or as a task. For us, this would
include the geography of the Earth, the information on
the maps, and the speed of processing and computation.
The geography of the actual world is what we are using
to get our alignment for the re-ranking but is entirely
out of our system’s control. It is part of the environment
in which our system operates. The information on the
historical maps is out of our system’s control as well
as the user’s control; the data that is on the maps is
created by the mapmakers originally and thus is static.
The CPU and the amount of RAM that our computers are
equipped is another external effect. We strive to optimize
our code program by vectorization and simplification in
order to finish the calculation in a reasonable amount of
time.
IV. E XPERIMENTS
1 M ETHOD
We tested our system on eight test maps, each showing
a single state. These maps stayed consistent throughout
our several tests. We ran five cases over these maps:
limited parse, limited gazetteer, point feature treatment
only; limited parse, full gazeteer, point feature treatment
only; full parse, limited gazetteer, point feature treatment
only; full parse, full gazetteer, point feature treatment
only; and full parse, full gazetteer, point feature and area
feature treatment. These are detailed more below.
1.1 Parameters
Our system loads the coordinates of the lake centroid
points from the gazetteer and replace the point coor-
dinates with polygon coordinates for linked lakes. We
use the polyconic projection and an affine alignment
found by single control points to project the spherical
coordinates of the polygons into the image plane. The
inlier bias γ is set at 0.5 since we assume that the
distribution is normal.
Since map-makers through out history adapt different
geodetic standards, for example, different models of the
Earth, various sphere-to-plane projection algorithm, we
should expect error in the alignment and the projection
process between the historical maps and the contempo-
rary lake shape data. In prior research, the model defined
a Gaussian probability around the single point features
4
since the toponyms are not always found exactly over
the point coordinates. Similarly, we are not expecting to
always find lake names labeled inside the lake shapes,
so we need to take the distance from a lake shape to its
name label into consideration. Therefore we want to find
a reasonable scale factorbased on the above two factors
to assign probabilities around both the point coordinates
(σp ) and the lake polygon coordinates (σA ). This way, we
can avoid false negatives where the lake name is actually
pretty close to the lake shape so we will not falsely
penalize its probability if the name is not inside the lake.
Within each lake, the probability is kept uniform and the
joint probabilities get smaller and smaller as progressing
outwards. See Figure 2.
5060
5080
5100
5120
5140
5160
5180
5200
5220 Lake
4580 4600 4620 4640 4660 4680 4700 4720 Probabilities
4740 4760
Figure 2. This figure shows a visualization of the probabil-
ity model around Rainbow Lake in MIchigan. The colored
contour circles represent decreasing probabilities as they
get farther from the lake shape. The width between the
contour circles is decided by the scale factor δA of 35
pixels.
However, for our purpose in this experiment, we made
no change to the original code for the point feature
scale factors (σp ) , which means we only considered
the error from alignment for point features. For area
features, since all four training maps focus on displaying
the counties and major towns and not many lakes are
presented, we only looked at the distance from the
point that indicates the location of a town to the label
of the name of the town. We randomly selected ten
towns on each map and manually recorded the name-
point distance in pixels. For now, we simply evaluated
the arithmetic mean and median of the raw residual
distances from alignment and name-point distances and
decided to use a αA of 35 pixels. We realize that the
simplified approach for finding the scale factors may
have resulted in problems in our experiment.
1.2 Data Format
Our experiment relies on both the GNIS dataset updated
in 2014 and The National Atlas database of US Water-
bodies and Wetlands updated as well in 2014. The TNA
data contain lake shapefiles which consist of polygons
specified by the coordinates of the vertices. However, the
TNA databases does not include any historical names
and as we work on historical maps, we considered it
important to link these two datasets.
From our five testing experiments, you can see our
results below. For each testing case, we recorded the per-
cent word error, the percent character error, the number
of words in each gazetteer, and the harmonic mean of
correct words’ ranks below.
Table 1
−5
Key to the Tables
x 10
9
Number of Words in the Percent Word Error
8
Gazetteer
Percent Character Error Harmonic Means of Correct
Words’ Ranks
7
6 In Table2, we show the results of when we used a
limited gazetteer, a limited parse, and point treatment
5
only. Civil and Populated Place were included solely in
4
the gazetteer, with the same being used solely in the
parse scores. For the prior scores, only civil and popu-
3 lated places were allowed within the possibilities with
the same being used solely for the posterior probabilities.
2 All strings were treated as point features.
4780
1 Table 2
Limited Gazeteer, Limited Parse, Point Treatment Only
1801 58.19
41.04 1.930
In Table3, we show the results of when we used a
limited gazetteer, an unlimited parse, and point treat-
ment only. Civil and Populated Place were included
solely in the gazetteer, but within the parse scores, more
categories were used within the gazetteer . For the prior
scores, more possible entries were given into the system
then were given into the posterior system. All strings
were treated as point features.
Table 3
Limited Gazetteer, Unlimited Parse, Point Treatment Only
1801 57.85
40.90 1.924
In Table 4, we show the results of when we used an
unlimited gazetteer, a limited parse, and point treatment
only. Within the parse scores, only Civil and Populated
Place were used; that means that for the prior scores, less
entries were given then into the posterior system. For
the posterior system with the gazetteer, Reservoir, Lake,

Swamp, and Bay were added to Civil and Populated
Place to generate the possible toponyms. All strings were
treated as point features.
Table 4
Limited Parse, Full Gazetteer, Point Treatment Only
1879 57.16
39.62 1.900
In table 5, we show the results of when we used
an unlimited gazetteer, an unlimited parse, and point
treatment only. Within the parse scores, all categories
were used which means that for prior scores, a large
number of entries were given–around the same as in
the posterior system. For the posterior system with the
gazetteer, Reservoir, Lake, Swamp, and Bay were added
to Civil and Populated Place to generate the possible
toponyms. All strings were treated as point features. This
gives us a real ground truth to see how our area method
affects our results.
Table 5
Full Parse, Full Gazetteer, Point Treatment Only
1879 57.59
39.12 1.911
In table 6, we show the results of when we used
an unlimited gazetteer, an unlimited parse, and point
treatment only. Within the parse scores, all categories
were used which means that for prior scores, a large
number of entries were given–around the same as in
the posterior system. For the posterior system with the
gazetteer, Reservoir, Lake, Swamp, and Bay were added
to Civil and Populated Place to generate the possible
toponyms. However, in this case all area features were
not treated as point features. First, a gazetteer was
created specifically for the map in question of all feature
categories selected by the user. Next, we replace the
points for area features (lakes, swamps, reservoirs, bays)
within our gazetteer. Then, for point features (Civil and
Populated Places) we treat them the same as in all other
tables. For area features, however, we use the method
detailed in [?].
Table 6
Full Parse, Full Gazeteer, Point Treatment and Area
Feature Treatment
1879 57.81
39.53 1.917
In the table found at the top of the next page, per-
centage change is taken across each of the tables; the
equation for that is shown in Algorithm 1.
(100−testCaseP ercent)
Algorithm 1 %Change = 100 × (100−baseCaseP ercent)
5
Negative numbers show when our error went down,
while positive numbers reflect when our error went
up. Overall, you can see that our error did successfully
decrease with all of the changes to our system. However,
you can also see that in cases four and five, word error
went up. That means that adding in the full gazetteer
plus the full parse overall hurt the system more than
adding in just one or the other’s full data set. From that
we can conclude that there were too many extraneous
possibilities that were added into the system for it to
be able to accurately select the correct toponym. From
T4 to T5 we also see positive changes, where the only
difference is our new area feature model. From this, we
can see that our area model increased error.
V. C ONCLUSIONS
Moving forward, there are several changes to our system
that might improve our results. For one, using parse
scores for the 2014 GNIS data may improve our overall
scores by giving us a more accurate dataset to work with.
As the GNIS data changes every year and is updated
frequently to handle errors or multiple entries, we feel
that this new data would improve our accuracy. Also,
the 2014 data is what is used to form the gazetteer later
as well as was used in our entire linking process, thus
the change might make for spurious non-matches. In
addition, a better way to figuring out our scale factor
may increase our accuracy. Our current system uses both
the alignment on our training maps and the distance of
point features to their labels on those maps and there
may be a better way to generate that scale factor on a per-
map basis or a better way of incorporating the alignment
and the distance of point features.
In addition, doing area tests with limited gazetteers or
limited parses solely might show some interesting results
due to our conclusion about adding in both full gazetteer
and full parse caused more errors. Once we do have a
σA that is maximized we might find a better solution to
removing extraneous features from there and optimizing
that part of the system. This would result in a best case
scenario rather than a likely case scenario, but would
give us the full breadth of what our system could do.
Overall, we feel that our experiment concluded in
showing that very small amounts of change happened
throughout our system overall. However, if you sum
from table two to table five changes overall, all of our
changes did end in a final negative result—thus, error
did decrease overall. However, that seems to be more
from adding in the full gazetteer rather than our area
system at all unfortunately. Due to the dependency
on the parse scores and the scale factor, we feel that
optimizing the scale factor and updating our parse scores
could end up leading to an overall positive result in the
future.
R EFERENCES
[1] Cartography Associates. David Rumsey historical map collection.
6

Table 7
Table of Parses and Change Throughout

Table 1 Table 2 Table 3 Table 4 Table 5 Overall Change from T1 to T5

Word Error Percent Change - -0.8066 -1,6106 1.0139 0.5215 -0.008818
Character Error Percent Change - -0.2369 -2.1199 -0.8213 0.6780 -0.025001