Background

 Earth is 500 Tera-meters square  Someday

 USA is 10 tm2  multi-spectral
 100 TM2 land in 70ºN to 70ºS image
 of everywhere
 We have pictures of 6% of it
 once a day / hour
 3 tsm from USGS
 2 tsm from Russian Space Agency 1.8x1.2 km2 tile

 Compress 5:1 (JPEG) to 1.5 TB. 10x15 km2 thumbnail

 Slice into 10 KB chunks
20x30 km2 browse image
 Store chunks in DB
 Navigate with 40x60 km2 jump image
 Encarta™ Atlas
 globe
 gazetteer
 StreetsPlus™ in the USA
 USGS Digital Ortho Quads
(DOQ)
 US Geologic Survey
 3 TeraBytes
 Most data not yet published
 Based on a CRADA
 TerraServer makes data available.
1x1 meter
4 TB
Continental
US
New Data
Coming

USGS “DOQ”
Russian Space Agency(SovInfomSputnik)

SPIN-2 (Aerial Images is Worldwide Distributor)

 1.5 Meter Geo Rectified imagery of (almost) anywhere

 Almost equal-area projection
 De-classified satellite photos (from 200 KM),
 More data coming (1 m)


Want to sell imagery on Internet.
Putting 2 tm2 onto TerraServer.
SPIN-2
 Demo

http://www.TerraServer.com

Microsoft

BackOffice
SPIN-2
 Hardware

STC E n te r p r is e S t o r a g e A r r a y
A lp h a
9740 48 48 48 48 48 48 48
DLT Server 9 GB 9 GB 9 GB
9 GB 9 GB 9 GB 9 GB
T ape D r iv e s D r iv e s D r iv e s 8400 D r iv e s D r iv e s D r iv e s D r iv e s
L ib r a r y 8 x 440M H z
A lp h a c p u s
10 G B D R A M
M a p S it e
100 M bps S erver
E t h e r n e t S w it c h DS3 In te r n e t S ervers

1TB Database Server

AlphaServer 8400 4x400. 10 GB RAM SPIN-2

324 StorageWorks disks

10 drive tape library (STC Timber Wolf DLT7000 )
 Software
Web Client
Image Internet Java
Server Information HTML
Viewer
Active Server Pages Server 4.0
broswer
MTS
The Internet

Terra-Server
Stored Procedures
Internet Info Internet Information
Server 4.0 Server 4.0
Sphinx Microsoft Automap Microsoft
(SQL Server) ActiveX Server Site Server EE

Terra-Server DB Automap Server Image Delivery SQL Server

Application 7
Terra-Server Web Site
Image Provider Site(s)
 System
Management &
Maintenance
 Backup and Recovery
 STC 9717 Tape robot
 Legato NetWorker™
 Sphinx Backup/Restore Utility
 Clocked at 80 MBps!!
 SQL Server Enterprise Mgr
 DBA Maintenance
 SQL Performance Monitor
TerraServer File Group Layout
 Convert 324 disks to 28 RAID5 sets
plus 28 spare drives
 Make 4 NT volumes (RAID 50)
595 GB per volume
 Build 30 20GB files on each volume
 DB is File Group of 120 files
HSZ70 B HSZ70 B HSZ70 B HSZ70 B HSZ70 B HSZ70 B HSZ70 B

HSZ70 A HSZ70 A HSZ70 A HSZ70 A HSZ70 A HSZ70 A HSZ70 A

E: F: G: H:
 Gazetteer Design
 Classic Snowflake Schema
 Fast First hint to Optimizer
PlaceGrid

ZGridID
CountrySearch Place
BestPlaceName
Country XDistance
AlternateName PlaceID
YDistrance
CountryID ImageFlag
CountryID
GazSrcID AlternateName 50,000,000
CountryName
Name
1148 UNcode
CountryID FeatureType
StateID
264
TypeID TypeID
StateSerach
State GazSourcID Description
AlternateName Latitude
CountryID StateID Longitude 13
StateID CountryID UGridID
ZGridID GazetteerSource
FreatureID StateName
GazSrcID DOQdate
1083 GazSrcID
SPIN2date
3776 Description
1,089,897 1
 Image Data Design
 Image pyramid stored in DBMS (250 M recs)
Pick TileMeta
ImageMeta Log UGridHits
OriginalMetaData
Name URL ImgMetaID
OrigMetaID ImgMetaID URL OrigMetaID
Description Time
SrcID OrigMetaID UGridID SrcID
Link <extensive
ImageSource ImgStatus ZTileGridID ImgStatus
PickDate list of action
Agency ImgDate count ImgDate
parameters
SourcePhotoID ImgTypeID ImgTypeID
JumpPixHeight 10 xxx
SourcePhotoDate TilePixHeight
JumpPixWidth xxx
SourceDEMDate TilePixWidth
MetaDataDate BrowsePixHeight CutCol
ProductionSystem BrowsePixWidth CutRow
ProductionDate ThumbPixWidth MidLat
DataFileSize ThumbPixHeight MidLong
Compression CutCol NELat
HeaderBytes CutRow NELong
… MidLat NWLat
MidLong Jump Browse Thumb Tile
80 other fields NWLong
NELat SELat
NELong UGridID UGridID UGridID UGridID SELong
NWLat ZGridID ZGridID ZGridID ZGridID SWLat
NWLong ZTileGridID ZTileGridID ZTileGridID ZTileGridID SWLong
650 k SPIN2 SELat ImgData ImgData ImgData ImgData UGridID
ImgDate ImgDate ImgDate
1
ImgDate
2 M USGS SELong UTMZone
SWLat ImgTypeID ImgTypeID ImgTypeID ImgTypeID XUtmID
ImgSource SWLong ImgMetaID ImgMetaID ImgMetaID ImgMetaID YUtmID
UGridID SrcID SrcID SrcID SrcID XGridID
ImgType EncryptKey EncryptKey EncryptKey EncryptKey
SrcID UTMZone YGridID
SrcName XUtmID File Name File Name File Name File Name ZGridID
ImgTypeID
SrcTblName YUtmID
ImgFileDesc .65 M SPIN2 .65 M SPIN2 .65 M SPIN2 16 M SPIN2
SrcDescription XGridID 16 M SPIN2
ImgFileExt 1.5 M USGS 1.5 M USGS 1.5 M USGS 96 M USGS
GridSysID YGridID 96 M USGS
MimeStr
ImgTypeID ZGridID

4
2 650 k SPIN2
2 M USGS
 Image Delivery and Load
DLT
Tape
“tar”
Wait 4
\Drop’N’ DoJob LoadMgr
Load
DLT NT DB
Tape Backup

LoadMgr LoadMgr ...

ESA
Alpha Alpha
60 10: ImgCutter
Server 4.3 GB Server ImgCutter
20: Partition

4100 Drives 4100 30: ThumbImg

100mbit 40: BrowseImg
EtherSwitch 45: JumpImg
\Drop’N’
50: TileImg
\Images 55: Meta Data
60: Tile Meta
70: Img Meta
80: Update Place

Enterprise Storage Array

STC Alpha
108 108 108
DLT 9.1 GB 9.1 GB Server
9.1 GB
Tape Drives Drives Drives 8400
Library
 Terra-Server Tables
 USGS DOQ Data
 48,000 DOQ images are input (45-55mb / image)
 Creates 864,000 Jump, Thumb, & Browse images (3.5 m rows)
 Creates 55.3 m Tile images (110.6 m rows)
 SPIN-2 Data
 3200 278 MB images are input (approximate size)
 Creates 620,800 Jump, Thumb, & Browse images (2.5 m rows)
 Creates 15.5 m Tile images (31 m rows)
 Gazetteer Data
 1.1 m named places (Encarta World Atlas)
 45 m cell names
 Total Rows = 228 M today (and growing fast)
 Other Details
 Active Server pages
 faster and easier than DB stored procedures.
 Commerce Server is interesting
 Images the Inventory
 no SKU,
 millions of them
 USGS built their own
 they are very smart, but it is easy
 masquerade as a credit-card reader.
 The earth is a geoid, and
 Every Geographer has a coordinate system (or two).
 Tapes are still a nightmare.
 Everyone is a UI expert.
 SQL 7 Testimonial
 We started using it March 4 1997
 SQL 7 Pre-Alpha
 SQL 7 Alpha
 SLQ 7 Beta 1
 SQL 7 Beta
 Loaded the DB twice
 (we made application mistakes)
 Now doing it “right”
 Reliability: Great! SQL 7 never lost data
 Ease of use: Great!
 Functionality: Great!
 RAGS:
RAndom SQL test Generator
 Microsoft spends a LOT of money on testing.
 Idea: test SQL by
 generating random correct queries
 executing queries against database
 compare results with SQL 6.5, DB2, Oracle, Sybase
 Being used in SQL 7.0 testing.
 Very productive test tool
 SELECT TOP 3 T1.royalty , T0.price , "Apr 15 1996 10:23AM" , T0.notes
FROM titles T0, roysched T1

Sample
WHERE EXISTS (
SELECT DISTINCT TOP 9 $3.11 , "Apr 15 1996 10:23AM" , T0.advance , (
"<v3``VF;" +(( UPPER(((T2.ord_num +"22\}0G3" )+T2.ord_num ))+("{1FL6t15m" +
RTRIM( UPPER((T1.title_id +((("MlV=Cf1kA" +"GS?" )+T2.payterms )+T2.payterms
))))))+(T2.ord_num +RTRIM((LTRIM((T2.title_id +T2.stor_id ))+"2" ))))),

Rags
T0.advance , (((-(T2.qty ))/(1.0 ))+(((-(-(-1 )))+( DEGREES(T2.qty )))-(-((
-4 )-(-(T2.qty ))))))+(-(-1 ))
FROM sales T2
WHERE EXISTS (
SELECT "fQDs" , T2.ord_date , AVG ((-(7 ))/(1 )), MAX (DISTINCT

Generated
-1 ), LTRIM("0I=L601]H" ), ("jQ\" +((( MAX(T3.phone )+ MAX((RTRIM( UPPER(
T5.stor_name ))+((("<" +"9n0yN" )+ UPPER("c" ))+T3.zip ))))+T2.payterms )+
MAX("\?" )))
FROM authors T3, roysched T4, stores T5
WHERE EXISTS (

Statement
SELECT DISTINCT TOP 5 LTRIM(T6.state )
FROM stores T6
WHERE ( (-(-(5 )))>= T4.royalty ) AND (( (
( LOWER( UPPER((("9W8W>kOa" +
T6.stor_address )+"{P~" ))))!= ANY (

SELECT TOP 2 LOWER(( UPPER("B9{WIX" )+"J" ))

FROM roysched T7
WHERE ( EXISTS (

SELECT (T8.city +(T9.pub_id +((">" +T10.country )+

UPPER( LOWER(T10.city))))), T7.lorange ,
((T7.lorange )*((T7.lorange )%(-2 )))/((-5 )-(-2.0 ))
This statement yields an error (now fixed) FROM publishers T8, pub_info T9, publishers T10
WHERE ( (-10 )<= POWER((T7.royalty )/(T7.lorange ),1))
AND (-1.0 BETWEEN (-9.0 ) AND (POWER(-9.0 ,0.0)) )
)
--EOQ
) AND (NOT (EXISTS (
SELECT MIN (T9.i3 )
FROM roysched T8, d2 T9, stores T10
WHERE ( (T10.city + LOWER(T10.stor_id )) BETWEEN (("QNu@WI" +T10.stor_id
)) AND ("DT" ) ) AND ("R|J|" BETWEEN ( LOWER(T10.zip )) AND (LTRIM(
UPPER(LTRIM( LOWER(("_\tk`d" +T8.title_id )))))) )
GROUP BY T9.i3, T8.royalty, T9.i3
HAVING -1.0 BETWEEN (SUM (-( SIGN(-(T8.royalty ))))) AND (COUNT(*))
)
--EOQ
) )
)
--EOQ
) AND (((("i|Uv=" +T6.stor_id )+T6.state )+T6.city ) BETWEEN ((((T6.zip +(
UPPER(("ec4L}rP^<" +((LTRIM(T6.stor_name )+"fax<" )+("5adWhS" +T6.zip ))))
+T6.city ))+"" )+"?>_0:Wi" )) AND (T6.zip ) ) ) AND (T4.lorange BETWEEN (
3 ) AND (-(8 )) ) )
)
--EOQ
GROUP BY ( LOWER(((T3.address +T5.stor_address )+REVERSE((T5.stor_id +LTRIM(
T5.stor_address )))))+ LOWER((((";z^~tO5I" +"" )+("X3FN=" +(REVERSE((RTRIM(
LTRIM((("kwU" +"wyn_S@y" )+(REVERSE(( UPPER(LTRIM("u2C[" ))+T4.title_id ))+(
RTRIM(("s" +"1X" ))+ UPPER((REVERSE(T3.address )+T5.stor_name )))))))+
"6CRtdD" ))+"j?]=k" )))+T3.phone ))), T5.city, T5.stor_address
)
--EOQ
ORDER BY 1, 6, 5
)
 Always UP: System pairs
 Two clusters at two sites
 Changes from one
sent to other
 When one site fails
other provides service
 Masks
 Hardware/Software faults
 Operations tasks (reorganize, upgrade
move
 Environmental faults (power fail)
 Outline

 Highlights of MS
Research
 DB-Related Research