You are on page 1of 50

Neodata:

Neodata: From Crowdsourcing To Cloudsourcing And Where It Will Take Us Christopher Osborne I was feeling

From Crowdsourcing To Cloudsourcing And Where It Will Take Us

Christopher Osborne

I was feeling a little mischevious and threw in as many buzzwords into this presentation title as possible. I even made up a few, but I will be talking about how data is changing and the new ways we are trying to visualise it.

About Me

In order to understand my perspective you need to know a little about where I come from

Taken by @sigizmund

#geomob - gmdlondon.ning.com

I run a bimonthly geoweb/neogeography meetup called #geomob, attended by lots of internet startups where we discuss the latest developments in web mapping and location technology

I actually started o ! as a geographer, with proper paper maps and everything

I actually started o ! as a geographer, with proper paper maps and everything

A turning point was when I became a web GIS developer at Brent Council, and

A turning point was when I became a web GIS developer at Brent Council, and realised there was a better way to present maps online, and started doing web development in my free time

I then moved to Transport for London, where I redesigned their data infrastructure. It was

I then moved to Transport for London, where I redesigned their data infrastructure. It was there that I became fascinated with the idea of sensor networks. They manage a huge infrastructure network of roads, railways, assets such as buses and trains, and there are sensors all over the network. Induction loops under the road that measure the amount of tra " c passing above, Oyster

card transactions, air quality sensors, GPS on buses, cameras that scan licence plate numbers across London

A huge network of sensors

Unfortunately all of this data was in silos and going to waste. I designed a data architecture to make all the data available in real time and spatially referenced so that real-time information can be used to make decisions influencing the road network

Neogeography

MASHUP:

{data source A + data source B} * remix = new service

Cloud is my database

Open data is innovation driver

service Cloud is my database Open data is innovation driver Lots of talk of neogeography and

Lots of talk of neogeography and mashups. A mashup is just a remix of several different data sources into one new service, and we have seen a huge number of map mashups being produced as it gets . This is wonderful, we have people able to quickly and easily create web mapping, but as you may have seen a lot of cartographically poor mashups. Its important to distinguish between the quick and dirty mashups and the professional polished mashups. A map of my holiday snaps doesn’t need much design or cartography but to convey a large amount of information you need to pay attention to cartography and visualisation. When we talk about neogeography I define it as a state of mind, the internet is my database, a huge nebulous entity of millions of atoms that forms what we call the cloud The key thing here is that the availability of open data or data that is ‘open enough’ that enables innovation to take place at zero or low cost

So talking about doing “proper cartography” in mashups, here is one that I made: “Where Can I Live?” A tool to show you locations in London within a convenient commute time from your place of work, within your budget. You specify your work location, how long you wish to travel and how much you can afford for the type of property you want to buy. We tried really hard to show just the right level of detail on the map, and keep the cartography in keeping with the style of the site, we layer on tube lines and visualise the locations with a little clock to show you how long it takes to travel there. We also graph house price data and display individual property listings This is what a professional mashup should look like, paying attention to cartography and style

This is a simple example of a typical mashup architecture, its simple, lightweight, open source,

This is a simple example of a typical mashup architecture, its simple, lightweight, open source, but the main distinction is that I don’t own any of this data. The cloud is my database and I pull in data from wherever I want and display it in a web browser.

So doing all this work in Web2.0 and working for a large government body I

So doing all this work in Web2.0 and working for a large government body I learnt a lot about architecture and data. You start with a huge amount of different systems and data sources, each one designed for a specific task

Government / Enterprise

Unfortunately this is what government and enterprise IT systems looks like, a huge greasy mess of spaghetti. Its virtually impossible to track down what systems do what let alone provide real-time data and information.

Web 2.0

Whereas web2.0 is all about speed, lightweight architectures that allow data to flow at near real-time speeds I was lucky that Keir Clarke introduced mashups and some real time mapping yesterday, I’m going to talk about the challenges of visualising huge amounts of real-time data

Current Web Mapping

Tiles, slippy maps. Amazing five years ago and you can throw some data on top but its pre-rendered and you can’t change any of the mapping

OpenStreetMap gives you access to the underlying data and you can create your own beautiful maps. OpenCycleMap is the classic example: features relevant to cyclists are highlighted, contours to show relief, POIs such as pubs

Cartagen is vector rendering of OpenStreetMap data in the browser. No plugin and no pre-rendered tiles, its early days and is a bit rough and ready but the map can change according to user interaction

OS Licence

2.2.4 no output or display from the Consumer Website Service contains Licensed Data in Vector Format or allows Licensed Data in Vector Format to be reconstructed

None of this can happen with Ordnance Survey data, the licence is too restrictive

Crowdsourced Geodata OpenStreetMap

At Ito World we use and contribute to OpenStreetMap

You can see just how much wider and deeper the data coverage is becoming

You can see just how much wider and deeper the data coverage is becoming

The UK and Europe are now very well mapped, particularly the Netherlands etc thanks to

The UK and Europe are now very well mapped, particularly the Netherlands etc thanks to the contribution of commercial AND data

And its happening all over the world

And its happening all over the world

We made this wonderful visualisation of all the OpenStreetMap edits for 2008, gives an idea of the power of OSM

A Map = Data x Visualisation

the basics, this hasn’t changed but the data we use is

I’m showing this because the man who created those streams in OpenStreetMap, Steve Chilton, is

I’m showing this because the man who created those streams in OpenStreetMap, Steve Chilton, is sitting right there.

We are working on wonderful ways of rendering maps

We are working on wonderful ways of rendering maps

Data is Changing

Visualisation is Changing

Maps are Changing

So the data we use is changing, we need to visualise it di! erently and we are creating new maps

Because of this, the internet. This is a visualisation of the blogosphere, you can see these large bubbles are blogs with a lot of readers and links back to them. You see how this is all connected

You come back a bit and you start to see the formation of a galaxy or cloud. Important - this is all 2-way, web2.0 is not just receiving a document from the web it is a constant exchange of data. (Twitter, blogs, comments, geotagged photos, facebook,

Neodata:

Big

Cloudsourced

Real-Time (ish)

Great example of the way a simple web service to share photographs has evolved with user input and is now a huge source of metadata

http://code.flickr.com/blog/ Flickr Alphashapes - using geotagged photos to create polygons for country and area

Flickr Alphashapes - using geotagged photos to create polygons for country and area boundaries. Psychogeographical space

Imagine what happens when we have billions of geotagged photos

Cloudsourcing - The Sensor Web

this is just a flippant term I coined to describe the idea of huge amounts of data that you can pull from the cloud. We are now seeing the start of the sensor web

we are becoming connected to the web everywhere. You are sending data back whenever you

we are becoming connected to the web everywhere. You are sending data back whenever you interact with your location enabled smartphone

We hold the equivalent of the tricorder in our hands right now

We hold the equivalent of the tricorder in our hands right now

“I want to invent a tricorder for planet earth” - Mike Liebhold

http://tonchidot.com/

We Are All Sensors Now

http://www.citysense.com/ Citysense creates heatmaps and shows you the ‘hot’ areas of the city where people

Citysense creates heatmaps and shows you the ‘hot’ areas of the city where people are, showing you where all the popular bars and nightlife areas are. An old visualisation applied to real-time data

Not the best visualisation but if you use Google Maps then you are sending back

Not the best visualisation but if you use Google Maps then you are sending back data about the speed of your movement, used to generate tra " c speed data

Self Organising Maps

So how do we visualise huge amounts of real-time data - one way is to use a self organising map

http://senseable.mit.edu/nyte/visuals.html

http://senseable.mit.edu/nyte/visuals.html

http://neworleans.citymurmur.org /

http://neworleans.citymurmur.org/

I blog www.cloudsourced.com

I tweet @osbornec