Data RSS - Sample Scenario

Pito Salas - rps@salas.com - March 23 2009

Introduction and Background
In a previous note, I described at a high level the vision for “Data RSS”. In a future note I will describe it in some more technical detail. Here I would like to lay out a scenario to illustrate how I imagine this to work and indeed why I am excited about this direction. Data RSS is a simple standard format for publishing and accessing data sets over Rest APIs. It’s a recipe that can be followed to make one or more data sets available over the web. It’s a recipe that can be followed to write applications, widgets and web sites who use data sets published by others. It is a fully decentralized approach where the two parties, publishers and accessors, do not have to coordinate with each other. Each can work independently of the other because they are following a standard “plug and socket” scheme. For our purposes, a data set is a table of information, numeric, textual, dates and times and currencies. A data set can be small (for example, the names and phone numbers of all US Senators). A data set can be huge (for example, the list of all purchase orders issued by City Hall over a period of time.) Data RSS is about enabling a kind of ecology of data publishers and accessors who don't need to coordinate with each other, so that when for example the DOX starts publishing data, existing widgets and reporting tools, and notification managers, etc etc can all immediately be hooked in. The effect is that, for example, the DOX doesn't feel like, why did I bother writing this whole REST API for my data, now I have to go convince people to use it.

Scenario
How would it work? Lets take a hypothetical example. Apps for America developer billnotify.org creates a notification tool so that a user can ask to receive an email whenever a piece of legislation is sponsored by a congressman from some state which has some word in the title, the user gets an email. Simple idea, assuming there's an API where the titles and states of authorship of a piece of legislation is available. There happens to be such an API. GovTrack.us provides an API which returns information about bills in congress, including what congressperson sponsored it, and the name of the bill. So billnotify.org has a reasonably straightforward approach: 1. Every day, call the GovTrack.us API to see what new bills have been offered. 2. Look up the sponsoring congress person and their state 3. Notify any billnotify.org users if a bill matching their criteria have been offered.

Data RSS - Sample Scenario

Pito Salas - rps@salas.com - March 23 2009

Now let's say the State Department makes travel advisories available through another REST API. They would like to allow someone to set up an email notification whenever an advisory is published for a certain state. Sounds familiar? The State Department might note that there’s already a service, billnotify.org,  that does this for legislation. They can either try and convince billnotify.org to broaden their service to include travel advisories, or get someone to create a travelnotify.org service. And now, with Data RSS Imagine instead, that billnotify.org talks Data RSS. Let’s rename it to be just mynotify.org. Here is how the world would look: • Both the state department and govtrack.us would be publishing their information in Data RSS format -- http://datarss.govtrack.us and http://datarss.state.gov. • mynotify.org would ask the user what information they wanted to be notified about, and they would enter one of those two urls. • mynotify.or would access the corresponding sites, and display pick lists for the user to indicate that they wanted to be notified about a bill or alternatively about a travel advisory. N.B. Tomorrow, when the FDA started delivering food recalls as Data RSS, the user would be able to start getting notifications about those as well without any coordination with the makers of mynotify.org.

Conclusion
You might say, well this is what the whole REST API efforts are for. But with the Data RSS scheme we achieve a decoupling between the ones creating and publishing the data from the ones accessing it. In the story above, mynotify.org didn’t need to help or participate or change to allow the FDA food recalls. This is a huge friction remover. Both publishers and users get payback for their efforts without having to evangelize and get custom code written by others. What this is and is not By the way, this is specifically NOT about a central database. Just the opposite. It's about decentralization and decoupling. Data RSS is ‘simply’ a standard and simple format for describing and delivering data sets. The magic is in the fact that it is a neutral standard designed to cover 90% of our needs. Clearly there will be more complex data models or parts of data models that are too complex or specialized to work under a generic Data RSS scheme. But I am really convinced that a huge amount of leverage and coverage could be achieved within a relatively simple model.