Exploring Public APIs with MongoDB and Analytica

Nosh Petigara nosh@analytica.com @noshp

Today • MongoDB and public APIs • What is Analytica? • Demo – Analytica shell (twitter data) – Analytica for Excel (StackOverflow data) .

MongoDB and public APIs • Most APIs talk JSON – MongoDB’s native JSON import • APIs vary wildly (internally and between one another) – MongoDB is schema-free • Data import is only half the battle – MongoDB’s query language and aggregation framework .

stackexchange.Some data sets to explore • Twitter API (JSON) – https://dev. XML) – http://developer.crunchbase.twitter.nytimes.com/docs .com/ • Crunchbase API (JSON) – http://developer.com/ • NYTimes (JSON.com/ • Stackoverflow (JSON and CSV) – http://data.

csv .jso n –uUSERNAME:PASSword| ./mongoimport –d twitter –c tweets • Importing JSON files – .json • CSV – ./mongoimport –d db –c coll --type csv -headerline myfile.twitter.Importing data sets • Streaming JSON directly into MongoDB – curl https://stream./mongoimport –d mydb –c mycoll file.com/1/statuses/sample.

com] .Analytica • Analytics & reporting platform for MongoDB – Natively understands JSON/document hierarchy – Tailored for analytics (not querying) – Works directly on MongoDB • Discovery. analysis. visualization cycle • In private beta [http://analytica.

What can you do with Analytica? • Inspect and extract data • Augment your data model • Calculate & aggregate • Filter and transform data • Join collections .

Demos • Today – Twitter stats [using the Analytica Shell] – Stackoverflow community analysis [using Analytica for Excel] • Not shown – REST API – Analytica web (Coming soon) .

Demo 1: Twitter data .

followers_count) Popular hashtags set twitter.populartags.tweetcount = count(tweets) set twitter.tweets.where(retweet_count <> 0))) Follower counts max(twitter.hashtags.tweets.Some other examples Tweets vs. replies count(select(twitter.byhashtag = group(tweets.text)) set twitter.byhashtag.text .user.populartags = orderdesc(byhashtag. retweets vs.by(entities.by(tweetcount)) get twitter.

Demo 2: StackOverflow User Profiles .

com or info@analytica.com • @analytica_inc on twitter .com • Get in touch – nosh@analytica.Next steps • Private beta – http://analytica.

Sign up to vote on this title
UsefulNot useful