Archives
How To Mine Your GMail with Google Takeout and MongoDB
Posted on February 14, 2014 Leave a Comment
Google has really been on the up-and-up lately with a service called Google Takeout that allows you to export your data from its cloud. For the thoughtful cloud user who is becoming increasingly concerned about privacy, accidental data loss, or data ownership, this is a product that’s sure to please. Likewise, for the data mining enthusiast, quantified-self […]
Understanding the Reaction to Amazon Prime Air (Or: Tapping Twitter’s Firehose for Fun and Profit with pandas)
Posted on December 19, 2013 2 Comments
On Cyber Monday eve, Jeff Bezos appeared in a 60 Minutes segment and revealed to the world that he’s been working on an experimental effort called Amazon Prime Air. The general idea behind Amazon Prime Air is that Amazon may one day deliver relatively lightweight items directly to your doorstep in less than 30 minutes […]
What Do Tim O’Reilly, Lady Gaga, and Marissa Mayer All Have In Common?
Posted on November 22, 2013 4 Comments
This post examines the followers of some popular Twitter users as the final installment of a multi-part series about exploring Twitter influence by asking the (Freakonomics-inspired) question, What do Tim O’Reilly, Lady Gaga, and Marissa Mayer all have in common? Although it may initially seem like an obnoxious question to ask, some of the answers may intrigue you […]
Super Simple Storage for Social Web Data with MongoDB (Computing Twitter Influence, Part 4)
Posted on November 20, 2013 3 Comments
In the last few posts for this series on computing twitter influence, we’ve reviewed some of the considerations in calculating a base metric for influence and how to acquire the necessary data to begin analysis. This post finishes up all of the prerequisite machinery before the real data science fun begins by introducing MongoDB as a […]
An Approximate Solution for TL;DR [~50 Year Old Text Summarization Hack Presented as a ~1.7MB Animated GIF]
Posted on November 13, 2013 1 Comment
Suffering from information overload? Too much TL;DR happening in your life? Attention span just isn’t what it used to be? Watch this short ~30 second screencast (a ~1.7MB animated GIF) that demonstrates a 50+ year old hack for summarizing news articles and other types of online content. After all, it seemed fitting that the presentation of a […]
Getting Started with Twitter’s API: From Zero to Firehose in ~2.5 Minutes
Posted on November 12, 2013 4 Comments
Mining the Social Web‘s goal is to teach you how to transform curiosity into insight, and its virtual machine features two IPython Notebooks that are designed to get you up and running with Twitter’s API as quickly as possible. The following ~2.5 minute screencast shows how to generate OAuth credentials, establish a Twitter API connection, and make API […]
Mining Social Web APIs with IPython Notebook [Slides]
Posted on October 30, 2013 1 Comment
Thanks so much to everyone who attended the Mining Social Web APIs with IPython Notebook workshop. It was really inspiring to see so many of you get your hands dirty hacking on data (as opposed to just talking or thinking about it.) It’s a lot of work to design a 3 hour workshop for such […]
Now Serving: Full-Text Sampler in IPython Notebook Format
Posted on October 25, 2013 3 Comments
The 2nd Edition of Mining the Social Web has officially soft-launched (the “hard-launch is at my Strata workshop next week), and as of late last week you could download either a PDF file or view an ebook excerpt of the first chapter that introduces data mining with Twitter’s API. Additionally, as of just a few hours ago, the full-text of the first […]
How To Harvest Millions of Twitter Profiles Without Violating the ToS (Computing Twitter Influence, Part 3)
Posted on October 22, 2013 1 Comment
In the last post in this continuing series on computing Twitter influence, we developed a wrapper function called make_twitter_request that handles the various sorts of HTTP error codes and network failures that you are likely to experience as you aspire to acquire non-trivial amounts of data from Twitter’s API. Although you are somewhat unlikely to […]
Why Is Twitter All the Rage?
Posted on October 9, 2013 4 Comments
Next week, I’ll be presenting a short webcast entitled Why Twitter Is All the Rage: A Data Miner’s Perspective that is loosely adapted from material that appears early in Mining the Social Web (2nd Ed). Given that the webcast is now less than a week away, I wanted to share out the content that inspired the topic. This […]
Mining the Social Web 