Archives

What Do Tim O’Reilly, Lady Gaga, and Marissa Mayer All Have In Common?

lady-gaga-nerd-glasses

This post examines the followers of some popular Twitter users as the final installment of a multi-part series about exploring Twitter influence by asking the (Freakonomics-inspired) question, What do Tim O’Reilly, Lady Gaga, and Marissa Mayer all have in common? Although it may initially seem like an obnoxious question to ask, some of the answers may intrigue you […]

Read More

Super Simple Storage for Social Web Data with MongoDB (Computing Twitter Influence, Part 4)

Icon_MongoDB_by_xkneo

In the last few posts for this series on computing twitter influence, we’ve reviewed some of the considerations in calculating a base metric for influence and how to acquire the necessary data to begin analysis. This post finishes up all of the prerequisite machinery before the real data science fun begins by introducing MongoDB as a […]

Read More

How to Deliver a Successful Tech Workshop with Vagrant and AWS

the-cloud-dropbox

At Strata, I delivered workshop called Mining the Social Web with IPython Notebook, and in order to ensure that the workshop would meet its objectives and be a smashing success, I knew that a few constraints had to be considered: Everyone must be able to follow along with the examples. (The goal of the workshop is […]

Read More

An Approximate Solution for TL;DR [~50 Year Old Text Summarization Hack Presented as a ~1.7MB Animated GIF]

hp-luhn

Suffering from information overload? Too much TL;DR happening in your life? Attention span just isn’t what it used to be? Watch this short ~30 second screencast (a ~1.7MB animated GIF) that demonstrates a 50+ year old hack for summarizing news articles and other types of online content. After all, it seemed fitting that the presentation of a […]

Read More

Getting Started with Twitter’s API: From Zero to Firehose in ~2.5 Minutes

7df3h38zabcvjylnyfe3

Mining the Social Web‘s goal is to teach you how to transform curiosity into insight, and its virtual machine features two IPython Notebooks that are designed to get you up and running with Twitter’s API as quickly as possible. The following ~2.5 minute screencast shows how to generate OAuth credentials, establish a Twitter API connection, and make API […]

Read More

Mining Social Web APIs with IPython Notebook [Slides]

Mining Social Web APIs with IPython Notebook

Thanks so much to everyone who attended the Mining Social Web APIs with IPython Notebook workshop. It was really inspiring to see so many of you get your hands dirty hacking on data (as opposed to just talking or thinking about it.) It’s a lot of work to design a 3 hour workshop for such […]

Read More

Now Serving: Full-Text Sampler in IPython Notebook Format

serving-small

The 2nd Edition of Mining the Social Web has officially soft-launched (the “hard-launch is at my Strata workshop next week), and as of late last week you could download either a PDF file or view an ebook excerpt of the first chapter that introduces data mining with Twitter’s API. Additionally, as of just a few hours ago, the full-text of the first […]

Read More

How To Harvest Millions of Twitter Profiles Without Violating the ToS (Computing Twitter Influence, Part 3)

dataharvest46_460

In the last post in this continuing series on computing Twitter influence, we developed a wrapper function called make_twitter_request that handles the various sorts of HTTP error codes and network failures that you are likely to experience as you aspire to acquire non-trivial amounts of data from Twitter’s API. Although you are somewhat unlikely to […]

Read More

Writing Paranoid Code (Computing Twitter Influence, Part 2)

fail-whale

In the previous post of this series, we aspired to compute the influence of a Twitter account and explored some relevant variables to arriving at a base metric. This post continues the conversation by presenting some sample code for making “reliable” requests to Twitter’s API to facilitate the data collection process. Given a Twitter screen […]

Read More

Arriving at a Base Influence Metric (Computing Twitter Influence, Part 1)

influence

This post introduces a series that explores the problem of approximating a Twitter account’s influence. With the ubiquity of social media and its effects on everything from how we shop to how we vote at the polls, it’s critical that we be able to employ reasonably accurate and well-understood measurements for approximating influence from social media […]

Read More