Firehose Data Science: Real-Time Analytics of Twitter Feeds

Invited Talk | Day 3 | 9:20 am | 40 Minute Duration | Grand Gallery E/F
  • David Corliss
    Ford Motor Company, Manufacturing Forecasting Data Scientist

Firehose Data Science: Real-Time Analytics of Twitter Feeds

Invited Talk | Day 3 | 9:20 am | 40 Minute Duration | Grand Gallery E/F
The application of data science methods to social media feeds presents many challenges, including the creation of APIs to access the input stream, working with big data – especially high volume and velocity – and real-time decisioning. This paper demonstrates accessing Twitter feeds using a Java API, applying machine learning methods in SAS and R to classify tweets, and using these data to support further statistical analysis. The example used in the presentation involves mining the Twitter “firehose” for hate speech with a decision tree for classification, plotting and mapping the time series in real-time, and investigating the hypothesized connection between hate speech and acts of violence against persons targeted in the speech.