Class Projects - Fall 2014


Twitter List management web application

This is "Funnelist Demo" by Northwestern U. Knight Lab on Vimeo, the home for high quality videos and the people who love them.
What it does

Funnelist is a Twitter List management web application for journalists usings lists on Twitter. It is useful to any journalist who uses a Twitter list to aggregate a subset of tweets around a certain topic. Although it was designed with journalists in mind, anyone who enjoys the list functionality provided by Twitter is able to better manage their lists with Funnelist’s one-click user addition/deletion functionality, user suggestions, and metric-based sorting functionality.

All of these features take away the pain points that journalists experience when managing lists through the Twitter client, and allows them to spend less time grooming their lists and more time digesting the tweets that have been aggregated for them.

How it works

When a user inputs her Twitter username and the name of one of her lists, Funnelist takes them and immediately begins building a series of requests to make to the Twitter REST API and our MongoDB. It makes an API call asking for all the current members of the list and passes that off to a function that can calculate various metrics for the users. We chose to highlight follower/following ratio, verified status and tweeting frequency as metrics to show to the user, but other pieces of information are available.

It also updates the MongoDB with the latest tweets from that list’s timeline and fetches that and other tweets from the database (to take advantage of multiple queries that may be done in a given time period). With this compiled dataset of tweets, Funnelist builds a list of suggestions by seeing which users currently not on the list are mentioned within the user timeline. It does all of this in parallel when possible to take advantage of Javascript’s asynchronous loading. Finally, it passes along this compiled information through the Mustache templating engine to create a personalized page that can be presented statically to the user.

From there, the user has the ability to execute multiple actions. We added a front-end sorting algorithm so that the user could see which members of their list performed well for certain metrics, and also added a feature where they could type in various keywords to see which users are mentioning certain topics. Under the suggestions column, Funnelist can show the user the tweets that suggested list candidates are mentioned in. It also provides list management tools to the user - current list members can be removed and suggested list candidates can be added with the click of a button, which makes a POST request to the Twitter API and updates the list.

Key Technologies

  • Node.js
  • Using the Express web application framework
  • Modules: async, express, body-parser, hogan-express (for Mustache), mongoose, morgan, twit
  • Heroku Cloud Platform Service
  • MongoLab add-on for implementing MongoDB
  • Twitter REST API (accessed through ‘twit’ module)
  • Mustache templating engine
  • Bootstrap CSS Framework

Next Steps

While our product gives journalists a new layer of list management capabilities that extend the offerings Twitter currently provides, we have a few enhancements in mind for the future. An obvious improvement would be incorporating OAuth so that anyone could easily use Funnelist with a one click login system.

Our current version requires a user to have their Twitter API key and other credentials readily available. Also, our system was limited by the Twitter API’s rate limiting. If we had access to the “firehose”, we would be able to provide more useful user suggestions, enhanced metrics, and a larger set of tweets to display.

Improvements to Funnelist’s interface and user experience could make the product even more intuitive, making a journalist’s life easier. While we chose to give the list management power to the user in this prototype, a fully-realized version of Funnelist may feature passive list management functionality that automatically adds and removes users without requiring the user to do anything.

Architecture Diagram for Funnelist


Student Team:

Aashish Prabhav Chalasani

Clayton Brooks Gentry

David Ryan

Matthew Scott Schley

Faculty Guidance:

Rich Gordon

Larry Birnbaum