TwXplorer helps make Twitter search more useful by helping users find and explore chatter about topics that interest them.

What it does

TwXplorer aims to make Twitter search more useful for journalists and other researchers by helping them identify and explore Twitter chatter about a particular topic.

The tool allows users to find tweets they’re interested in — just like regular Twitter search — but goes several steps further by showing users the most frequent terms, links and hashtags used in those tweets. TwXplorer helps users quickly see the focus and nature of specific discussions on Twitter.

Early users have used twXplorer to:

  • Discover unexpected, but relevant content.
  • Find hashtags to follow or add to tweets.
  • Refine searches through a “drill-down” approach.
  • Capture snapshots of the buzz on Twitter at a specific moment in time.
  • See what people on specific Twitter lists are talking about most.

TwXplorer began as a project in Northwestern’s Collaborative Innovations class. Read more about the original project here.

Read more about twXplorer in The Atlantic , The Buttry Diary, AllTwitter and Nieman Lab.

How it works

OAuth sign-in: Before using twXplorer, users sign in with their Twitter ID. This has several implications. First, it means Twitter will identify your search request as coming from you rather than twXplorer as a website, which means many people can use twXplorer concurrently without fear of running up against the limits of Twitter’s API. Second, it means we can access your Twitter lists for the twXplorer lists feature. And finally, it means we can associate saved searches with your Twitter account rather than making you create a separate twXplorer login.

Search by language: By default, the twXplorer search looks for tweets in the language associated with your Twitter profile. But you can search for tweets in 12 languages. Twitter offers “best-effort” language detection, which is not perfect but can help you find tweets written in those languages.

Find up to 500 tweets: To provide a relatively swift response, and comply with Twitter’s API limits, twXplorer finds the 500 most recent tweets that include your search terms, then does not display those that Twitter codes as retweets (“new style” retweets as opposed to those where the content is preceded by RT). If twXplorer reports finding 400 tweets, it means it found 400 unique tweets — which you can scroll through — and 100 “new style” retweets.

Zero in on the most relevant terms: TwXplorer excludes common words like “the.” Then it looks not only for single words, but also user mentions (such as @KnightLab) and hashtag text (#chicago counts as “chicago”). It also looks for “bigrams” (two-word phrases) that show up more than once. If a bigram is common (say, “white house”), twXplorer doesn’t count them again as separate terms.  TwXplorer groups terms together when they have a common stem (“look,” “looks” and “looking” are counted as the same term).

Count the terms: The number that appears next to any term, hashtag or link is the number of tweets that include that term. The counts include terms used in retweets, although the retweets are not all displayed.

Drill down: When you click on any term, hashtag or link in your search result, twXplorer returns only the subset of search results containing the term you clicked on. If you click on a second term, the subset of tweets is narrowed even further. For instance, if you filter separately by “chicago” and “bulls,” twXplorer will display search results only for tweets containing both terms.

Next Steps

TwXplorer is still a new project, but users have submitted a few requests for additional features, including:

  • An API
  • Sentiment analysis of relevant tweets
  • A better block list
  • The ability to capture and analyze specific time frames
  • The ability to filter or search by location

TwXplorer's list of top terms.


Live project

Blog posts and updates on the tool

Code repository

More about the student version of twXplorer

Initial Concept: Larry Birnbaum and Rich Gordon

Knight Lab Team:

Developer: Jennifer Wilson

Engineering Director: Joe Germuska

Design: Aaron Salmon , Rebecca Lai

Functional Design: Larry Birnbaum and Jessica Soberman

Initial Student Team: Allen Zeng (computer science), Jeanette Huang (computer science), Miguel Huerta (journalism)

Faculty Guidance: Larry Birnbaum, Rich Gordon, and Kris Hammond (with assistance from Shawn O’Banion)