Fetch is a privacy web application that complies social media data on individuals.
What it does
Fetch is a social media data miner that pulls information from a person’s various social media accounts and compiles the results into an easy-to-digest profile. Looking beyond the obvious data points like a person’s follower count or registration date, Fetch aims to find meaning from a social media profiles. For example, if somebody tweets a link to a song they like, Fetch can make a Spotify playlist for them. Fetch also gathers their most popular posts across Instagram and Twitter, displays a map of all the places that they have tagged themselves, and more. Fetch provides a small glimpse into a person—their history, interests and personality—based on their online presence.
Fetch can be used as a tool for developing greater digital-self-understanding as well as learning more about others in preparation for social or professional events.
How it works
In order to generate a profile Fetch completes the following steps:
- A user begins to type a person’s name into the Fetch homepage. After six characters have been entered, typeahead kicks in and begins to search Twitter for users with the same name along with their corresponding Twitter handles. As the user types more characters, the Twitter typeahead refreshes with more accurate options.
- The user then selects the right full name and Twitter username combination and hits enter. Now all the fun stuff starts on the server.
- The server takes the full name and Twitter username and checks whether the profile has been cached on the server. If a cached version is found, the cache file is loaded. Otherwise, the Twitter API is requested for recent tweets.
- The tweets are iterated through once, gathering all images, links, hashtags, locations, and follower/following counts.
- Now that the Twitter information has been gathered, we use the Instagram API and run our follower/following name matching algorithm to find the most-likely Instagram user for the given Twitter user.
- Once the Instagram user has been found, we use the Instagram API to gather images.
- We then display the most common hashtags, most popular images from Twitter and Instagram on a view. The view also displays a map with pins on the different locations that we parsed from Instagram and Twitter, and Spotify playlists and Youtube videos from Twitter.
- Twitter API
- Instagram API
- Spotify API
- Youtube API
- Google Geocoding API
While we were to amass plenty of data through Twitter and Instagram APIs, figuring out how to find and pull this information from our database as well as present it was a challenge. We hit a few road bumps along the way, most notably Twitter’s rate limit and the June 1, 2016 invalidation of our Instagram API token.
With more time we’d like to mine the text of the tweets and Instagram for more information about a person, including the places they may have been, their interests and sentiment on given topics. This would require a better natural language processing API. We’d also like to expand into other social media sites, such as LinkedIn.