Unit 5 Lab 2: Data and Functions, Page 1

Working With Large Data Sets

EKs covered: 3.1.2F, 3.2.1A, 3.2.1B, 3.2.1C, 3.2.2A, 3.2.2B, 3.2.2C, 3.2.2D, 3.2.2G

What about data with millions of pieces of information, instead of a few hundred? Large data sets present challenges and opportunities for discovering new information.

Using a Web browser, open the Baby Name Voyager. This visualization shows the 1000 most popular names of boys and girls born in the United States for every year from 1880 to 2014.

If the graph becomes unresponsive or blank, reload it.

What was the most popular girl's name in the 1900s? In the 1960s?
What boys' names are much less popular today than they were in 1880?
Type in what you think is the most popular name in your school. Is this name still popular for new babies?
What else can you find? Find some interesting information in the data, then prepare to show it to your class.

The Baby Name Voyager is an impressive visualization of a large data set. This data comes from the Social Security Administration, a text file for each year from 1880 to 2014. Very few of the insights in this data would be learned just from reading these files!

Large data sets present unique challenges and opportunities:

Large data sets can be complex or nearly unmanageable. Storing, processing, and editing large data sets is difficult.
Because of their size and complexity, large data sets can be difficult to usefully analyze. Computing power is frequently necessary for this analysis, providing access to trends or connections that would otherwise not be seen.
One way in which computing power can be found for a massive computation is crowdsourcing, in which people all over the Internet are asked to volunteer the time that their personal computers would otherwise spend doing nothing, e.g., while the person is asleep. Examples include SETI @ Home and Folding @ Home.
Some large data sets are built by human collaboration, and can include text, sound, images, and video.
Wikipedia, YouTube, and Twitter are all large data sets! Twitter's list of Trends is generated by analyzing this data as people generate it.
Collaborative analysis is very useful when working with large data sets, often resulting in new information that could not be learned by an individual.
Working with large data sets is often about making connections and identifying trends.
Large data sets often contain personally identifiable information, such as names, locations, email addresses, and passwords. It can be difficult to work with large data sets while protecting personal privacy.

Visualizations and interactive tools are especially valuable Running Data in NYC when working with large data sets, giving people the opportunity to study what might otherwise be incomprehensible. This map from YesYesNo was generated from runners contributing their tracking data.

Work with the Social Security birth data to produce a visualization of your own. You can start with this file, with other years' data available here to download.
Think of a large data source you've produced, and visualize it using Snap!. Remember, large data sets can include text, sound, images, and video.