Tag Archives: analytics

Teaching Students to Analyze Twitter data with Excel pivot tables: Social Media Analytics Assignment (Post 2 of 4)

In my last post, I discussed a new assignment that I’m using this semester in my Communication research class (all posts on that class).

That social media analytics project assignment contains 3 parts. Each part is related but unique, allowing students to pick up a new skill set. In this post, post 2 of 4 in the series I’m writing about this assignment, we’ll discuss part 1 of the assignment. If you haven’t read the assignment overview post, I encourage you to do so before proceeding. There you will see a copy of the assignment discussed in the below post.

Part 1 of the assignment asks student teams to analyze the Twitter data provided by their clients by creating pivot tables in Microsoft Excel.

social media analytics pviot tables excel Twitter data

If you aren’t familiar with pivot tables, they enable you to filter and visualize spreadsheets. This allows you to focus in on specific data points and quickly extract insights from large data sets.

I got the inspiration to create this part of the assignment from a very helpful conversation I had with Professor Stefanie Moore at Kent State University. A big thank you to Stefanie for taking the time to chat with me and for providing me with insights to how she teaches analytics. I am really impressed and inspired by what Professor Moore is doing at Kent State.

Preparation: Getting Twitter Data
In order to analyze Twitter data using pivot tables in Excel, you need to first download Tweets from Twitter’s analytics (ads) page. If you’ve never done this before, it is really quite easy.

The reason we use Twitter is because Twitter enables you to extract a ton of valuable account data from your account in the form of a CSV spreadsheet. But, as an aside, you could analyze just about any data with pivot tables.

My students were required to get the Twitter data from a client. Therefore, I created a step-by-step guide that they could provide to the client so that the client could extract the appropriate data and supply it to me.

To ensure we had enough data, I instructed the students to ensure that their client was posting at least a few times per week. I asked students to get 6 months of Twitter data if possible. In short, I wanted to ensure that there were at least 50 Tweets from the client in the time period we collected. This number is somewhat arbitrary. And ideally you’d like to have more. But, 50 Tweets is enough to sort and play with.

Here are the steps for extracting Twitter data from an account:

Step 1:  log into your organization’s Twitter account at http://twitter.com. Next, select your account profile picture (as shown below) and select “Analytics.”


Step 2: A new window will appear. Click “Tweets” from the menu at the top. Then, select the date range (see below). A menu will open. Please select a date range of at least 3 to 6 months back so that there are enough Tweets for the students to analyze.
Important: Click “Update” to change the selected date range.
In the below example, I selected Feb 1 through May 1 (3 months).

Step 3: Once the dates have been selected, click “export data.” A new window will appear. Click “save file” to save the file to your computer. Email that file (it should be a .CSV file named something starting with: “tweet_activity_metrics…”). You have your data. If someone else is downloading the data – such as a class client – , they will need to email the file to you or your student.

Using Pivot Tables to Analyze Twitter Data

A few days were set aside in class to work with the pivot tables and learn how to answer the questions students were asked to answer in the project. On day 1, I provided a brief lecture  (about 10 minutes). And then I instructed students to begin working with the lab guide I had created. If you’re a longtime reader of this blog, you know I am big on creating lab guides to assist students in learning software.

See the lab guide students used to learn to analyze their Twitter data using pivot tables: http://bit.ly/435_pivottableslab

While working with the lab guide, students were to have a copy of the assignment that contained the research questions they needed to answer using the pivot tables. Those research questions were:

  1. Which Twitter posts received the most (Fill in the blank – you need to decide what variables are important engagement data for your client. You’ll need more than 1 variable. And, you’ll want to show more than just the top Tweet for that variable, but the top few)?
  2. What is the client’s Twitter engagement by month? (again, you choose the appropriate engagement metrics)
  3. Come up with 1 other RQs for important data points you extract from your pivot table analysis that you believe will be of value to your client.

For the above questions, students needed to pick what engagement metrics they wanted to analyze. There are several engagement metrics in the CSV file when you download it from Twitter. Examples include retweets and favorites.

For research question #3, most groups analyzed engagement by Tweet category. As you’ll see in the lab guide, students learned how to comb through their Tweets and identify common themes by which to categorize their Tweets. Examples may include promotional Tweets, humorous Tweets, Tweets that ask a question, etc.

The above 3 research questions are just a sampling of what you could do with the pivot tables.

In Summary

In the next post, we will discuss part 2 of this assignment which gets students using Microsoft Social Engagement to answer some research questions about their client.  I will be publishing that post in 2 weeks.

In the meantime, if you want to get your feet wet, I encourage you to download your own Twitter data and walk through the lab guide above. Or, check out some of the sources listed below to learn how to analyze Twitter data with pivot tables.

As you will see when you take a look at the lab guide, you must first clean the data so that Excel can analyze it. I then walk you through a number of different ways you can analyze your Twitter data.

The fact is that I was a bit of a newbie to pivot tables when I created this assignment. To build the above-discussed lab guide I provided students to help them through learning how to use pivot tables, I relied heavily on several key resources. Much of what is in the lab guide is built directly on what I learned from these sources. To learn directly from the sources I learned from, check out the sources below. A big thank you to all of them for sharing their knowledge publicly. I hope I was able to honor them in adapting their work for a classroom assignment.

Update: You can now read the follow up posts to this blog series.

Sources:

 

The New Social Media Analytics Assignment for my Comm Research Class (Post 1 of 4)

A few months ago I wrote about how students in my social media class were using Microsoft Social Engagement to track metrics and do some social listening. At the time, I said I’d follow up with a post about how we were using the software in my communication research class. Well, the time has come! But, this post will do more than dive into how we are using Microsoft Engagement in my class. It will share with you a whole new project my research students are doing.

This is post #1 in a 4 part series on a new assignment my students are working on in my communication research class. The assignment spreads over several weeks with a good amount of time in class working in the computer lab. The project is the result of continued and ongoing efforts I’ve been making in a few classes to enhance student education in social media analytics. The project replaces the sentiment analysis assignment I wrote about a few years ago.

This post will cover an overview of the assignment (A copy of the assignment is below). Post #2 will discuss using pivot tables to analyze Twitter data. Post #3 will discuss Microsoft Social Engagement. Post #4 will discuss Netlyitic.

Update: Post #2 on pivot tables is now available, as is Post #3 on MS Engagement and Post #4 on Netlytic.

First, let me provide some context. In my communication research class (see all posts related to the class), students work in teams to complete 3 projects. Each project gets progressively more difficult. The project we are going to discuss today is project #2.

Overview of Social Media Analytics Project for A Client

The purpose of the assignment is for students to get experience performing a social media analytics audit of a client using a variety of social media analytics and social network analysis tools. The goal is for the students to try and understand their client’s current use of social media and provide insights and recommendations for enhancing that client’s social media presence.

Each team was tasked with going out and finding a client that would agree to participate. While I had hoped that most groups would approach local businesses, they tended to focus more on on-campus groups like athletic teams. This may have been a result of convenience because each team had to acquire several months worth of Twitter data from their client. I will explain that in further detail when we discuss pivot tables in post #2. So students tended to go to on campus organizations where they already knew who ran the Twitter account.

The three main components of the project are:

  1. Client Social Media Profile & Engagement Analysis
    1. Students use Pivot Tables to explore your client’s posts on social media and analyze their overall engagement. For example, students determine the top posts by their client which made that have gotten the most likes.
  2. Analyzing Trends
    1. Students use Microsoft Social Engagement to monitor and analyze the conversation surrounding the client’s brand.
  3. Social Network Analysis
    1. Students use Netlytic.com to build visual representations of their client’s social network on Twitter or Instagram and do some basic analysis.

For each component, I have created a set of research questions that students answer using the appropriate software. The students adapt the research questions a bit to their context when necessary. You can see the research questions in the assignment below.

The Plan in the Classroom

On day 1, I provide a 10 minute lecture on pivot tables. The rest of the class is a lab for students to work on learning how to create pivot tables to analyze Twitter data and answer the RQs.

On day 2, I give a 20 minute lecture about the social engagement software and talk a little about sentiment analysis so students understand what it is when they look at it in the Microsoft software.

Day 3 is a lab day to work on whatever they weren’t able to get done in the pivot tables or the social engagement software.

On day 4, I lecture about social network analysis and some basic concepts. (We cover some other material this day about writing research papers).

On day 5, we finish talking about social network analysis – about 15 minutes – and the students analyze their client’s data.

Research Write Up

After students complete all 3 parts of the project, they then have to write up their study. The research paper format I use in this class is inspired by Don Stacks book, Primer in Public Relations Research.

In the past, by the second project students are writing brief literature reviews. However, because this is the first time I’ve run this project and it has been a lot of work, I called an audible and removed the requirement for the lit review in this project. So, you will see in the assignment below that those requirements have been withheld.

Thus, by the second project students have been taught about writing research problem overviews (problem statement, campaign goals & objectives, research objective & RQs/hypotheses), methods, results and discussion sections.

The students write up their reports. And they are encouraged to share them with their client.

Limitations & Final Thoughts

There are a few drawbacks I’ve experienced thus far with this project.

First, there is a lot of info coming at the students with this project. The assignment sheet itself is several pages long. As such, it is important to explain things several times and work with the students as they are doing this project.

Students need to be responsible for getting the data for this project from their client, creating their own Netlytic account and setting it up to collect data. And, they need to provide me with who their client is and some competitors of the client far enough in advance that I can program it into Microsoft Social Engagement (I’ll go into more depth on this in the individual posts about each section). We had a few groups that made mistakes along the way and were short on data or had to do some last minute scrambling.

The data collection periods across the Twitter CSV file, the Microsoft Social Engagement and the Netlytic are not consistent. This is simply a result of the classroom setting and a lack of full control over when data collection happens. For example, a team’s client may have sent their Twitter data which covers the last 6 months one day, a teammate set up Netlytic to collect data another day, and the day I set up the Microsoft Social Engagement to collect data on their client on a third day.

With these another limitations in mind, the project has been fun thus far this semester. A major benefit of this assignment is that most of the tools used in this assignment are free or inexpensive and not too difficult to learn (and thus teach your students).

Over the next few posts, I will offer some depth on each section of the project.  So check back soon! For now, you can get a copy of the assignment below.

Update: You can now read the follow up posts to this blog series.

An Assignment and Spreadsheet for Teaching Students to Track Social Media Metrics in my Social Media Class (Post 1 of 2)

In the social media education community, there has been a lot of discussion about teaching social media metrics and analytics to students. This has been a challenge and frustration for myself and many others. Access to industry tools is cost prohibitive for many universities, making it difficult for us as educators to prepare our students for this aspect of their careers.

I’ve worked hard over the last few years to try and enhance how I’m teaching these concepts. And I’m not where I want to be. But I know there are many fellow educators also on this journey with me. So, I’d like to share how I teach students to track social media metrics as part of a semester long assignment and a few modifications I have recently made to enhance that aspect of my teaching.

I’ve split this topic into two blog posts for length purposes. In both of these posts, we’ll focus on my social media class (2016 syllabus; and all articles about this class). In this post, we’ll talk about the spreadsheet for tracking metrics. In the follow up post, we’ll discuss Microsoft Social Engagement and how I integrate it into the metrics assignment portion of the class.

Update: The follow up post on Microsoft Social Engagement is now available.

My aim in my social media class is to introduce metrics to students both in lecture & discuss (which I’ve been doing for some years) as well as by use of software. Then, when students get into the Communication Research class (2015 syllabus; articles about this class), they will get more in-depth learning about analytics. I’ve increased/improved my focus on this area in that class for next spring. And my long term hope is to really build that part of the class out. During the upcoming spring semester, I will write a blog post about what we will be doing with analytics. And, at that time, I will share all of my assignments and handouts.

Okay, back to my social media class. In past years we’ve used Twitter Analytics – which has been the best, free tool. Unfortunately, other platforms have been limited in their analytics. We’ve used a slew of free tools that have been here today, gone tomorrow.

This year, we still faced the challenges of relying on Twitter Analytics and whatever free tools we could find. But I also added a brief introduction to Microsoft Social Engagement (which will be discussed in the next post in this series).

But first, let’s discuss how I teach students to track performance metrics in my social media class.

In my social media class, students are divided into teams. Each team is in charge of running a social media platform for our department’s social media. In the past, I had my students use a spreadsheet developed by Jeremy Floyd to track metrics. At the time, I modified the spreadsheet for our purposes. At the start of this semester I modified the spreadsheet further simplify it and to add a section on Microsoft Social Engagementƒ (again, which I will discuss in the next blog post).

Here is a copy of the spreadsheet as it was distributed in my Fall 2016 class which you can use in following along with the below post. You can also download a copy for yourself to modify and use as you prefer. Again, credit goes to Jeremy Floyd for the original incarnation of this spreadsheet.

In lecture, I teach students about the activity, engagement and performance metrics discussed in Kim’s book, Social Media Campaigns: Strategies for Public Relations and Marketing. I also emphasize the importance of choosing metrics that are tied to goals. (You’ll see a tab in the spreadsheet discussed below, where students are to determine their objectives and what metrics would be important to those objectives).

Student teams begin with the planning tab, then they establish their metrics goals to use the spreadsheet to establish benchmarks and KPIs for their platform and track metrics over the semester. They then move over to reporting tab to track weekly metrics.

Tip. You can see tips by mousing over the small triangles in the upper right corner of some cells, as shown below. I’ve created these to help students when working on their spreadsheets in groups.

In the image below, you can see the ‘reporting’ tab of this spreadsheet. We start tracking in week 9 of the semester, but you can modify this as you like. After each week, you’ll see the percentage change. Of course, you can also modify what you are tracking. I throw in a number of potential metrics to track for different platforms. But, students can delete all the rows they don’t need and modify the individual metrics for that platform as needed. The metrics identified in the spreadsheet are just a guide.

I’ve also divided the spreadsheet up into different platforms so each team can pick their platform (as shown in image below) for tracking the success of their posts. The idea here, is that by tracking these posts across time, students can begin to analyze these metrics for trends (though, I don’t have any ways to quickly analyze and visualize this data at this time). This could help them learn when the best time to post is. However, you could also add variables about the post that can help them identify which is the type of content that is most successful. Other spreadsheets I’ve seen track variables such as whether an image was used, what hashtags are used, if links are used, etc. So, again, you can modify the optimization section as you see fit. I discuss other variables to track, but focus on the ones in this spreadsheet so as to not overwhelm students. I’ve found if I ask students to track too many things, nothing gets tracked as they get overwhelmed. So choose what you want them to track, and stick with it.

I’ve relied on Kim’s metrics categories for metrics students can track. Also, please know the metrics I have identified isn’t perfect and modification of what I’ve identified may be needed – some of my initial metrics may not work, or changes have occurred.

Integrating The Metrics Into the Semester-Long Assignment

As noted above,  across the entire semester of my social media class, students are strategizing, building and executing social media for my class. As a part of that, they present their content to the class for approval at intervals throughout the semester.  In the latter half of the semester, the students present their current metrics to the class alongside the content they are proposing for the next content time period. At the end of the semester, we discuss their metrics, whether they met their KPIs and during what week they did, and what they learned from them.

While the above enables us to track interaction with our social content and extract some insights, it doesn’t account for listening to competitors, following trends, etc. It also doesn’t take deeper analytics and the extraction of insights into consideration. We don’t do anything to plot or discern specific insights – I am saving that for the Communication Research class this spring. Said another way, the assignment and use of this spreadsheet in my social media class, as I executed it in Fall 2016, was really more about tracking metrics, following change and teaching students  to see the impact (outcome) of their efforts on social media, while connecting those back to objectives and KPIs.

In the next blog post, I go into the “social listening” tab of the spreadsheet and discuss how students got a little hands on use with Microsoft Social Engagement in my social media class during fall 2016.

In the meantime, if you have any thoughts or suggestions or resources you’d like to share about teaching metrics to students, please share them with me and the readers via a comment in the post or Tweet me. This is an important journey for all of us as we work to enhance hands-on metrics learning for our students.

I hope you found this post helpful. If you did, please share it. It helps a lot.

-Cheers!
Matt

#Hokies Tweets Network Visualization: How I extracted Tweets via TAGS 6 and visualized them in Gephi

Click to see larger or download.
Click to see larger or download.

A professional development goal of mine is to learn a lot more about social network analysis and visualization of social media data. This area has grown increasingly valuable and important in our field.  And I believe we all need to have at least a base knowledge of social data and how to play with it.

With my wife traveling for work and rainy weather here in West Virginia, this weekend presented a great opportunity to finally get my feat wet (no pun intended).

As you may know, my beloved Virginia Tech Hokies haven’t been playing so well this college football season.  So I decided to use Saturday’s game as an opportunity to play with Twitter data and Gephi, an open source data visualization program.

I’ll explain what I did below to make the above visualization in case you’d like to try this for yourself. This is a simple approach and I think you’ll find you can do it if I can learn it in a weekend! I started Saturday morning with zero knowledge of graph theory, social network analysis, how to use Gephi, and how to pull down Tweets.

I’m writing this up because I found several tutorials online. But, none of them quite came together to show me how to do all the parts in one tutorial. A major reason is that the Twitter API has changed since many tutorials available online were built. So, the ways offered for getting the Twitter data on those tutorials no longer works.  As such, getting Twitter data is a challenge if you don’t know a little programming with Python, etc (Needless to say, I don’t).

Fortunately, each of the tools together below made this first experiment in Twitter data visualization possible.

Here’s how I did it:

1) I used the TAGS v.6 Twitter Archiving Tool to gather Tweets with the hashtag #hokies. This is an amazing, free tool – thank you so much to Martin Hawksey for this! You can learn to use the TAGS archiver fairly easily via Google Docs. The only real slow down is that you have to get a Twitter API key via your Twitter account.

I ended up gathering 1583 Tweets between 3:19am – after midnight before the game – and the majority of the way through the game at 2:43. So, whatever Tweets going back I could pull when I extracted the data at 2:43; not a great picture of the #Hokies conversation, but it worked for this exercise.

2) I used @DFeelon’s spreadsheet converter to convert the TAGS spreadsheet to a file I could put into GEPHI to do the visualization. Thanks Deen!

His converter pulls only the first Twitter account that is mentioned in the Tweet or in a RT – so any additional persons mentioned in a Tweet were not counted. You can learn more about it here on Deen’s blog. It is easy to use. In short, I copied my Tweeter and Tweet text into his spreadsheet, and voila! This created my edge file in CSV for GEPHI with 2 columns (vertices, or nodes) – the first column being the person who sent the Tweet and the second column being the person to whom the Tweet was directed.

3) I noticed that some mentions of Twitter account handles were all lowercase whereas others were not. This had created duplicate nodes. That is, in some instances, one Twitter account had been split into two: an all lowercase version and the original. So, I simply made all text lowercase to address this problem. I used Google Refine to clean my CSV file because I want to learn to use this program. But, you could change the case in Excel or any spreadsheet software.

4) I then loaded the cleaned CSV file into Gephi (download it here) so I could do the visualization.

5) I spent a lot of time on Saturday reading about visualization and getting a basic knowledge of graph theory and how to use Gephi. While I’ve still got a lot to learn, I decided to follow a tutorial for my first “go round.” It seemed like a great opportunity to put together concepts and tools in Gephi that I’d learned in a guided environment. So, I followed the instructions on the latter half of this YouTube video for how to visualize the data and export it into the file you see with this post. The tutorial is by Michael Bauer via the International Journalism Festival. Of note, the first half shows you how to extract data using Twitter’s old API and that process no longer works. So you can take your CSV file gained through the process above, import it into Gephi, and pick up with the tutorial at 1:05:46.

So, that’s it!

A few quick things about this visualization:

As indicated by the size of the Twitter account name, we can see that Virginia Tech sports beat writer Andy Bitter for the Roanoke Times had the largest number of Tweets directed at him regarding the game (that is, his node – his Twitter account – had the most degrees. The degrees are the number of edges, or connections one node has to another). This makes sense. I’ve followed the #Hokies conversation on Twitter for years and Andy has been a constant presence and leader in providing news and analysis of Tech.

The communities are indicated in colors. I used the modularity script in Gephi to identify these, as is shown in the above-noted YouTube video. In short, you can use the color coding to make a basic clustering of who is talking to who.

In Closing:

While I’ve got a ton to learn, I’m thrilled with the progress I’ve made in just over a weekend from not knowing the first thing about graph theory, basic spreadsheet formatting for nodes and edges, or how to visualize a social network, to building my first visualization. And, while my goal is not to become a data scientist, I am excited to continue to learn and grow a base knowledge in this area. I know I am just scraping the tip of the iceberg.

I’d love to hear your thoughts and tips on how I can improve my knowledge and skills! Also, please feel free to share your tips, tutorials, and experiences with social data.

 

Note: Thanks to Nathan Carpenter at the ISU SMACC for helping me get started with data gathering and visualization by generously sharing his experiences and tools!