Tag Archives: communication research class

Here’s My Communication Research Class Assignment on Analyzing Media Placement

In my opening post to the Spring 2018 semester, I reviewed several new assignments and activities I will be bringing into my classes this semester.

In this post, I will discuss the quantitative content analysis assignment that students will complete in my COMM 435 Communication Research course. The project simulates an analysis of earned media placement.

[Here are all the posts about that class, including discussion of past projects].

Background: About the Class

My aim with the communication research class is to offer our students experience learning about a variety of approaches to conducting research. My goal is to mix methodology (i.e., the study of research methods) with practical applications that students may run into in their careers. This course is not a graduate school prep course. It is designed for students who are planning to go into industry.

I think a struggle that many of us have is that there is a sense that we need to prepare students to be able to do the “new and cool stuff” (e.g., social media data analysis) in the research class, while balancing educating students about the research process, research ethics, designing measurements, building methods, gathering data, analyzing data, interpreting data, etc.

Unfortunately, we only have this one research class in our department as we are a small department serving a wide array of needs. I know that some other universities have advanced research or analytics courses. Thus, for me, I need to fit in both emerging methods and what some might see as traditional, evergreen methods:  content analysis, surveys, focus groups, interviews.

With this in mind, my students complete 3 assignments in the class, with each assignment focusing on a different topic. Each assignment is situated in a hypothetical, but plausible situation. I present the situation to the students via the assignment, and then we go through the steps of learning how to solve the problem put forth in the assignment. The assignments explore:

  1. Content analysis of media artifacts (in the form of an analysis of earned media placement)
  2. Social data analysis
  3. Surveys, interviews, focus groups.

Each project is done in teams. This blog post will focus on project 1, content analysis of media artifacts.

As an side, if you’re interested in learning about the social data analysis assignment, last year I wrote a 4-part series on that assignment that I’ve gotten a lot of comments and questions about.

Project 1: The Set Up

Learning research methods is a challenge for anyone new to it. Undergraduates sometimes express a strong aversion to the topic.

I’ve found that a content analysis of media artifacts is the most approachable method for introducing students to the systematic nature of doing research.

During the first few weeks of class, students are learning about research (e.g., the process, concepts such as reliability & generalizability, what research methods are, sampling, etc.).

After that, I introduce a hypothetical situation that the students will have to solve for their first project. Each year, I change up the situation a little bit. But the nuts and bolts have remained the same for the last 3 years.

I use the format from the Stacks book to set up the hypothetical situation students will address. You can see the entire text for the situation in the assignment at the bottom of this post. I will be referring to it in the paragraphs below.

The hypothetical is that the students work for an agency representing Netflix. Netflix is facing greater competition from other online streaming services like Amazon. To keep its competitive edge, Netflix is working to create shows that will appeal to a key market: 30-somethings. Stranger Things is one such show. Season 2 just launched.

Because the success of Netflix shows is widely influenced by critical acclaim from media, a media relations campaign was undertaken to position Netflix positively relative to itself competitors as a streaming service by way of the show.  The objectives of the campaign were to gain positive coverage of the premiere of season 2 of Stranger things.

The students enter the situation after the campaign has been executed and the campaign is now in the evaluation stage. Their job is to evaluate whether the media coverage was earned and what the nature of that coverage was.

Data Collection

I used to gather a sample of news articles from LexisNexis and provide them to the students. This semester, the students will gather the data set themselves using the Meltwater social intelligence software. I’m excited about this because it gets the students into Meltwater and thinking about the use of the tool’s dashboard features. In addition, students are learning that they have the ability to pull down data for further analysis outside of Meltwater.

[You can learn more about the Meltwater University program in this blog post.]

The Meltwater software enables users to gather news articles from a given time period. Searches are conducted using keywords. Stranger things season 2 launched October 27, 2017. So the data set is built around the season premiere. I don’t have a strong research justification for the exact date range chosen. Rather, I chose it because it produced a manageable number of articles for each student to have to code.

Also, please note that I do not operationalize what “top news sources” are from the assignment objectives. Instead, for purposes of the exercise, I have students pick the top 5 sources related to their search results to analyze.

You can see the procedure for gathering the data via the Meltwater for Media Article Content Analysis lab guide I created.

Data Analysis

There is a lot you could do with Meltwater to analyze the articles related to the launch of season 2 of Stranger Things.  If I had more time for this project, we’d dig into a lot of the dashboard tools. For now, students are only focusing on the quantitative content analysis of news articles.

I provide the students with details about specific research questions they are trying to answer related to media coverage: placement, share of voice, and whether or not the campaign’s 3 key messages made it into the press.

The data analysis is a simple quantitative content analysis of media artifacts. A simple coding sheet is provided. We discuss inter-coder reliability. And each student codes his/her media articles by hand using the coding sheet.

In class, we go over the coding sheet. And, in addition to the items on the coding sheet, which align with the research questions, students come up with their own item to code and to report in their paper. I do this exercise to get the students thinking about other things they could look for in the articles that might be useful.

It is worth noting that during a class activity earlier in the semester, students design their own coding sheets to evaluate car commercials and they learn quite a bit about the ups and downs of creating coding sheets. But, for the project, I create and provide the coding sheet. The operationalizations from the coding sheet are based on our class text, Paine’s “Measure what Matters: Online Tools for Understanding Customers, Social Media, Engagement, and Key Relationships.” For example, in class we discuss what we mean when we say that share of voice is “exclusive” or “dominant.”

While some of the coding can be done in class, students finish the coding as homework.

You can see the coding sheet that we will use this semester at the bottom of this blog post.

The students take their coded data and enter it into a spreadsheet so that we can quickly run frequency reports using SPSS.

The Project Write Up

With project 1, the write up that students produce is limited to providing a problem overview, the results, and a brief discussion section along with an appendix of their coding sheet. In the second and third projects students are asked to produce more and more of what a research paper might look like. But, because this is not an academic research class, I try to balance introducing students to a more academic style of research writing with a style that is more suitable to a report they might right in industry. I use a similar format to the format presented in the Stacks book.

I provide students with several handouts to help them write up their results.

One opportunity this assignment always presents, is a discussion about the limitations “simply measuring” the items on the coding sheet without looking at any context. As such, students are to go into their data and identify the features of the articles that support their results. Thus, they find example headlines and quotes to demonstrate, say, an example of a key message that was amplified.

Some Limitations

This project is, of course, limited in several rather important respects. However, I’ve created this project because it provides a great opportunity to introduce students to research, what a content analysis is, how to use a coding sheet, inter-coder reliability (invariably there are disagreements into how aspects of articles should be coded), and more.  Further, the project presents these learning opportunities within the context of learning a little bit about how one might evaluate earned media coverage. For example, students have learned about key messages in other classes. Now, they are learning about how those key messages may make their way into media articles and how the media represents them.

The project accomplishes this while situating the assignment in a campaign that is hypothetical but that is based on real events: Stranger Things Season 2 is real. Many of my students love the show and have watched it. They are reading real media articles about the show.  Further, students are situating this project as a campaign evaluation because the entire project is situated within the narrative that the students already executed the campaign and now they are evaluating it.

A Thought About the Key Messages Portion of the Coding Sheet

Each year I have changed up the Netflix show that we analyze and have thus changed the hypothetical backstory that accompanies it.

The key messages on the coding sheets are messages that I made up. They do not change much year to year, other than to bring them into the context of the show we’re evaluating. For example, this year I changed key message number two to emphasize the theme of nostalgia, which relates to the 30-something audience we are trying to target. I write the messages to be purposefully broad enough that they always end up achieving a good amount of frequency in the data set. The key messages are based on my general knowledge of Netflix. This year, it is possible that we won’t get many hits on key message number two. But, we’re bound to have some success with all three. However, if you choose to do this project and have some time, a better way to write the key messages would be to read through the data set ahead of time and develop them based on your own content analysis of the articles.

I hope that you found this blog post interesting and helpful. If you have ideas on how I can improve this project, please leave a comment or Tweet me. If you decide to use a version of this project in your own class, please stop on back and let readers know how it went or ways that you built upon it.

Don’t forget to check out the assignment below and the accompanying coding sheet.

Cheers!

– Matt

Project 1: Media Placement Assignment Handout

Project 1 Coding Sheet

Note: The hypothetical situation above uses the names of a real brand, Netflix, and its product. However, the situation is entirely made up and exists for educational purposes. Netflix logo is copyright Netflix.

Teaching The Applied Communication Research Class

Metrics, Metrics, Metrics! I hear it everywhere I turn. 🙂 More than ever, we need to be teaching our students research skills.

This Spring 2014 semester I am really excited to be teaching an applied Communication Research class!

For two years at Utah Valley University, I taught communication research with an emphasis on academic research. You can see the syllabus for that class. In that class, student groups planned, wrote up, and executed a semester long academic research study. Though many professors don’t prefer to teach this class, research is one of my favorite classes to teach. I’ve had numerous undergraduate students present their research at undergraduate research conferences and earn travel grants to do so. This is a super valuable experience for those considering grad school. Though it is very time demanding, and some feel teaching others how to conduct research is tedious, I didn’t find it that way at all. Seeing students get that “aha” moment in research and seeing them succeed makes teaching the class very rewarding.

This semester, I’ll be focusing on the more practical uses of research with an emphasis on using research for strategic purposes. This class emphasizes research across new media, legacy media, and interpersonal and online environments. Students will learn both quantitative and qualitative methods.

Our textbook is Paine’s “Measure what Matters: Online Tools for Understanding Customers, Social Media, Engagement, and Key Relationships.” I considered the Stacks book as well, but liked the emphasis on new media in Paine and felt her book may be more accessible to students, as students can be intimidated by a research class.

This hands on class will emphasize the following research skill sets:

  • How to conduct content analysis using a coding sheet.
  • How to conduct a computer-assisted content analysis
  • How to conduct interviews and focus groups
  • How to conduct quantitative electronic surveys using iPads

Students will work in teams to conduct 3 applied projects. The first 2 projects are real-world problems I set up and the students have to solve, and in the 3rd project they have to identify a problem, write a proposal, and execute:

  • Media placement evaluation – Answering questions such as, placement, share of voice, and whether key messages are included in media coverage and to what extent. Done via content analysis of media clippings.
  • Sentiment analysis of social media content – What are people saying about your brand on social media, and what is sentiment towards it? Done via computer-assisted content analysis of Twitter posts.
  • Audience Research – Focuses on 1 of the 5 key PR variables discussed by Stacks (2011): Confidence, credibility, relationship, reputation (which may include awareness), or trust. Students will choose 2 of the following: interviews, focus groups, and surveys.

Students will be introduced to the following software:

  • Computer-assisted content analysis (Yoshikoder will be used as it is free and easy to learn)
  • Digital Survey programming with XLS Forms
  • Open Data Kit Collector – field data survey collection software (we will be using this with the XLS forms on the free FormHub.com online form tool).
  • SPSS – We won’t get too far into SPSS due the other demands on the students time, but students will learn data entry, descriptive statistics, and correlation analysis.

I’ll be posting the syllabus for the class soon! As the semester goes along, I hope to get up a number of blog posts expanding on the class, assignments, and so forth. So check back!

Have you taught research – what do you emphasize in your class? How can I improve my class? What key skill sets should we be teaching  future practitioners?

-Cheers!

-Matt

– top photo CC by IntelFreePress