This post may contain affiliate links. Please read my disclosure for details.

In the last two posts, I’ve been discussing the Yoshikoder sentiment analysis project in my Communication Research class here at Shepherd University.

My first post looked at the project in general. And the second, most recent post, looked at how to teach computer-assisted content analysis using the Yoshikoder computer-assisted content analysis software and the activities I provide my students to prepare them for the project.

I encourage you to check out those posts for background and set up! Ok, now on to sharing the assignment itself and providing a brief overview of it.

As I’ve stated elsewhere, the purpose of this assignment is to

1) give students a hands-on look under the hood of sentiment analysis – that is, to understand HOW it works and its flaws.

2) To teach students via hands=on experience about quantitative content analysis, particularly computer-assisted content analysis

3) To teach them how to conduct a computer-assisted content analysis using software (Yoshikoder)

So here’s the set up to the assignment (which you can see below). This hands-on learning project is based on a real brand and a realistic but made up scenario. I do this with both this assignment, and my first project in this class.  Specifically, I provide The Situation or Problem / Campaign goals and objectives (of an imaginary campaign that is ongoing or happened) / benchmarks / KPIs.

In this case, the situation had to do with a popular online retail brand and rising customer complains and dissatisfaction as the brand has grown beyond its core base of loyal customers in recent years.I’ve redacted the brand and the situation from the below assignment. But you can fill in your own.

I rely on Stacks (2011) model for writing the problem, goals, objectives.  While I provide the research objective(s) in my first project, in this project students must come up with the research objective(s) and RQ(s).

I then provide some benchmarks. In this scenario, at a certain point in time sentiment was strong (let’s say, 70% positive). And then after the hypothetical situation, it dropped (say, to 50%). The students have been recently introduced to the concepts of benchmarks and KPIs via a brief lecture, so this is their first experience with these concepts. They are given 1 KPI (let’s say 65% positive sentiment) against which to measure their success. Keep in mind that the situation assumes that a campaign already took place aimed at addressing decreased customer satisfaction and negative comments on Twitter addressed at the brand of choice. We are now seeking to assess whether this campaign that happened successfully increased sentiment towards the brand (at a deeper level, repaired relationships and the image of the brand among the online community).

There are other important considerations students must make:

1) Since we’ve discussed sentiment and its flaws, they need to think about the valence of sentiment (The AFINN dictionary scores terms from -5 to +5), and they need to research and understand how AFINN was designed and works (I provide some sources to get them started). If you’re not familiar with the AFINN dictionary, it was designed for sentiment analysis of microblogs.It is a free sentiment dictionary of terms you can download and use in Yoshikoder. 

For more details on the assignment, check out the assignment embedded below and the requirements for what must be turned in.

As I’ve noted in a previous post, this project isn’t perfect. But it is a fairly straightforward and accessible learning experience for students who are in their first semester of experiencing how research can be conducted. It covers a wide array of experiences and learning opportunities – from discussion of what sentiment is, to understanding its flaws, to understanding the flaws of quantitative content analysis, to learning to apply a number of key research terms, as well as providing exposure to how to write research reports. The project itself is bolstered by several lectures, it comes about 1/2 way through the semester, and takes several days in the classroom of hands on learning. Students of course finish the writing up outside of class. But we do the analysis all in class to ensure students are getting my help as the “guide on the side.”

My previous post covers some activities we do to build up to this assignment.

So that’s all for now! Please feel to use this assignment, to modify it, and improve it. If you do, come back and share how you have or how you would improve upon it and modify it in the comments below!

If you want to know more about my Communication Research class, please see this post which includes the syllabus.