Teaching Computer-Assisted Content Analysis with Yoshikoder

Last blog post I discussed the second project in my applied research class, a sentiment analysis of Tweets using Yoshikoder - a free computer-assisted content analysis program from Harvard.

As promised, I want to share my assignment, and my handout for students that teaches them how to use Yoshikoder. Before we do the project, however, I do a brief in class activity to get students learning how to use Yoshikoder. So let’s start there for today’s post. And next post, I’ll share the assignment itself.


What I like to do, is present the problem to the students via the project assignment. Then, we go back and start learning what we’d need to do to solve the problem. So, after lecturing about what sentiment analysis is and why it is important, I get students introduced first to the idea of constructing a coding sheet for keywords by taking a list of keywords and adding them to categories.

First, we talk about the idea in class, and I show them some simple examples, like: If I wanted to code a sample for the presence of “sunshine” – what words would I need? Students brainstorm things like  start, sun, sunny, sunshine, etc., etc.

We discuss the importance of mutual exclusivity, being exhaustive, etc.

I show an example from my dissertation which looked at agenda setting topics on Twitter.

On the class day before I introduce Yoshikoder to the class, students do a practice assignment where I give them a list of random terms related to politics and elections. They then have to create “positive” and “negative” content categories using the terms. The terms aren’t necessarily well fit for this exercise, which gets them thinking a bit… They then hand code a sample of Tweets I provide about two different politicians. I tend to use the most recent election. So, in this case Obama and Romney. They are frustrated by having to hand code these Tweets – but a little trick is to do a search for the exact phrases in the Tweet files on the computer and they are done fairly quickly. Ok, so on the next class period:

1) Practice with Yoshikoder We do the same basic task, but this time they learn to program their “positive” and “negative” categories into Yoshikoder. They then load the Tweets (which I have saved as a txt file) and analyze them for the presence of their positive and negative content categories. This is a great point to stop and have students assess the reliability between what they hand coded and what the computer coded. Often, there will be discrepancies. And this makes for a great opportunity for discussion.

Here is the activity that I use in class. I also provide Tweets that I’ve downloaded using the search terms for the politician/candidate I’m using in the activity (e.g., Obama; Romney) in plain text format so Yoshikoder can read it. Also, see the below handout which I provide students to show them how to use Yoshikoder and how to program, and run the analyses I just described.

As I mentioned above, I create a handout that I like to give students that explains the different functionalities of Yoshikoder and how to run the analyses. As I’ve discussed elsewhere, I like to provide handouts. And the one below isn’t one of my more elaborate handouts. But it provides a quick overview with some screen shots to show what buttons need to be clicked. This is super helpful if you are trying to learn Yoshikoder, or want to use it alongside the activity (discussed in this post or the project discussed in my last post, and which I will provide in my next blog post).

Enjoy! .

If you’d like to learn more about using Yoshikoder, I found this great tutorial:

- Cheers! Matt

Applied Research Class: Sentiment Analysis Project Reflection

I began this semester with the intention of blogging a bit about my applied research class. I provided an overview of it and a copy of the syllabus on an earlier post. But since writing that post, I’ve yet to do a follow up… until now.

First, let me say that more and more I am trying to decrease my lecturing and spend more time in class with hands on learning, having my students learn by doing rather than just listening - sort of like the flipped classroom Gary Schirr has been discussing recently on his blog.   So this class is really pushing in class projects and experiential learning. Following this approach, in order to introduce students to research, I provided students with the instructions and a lot of structure for their first two projects.

I want to use our second research project as an example. Then, I’ll talk about the pros and cons. The second project was a sentiment analysis of Tweets about a brand I chose and a (realistic but not necessarily real) scenario.

My goals with this project were to teach students:

  1. About computer-assisted content analysis. We focused on how it is different from a hand-coded quantitative content analysis (which was the focus of our first project). And its strengths and weaknesses.
  2. How to do a basic computer-assisted content analysis using Yoshikoder, an easy to use, free App that works on Mac and PC. So my students can use it at home if needed!
  3. About sentiment analysis – what it is, why it is used by organizations to evaluate the online conversation about their brand, and its strengths and weaknesses.
  4. How to write up a research report (In the first project, I provided the project overview and requested results and discussion. In the second project, I added a literature review and methods section, and had them write the research objective and research question).

Why I chose to do this project this way: A number of social media analytics tools today are offering sentiment analysis.  There are also sites like socialmention.com that will provide you with a free sentiment analysis of a search term. But how are these analyses conducted? What are their strengths and weaknesses? Are they reliable? Do they mean anything at all? And what do we need to be careful of before accepting them, and thus drawing inferences from them?

So what I wanted my students to do, was to SEE how a sentiment analysis would be conducted by some of those high-price (or no price!) analytic tools. In other words, I want my students to get their hands dirty as opposed to allowing some distant and hidden algorithm to do the analysis for them. I believe gaining hands on experience with this project provides students a more critical lens through which to see and evaluate a sentiment analysis of social media messages.

The Set Up: I provide in the assignment: The Situation or Problem / Campaign goals and objectives (of an imaginary campaign that is ongoing or happened) / benchmarks / KPIs. In this case, the situation had to do with a popular online retail brand and rising customer complains and dissatisfaction as the brand has grown beyond its core base of loyal customers in recent years.

I provide students with the sample of about 1000 Tweets I downloaded and formatted to play nicely with Yoshikoder. The sample comprises of mentions of the brand. This ensures students are all looking at the same dataset, and streamlines (or eliminates I should say) the data collection process to help students focus on other elements of the assignment. For the sentiment analysis,

I rely on the AFINN dictionary, which was designed for sentiment analysis of microblogs. Students learn about what the AFINN is and a little about how linguistic analysis dictionaries are created through research. Students then analyze the Twitter dataset using the AFINN dictionary to determine the sentiment scores. There is no fancy stats being done here. By checking the sentiment analysis output, they simply determine if their KPI (which was a % of positive Tweets about the brand) was met. In this case, the result they are looking for is a % – so simple division. Not scary at all, no SPSS training needed (that comes with a later project).

They also look at the valence of the sentiment (with a range of + or -5) and explore the meaning of that. The students use this information, along with class lecture, other exercises on how to write research reports, etc., to produce their project #2 report.

Again, to reiterate an important point, we discuss the benefits and of this analysis as well as its real weaknesses. Students always bring up the fact that the results lack context – what if someone used the word “bad” meaning good? What about sarcasm? I show them how to use Yoshikoder to look at Keywords in context as a way of addressing this.

The Benefits and Drawbacks of This (and these types of) Projects As I said above,  I am really trying to move away from lecture in favor of experiential learning. Here are some things I’ve noticed. Some may be benefits, others drawbacks, and others a bit of both…

  • The focus on this project is not on the stats or the analysis and I provide a lot of the needed information – so it makes for a good ‘getting your feet wet’ project that teaches students other important elements of research.
  • It would be nice to teach them more advanced methods of analysis – but I do cover that a bit more later in the semester.
  • Students learn through their mistakes and from my feedback as opposed to me paving the way for them and simply asking them to drive down the smooth road.
  • I provide a LOT of handouts on how to write different sections of a research report, etc. They are detailed… sometimes too detailed and I fear students don’t read them because it is information shock.
  • Sometimes, I wish I had more time to teach them how to avoid the simple mistakes I see in their work, particularly their research reports. I say to myself, “oh man, I thought I told them how to do that.” Or, “Why didn’t you read the handout that explains how to structure this!?”
  • They likely won’t do sentiment analysis like this every again – but at least they’ll understand it!
  • They get to see the results for themselves and get a sense that they discovered the results.
  • Class time is busy – our class rushes by and we don’t always get to cover everything I want to. As a person who likes order and time management, I am having to “let go a little” and let things happen. This is helping me grow. I wonder if it is helping my students though…
  • I know I enjoy doing these sorts of projects a lot more than standing and lecturing, lecturing, lecturing about research. I feel it has made research a lot more “real” and hands on to them.

So that is my overview of the project in general, and some thoughts. It isn’t perfect but it seems to have gone well and I really enjoyed doing it. I’d love any feedback or suggestions you may have to make this the best possible experience for my students. And of course, feel free to adapt, modify, or improve upon this idea.

In an upcoming post(s), I’ll share the assignments (I want to move my documents over to SlideShare due to the pay wall on Scribd). And I will provide some basic info on how to use the Yoshikoder software.

Cheers! -Matt

photo CC by netzkobold

#TryThis! A Technique For Enhancing Teamwork, Discussion In Class

This is the 3rd and final #TryThis! post I had planned from several months ago. I never got a chance to post it… until now!

Wouldn’t it be great if there was a way to get students thinking about topics, discussing them, taking a stance, and then having to defend that stance and engage in critical discussion with their classmates? Wouldn’t new ideas emerge, different ways of evaluating a problem be presented, and overall learning be enhanced?

Of course! Students would get practice examining case studies, solving ethical or legal scenarios, and making decisions about real-world situations they may face in the workplace. The possibilities are inspiring!

If we follow Bloom’s Taxonomy, then we should strive for our students to not only understand information, but be able to to analyze and evaluate information.

But it sometimes feels hard to find strategies to motivate students to talk in class. And, let’s be honest: it is just easier to lecture and have students regurgitate on a test, isn’t it? These things, and fear of what will happen when we let go of the “power position” in the front of the classroom, may keep us from truly developing the active classroom we strive for.

One strategy to encourage analysis, evaluation, and discussion in class that I have found helpful is: simultaneous response. I learned about simultaneous response at a pedagogy seminar I attended a few years ago on team-based learning (TeamBasedLearning.org) featuring a very well-known educator Larry K. Michaelsen. Now, I admit I don’t execute it perfectly. Larry had a # of very important rules for its effectiveness. But I’ve managed to lose them over time. Still, I apply the general concept.

It works like this:

The professor poses a question to the class (I usually have it on a slide).

The question asks the students to evaluate a problem/scenario and arrive at a solution. The question usually follows a brief lecture on the related material.

Multiple choice answers are given – usually 3-5, but you can do yes/no, true/false, or agree/disagree as well.

In teams, students discuss the question and arrive at an answer. Students are able to discuss differing opinions related to course content, notes, etc.

Teams write their answer (A, B, C, D, etc) on a piece of paper.

When given the signal, all teams hold up their answer simultaneously.

Why it is Effective

First, students get to discuss the problem in a group – so they discuss the problem and related concepts. I let them use their notes, so they are actively discussing and learning. It is a competition so they want to get it right – they are motivated – and arrive at a group answer. The critical element of discussion gives them time to evaluate the merits of alternative answers they are leaning toward.

Because all teams must hold up their answer simultaneously, they cannot shy away from what they’ve committed to. Normally, quiet students or students who aren’t sure of themselves will just not talk in class. As a result, the most talkative students tend to dominate class discussion – and only 3 or 4 students are talking. But with this technique, their team is forced to defend itself. Naturally, discussion occurs and students see different ways of looking at things. In fact, I’ve found that students get a bit more talkative in this scenario, perhaps because they feel a sense of competition or a need to assert themselves.

Here is a simple example I’ve used in my Principles of PR class.

The topic was law and social media. The activity was predicated with a brief lecture on legal issues PR practitioners must consider, and an article highlighting how new media have posed new challenges to issues like defamation. http://bit.ly/PPR_Defamation . In the real life scenario, Horizon Realty sued a tenant for tweeting the below quote, claiming defamation. The lawsuit against a tenant was thrown out because the defendant merely suggested that Horizon thought it was “ok” for her to sleep in a moldy apartment. The judge ruled that the tweet was too vague to satisfy the strict test for defamation.

Question: In your assessment, do you agree with what happened in the case where Horizon Realty sued their tenant  for Tweeting “You should just come anyway. Who said sleeping in a moldy apartment was bad for you? Horizon realty thinks it’s ok.”? (Be ready to explain!)

A. Agree.

B. Disagree.

While this is a question asking their INFORMED OPINION, students are reminded after the discussion that the burden of proof is on the defamed and therefore, if one was to follow the law as it is applied, then one would would be inclined to agree.

Questions such as this raise great discussions and empower students to express their knowledge and their opinions about what should have been done in a scenario, or in regard to an issue, law, etc. Of course, other types of questions can be asked where one’s opinion does not come into play. This technique is adaptable. I used it here in a lower-division course, but it can also be used in upper division courses for just about any subject matter. The purpose is to drive students to use their knowledge of a subject to analyze material and make real-world decisions.

I use this technique a lot and I love it. It is one of my favorite because students engage with it. Of all the techniques I have used, it has been the best way of getting all students involved!

Key Things to Keep in Mind:

  1. The question is important – I have written a few duds myself and am always looking to find ways to write better questions for this exercise. To me, a good question is one that 1) challenges the students to think about the application of materials and how they come together for the problem, and that 2) inspires student discussion.
  2. Teach them the rules – This is essentially a game. Students are getting to compete! But it is probably something quite different from what they’ve done in other classes. So it is important that you clearly explain how the exercise works. I like to have a few practice questions – 1 or two “easy ones” that are unrelated to course content to get them to understand how it works.

That’s all for now! If you use this technique or try it, I’d love to hear your thoughts!



photo credit: Bob Xu via photopin cc

Web Round up: Tools, Academic News, and Zuck’s status update on Web Freedom

Wow! Spring break has flown by! I can’t believe it is Friday already! While my spring break has been mostly dedicated to catching up on work or trying to get ahead on projects, I took some time to relax and got out and hiked with Scout on Tuesday during our first day of truly nice weather this year (Instagram photos below and to the right of course :) ).

Today’s post is a quick look at some great articles from around the web I’ve been reading over the past week or so that you may have missed.

Thanks so much to the wonderful people on Twitter who shared many of these – you are my go to source for news!

Social Media Tools:

29 Social Media Tools Recommended By the Pros by the always instructive Social Media Examiner

50 Top Tools for Social Media Monitoring, Analytics, and Management by Social Media Today

The (Potential) Pitfalls of Social Media Tools

Duck Dynasty, Amazon Show The Pitfalls Of Big Data – Highly recommend! This article cautions us about relying too much on the info we gather about audiences on social media tools when it comes to informing our decisions. We forget that the audience on social media is not representative of the wider population, and more specifically, our target audience. Reminds me of a lesson I learned in my first research methods class about sampling. :)

Academia News and Issues

Lost faculty job offer raises questions about negotiation strategy – Super interesting look at job negotiation and the issues surrounding it. This article tells the story of how negotiating a tenure-track position for one faculty backfired when a university pulled the offer. With job negotiating such a complex and difficult task, this is certainly worth a read.

What Should Students Call Their Professors? – This one raised quite the discussion on my Facebook page among faculty friends. The opinions were diverse. I think the article makes a great point that many students simply don’t know what to call their professors due to the wide array of persons teaching them, from graduate students, to MAs, to Ph.Ds., and so forth.

Zuckerberg on Internet Freedom

Lastly, Mark Zuckerberg posted the following on his Facebook page as a status update I thought I’d share. Though it doesn’t relate to the above, it is noteworthy. Reactions have been mixed, with some calling Zuckerberg a hypocrite given his company’s focus on collecting and using information to market to individuals. Here is his status update (I copied and pasted it):

As the world becomes more complex and governments everywhere struggle, trust in the internet is more important today than ever.

The internet is our shared space. It helps us connect. It spreads opportunity. It enables us to learn. It gives us a voice. It makes us stronger and safer together.

To keep the internet strong, we need to keep it secure. That’s why at Facebook we spend a lot of our energy making our services and the whole internet safer and more secure. We encrypt communications, we use secure protocols for traffic, we encourage people to use multiple factors for authentication and we go out of our way to help fix issues we find in other people’s services.

The internet works because most people and companies do the same. We work together to create this secure environment and make our shared space even better for the world.

This is why I’ve been so confused and frustrated by the repeated reports of the behavior of the US government. When our engineers work tirelessly to improve security, we imagine we’re protecting you against criminals, not our own government.

The US government should be the champion for the internet, not a threat. They need to be much more transparent about what they’re doing, or otherwise people will believe the worst.

I’ve called President Obama to express my frustration over the damage the government is creating for all of our future. Unfortunately, it seems like it will take a very long time for true full reform.

So it’s up to us — all of us — to build the internet we want. Together, we can build a space that is greater and a more important part of the world than anything we have today, but is also safe and secure. I’m committed to seeing this happen, and you can count on Facebook to do our part.


I hope you’ve had a great spring break!



Google Scholar Power User’s Guide: Research Recommendations (3 of 3)

This is the third and final post on what’s been a popular series of posts – becoming a Google Scholar power user. Post #1 explored the advanced search features of Google Scholar, and post #2 explored why you should have a Google Scholar profile.

This post about the Recommendations feature comes last because you must have a Google Scholar profile in order to use it.

In past blog posts, I’ve written about great ways to find research articles for your literature review. Specifically, I’ve talked about using Google Scholar search, and the Mendeley search option.

Google Scholar recommended research is another way of finding research articles that I’m loving. It is super easy to use and I’ve found tons of articles I wouldn’t have found before. As you recall, these recommendations are based on your citations – in other words, what you’ve published online. So of course they are going to be tailored to your research interests.

Google Scholar recommendations

Click to enlarge.

If you have your Google Scholar profile set up, Google Scholar will recommend new research articles to you based on your publications. So the recommendations are almost always super relevant and helpful for future studies!

To access these, go to scholar.google.com and click “My Updates” at the top.

Also, Google Scholar will often list the most recent recommendations under the search bar at scholar.google.com. You can get the rest by clicking “see all updates” (see photo above)

Here are my recommended articles today:

Click to enlarge.

Click to enlarge.

Tailored scholarly article recommendations – what could be better?

Hope you enjoyed this series of posts on Google Scholar! If you did, please share this post!

Other articles in series:



Become A Google Scholar Power User: Scholar Profiles (2 of 3)

Thanks to everyone who read my first post in this three-part series on becoming a Google Scholar Power User:  advanced features of Google Scholar search.  It was one of the most popular posts I’ve had on this blog! If you haven’t read it, check it out. If you have, let’s get started with part 2! Here we go! (Note: Post #3 is also available now)

To get the most out of Google Scholar, including the library feature discussed in the last post, you’ll need a Google Scholar profile. If you don’t have one, I highly recommend setting one up. Here’s why:

A Google Scholar profile has many benefits. It is a public profile that provides author photo, institutional affiliation and contact information, and importantly an interactive list of all articles a particular scholar has published. (Get started with Google Scholar profiles). Google Scholar automatically populates your list of research articles by associating your name with articles it has indexed. You can also add co-authors, and keywords for your research agenda.

Click to enlarge. A look at my Google Scholar Profile

Click to enlarge. A look at my Google Scholar Profile

Here are 8 great benefits of a Google Scholar Profile:

1) Help people find your work. Remember that author feature I discussed int he previous post? Google Scholar search results allow searchers to click an author’s name to see what else they’ve published. This links directly to your Google Scholar profile with the list of all your research articles. If you have a website/blog, you can add a link so people can connect with you.

2) Track Citations of Your Work. What’s really great, is your Google Scholar profile summarizes some really informative stats.

This includes the number of citations for each article you’ve published, and up to date summary statistics across time. For example, here is my Google Scholar profile.

These numbers are helpful to have! For example, when doing my mid-tenure review, I provided a brief context / explanation of each publication I had (this was a recommendation made to me by a faculty member). The purpose was to provide an explanation of the significance of the work, and how it related to my research agenda. I noted the # of citations particular publications had received according to Google Scholar.

3) Follow New Citations. You can easily follow new citations of your work, and get Google Alerts emails when articles are published that mention your published works. This is great to know who is citing you, the impact your work is having, as well as finding articles related to your research interest that you may want to read.

4) Follow New Article. Have something in press and want to know when it is published? Go to your profile and click “Follow new articles.” You’ll get an email alert when the time comes!

5) Library. As discussed in the previous post on Google Scholar, you need a profile to use this feature. See explanation in last post.

6) Recommendations – Having a Google Scholar profile enables you to get recommendations. I’ll discuss those in our next post.

7) Easily Find The Work of Your Favorite Scholars You can see the work of scholars whose research you enjoy by searching an authors name in Google Scholar search, or directly from your profile page (assuming they have a Google Scholar profile).

8) Follow Fav Scholars. Want to get alerts every time a particular scholar publishes something new? Go to their scholar profile and click “follow new articles.” You’ll get an email alert every time they publish something. You can also follow new citations of their work, to receive emails every time someone cites their work.

That’s all for now! Update: Post #3 is now available: why you should have a Google Scholar profile.

What other benefits of Google Scholar profiles are there? How do you use Google Scholar profiles?

See all my academic Tips, Tricks, and Productivity Posts.

Become a Google Scholar Power User (1 of 3)

Update: Posts #2 (Google Scholar Profiles) and #3 (Google Scholar Recommendations) are now available.

I love Google Scholar. It is useful for not only your research agenda, but also it is a tool to teach your students about.

When teaching students about finding academic research, no discussion is complete without Google Scholar. In fact, I tell my students this is my go to source… though I’m sure the library probably wouldn’t be happy to hear that.

GScholars primary purpose is as a search engine for scholarly articles. Simply goto: scholar.google.com and search for an article title, subject (e.g., a theory, construct), keyword, or author.

There’s much more to GScholar than a simple search. GScholar has some little known (and some very new!) features that are very useful. Here’s how to become a Google Scholar Power User, Part 1 – Advanced Search Features: (See: Post #2 in this series on Google Scholar Profiles).

Great Benefits!

It works just like Google, but it indexes academic publications. Many libraries are linked with Google Scholar, such that if you search Google Scholar and an article comes up that your school library has, you can access it directly through your search engine results. (While this often happens automatically when on campus, you can find out if your library has the article when away from campus as well. Go to your settings, click library, and do a search for your universities library. Then click save).

Advanced Features:

1) Advanced Article Access

Another great benefit is that sometimes articles are hosted online in various places, and you can find access to those articles that you would otherwise not have access to through your library.

Identifying this access is easy. To the right of the search result, you will see [PDF] available from XYZ or [HTML] available from. For an example, here is a search for my own research .  Notice how the “Getting Political on Social Network Sites” is available as HTML from the online journal First Monday.

Example of Google Scholar Search Results. See explanation of features below.

Example of Google Scholar Search Results. See explanation of features below. Click this image to enlarge it.

2) More Access Options

See an article that you want to get access to, but your library doesn’t have it and you don’t see a PDF version offered in the search results? Click “## versions” directly under the search result description. Sometimes, the primary result that Google shows does not provide access to the article, but alternative versions do. By clicking “## versions”, you may find that another version of the article is available online.

3) Related Articles

When conducting research, we’re often looking for research on a particular theory, construct, etc. So, if we find an article that fits our search goals, wouldn’t it be great to see what other articles are similar? The “Related Articles” link under the search entry does just that. For example, if I find an article on agenda setting in social media in a search result, and want to see more like it, rather than try a new search query, I’ll click “related articles” under the article I like for a whole host of articles related to agenda setting on social media.

4) Cited By Feature

The cited by feature offers a similar benefit to the “related articles” feature. It is very helpful because you can see who has cited this work. Why is that great? Because if the article you have found is of interest, likely those works that cited it are related and may be helpful! More so, they may have built on that study and thus their theory and findings may provide more recent insights and advancements to the topic you’re studying.

5) Author

If you want to see more articles by a specific author, click an author’s name (such as my name in the example above). This will take you to the author’s profile on GScholar (if they have one) where you can see all the articles they’ve published. This is something I’ll discuss more in a future post.

6) Cite

Want the APA or MLA citation format for the article in your search result? Click cite. A window pops up and you can choose the citation you need. Quick. Easy. Super helpful. You can also import into particular format styles.

7) Google Scholar Library

This is a new feature and one I just discovered. It works sort of like how Mendely lets you create a library of articles. You need a Google Scholar account to use this feature – which is of course free and connected to your Google account.

What you get, is an online custom list of articles. When you enable Library, you are asked if you want to import all articles you’ve cited. That is, Google indexes all the articles you have cited in the online publications Google has associated with you.

To access your library click “My library” at scholar.google.com. To add articles to your library, in a search result, you can click “save” to save that article directly from search results into your library.

It seems the deleting articles requires you to click on them individually and then click delete. I found no mass editing.

You can set up labels to organize articles into categories. For example, I may have a label “politics” another “social networks” and another “blogs.” By clicking an article in your library, then clicking the ‘labels’ drop down you can create and select labels.

This is a brand new feature and it has a lot of potential. Given that I’m a big Mendeley user, the library feature may be redundant. But I’m going to play with it and see if it has added benefit. I do like the idea that it provides direct access to articles available online.

Here is the How to and FAQ of Google Scholar Libraries 

That’s all for now!

The next 2 posts get beyond the basics of Google Search search results into Recommendations and Google Profiles. Check them out:

  1. Post #2 Scholar Profiles.
  2. Post #3 Scholar Recommendations


Do you use Google Scholar for search? What are your favorite features? What do you think about the new library feature?