Posted: December 31st, 2014

Sampling strategy and codebook for content analysis

Sampling strategy and codebook for content analysis

BACKGROUND. You work for the United Nations, who just issued (through their Intergovernmental Panel on Climate Change) a scientific consensus statement indicating that humans almost certainly cause global warming. At the same time, you are concerned that newspapers who report on global warming (also called climate change) may be presenting the issue with a “false balance” – presenting arguments for and against humans causing climate change, when there is a scientific consensus on one side of the issue. A previous study found evidence for this false balance (Boykoff & Boykoff, 2004); your job is to assess whether patterns of coverage since January 1st, 2012 reflect a similar tendency to report with a false balance.

YOUR TASKS. Your job is to develop a sampling strategy and codebook for a content analysis of New York Times, Washington Post, Los Angeles Times, and Wall Street Journal news stories related to global warming and/or climate change published since January 1st, 2012. In order to do so, you will need to complete (most) of the 6 steps involved in doing a content analysis.

We have already completed Step 1 (see lecture slides for the six steps) for you: The population of interest for this assignment includes all news stories related to global warming and/or climate change since January 1st, 2012. To obtain copies of the text of these news stories, you should use Lexis-Nexis and enter the following search term for stories that have appeared in the New York Times, Washington Post, Los Angeles times, & Wall Street Journal (you may choose whether to focus on print stories only, blogs, or both – see Lexis-Nexis’ list of sources for these papers):

(“global warming” OR “climate change”) AND DATE is after 1/1/2012

You may not be able to analyze all of these news stories (e.g., conduct a census), depending on how many newspaper articles are retrieved with this search term, so you may need to figure out a sampling strategy that will allow you to test whether or not these newspapers cover global warming and/or climate change in a falsely balanced way using a sample of news stories.

Task 1. In approximately two single-spaced, typed pages (12-pt font), answer each of the following questions: (4.5 points in total)

1)    DEVELOP YOUR MEASURES (step 2 from lecture)

a.    Provide a one-sentence conceptual definition of each variable involved in the research question. You’ll need to provide a conceptual definition for (a) news stories related to global warming and/or climate change, and (b) false balance. You will find the following articles helpful to look at as a starting point. You may start from the authors’ conceptualizations of related terms like “information bias,” “balanced reporting,” and “false balanace”, but you should come up with your own definitions of what would constitute a “false balance.” (1 point)

Boykoff, M. T., & Boykoff, J. M. (2004). Balance as bias: Global warming and the US prestige press. Global Environmental Change, 14, 125-136. doi: 10.1016/j.gloenvcha.2003.10.001. Available online at:
http://www.eci.ox.ac.uk/publications/downloads/boykoff04-gec.pdf

Dixon, G., & Clarke, C. (2013). The effect of falsely balanced reporting of the autism-vaccine controversy on vaccine safety perceptions and behavioral intentions. Health Education Research, 28, 352-359. doi:10.1177/1075547012458290. Available online at: http://her.oxfordjournals.org/content/28/2/352.long

o    Provide a conceptual definition for news stories related to global warming and/or climate change, being sure to (1) address what you mean by news stories and (2) what counts as being related to these topics
o    Provide a conceptual definition for “false balance” including key dimensions or elements. The definition should explain clearly (1) what do you mean by “balance” and (2) why such balance is considered biased or false (.5 pts)
o    Note: These definitions should (1) be written in your own words (not copying the authors’ terms but citing the authors if they influence your definition), (2) represent a conceptual, not operational definition

2)    DEVELOP A SAMPLING STRATEGY (step 3 from lecture)

a.    Specify how many news stories you’ll need to analyze to test your research question. (1 point)

In other words, calculate your sample size(s). Describe why you chose this particular sample size. To make these decisions, you’ll want to specify the level of confidence and error you are willing to tolerate for this study. You will want to make use of a sample size calculator in making your decisions.

b.    Describe your sampling strategy using terms and concepts we’ve learned about in class and/or in the textbook.
In this description, be sure that you answer each of the following questions: (i) Among all types of news stories (e.g., news stories, editorials, blog posts, opinion pieces, etc.), which ones will you choose as news stories related to global warming and/or climate change? (HINT: this decision should match your conceptual definition of what constitutes a news story related to the topic).

You will start by using the Lexis-Nexis search term listed above to search for relevant stories to include in your sample. (HINT: We will demonstrate how to use Lexis-Nexis in class on approximately Monday, October 7th). The set of articles that are returned from your search term will serve as the population for this assignment (HINT: depending on which types of New York Times, Los Angeles Times, Washington Post, and Wall Street Journal databases you include in your search, your population may be smaller or larger).

You will then need to describe the process of selecting a sample from this population by answering the following questions: (ii) How will you select specific news stories, from the overall population of news stories, to include in your sample? Will this process involve non- random sampling (convenience or snowball) or random sampling (simple or multi-stage cluster)? You can propose any strategy that you consider appropriate, but be sure to justify your decision and identify tradeoffs associated with that decision.

3)    DESCRIBE HOW YOU WILL ASSESS CODER RELIABILITY (step 4 from lecture)

a.    Describe the process that you will undertake to assess coder reliability. How will you assess whether coders are reliable in applying the codebook? Will you have multiple people double-code some of the same articles? How many will you double-code? Describe the process that your coders will follow. (1 point)

o    Name some procedures that you learned from the lecture about inter-coder reliability (.5 pts)
o    Provide a clear and thoughtful description of the process and what it entails (.5 pts)

Task 2. On a separate page (or pages), provide a draft of the codebook you will use to content analyze the news stories (think of these as your operational definitions). The codebook itself should be separate from the 2-page document (4.5 points total)

Your codebook should include no more than 5 coding decisions for your coders, per news story. In other words, you have to decide on just 5 things to assess in the news story you decide to include in your study. The codebook should include (1) the names of (no more than 5) indicators of “false balance”; and (2) specific instructions about how to measure each indicator. You will get to provide your rationale for why you decided to choose these indicators in your 2-page summary document. It is often useful, in a codebook, to illustrate your directions for measuring each item with an example. As part of the grading for this assignment, we are going to take your codebook and see if we could use it to accurately code news stories on the topic.

In addition to providing the codebook itself, provide a brief rationale for each measure (as part of the 2-page summary; the codebook should be separate from this). Why did you choose to measure this as an indicator of false balance? How do these indicators reflect your conceptual definition? It is always a good idea to look at how other people have gone about coding false bias in news stories (and how they sampled newspaper articles to find them!). There is a sizeable literature on this topic.

It is always a good idea to start with questions that other scholars have used before. If you think previous measures are good questions and capture some aspect of the construct perfectly, you can use these measures, but be clear that you are quoting them directly. If you think previous measures are bad questions and you can improve on them, do so. You should begin by lookingat Boykoff & Boykoff (2004) (http://www.eci.ox.ac.uk/publications/downloads/boykoff04-gec.pdf) and Dixon and Clarke (2013) (http://her.oxfordjournals.org/content/28/2/352.long), as well as
at least one other reference (each should be cited in your paper in APA format):

If you decide to use other people’s measures, or develop your own based directly on what others have done before, be sure to cite these articles in your assignment using APA format (please include both in-text citations and a reference list at the end of your assignment). Failure to do so would constitute plagiarism, a violation of the academic code of conduct at Cornell.

Task 3. Select four news stories from your sample. Code them using your codebook. Enter your codes in the spreadsheet. (step 5 from lecture)

These four do not have to be randomly selected or correspond to your proposed sampling strategy – just pick any four to try it out yourself.  All of your codes should be quantitative in nature – a number should correspond to each variable in each story. For example, if you were coding for the presence or absence of emotional display, you could assign “1” when it is shown or “0” when it is not.  Attach full-text of each news story that you coded to the end of your assignment.

PLACE THIS ORDER OR A SIMILAR ORDER WITH US TODAY AND GET AN AMAZING DISCOUNT 🙂