Content Analysis Preliminary Findings: Part I, Agenda

This is the first of three blog posts which outline some preliminary findings from our content analysis of alternative media in the UK. This content analysis is the first of three phases of project research. In this post, we provide an overview of some of the broad features about the agendas of the outlets we are studying, including how much of it is fact- or opinion-driven, what topics receive the most focus, and the degree of media critique. Prior to peer-review, this is not intended as an in-depth or final account.

Here’s the sample size and distribution from our first phase of content analysis, dubbed the ‘rise’ phase. The sample periods refer to four periods of time (6 – 25 October 2015; 9 – 29 October 2016; 30 April – 7 June 2017 [General Election]; and 8 – 28 October 2018). As well as allowing us to make broader inferences about content, this sample also itself tells us something about output and routine.

For example, you can clearly see which outlets produce the most and least material. Both The Canary and Guido Fawkes stand out. As you can also see here, these outlets clearly produced far more material during the general election, even taking into account the extra two weeks of data collection.

It is important to note some caveats. Novara Media is underrepresented because it produces much more video and podcast content than the other outlets; our analysis was textual. Moreover, some sites didn’t produce any material (Skwawkbox in 2016) or didn’t yet exist (Westmonster prior to 2017).

Turning to some percentages, our analysis reveals the degree to which content is “fact-driven” or “opinion-driven”. By this, we mean whether a story was primarily focused on providing information (we didn’t fact-check as part of our analysis) or whether it focussed on providing opinion. This divide roughly equates to a divide between “news” and “commentary”. This analysis allowed for opinion within fact-driven pieces, and vice-versa, so it should not be interpreted as either/or.

As these charts show, there are some key differences. Left-wing sites, such Evolve Politics and The Canary, clearly produce more opinion-driven content. Right-wing sites, like Westmonster and Breitbart London, produce more fact-driven content. For more distinct sites such as Guido Fawkes, with a firm Westminster, breaking-news focus, this focus on news is unsurprising. Likewise, with Another Angry Voice, which does not self-identify as a news site, it is unsurprising to see a high degree of commentary.

Again, some caveats. The Canary’s opinion-led percentage here is somewhat skewed by a surge of commentary during the 2017 general election. In other years, around 45% of its content was fact-driven. Secondly, this fact-driven variable doesn’t always equate to original reporting. As we will show in another blog post, much of the sources used by these sites are recycled from mainstream outlets.

In terms of the topic of the content, there are three standout categories (we applied seventeen codes here). By “party political”, we mean a story that places an emphasis on policy, process, or events concerning a political party and its associated actors. By “media”, we simply mean a story that focuses on the media. Likewise, “EU Affairs” meant anything about the European Union (in practice, this meant Brexit stories).

As these charts show, party political content makes up a large proportion of the content. This suggests a fairly ‘hardcore’ political agenda, which appears more emphatic among the left-wing sites. EU affairs make up a strong proportion of the content produced by right-wing sites, with Westmonster, an explicitly pro-Brexit outlet, leading the way. Media stories also feature highly, with around 1 in 5 articles by The Canary and Conservative Woman being about the media.

This takes us onto the final finding in this blog post: articles that contain material that is critical of the media. In our project, this is the focus of a specific analysis and what you see here is effectively the construction of our subsample. To qualify, an article only needed to contain a single mention, that is, a sentence-level unit of analysis, which critiqued the media in some way.

The findings here clearly reveal that left-wing sites provide much more criticism than right-wing sites, with the exception of Conservative Woman. The fairly simple hypothesis here is that the emergence of the ‘alt-left’ is a critical reaction to the right-wing British tabloid press. However, it still shows that right-wing sites are willing to engage in criticism.

In our next blog post, we will explore how the sites represent political parties.