Computer-Assisted Understanding of Stance in Social Media

Formalizations, Data Creation, and Prediction Models

University of Duisburg-Essen

Language:

Michael Wojatzki

Michael Wojatzki completed his doctorate in January 2019 at the graduate school "User-Centred Social Media" at the University of Duisburg-Essen. Besides his academic career, Michael has worked for several companies in the information industry and software development. His research focuses on automatic, AI-based systems that recognise and analyse opinions shared in social media.

Let's have a?

Expertise

  • Data science
  • Artificial intelligence (machine learning)
  • Opinion mining / public opinion research

Of interest to

  • Data scientists
  • Computer scientists
  • Social media manager
  • Pollsters
Pietro Jeng/Unsplash
Michael Wojatzki

Michael Wojatzki completed his doctorate in January 2019 at the graduate school "User-Centred Social Media" at the University of Duisburg-Essen. Besides his academic career, Michael has worked for several companies in the information industry and software development. His research focuses on automatic, AI-based systems that recognise and analyse opinions shared in social media.

Let's have a?

Expertise

  • Data science
  • Artificial intelligence (machine learning)
  • Opinion mining / public opinion research

Of interest to

  • Data scientists
  • Computer scientists
  • Social media manager
  • Pollsters

Interview

Anja Zeltner
Author

What does „stance“ mean in the context of social media?

Michael Wojatzki
is typing…
Anja Zeltner
Freie Autorin

What does „stance“ mean in the context of social media?

Michael Wojatzki
Doktorand

Stance refers to an evaluation, either positive or negative, which is directed towards a person, thing or idea. People’s stance drives them to vote for a certain party or candidate, to buy a certain product, or to avoid or approach people. As people today express their stance in large quantities on social media sites, social media can be an important source for assessing the stance of groups or society as a whole.

Anja Zeltner
Freie Autorin

You analysed if it is possible to predict how people will position themselves to controversial topics, for instance climate change or gender equality. What are your results?

Michael Wojatzki
Doktorand

It is actually possible to make such predictions. However, two prerequisites must be given: First, the system undertaking the analysis needs training data to understand a certain topic. Such data can be obtained, for example, from whether people liked relevant Facebook posts. Second, prediction models are always topic-specific. If you trained such a model to predict positions on climate change, it cannot readily be used to predict positions on gender equality. Even under these conditions, predicting the position of individuals is very challenging. The difficulty for computers here is that individuals often have conflicting positions—e.g. find it immoral to eat meat but still do it. Predictions at a group level can be made with high accuracy. In such a case, we would try to figure out what percentage of a group agrees or disagrees with a position.

Anja Zeltner
Freie Autorin

How can stance analysis in social media affect society positively or negatively?

Michael Wojatzki
Doktorand

In machines that are able to automatically understand stance on a large scale, there is great potential for but also a danger to society. For example, if people start to use stance detection to only engage with content that aligns with their own stance, existing echo-chambers will be amplified and discourses on controversial issues—that incorporate every member of society—will not take place. Ultimately this may lead to a more fractured society which is unable to find consensus. In addition, stance detection also provides the technical basis for an all-pervading censorship or the persecution of political dissidents. At the same time, automatic stance detection can improve the efficiency with which social media users or organisations discover, group or filter social media posts that express stance towards targets they are interested in. In this way, automatic stance detection can help the society as a whole to communicate more efficiently, and thus to make better decisions.

Keywords

Social Media, Stance

Abstract

Stance can be defined as positively or negatively evaluating persons, things, or ideas (Du Bois, 2007). Understanding the stance that people express through social media has several applications: It allows governments, companies, or other information seekers to gain insights into how people evaluate their ideas or products. Being aware of the stance of others also enables social media users to engage in discussions more efficiently, which may ultimately lead to better collective decisions.

Since the volume of social media posts is too large to be analyzed manually, computeraided methods for understanding stance are necessary. In this thesis, we study three major aspects of such computer-aided methods: (i) abstract formalizations of stance which we can quantify across multiple social media posts, (ii) the creation of suitable datasets that correspond to a certain formalization, and (iii) stance detection systems that can automatically assign stance labels to social media posts.

We examine four different formalizations that differ in how specific the insights and supported use-cases are: Stance on Single Targets defines stance as a tuple consisting of a single target (e.g. Atheism) and a polarity (e.g. being in favor of the target), Stance on Multiple Targets models a polarity expressed towards an overall target and several logically linked targets, and Stance on Nuanced Targets is defined as a stance towards all texts in a given dataset. Moreover, we study Hateful Stance, which models whether a post expresses hatefulness towards a single target (e.g. women or refugees).

Machine learning-based systems require training data that is annotated with stance labels. Since annotated data is not readily available for every formalization, we create our own datasets. On these datasets, we perform quantitative analyses, which provide insights into how reliable the data is, and into how social media users express stance. Our analysis shows that the reliability of datasets is affected by subjective interpretations and by the frequency with which targets occur. Additionally, we show that the perception of hatefulness correlates with the personal stance of the annotators. We conclude that stance annotations are, to a certain extent, subjective and that future attempts on data creation should account for this subjectivity. We present a novel process for creating datasets that contain subjective stances towards nuanced assertions and which provide comprehensive insights into debates on controversial issues.

To investigate the state-of-the-art of stance detection methods, we organized and participated in relevant shared tasks, and conducted experiments on our own datasets. Across all datasets, we find that comparatively simple methods yield a competitive performance. Furthermore, we find that neuronal approaches are competitive, but not clearly superior to more traditional approaches on text classification. We show that approaches based on judgment similarity – the degree to which texts are judged similarly by a large number of people – outperform reference approaches by a large margin. We conclude that judgment similarity is a promising direction to achieve improvements beyond the state-of-the-art in automatic stance detection and related tasks such as sentiment analysis or argument mining.

volltext_auf_opend

The full text of this dissertation is available on OpenD. Online and OpenAccess.

Read it now

Suggested citation

Wojatzki, Michael Maximilian. Computer-Assisted Understanding of Stance in Social Media: Formalizations, Data Creation, and Prediction Models. Universität Duisburg-Essen, 2019, doi:10.17185/duepublico/48043.

Repository

duepublico.uni-duisburg-essen.de

Identifiers

urn:nbn:de:hbz:464-20190201-140926-6

doi: 10.17185/duepublico/48043