TU Darmstadt / ULB / TUbiblio

Practice-Oriented Authorship Verification

Halvani, Oren (2021)
Practice-Oriented Authorship Verification.
Technische Universität Darmstadt
doi: 10.26083/tuprints-00019861
Ph.D. Thesis, Primary publication, Publisher's Version

Abstract

The question of the authorship of texts has occupied numerous areas both within and beyond research for a very long time. Knowing whether a particular person is a potential author of a text is of central importance for countless application scenarios. In practice, there are many examples where a document has an alleged author, but the authorship is disputed by another party. These include, for example, theses suspected to have been written by someone else (e. g., ghostwriters), testaments purportedly written by the individuals involved, messages disseminated from compromised email or social media accounts, or claims for damages from allegedly real policyholders. The underlying task in all these examples is authorship verification (AV), which concentrates on the fundamental question of whether two texts were written by the same person. AV represents an important sub-discipline of authorship analysis that has been researched for decades. However, when looking at much of the existing research in the field of AV, it becomes apparent that the focus is mainly on the detection accuracies of AV methods. Other important issues dealing with the robustness, reliability, sensitivity to topic influences, and generalizability of AV methods, as well as the interpretability of verification results, receive much less attention in comparison. The latter, interpretability, is of considerable interest. The reason for this is that verification results in the form of probabilities, similarity scores, and binary predictions (same-authorship/different-authorship) are generally insufficient on their own to be used in practice (especially in forensic contexts).

The aim of this thesis is to shed light on these and other aspects of AV and to provide definitions, concepts and approaches that can and have been used in realistic scenarios. We first provide an improved systematization of AV, in which we concentrate on relevant key topics. We explain the importance of preprocessing in the context of AV and propose an effective technique that masks topic-related words in documents. In this way, potential biases with respect to verification results can be successfully counteracted. We also propose new categories of features that can be used in AV (and other related disciplines) for different types of texts, e. g., newspaper articles, reviews, chat logs, emails, and scientific texts. Furthermore, we present a taxonomy of characteristics of AV methods that can be used to compare AV approaches in more detail. These characteristics can be used to assess the extent to which AV methods are suitable for practical use, regardless of their detection accuracy. Moreover, we highlight shortcomings of existing evaluation methodologies in the literature and, in particular, address the weaknesses of some performance measures that are still used today to evaluate AV methods. In this context, we present an alternative evaluation approach that aims to reflect a problematic aspect of AV methods in terms of their applicability in practice. After an in-depth analysis of a variety of existing AV methods, we propose alternative approaches that aim to counteract the identified issues. To supplement this, we offer various (visualization) techniques that allow human experts such as investigators to interpret the verification results of our AV methods. Based on a set of self-compiled corpora covering different genres and topics, we afterwards evaluate our AV approaches against competitive baseline methods. These corpora focus on different challenges, such as verification cases with cross-topic conditions, documents written in two widely separated time periods, and documents with excessive use of slang. Thus, the suitability of the methods is evaluated and analyzed from different perspectives.

Item Type: Ph.D. Thesis
Erschienen: 2021
Creators: Halvani, Oren
Type of entry: Primary publication
Title: Practice-Oriented Authorship Verification
Language: English
Referees: Waidner, Prof. Dr. Michael ; Savoy, Prof. Dr. Jaques
Date: 2021
Place of Publication: Darmstadt
Collation: 244 Seiten
Refereed: 28 June 2021
DOI: 10.26083/tuprints-00019861
URL / URN: https://tuprints.ulb.tu-darmstadt.de/19861
Abstract:

The question of the authorship of texts has occupied numerous areas both within and beyond research for a very long time. Knowing whether a particular person is a potential author of a text is of central importance for countless application scenarios. In practice, there are many examples where a document has an alleged author, but the authorship is disputed by another party. These include, for example, theses suspected to have been written by someone else (e. g., ghostwriters), testaments purportedly written by the individuals involved, messages disseminated from compromised email or social media accounts, or claims for damages from allegedly real policyholders. The underlying task in all these examples is authorship verification (AV), which concentrates on the fundamental question of whether two texts were written by the same person. AV represents an important sub-discipline of authorship analysis that has been researched for decades. However, when looking at much of the existing research in the field of AV, it becomes apparent that the focus is mainly on the detection accuracies of AV methods. Other important issues dealing with the robustness, reliability, sensitivity to topic influences, and generalizability of AV methods, as well as the interpretability of verification results, receive much less attention in comparison. The latter, interpretability, is of considerable interest. The reason for this is that verification results in the form of probabilities, similarity scores, and binary predictions (same-authorship/different-authorship) are generally insufficient on their own to be used in practice (especially in forensic contexts).

The aim of this thesis is to shed light on these and other aspects of AV and to provide definitions, concepts and approaches that can and have been used in realistic scenarios. We first provide an improved systematization of AV, in which we concentrate on relevant key topics. We explain the importance of preprocessing in the context of AV and propose an effective technique that masks topic-related words in documents. In this way, potential biases with respect to verification results can be successfully counteracted. We also propose new categories of features that can be used in AV (and other related disciplines) for different types of texts, e. g., newspaper articles, reviews, chat logs, emails, and scientific texts. Furthermore, we present a taxonomy of characteristics of AV methods that can be used to compare AV approaches in more detail. These characteristics can be used to assess the extent to which AV methods are suitable for practical use, regardless of their detection accuracy. Moreover, we highlight shortcomings of existing evaluation methodologies in the literature and, in particular, address the weaknesses of some performance measures that are still used today to evaluate AV methods. In this context, we present an alternative evaluation approach that aims to reflect a problematic aspect of AV methods in terms of their applicability in practice. After an in-depth analysis of a variety of existing AV methods, we propose alternative approaches that aim to counteract the identified issues. To supplement this, we offer various (visualization) techniques that allow human experts such as investigators to interpret the verification results of our AV methods. Based on a set of self-compiled corpora covering different genres and topics, we afterwards evaluate our AV approaches against competitive baseline methods. These corpora focus on different challenges, such as verification cases with cross-topic conditions, documents written in two widely separated time periods, and documents with excessive use of slang. Thus, the suitability of the methods is evaluated and analyzed from different perspectives.

Alternative Abstract:
Alternative abstract Language

Die Frage nach der Autorschaft von Texten beschäftigt seit langem zahlreiche Bereiche innerhalb und außerhalb der Forschung. Zu wissen, ob eine bestimmte Person ein potentieller Autor eines Textes ist, ist für unzählige Anwendungsszenarien von zentraler Bedeutung. In der Praxis finden sich viele Beispiele, in denen ein Dokument einen vermeintlichen Autor hat, die Autorschaft aber von anderer Seite bestritten wird. Dazu zählen beispielsweise Dissertationen, bei denen der Verdacht besteht, dass sie von einer anderen Person (z. B. Ghostwriter) geschrieben wurden, Testamente, die angeblich von den Betroffenen stammen, Nachrichten, die von kompromittierten E-Mail- oder Social-Media-Konten verbreitet werden oder auch Schadensersatzansprüche von angeblich realen Versicherungsnehmer. Die zugrundeliegende Aufgabe in all diesen Beispielen ist die Autorschaftsverifikation (AV), welche sich mit der fundamentalen Frage beschäftigt, ob zwei Texte von der gleichen Person geschrieben wurden. Die AV stellt eine wichtige Teildisziplin der Autorschaftsanalyse dar, an der seit Jahrzehnten geforscht wird. Bei der Betrachtung eines Großteils existierender Forschungsarbeiten auf dem Gebiet der AV fällt allerdings auf, dass der Fokus hauptsächlich auf den Erkennungsgenauigkeiten von AV-Methoden liegt. Andere wichtige Fragen, die sich mit der Robustheit, der Zuverlässigkeit, der Sensitivität gegenüber thematischen Einflüssen und der Generalisierbarkeit von AV-Methoden sowie der Interpretierbarkeit von Verifikationsergebnissen befassen, finden demgegenüber deutlich weniger Beachtung. Letzteres, die Interpretierbarkeit, ist von erheblichem Interesse. Der Grund dafür ist, dass Verifikationsergebnisse in Form von Wahrscheinlichkeiten, Ähnlichkeitsscores und binären Vorhersagen (Gleiche-Autorschaft/Ungleiche-Autorschaft) für sich allein im Allgemeinen nicht ausreichend sind, um in der Praxis (insbesondere in forensischen Kontexten) verwendet zu werden.

Das Ziel dieser Arbeit ist es, diese und andere Aspekte der AV zu beleuchten sowie Definitionen, Konzepte und Ansätze zu liefern, die in realistischen Szenarien eingesetzt werden können und wurden. Wir erarbeiten zunächst eine verbesserte Systematisierung von AV, wobei wir uns auf relevante Schlüsselthemen konzentrieren. Wir erläutern die Bedeutung der Vorverarbeitung im Kontext von AV und schlagen eine effektive Technik vor, die themenbezogene Wörter in Dokumenten maskiert. Dadurch kann potenziellen Verzerrungen in Bezug auf die Verifikationsergebnisse erfolgreich entgegengewirkt werden. Des Weiteren schlagen wir neue Kategorien von Merkmalen vor, die im Rahmen von AV (und anderen verwandten Disziplinen) für verschiedene Arten von Texten verwendet werden können, wie beispielsweise Zeitungsartikel, Rezensionen, Chat-Protokolle, E-Mails und wissenschaftliche Texte. Darüber hinaus stellen wir eine Taxonomie von Eigenschaften von AV-Methoden vor, mit der sich AV-Ansätze detaillierter vergleichen lassen. Anhand dieser Eigenschaften lässt sich beurteilen, inwieweit AV-Methoden unabhängig von ihrer Erkennungsgenauigkeit für den praktischen Einsatz geeignet sind. Weiterhin zeigen wir Unzulänglichkeiten bestehender Bewertungsmethoden in der Literatur auf und gehen insbesondere auf die Schwächen einiger Leistungsmaße ein, die heute noch zur Bewertung von AV-Methoden verwendet werden. In diesem Zusammenhang stellen wir einen alternativen Bewertungsansatz vor, der einen problematischen Aspekt von AV-Methoden im Hinblick auf ihre Anwendbarkeit in der Praxis widerspiegeln soll. Nach einer eingehenden Analyse einer Vielzahl existierender AV-Methoden schlagen wir alternative Ansätze vor, die darauf abzielen, den identifizierten Problemen entgegenzuwirken. Ergänzend dazu bieten wir verschiedene (Visualisierungs-)Techniken an, die es menschlichen Experten wie beispielsweise Ermittlern und Ermittlerinnen ermöglichen, die Verifikationsergebnisse unserer AV-Methoden zu interpretieren. Basierend auf einer Reihe selbst erstellter Korpora, die verschiedene Genres und Themen abdecken, evaluieren wir anschließend unsere AV-Ansätze gegen konkurrierende Baseline-Methoden. Diese Korpora fokussieren sich auf unterschiedliche Herausforderungen, wie beispielsweise Verifikationsfälle mit themenübergreifenden Bedingungen, Dokumente, die in zwei weit voneinander entfernten Zeiträumen geschrieben wurden sowie Dokumente mit übermäßigem Gebrauch von Slang. Dadurch wird die Eignung der Methoden aus verschiedenen Perspektiven analysiert und bewertet.

German
Status: Publisher's Version
URN: urn:nbn:de:tuda-tuprints-198617
Classification DDC: 000 Generalities, computers, information > 004 Computer science
400 Language > 400 Language, linguistics
Divisions: 20 Department of Computer Science
20 Department of Computer Science > Security in Information Technology
Date Deposited: 15 Nov 2021 13:03
Last Modified: 22 Nov 2021 09:45
PPN:
Referees: Waidner, Prof. Dr. Michael ; Savoy, Prof. Dr. Jaques
Refereed / Verteidigung / mdl. Prüfung: 28 June 2021
Export:
Suche nach Titel in: TUfind oder in Google
Send an inquiry Send an inquiry

Options (only for editors)
Show editorial Details Show editorial Details