TU Darmstadt / ULB / TUbiblio

Detecting Humorous Images by Caption Analysis

Ockenfels, Malou and Miller, Tristan and Puzikov, Yevgeniy (2019):
Detecting Humorous Images by Caption Analysis.
In: 2019 Conference of the International Society for Humor Studies, Austin, TX, USA, 24.06.2019--28.06.2019, [Conference or Workshop Item]

Abstract

The automatic recognition of verbal humour has become an established work area in natural language processing (NLP), but the detection of humour in visual media is still in its infancy. In this paper, we describe and evaluate NLP methods for detecting humorous images by analyzing descriptive captions. We present a data set of 40 scenes manually annotated with English-language captions and funniness scores, as well as various knowledge-based and data-driven methods that use the captions alone to predict the funniness of the associated scene. Our knowledge-based methods, inspired by (verbal) humour-theoretic notions of incongruity and surprise, use semantic frames, selectional preferences for verb dependencies, and/or n-gram frequencies, while our data-driven methods include bag-of-words models and pre-trained word embeddings used as features in various machine learning classifiers: naïve Bayes, support vector machine (SVM), random forest, and a multilayer perceptron. On our data, the bag-of-words model with an SVM achieves the best classification performance, approximating the human upper bound. Our analysis of false negatives indicates that the element of incongruity is absent, or at least not obvious, in many funny scenes or their descriptive captions.

Item Type: Conference or Workshop Item
Erschienen: 2019
Creators: Ockenfels, Malou and Miller, Tristan and Puzikov, Yevgeniy
Title: Detecting Humorous Images by Caption Analysis
Language: English
Abstract:

The automatic recognition of verbal humour has become an established work area in natural language processing (NLP), but the detection of humour in visual media is still in its infancy. In this paper, we describe and evaluate NLP methods for detecting humorous images by analyzing descriptive captions. We present a data set of 40 scenes manually annotated with English-language captions and funniness scores, as well as various knowledge-based and data-driven methods that use the captions alone to predict the funniness of the associated scene. Our knowledge-based methods, inspired by (verbal) humour-theoretic notions of incongruity and surprise, use semantic frames, selectional preferences for verb dependencies, and/or n-gram frequencies, while our data-driven methods include bag-of-words models and pre-trained word embeddings used as features in various machine learning classifiers: naïve Bayes, support vector machine (SVM), random forest, and a multilayer perceptron. On our data, the bag-of-words model with an SVM achieves the best classification performance, approximating the human upper bound. Our analysis of false negatives indicates that the element of incongruity is absent, or at least not obvious, in many funny scenes or their descriptive captions.

Divisions: 20 Department of Computer Science
20 Department of Computer Science > Ubiquitous Knowledge Processing
Event Title: 2019 Conference of the International Society for Humor Studies
Event Location: Austin, TX, USA
Event Dates: 24.06.2019--28.06.2019
Date Deposited: 23 May 2019 13:41
Export:
Suche nach Titel in: TUfind oder in Google

Optionen (nur für Redakteure)

View Item View Item