TU Darmstadt / ULB / TUbiblio

Towards User-Centered Active Learning Algorithms

Bernard, Jürgen and Zeppelzauer, Matthias and Lehmann, Markus and Müller, Martin and Sedlmair, Michael (2018):
Towards User-Centered Active Learning Algorithms.
In: Computer Graphics Forum, pp. 121-132, 37, (3), ISSN 01677055,
DOI: 10.1111/cgf.13406,
[Online-Edition: https://doi.org/10.1111/cgf.13406],
[Article]

Abstract

The labeling of data sets is a time-consuming task, which is, however, an important prerequisite for machine learning and visual analytics. Visual-interactive labeling (VIAL) provides users an active role in the process of labeling, with the goal to combine the potentials of humans and machines to make labeling more efficient. Recent experiments showed that users apply different strategies when selecting instances for labeling with visual-interactive interfaces. In this paper, we contribute a systematic quantitative analysis of such user strategies. We identify computational building blocks of user strategies, formalize them, and investigate their potentials for different machine learning tasks in systematic experiments. The core insights of our experiments are as follows. First, we identified that particular user strategies can be used to considerably mitigate the bootstrap (cold start) problem in early labeling phases. Second, we observed that they have the potential to outperform existing active learning strategies in later phases. Third, we analyzed the identified core building blocks, which can serve as the basis for novel selection strategies. Overall, we observed that data-based user strategies (clusters, dense areas) work considerably well in early phases, while model-based user strategies (e.g., class separation) perform better during later phases. The insights gained from this work can be applied to develop novel active learning approaches as well as to better guide users in visual interactive labeling.

Item Type: Article
Erschienen: 2018
Creators: Bernard, Jürgen and Zeppelzauer, Matthias and Lehmann, Markus and Müller, Martin and Sedlmair, Michael
Title: Towards User-Centered Active Learning Algorithms
Language: English
Abstract:

The labeling of data sets is a time-consuming task, which is, however, an important prerequisite for machine learning and visual analytics. Visual-interactive labeling (VIAL) provides users an active role in the process of labeling, with the goal to combine the potentials of humans and machines to make labeling more efficient. Recent experiments showed that users apply different strategies when selecting instances for labeling with visual-interactive interfaces. In this paper, we contribute a systematic quantitative analysis of such user strategies. We identify computational building blocks of user strategies, formalize them, and investigate their potentials for different machine learning tasks in systematic experiments. The core insights of our experiments are as follows. First, we identified that particular user strategies can be used to considerably mitigate the bootstrap (cold start) problem in early labeling phases. Second, we observed that they have the potential to outperform existing active learning strategies in later phases. Third, we analyzed the identified core building blocks, which can serve as the basis for novel selection strategies. Overall, we observed that data-based user strategies (clusters, dense areas) work considerably well in early phases, while model-based user strategies (e.g., class separation) perform better during later phases. The insights gained from this work can be applied to develop novel active learning approaches as well as to better guide users in visual interactive labeling.

Journal or Publication Title: Computer Graphics Forum
Volume: 37
Number: 3
Uncontrolled Keywords: Interactive Machine Learning, Evaluation, Visual analytics
Divisions: 20 Department of Computer Science
20 Department of Computer Science > Interactive Graphics Systems
Date Deposited: 29 Aug 2019 12:46
DOI: 10.1111/cgf.13406
Official URL: https://doi.org/10.1111/cgf.13406
Export:
Suche nach Titel in: TUfind oder in Google

Optionen (nur für Redakteure)

View Item View Item