TU Darmstadt / ULB / TUbiblio

On Soft-Biometric Information Stored in Biometric Face Embeddings

Terhorst, Philipp ; Fahrmann, Daniel ; Damer, Naser ; Kirchbuchner, Florian ; Kuijper, Arjan (2021):
On Soft-Biometric Information Stored in Biometric Face Embeddings.
In: IEEE Transactions on Biometrics, Behavior, and Identity Science, (Early Access), IEEE, ISSN 2637-6407,
DOI: 10.1109/TBIOM.2021.3093920,
[Article]

Abstract

The success of modern face recognition systems is based on the advances of deeply-learned features. These embeddings aim to encode the identity of an individual such that these can be used for recognition. However, recent works have shown that more information beyond the user’s identity is stored in these embeddings, such as demographics, image characteristics, and social traits. This raises privacy and bias concerns in face recognition. We investigate the predictability of 73 different soft-biometric attributes on three popular face embeddings with different learning principles. The experiments were conducted on two publicly available databases. For the evaluation, we trained a massive attribute classifier such that can accurately state the confidence of its predictions. This enables us to derive more sophisticated statements about the attribute predictability. The results demonstrate that the majority of the investigated attributes are encoded in face embeddings. For instance, a strong encoding was found for demographics, haircolors, hairstyles, beards, and accessories. Although face recognition embeddings are trained to be robust against non-permanent factors, we found that specifically these attributes are easily-predictable from face embeddings. We hope our findings will guide future works to develop more privacy-preserving and bias-mitigating face recognition technologies.

Item Type: Article
Erschienen: 2021
Creators: Terhorst, Philipp ; Fahrmann, Daniel ; Damer, Naser ; Kirchbuchner, Florian ; Kuijper, Arjan
Title: On Soft-Biometric Information Stored in Biometric Face Embeddings
Language: English
Abstract:

The success of modern face recognition systems is based on the advances of deeply-learned features. These embeddings aim to encode the identity of an individual such that these can be used for recognition. However, recent works have shown that more information beyond the user’s identity is stored in these embeddings, such as demographics, image characteristics, and social traits. This raises privacy and bias concerns in face recognition. We investigate the predictability of 73 different soft-biometric attributes on three popular face embeddings with different learning principles. The experiments were conducted on two publicly available databases. For the evaluation, we trained a massive attribute classifier such that can accurately state the confidence of its predictions. This enables us to derive more sophisticated statements about the attribute predictability. The results demonstrate that the majority of the investigated attributes are encoded in face embeddings. For instance, a strong encoding was found for demographics, haircolors, hairstyles, beards, and accessories. Although face recognition embeddings are trained to be robust against non-permanent factors, we found that specifically these attributes are easily-predictable from face embeddings. We hope our findings will guide future works to develop more privacy-preserving and bias-mitigating face recognition technologies.

Journal or Publication Title: IEEE Transactions on Biometrics, Behavior, and Identity Science
Number: Early Access
Publisher: IEEE
Uncontrolled Keywords: Face recognition, Biometrics, Machine learning, Deep learning
Divisions: 20 Department of Computer Science
20 Department of Computer Science > Interactive Graphics Systems
20 Department of Computer Science > Mathematical and Applied Visual Computing
Date Deposited: 13 Jul 2021 08:41
DOI: 10.1109/TBIOM.2021.3093920
Export:
Suche nach Titel in: TUfind oder in Google
Send an inquiry Send an inquiry

Options (only for editors)
Show editorial Details Show editorial Details