TU Darmstadt / ULB / TUbiblio

Robust and Efficient Aggregation for Distributed Learning

Vlaski, Stefan ; Schroth, Christian ; Muma, Michael ; Zoubir, Abdelhak M. (2022)
Robust and Efficient Aggregation for Distributed Learning.
30th European Signal Processing Conference. Belgrade, Serbia (29.08.-02.09.2022)
doi: 10.23919/EUSIPCO55093.2022.9909822
Konferenzveröffentlichung, Bibliographie

Kurzbeschreibung (Abstract)

Distributed learning paradigms, such as federated and decentralized learning, allow for the coordination of models across a collection of agents, and without the need to exchange raw data. Instead, agents compute model updates locally based on their available data, and subsequently share the update model with a parameter server or their peers. This is followed by an aggregation step, which traditionally takes the form of a (weighted) average. Distributed learning schemes based on averaging are known to be susceptible to outliers. A single malicious agent is able to drive an averaging-based distributed learning algorithm to an arbitrarily poor model. This has motivated the development of robust aggregation schemes, which are based on variations of the median and trimmed mean. While such procedures ensure robustness to outliers and malicious behavior, they come at the cost of significantly reduced sample efficiency. This means that current robust aggregation schemes require significantly higher agent participation rates to achieve a given level of performance than their mean-based counterparts in non-contaminated settings. In this work we remedy this drawback by developing statistically efficient and robust aggregation schemes for distributed learning.

Typ des Eintrags: Konferenzveröffentlichung
Erschienen: 2022
Autor(en): Vlaski, Stefan ; Schroth, Christian ; Muma, Michael ; Zoubir, Abdelhak M.
Art des Eintrags: Bibliographie
Titel: Robust and Efficient Aggregation for Distributed Learning
Sprache: Englisch
Publikationsjahr: 18 Oktober 2022
Verlag: IEEE
Buchtitel: 30th European Signal Processing Conference (EUSIPCO 2022): Proceedings
Veranstaltungstitel: 30th European Signal Processing Conference
Veranstaltungsort: Belgrade, Serbia
Veranstaltungsdatum: 29.08.-02.09.2022
DOI: 10.23919/EUSIPCO55093.2022.9909822
URL / URN: https://ieeexplore.ieee.org/document/9909822
Kurzbeschreibung (Abstract):

Distributed learning paradigms, such as federated and decentralized learning, allow for the coordination of models across a collection of agents, and without the need to exchange raw data. Instead, agents compute model updates locally based on their available data, and subsequently share the update model with a parameter server or their peers. This is followed by an aggregation step, which traditionally takes the form of a (weighted) average. Distributed learning schemes based on averaging are known to be susceptible to outliers. A single malicious agent is able to drive an averaging-based distributed learning algorithm to an arbitrarily poor model. This has motivated the development of robust aggregation schemes, which are based on variations of the median and trimmed mean. While such procedures ensure robustness to outliers and malicious behavior, they come at the cost of significantly reduced sample efficiency. This means that current robust aggregation schemes require significantly higher agent participation rates to achieve a given level of performance than their mean-based counterparts in non-contaminated settings. In this work we remedy this drawback by developing statistically efficient and robust aggregation schemes for distributed learning.

Freie Schlagworte: emergenCITY, emergenCITY_CPS
Fachbereich(e)/-gebiet(e): 18 Fachbereich Elektrotechnik und Informationstechnik
18 Fachbereich Elektrotechnik und Informationstechnik > Institut für Nachrichtentechnik
18 Fachbereich Elektrotechnik und Informationstechnik > Institut für Nachrichtentechnik > Robust Data Science
18 Fachbereich Elektrotechnik und Informationstechnik > Institut für Nachrichtentechnik > Signalverarbeitung
LOEWE
LOEWE > LOEWE-Zentren
LOEWE > LOEWE-Zentren > emergenCITY
Hinterlegungsdatum: 09 Dez 2022 09:11
Letzte Änderung: 01 Mär 2023 15:34
PPN: 505397692
Export:
Suche nach Titel in: TUfind oder in Google
Frage zum Eintrag Frage zum Eintrag

Optionen (nur für Redakteure)
Redaktionelle Details anzeigen Redaktionelle Details anzeigen