Heinrich, Roman ; Binnig, Carsten ; Kornmayer, Harald ; Luthra, Manisha (2024)
Costream: Learned Cost Models for Operator Placement in Edge-Cloud Environments.
40th IEEE International Conference on Data Engineering (ICDE 2024). Utrecht, The Netherlands (13.05.2024 - 16.05.2024)
doi: 10.1109/ICDE60146.2024.00015
Konferenzveröffentlichung, Bibliographie
Kurzbeschreibung (Abstract)
In this work, we present Costream, a novel learned cost model for Distributed Stream Processing Systems that provides accurate predictions of the execution costs of a streaming query in an edge-cloud environment. The cost model can be used to find an initial placement of operators across heterogeneous hardware, which is particularly important in these environments. In our evaluation, we demonstrate that Costream can produce highly accurate cost estimates for the initial operator placement and even generalize to unseen placements, queries, and hardware. When using Costream to optimize the placements of streaming operators, a median speedup of around 21 × can be achieved compared to baselines.
| Typ des Eintrags: | Konferenzveröffentlichung |
|---|---|
| Erschienen: | 2024 |
| Autor(en): | Heinrich, Roman ; Binnig, Carsten ; Kornmayer, Harald ; Luthra, Manisha |
| Art des Eintrags: | Bibliographie |
| Titel: | Costream: Learned Cost Models for Operator Placement in Edge-Cloud Environments |
| Sprache: | Englisch |
| Publikationsjahr: | 23 Juli 2024 |
| Verlag: | IEEE |
| Buchtitel: | Proceedings: 2024 IEEE 40th International Conference on Data Engineering |
| Veranstaltungstitel: | 40th IEEE International Conference on Data Engineering (ICDE 2024) |
| Veranstaltungsort: | Utrecht, The Netherlands |
| Veranstaltungsdatum: | 13.05.2024 - 16.05.2024 |
| DOI: | 10.1109/ICDE60146.2024.00015 |
| Kurzbeschreibung (Abstract): | In this work, we present Costream, a novel learned cost model for Distributed Stream Processing Systems that provides accurate predictions of the execution costs of a streaming query in an edge-cloud environment. The cost model can be used to find an initial placement of operators across heterogeneous hardware, which is particularly important in these environments. In our evaluation, we demonstrate that Costream can produce highly accurate cost estimates for the initial operator placement and even generalize to unseen placements, queries, and hardware. When using Costream to optimize the placements of streaming operators, a median speedup of around 21 × can be achieved compared to baselines. |
| Freie Schlagworte: | Costs, Accuracy, Predictive models, Data engineering, Hardware, Data models, distributed stream processing, operator placement, graph neural networks, edge-cloud, IoT, zero-shot learning |
| Fachbereich(e)/-gebiet(e): | 20 Fachbereich Informatik 20 Fachbereich Informatik > Data and AI Systems DFG-Sonderforschungsbereiche (inkl. Transregio) DFG-Sonderforschungsbereiche (inkl. Transregio) > Sonderforschungsbereiche DFG-Sonderforschungsbereiche (inkl. Transregio) > Sonderforschungsbereiche > SFB 1053: MAKI – Multi-Mechanismen-Adaption für das künftige Internet DFG-Sonderforschungsbereiche (inkl. Transregio) > Sonderforschungsbereiche > SFB 1053: MAKI – Multi-Mechanismen-Adaption für das künftige Internet > C: Kommunikationsmechanismen DFG-Sonderforschungsbereiche (inkl. Transregio) > Sonderforschungsbereiche > SFB 1053: MAKI – Multi-Mechanismen-Adaption für das künftige Internet > C: Kommunikationsmechanismen > Teilprojekt C2: Informationszentrische Sicht |
| Hinterlegungsdatum: | 11 Okt 2024 12:49 |
| Letzte Änderung: | 11 Okt 2024 12:49 |
| PPN: | |
| Export: | |
| Suche nach Titel in: | TUfind oder in Google |
 |
Frage zum Eintrag |
Optionen (nur für Redakteure)
 |
Redaktionelle Details anzeigen |
Drucken
Impressum