Krämer, Michel ; Würz, Hendrik M. ; Altenhofen, Christian (2024)
Executing cyclic scientific workflows in the cloud.
In: Journal of Cloud Computing : Advances, Systems and Applications, 2021, 10 (1)
doi: 10.26083/tuprints-00023616
Artikel, Zweitveröffentlichung, Verlagsversion
 | Es ist eine neuere Version dieses Eintrags verfügbar. |
Kurzbeschreibung (Abstract)
We present an algorithm and a software architecture for a cloud-based system that executes cyclic scientific workflows whose structure may change during run time. Existing approaches either rely on workflow definitions based on directed acyclic graphs (DAGs) or require workarounds to implement cyclic structures. In contrast, our system supports cycles natively, avoids workarounds, and as such reduces the complexity of workflow modelling and maintenance. Our algorithm traverses workflow graphs and transforms them iteratively into linear sequences of executable actions. We call these sequences process chains. Our software architecture distributes the process chains to multiple compute nodes in the cloud and oversees their execution. We evaluate our approach by applying it to two practical use cases from the domains of astronomy and engineering. We also compare it with two existing workflow management systems. The evaluation demonstrates that our algorithm is able to execute dynamically changing workflows with cycles and that design and maintenance of complex workflows is easier than with existing solutions. It also shows that our software architecture can run process chains on multiple compute nodes in parallel to significantly speed up the workflow execution. An implementation of our algorithm and the software architecture is available with the Steep Workflow Management System that we released under an open-source license. The resources for the first practical use case are also available as open source for reproduction.
| Typ des Eintrags: | Artikel |
|---|---|
| Erschienen: | 2024 |
| Autor(en): | Krämer, Michel ; Würz, Hendrik M. ; Altenhofen, Christian |
| Art des Eintrags: | Zweitveröffentlichung |
| Titel: | Executing cyclic scientific workflows in the cloud |
| Sprache: | Englisch |
| Publikationsjahr: | 8 April 2024 |
| Ort: | Darmstadt |
| Publikationsdatum der Erstveröffentlichung: | 6 April 2021 |
| Ort der Erstveröffentlichung: | Berlin ; Heidelberg |
| Verlag: | SpringerOpen |
| Titel der Zeitschrift, Zeitung oder Schriftenreihe: | Journal of Cloud Computing : Advances, Systems and Applications |
| Jahrgang/Volume einer Zeitschrift: | 10 |
| (Heft-)Nummer: | 1 |
| Kollation: | 26 Seiten |
| DOI: | 10.26083/tuprints-00023616 |
| URL / URN: | https://tuprints.ulb.tu-darmstadt.de/23616 |
| Zugehörige Links: | |
| Herkunft: | Zweitveröffentlichung DeepGreen |
| Kurzbeschreibung (Abstract): | We present an algorithm and a software architecture for a cloud-based system that executes cyclic scientific workflows whose structure may change during run time. Existing approaches either rely on workflow definitions based on directed acyclic graphs (DAGs) or require workarounds to implement cyclic structures. In contrast, our system supports cycles natively, avoids workarounds, and as such reduces the complexity of workflow modelling and maintenance. Our algorithm traverses workflow graphs and transforms them iteratively into linear sequences of executable actions. We call these sequences process chains. Our software architecture distributes the process chains to multiple compute nodes in the cloud and oversees their execution. We evaluate our approach by applying it to two practical use cases from the domains of astronomy and engineering. We also compare it with two existing workflow management systems. The evaluation demonstrates that our algorithm is able to execute dynamically changing workflows with cycles and that design and maintenance of complex workflows is easier than with existing solutions. It also shows that our software architecture can run process chains on multiple compute nodes in parallel to significantly speed up the workflow execution. An implementation of our algorithm and the software architecture is available with the Steep Workflow Management System that we released under an open-source license. The resources for the first practical use case are also available as open source for reproduction. |
| Freie Schlagworte: | Scientific workflow management systems, Workflow scheduling, Cloud computing, Distributed systems |
| ID-Nummer: | Artikel-ID: 25 |
| Status: | Verlagsversion |
| URN: | urn:nbn:de:tuda-tuprints-236167 |
| Sachgruppe der Dewey Dezimalklassifikatin (DDC): | 000 Allgemeines, Informatik, Informationswissenschaft > 004 Informatik |
| Fachbereich(e)/-gebiet(e): | 20 Fachbereich Informatik 20 Fachbereich Informatik > Graphisch-Interaktive Systeme 20 Fachbereich Informatik > Fraunhofer IGD |
| Hinterlegungsdatum: | 08 Apr 2024 12:24 |
| Letzte Änderung: | 11 Apr 2024 09:47 |
| PPN: | |
| Export: | |
| Suche nach Titel in: | TUfind oder in Google |
Verfügbare Versionen dieses Eintrags
- Executing cyclic scientific workflows in the cloud. (deposited 08 Apr 2024 12:24) [Gegenwärtig angezeigt]
 |
Frage zum Eintrag |
Optionen (nur für Redakteure)
 |
Redaktionelle Details anzeigen |
Drucken
Impressum