TU Darmstadt / ULB / TUbiblio

Executing cyclic scientific workflows in the cloud

Krämer, Michel ; Würz, Hendrik M. ; Altenhofen, Christian (2021)
Executing cyclic scientific workflows in the cloud.
In: Journal of Cloud Computing, 10 (1)
doi: 10.1186/s13677-021-00229-7
Artikel, Bibliographie

Dies ist die neueste Version dieses Eintrags.

Kurzbeschreibung (Abstract)

We present an algorithm and a software architecture for a cloud-based system that executes cyclic scientific workflows whose structure may change during run time. Existing approaches either rely on workflow definitions based on directed acyclic graphs (DAGs) or require workarounds to implement cyclic structures. In contrast, our system supports cycles natively, avoids workarounds, and as such reduces the complexity of workflow modelling and maintenance. Our algorithm traverses workflow graphs and transforms them iteratively into linear sequences of executable actions. We call these sequences process chains. Our software architecture distributes the process chains to multiple compute nodes in the cloud and oversees their execution. We evaluate our approach by applying it to two practical use cases from the domains of astronomy and engineering. We also compare it with two existing workflow management systems. The evaluation demonstrates that our algorithm is able to execute dynamically changing workflows with cycles and that design and maintenance of complex workflows is easier than with existing solutions. It also shows that our software architecture can run process chains on multiple compute nodes in parallel to significantly speed up the workflow execution. An implementation of our algorithm and the software architecture is available with the Steep Workflow Management System that we released under an open-source license. The resources for the first practical use case are also available as open source for reproduction.

Typ des Eintrags: Artikel
Erschienen: 2021
Autor(en): Krämer, Michel ; Würz, Hendrik M. ; Altenhofen, Christian
Art des Eintrags: Bibliographie
Titel: Executing cyclic scientific workflows in the cloud
Sprache: Englisch
Publikationsjahr: 6 April 2021
Verlag: Springer Nature
Titel der Zeitschrift, Zeitung oder Schriftenreihe: Journal of Cloud Computing
Jahrgang/Volume einer Zeitschrift: 10
(Heft-)Nummer: 1
DOI: 10.1186/s13677-021-00229-7
Zugehörige Links:
Kurzbeschreibung (Abstract):

We present an algorithm and a software architecture for a cloud-based system that executes cyclic scientific workflows whose structure may change during run time. Existing approaches either rely on workflow definitions based on directed acyclic graphs (DAGs) or require workarounds to implement cyclic structures. In contrast, our system supports cycles natively, avoids workarounds, and as such reduces the complexity of workflow modelling and maintenance. Our algorithm traverses workflow graphs and transforms them iteratively into linear sequences of executable actions. We call these sequences process chains. Our software architecture distributes the process chains to multiple compute nodes in the cloud and oversees their execution. We evaluate our approach by applying it to two practical use cases from the domains of astronomy and engineering. We also compare it with two existing workflow management systems. The evaluation demonstrates that our algorithm is able to execute dynamically changing workflows with cycles and that design and maintenance of complex workflows is easier than with existing solutions. It also shows that our software architecture can run process chains on multiple compute nodes in parallel to significantly speed up the workflow execution. An implementation of our algorithm and the software architecture is available with the Steep Workflow Management System that we released under an open-source license. The resources for the first practical use case are also available as open source for reproduction.

Freie Schlagworte: Cloud computing, Distributed systems
ID-Nummer: Artikel-ID: 25
Zusätzliche Informationen:

Art.No.: 25

Fachbereich(e)/-gebiet(e): 20 Fachbereich Informatik
20 Fachbereich Informatik > Graphisch-Interaktive Systeme
20 Fachbereich Informatik > Fraunhofer IGD
Hinterlegungsdatum: 25 Mai 2021 07:59
Letzte Änderung: 11 Apr 2024 09:48
PPN:
Export:
Suche nach Titel in: TUfind oder in Google

Verfügbare Versionen dieses Eintrags

Frage zum Eintrag Frage zum Eintrag

Optionen (nur für Redakteure)
Redaktionelle Details anzeigen Redaktionelle Details anzeigen