Shah, Aamer ; Kuo, Chih-Song ; Nomura, Akihiro ; Matsuoka, Satoshi ; Wolf, Felix (2019)
How File-access Patterns Influence the Degree of I/O Interference between Cluster Applications.
In: Supercomputing Frontiers and Innovations, 6 (2)
doi: 10.14529/jsfi190203
Artikel, Bibliographie
Kurzbeschreibung (Abstract)
On large-scale clusters, tens to hundreds of applications can simultaneously access a parallel file system, leading to contention and, in its wake, to degraded application performance. In this article, we analyze the influence of file-access patterns on the degree of interference. As it is by experience most intrusive, we focus our attention on write-write contention. We observe considerable differences among the interference potentials of several typical write patterns. In particular, we found that if one parallel program writes large output files while another one writes small checkpointing files, then the latter is slowed down when the checkpointing files are small enough and the former is vice versa. Moreover, applications with a few processes writing large output files already can significantly hinder applications with many processes from checkpointing small files. Such effects can seriously impact the runtime of real applications—up to a factor of five in one instance. Our insights and measurement techniques offer an opportunity to automatically classify the interference potential between applications and to adjust scheduling decisions accordingly.
| Typ des Eintrags: | Artikel |
|---|---|
| Erschienen: | 2019 |
| Autor(en): | Shah, Aamer ; Kuo, Chih-Song ; Nomura, Akihiro ; Matsuoka, Satoshi ; Wolf, Felix |
| Art des Eintrags: | Bibliographie |
| Titel: | How File-access Patterns Influence the Degree of I/O Interference between Cluster Applications |
| Sprache: | Englisch |
| Publikationsjahr: | 18 Juli 2019 |
| Ort: | Chelyabinsk |
| Verlag: | South Ural State University |
| Titel der Zeitschrift, Zeitung oder Schriftenreihe: | Supercomputing Frontiers and Innovations |
| Jahrgang/Volume einer Zeitschrift: | 6 |
| (Heft-)Nummer: | 2 |
| DOI: | 10.14529/jsfi190203 |
| Kurzbeschreibung (Abstract): | On large-scale clusters, tens to hundreds of applications can simultaneously access a parallel file system, leading to contention and, in its wake, to degraded application performance. In this article, we analyze the influence of file-access patterns on the degree of interference. As it is by experience most intrusive, we focus our attention on write-write contention. We observe considerable differences among the interference potentials of several typical write patterns. In particular, we found that if one parallel program writes large output files while another one writes small checkpointing files, then the latter is slowed down when the checkpointing files are small enough and the former is vice versa. Moreover, applications with a few processes writing large output files already can significantly hinder applications with many processes from checkpointing small files. Such effects can seriously impact the runtime of real applications—up to a factor of five in one instance. Our insights and measurement techniques offer an opportunity to automatically classify the interference potential between applications and to adjust scheduling decisions accordingly. |
| Freie Schlagworte: | DFG|320898076, DoE|DE-SC0015524, DFG, 320898076, DoE, SC0015524 |
| Fachbereich(e)/-gebiet(e): | 20 Fachbereich Informatik 20 Fachbereich Informatik > Parallele Programmierung |
| Hinterlegungsdatum: | 20 Mär 2024 13:43 |
| Letzte Änderung: | 20 Mär 2024 13:43 |
| PPN: | |
| Export: | |
| Suche nach Titel in: | TUfind oder in Google |
 |
Frage zum Eintrag |
Optionen (nur für Redakteure)
 |
Redaktionelle Details anzeigen |
Drucken
Impressum