Please use this identifier to cite or link to this item: http://hdl.handle.net/10125/62291

Scheduling Heuristics For Executing Scientific Workflows On Homogeneous Clusters With Globallyand Locally-Accessible Persistent Storage

File Size Format  
2018-08-ms-pandey.pdf 2.8 MB Adobe PDF View/Open

Item Summary

Title:Scheduling Heuristics For Executing Scientific Workflows On Homogeneous Clusters With Globallyand Locally-Accessible Persistent Storage
Authors:Pandey, Suraj
Contributors:Computer Science (department)
Keywords:Scientic Workows
DAG
Task Replication
Local/Global Storages
Heuristics
Date Issued:Aug 2018
Publisher:University of Hawaiʻi at Mānoa
Abstract:Many applications in science and engineering today are structured as scientic workows, i.e.,
task graphs with data dependencies between graphs, where tasks are implemented as standalone
executables and data dependencies are via les read/written from/to stable storage.
For many relevant application domains, these workows are both large and data-intensive.
Therefore, optimizing data accesses is crucial for ecient scientic workow executions.
Typical HPC (High Performance Computing) platforms used to run scientic workows
are commodity clusters, in which each compute node has access to private, small, highbandwidth
\local" storage, and to shared, large, low-bandwidth \global" storage. To date,
production Workow Management Systems (WMs), software infrastructures for executing
workows in practice, only use global storage. There is thus an opportunity to improve
workow performance by exploiting local storage. The diculty, however, is twofold. First,
the capacity of local storage is limited and often allows holding only a few workow les.
Second, storing data in local storage reduces parallelism because storage is private to a single
node. In this thesis, we design scheduling heuristics to orchestrate workow execution in this
context, with the objective of minimizing workow execution time. These heuristics decide
which les should be stored in which level of storage (local or global) and replicate tasks so
as to increase the availability of data across compute nodes and thus maintain parallelism.
We implement a simulation framework to evaluate and drive the design of these heuristics
using both real-world and synthetic workow congurations. We also implement a software
prototype for using these heuristics on HPC platforms. From experimental results obtained
in simulation and on an actual HPC cluster we are able to evaluate the relative merit of
our heuristics and draw conclusions about the most promising approaches and remaining
challenges.
Description:M.S. Thesis. University of Hawaiʻi at Mānoa 2018.
URI:http://hdl.handle.net/10125/62291
Rights:All UHM dissertations and theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission from the copyright owner.
Appears in Collections: M.S. - Computer Science


Please email libraryada-l@lists.hawaii.edu if you need this content in ADA-compliant format.

Items in ScholarSpace are protected by copyright, with all rights reserved, unless otherwise indicated.