File(s) under permanent embargo
Resource scheduling and provisioning for processing of dynamic stream workflows under latency constraints
journal contribution
posted on 2023-05-21, 14:39 authored by Alexander Brown, Saurabh GargSaurabh Garg, Erin MontgomeryErin Montgomery, Ujjwal K CThere has been an increasing demand for large-scale processing of stream workflows. Many streaming data sources, such as video and audio, differ in terms of the type and density of information they contain over time, which require dynamic workflow structures where paths change in response to data variations. Additionally, information is often only inferred from examining discrete chunks and hence, cannot be arbitrarily parallelised like other streaming data sources. Stream processes using these data sources often operate as dynamic workflows, where different paths are selected based on prior processing information, relating to differences in the nature of the data itself. To better enable low latency analysis in such scenarios, this work investigates strategies to schedule and provision streaming dynamic workflow scenarios of various complexities and degrees of unpredictability. It is found that, as scenarios become increasingly uncertain and unpredictable, it is better to use simpler strategies which do not make assumptions about workflow structures or completion times.
History
Publication title
Future Generation Computer SystemsVolume
131Pagination
166–182ISSN
0167-739XDepartment/School
School of Information and Communication TechnologyPublisher
Elsevier Science BvPlace of publication
Po Box 211, Amsterdam, Netherlands, 1000 AeRights statement
© 2022 Elsevier B.V. All rights reserved.Repository Status
- Restricted