There has been an increasing demand for large-scale processing of stream workflows. Many streaming data sources, such as video and audio, differ in terms of the type and density of information they contain over time, which require dynamic workflow structures where paths change in response to data variations. Additionally, information is often only inferred from examining discrete chunks and hence, cannot be arbitrarily parallelised like other streaming data sources. Stream processes using these data sources often operate as dynamic workflows, where different paths are selected based on prior processing information, relating to differences in the nature of the data itself. To better enable low latency analysis in such scenarios, this work investigates strategies to schedule and provision streaming dynamic workflow scenarios of various complexities and degrees of unpredictability. It is found that, as scenarios become increasingly uncertain and unpredictable, it is better to use simpler strategies which do not make assumptions about workflow structures or completion times.

Item Type:	Refereed Article
Keywords:	workflow scheduling, resource provisioning, stream processing, stream workflows
Research Division:	Information and Computing Sciences
Research Group:	Distributed computing and systems software
Research Field:	Distributed systems and algorithms
Objective Division:	Plant Production and Plant Primary Products
Objective Group:	Forestry
Objective Field:	Native forests
UTAS Author:	Brown, A (Mr Alexander Brown)
UTAS Author:	Garg, S (Dr Saurabh Garg)
UTAS Author:	Montgomery, J (Dr James Montgomery)
UTAS Author:	K C, U (Mr Ujjwal K C)
ID Code:	154080
Year Published:	2022
Deposited By:	Information and Communication Technology
Deposited On:	2022-10-27
Last Modified:	2023-01-10
Downloads:	0