Emerging service orchestration discovery and monitoring

Due to the popularity of web services on the Internet, it is important to have a clear view of their utilization behaviors. Despite asynchronous service invocations and distributed executions can provide better user experience, our views are blurred by out-of-order and fragmented service logs. Researchers have been trying various methods to reveal emerging service orchestration patterns, but nearly all of them have taken deterministic approaches. Hence, they do not natively cater for incomplete data and noises. In this paper, we propose to address these problems by using topic models aiming to reveal service orchestration patterns from sparse service logs. Probabilistic approaches do not only tolerate data defects, but their associated approximation methods also overcome combinatorial explosion. We first investigate the implications of sparsity on topic models. Secondly, we propose an extended time-series form of susceptible-infectious-recovered model to monitor the dynamics of emerging service orchestrations. We quantify their emerging-potential by estimated effective-reproduction-number, which is obtained incrementally by Bayesian parameter estimations. Guided by our proposed emerging-potential measure, one can profile and categorize emerging service orchestration patterns, and generate automated alerts on upcoming consumption peaks. In practice, our model enables service providers to better allocate their resources to meet demands dynamically. While our findings affirm that biterm topic model can be applied to service logs with short and sparse log entries, the effectiveness of our proposed monitoring solutions is also shown by experiments.