A Case Study on Multi-Criteria Optimization of an Event Detection Software under Limited Budgets

Zaefferer, M.1, a; Bartz-Beielstein,T.1, b; Naujoks, B.1, c; Wagner, T.2, d; Emmerich, M.3, e

Fakultät für Informatik und Ingenieurwissenschaften, Fachhochschule Köln, 51643 Gummersbach
Institut für Spanende Fertigung, Technische Universität Dortmund, Baroper Str. 303, 44227 Dortmund
Leiden Institute of Advanced Computer Science (LIACS), Universiteit Leiden, 2333 CA Leiden, The Netherlands

a) martin.zaefferer@fh-koeln.de; b) bartz@gm.fh-koeln.de; c) boris.naujoks@fh-koeln.de; d) wagner@isf.de; e) emmerich@liacs.nl


Several methods were developed to solve cost-extensive multi-criteria optimization problems by reducing the number of function evaluations by means of surrogate optimization. In this study, we apply different multi-criteria surrogate optimization methods to improve (tune) an event-detection software for water-quality monitoring. For tuning two important parameters of this software, four state-of-the-art methods are compared: S-Metric-Selection Efficient Global Optimization (SMSEGO), S-Metric-Expected Improvement for Efficient Global Optimization SExI-EGO, Euclidean Distance based Expected Improvement Euclid-EI (here referred to as MEI-SPOT due to its implementation in the Sequential Parameter Optimization Toolbox SPOT) and a multicriteria approach based on SPO (MSPOT). Analyzing the performance of the different methods provides insight into the working-mechanisms of cutting-edge multi-criteria solvers. As one of the approaches, namely MSPOT, does not consider the prediction variance of the surrogate model, it is of interest whether this can lead to premature convergence on the practical tuning problem. Furthermore, all four approaches will be compared to a simple SMS-EMOA to validate that the use of surrogate models is justified on this problem.


Sequential Parameter Optimization (SPO), Efficient Global Optimization (EGO), Surrogate Models, Hypervolume Indicator, Multi-Objective Optimization, Event Detection, Water-Quality Monitoring


In: Proceedings of the 7th International Conference Evolutionary Multi-Criterion Optimization (EMO 2013), 19.3.-22.3. 2013, Sheffield, UK, Pursehouse, R.; Fleming, P. J.; Fonseca, C. M.; Greco, S.; Shaw, J. (Hrsg.), ISBN 978-3-642-37139-4, S. 756-770, doi: 10.1007/978-3-642-37140-0_56