A single evaluation method or measure is not able to evaluate all relevant aspects in a complex setting where a multitude of stakeholders are involved. We reason that employing a multi-method evaluation, where multiple evaluation methods or measures are combined and integrated, allows for getting a richer picture and prevents blind spots in the evaluation outcome.
This collection is meant to grow over time and we are happy to share your resources and discuss best practices. Don’t hesitate to contact us!
Special Issue ‘Perspectives on Recommender Systems Evaluation’ in ACM Transactions on Recommender Systems.
PERSPECTIVES 2022: The workshop ‘Perspectives on the Evaluation of Recommender Systems’ is back at RecSys 2022.
We am happy to announce that we – Eva Zangerle, Christine Bauer, and Alan Said – will co-organize the workshop Perspectives on the Evaluation of Recommender Systems at RecSys 2021.
Christine Bauer will hold a tutorial on Multi-Method Evaluation of Adaptive Systems at UMAP 2021. UMAP – including the tutorial – will be held fully online.
While CHIIR 2020 had unfortunately to be cancelled due to current global situation with Covid-19, the paper accompanying the tutorial on multi-method evaluation (that was supposed to be held at CHIIR 2020) is published:
Christine Bauer is an Assistant Professor at the Department of Information and Computing Sciences at Utrecht University, The Netherlands. She is an Elise Richter laureate and received a grant for the project “Fine-grained Culture-aware Music Recommender Systems” (2017–2020) sponsored by Austrian Science Fund (FWF). Her research activities center on interactive intelligent systems. Thereby, she takes a human-centered perspective, where technology follows humans’ and the society’s needs. Her research and teaching activities are driven by her interdisciplinary background. She holds a Doctoral degree in Social and Economic Sciences, a Master degree in Business Informatics, and a Diploma degree in International Business Administration. In addition, she pursued studies in jazz saxophone. She has authored more than 100 papers in refereed journals and conference proceedings, and holds several best paper awards as well as awards for her reviewing activities.
Eva Zangerle is an Assistant Professor at the University of Innsbruck at the research group for Databases and Information Systems (Department of Computer Science), Austria. She earned her master’s degree in Computer Science at the University of Innsbruck and subsequently pursued her Ph.D. from the University of Innsbruck in the field of recommender systems for collaborative social media platforms. Her main research interests are within the fields of music recommender systems, social media analysis and information retrieval. Over the last years, she has combined these three fields of research and investigated context-aware music recommender systems based on data retrieved from social media platforms aiming to exploit new sources of information for recommender systems. She was awarded a Postdoctoral Fellowship for Overseas Researchers from the Japan Society for the Promotion of Science allowing her to make a short-term research stay at the Ritsumeikan University in Kyoto.
Workshop on the Perspectives on the Evaluation of Recommender Systems (PERSPECTIVES 2022) at RecSys 2022.
Workshop on the Perspectives on the Evaluation of Recommender Systems (PERSPECTIVES 2021) at RecSys 2021.
Tutorial on Multi-Method Evaluation of Adaptive Systems at UMAP 2021.