Resources on multi-method evaluation in research and development of interactive intelligent systems

A single evaluation method or measure is not able to evaluate all relevant aspects in a complex setting where a multitude of stakeholders are involved. We reason that employing a multi-method evaluation, where multiple evaluation methods or measures are combined and integrated, allows for getting a richer picture and prevents blind spots in the evaluation outcome.

We advocate for multi-method evaluation. On this website, we collect resources that help to apply multi-method evaluation in research and development.

This collection is meant to grow over time and we are happy to share your resources and discuss best practices. Don’t hesitate to contact us!

About

Christine Bauer

Photo of Christine Bauer

Christine Bauer is EXDIGIT Professor of Interactive Intelligent Systems at the Department of Artificial Intelligence and Human Interfaces (AIHI) at the Paris Lodron University Salzburg (PLUS), Austria. Her research activities center on interactive intelligent systems. Thereby, she takes a human-centered perspective, where technology follows humans’ and the society’s needs. Her research and teaching activities are driven by her interdisciplinary background. She holds a Doctoral degree in Social and Economic Sciences, a Master degree in Business Informatics, and a Diploma degree in International Business Administration. In addition, she pursued studies in jazz saxophone. She has co-authored more than 120 papers, and holds several best paper awards as well as awards for her reviewing activities. Furthermore, she is an Elise Richter laureate and received a grant for the project “Fine-grained Culture-aware Music Recommender Systems” (2017–2020) sponsored by Austrian Science Fund (FWF).

Eva Zangerle

Photo of Eva Zangerle

Eva Zangerle is an Assistant Professor at the University of Innsbruck at the research group for Databases and Information Systems (Department of Computer Science), Austria. She earned her master’s degree in Computer Science at the University of Innsbruck and subsequently pursued her Ph.D. from the University of Innsbruck in the field of recommender systems for collaborative social media platforms. Her main research interests are within the fields of music recommender systems, social media analysis and information retrieval. Over the last years, she has combined these three fields of research and investigated context-aware music recommender systems based on data retrieved from social media platforms aiming to exploit new sources of information for recommender systems. She was awarded a Postdoctoral Fellowship for Overseas Researchers from the Japan Society for the Promotion of Science allowing her to make a short-term research stay at the Ritsumeikan University in Kyoto.

Contact

Dr. Christine Bauer
Paris Lodron University Salzburg
Faculty of Digital and Analytical Sciences
Department of Artificial Intelligence and Human Interfaces (AIHI)
christine.bauer [at] plus.ac.at
https://christinebauer.eu/

Dr. Eva Zangerle
University of Innsbruck
Department of Computer Science
Databases and Information Systems
eva.zangerle [at] uibk.ac.at
https://www.evazangerle.at