Game Theory for Data Science: Eliciting Truthful Information (Synthesis Lectures on Artificial Intelligence and Machine Learning)

Intelligent systems often depend on data provided by information agents, for example, sensor data or crowdsourced human computation. Providing accurate and relevant data requires costly effort that agents may not always be willing to provide. Thus, it becomes important not only to verify the correctness of data, but also to provide incentives so that agents that provide high-quality data are rewarded while those that do not are discouraged by low rewards.

We cover different settings and the assumptions they admit, including sensing, human computation, peer grading, reviews, and predictions. We survey different incentive mechanisms, including proper scoring rules, prediction markets and peer prediction, Bayesian Truth Serum, Peer Truth Serum, Correlated Agreement, and the settings where each of them would be suitable. As an alternative, we also consider reputation mechanisms. We complement the game-theoretic analysis with practical examples of applications in prediction platforms, community sensing, and peer grading.

Boi Faltings is a full professor at École Polytechnique Fédérale de Lausanne (EPFL) and has worked in AI since 1983. He is one of the pioneers on the topic of mechanisms for truthful information elicitation, with the first work dating back to 2003. He has taught AI and multiagent systems to students at EPFL for 28 years. He is a fellow of AAAI and ECCAI and has served on program committee and editorial boards of the major conferences and journals in Artificial Intelligence.