Predicting Information Retrieval Performance (Synthesis Lectures on Information Concepts, Retrieval, and Services)

Information Retrieval performance measures are usually retrospective in nature, representing the effectiveness of an experimental process. However, in the sciences, phenomena may be predicted, given parameter values of the system. After developing a measure that can be applied retrospectively or can be predicted, performance of a system using a single term can be predicted given several different types of probabilistic distributions. Information Retrieval performance can be predicted with multiple terms, where statistical dependence between terms exists and is understood. These predictive models may be applied to realistic problems, and then the results may be used to validate the accuracy of the methods used. The application of metadata or index labels can be used to determine whether or not these features should be used in particular cases. Linguistic information, such as part-of-speech tag information, can increase the discrimination value of existing terminology and can be studied predictively.

This work provides methods for measuring performance that may be used predictively. Means of predicting these performance measures are provided, both for the simple case of a single term in the query and for multiple terms. Methods of applying these formulae are also suggested.

Robert Losee has been a professor at the University of North Carolina at Chapel Hill's School of Information and Library Science since 1986, after receiving a Ph.D. from the University of Chicago. He has taught courses in Information Retrieval, including an introductory graduate course, and an advanced Artificial Intelligence for Information Retrieval course. He has also taught courses in Information Theory, including a doctoral seminar in the area. His most recent monograph is Information from Processes: About the Nature of Information Creation, Use, and Representation (Springer, 2012).

Gary Marchionini is the Cary C. Boshamer Professor of Information Science in the School of Information and Library Science at the University of North Carolina at Chapel Hill. His Ph.D. is from Wayne State University in mathematics education with an emphasis on educational computing. His research interests are in information seeking in electronic environments, digital libraries, human-computer interaction, digital government and information technology policy. He has had grants or contracts from the National Science Foundation, the U.S. Department of Education, the Council on Library Resources, the National Library of Medicine, the Library of Congress, the Kellogg Foundation, and NASA, among others. He was the Conference Chair for the 1996 ACM Digital Library Conference and program chair for the 2002 ACM-IEEE Joint Conference on Digital Libraries. He is editor-in-chief for ACM Transactions on Information Systems and serves on the editorial boards of a dozen scholarly journals. He has published more than 150 articles, chapters, and conference papers in the information science, computer science, and education literatures. He founded the Interaction Design Laboratory at UNC-CH.