Semantic Web Services: Advancement through Evaluation

Over the last decade, a great amount of effort and resources have been invested in the development of Semantic Web Service (SWS) frameworks. Numerous description languages, frameworks, tools, and matchmaking and composition algorithms have been proposed. Nevertheless, when faced with a real-world problem, it is still very hard to decide which of these different approaches to use.

In this book, the editors present an overall overview and comparison of the main current evaluation initiatives for SWS. The presentation is divided into four parts, each referring to one of the evaluation initiatives. Part I covers the long-established first two tracks of the Semantic Service Selection (S3) Contest – the OWL-S matchmaker evaluation and the SAWSDL matchmaker evaluation. Part II introduces the new S3 Jena Geography Dataset (JGD) cross evaluation contest. Part III presents the Semantic Web Service Challenge. Lastly, Part IV reports on the semantic aspects of the Web Service Challenge. The introduction to each part provides an overview of the evaluation initiative and overall results for its latest evaluation workshops. The following chapters in each part, written by the participants, detail their approaches, solutions and lessons learned.

This book is aimed at two different types of readers. Researchers on SWS technology receive an overview of existing approaches in SWS with a particular focus on evaluation approaches; potential users of SWS technologies receive a comprehensive summary of the respective strengths and weaknesses of current systems and thus guidance on factors that play a role in evaluation.

M. Brian Blake is a Professor of Computer Science and Associate Dean for Research in the College of Engineering at the University of Notre Dame, IL, USA.  His research has focused on internet computing, service-oriented computing, and agent-based workflow systems.  He was the founder of the WS Challenge, which concentrates on evaluating the software engineering quality of service composition systems.

Liliana Cabral is a Research Fellow at the Open University, UK. She developed technologies and presented a number of tutorials on Semantic Web Services in related conferences, and has also been a member of several projects from the European Commission concerning Semantic Web Services. Liliana is currently the chair of the SWS Challenge initiative and a member of the SEALS (Semantic Evaluation at Large Scale) project (EC FP7).

Birgitta König-Ries holds the Heinz-Nixdorf Endowed Chair for Distributed Information Systems at the University of Jena, Germany. Prior to this she was with the Technical University of Munich, Florida International University, the University of Louisiana at Lafayette, and the University of Karlsruhe. Her group is working on Semantic Web Services and on portal technology, with a special focus on the evaluation of SWS frameworks.

Ulrich Küster has received a PhD on the evaluation of Semantic Web Service technology from Friedrich-Schiller-University Jena, Germany. He co-chaired the SWS Challenge initiative for several years, served in the steering committee and as a track chair for the S3 Contest initiative and has held tutorials about the evaluation of semantic technologies.

David Martin is a Software Engineering Manager in the Siri team at Apple (Cupertino, California, USA), and previously was a Senior Computer Scientist in the Artificial Intelligence Center of SRI International. His research areas include agent-based systems, Semantic Web and Semantic Web Services, intelligent assistance, andknowledge-based systems engineering. David was a Principal Investigator for DARPA's DAML program that created the Semantic Web initiative, and had a leading role in the development of OWL for Services (OWL-S) and the Semantic Web Services Language (SWSL). He has served as an organizer of many workshops related to Web Services, including the combined Service Matching and Resource Retrieval and Semantic Service Selection (SMR2 / S3) series of workshops and the first SWS Challenge workshop.