Inter-Rater Reliability Using SAS: A Practical Guide for Nominal, Ordinal, and Interval Data

Kilem Li Gwet

ISBN 10: 0970806264 / ISBN 13: 9780970806260

Published by Advanced Analytics, LLC 2010-11-15, 2010

About this Item

Seller Inventory # 6666-IUK-9780970806260

Bibliographic Details

• Title: Inter-Rater Reliability Using SAS: A Practical Guide for Nominal, Ordinal, and Interval Data
• Publisher: Advanced Analytics, LLC 2010-11-15
• Publication Date: 2010
• Language: English
• ISBN 10: 0970806264
• ISBN 13: 9780970806260
• Binding: Paperback
• Condition: New

About this title

Synopsis

The primary objective of this book is to show practitioners simple step-by-step approaches for organizing rating data, creating SAS datasets, and using appropriate SAS procedures, or special SAS macro programs to compute various inter-rater reliability coefficients. The author always starts with a brief and non-mathematical description of the agreement coefficients used in this book, before showing how they are calculated with SAS. The non-mathematical description of these coefficients is done using simple numeric examples to show their functionality. The author offers practical SAS solutions for 2 raters as well as for 3 raters and more.
The FREQ procedure of SAS offers the calculation of Cohen's Kappa as an option, when the number of raters is limited to 2. The introduction of this feature is without doubt a very welcome addition to the system. But in addition to offering only Kappa as the only agreement coefficient, the use of FREQ to compute Kappa is full of pitfalls that could easily lead a careless practitioner to wrong results. For example, if one rater does not use one category that another rater has used, SAS does not compute any Kappa at all. This problem is referred to in chapter 1 as the unbalanced-table issue. Even more seriously, if both raters use the same number of different categories, SAS will produce "very wrong" results, because the FREQ procedure will be matching wrong categories to determine agreement. This issue is referred to in chapter 1 as the "Diagonal Issue." There are actually a few other potentially serious problems with weighted Kappa that the author has identified. They are all clearly documented in this book, and a plan for resolving each of them is proposed.

About the Author

KILEM L GWET has a doctorate degree in Statistics from Carleton University. He is a statistical Consultant, a researcher and an instructor. He has over 15 years of experience in various industries, with clients including Booz Allen & Hamilton, American Health Information Management Association (AHIMA), Centers for Medicare & Medicaid Services (CMS), IMPAQ International and many others. He has several publications on Inter-rater and Intra-rater reliability assessment in peer-reviewed journals. He currently lives in Maryland.