Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form

How well do raters agree on the development stage of Caenorhabditis elegans?

Ferguson, AA and Bilonick, RA and Buchanich, JM and Marsh, GM and Fisher, AL (2015) How well do raters agree on the development stage of Caenorhabditis elegans? PLoS ONE, 10 (7).

Published Version
Available under License : See the attached license file.

Download (1MB)
[img] Plain Text (licence)
Available under License : See the attached license file.

Download (1kB)


The assessment of inter-rater reliability is a topic that is infrequently addressed in Caenorhabditis elegans research, despite the existence of sophisticated statistical methods and the strong interest in the field in obtaining reliable and accurate data. This study applies statistical modeling as a robust means of analyzing the performance of worm researchers measuring the stage of worm development in terms of the two independent factors that comprise "agreement", which are (1) accuracy, representing trueness, a lack of systematic differences, or lack of bias, and (2) precision, representing reliability or the extent to which random differences are small. In our study, multiple raters assessed the same sample of worms to determine the developmental stage of each animal, and we collected data linking each scorer with their assessment for each worm. To describe the agreement of the raters, we developed a structural equation model with latent variables and thresholds, which assumes that all the raters are jointly scoring each worm. This common factor model separately quantifies the two aspects of agreement. The stage-specific thresholds examine accuracy and characterize the relative biases of each rater during the scoring process. The factor loadings for each rater examine the precision and characterizes the random error of the rater. Within our group, we found that the overall agreement was good, while certain adjustments in particular raters would have decreased systematic differences. Hence, the use of developmental stage as an experimental outcome can be both accurate and precise.


Social Networking:
Share |


Item Type: Article
Status: Published
CreatorsEmailPitt UsernameORCID
Ferguson, AAaaf6@pitt.eduAAF6
Bilonick, RArab45@pitt.eduRAB45
Buchanich, JMjeanine@pitt.eduJEANINE
Marsh, GMgmarsh@pitt.eduGMARSH
Fisher, AL
ContributionContributors NameEmailPitt UsernameORCID
Date: 14 July 2015
Date Type: Publication
Access Restriction: No restriction; Release the ETD for access worldwide immediately.
Journal or Publication Title: PLoS ONE
Volume: 10
Number: 7
DOI or Unique Handle: 10.1371/journal.pone.0132365
Institution: University of Pittsburgh
Schools and Programs: School of Public Health > Biostatistics
School of Public Health > Epidemiology
School of Dental Medicine > Dental Science
School of Medicine > Ophthalmology
Refereed: Yes
Date Deposited: 23 Aug 2016 14:21
Last Modified: 30 Mar 2021 13:55


Monthly Views for the past 3 years

Plum Analytics

Actions (login required)

View Item View Item