Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form

How well do raters agree on the development stage of Caenorhabditis elegans?

Ferguson, AA and Bilonick, RA and Buchanich, JM and Marsh, GM and Fisher, AL (2015) How well do raters agree on the development stage of Caenorhabditis elegans? PLoS ONE, 10 (7).

[img]
Preview
PDF
Published Version
Available under License : See the attached license file.

Download (1MB)
[img] Plain Text (licence)
Available under License : See the attached license file.

Download (1kB)

Abstract

© 2015, Public Library of Science. All rights reserved.. This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication. The assessment of inter-rater reliability is a topic that is infrequently addressed in Caenorhabditis elegans research, despite the existence of sophisticated statistical methods and the strong interest in the field in obtaining reliable and accurate data. This study applies statistical modeling as a robust means of analyzing the performance of worm researchers measuring the stage of worm development in terms of the two independent factors that comprise "agreement", which are (1) accuracy, representing trueness, a lack of systematic differences, or lack of bias, and (2) precision, representing reliability or the extent to which random differences are small. In our study, multiple raters assessed the same sample of worms to determine the developmental stage of each animal, and we collected data linking each scorer with their assessment for each worm. To describe the agreement of the raters, we developed a structural equation model with latent variables and thresholds, which assumes that all the raters are jointly scoring each worm. This common factor model separately quantifies the two aspects of agreement. The stage-specific thresholds examine accuracy and characterize the relative biases of each rater during the scoring process. The factor loadings for each rater examine the precision and characterizes the random error of the rater. Within our group, we found that the overall agreement was good, while certain adjustments in particular raters would have decreased systematic differences. Hence, the use of developmental stage as an experimental outcome can be both accurate and precise.


Share

Citation/Export:
Social Networking:
Share |

Details

Item Type: Article
Status: Published
Creators/Authors:
CreatorsEmailPitt UsernameORCID
Ferguson, AAaaf6@pitt.eduAAF6
Bilonick, RArab45@pitt.eduRAB45
Buchanich, JMjeanine@pitt.eduJEANINE
Marsh, GMgmarsh@pitt.eduGMARSH
Fisher, AL
Contributors:
ContributionContributors NameEmailPitt UsernameORCID
EditorDupuy, DenisUNSPECIFIEDUNSPECIFIEDUNSPECIFIED
Date: 14 July 2015
Date Type: Publication
Access Restriction: No restriction; Release the ETD for access worldwide immediately.
Journal or Publication Title: PLoS ONE
Volume: 10
Number: 7
DOI or Unique Handle: 10.1371/journal.pone.0132365
Institution: University of Pittsburgh
Schools and Programs: Graduate School of Public Health > Biostatistics
Graduate School of Public Health > Epidemiology
School of Dental Medicine > Dental Science
School of Medicine > Ophthalmology
Refereed: Yes
Date Deposited: 23 Aug 2016 14:21
Last Modified: 02 Feb 2019 16:56
URI: http://d-scholarship.pitt.edu/id/eprint/28421

Metrics

Monthly Views for the past 3 years

Plum Analytics

Altmetric.com


Actions (login required)

View Item View Item