Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form

Validity Evidence of a Multiple-Choice Test and a Performance Test in an Employment Setting

Lofgren, Robert Eric (2005) Validity Evidence of a Multiple-Choice Test and a Performance Test in an Employment Setting. Doctoral Dissertation, University of Pittsburgh. (Unpublished)

Primary Text

Download (3MB) | Preview


A selection procedure consisting of both multiple-choice (MC) paper-and pencil and hands on performance assessment (PA) elements was developed for a large consumer products manufacturing company in the southeastern United States for the purpose of ensuring that workers possessed the necessary knowledge, skills, and abilities required for work at a new facility. Two 125-item alternate tests using an MC format and 7 PA exercises were initially developed for the job referred to in the present study as L2/L3 Production Technician. The purpose of this study was to examine the validity evidence for the two alternate multiple-choice (MC) job knowledge tests and seven performance assessment (PA) exercises that were developed for employment selection purposes. The study sample included 432 Form A and 324 Form B examinees who took both the MC test and the PA exercises. Factor analysis results revealed that the same construct, labeled as applied mechanical knowledge, was measured by both the MC tests and the PA exercises. Item and test analysis results supported the use of Form A and Form B as alternate test forms. The decision consistency between the MC tests and the PA exercises did not appear to be sufficient to recommend that either form of the MC test alone could be used to select qualified L2/L3 Production Technicians. The correlations between MC score and PA total score were .627 for Form A and .612 for Form B. As part of a content analysis, subject-matter experts rated a large number of MC items as either having "no relationship" or "small relationship" to the PA exercises. However, subject-matter experts did rate the PA exercises as having a great importance to the job of L2/L3 Production Technician.


Social Networking:
Share |


Item Type: University of Pittsburgh ETD
Status: Unpublished
CreatorsEmailPitt UsernameORCID
Lofgren, Robert Ericrelst18@pitt.eduRELST18
ETD Committee:
TitleMemberEmail AddressPitt UsernameORCID
Committee ChairLane, Suzanne
Committee MemberStone, Clement A
Committee MemberIrrgang, James J
Committee MemberPingel, Louis A
Date: 12 December 2005
Date Type: Completion
Defense Date: 10 November 2005
Approval Date: 12 December 2005
Submission Date: 1 December 2005
Access Restriction: No restriction; Release the ETD for access worldwide immediately.
Institution: University of Pittsburgh
Schools and Programs: School of Education > Psychology in Education
Degree: PhD - Doctor of Philosophy
Thesis Type: Doctoral Dissertation
Refereed: Yes
Uncontrolled Keywords: content analysis; validity evidence; skills testing; multiple-choice test; employee selection; performance assessment
Other ID:, etd-12012005-124415
Date Deposited: 10 Nov 2011 20:07
Last Modified: 15 Nov 2016 13:52


Monthly Views for the past 3 years

Plum Analytics

Actions (login required)

View Item View Item