Lofgren, Robert Eric
(2005)
Validity Evidence of a Multiple-Choice Test and a Performance Test in an Employment Setting.
Doctoral Dissertation, University of Pittsburgh.
(Unpublished)
Abstract
A selection procedure consisting of both multiple-choice (MC) paper-and pencil and hands on performance assessment (PA) elements was developed for a large consumer products manufacturing company in the southeastern United States for the purpose of ensuring that workers possessed the necessary knowledge, skills, and abilities required for work at a new facility. Two 125-item alternate tests using an MC format and 7 PA exercises were initially developed for the job referred to in the present study as L2/L3 Production Technician. The purpose of this study was to examine the validity evidence for the two alternate multiple-choice (MC) job knowledge tests and seven performance assessment (PA) exercises that were developed for employment selection purposes. The study sample included 432 Form A and 324 Form B examinees who took both the MC test and the PA exercises. Factor analysis results revealed that the same construct, labeled as applied mechanical knowledge, was measured by both the MC tests and the PA exercises. Item and test analysis results supported the use of Form A and Form B as alternate test forms. The decision consistency between the MC tests and the PA exercises did not appear to be sufficient to recommend that either form of the MC test alone could be used to select qualified L2/L3 Production Technicians. The correlations between MC score and PA total score were .627 for Form A and .612 for Form B. As part of a content analysis, subject-matter experts rated a large number of MC items as either having "no relationship" or "small relationship" to the PA exercises. However, subject-matter experts did rate the PA exercises as having a great importance to the job of L2/L3 Production Technician.
Share
| Citation/Export: |
|
| Social Networking: |
|
Details
| Item Type: |
University of Pittsburgh ETD
|
| Status: |
Unpublished |
| Creators/Authors: |
|
| ETD Committee: |
| Title | Member | Email Address | Pitt Username | ORCID |
|---|
| Committee Chair | Lane, Suzanne | | | | | Committee Member | Stone, Clement A | | | | | Committee Member | Irrgang, James J | | | | | Committee Member | Pingel, Louis A | | | |
|
| Date: |
12 December 2005 |
| Date Type: |
Completion |
| Defense Date: |
10 November 2005 |
| Approval Date: |
12 December 2005 |
| Submission Date: |
1 December 2005 |
| Access Restriction: |
No restriction; Release the ETD for access worldwide immediately. |
| Institution: |
University of Pittsburgh |
| Schools and Programs: |
School of Education > Psychology in Education |
| Degree: |
PhD - Doctor of Philosophy |
| Thesis Type: |
Doctoral Dissertation |
| Refereed: |
Yes |
| Uncontrolled Keywords: |
content analysis; validity evidence; skills testing; multiple-choice test; employee selection; performance assessment |
| Other ID: |
http://etd.library.pitt.edu/ETD/available/etd-12012005-124415/, etd-12012005-124415 |
| Date Deposited: |
10 Nov 2011 20:07 |
| Last Modified: |
15 Nov 2016 13:52 |
| URI: |
http://d-scholarship.pitt.edu/id/eprint/9909 |
Metrics
Monthly Views for the past 3 years
Plum Analytics
Actions (login required)
 |
View Item |
![[feed]](/images/feed-icon-32x32.png){kind=icon}