Please use this identifier to cite or link to this item: http://hdl.handle.net/10125/48504

Faculty Confidence and Engagement Survey: Validation and Analysis with the Rasch Model

File SizeFormat 
Hill2013_05_AIR_Forum_FaCES_RaschAnalysis.pdf721.33 kBAdobe PDFView/Open

Item Summary

Title: Faculty Confidence and Engagement Survey: Validation and Analysis with the Rasch Model
Authors: Hill, Yao Zhang
Keywords: Rasch models
faculty confidence and engagement
survey analysis method
evaluation of faculty professional development
Issue Date: May 2013
Citation: Hill, Y. Z. (2013, May). Faculty confidence and engagement survey: Validation and analysis with the Rasch model. Paper presented at the Association for Institutional Research Annual Forum, Long Beach, CA.
Abstract: This presentation explains how the Item Response Theory Rasch model was used to validate and analyze the Faculty Confidence and Engagement Survey (FaCES) administered at a large urban community college. The presentaton illustrates the advantages of using the Rasch model to analyze survey ratings, the major steps in the analysis, and the important statistics to be used for instrument revision, interpretation, and reporting purposes. The presenter shares major findings related to the survey’s construct validity and findings on faculty/staff’s level of confidence and engagement in different aspects.
Description: The study explored using the Item Response Theory (IRT) Rasch model to validate and analyze the Faculty Confidence and Engagement Survey (FaCES) used in a large urban community college. The FaCES survey was developed collaboratively by the professional development leaders on campus. It aimed to gauge faculty/staff’s confidence and engagement in areas such as teaching practices, primary professional responsibilities, professional development, collaboration, scholarship, willingness to help and seek help, sense of trust, institutional responsibilities, and etc. The survey utilized 36 five-point Likert scale items. The data used in the analysis came from 194 respondents in the Spring 2012 survey administration.
Using IRT Rasch Model analysis advances the technique in survey rating data validation and analysis for the following reasons:
1. The traditional factor analysis technique in survey construct validation can be misleading because it treats survey ratings as interval scale rather than ordinal scale, and because the resulting factors were treated as equivalent in data interpretation, while statistically they contribute differently to the variance in the data.
2. Rasch IRT analysis allows the evaluators and researchers to discover the “true” ratings, and thus, more accurate answers about these ratings. In Rasch analysis, using probability estimation, an item’s endorsability/difficulty and a person’s level of confidence can be estimated simultaneously using logit scores. The resulting logit scores are on a true interval scale.
3. “Not all items contribute to the same latent trait, and of those items that do, they do not all contribute equally” (Bond & Fox, 2007). The presenter will share the major steps in the Rasch model analysis, and the important statistics that should be used for instrument revision, interpretation, and reporting purposes.
The major findings of the study are as follows: (1) the instrument satisfactorily measured the holistic construct of faculty/staff’s confidence and engagement; (2) the engagement items related to teaching practices and duties/responsibilities within one’s locus of control were more easily endorsed and the items related to scholarship, collaboration, duties/responsibilities outside one’s locus of control, or involving support from administration were more difficult to endorse.
URI/DOI: http://hdl.handle.net/10125/48504
Appears in Collections:Workshops and Presentations



Items in ScholarSpace are protected by copyright, with all rights reserved, unless otherwise indicated.