Psychometrics in Learning and Development Part II
UCLA doctoral candidate, Tianying (Teanna) Feng, is going to guide us in improving our understanding of educational and social research by introducing psychometrics and its application to Learning and Development.
About the Author:
Teanna is a doctoral student in the Education – Social Research Methodology program and a master’s student in the Statistics program at the University of California, Los Angeles (UCLA). She has been working at UCLA’s National Center for Research on Evaluation, Standards, and Student Testing (CRESST) since 2017, where she contributes to research on educational games designed to facilitate K-12 students’ understanding of STEM content.
Examples of psychometrics in educational settings
Psychometrics plays an important role in the design, administration, and analysis of instruments, including questionnaires and tests, which are ubiquitous in educational settings. For example, results obtained from a psychometric model can tell us about the characteristics of the test (e.g., internal consistency) and/or its individual items (i.e., test or survey questions), which help test developers decide whether they should revise the current set of items. Some characteristics of test items that developers consider include:
- Item Discrimination: How well a test item differentiates students who know the content on the test from those who do not. This means that a student who knows more about the topic should achieve a higher score on the item.
- Item Bias: Which item may cause students with the same level of ability but different characteristics (e.g., culture or gender) to respond differently. It is closely related to the principle of assessment fairness, discussed above.
- Clarity: Which item has answer options that may be unclear.
- Item Difficulty: Which item is too easy or too difficult, in which case the item may not provide an accurate assessment of what the students know.
Examples of psychometrics in the workplace
As with any type of assessment, one should consider whether a given psychometric tool has been rigorously validated, and whether such a tool is indeed being used in a fair, reliable, and valid way. The same concerns are also applicable to the use of psychometrics in the workplace to evaluate the following:
- Predict aspects of employee well-being, such as physical health, mindset, environment, and economic security (Donaldson & Donaldson, 2020).
- Facilitate hiring and development processes (Alloway & Cissel, 2017; Dattner, 2013; Wilkie, 2013).
- Identify counterproductive behavior at work (Rust et al., 2020).
- Aid people analytics (Baer and D’Silva, 2018; Ledet et al., 2020), such as analyzing employee engagement trends and building customer profiles to deliver targeted services.
Other uses of psychometrics focus on the development and validation of questionnaires and tests especially designed to predict work outcomes and organizational fit.
Summary
Testing and measurement are ubiquitous in our everyday lives. This article introduces the concept of psychometrics and provides examples of its applications. The tools of psychometrics, when properly used, can help developers refine their content, educators obtain reliable and valid feedback about student performance, and companies strengthen their hiring, selection, and training processes.
References
Alloway, T. P., & Cissel, H. (2017, April). Psychometric testing in the workplace. The Score. http://www.apadivisions.org/division-5/publications/score/2017/04/psychometric-testing
American Psychological Association (APA). (n.d.). Psychometrics. In APA dictionary of psychology. Retrieved November 14, 2021, from https://dictionary.apa.org/psychometrics
American Educational Research Association (AERA), American Psychological Association (APA), & National Council on Measurement in Education (NCME). (2014). Standards for educational and psychological testing. American Educational Research Association.
Baer, T., & D’Silva, V. (2018, October). ‘All in the mind’: Harnessing psychology and analytics to counter bias and reduce risk. McKinsey & Company. https://www.mckinsey.com/industries/financial-services/our-insights/all-in-the-mind-harnessing-psychology-and-analytics-to-counter-bias-and-reduce-risk
Dattner, B. (2013, September). How to use psychometric testing in hiring. Harvard Business Review. https://hbr.org/2013/09/how-to-use-psychometric-testin
Donaldson, S. I., & Donaldson, S. I. (2020). The positive functioning at work scale: Psychometric assessment, validation, and measurement invariance. Journal of Well-Being Assessment, 4(2), 181-215.
Ledet, E., McNulty, K., Morales, D., & Shandell, M. (2020, October). How to be great at people analytics. McKinsey & Company. https://www.mckinsey.com/business-functions/people-and-organizational-performance/our-insights/how-to-be-great-at-people-analytics
Rust, J., Kosinski, M., & Stillwell, D. (2020). Modern psychometrics: The science of psychological assessment. Routledge.
Salkind, N. J. (2018). Tests & measurement for people who (think they) hate tests & measurement. SAGE.
Wilkie, D. (2013). How reliable are personality tests? The Society for Human Resource Management. https://www.shrm.org/resourcesandtools/hr-topics/employee-relations/pages/how-reliable-are-personality-tests.aspx