Psychometrics in Learning and Development
UCLA doctoral candidate, Tianying (Teanna) Feng, is going to guide us in improving our understanding of educational and social research by introducing psychometrics and its application to Learning and Development.
About the Author:
Teanna is a doctoral student in the Education – Social Research Methodology program and a master’s student in the Statistics program at the University of California, Los Angeles (UCLA). She has been working at UCLA’s National Center for Research on Evaluation, Standards, and Student Testing (CRESST) since 2017, where she contributes to research on educational games designed to facilitate K-12 students’ understanding of STEM content.
Psychometrics: What is it, and how do we use it in Learning & Development? by Teanna Feng
Whether it is a personality test used in hiring, a standardized test for college entrance, a survey asking for public opinion, we all have experienced some form of testing—both formal and informal—in our lives. The development, administration, and analysis of many tests, especially in high-stake settings, requires psychometrics. Let’s start with a working definition of psychometrics and then explore examples of how psychometric tools are applied in the field of Learning and Development.
What is psychometrics?
If we look at the word “psychometrics,” we can break it down into two parts: “psycho” means mental and “metric” means measurement. The American Psychological Association (APA) Dictionary of Psychology defines “psychometrics” as a scientific discipline “concerned with the quantification and measurement of mental attributes, behavior, performance, and the like, as well as with the design, analysis, and improvement of the tests, questionnaires, and other instruments used in such measurement” (APA, n.d.).
We often use psychometrics (e.g., assessment tools, measurement instruments, and statistical models) for the evaluation, selection, placement, diagnosis, and classification of individuals. Its tools are widely used in practice, including in educational assessments, clinical diagnoses, and workplace analytics.
What can be measured with psychometric tools?
Table 1 (Salkind, 2017, p. 35) shows a non-exhaustive list of what we can measure with psychometric tools, which include questionnaires and standardized tests. Note that, regardless of what we want to measure, fairness, reliability, and validity are three important and fundamental principles (AERA et al., 2014) that guide the development and use of assessments.
A fair assessment, for example, should only measure what it aims to measure, independent of other characteristics of the test takers. For example, if a test aims to assess students’ algebra skills, then students who are presumably at the same algebra level but come from different cultures should perform similarly on the test.
A reliable assessment should produce consistent scores, such as across time, equivalent test forms, or individual items targeting the same concept. We refer to the last type of consistency as internal consistency. For example, a survey of persistence factors may ask students to what extent they agree with the following statements:
- I give up easily when I think a task is too difficult.
- I am reluctant to tackle a challenge.
- I persist in face of failure.
This survey likely has a good internal consistency if students agree with the first and second statements and disagree with the third, given the third statement is the only one worded positively in relation to “persistence.”
A valid assessment measures what it claims to measure. For example, if an algebra test is designed to measure students’ understanding and mastery of algebra, the validity of the test is (a) conceptual, or, how well the items on the test target relevant algebraic techniques and concepts; and (b) empirical, or how much evidence we have to claim that this test indeed measures students´ understanding and mastery of algebra, in such a way that their scores can be used to infer their knowledge of algebra.
Table 1
An Overview of Different Types of Tests with Examples
| Type of test | What it measures | Some examples |
| Achievement | Level of knowledge in a particular domain | · High School Placement Test· Early School Assessment· Norris Educational Achievement Tests· Test of Basic Adult Education |
| Personality | Unique and stable set of characteristics, traits, or attitudes | · Aggression Questionnaire· Basic Living Skills Scale· Dissociative Features Profile· Inventory of Positive Thinking Traits· Employer personality tests, such as the Occupational Personality Questionnaire |
| Aptitude | Potential to succeed | · Differential aptitude tests· Scholastic Aptitude Test (SAT)· Aptitude Interest Category· Evaluation Aptitude Test· Wilson Driver Selection Test |
| Ability or intelligence | Skill or competence | · Wechsler Intelligence Scale for Children· Stanford-Binet Intelligence Test· Cognitive Abilities Test· General clerical ability tests· School Readiness test |
| Performance | Basic performance of particular tasks | · Driver’s license road test· Assessment in Nursery Education· Functional Communication Profile· The Egan Bus Puzzle Test |
| Vocational or career | Job-related interests | · Adaptive Functioning Index· Career Interest Inventory· Prevocational Assessment Screen· Rothwell-Miller Interest Blank· Vocational Adaptation Rating Scales |
Note. Aptitude tests measure an individual’s potential to succeed in an occupational or educational setting or the potential to acquire relevant skills or knowledge. Ability tests focus more on an individual’s current level of skill, competence, or knowledge. There are, however, ability tests that are also seen as aptitude tests.
Glossary
Measurement: The assignment of labels or values to specific attributes of objects or people’s behaviors, through systematic and standard procedures.
Fairness: Fairness is about assessing individuals equitably, irrespective of personal characteristics or experiences external or irrelevant to the targeted skill or knowledge.
Reliability: Reliability is the consistency of outcomes for the same group of individuals.
Validity: Validity is the extent to which empirical evidence and theoretical rationales support the conclusions and interpretations of assessment outcomes.
References
Alloway, T. P., & Cissel, H. (2017, April). Psychometric testing in the workplace. The Score. http://www.apadivisions.org/division-5/publications/score/2017/04/psychometric-testing
American Psychological Association (APA). (n.d.). Psychometrics. In APA dictionary of psychology. Retrieved November 14, 2021, from https://dictionary.apa.org/psychometrics
American Educational Research Association (AERA), American Psychological Association (APA), & National Council on Measurement in Education (NCME). (2014). Standards for educational and psychological testing. American Educational Research Association.
Baer, T., & D’Silva, V. (2018, October). ‘All in the mind’: Harnessing psychology and analytics to counter bias and reduce risk. McKinsey & Company. https://www.mckinsey.com/industries/financial-services/our-insights/all-in-the-mind-harnessing-psychology-and-analytics-to-counter-bias-and-reduce-risk
Dattner, B. (2013, September). How to use psychometric testing in hiring. Harvard Business Review. https://hbr.org/2013/09/how-to-use-psychometric-testin
Donaldson, S. I., & Donaldson, S. I. (2020). The positive functioning at work scale: Psychometric assessment, validation, and measurement invariance. Journal of Well-Being Assessment, 4(2), 181-215.
Ledet, E., McNulty, K., Morales, D., & Shandell, M. (2020, October). How to be great at people analytics. McKinsey & Company. https://www.mckinsey.com/business-functions/people-and-organizational-performance/our-insights/how-to-be-great-at-people-analytics
Rust, J., Kosinski, M., & Stillwell, D. (2020). Modern psychometrics: The science of psychological assessment. Routledge.
Salkind, N. J. (2018). Tests & measurement for people who (think they) hate tests & measurement. SAGE.
Wilkie, D. (2013). How reliable are personality tests? The Society for Human Resource Management. https://www.shrm.org/resourcesandtools/hr-topics/employee-relations/pages/how-reliable-are-personality-tests.aspx
Illustration credit for cover art https://www.freepik.com/free-vector/exams-concept-illustration_9233873.htm#query=test&position=0&from_view=keyword