For decades, cognitive testing has been a cornerstone of the assessment world, an indispensable part of the recruitment process, especially when it comes to evaluating candidates with limited work experience. But now the wind seems to have turned.
Recently, research has begun to question these old truths from several directions. A study by Stephen Woods and Fiona Patterson has shaken up the established order that has been unchanged for decades.
First of all, many of the studies included in meta-analyses (where several studies are reviewed and analyzed together) have been statistically adjusted for "range restriction". This has effectively inflated the supposed predictive validity of an assessment, assuming that one only analyzes data for the best candidates hired. However, if we do this for some assessments but not others, we run the risk of comparing apples with pears. So perhaps it is not so strange that cognitive tests seemed to come out on top.
In addition, many of the studies are decades old, and the world of work has changed enormously. Partly, it has well documented The Flynn effect showed that average scores on cognitive tests have gradually improved since the mid-20th century, largely due to improvements in education and learning at the societal level.
Firstly, the nature of working life has changed significantly with the ubiquitous use of computers in the workplace. And in the latest phase of technology development, tools like Gen AI have already become a resource that white-collar workers around the world are turning to.
As a result, the gap between the importance of traditional cognitive reasoning skills and the availability of these skills in the labor market has narrowed. At the same time, new skills have come to the fore, such as complex thinking across multiple domains (rather than in a linear fashion as in cognitive tests), as well as skills in collaboration, creativity, innovation and adaptability.
These changes mean that the assumed superiority of cognitive reasoning may no longer hold, and a new review of meta-analytic data actually shows this. In fact, when you use a range of other assessment methods together, the incremental value of cognitive testing may be more nuanced than previously thought.
So, what does this mean for how we judge?
Cognitive tests are still a valid tool in the assessment arsenal, but the risks of negative impact need to be balanced against the benefits. They still play a role in a rounded assessment of a person, but we need to mitigate the negative risks of negative impact to maximize diversity, equality and inclusion.
In practical terms, the large-scale use of cognitive testing alone creates a risk of systemic negative impact. Even when the limits are within reason to avoid violating the four-fifths rule, it is still not zero negative impact. There is often legally permissible, yet widespread and persistent negative impact.
The key to managing this is to build an assessment process that is multi-method within an assessment, intentionally maximizing prediction while minimizing negative impact. That's why since the beginning Sova has pioneered the use of blended assessments to solve this very problem, with clear and demonstrable benefits for many large companies.
Now we see how new research, as mentioned above, and leading industry practices show that for recruitment at scale, the use of cognitive tests for selection is no longer best practice. Instead, a blended approach that focuses on creating the fairest and most objective process possible is the new standard all organizations can work to.
There is no longer any need to compromise fairness, the cornerstone of every high-quality modern assessment journey.
