Are you using the right testing tool?
If you’ve applied for a job, chances are you’ve taken some kind of psychometric test. For what it’s worth, psychometric assessments have been around since the early 20th century. And they’re widely used; the commonly quoted statistic is that as many as 80% of the Fortune 500 use these kinds of tests in the recruitment process.
It’s a booming $750 million-plus industry because organizations recognize that making the right talent decision is big business. Unfilled vacancies cost $160B a year in productivity loss. The average cost-per-hire is just over $4k. And the cost of making the wrong hire can be 2.5x that person’s salary.
Ultimately, talent leaders are looking for data to back their talent decisions. The dream is to take a peek behind the wizard’s curtain and determine, with certainty, that the hire they’ve made has the right skills, will commit to the role and the company, and that they will stick around.
With unstructured interviews being such a terrible – to be frank – and unreliable predictor of future success in a role, the obvious choice is to find an instrument that will bring much-needed objective data into the process.
To complete the data set, one approach is to use psychometric assessments as part of the recruitment process. To gauge whether the candidate’s personality and cognitive abilities match the requirements of an open role. And as stated before, the big players are big fans.
But just because they do – does that mean you should?
What is psychometric testing?
Psychometric means measurement of the mind. A psychometric test, when applied in a recruitment context, aims to provide an objective, unbiased and standardized assessment of a candidate’s personality, behaviours, motivation, career interests, competencies and intellectual abilities. An ambitious goal, to be sure.
They fit into the data arsenal of your talent strategy by helping you see past the résumé, past the voice on the other end of the phone screen and past the well-rehearsed, well-dressed, charismatic person in the interview room.
Hiring teams often apply psychometric tests to whittle down the number of applicants for a role to determine the best fit. In many cases, organizations are driving large amounts of candidates through testing at the start of the hiring process to identify a smaller “suitable” pool of applicants, known as a shortlist.
Efficiency gains, both for the talent acquisition/recruitment team and hiring managers are cited as the primary reason for using such tests to narrow down the candidate pool. This is especially critical where teams are dealing with high-volume roles, or have a high volume of reqs open. And it’s not just about minimizing internal time and costs. When the best talent is off the market in just 10 days, it’s crucial that organizations can quickly and accurately make the right hiring decisions.
Misinterpreting the results
“People seem to think that having numbers is always better than not having numbers,” says professor in Organisational Psychology, Orin Davis. “The reality is that numbers are only ever as good as the instruments they came from, and a great many tests tend to be rather poor, especially when applied to individuals and contexts.”
“It is one thing to measure whether a person tends to be more or less open to experiences relative to the average person (which is readily done with the Big 5 personality test), and quite another to say that a specific person will be open to a specific experience (impossible to do with a personality test).”
Well-researched and validated psychometric tests aren’t in and of themselves problematic. Some are incredibly useful. The problems largely lie in a misinterpretation of the term, and the contexts in which they’re used.
Things start to get a bit tricky because “psychometric test” and “personality test” are often used interchangeably. Personality tests hone in on an individual’s deeply held values and beliefs that shape their behaviour. The purported purpose of assessing someone’s personality is to find some of those icky bits of behavior that will impact someone’s performance on the job.
Are they really the best person for the role?
Will they work well with the team?
How will they really react when the proverbial hits the fan?
Let’s address the suitability of personality testing for job roles (hint: it’s not good). Tests range from robust and well-meaning, to questionable and unhelpful, to downright discredited. Whether you’re using MBTI that was invented out of thin air or the well-regarded Big 5, if you’re using a personality test to gauge whether someone will be a top performer, be wary of the weighting you give to these assessments.
“Most psychometric assessments are not inherently biased themselves, but rather the interpretation or decision-making of the process they contribute to is biased,” notes renowned talent assessment thought leader and I/O psychologist, Dr. Charles Handler.
“The most glaring (bias) is that people do not always know what the constructs mean or imply,” notes Davis. If you are giving a test of creative personality without knowing what a creative person is, or how many different validated personality assessments there are for creatives, you are operating with a crippling bias and are better off not using any test at all.”
An overwhelming number of those who use psychometric tests for recruitment are untrained in how to use them. Add in time-poor hiring leaders dealing with substantial volumes of candidates who need to grasp at anything to manage the load and you’re left with decisions that are over-reliant on personality data to make a decision.
Dr Handler adds, “While the score data from an assessment itself may not be biased, if there is not a cut score or any mandated score level that’s required for someone to be eligible to be hired, then there’s an opportunity for those interpreting the results to inject their own bias or subjectivity into the ultimate hiring decision.”
This is typically not the fault of the assessment itself, more of the fault of the process or just an artefact of human nature.”
Dr Charles Handler
Context is everything
It’s also easy to glaze over the context in which personality assessments are employed. When you are making reflexive or ill-informed hiring decisions on this basis, how can you be sure the result you are looking for is right for the role?
“The hard part about personality assessments,” says HR Tech and AI analyst, Ben Eubanks, “is that they’re often seen in the same way you would look at an assessment of math or history: there’s a “right” answer and a “wrong” one.”
“The truth is that assessments are much more dynamic, and the average hiring leader or talent professional may not realize that multiple types of personalities might be well suited to a role or job family.”
Ben Eubanks
Bold HR’s Rebecca Houghton adds, “Dig a little deeper and you will start to notice a difference in terms of intrinsic motivation that will be stronger in one [candidate] than the other, which will drive higher performance overall. The problem here is that in order for us to know which one, we have to know our environment and context really, really well and we don’t.”
A useful assessment for one role or company may be irrelevant for another. “Personality assessments rarely assess the environment, they just assess the person,” adds Houghton. “So the concept of using personality profiling as a matching tool is totally skewed.”
So, should we be using the results of personality tests to make hiring decisions? “Not to determine if someone is going to be successful in the role,” says Andrew Gemmel, an experienced recruiter. “They are simply a guide or enrich information to some extent about their base level personality and how they may approach the role.”
This is true even for high-value roles, such as executives. For these roles, “there’s often value in a more sophisticated personality assessment tool that is interpreted by an expert psychologist who can then brief those involved in the hiring decision,” notes Dr Handler. “Even in these cases personality data should never be the sole predictor, it is just another signal or data point that provides insight to those involved in the hiring decisions.”
Brain power
What about cognitive tests? What if, instead of values and beliefs, we focus on a candidate’s mental ability and aptitude? Again, it comes down to context.
Cognitive tests are only as good as our insight into the level of abilities needed to perform a role. “Hiring organizations generally lack a good insight to our existing baseline”, comments Houghton, “so we tend to ‘invent’ cognitive standards that people need to meet without spending too much time on the job design, existing standards of high performance, and contextual requirements.”
“If the role has no problem solving requirements, then I’d challenge why we’re measuring problem-solving skills, which is what cognitive tests are for.”
Rebecca Houghton
Let’s come back to our hiring manager drowning in applications. Why can’t they just take the smartest of the bunch and put them into a role?
Consider that a common application of cognitive testing is to filter through applicants for low-skilled roles. Using them in this manner, says Davis, “is tantamount to inquiring whether we should be asking ski instructors if they know how to swim. One can suppose that a ski instructor who knows how to swim probably does have good coordination and motor control, and that is a good indicator that one is more likely to be able to ski well. But it is probably far more helpful and expedient to focus on their skiing and teaching capabilities.”
Gemmel cautions the importance of context as well. “IQ has different aspects and high volume roles may benefit from only certain parts of stronger cognitive indicators. Even then we won’t know if it is relevant, useful or in context to the role at hand over the medium to longer term. The context becomes very important when hiring. A high IQ or strong cognitive bias in whatever area won’t mean you have hired the right person.”
Why (some) companies love psychometric tests
The truth is, humans love to put people into buckets. To that extent, psychometric tests are an incredible resource. They can be powerful to quickly categorize someone’s working style, how they like to be managed and where their development potential lies. A la Emma Goldberg’s cheeky comparison to astrological signs, your Color Code, Enneagram type, DiSC profile or MBTI style can be a quick route to “belonging” with a team or workplace.
The bigger truth is, humans like to know other humans live in the same bucket as them. A personality “badge” gives them perceived similarity with colleagues that can help them gravitate to those they want to have a beer with… and those they perceive we don’t want to. The danger in looking for candidates with the personality we think might “fit”, outside of the context of role or capabilities, is that you end up homogenising your talent pool to fit your bucket.
Davis notes this form of bias as “the cloning error”. “People assume that a person who was successful in a role has a measurable set of personality characteristics and strengths that constitute the only way to be successful in that role,” explains Davis. “As such, replacing that person requires finding someone just like them — a clone.”
“So, if your most successful salesperson was an agreeable extravert whose top strengths are bravery, honesty, and kindness, you will require anyone you interview to match that profile. For the same price, you might as well match their demographics!”
“This approach is riddled with errors over and above this obvious one, not the least of which is that conditions of a job can change rapidly — new coworkers, new bosses, new company policies, new resources, new tech, et cetera. As soon as the context changes, people need to adapt, and that can include working in different ways and applying one’s personality and strengths in different ways.”
From Davis, “culture is a much broader construct and has room to accommodate a whole host of personalities that can each be adapted to fit into the culture in different ways.”
Testing is supposed to mitigate bias in hiring decisions. But does psychometric testing achieve this? “IQ is a natural bias either way,” comments Gemmel. “Too smart” or “not smart enough” for the position is a bias in its simplest form. In terms of personality “too introverted or extroverted comes to mind”.
“Neither are indicators of success. Just simply a base level of information you “might” find useful. Though if relied on with a heavy emphasis (bias) you may miss out on a star of the future.”
Andrew Gemmel
Additionally, there is fresh concern around the bias within AI-based tools. Dr Handler says, “This is still an issue that is got a lot of controversy around it and there’s not a lot of definite proof one way or another. it’s more a factor of the idea of garbage in and garbage out.”
Therefore when applying AI assessment tools, it’s critical to examine both internal processes and the vendor’s approach to identifying and mitigating bias patterns.
Fit for purpose
If not during the hiring process, when can you reliably use psychometric tests? Gemmel recommends that hiring leaders “use them to understand the nature of the person at hand and how this information might help you, as leaders and managers, to communicate and relate to them during their development.”
“The best way to use psychometric and personality assessments is as a single point in a larger decision-making process,” says Eubanks. “If you have a mechanism like Vervoe offers to capture someone’s skills and abilities related to the job, a personality assessment might give you additional indicators on their fit for specific types of roles.”
“At the end of the day, the best personality doesn’t make up for someone’s inability to do the job, perform the work, or serve the customer. Those are requirements in the fast-paced world of business today.”
Ben Eubanks
Should psychometric tests be used for hiring?
The summary from our experts is resoundingly, “proceed with caution”.
While these assessments may be useful for improving team dynamics, to guide development or as a smaller piece of your bigger talent puzzle, they are not the best solution to base your hiring decisions on.