The Complete Guide to Skill Testing

Share on facebook
Share on twitter
Share on linkedin

With so many candidates in the market, it’s more important than ever to create an engaging and human candidate experience. But you need to balance that with finding the best talent for your role.

Skill testing can give recruiters a competitive advantage in today’s job market. Candidates who are hired on merit, rather than background, tend to stay longer and perform better over the long term. Here’s how to use skills assessments to fill your open positions, no matter how many applicants you are dealing with.

What We’ll Cover

What is a Skill Test?

A skills test is an assessment used to provide an unbiased, validated evaluation of a candidate’s ability to perform the duties listed in the job description.

Typically, a skills test asks a variety of questions in different formats to see how candidates perform on-the-job tasks. A good skills test includes questions that are capable of being answered by someone already doing the job and can accurately measure key performance metrics. Questions should also be specifically tailored to relate to the responsibilities of an open position. Many skills tests include immersive experiences, like coding challenges or job simulations, to mimic how a candidate performs when faced with a real-life scenario.

Get more hiring resources delivered straight to your inbox.

Other types of job-readiness evaluations deploy validated psychometric assessments to identify those in-demand soft skills: things like motivation, conscientiousness, resilience, and emotional intelligence. A personality assessment varies from a skills test in that it predicts how a person will behave in a specific scenario, rather than their ability to complete a task.

While skills test cover task-related abilities, like coding, copywriting, or sales, some pre-employment assessments integrate the less tangible capabilities – things like teamwork and leadership. These qualities are sought after by executives at more than 900 companies, according to a Wall Street Journal survey of executives.

Yet, 89% of those surveyed said they have a “very or somewhat difficult time finding people with the requisite attributes.” Where traditional hiring methods fall short, a skills test can easily clarify a candidate’s true talent.

Overall, skills tests can play a critical role in predicting on-the-job success. More so than resumes or job interviews, a skills test can assess the true potential of a new hire to go the distance with the company. Here’s how skill testing works, and why more companies than ever are starting to integrate skill testing into the recruitment and hiring process.

How Skill Testing Works

Skill testing works best when the questions being asked are specifically crafted to the role and needs of the team hiring the new candidate. In designing a skills test, combine different types of questions to get a 360-degree view of how a candidate will perform in different scenarios.

There are a variety of ways to set up a skills test – and we’ll get into the mechanics of how to actually run the assessment in the next section. But, designing a thoughtful aptitude test takes some initial foresight on behalf of the hiring manager and team. Research by Deloitte suggests this sample process for selecting and implementing skill testing questions:

  1. Define the “human elements” needed to perform the job
  2. Compile questions that will measure and predict these human elements
  3. Use the data gathered by the skills assessments to empower the next round of the screening process
  4. Post-hiring, evaluate the efficacy of the hiring assessment to ensure the questions delivered the best result.

Ultimately, the best use for an aptitude test is to help recruiters move away from the resume and allow candidates to prove they are the real deal. Crafting the right series of questions should be a collaborative process between the recruiting team and the team hiring the new employee. Here’s how these teams can set up and run a skills test.

How to Set Up and Run a Skill Test

In designing a skills test or pre-employment assessment, there are a few specific steps to take in order to thoughtfully structure your questions. Vervoe recommends following these best practices in setting up and running your skills test. These tips can help with candidate engagement and lead to high rates of completion.

  • Your skills test should include a minimum of six questions; somewhere in the eight to ten range is best.
  • At least a few questions should require text answers; start with a text-based response in the first question, rather than a video or immersive question.
  • At least one question should be multiple choice.
  • Include an “immersive” style question, in which the candidate edits a document, spreadsheet, or presentation.
  • To retain a candidate over the entire experience, start with easier questions and build up to more difficult ones later in the assessment.
  • Try to minimize use of timers to account for technical difficulties and give the candidate the best chance of success.

We also suggest that video responses not be timed; there are too many technical issues that can result from a candidate trying to film a one-way video interview. If you do wish to set a time limit, make sure it’s at a minimum of five minutes.

Running a skills test through Vervoe, or any other platform, is relatively straightforward. Vervoe’s skills assessments let you select questions from a library of assessment tools, or design your own questions based on the specific needs of your company. The Expert Assessment Library offers questions and trials created by experts in their fields, meaning they have at least 3+ years of experience in their specific area of expertise. You can preview questions from any of the assessments and add them seamlessly through the Vervoe platform.

Now that you know how to set up an aptitude test, when should you deploy this tool during the hiring process?

Using Skill Tests During Hiring

Timing is everything when it comes to adding a skill assessment to your hiring process. Research by Harvard Business Review revealed that skills tests should come early in the hiring process. According to their study, “Many service companies, including retailers, call centers, and security firms, can reduce costs and make better hires by using short, web-based psychometric tests as the first screening step. Such tests efficiently weed out the least-suitable applicants, leaving a smaller, better-qualified pool to undergo the more costly personalized aspects of the process.”

Skill tests should be used to screen candidates in, not out. The issue many recruiters face is that the volume of candidates makes it impossible to carefully consider each person’s ability. Smart algorithms and AI tools can turbo-charge candidate assessments by scoring results quickly and removing human bias from the equation.

Vervoe’s algorithm scores candidates using a multi-layered approach. Candidates are ranked based on how well they performed, rather than filtered out if they didn’t achieve a certain benchmark. The top candidates easily rise to the top; but no one misses out on being considered for the next round. When used early in the hiring process, skill tests can select a more diverse pool of applicants to continue onto the next phase.

Skill Test Examples and Templates

There are many ways to set up a skills test, depending on the position for which you are hiring. Pre-employment skills tests can cover a range of positions: administrative assistant, finance and accounting, and call center reps are just a few roles that companies hire for using skills assessments.

Niche skill test for coding

Excel skill tests, coding skill tests, typing skill tests, and other computer skill tests are the most common forms of pre-employment assessments. Some companies focus on questions that are task-related, e.g. “Create a Powerpoint Slide that has a video embedded in the presentation.” Questions can get hyper specific to test a niche skill, like a coding language, or be posed more broadly to test the general requirements for success at a certain level.

Some companies choose to focus on verifying the skills that will help a candidate succeed beyond the immediate position. This approach skews closer to a pre-employment assessment, with questions designed to reveal if a candidate can climb the corporate ladder, adapt in a challenging work environment, or respond under pressure. For example, one call center rep test included questions such as, “You have an elderly customer on the phone who is having trouble understanding your instructions. A colleague is also trying to transfer a call from a customer you served before, and you have a scheduled follow-up call happening in 5 minutes. How would you handle and prioritize in this situation?”

Multiple choice, open-ended questions, and pre-recorded video responses are all great ways to see if a candidate has what it takes to do the job well. But, do candidates enjoy answering these types of questions?

Do new hires like doing skill tests?

By most accounts, candidates appreciate the opportunity to showcase what makes them great at their job. Orica, the world’s largest provider of commercial explosives, integrated skill testing into their interview process to the delight of their job candidates. In revamping the interview process for graduate students looking to join the Orica team, recruiters consolidated their online evaluation components into one platform, Vervoe. The skill assessment combined questions focusing on skills, logic, and values.

An average of 86% of candidates completed the online process, and the reviews were mostly positive. Here’s what the candidates had to say about the skills test:

“The tests required total engagement and thought, and were a clear demonstration of what makes Orica different from any other company.”

“I think the questions were very diverse and it allowed me to showcase myself, my skills and abilities in different ways.”

It gave me an opportunity to showcase who I am as well as challenge my skills”

This is just one example of how a skill test can change the entire interview process for a potential new hire. In a job market where people spend an average of 11 hours a week looking for a new job, it’s easy to get burned out, fast. Every job description starts to look the same; every interview begins to feel stale.

When given the opportunity to showcase their talent through real-world tasks, job candidates will jump at the chance to be engaged with the job description, rise above their resume, and challenge themselves. Companies that use Vervoe’s assessments experience a 97% candidate completion rate, which is among the highest engagement rates in the industry. Candidates love the opportunity to stand out from the crowd. Even if they aren’t hired, skills testing offers a break from the repetition of the stale interview experience.

What are the benefits of a skill test?

The benefits of a skills test aren’t limited to the candidate experience.

Recruiters looking to hire diverse, high-performing teams with better efficiency and consistency can use pre-employment tests to their advantage. Skills tests are a better predictor of performance than resume screenings or traditional interviews alone. Resume screenings are bad for three reasons. First, studies suggest that it’s common for candidates to lie on their CV. The person you think you’re hiring may not actually possess the qualifications you think they do.

Second, resumes only provide a high-level view of a candidate’s credentials and work experience. These items don’t offer qualitative insight into actual on-the-job performance. Coupled with recruiting biases that are built into the process, the third threat is that recruiters are privileging candidates based on background and demographics, rather than talent. Perhaps this is why new hires crash out as often as they do. According to one study, 46% of new hires “fail” within the first 18 months of being hired.

Skill tests can help take some of the bias out of the interview process, give recruiters a new evaluation metric to consider, and lead to happier, long-term hires. There’s ample evidence to suggest they really do work better than many of the other traditional hiring methods recruiters have relied on in the past.

Do skill tests work?

In our experience, skill testing works better than traditional hiring methods – with some caveats.

Without a doubt, aptitude tests can be used to replace resume screening. This style of sorting through candidates increases the chance that the best candidates will be unfairly eliminated. Good people get screened out, rather than screened in. So-called “pedigree proxies” – resumes and cover letters – are not indicative of job performance, yet they are often the quickest way a recruiter or algorithm can think of to cut down on their stack of candidate resumes.

Skills tests improve time to hire while allowing the hiring manager to see how someone will do the job, before they get the offer. This reduces turnover costs, which add up quickly: the cost of making the wrong hire can be up to 2.5x salary, easily over $100,000. Working with Vervoe’s skills assessments, on the other hand, can help a recruiter identify the best people at under $100 per hire.

The best skills tests, however, need the right formula to help the candidates succeed. Some recruiters focus narrowly on the skills that will help a new hire succeed in the immediate position for which they are hiring. Yet, many CEOs emphasize the importance of soft skills – things like leadership and teamwork.

New hires may end up being disappointed and leaving because they lacked the soft skills needed to adapt to their new team, not necessarily the skills to perform the job. Recruiters must integrate questions into their skill assessment that focus on critical soft skills that predict long-term success. These validated psychometric assessments are key to assessing “culture fit” without defaulting to recruiter bias.

Is skill testing valid?

  • Any test that directly mimics what a person will do on the job can be considered “validated.”
  • Tests of personality and soft skills are a riskier prospect even when they are “validated,” because they often lack the proper validation required to be EEOC compliant.
  • Positive candidate experience and perceived fairness are two of the primary reasons why skill testing is an effective and expedient hiring practice.

The value of testing candidates prior to hiring them is having an expedient way of assessing their fitness for the job. The decisions made based on a test, however, cannot be any better than the information provided by the test, which makes it extremely important that the test be an accurate representation of the constructs that it represents.

This is somewhat difficult, however, because the constructs being assessed, such as knowledge, skills, attitudes, and cognitive processes, cannot be known objectively and must be inferred. To that end, there are many statistical measurements that are used to assess the degree to which tests are reliable (consistent) and valid (accurate).

There are many types of psychometric validity, and it is a rare test (if such even exists) that hits every type. Looking specifically at tests for finding job fit, there are a few different types of validity that are particularly relevant, not just to ensure that the hire is a good one, but to ensure compliance with EEOC regulations.

Types of Validity

Face Validity:
Does it look like the test is assessing what it claims to measure?

Face validity is the most basic form of validity, and requires general consensus that someone taking this test would need to exhibit the constructs that the test is assessing. For instance, a set of math problems being used to assess arithmetic ability has more face validity than a set of word problems, because the latter are assessing a combination of arithmetic skills and reading comprehension (which means the test is assessing arithmetic skills in a specific context, but not in general), etc.

For some types of skill tests, this is the main type of validity available, and sometimes the only one that can be obtained when a test is first created.

Generally, these sorts of skill tests require job candidates to engage in tasks that could theoretically be completed by anyone who can do the job. As long as the tasks mirror activities that the individual will need to be able to complete in order to do the job properly, the questions have face validity and are EEOC compliant.

That said, a company should always take care not have “disproportionate impact” upon a particular demographic group, as this can be an indicator that something is wrong with the test. For instance, if only 75% of members of Demographic A pass the test, while it’s 93% for other demographics, the test still may be EEOC compliant, but an investigation is a must.

Content Validity:
Does the tests cover the full range of the construct that it is supposed to measure

Content validity refers to whether a given test covers a representative sample of the full range of the construct that it is measuring. For job tests, the matter of validity should be focused most on this point. In addition to ensuring that the questions appear to be about topics related to the position (face validity), the set of questions (as a group) needs to assess a sufficient range of tasks so that the evaluator can know that the set of answers paints a picture of whether the individual is capable of doing the collection of tasks required by the job.

As long as the set of questions provides a representative sample of the tasks the candidate will need to do on the job, and someone who can do the job properly can definitely provide solid answers to the questions, the questionnaire has content validity and is EEOC compliant. (Again, it is important to be aware of the possibility of disproportionate impact, and to be prepared to alter the task[s] if you are seeing a particular demographic scoring lower than another.)

Construct Validity:
Does the test actually measure the theory-based construct that it claims to measure?

In hiring, construct validity has the greatest bearing on tests that do not directly assess knowledge and skills that will be used on the job. Rather, it is about determining whether, say, a test of attention to detail (an example of a “soft skill”) or extraversion (personality trait) actually demonstrates that the person has the indicated characteristic.

When HR people ask if a test is “validated” or has “psychometric validity,” this is usually what they are talking about. But, when it comes to tests of skills to be used directly on the job, construct validity is far less relevant compared to face and content validity.

Tests of general cognitive ability and/or personality, however, absolutely need to have construct validity to be EEOC compliant, because they are indirectly related to successfully completing the actual tasks of the job. For a personality or “soft skill” test to be valid, it must assess a theoretical construct whose existence can be defended through a review of the scientific literature, and statistics must show that the test questions do indeed work together to describe a coherent construct.

Predictive/External Validity:
Does the information we learn from test performance predict/apply to performance in other situations?

Predictive/External validity is about whether the test predicts performance in areas that have not been directly tested. Even though a test may have the above types of validity, there remains a difference between the tasks that are tested in a context-free fashion, and doing the actual tasks in the context of the job.

If a company is going to use a test, it is always helpful to prove that the test actually predicts high performance on the job and/or good fit with the company/team. For a test of skills that will definitively be used on the job, predictive validity is very helpful, but technically not required, for EEOC compliance. But, for any other type of test, this is crucial for a company to have, and they have to run the numbers within their own company.

One of the most important points to recognize is that predictive validity does not transfer between companies. If a study proves that Test A predicts high performance at a Big 10 consulting firm, that does not help a Big 4 accounting firm at all (especially if they are having to defend themselves against an EEOC-related suit). What good is a personality test (for example) if you cannot prove that it distinguishes between high and low performers in your company?

Unfortunately, most tests cannot do such a thing, in part because most people who run tests of predictive validity do so only to prove that a particular combination works – what they fail to do, however, is prove that it is the best or only combination that works (this can be done, but it is very difficult and work-intensive to do). In turn, this means they haven’t actually proven that other scores should be causing people to be rejected, which can be risky and can cause companies to miss out on fantastic talent.

Moreover, a given company can prove that a test has predictive/external validity only after using it for a while and measuring on-the-job performance, which means that the company cannot initially assume that these indirect measures of job success are valid.

One of the primary values of using skill-based testing in a hiring process is that there is a clear correspondence between the items on the tests and the tasks that will be performed on the job. As such, skill tests have both face validity and content validity, which makes them EEOC compliant. In addition, these tests expediently show which candidates are nominally capable of doing the job before employers spend time assessing a candidate’s application materials. By contrast, indirect tests of future job performance, like personality tests and assessments of general cognitive abilities and/or “soft skills” require extensive assessment to confirm construct and predictive/external validity, which makes them a riskier and less expedient move.

In all cases, however, it is imperative that employers keep an eye out for demographic disparities in performance, and to launch an immediate investigation if a significant one is found. At Vervoe, we are constantly monitoring our tests to ensure that candidates are taking a fair test that assesses the skills they will use to get the job done well. As an added benefit, many companies find that using fair tests also yields a more diverse slate of finalists that is more representative of the wide range of talent that exists in the world.

Though personality and cognitive assessments are very hard to validate for the hiring process, they can be helpful in onboarding and coaching new hires in a way that is tailored to their uniqueness, which can give them a smooth and powered-up start. But, what matters most of all is the candidate experience. An enjoyable candidate experience, combined with a perception that the process is fair (along with the test actually being fair!), is the best way to ensure that candidates will not only show their capabilities to the company but have a positive view of the company regardless of the outcome of the hiring process.

Skill tests vs. interviewing

In conclusion, we’ll leave you with few thoughts on skill tests compared to interviews.

First, interviews, in general, need a total overhaul. Recruiters have been asking the same, outdated interview questions for decades. Many candidates get overwhelmed by the performance anxiety inherent in the interview and may make (forgivable) mistakes. Nevertheless, many recruiters like the security of meeting someone before making an offer.

Many recruiters seek the same insight from a group interview or case study that they would get from an individual skill test. Unfortunately, using these methods can’t give you the same valuable information as a straightforward aptitude assessment. Case studies can be too conceptual; rather than seeing how a candidate will approach the work listed in the job description, case studies ask abstract questions. The goal of asking “how many tennis balls can fit on a Boeing 757” is not to see if the candidate can guess the right answer, but to see how they approach the question and reason through their response. But this knowledge doesn’t always serve a recruiter with the best predictor of on-the-job success.

Group interviews provide more insight – into a candidate’s teamwork, leadership, and communication, for example. Yet, in a group scenario, extroverts tend to dominate. It can be difficult to see how each candidate performs as an individual while trying to consider the group at once.

In summary, skill testing is all about understanding whether a candidate can do something or knows something. It’s about verifying their ability to go the distance with your company. Pre-employment assessments differ slightly in that they focus on predicting how a candidate will behave in certain scenarios, not what they can do. By combining questions from skills testing and pre-employment assessments, recruiters can get a more accurate picture of the candidate’s ability.

Additional Resources

For more reading, check out some of these great resources.

Related articles

Hiring Resources
Emily Heaslip

The Complete Guide to Remote Recruiting

Even before COVID-19 precipitated the need for social distancing, remote work was on the rise. There’s been a 44% increase

The 10 Best Applicant Tracking Systems