Bas van Hateren

Q: How does Bas van Hateren see AI changing pre-employment assessments?

A: He sees three trends. The first is that candidates use AI on a massive scale to complete assessments. Several researchers have shown that an AI almost always passes the cut-off on a questionnaire-based assessment. Van Hateren mentions Patrick Dunlop in Australia, whose agent matched roughly 85 percent of the profile you would want for a role. In intelligence tests too, the latest models now score at the level of a higher education graduate or above in the exercises his team runs. The second trend is a wave of startups offering AI-based tests. The third, which he is enthusiastic about, is low-threshold chatbots that ask applicants about their skills.

Q: Why are personality and intelligence tests becoming less reliable, according to Bas van Hateren?

A: Not because of the test, but because of the candidate. The reliability of the test itself has not changed. The question is whether the candidate still completes it themselves. Anything that does not need to be actively controlled can be taken over by an AI. Because AI can now operate your computer as well, providers need to build in failsafes for what must not be taken over. Van Hateren therefore expects a shift towards visual tests that you really have to control yourself, such as games and game-based assessments. That is something AI still does less well remotely.

Q: What is the first question to ask an assessment vendor, according to Bas van Hateren?

A: What is the test-retest reliability? If you remember one thing from this episode, he says, let it be this. Below 0.7 you should not touch it, and ideally you want it above 0.8. The number answers a simple question. If someone takes the same test twice within a month, does the second score come close to the first? If a candidate scores 20 percent higher at employer B a week after taking the test at employer A, simply because they have figured the test out, then the test is measuring exactly what you do not want in it.

Q: How should you judge new AI-based assessment vendors, according to Bas van Hateren?

A: Open-minded, with a hard floor. He is positive in advance about anything new, because in his view selection needs a new paradigm. But as soon as he asks new vendors to back up their reliability or validity, many of them stop responding. He mentions an Israeli provider of work sample tests for roles such as warehouse work, which had never checked whether its scores were related to performance on the job. Doing warehouse work on a computer is not the same as doing it in a warehouse. A test must be reliable, measure what it claims to measure and be able to prove that somewhere. For Van Hateren those old truths are beyond dispute.

Q: Are chatbots that screen candidates for skills reliable, according to Bas van Hateren?

A: Reasonably reliable, he expects, although that also needs testing. These chatbots ask about what is missing from a CV, for example when someone did something and how well, sometimes using the STAR method. Given what we know about structured interviewing, it is plausible that this works well. Here the test-retest question sits in the analysis. Does an LLM give the same judgement every time it sees exactly the same answer? Van Hateren refers to US research showing that an LLM running on its own hardware stays within that 0.8 reliability, and that it is the load on an LLM during analysis that causes the variation in outcomes. Assessment vendors may have to set up their own computing capacity.

Q: How do European and US rules on hiring assessments differ, according to Bas van Hateren?

A: Europe looks at the input, the US at the output. In Europe the question is whether the test is sound. In the US the question is whether the test produces the desired outcome, meaning whether the group that passes has roughly the same make-up as the group that applied. Van Hateren gives the example of traders, where reaction speed is a demonstrable part of the job and men score higher on it on average. In Europe it is then acceptable that mostly men pass the selection, in the US it is not. He learned a second difference from Charles Handler, his counterpart in the US. There you are legally exposed if you have no thorough, documented job analysis. In the Netherlands he has met almost nobody in all these years who has documented one.

Q: Why does Bas van Hateren think the EU AI Act sets the bar too high for hiring AI?

A: Because there is no bigger black box than the one between our ears. When a human selects, it is suddenly fine, even though human assessors are often worse and more biased than the digital systems that are judged not good enough. Because of the strict requirements, organisations fall back on those human systems. Van Hateren thinks it is fair that the bar for AI is higher than for humans, because AI can also do better. The question is whether we are setting it at an impossible height. On top of that, under the GDPR companies may no longer share performance data with external test vendors, which makes it hard to validate new tests properly.

Q: Why should you validate an assessment on your own roles, according to Bas van Hateren?

A: Because what is no longer possible externally is still allowed internally. Companies may use their own performance data in-house, and that was one of the reasons he started TA Audit. Van Hateren therefore looks less at the general validation of a test and more at whether the test measures what matters for this role in this company. Researchers first respond that you cannot simply set validation aside, but in conversation they often conclude that validating against your own roles and your own performance data is more reliable. He saw how it goes wrong at a large government body that selected on a skills model from the 1980s, while half of those jobs no longer existed.

Q: How will job applications work in five years, according to Bas van Hateren?

A: More like a portfolio, with your own AI agent applying on your behalf. That agent has had every conversation with you and probably knows better who you are than you do. An employer then has its own agent question those candidate agents, a hundred thousand in a few hours. He also sees employers that simply assume candidates use AI. Someone at a startup factory asked candidates for the chat log of their AI on a take-home assignment. Not for the result, but to see how much direction and pushback the candidate gave. Van Hateren also expects more relatively small organisations working with partners rather than employees. He advises his friends' children to start their first business while still at secondary school.

Q: Which skills will matter more because of AI, according to Bas van Hateren?

A: Judgement, critical investigation and real creativity. For judgement, he says, we still have poor tests. In his book with Kevin Wheeler he describes the true resource investigator, not in the Belbin sense, but someone like Dutch politician Pieter Omtzigt. Someone with a nagging feeling that something is missing, who can back it up and keeps challenging AI. And the oddball creatives, the people in your company you now look at and think, where did you get that idea. Soon they will be the only ones who are still truly creative, just as Gaudí once made things people frowned at that a hundred years later became the Sagrada Família.

Q: Which work will AI replace, and what remains, according to Bas van Hateren?

A: The work that sticks exactly to the script. He uses the example of a call centre with three lines. The first line follows instructions, the second is very good and the third line solves problems that have never come up before. A third-line agent once told him he is mainly very good at googling, because every problem he solves becomes the new instruction for the second line. According to Van Hateren that first line is largely being replaced by AI bots, and only a few first-line agents can move up because it takes a different mindset and skill set. That says nothing about how good they were, the job simply no longer exists. He also knows a team of ten building a product where its biggest competitor has around a hundred developers, with a token budget of about 200,000 euros a month and new features shipped within a week.

Q: Why does self-awareness matter more in an AI-driven job market, according to Bas van Hateren?

A: Because many people overestimate themselves, and a changing job market will confront them with that. Everyone is unconsciously competent, but also unconsciously incompetent. He refers to research in which about 80 percent of employees rank themselves among the best 50 percent. His tip is to ask yourself who you compare yourself with. An elite sports coach told him that with young talent they watch whether a player wants to measure themselves against the stronger group or shine among the weaker one. He saw the value of self-awareness at an HR department where he worked as an interim. After everyone took an assessment, people became aware of their irritating traits and meetings became more and more enjoyable.

Q: How should people and AI work together, according to Bas van Hateren?

A: As one team. He quotes Jack Dorsey, who wants to replace middle management with AI. Middle management filters the information that should reach the CEO and divides tasks within its own team, while an AI knows who in the whole organisation has the right skills. What remains is the coaching role, which is now often neglected because managers are busy checking. According to Van Hateren your team will consist of people and AI agents, whom you question as full colleagues and whose strengths and weaknesses you know. That also requires self-awareness about your own strengths and weaknesses. What he still sees too often is an aversion to AI out of human pride, while most of the value lies precisely in working together.