Scientifically profound, psychological test procedures for the selection of staff should predict the success of applicants for a vacancy as precisely as possible. This is only possible if the results are independent, reliable and in addition to that as well valid for the respective profession. We have summarised the quality criteria of scientific tests for you.
OBJECTIVITY: INDEPENDENT RESULTS, UNAFFECTED BY EXTERNAL INFLUENCES
Objectivity is one of the three main quality criteria to be met by a scientific test. A scientific test is objective when no external basic or marginal conditions affect the process and its evaluation.
Standardised computer-based tests in which the responses of your candidate are automatically converted into the corresponding test value achieve an extremely high level of objectivity.
By way of comparison, in the past, test methods like these were completed with pen and paper; the test managers then had to evaluate all the results – an error-prone process which can now be fully replaced with web-based test procedures with the highest level of objectivity.
In addition to that, web-based recruitment procedures work independently of demographic information (such as gender or ethnicity). Web-based pre-selection using tests is thus demonstrably fairer than pre-selection using candidates’ documents – a process which studies have shown to be particularly susceptible to discrimination of minorities.
RELIABILITY: CONSISTENTLY RELIABLE RESULTS – EVEN WITH WRONG ENTRIES
If a test is reliable it has a high level of measuring accuracy. In order to ensure the reliability of the results, the applicant is asked several relatively similar questions or asked to complete similar tasks (“items”) for each characteristic, which can later be condensed into a comprehensive description. The amount of questions ensures that each characteristic is examined in its entirety. In practice, this sometimes leads to a perceived redundancy of the test tasks, but it is necessary in order to minimise random influences.
At the same time the developed questions must be clearly differentiated from each other, from feature to feature to avoid inaccurate statements and an overlap of different test dimensions. Otherwise it would be impossible to generate a differentiated candidate profile.
The reliability of a test result is shown by the so-called reliability coefficient.
It is a universal test scale that lies between -1 and 1. In examining the reliability of a test, this value provides information on the extent to which the individual test tasks are related to each other – the higher the reliability coefficient, the more accurate and specific the test. Conversely, a low reliability coefficient indicates that the individual responses show a comparatively low level of similarity in response patterns; a test is then more susceptible to misinterpretations. The more reliable the test, the more accurate it is regarding the measured characteristic of a candidate – you as an HR decision maker will obtain a reliable result that will help you make an informed decision.
Reliability thus also means that a scientific test method is designed so that even with individual wrong entries, a reliable statement can be made for every value.
An example: if a candidate accidentally puts a cross in the wrong box for one of ten questions about his performance motivation because he is nervous, the test still provides reliable information about whether the candidate can be considered sufficiently motivated.
Reliable tests produce reliable results – which would be very similar if the test were to be repeated under the same conditions.
This quality criterion is important because only then can the long-term validity of the test result be assumed – a key prerequisite in order to justify personnel decisions based on test results.
VALIDITY: VALID STATEMENTS WITH HIGH PREDICTIVE POWER
The validity of a test procedure indicates that the test actually predicts the later work performance that has been defined as relevant for the particular profession – the test results predict exactly what they are meant to predict.
The first step towards a valid test is therefore, the question “what”: what do you want to measure? A requirement analysis in your company is used to determine in advance which criteria a candidate has to fulfil for a certain position – which is the central aspect for the professional success of a candidate who is the subject of a prediction. Using the results of the requirements analysis we define the contents of the test procedures. This is followed by the question “how?”: how should the requirements A, B and C be measured?
The use of test methods which have a so-called prognostic validity, i.e., a statistically significant relationship with a relevant performance criterion (for example, a superior assessment or customer satisfaction values), is central.
In addition, appropriate procedures for each situation should be used if possible. An example: For IT occupations, numerical skills and logical thinking are part of the main requirements. Candidates are therefore given intelligence tasks in the test involving numerical and figural content among others. These include, for example, rows of numbers and matrices - the content validity of these two features is thus fulfilled. A pure focus on writing skills here would be not effective, but may well be part of the diagnosis, when additional project management or implementation tasks are provided. The validity of a test is therefore always dependent on the question as to what is to be predicted – the method of data collection is coordinated with current requirements of the position. In this way, requirement criteria can be taken into account that are less related to the job itself than to its constraints. For example, characteristics such as a high degree of openness for new experiences can be defined as an additional requirement in order to predict whether the person will also be successful at locations abroad.
THE TEST FOR THE TEST
In order to continuously improve the quality of our scientific test procedures, we at HR Diagnostics carry out regular evaluation studies. Together with our customers, we examine, among other things, the extent to which the predictions of the tests for professional success coincide with reality – whether, for example, the high test result of an applicant is actually accompanied by top performance in the profession. This can be done in retrospect or before the project is introduced by the already established candidates carrying out the test and the results compared with the available performance data.
The strategic orientation of a company can also be taken into account: the desire for a new management style, a more innovative orientation or the change from a purely sales orientation to customer orientation change both a company and the requirements for future employees. We have these developments in mind and adapt our test procedures to your company strategy – in accordance with scientific criteria.
