The analysis of combinations of behavioural competencies to predict the success of a professional in his or her post.
Download our white paper : Evidence Based Executive Search
Executive search & leadership assessment
The goal of this white paper is to describe how Uman Partners have developed a proprietary approach that is quantitative, predictive and explanatory, relevant and robust, to determine the different possible combinations of key skills and environmental parameters (business culture, management, etc.), in order to maximise the probability of success of a professional in his or her future post.
This is done using analytics (“Data Science”) technologies that allow more detailed, more precise and richer results to be produced on the subject of their operational performance, than the tools created today by the big market players using traditional statistical approaches.
The founders of Uman Partners, as well as having much experience in the search industry, have been pioneers in Europe in the development and implementation of this type of mathematical approach for companies since the mid-2000s, and have contributed to the development of these technologies that are today recognised and integrated into large industrial or consulting groups (Dassault Systèmes, Deloitte, BearingPoint, etc.).
A BENCHMARK FOR COMPETENCIES; BEHAVIOURAL COMPETENCIES AS A DETERMINING CRITERIA FOR SUCCESS
Most large industrial groups, big players in the service industry or Human Resource consulting and head hunters have based their evaluations first and foremost on technical skills. This was an essential and necessary pre-requisite, but it has not been sufficient.
That is why many behavioural competency frameworks that can be seen as “universal” were developed, designed by public or private organisations, applying their segmentation on qualitative studies, which we cannot call into question today, since their results all concurred.
These frameworks map out in an exhaustive but disjointed way all (leadership) behavioural competencies of managers, experts, or future managers. Depending on the approach, the competencies are described according to two or three different ways of carrying them out:
– Proficiency in the competency
– Overstretched competency
– Lack of proficiency in the competency
And each competency can be qualified as:
– Rare / frequent among professionals
– Difficult / easy to acquire for professionals, depending on whether it is innate or not.
Furthermore, various public quantitative studies from independent organisations, synthesising 85 years of research, have shown the high superiority of the predictive capacity of interviews structured around the measurement of (leadership) behavioural competencies of Talent, with regard to their level of success in their future post (Wiesner & Cronshaw* – 1988, McDaniel, Whetzel, Schmidt & Maurer – 1994, Schmidt & Hunter** – 1998).
*The superior reliability of the structured interview confirmed by the analysis of 51,459 interviews (Wiesner & Cronshaw, 1988)
**An analysis of the predictive validity of 19 different selection methods as well as the combined use of different methods (Schmidt & Hunter, 1998)
A summary of these results enables a classification of the 6 most used methods for their predictive capability:
PERCENTAGE OF RELIABILITY OF SELECTION TECHNIQUES*
– Intuitive face-to-face interviews: 20%
– Taking up references: 26%
– Assessment centre: 36%
– Panel interviews: 37%
– Skills tests: 53%
– Structured interviews based on the evaluation of skills: 70%*
* This percentage is the probability of successfully predicting the professional’s success in his or her post following the exercise (Assessment, Test, etc.).
It is, therefore, a major challenge to determine the combination of behavioural competencies that a professional should possess, for each type of post and in each environment (or transversely independent of the environment), for success in a post (“success” according to the criteria defined here later).
We will describe how, with very few data points and with simple means of measuring competency, Uman Partners can determine:
– Rules describing the combinations of competencies (preferable and differentiating, ranked according to their influence) that express the success of a Talent for any given post. It is in fact very frequently the case that requirements for proficient competencies are over-specified. Uman Partners provides the correct requirement, and thus facilitates the search for Talent (avoiding the “search for the impossible” that happens so frequently).
– Recommendations on integrating the Talent into his or her post with a factual description of his or her competencies and the degree to which they are proficient (resulting in a personalised training plan depending on the difficulty level of acquiring the competency); as well as a recommendation on complementary competencies that the professional should enhance to compensate for his or her partial or complete weaknesses in the required competencies.
These unique recommendations are accessible thanks to the technologies used by Uman Partners.
– The leverage effect of each of the rules (the multiplying factor of the frequency of success of individuals in comparison with the whole of the population measured).
METHOD FOR MEASURING BEHAVIOURAL COMPETENCIES
This is the crucial point to demonstrate the validity of the results compiled by Uman Partners: the measurement of behavioural competencies in an individual. We will see in the “Calculation Method” paragraph below that the relevance and precision of the measurement (as empirical as the measurement means / method is) is validated retrospectively by the calculation.
Public searches indicate that behavioural competencies should be validated:
– by their relationship to the action;
– by their relationship to the context;
– by the modes of exchanges with the “client”;
– by the modes of cooperation within the team*.
* wording presented here according to Sandra Bellier, Director of Research and Development at Cegos
This is how Uman Partners, like other market players, measures how much the candidate has mastered the competency using sets of questions posed to the individual that are mentally placed against their relationship to the action, context, modes of exchange and modes of cooperation.
Uman Partners has constructed a structured database in the following way:
– In the rows, the people interviewed who are currently in post are indexed
– In the columns, are the benchmark competencies, and the environmental characteristics of the post (sector, company size, job title, company characteristics, cultural parameters, etc.).
– The last column is the descriptive variable of success in his or her previous post (or the job that is becoming their previous post) of the person interviewed.
We have deliberately chosen for this case a binary output: 1 for successful; 0 for unsuccessful.
But this is the result of at least 4 components:
– The person has a proposal for changing position after less than 3 years
– The revenue within his / her scope has increased by more than x%
– The profitability of his / her scope has increased by more than y%
– Staff numbers in his / her scope have increased by more than z%
– The person has brought a significant impact to his / her scope of responsibility (to be assessed or at the discretion of experts who are senior consultant).
The Person / Competency cells contain a grade between 1 and 10 awarded by the consultants that interviewed the person, if the competency has been measured. If not, the box is left empty. (A mark of between 1 and 3 indicates that proficiency in the competency is insufficient; between 4 and 7 indicates that that it is mastered; 8 and above that it is over-stretched). This grade between 1 and 10 provides a nuanced description and a detailed analysis of results.
Important comment on the amount of data necessary:
Market analysis technologies facilitate the processing of very sparse databases. Thus, the design of the Uman Partners’ tools does not require testing all the competencies in the framework for each person interviewed, but only 15% of them.
Once the database is constructed, the analysis is made following an iterative process that is specific to the technology in question. This is an inductive approach to artificial intelligence (rather than deductive).
It can be summarised in the following way:
First of all, every point in the database is represented in a space in n dimensions (n being the number of variables considered, that is the benchmark competency and the environmental variable):
Next, using successive iterations, two points from the same family are taken at random in the space to define a proficiency area in which the local influences will be analysed combined with the variables and the optimal ranges for each variable, to maximise the density of points representing the searched for result.
We see in this diagram on the right, that the analysis allows us to find an area that is denser (in the green area), bringing together a great number of successful professionals in a very marked way.
The analysis of local influence (i.e.: for each hypercube) in each variable allows us to determine the rules describing the combinations of competencies (and environmental variables) that describe and predict the success of a professional in his or her post:
People with the following behavioural competencies in the described environment:
– Speed of learning 7
– Ability to manage ambiguous situations >8
– Ability to analyse numerical data >6
– Results-driven >6
– In a group with more than 10,000 employees
Are 5.4 times more often successful in their post as Project Director
For each post, several profile types may thus deliver success more frequently. Therefore several rules are generated for each post.
(This analysis is made for each type of post.)
– All the rules found for each post then allow us to construct a simple tool that simulates down-graded situations and thus allows:
– the measurement of risk incurred in recruiting a candidate for whom certain required competencies are not mastered or not well mastered.
– to determine which other competencies can partially compensate (and “how much”) for the absence or lack of certain key competencies
– to define the coaching and training focuses of the candidate during his or her integration (on-boarding).
– The relevance of measuring competencies, the choice of competencies measured for each candidate and the success indicator, is validated by a specific calculation. Its principle is simple: we are seeking to verify that the correlation established between competencies, the environment and success is not linked simply to chance (i.e that the orange Os in the diagrams — the successful people — are not grouped together by chance). To do this, we generate a database in which we randomise the output and we compare the results obtained with the real database.
OPERATIONAL TOOLS SHARED BETWEEN UMAN PARTNERS CONSULTANTS AND CLIENTS
UMAN PARTNERS has thus produced a tool for the consultants and their clients that is very easy to use.
It is made up of 5 elements:
– A set of benchmark competencies that describe in a detailed manner each competency in 3 ways: proficient / not proficient / overstretched
– For each competency, a note that qualifies it as “rare” or “common” in the general population (according to thresholds selected statistically)
– For each competency, a note that qualifies it as “easy to acquire” or “difficult to acquire” in the general population (according to thresholds selected statistically)
– For each competency, a series of 6 role-play questions whose responses measure the degree to which the competency has been mastered. These questions are to be shared between different interviewers (consultants, client stakeholders) and the pooling of the responses facilitate a robust evaluation of whether the competency has been mastered
– For each post type, a set of descriptive rules (between 1 and 4) of successful profiles in their environment. Each rule is graded with a factor corresponding to the loss of the leverage effect of the rule if the factor is removed.
For example, if a rule describes a profile that has 3.5 times more probability of success than the average in the observed population, and one of the factors (one competency for example) is assigned a factor of 2, that means that a talent who does not possess this competency belongs to a population whose probability of success is not 3.5 times more, but 3.5-2=1.5 times more than the overall population.