Case Studies

Read about some of our successful People Analytics-Projects.

Case Study 1

Evaluation and Optimization of a Selection Process for Insurance Sales Agents

 

Client: a German insurance company operating internationally

 

Starting point: The client wanted to install a new selection process for field sales agents. The new process was going to be composed of three parts, each comprising a number of evaluative instruments:

  • a structured interview with various interview questions
  • an intelligence test with tests for verbal, numerical and figurative intelligence
  • an exercise part with various role play exercises

Our task was to evaluate the new process and, if necessary, improve it.

Step 1: Evaluation of the new selection process
Questions to be answered

  • What is the overall predictive quality of the new selection process, i.e. how well does it predict a candidate’s future success as an insurance sales agent?
  • Are some parts of the process better than others at predicting future success?

Required data

  • Results from the new selection process for a sufficient number of candidates
  • Quantitative data about the subsequent job success (sales performance) of those candidates

Data gathering

 

In order to evaluate the predictive quality of the new selection process, it was necessary to measure subsequent performance both for candidates who passed the new process and for candidates who did not perform well or failed the process. Otherwise, variance would have been too limited and we would not have been able to use variance-based analytic methods such as regression analyses.

 

This meant that participants for the study had to be chosen from among the sales agents already working at the insurance company. In order to avoid possible negative consequences for participants, all data was gathered and evaluated anonymously. 80 sales agents were chosen to go through the new selection process and receive a score of their aptitude. Subsequently, we evaluated the sales performance of each candidate during a period of 18 months, using the number of sold non-life insurance policies and the value of sold life insurance policies as quantitative measures of success.

Evaluation

 

In order to evaluate the predictive quality of the selection process, participants’ aptitude scores were correlated to their performance data by means of a multiple regression analysis. Additionally, a factor analysis was performed in order to assess the internal consistency of the new selection process.

 

In the first stage of the factor analysis, the three parts of the new selection process were evaluated separately in order to determine whether their instruments (e.g., the interview questions) showed correlations among themselves, that is to say, whether they showed a tendency to produce similar values for any given candidate independently of the characteristics they were supposed to measure.

 

In the second stage of the factor analysis, we tested for correlations between the three parts of the selection process in order to determine whether they showed a tendency to produce similar values for any given characteristic and candidate.

Results

The regression analysis showed no correlation between a good score in the selection process and future success on the job. Among the three parts of the process, only the intelligence test showed some predictive power, although correlations were only in a medium range.

 

The only characteristic that showed a significant predictive value for future success was previous knowledge of the insurance sector at the time of hiring.

The factor analysis revealed significant correlations between instruments when looking at each of the three parts separately, but no correlations across the three parts of the process.

 

These results were the opposite of what would be expected of a valid selection process (namely, low correlations between instruments but high correlations between parts). This was taken as evidence that the selection process was dominating over the characteristics to be measured, or in other words, that the process was measuring artefacts.

Step 2: Identification of predictors for the success of sales agents
Questions to be answered

  • Are there any personality traits that can be measured during a selection process and that correlate significantly with success as a sales agent?
  • Are there any factors on the organizational level that correlate significantly with the success of sales agents?

Required data

  • Quantitative data regarding the manifestation of certain personality traits among the 80 participants (e.g., goal orientation, enthusiasm, conscientiousness, etc.)
  • Quantitative data regarding the manifestation of certain characteristics by the organization (e.g., concern on the part of superiors for the professional advancement of sales agents, trust and appreciation for sales agents on the part of their superiors, etc.)

Data gathering

 

An anonymous survey was carried out among the 80 participants from Step 1 as well as their line managers and heads of agency. The personality traits were measured by means of a self-assessment by the 80 participants as well as assessments by their superiors.

Evaluation

 

The quantitative results of the survey were correlated with the performance data gathered in Step 1 by means of a multiple regression analysis.

Results

A number of factors were determined to have a significant or even strong correlation with success as a sales agent:

  • On the individual level: clarity about goals, size of goals, enthusiasm, sales orientation, endurance, adherence to deadlines for routine tasks
  • On the organizational level: concern on the part of superiors for the professional advancement of a sales agent, trust and appreciation on the part of superiors, sympathy on part of superiors for personal issues of the sales agent

Step 3: Development and validation of an optimized selection process
Question to be answered

If the selection process is modified to include a quantitative assessment of the individual predictors of success identified in Step 2, then what is the predictive quality of the modified selection process regarding the future success (sales performance) of candidates?

Required data

  • Results from the modified selection process for each candidate
  • Quantitative data regarding the success (sales performance) of each candidate

Data gathering

 

First, the new selection process was modified in the light of the insights gained from Step 2. The role play exercises were discarded and instead the interview was amended with questions designed to measure the personality traits that had been identified as predictors of success. Subsequently, the 80 participants went through the modified selection process and received new aptitude scores.

Evaluation

 

The results of the modified selection process were correlated with the existing performance data from Step 1 by means of a regression analysis.

Results

The modified selection process showed good predictive power for success as a sales agent. On average, the sales performance of agents who passed the modified selection process was significantly higher than that of agents who did not pass the modified process.

Within the subgroup of agents who had no prior knowledge of the insurance industry at the time of hiring, the difference in sales performance was strongest, with “apt” sales agents outperforming “unapt” agents by almost 100%.

Case Study 2

Client: a major German insurance company

 

Starting point: The client was faced with high unwanted attrition of sales agents within the exclusive sales organization.

 

We were hired to determine the causes of attrition and suggest measures to reduce it.

Questions to be answered

  • What are the reasons given by dropped-out sales agents for leaving the company?
  • How do dropped-out sales agents rate their working conditions and superiors within the organization compared to sales agents who remained with the company?
  • Are there any significant differences in personality traits between agents who left and agents who remained with the company?

Required data

  • Qualitative data regarding the reasons given by dropped-out sales agents for leaving the company
  • Quantitative data from dropped-out and remaining sales agents regarding their satisfaction with working conditions and superiors
  • Quantitative data from dropped-out and remaining sales agents regarding their success (sales performance), their aptitude for sales activities and other personality traits

Data gathering

 

A survey was conducted among 35 dropped-out and 34 remaining sales agents. For an additional perspective, the former line managers of the dropped-out agents were questioned as well.

Evaluation

  • Comparison of the perspectives of dropped-out sales agents and their former line managers through means comparisons on the answers to the questions from the survey
  • Multiple regression analysis to identify personality traits with a strong correlation to success
  • Absolute and relative frequency evaluations on the answers from the survey

Results

  • Reasons for leaving: About one third of dropped-out sales agents said they had left the company because they had realized their own inaptitude for sales activities (an assessment that was confirmed by their former line managers).

The other most frequently given reasons were: conflicts with superiors, lack of reaction on the part of superiors to voiced dissatisfaction, dissatisfaction with earnings, and a better offer from a competing company.

  • Satisfaction with working conditions and superiors: Satisfaction with development opportunities, value of in-force business and sense of involvement was significantly lower among dropped-out sales agents than among those who had remained with the company.

Regarding their former superiors, dropped-out sales agents criticized a lack of sales support, insufficient encouragement to develop professionally, low reliability in keeping promises, as well as a lack of human attention, appreciation, trust and sympathy.

  • Personality traits: The biggest difference between dropped-out and remaining sales agents was in the number of previous jobs held before joining the insurance company, which was significantly higher among drop-outs than among remaining agents.

  • Additional insights from the superiors’ survey: The survey revealed that the variable part of the remuneration for higher levels of sales management at the insurance company depended heavily on the number of new hires. This can be seen as a structural factor favoring attrition as it promotes hiring of candidates without regard to their actual aptitude for sales.

Furthermore, superiors confirmed that in many cases the organization had responded inadequately to the expressed dissatisfaction of sales agents who then ended up leaving the company. Structural factors were given as the principal reason for this inadequate response. On the one hand, high administrative workloads and manager-to-staff ratios stand in the way of intensive personal supervision of sales agents, with the consequence that superiors often do not find out about an agent’s intention of leaving the company until it is too late.

 

On the other hand, retaining good sales agents requires more flexible structures such as shorter chains of command and individually adaptable remuneration and employment models.