By Stephen D. Silver, MBA and Stephen Berke, MBA, Data Management.

Competence.

It’s easy to assess after someone has been hired. But for the information manager who wants to increase the likelihood for success, using personnel testing in regular selection procedures has proved beneficial.

Using personnel testing can lead to meaningful improvements in turnover, productivity, training effectiveness and morale by using job-related, performance-oriented tests.

DP faces people

Managers are trained to forcefully and decisively attack business problems. Instructed in analytical problem-solving techniques, Dpers try to get a handle on the day-to-day technical difficulties. While problems involving hardware and software decisions, operational problems and budgets are solved, people problems tend to be downplayed or ignored. People problems tend to be more complex, less easily solved, and are not obviously related to the success of the shop nor reported on in financial statements.

Overlooking people problems is the leading cause of failure of business systems development. A sampling of Walden’s client base reveals some important findings. At least 60 percent of the shops surveyed had turnover rates of programmers, analysts and operators of at least 40 percent, with rates of 50 percent to 65 percent not uncommon. This represents a complete change of staff at this critical level of skill every two to four years. How can any planning be successful with this human resource problem?

Moreover, most companies recruit at least 75 percent of their middle and senior management IT staff from outside the company. Relatively few organizations had formal employee appraisal systems in their data processing organization.

The bottom line of these people problems tends to be low efficiency, uncertain employee loyalty and induced job-hopping. Most of the classic problems that surface – projects seriously behind schedule, incomplete conversions, disaffected users and inefficient systems – revolve around the forgotten people resource.

If this effort were spent at the selection stage and the best tools available were used, DP will slowly build a much better team, thereby improving the entire data processing resource.

Common selection techniques

There are only a limited number of selection techniques available to DP management. Yet the decision to hire is often made “on good looks alone.” A succession of poor hiring decisions can be disastrous, especially when they involve a critical project or a senior position.

The most common employee selection techniques are: 1) Evaluation of past work history from the curriculum vitae; 2) interview; 3) reference checks; and 4) objective testing. None of these techniques if used alone is completely effective, and each is more or less subjective (except testing) as opposed to being objective.

Walden has found that in at least 60 percent of new hires, the candidate does not meet initial company expectations, and approximately 44 percent of those who survive one year leave in the second year of employment. Given this tendency, it is imperative that better selection techniques be used.

Evaluation of past work history would appear to be a fairly objective method.

Yet this method is deceiving. A curriculum vitae or resume can appear to describe a first class work history; yet the employee may have been a weak performer, or may have relied on others to carry him or her through. Calling past employers to verify the job history is a highly questionable practice since many persons are wary of providing unfavorable information on the telephone. Taking the time to write for information usually means you have lost the candidate to another shop.

The reference check has a similar weakness.

Candidates seldom provide references of persons who will speak poorly of them. Unless the candidate is known personally by a responsible member of your company, any positive inferences from the reference check must be heavily discounted, to a degree, depending on the situation.

The interview is the most common selection technique used. However, the interview often is abused because, in most cases, it is almost entirely subjective. Even with a systematized and validated interview, it is impossible to remove all the hidden biases that can imperceptibly enter the interview. Style of dress, an extroverted personality and perceived motivation can improve the “aura” of the candidate. Also, some people are adept at taking interviews, while many other highly capable persons are poor at marketing their most valuable commodity – themselves.

These subjective techniques must be supplemented by an objective, job-related test to measure skills related to the position. The interview is much more valuable when the interviewer has an objective report on the candidate’s abilities and work habits. Managers omitting this important factor may not only hire weak performers, but may reject superior candidates as well.

How can testing help?

If all candidates for both entry-level and experienced positions, as part of a pre-defined, consistently applied policy, are required to take a test, better qualified persons will be selected.

This has been proved time and time again. The graph (See Figure 1.) reflects what happened at a very large bank with approximately 2,200 data processing employees at all skill levels. The conclusion is that DP staffers who were hired with testing as part of the selection process were rated higher by their managers than those candidates who were selected without having been tested by a margin of about 20 percent, over the eight year study period. Ratings of supervisors ranged from low competent to excellent, and the graph also reflects the percentage of the programming staff in each supervisory rating category.

article5_graph

Performance ratings

  • TC: Candidates testing for programming aptitude before hiring.
  • NC: Candidates not tested for programming aptitude before hiring.

———————————————————-

The use of testing before hiring also tends to reduce employee turnover.

The most common reasons for programmers leaving their jobs are: A higher salary offer and greater opportunity for advancement. When superior programmers are not recognized as such, and where the programming supervisor does not have adequate flexibility to suitably reward the superior performer, turnover will increase. The use of objective testing will provide a tool for data processing managers to keep track of persons with superior potential.

A recent study at a leading insurance company found that, when using a recognized aptitude test, the company realized substantial cost savings and reduction of turnover (See Figure 2.).

Company cost per trainee Dropout rate Total cost Net savings
Using existing methods $25,000 7.5% $187,500 -
Using recognized test $25,000 3.3% $82,500 $105,000
Less cost of tests $6,000
Savings $99,000
Figure 2. Cost benefit analysis

———————————————————-

Besides cost savings, many companies also realize improved morale and performance when better qualified persons are hired and where management can recognize and reward the superior. Companies make better use of their recruiting and selection effort as well. This occurs because superior people are not rejected based on subjective reasons by recruiters. In so many companies, the interviewer will not have any future contact with the applicant, yet is allowed to make subjective hiring decisions without the ultimate responsibility for performance.

How to evaluate aptitude tests

Once the decision is made to use personnel testing, how does DP management go about choosing which test to use? A suggestion is to consider at least three guidelines: Tests should be relevant, reliable and predictive of on-the-job success.

RelevancyA – A relevant test measures those abilities that are critical to job success. These factors, called job criteria, differ greatly for the various data processing positions, but within each position, they do not appear to have changed very much over the years. Specifically, this means DP should not use a single test for all data processing jobs. The abilities that make for successful computer operators, seldom assure success as a programmer. The same goes for systems analysis work. A test should be specifically designed to measure the abilities of each position.

ReliabilityA – A reliable test should be shown to have predicted success on-the-job over a long period of time and in a wide range of applications. This information can usually be obtained from the publishers or by contacting long time users of the test.

Predicting on-the-job successA – This ability probably is the most important factor in selecting a test. This ability also is important in the legal framework of testing. Management should ensure that required studies have been done to prove that the tests are valid for the job skills that they are used to evaluate. Although it is desirable for clients to do a validation study, this is not always legally required. An employer may, under many circumstances, rely on the validation studies of other organizations with similar job descriptions of each position. A study should demonstrate the “close agreement” by statistical measures of test scores and supervisory ratings of performance on-the-job by the candidate. A typical result of such a study is shown in Figure 3.

———————————————————-

Validity coefficients computer between the AABP (raw-score, coded adjectival descriptor, and percentile equivalent) and four criterion dimensions are statistically significant and are listed below.

Validity coefficients
Job criteria Test raw scores Test percentile equivalents
PP .49 .49
PA .61 .65
MS .45 .46
SA .55 .55
  • PP – Programming Performance
  • PA – Programming Aptitude
  • MS – Managerial/Supervisory Aptitude
  • SA – Systems Analysis Aptitude
Figure 3. Typical validation study result

———————————————————-

Optimum test fairness

Not only is the questions of test fairness socially desirable, but government regulations for hiring mandate that employers should be as unbiased as possible in hiring decisions.

For the selection of prospective programmers, analysts or operators, Walden’s feeling is that fairness is improved by using tests. When evaluating applicants for data processing skill, a test that offers the following characteristics is optimum.

No multiple-choice tests –Multiple-choice tests do not allow the candidate to show what he or she can really do on-the-job. As well, this type of test will tend to favor the person who is “test-wise” and is experienced at test-taking. Better suited are the tests which simulate actual on-the-job performance, and where partial credit is given for partial work done.

Moreover, a multiple-choice test does not allow the evaluator to consider work habits which are sometimes very important to job success. These include ability to document and annotate one’s work, the ability to interpret intricate specifications, the ability to desk-check one’s work carefully and the ability to sustain concentration over a long period of time for programmers and analysts.

No time-limited tests — A test which is time-limited will tend to penalize persons who may be slower, yet very careful in work habits. The employer may reject persons who could potentially become superior performers but who may take longer than the test allows. Speed of programming, for example, is not always related to excellence of work. Less time debugging the program on the computer may provide higher returns than the cost of time taken to actually write the programs.

Physical security over the test taking procedure — Many tests are offered where the candidate can obtain a copy of the test quite easily, or has taken the test before without the potential employer’s knowing about it. To be fair to all candidates and the employer, DP managers should only consider tests where a record is kept of all candidates who have taken the test. Also, separate norms should be developed for repeaters, if they are to be required to take the test a second time.

Important DP hiring criteria

The Walden organization has evaluated over 200,000 candidates for positions in data processing departments for the past 25 years. Surprisingly, the essential traits for success in the profession have not really changed. Even with the furious pace of technology that has created almost an unrecognizable computer department is most organizations today, this research has isolated several important traits without which, success is almost impossible to achieve. Thus, it is critical to weed out of the hiring stream all candidates who do not possess these skills.

For programming candidates, it is recommended that the employer measure and select according to these traits: Logic ability, interpretation of specifications, documentation, sustained concentration, ability to follow instructions, extremely high accuracy, relatively quick working pace and high attention to detail.

For candidates aspiring to the systems analyst or programmer/analyst position, extensive research indicates these traits: Ability to plan logical procedures for multi-step work flow, ability to interpret intricate specifications, alertness in finding alternate ways of proceeding, documentation of each step, relative effectiveness and efficiency of design, quality of system design organization, understanding organization interrelationships and ability to develop systematic procedures.

Keep these traits in mind when initiating a search. If these traits are discovered in an applicant, measure them accurately and use the results of the selection techniques to “zero in” on the best of the group, success is inevitable.

Government regulations and testing

Many executives who are new to testing ask whether they are permitted by law to use objective tests as part of their selection process. The answer is yes. However, employers must conform to a set of regulations designed by Federal government agencies to ensure fairness, and that the tests or other selection or appraisal techniques actually do measure what they claim to do. Before you start the process, you should check with the professionals within your organization to make sure that you are operating within the guidelines established. For performance or job-related tests, the rules of the game can generally be established and justified. The validation process and legal framework are too complex to cover here. For more information, a test vendor or appropriate professional can be helpful.

Don’t hire on good looks alone

Information managers have gone a long way to solve people problems if they can resist the short term expedient of hiring “on good looks alone!” Remember, success as a manager will result from the skillful interplay of all available resources — hardware, software and personnel. Up the odds for success by using testing to reduce the odds of poor staff selection.

About the Authors

Silver and Berke, president and vice president, respectively of Walden Personnel Testing & Consulting Inc., market a leading series of job related tests for evaluating data processing personnel. The tests have been used by over 2,000 organizations since 1968 in the selection, training and promotion of data processing staff. For more information contact Walden Personnel Testing & Consulting Inc. at 1-866-383-8446 -Ed.