What is Validity and why is it important when selecting an assessment?
Employee Assessment- This is important because in order to use an assessment properly (legally), you need to know the assessment measures what it says it measures. For example, using an assessment validated for post-employment purposes as a hiring tool is an inappropriate use of the instrument and can have severe legal implications. (See additional information on validity listed under definitions at the end of this section.)
What is Reliability and why is it important?
Reliability refers to how dependably or consistently an assessment measures a characteristic. In general, lower test-retest reliability indicates the assessment is not measuring core behavioral traits but is assessing states which are subject to change with mood or circumstances. Knowing the level of reliability for each attribute within an assessment is critical when using the assessment in the selection process to predict future performance is critical. A good test manual should provide detailed information on the types of reliabilities reported, how reliability studies were conducted, and the size and nature of the sample used to develop the reliability coefficients. Independent reviews also should be consulted. (See additional information listed under definitions at the end of this section.)
Do your assessments comply with all state and federal requirements (EEOC, Title VII of the Civil Rights Act, Age Discrimination in Employment Act, ADA, etc.)?
Yes, our assessments, when properly administered, represent non-biased assessment tools which conform to the requirements of federal and state law.
What is meant by “Adverse Impact”?
One of the basic principles of the Uniform Guidelines is that it’s unlawful to use a test or selection procedure that creates adverse impact, unless justified. Adverse impact occurs when there is a substantially different rate of selection in hiring, promotion, or other employment decisions that work to the disadvantage of members of a race, sex, or ethnic group. As noted above, our assessments comply with federal and state law; nonetheless, employers using our assessments should remain aware of the effects of the specific administration of any actual assessment and the specific results achieved.
What is the number one predictor of success on a job?
Based on research, “General Mental Abilities” (a.k.a. “Thinking Style) is the number one predictor of success on a job. This doesn’t mean “smarter” is better; it simply means matching a person to the demands of the job (i.e., being fully engaged) is the best predictor of future success. Any selection or placement assessment which doesn’t include a comprehensive cognitive section is missing a significant piece of the selection (hiring) puzzle. (Reference: Psychological Bulletin, September 1998, Vol. 124, No. 2, 262-274, “The Validity and Utility of Selection Methods in Personnel Psychology – Practical and Theoretical Implications of 85 Years of Research Findings” by Frank L. Schmidt, Department of Management and Organization University of Iowa and John E. Hunter, Department of Psychology Michigan State University)
What is the difference between a pre- and a post-employment test?
Pre-employment tests are those which are validated for use in the hiring process while post-employment tests are those which are validated for use after hire. Some assessments (such as an Integrity Test) are validated for use only during the hiring process. Others, such as our ProfileXT, are validated for use in both the pre- and post-employment process; while others (such as team-building or personal development assessments) are only validated for use post-employment use.
What is meant by a Performance Model?
A Performance Model (also called a “benchmark” or “success” pattern, “or “job match pattern”) refers to a composite picture of the characteristics of top performers produced by assessing and analyzing a sample group. Performance Models are developed to help employers understand what characteristics and traits tend to be associated with successful performance in a specific position. Performance Models are developed utilizing the actual assessment results from the top performers using the PXT Select™ assessment. Thus, the Performance Models reflects the characteristics which are typically found in those who are found to be successful at the job under study. Typically, when someone’s score falls inside the desired pattern, their overall job match percent is higher than the job match percent for those who fall outside the job match pattern. The higher the overall job match percent reported, the higher the expectancy that individual will fit well into the job under consideration.
Do you have to be “certified” in order to administer and interpret your assessments?
No, all our assessment results are designed to be used and understood by the average manager. All the reports are written in clear, easily understandable language and require no interpretation.
Why do you refer to tests as assessments?
Quite simply, the word test implies someone taking the instrument can fail; and that’s just not the case, you can’t fail who you are as a human being. You may be a poor fit for a particular job, but that doesn’t mean you’ve failed as a human being; it only means you were a poor match for that particular job. The term “assessment” is a much softer and more acceptable term to use when describing psychological tests.
What is an Integrity assessment and why is it only used for pre-employment testing?
Integrity assessments are essentially “honesty” tests (our Step One Survey II is an example of this type of assessment.); a specific type of personality test which is broadly categorized into two types.
- Overt integrity tests gauge involvement in and attitudes toward theft and employee delinquency. Test items typically ask for opinions about frequency and extent of employee theft, leniency or severity of attitudes toward theft, and rationalizations of theft. They also include direct questions about admissions of, or dismissal for, theft or other unlawful activities.
- Personality-based measures typically contain disguised-purpose questions to gauge a number of personality traits. These traits are usually associated with a broad range of counterproductive employee behaviors, such as insubordination, excessive absenteeism, disciplinary problems, and substance abuse.
According to the U.S. Department of Labor, “it is generally recommended these tests be used only for pre-employment screening. Using the test with present employees could create serious morale problems. Using current employees’ poor scores to make employment decisions may have legal repercussions when not substantiated by actual counterproductive behavior.” A number of states currently have statutes restricting the use of honesty and integrity measures. At least one state has an outright ban on their use. Consult regulations in your state that govern the use of honesty and integrity tests before using them.
What are the differences between a normed (normative) assessment and one that is ipsative and which is preferable?
Normative is a type of scoring produced by testing a large population and generating a normal bell curve distribution of the results. The distribution is then divided into standard tenths creating a quantified, normal scale with which to measure and compare individuals. (This type of scoring is typical of many of our assessments.) Normative assessments are typically validated for both pre- and post-employment use and are extremely helpful in predicting future performance.
Ipsative literally means “of the self” and it is a type of scoring generated by forced choice items (e.g., select the word which MOST describes you and the word which LEAST describes you…). “Assessments” developed using such (scoring) instruments (such as the Myers-Briggs, DISC, etc.) are “innocent of any psychological meaning.” (Kline 1993). Ipsative assessments have the added burden of not containing distortion scales; and, as a consequence should never be used in the hiring or selection process. In many cases, their Technical Manuals specifically prohibit their use for hiring or selection purposes (unfortunately, these assessments are often sold as selection tools).
What is the significance of distortion and why is it important?
The Distortion Scale deals with how candid and frank the respondent was while taking the assessment. In the case of our assessments, the lower the distortion score, the greater the level of concern the results may have been distorted.
The distortion score refers to the reliability of the results, not the honesty of the individual. A low score would suggest the individual might have distorted their responses. This could possibly happen because on an attempt to portray a picture of how they would like to be seen rather than an accurate picture of how they are. The distortion score should never be used as a basis for disqualification of an individual.
What is meant by Faking?
This refers to attempts by the test participant to misrepresent their true behavior through exaggeration, distortion, equivocation, avoidance, or some other means. (See Distortion.
What does the term Psychometrics mean?
Psychometrics is the science of measuring the characteristics of human behavior, personality, cognitive abilities, interest, and/or aptitudes.
What is the difference between a personality test and an assessment that measures core competencies?
Personality-style tests (ipsative assessments like the DISC and Myers-Briggs) measure traits like extroversion, methods of processing information, and how people influence others. There is very little correlation between these tests and subsequent performance. One problem is these tests measure preferences, not competencies. For these reasons, never use personality tests to predict future performance.
Why should you ask a test provider to furnish a Technical Manual?
The “Technical Manual” is a step-by-step description of how the assessment was constructed. It outlines the various constructs used by the assessment and the basis of their formulation. Simplistically, it demonstrates the science behind an assessment and provides the user with information concerning the test’s development, purpose, validity, and reliability. Without a Technical Manual, an employer can’t assess the above and, in the event of an EEOC challenge, is most often in a defenseless position to justify their use of the assessment.
Are Internet-based assessments better than paper/pencil versions?
Not necessarily, it’s usually a matter of convenience. Advantages to Internet-based (or computer-based) assessments include:
- Administration procedures are the same for all assessment takers.
- The need for test administrators is reduced.
- The results can be available immediately.
- The assessment can be administered without delay to walk-in applicants.
According to the U.S. Department of Labor’s Testing and Assessment: An Employer’s Guide to Good Practices, any test or procedure used to measure an individual’s employment or career-related qualifications and interests can be considered a personnel assessment tool. There are many types of personnel assessment tools. These include traditional knowledge and ability tests, inventories, subjective procedures, and projective instruments. In their guide, the term test is used as a generic term to refer to any instrument or procedure that samples behavior or performance. Personnel assessment tools differ in:
- Purpose, e.g., selection, placement, promotion, career counseling, or training
- What they are designed to measure, e.g., abilities, skills, work styles, work values, or vocational interests
- What they are designed to predict, e.g., job performance, managerial potential, career success, job satisfaction, or tenure
- Format, e.g., paper-and-pencil, work-sample, or computer simulation
- Level of standardization, objectivity, and quantifiability—Assessment tools and procedures vary greatly on these factors. For example, there are subjective evaluations of resumes, highly structured achievement tests, interviews having varying degrees of structure, and personality inventories with no specific right or wrong answers.
All assessment tools used to make employment decisions, regardless of their format, level of standardization, or objectivity, are subject to professional and legal standards. For example, both the evaluation of a resume and the use of a highly standardized achievement test must comply with applicable laws. Assessment tools used solely for career exploration or counseling are usually not held to the same legal standards.
What’s the difference between a personnel test and a personnel assessment process?
A personnel test or a procedure provides only part of the picture about a person. On the other hand, the personnel assessment process combines and evaluates all the information gathered about a person to make career or employment-related decisions.
What weight should be given to the results of an assessment?
The publisher of our assessments states the results of our assessments should never be more than one-third of the input to a hiring decision. Specifically, “As discussed in the User’s Guide for this product, this job pattern approach to matching individuals to a position provides information of great value and should be an important part of the placement decision. However, the user is reminded that the results from any test should never make up more than a third of the final decision.” Other factors include background checks, the interview(s), references, education, work experience, and the like.
Why do organizations conduct assessments?
Organizations use assessment tools and procedures to help them perform the following human resource functions:
- Selection. Organizations want to be able to identify and hire the best people for the job in a fair and efficient manner. A properly developed assessment tool may provide a way to select successful salespeople, concerned customer service representatives, and effective workers in many other occupations.
- Placement. Organizations also want to be able to assign people to the appropriate job (or job) level. For example, an organization may have several managerial positions, each having a different level of responsibility. Assessments may provide information that helps organizations achieve the best fit between employees and jobs.
- Training and development. Tests are used to find out whether employees have mastered training materials. They can help identify those applicants and employees who might benefit from either remedial or advanced training. Information gained from testing can be used to design or modify training programs. Test results also help individuals identify areas in which self-development activities would be useful.
- Promotion. Organizations may use tests to identify employees who possess managerial potential or higher level capabilities so that these employees can be promoted to assume greater duties and responsibilities.
- Career exploration and guidance. Tests are sometimes used to help people make educational and vocational choices. Tests may provide information that helps individuals choose occupations in which they are likely to be successful and satisfied.
- Program evaluation. Tests may provide information which the organization can use to determine whether employees are benefiting from training and development.
What does the expression “Use tests in a purposeful manner” mean?
Assessment instruments, like other tools, can be extremely helpful when used properly, but counter-productive when used inappropriately. Often inappropriate use stems from not having a clear understanding of what you want to measure and why you want to measure it. Having a clear understanding of the purpose of your assessment system is important in selecting the appropriate assessment tools to meet that purpose. This brings us to an important principle of assessment – it’s critical to have a clear understanding of what needs to be measured and for what purpose.
Assessment strategies should be developed with a clear understanding of the knowledge, skills, abilities, characteristics, or personal traits you want to measure. It is also essential to have a clear idea of what each assessment tool you are considering using is designed to measure
What is meant by using the “Whole Person Approach” to testing?
Using a single test or procedure will provide you with a limited view of a person’s employment or career-related qualifications. Moreover, you may reach a mistaken conclusion by giving too much weight to a single test result. On the other hand, using a variety of assessment tools enables you to get a more complete picture of the individual. The practice of using a variety of tests and procedures to more fully assess people is referred to as the whole-person approach to personnel assessment. This will help reduce the number of selection errors made and will boost the effectiveness of your decision making. Do not rely too much on any one test to make decisions. Use the whole-person approach to assessment.
What does it mean to provide “reasonable accommodation in the assessment process to people with disabilities”?
Accommodation in the assessment process may involve ensuring physical accessibility to the test site, modifying test equipment or tests, or providing qualified assistance. Giving extra time on certain kinds of tests to test takers with dyslexia or other learning disability, and administering a larger print version of a test to a person who is visually impaired are examples of reasonable accommodation.
What is a Cultural Survey and when should an organization use one?
Cultural surveys typically are designed to measure employees’ total workplace experiences; e.g., levels of satisfaction in their jobs, compensation, management, etc. The information collected provides senior-level leaders with a vital perspective on the current reality in their organization’s workplace and areas of concern affecting the total workplace experience. Cultural surveys should only be undertaken when senior level management is prepared to take action as might be indicated in the surveys.
Definitions
Reliability
Reliability measures the probability that an individual retaking the test would obtain a similar test score. There are three (3) tests of reliability: (i) test-retest reliability, (ii) split-half reliability, and (iii) coefficient alpha reliability. The U.S. Department of Labor in its Guide states, “The reliability of a test is indicated by the reliability coefficient. It is denoted by the letter ‘r’ and is expressed as a number ranging between 0 and 1.00, with r = 0 indicating no reliability, and r = 1.00 indicating perfect reliability. Do not expect to find a test with perfect reliability. Generally, you will see the reliability of a test as a decimal, for example, r = .80 or r = .93. The larger the reliability coefficient, the more repeatable or reliable the test scores.” The guide goes on to state that “reliability coefficient values below .70 “may have limited applicability”. The vast majority of the assessments, especially four-quadrant social style assessments, have reliability coefficient factors below .70. Our PXT SelectTM assessment reliability coefficients average .85 and all attribute reliability coefficient factors are well above the .70 minimum threshold.”
Validity
Validity refers to the specific characteristic the test measures and how well the test measures that characteristic. This simply means test scores have a legitimate relationship with some aspect of job performance; i.e., the test measures what it says it measures. There are four general types of validity: face, content, construct, and criterion:
- “Face” validity simply means the test looks legitimate; the questions or items on the test include words or items a person would expect to encounter on the job.
- Constructs validity refers to the extent to which a psychological measure may be viewed as an accurate measure of a particular construct.
- Content-related validity requires a demonstration the content of the test represents important job-related behaviors; i.e., the content of the test looks like the content of the job. A typing test is content-valid when the job involves typing letters.
- Criterion-related validity correlates success on the test with the successful carrying out of critical job duties; i.e., validation is established when there is a significant positive correlation between comparative success on the test and comparative success with regard to some significant measure of job performance.
For example, the PXT SelectTM Technical Manual describes in understandable language how each of the three validity measurement used (criterion, content, and construct) were determined, including detailed explanations of how the tests were conducted.