Discussion This study attempted to

Discussion


This study attempted to examine the internal consistency reliability and validity of the 40-item Watson-Glaser among job incumbents. The corrected internal consistency reliability of .85 shown in this study indicates that the Watson-Glaser total score possesses good internal consistency reliability for this sample of job incumbents.

Previous researchers have reported higher internal consistency reliability coefficients of the Watson-Glaser. For example, Gadzella, Baloglu, & Stephens (2002) reported internal consistency reliability coefficients of .91 for 30 men, .83 for 105 women, and .91 for 135 students majoring in Education. Since the reliability coefficient is a correlation coefficient, the relatively high and narrow range of ability among the participants in this study might have attenuated the internal consistency reliability obtained in this study, resulting in the uncorrected reliability coefficient of .76. Samples with restricted variances can lead to reliability coefficients being spuriously low (Nunnally & Bernstein, 1994).

The results indicate that critical thinking ability as measured by the 40-item Watson-Glaser is significantly related to job performance. All the criterion-related validity coefficients reported in Table 1 suggest that the 40-item Watson-Glaser could be regarded as “very beneficial” (U.S. Department of Labor, 1999, p. 3-10) for use in relating critical thinking ability to the following aspects of performance: Analysis and Problem Solving, Judgment and Decision Making, Professional/Technical Knowledge and Expertise, and Overall Performance. The results in Table 1 also indicate that the Watson-Glaser test is “likely to be useful” (U.S. Department of Labor, 1999, p. 3-10) in relating critical

thinking ability to Overall Potential. There is also convergent validity evidence from previous studies relating the Watson-Glaser to other cognitive ability tests. Such evidence can be found in Watson and Glaser (1994) showing significant relationships between scores of mid-level management applicants on the Watson-Glaser and their scores on the following tests: Wesman Personnel Classification Test, Verbal (.66), Employee Aptitude Survey – Verbal Reasoning (.51), Employee Aptitude Survey – Verbal Comprehension (.50), Employee Aptitude Survey – Numerical Reasoning (.41), Employee Aptitude Survey – Space Visualization (.26). For a sample of executive management applicants, the correlation between their scores on the Watson-Glaser and their scores on Differential Aptitude Tests – Abstract Reasoning – was .40 (Watson & Glaser, 1994).

Many organizations typically use selection tests in their hiring process. The results obtained in this study suggest that in jobs such as those of Analysts where critical thinking ability is important for successful performance, the Watson-Glaser is likely to be beneficial as part of the external or internal selection process for the job. However, since successful performance typically depends on several factors and aspects related to the job, no single test is sufficient to cover all the aspects of performance. As such, combining information from the Watson-Glaser with other sources of information (for example, interviews, work samples, and records of past performance) will likely enhance the validity of the selection process.

Of course, it is important to note that the practical value of a selection test depends not only on its validity but also on such factors as the base rate

for success on the job (that is, the proportion of people who would be successful in the absence of any selection tool), the selection ratio (that is, the ratio of applicants to the number of vacancies to be filled), adverse impact associated with the test, the cost of hiring error, and the cost of the test itself (Anastasi & Urbina, 1997; Cascio, 1997; Schmidt & Hunter, 1998; U.S. Department of Labor, 1999).


Limitations and Suggestions for Future Research

One of the difficulties of conducting real-life validation studies is getting enough of the needed participants to provide the required data. Consequently, this study was challenged by the constraints of getting otherwise busy employees to take the Watson-Glaser for research purposes and for the supervisors of these employees to also independently provide performance ratings on their subordinates.
Some researchers might be interested in how examinee scores on the Watson-Glaser relate to national norms. The focus and constraints of this study, however, necessitated the use of a sample that was more occupation-specific than national in scope. When using cognitive ability tests for purposes of talent assessment, comparing scores of candidates against a norm population of relevant occupation groups is usually more applicable than using the general census-type “national norms” for such an occupation-specific purpose.

Given the relevance of critical thinking in the employment context, as well as the popularity of the Watson-Glaser as a measure of critical thinking ability, the organizational literature would benefit from more published studies that relate critical thinking ability to performance. For example, in addition to more concurrent validation studies, it also would be beneficial for future researches to try and publish predictive validation studies relating scores on critical thinking tests to subsequent performance.



Pearson

The Talent Assessment business of Pearson develops and delivers assessments for employee selection and development that increase workforce performance. With more than 80 years of experience in the employee assessment field, Pearson offers award-winning products such as the Watson-Glaser Critical Thinking Appraisal, Bennett Mechanical Comprehension Test, and the Golden Personality Type Profiler, and has a growing global presence with offices in seven countries. Its clients include half of the Fortune 500, expanding small businesses, and all organizations that make identifying talent and unlocking employee potential a top priority. It is a part of Pearson (NYSE: PSO), the international media company. Pearson’s other primary businesses include the Financial Times Group and the Penguin Group.

Learn more about Pearson’s Talent Assessment products at www.AssessTalent.com.