A Sample Thesis Questionnaire
 

Chapter 3: Aspects of Personality Questionnaire

            This chapter concentrates on the different aspects of personality questionnaire. Specifically, focus will be on the choice, evaluation and issues related to the use of personality questionnaires.

Choice of Questionnaire

There is a wide array of published trait questionnaires that are potentially relevant to certain human resource needs. Personality questionnaires range from those that aim to assess general qualities, such as measures of the Big Five, to those that measure more narrowly drawn traits that may be critical in certain situations. There are no definite rules for choosing between the different questionnaires. However, it is essential that the user of these personality questionnaires have a clear objective as to what the test should assess or measure. The goal of using personality tests helps in limiting the choices of questionnaire the user can apply and ensure that useful results are obtained.

Evaluation of questionnaires

There are some well-established benchmarks that may be used for evaluating the questionnaires. One of which is reliability, referring to whether repeated measurements will give similar questionnaire scores. However, a questionnaire may be reliable for the wrong reasons. The user must ensure that questionnaire items do not increase the tendency for biases. The validity of the questionnaire is also an important factor for assessing the quality of a personality test. In evaluating personality tests it is important that the questionnaire possesses good reliability, stability and validity. If it has subscales, their differentiation should be supported by factor analysis. Evaluation of internal consistency (i.e., reliability) and stability over time is straightforward. Generally, researchers take a reliability value of 0.7 as the minimum for research use, although 0.8 or more is preferable. Individual assessment requires a reliability of 0.9 or better. Determining factor structure may raise technical issues such as the nature of the factor structure to be used, although, if the factor structure is robust, choice of analytic method should have minor effects only.  

Assessment of validity may be a little more complex, as I will now discuss. As previously discussed, the key element of validity is criterion validity–the ability of the questionnaire to predict meaningful criteria such as emotional states, abnormal behaviours and job performance. Criterion validity has two main aspects. These include the concurrent (present) and predictive (future) validity. Both aspect can be useful; for instance, the clinician may want an index of current behavioural disturbance, while the personnel manager needs to predict future job performance, following training. In any case, the validity coefficient expresses how strongly the trait predicts the criterion; the trait may not be of much practical use if the coefficient is too low. It is also important to establish whether the validity coefficient generalises across different contexts; it is dangerous to assume that a single study establishes validity, even if the coefficient is high.  

Face validity is the least important of the remaining aspects of validity, although lack of face validity may sometimes alienate respondents. Content validity is especially important in the early stages of research, before the development of a detailed nomological network that demonstrates the meaning of the construct from its relationships to other indices and behavioural outcomes. Convergent and divergent validity are usually considered together. For example, an extraversion–introversion scale should correlate moderately high with related constructs such as sociability and assertiveness (convergent validity). If it fails to do so, the scale is probably not measuring extraversion. It should also show small correlations with other constructs that are known to be distinct from extraversion, such as neuroticism and intelligence (divergent validity).  

Establishing divergent validity is especially important in developing scales for new constructs, which, all too often, turn out to be similar to existing ones. Incremental validity is related to divergent validity. It refers to tests of whether the scale predicts criteria if other constructs correlated with both the scale and the criterion are statistically controlled, typically using partial correlation or multiple regression. If I had a new scale for stress vulnerability, incremental validity would be demonstrated if the scale predicted anxiety symptoms with neuroticism and extraversion controlled, for example. Finally, construct validity refers to the often elusive theoretical basis for the trait, and its psychological meaning. The relevance of theory to the practitioner varies according to the nature of the practical problem. Sometimes, prediction proceeds on an actuarial basis. That is, if it is known that a battery of scales predicts performance on some job (with good validity generalisation), the scales can be used for personnel selection without too much concern about theory. However, this approach is often negated by the existence of moderator variables, that is, additional variables that influence the association between the trait scale and the criterion. For example, correlations between traits and job performance depend critically on factors such as the nature of the work, the stressfulness of the work environment, and the level of stimulation or arousal it affords. Although the influence of moderator variables can be mapped out empirically, prediction is enhanced when the user can utilize theory to determine when a trait is or is not likely to be predictive.  

Issues

The user of psychometric tests is, of course, bound by the same ethical principles as any other psychologist. Lanyon and Goodstein (1997) discuss some misuses of tests, which would be contraindicated by the APA code. Naturally, it is unethical to use professionally a test whose validity has not been established. Even if the test has been systematically developed, problems may arise when there is no clear criterion for the construct that is assessed, and when tests are interpreted on the basis of common sense or the tester's personal insights. Such problems are often more acute for projective tests than for trait measures. More subtly, tests that are valid for one purpose may be misused in a different context. Lanyon and Goodstein pointed out that tests developed for use in psychiatric settings such as the Rorschach and MMPI may not be suitable as selection devices in industry, especially when administered by people with no clinical training. Several countries, including the UK, have formal systems for accrediting test users to counter such problems.

Ethical obligations are discharged within a legal framework, which, of course, differs from nation to nation, and, in the USA, from state to state. Laws typically deal with issues such as confidentiality and data protection, protection of privacy, and fairness in occupational selection. Naturally, the practitioner requires familiarity with such laws, especially in an increasingly litigious society. If a trait assessment is a factor in a job applicant not being hired or promoted, the psychologist may have to justify the relevance of the trait in court. Occasionally, legal decisions may seem capricious. In 1996, the police force of New London, Connecticut, obtained some notoriety for refusing employment to an applicant whose mental ability was deemed too high (corresponding to an IQ of about125). The police department successfully argued in court that applicants who score too high could get bored with police work and leave soon after receiving costly training.  

Arthur and colleagues (2001) reviewed some legal implications of organisational personality assessment in the USA. They pointed out that personality measures may be less vulnerable than mental ability tests to the perception that they are unfair to minority applicants. Indeed, they quote a statement made by Hogan et al. (1996, p. 475):  

“… we want to suggest in the strongest possible terms that the use of well-constructed measures of normal personality in pre-employment screening will be a force for equal employment opportunity, social justice, and increased productivity.” 

Arthur and associates (2001) pointed out two unresolved problems in line with the use of personality questionnaire within the occupational field. First, although fakers may be identified as having very high scores on desired traits, rejecting a job applicant because they score too highly might be difficult to justify legally. Second, the well-replicated sex differences in some personality traits lead to conflict between legal and scientific principles. The use of score adjustments or differential cut-offs in the use of employment-related tests, on the basis of race, colour, religion, sex or national origin was made unlawful by the US Civil Rights Act. This is implemented primarily to prevent racial discrimination. However, it is normal and scientifically justified practice in personality assessment to use separate norms for men and women, a procedure that in fact promotes fairness in occupational selection. It remains to be seen how this issue will play out in future court cases.