RELIABILITY AND VALIDITY OF THE Momentum CPI QUESTIONNAIRE (MCPIQ)

A study of the reliability and validity of the Momentum CPI Questionnaire has been conducted at the London School of Economics, with the following extremely positive results:

Reliability:
The MCPIQ has a reliability of 0.992. The average error is 0.008, or 0.8%

Validity:
The MCPIQ has a validity of 0.933. The average error is 0.067, or 6.7%.

Survey instrument
The Momentum CPI Questionnaire consists of 144 pairs of behavioural statements (phrased as "I do X" or "I do Y") and requires the respondent to allocate five (5) points between the two alternatives. Hence, there are six available options: 5-0, 4-1, 3-2, 2-3, 1-4, and 0-5. Choices may be between two favourable or unfavourable alternatives, however the respondent is always required to express preference for one statement over the other. There is no opportunity to allocate equal weighting to both alternatives.

Reliability test
Reliability means 'replicability' and may be measured by whether, and to what extent, responses provided on a test, re-test basis are identical. To assess the reliability of the Momentum CPI Questionnaire (MCPIQ), 23 managers were asked to complete the questionnaire twice, first on day one and a second time on day six. In order not to influence their responses, they were provided with no background information about the measures or the underlying model. The second questionnaire contained the same statement pairings as the first one but in a different order. This attempted to control for prior response recall, and for direct comparison with prior responses. Participants in the study were managers working for existing clients of Behavioural Science Systems.

Validity test
In the current context, validity refers to the degree to which the MCPIQ measures what it purports to measure. The MCPIQ purports to identify and measure the behaviours managers exhibit in their current jobs. The question therefore is whether the results of the questionnaire - i.e. the reported behaviour - can be verified by other means. This was done by comparing managers' self-assessments of behaviour derived from the MCPIQ with objective assessment of their behaviour by observers.

To assess the validity of the MCPIQ, 12 pairs of managers were asked to complete the questionnaire. Manager A (the subject) responded to the MCPIQ on the basis of how he/she saw him/herself behaving in their current job. Manager B (the observer) completed the MCPIQ as he/she saw Manager A behaving in A's job. The paired respondents completed the MCPIQ without discussion or consultation with each other. (Note: Observers were selected on the basis of their working proximity with Subjects - i.e. the degree to which they were in a position to actually observe the Subjects' behaviour at work.)

Method of analysis
The data were collated to match up the paired responses to identical questions. For the reliability test the data pairs comprised managers' initial and staggered responses. For the validity test the data pairs consisted of a Subject's and an Observer's responses.

Analysis tested the degree of difference in paired responses. Difference was measured in terms of units - i.e. the difference between a 3-2 response in the first test and a 2-3 response in the second test is one unit, whereas the difference between a response of 3-2 in the first test and 1-4 in the second test is two units difference . The smaller the difference, the greater the reliability or validity.

The formula used to calculate this difference is a modified form of the average deviation equation:

REFERENCES
Block, J. A. (1957) 'A comparison between ipsative and normative ratings of personality', Journal of Abnormal and Social Psychology, vol. 54, pp. 50-54

Hicks, L. E. (1970) 'Some properties of ipsative, normative and forced-choice normative measures', Psychological Bulletin, vol. 74, no. 3, pp. 167-184

__________________________________________________________________________

1.The issue of whether data of this kind can be expressed as cardinal numbers has already been addressed by Block (1957: 52), who stated that 'there is an almost complete functional identity between the results obtained from ipsative ratings treated normatively and conventionally acquired normative rating data. The one can be substituted for the other, with insignificant loss of discriminations.' In using the term 'ipsative', Block was referring to 'partially ipsative' measures (see Hicks, 1970: 170). The data obtained through MCPIQ is partially ipsative; therefore, analysis may unproblematically take place along the lines stipulated in the main text.