Controlling Psychometric Measures for Method Effects by Means of Factor Analysis

Characteristics of psychological measurement constitute the background of the research topic: psychological attributes of humans are not directly observable. Instead, these attributes are measured indirectly by means of indicators. Various types of measurement models and analysis strategies have been designed to assure access to the latent attributes. However, these strategies do not guarantee valid measurements since they can exert a unique influence on the outcome of measurement. Hence, test scores as an outcome of measurement not only vary due to the psychological attribute of interest but also due to the method of measurement. Such method-dependent variation is referred to as a method effect; if uncontrolled, these effects impair the quality of measurement. Various methods for dealings with such unwanted sources of variation have been developed. Most famous is the multitrait-multimethod methodology, which separates variance related to the construct of interest from the methods used to measure the construct.

Since elaboration of assessment methods has so far not proven especially useful for preventing method effects, the focus is on the development of statistical methods for dealings with such unwanted sources of variation. Most famous so-far is the multitrait-multimethod methodology, which is assumed to separate variation related to the construct of interest from variation due to methods used to measure the construct. A number of other method effects have so far been described in the measurement literature (e.g., ceiling effect, difficulty effect, item-position effect, wording effect, speed effect/speededness, agreement bias). Further advancement in identifying of and controlling for method effects is of crucial importance for improving psychological measurement and, consequently, advancing psychological science. 

A promising way of dealing with method effects is the decomposition of the measurement into components related to the psychological attribute of interest and to the method effects by taking available knowledge on the construct and the method effect into consideration. This knowledge replaces the experimental design on which the multitrait-multimethod methodology is based. The component representing the psychological attribute that is to be measured is separated from the component that is due to the assessment method. The ideal result of such decomposition is a purified score that more precisely reflects the psychological attribute of interest.  Recent studies have shown the usefulness of this statistical procedure when, for example, data were distorted by the item-position effect or speededness of the measurement.

The Research Topic shall provide an outlet opportunity for new research along this line of research in the first place. The research work may be focused on either the detection of method effects or the control of such effect. Furthermore, reports of applications to real data are encouraged that support the transfer of the methodological developments to substantive research. Original research articles are welcome that 

·       describe so far unknown method effects;

·       present new methodological developments for detecting method effect;

·       present new developments in the methods for separating variation due to method effects and genuine variation;

·       evaluate and/or compare methods for dealing with method effects;

demonstrate known method effects in new types of real data.