Ecological validity (perception)

Brunswik's concept of ecological validity is tied to his concept of representative design. In a representative design, the variances and correlations of some dependent variable Y and independent variables X1, X2, and X3 match their values in some specific real-world ecology.

Quoting Hammond, "'Generalizability of results concerning... the variables involved [in the experiment] must remain limited unless the range, but better also the distribution... of each variable, has been made representative of a carefully defined set of conditions' (1956, p. 53). Brunswik's admonition regarding the representativeness of the formal aspects of the conditions of experiments also includes the (ecological) intercorrelation among the independent variables in the experiment, thus challenging the typical factorial design in which variables are set in orthogonal relation to one another."^[1][2]

To understand why the ecological validity of a cue will change if the design is not representative, consider two admissions officers, at schools A and B. School A is a highly selective university and B is a nonselective college. Admissions officers at A and B may learn to predict freshman grade-point average – GPA – (Y) of applicants to their respective colleges on the basis of applicants' high school GPA (X1), ACT test score (X2), and a rating of the quality of the student's essay on a 1 to 5 scale (X3). Because, in multiple regression, the weights of X1, X2, and X3 depend on their correlations and their variances, one would likely find very different regression weight (and therefore ecological validity of X1) of applicants at A versus B.

Brunswik believed that people learn over time to weight cues that will predict the criterion Y in a particular environment where they operate and receive feedback. If, in a particular environment where the judge normally operates, X1 and X2 are highly related, one can learn to predict Y using a subset of the cues to predict the criterion without loss of accuracy. But if the same person is put in a new situation with different ranges of the cues and different correlations among them, performance in predicting the criterion will suffer. This is similar to saying that Admissions officer A might have a hard time using what she had learned from experience at her selective employer if now attempting to predict freshman GPAs of applicants at B's university. Brunswik believed similar problems arise when researchers create experiments where the independent variables are not distributed in a way that matches the participants' local environments—for example, by making independent variables uncorrelated or by holding all but one variable constant.

Brunswik's students have written that the now-common use of ecological validity to describe a type of experimental validity was a corruption of his original terminology.^[1] Social scientists routinely refer to the ecological validity of an experiment as a rough synonym to Aronson and Carlsmith's (1968) concept of the mundane realism^[3] of the experimental procedures—mundane realism refers to the extent to which the experimental situation is similar to situations people are likely to encounter outside of the laboratory. Hammond published a detailed critique of this misuse (1998). Another common misuse of ecological validity is as a synonym for external validity.