Validity in prescription-driven research is approached in different ways than descriptive research. The first difference deals with what some researchers call 'messy situations' (Brown 1992; Collins, Joseph, and Bielaczuc 2004). A messy situation is a real-life, a highly multivariate one is where independent variables cannot be minimized nor completely accounted for. In explanatory science, experiments are in controlled laboratories, where variables can be minimalized. The complex nature of a real-life intervention means that the success or failure (effect) of the intervention may be difficult to conclusively link to the intervention itself (cause). This aspect of knowledge claims from science is seen as extremely problematic for positivist scientists looking for explanations. However, scientists using a pragmatic paradigm respond to this concept in two ways; first by questioning the value of research carried out in a controlled situation (Brown 1992; Hodkinson 2004; Kelly and Lesh 2000; Perrin 2000; Susman and Evered 1978; Walker and Evers 1999; Zaritsky et al. 2003) and secondly, by looking at causal effects through a different perspective.
The use of the phrase of Pragmatic Validity was first discussed in Worren, Moore & Elliott (2002), who contrasted it with Scientific Validity. This ideas has been taken up in the management literature to a considerable degree.
Many social science researchers assert that testing interventions in controlled laboratory settings is hardly feasible and not a reflection of the real world.[1] For them, real-life settings are needed in order to produce worthwhile research artifacts. These artifacts are validated by the adoption rate of the practitioners within the community of practice associated with the field.[2] Nowotny (2000) calls knowledge that has been validated by the multidisciplinary community of practice 'socially robust', meaning that it has been developed in (and for) contexts outside the laboratory and can be used by practitioners.
In the following statement, Cook (1983) refers to the well-known educational researcher Cronbach about multivariate causal interdependency and validity, and the need for understanding the complexity of the situation being researched.
Lawful statements of causation require full knowledge of this system of variables so that total prediction of the outcome can be achieved. From his belief in the systemic organization of causal connections and the utility of causal explanations of this type, Cronbach questions whether the experimentalists' isolation and manipulation of a small set of specific causal agents is sensitive to the real nature of causal agency, which depends on complex patterns of influence between multiple events and also involves characteristics of respondents, settings and times (p.78).
Thus, Cook (1983) actually questions the validity of causal explanations generated in a context-free setting (the goal of positivistic, explanatory research). Causal relationships in pragmatic research are looked at somewhat differently, which is apparent in the wording alone.
A statement about a causal relationship in positivistic research is something like the following; if you perform action x to subject y, then z happens. This assumes that the confounding variables have been ruled out, and the statement is always true, regardless of the situation (internally and externally valid). What I want to do now is use the concept of 'technological rules' in order to illustrate how causality is shown in prescriptive.
In pragmatic science, the goal is to develop knowledge that can be used to improve a situation. This we can call prescriptive knowledge. Prescriptive knowledge, according to van Aken (2004, 2004b, 2005) can take the form of a technological rule. A technological rule is "... a chunk of general knowledge linking an intervention or artifact with an expected outcome or performance in a certain field of application" (van Aken, 2005: p23). This rule can be formulated much the same way as my earlier example of a causal statement; 'if you perform action X to subject Y, then Z happens' (Note the cause and effect formulation). This type of algorithmic formulation is called a design solution (vanAken and Romme 2005). A design solution is usually a statistically proven quantitative model that can be taken as specific instruction (van Aken & Romme, 2005). On the other hand, there are more abstract technological rules that are used for designing solutions. These are heuristics that guide, but do not determine, the design process and are called design solutions (van Aken 2005; van Aken and Romme 2005). Design solutions are formulated in the following way; " If you want to achieve Y in situation Z, then you perform something like X" (van Aken & Romme, 2005; p. 6). In short, the resulting artifacts of pragmatic research can also be causal relationships, just typically not as specific or reductionist as those resulting from positivist research. The words 'something like' in the statement implicitly refer to the complexity in which the causal relationship is enacted. The causal agent (X, in the statement above) can also be seen as complex and multivariate (Cook, 1983). Testing these causal agents is done in context, much the same way as evaluation research tests social or economic programs (van Aken 2003) .