The Design of Experiments

At the time of publication, Fisher was a statistician at Rothamsted Research (formally known as Rothamsted Experimental Station) where he developed statistical methods to analyze agricultural data.^[2] His broader career included fundamental contributions to statistics, genetics, and evolutionary biology. The Design of Experiments built on Fisher’s earlier work, including his book Statistical Methods for Research Workers (1925), and laid the groundwork for the practical application of statistical tests in experimental science.^[1]

Much of Fisher’s approach grew out of earlier studies at Rothamsted, particularly a 1923 paper with Winifred Mackenzie on crop variation.^[3] In this work, Fisher tied the validity of experimental tests directly to the principle of randomization, arguing that randomly allocating treatments to plots could simulate the independence required for valid inference. This innovation established randomization as the “physical basis of the validity of the test” and became Fisher’s first principle of experimental design.^[3]

Fisher went on to integrate randomization with two further principles, replication and blocking, to form a general framework for designing experiments. These ideas, first applied to agricultural trials, were presented systematically in The Design of Experiments, helping to formalize experimental design as a statistical discipline.^[3]

One of the most notable sections of the book describes the origin of the null hypothesis. Fisher illustrated it with a wager, a woman claimed to be able to tell whether milk or tea had been poured first into a cup.^[1] In order to test this, Fisher designed an experiment in which she was presented with eight cups, four of each preparation.^[4] This came to be known as the Lady tasting tea experiment.^[5] He defined the null hypothesis as the assumption that she has no such ability to make the distinction, meaning her answers would be due to chance. Under this assumption, the probability of guessing all eight cups correctly was calculated to be 1/70. Fisher argued that if she succeeded to distinguish whether milk or tea had been poured first into a cup, the null hypothesis could be rejected. Fisher stated “the null hypothesis is never proved or established, but is possibly disproved, in the course of experimentation. Every experiment may be said to exist only in order to give the facts a chance of disproving the null hypothesis,”^[6] thus the concept of a null hypothesis was born. This discussion became central to the philosophy of statistical inference.^[6]

Beyond establishing the principles of randomization, replication and blocking, The Design of Experiments advanced the statistical understanding of error estimation in field trials.^[7] Before Fisher’s work, agricultural experiments often lacked a coherent theory for estimating experimental error, except in the simplest comparisons. Fisher formalized this process, allowing errors to be estimated directly from the data rather than assumed. His methods revealed the unreliability of earlier experiments that failed to account for natural variability, such as differences in soil fertility.^[7] Later editions of his related work, Statistical Methods for Research Workers, introduced further refinements, including the analysis of covariance, a technique developed to improve precision by adjusting for variation.^[7] Together, these innovations helped organize earlier agricultural practices into a coherent framework for experimental design and analysis.

Fisher has faced multiple criticisms and controversies through his decades-long career. His work had established a foundation for significance testing that stressed the indication of evidence against the null hypothesis, using p-values.^[8] Ronald Fisher argued that inductive inference was dire to the progression of scientific knowledge and that this is “experimentally demonstrable when we know how to conduct an experiment which will rarely fail to give us a statistically significant result.”^[9] Skeptics, such as Paul L. Morgan and Jacob Cohen, expressed that there were considerable irregularities in Fisher's disagreements with probability.^[8][10] Cohen had a specific critique regarding Fisher, “who emphatically rejected Bayesian theory of inverse probability but slipped into invalid Bayesian interpretations of NHST [null hypothesis significance testing].”^[11] Despite Fisher’s statistical legacy, the discourse highlights how his study has contributed to the extensive misinterpretation of NHST in modern research.^[11]

In addition to critiques of his work, Fisher has been known to be controversial due to his views on race and history of supporting eugenic policies.^[11]