The design of an experiment and its statistical analysis are intimately connected. These pages describe various designs with numerical examples of their statistical analysis. There may be some overlap with the pages on Statistical Analysis. Sample size is discussed separately.The designs, in most cases with a numerical example, are considered under the following three headings:
Note that this page has three sub-pages. To access them click on “More details”. To return to this page click on “12. Experimental designs” in the navigation bar.
1. Between-subject, single factor & completely randomised designs
These are the most common designs. They involve a single factor, usually called the “treatment”. This can have any number of levels which can be quantitative (e.g. doses of a test compound) or qualitative (e.g. diets A, B, C). Unequal numbers in each group present no problems.
Experimental subjects are assigned to treatments strictly at random without regard to any characteristics, so these are sometimes called “completely randomised between-subjects designs”. However, surveys such as comparisons of different strains involve the same statistical methods, so are also discussed here.
These designs are only suitable for experiments where the experimental material is homogeneous. If the experimental units are highly variable (e.g. in body weight, age, genotype etc.) or if they are so large that they become unwieldy, the use of a blocked design should be considered otherwise the experiment will need to be large or it will lack power.
These designs are usually analysed by a one-way analysis of variance (possibly following a scale transformation) if the outcome is quantitative or a chi-squared analysis if the outcome is counts/proportions. Non-parametric tests such as the Mann-Whitney or Kruskal Wallace test may be necessary when the assumptions underlying parametric methods are not met (see 11. Statistical analysis). More details
2. Blocked or stratified designs
The designs include randomised block, crossover (within-subject) and Latin square designs as well as some others not discussed here. They are very useful designs, and should be used more widely in biomedical research.Blocking breaks the experiment up into smaller “mini-experiments”:
For convenience. It is often easier to do an experiment which has been split up into smaller bits. Blocking ensures that this is done correctly so as to avoid bias and improve power.
To increase statistical power. It can remove the effects of any observable heterogeneity of the experimental material (e.g. it may be difficult to get a group of animals of sufficiently uniform age or body weight or it might be impossible to do all the measurements at the same time of day).
To take account of any natural structure of the experimental material, such as experiments involving neonatal animals which come in litters.
To remove time effects such as those due to diurnal or seasonal rhythms or just random fluctuations in the environment.
To ensure repeatability. If an experiment is done in small bits over a period of days, weeks or months significant differences among treatments will only be detected if each mini-experiment gives consistent results, subject to sampling error. These designs are common in in-vitro studies where the investigator sometimes uses words like “The experiment was repeated three times”. This implies that they did a randomised block experiment with three blocks, with the blocking factor being time.
These designs are usually analysed by a two-way analysis of variance without interaction (see numerical examples). The statistical analysis is much easier if there are equal numbers in each treatment group within a block. More details
3. Factorial designs
These designs look at the effect of two or more “factors” simultaneously. One factor is usually the “treatment”, others could be sex, genotype, age, previous treatment, diet etc. There can be any number of factors and any number of levels of each factor, but designs with more than four factors are rare in biomedical research, though common in industry.
These designs are extremely useful and some statisticians suggest that virtually all experiments should have this type of design because they provide extra information at little or no extra cost.
They are used to:
Study any interactions among factors. It is often important to know what factors influence the outcome of an experiment.
Increase the amount of information from an experiment without increasing the numbers of animals. They are almost like doing two or more experiments simultaneously with the same animals. Thus they are a useful “Reduction” strategy
Optimise subsequent experiments. They can be used to find the combination of factors which produces most sensitivity in subsequent similar experiments.
Blocking and factorial designs are not mutually exclusive. Both can be used in a single experiment. More details