Sample size determination

Determining the appropriate sample size is of great importance. Too small and important effects may be missed. Too large and the experiment will waste time, scientific resource and animals. Unfortunately, there is no perfect method, but the Power Analysis and the Resource Equation methods which are, in many ways, complementary are better than subjective judgement. Three methods for determining sample size are discussed here.

Previous experience or custom Previous experience can be helpful because power analysis and the resource equation methods only provide an indication of appropriate sample size. However, the custom in some disciplines, of always using group sizes of six or eight animals regardless of the type of experiment or number of groups is not acceptable, particularly when using factorial experimental designs or designs with more than 2-3 treatment groups. In these cases the number of animals needed will be seriously over-estimated. More formal methods such as the two described here are more appropriate.

Power analysis Power analysis is based on a mathematical relationship between a number of variables, as discussed on the next page. It is the method of choice for clinical trials which tend to be relatively simple and extremely expensive and is  recommended for any experiment where it can be reasonably easily applied, such as simple confirmatory experiments. It can be used for binary outcomes where the aim is to compare two proportions but can not be used for pilot or exploratory experiments and it is difficult to use for complex biological experiments having several treatment groups. If several different outcomes are being measured it is necessary to decide which of these is the most important and design the experiment to address this outcome. The weakness of the method is that it depends on a good estimate of the standard deviation, and the estimated numbers depend critically on this.

The “Resource equation” method The resource equation method depends on the law of diminishing returns. Adding one more experimental unit to a small experiment may give useful additional information, but if the experiment is already quite large it may not do so. It provides some simple rules on how large is large enough and how large is too large. It is very easy to use particularly for complex biological experiments with quantitative outcomes (not counts or binary outcomes such as dead/alive). It can be used for exploratory experiments, such as those involving microarrays, haematology and/or clinical biochemistry where many parameters are being measured and it is difficult to decide which one is the most important. It does not take account of the variability of the material, the probability of reaching erroneous conclusions or the magnitude of response likely to be of scientific interest so it is a blunt tool when compared with the power analysis.