When planning an experiment it is essential also to know that the results can be analysed. Planing the statistical analysis is an integral part of planning the experiment. So these pages give a brief introduction to the main statistical methods of analysing designed experiments. Details are given in the sub-pages.
What is a “statistical analysis”
An experiment usually results in some means or proportion affected of different groups such as control and treated animals. Means will differ because each animal is different. Proportions affected could differ by chance. Means and proportions may also differ as a result of the treatment. The aim of the statistical analysis is to calculate the probability that differences as great as or greater than those observed could be due to chance. If this probability is high, then chance may be the explanation, if it is low then a treatment effect may be the explanation. These days the actual calculations are almost always done using a computer.
Most measurement data where the aim is to compare means can be analysed using an analysis of variance (ANOVA), a t-test or a non-parametric method. Scores and proportions often use a chi-squared test, while dose-response relationships use regression analysis. Other methods may be needed when there are multiple outcomes. The methods described in the sub-pages give a brief introduction.
Note that this page has five sub-pages accessed by clicking on “more details”. Return from the sub-pages by clicking on 11. Statistical analysis on the main navigation bar.
The statistical analysis
Data checking and summarisation
The first step should always be to check the data once it has been entered into the computer. Any outliers or other doubtful observations should be identified and compared with the original data to ensure that they are not due to transcription errors. Outliers which seem to be valid data points should not be discarded at this stage, but their presence should be noted. More details
The analysis of variance (ANOVA) and the t-test
Given a set of means (say from a number of treatment groups) the ANOVA will calculate the probability that the observed or greater differences among them could have arisen by chance sampling variation. The ANOVA does not indicate which mean differs from which, assuming that there are more than two means. This has to be assessed either by extensions of the ANOVA itself or by using post-hoc comparisons.
The ANOVA and Student’s t-test are often the best way of analysing quantitative (i.e. measurement) data , provided certain assumptions are met. When there are only two treatment groups the ANOVA and the t-test are mathematically identical, but the ANOVA will also accommodate more than two groups and more complex designs involving blocking and additional factors. The t-test is not discussed here as the ANOVA can replace it.
In some cases a scale transformation is necessary in order for the assumptions required for a valid ANOVA are met. This is also covered in the section. More details
Regression
This method is used when the aim is to study a causal relationship between two variables, such as in obtaining a dose-response relationship. More details
Non-parametric methods
Where there is measurement data, but the variation is clearly not the same in each group and the residuals do not have a normal distribution and a transformation of scale does not correct this, then a non-parametric statistical method may be appropriate. More details.
Counts and proportions
Discrete data giving rise to counts and proportions can not be analysed using the above two methods. Such data can usually be analysed using a chi-squared test or a normal approximation. More details