10. Outcomes

The nature and type of outcome or character(s) to be measured has an important effect on the design of the experiment, particularly with respect to sample size.

Counts, scores and other attributes of individuals
Measurements data
Multiple outcomes
A C57BL/6 mouse homozygous for the obese gene Lepob
Counts, scores and other attributes of individuals

Attributes such as dead/alive or scores such as 0, +, ++ and +++ which classify the animals (or other experimental units) are a common form of outcome. These have what is called a nominal scale if there is no particular order such as with Male/Female or strain A/B/C, They have an ordinal scale if the categories have some natural order such as 0,+, ++ and +++ where +++ is expected to be larger than 0. If there are just two possible outcomes (e.g. dead/alive) then this is said to be a binary outcome. Such outcomes generally require a much larger sample size than if something can be measured, and the resulting data will need to be analysed using methods suitable for counts and proportions.

Measurements data

If the outcome is something which is measured, or counted (provided the counts are relatively high), then sample size can usually be much smaller than with attributes such as dead/alive. So where possible outcomes should be measurements of some sort. However, some measurement data has a very skewed distribution, with a lot of low numbers and a few very high ones. This is commonly seen with the concentration of a substance, where concentration can not be less than zero. Such data may need to be transformed to a different scale (very often a logarithmic scale) before it is amenable to a statistical analysis using the usual parametric methods (Studentís t-test and the analysis of variance). Where the observations are percentages constrained to be between zero and one hundred, then a scale transformation may again be appropriate if many of the observations are less than 20% or more than 80%.

Multiple outcomes

Many statistical textbooks seem to assume that each experiment only has a single outcome, character or trait which is measured. However, multiple outcomes such as clinical biochemical or haematological measurements are common. In micro-array experiments there may be many thousands of measurements on each individual.  Similarly, where individuals are followed over time data may be collected on a range of physiological variables such as growth, food consumption, and/or blood concentration of test substances. There are various ways of dealing with such data:

    A separate analysis for each outcome.

Each character is analysed separately using,say, a t-test or analysis of variance. The problem with this approach is the need to correct the p-values (see 11. Statistical analysis) for multiple tests.

Using a 5% significance level, the chance of a false positive result if one character is tested is   5%, but if two characters are analysed the chance of a false positive raises to 9.75% and with three characters it rises to 14.3%. This assumes that the outcomes are independent, but characters such as body weight at different ages will be correlated. Although there are methods of correcting for multiple testing (such as the Bonferroni method), they are really not suitable where many correlated outcomes are involved.

It is often informative to look at the magnitude of response for each outcome in terms of standard deviations (i.e. the difference between two treatment groups, divided by the standard deviation (SD) of the character). This converts all characters to the same units and shows which are most likely to be statistically significant as statistical significance depends largely on the response in standard deviation units.

    Reducing multiple outcomes by combining them

Where a single character is measured at several time points it may be appropriate to combine them in some way. The change over time (e.g. by fitting a regression line or looking at the difference between the first few and last few measurements), time to reach a peak, or area under a curve could be estimated for each animal, and this could be subjected to the statistical analysis.

    A multivariate analysis

Multivariate statistical methods combine multiple outcomes taking into account any correlations between them to give a few composite characters. Techniques such as Principle Components Analysis, Discriminant Function Analysis and various clustering methods can be used to analyse patters of response across multiple outcomes. However, these are advanced statistical methods requiring professional advice.