### Some Handy Definitions

If you’re not sure what some of the terms we used mean, the following list describes them.

**Population**

This is the entire group about which we want to know something. It’s important to realise that populations are not restricted to people. For example, populations can also refer to plants, animals, areas of land, hospitals, schools or whatever else is interesting to the experimenter. The population of interest needs to be clearly defined before the experiment begins. This can sometimes be difficult. Examples of populations are:

- All people in Australia with a certain disease;

- All plants of a certain type in the area of land surrounding a proposed mine site;

- All primary school students in Western Australia;

- All woylies in the Upper Warren region of Western Australia's South-West; or

- All babies born in a Sydney Metropolitan Area hospital.

**Sample**

The sample is the subset of units or people in the population who actually take part in the experiment. This is often, but not always, a random selection. There are many different sample designs, ranging from the simple random sample, where each person in the population has the same chance of being selected, through to more structured designs such as stratified, multistage and cluster samples. Each of these latter designs still involves random selection, but they need slightly different handling at both the design and analysis stages.

**Hypothesis**

Experiments are ideally designed to answer a specific question. This question is usually formulated as a hypothesis, which we test using the data collected in the experiment. For example, the question “Do people with a certain disease survive for longer using a new treatment than using the standard treatment?” could be formulated into the null hypothesis of “H_{0}: The new treatment is no different than the standard treatment”, which would be tested against the alternative hypothesis of “H_{1}: The new treatment is different to the standard treatment”. This is an example of a two-sided hypothesis test, since we are looking for a difference, but are not restricting the difference to being either positive or negative.

**Standard Error**

The **standard error** (which is the square root of the variance) is a measure of the precision of something estimated from a sample, for example a mean or a proportion. Since, by definition, any sample doesn’t have information from the whole population, anything we estimate from the sample will be an imperfect estimate – it will have **sampling error**. The term “error” is something of a misnomer here. Rather than implying a mistake, it refers to the difference between a sample and the population. As the sample size increases, the sample becomes more like the population, and so the associated standard error decreases. The standard error is also smaller if measurements are less variable.

**Significance Level**

The **significance level (α)** is the probability or chance that our experiment shows a significant difference when there really isn’t one. It’s the first of two ways we can get our hypothesis test wrong; hence it’s also known as the **Type I error (α)**

**Power**

The **power of an experiment (1 - β)** is the likelihood the experiment can detect a difference when that difference really exists. For example, suppose we expect a new cancer treatment to improve the six-month survival rate for patients when compared to the standard treatment, and we want to test this belief by conducting an experiment with a 90% power. Our experiment would have around a 90% chance of detecting a significant improvement in the six-month survival rate. However, it would also have around a 10% chance of failing to detect a significant improvement, even if that improvement really exists. This failure to detect a difference that is actually there is the second way we can get our hypothesis test wrong; hence it’s known as a **Type II error (β)**. When designing an experiment we want to try and minimise the chance of this type of error, which means maximising the power.

Back to Top

*March 2016*