Often in experiment planning, we want to collect as much data as possible, but collecting data can be expensive, and there may be risk to patients depending on the experiment.

Computing the power for a 2-sample test

When planning a study, we want to know how likely we are to detect an effect we care about. In other words, if there is a real effect, and that effect is large enough that it has practical value, then what is the probability that we detect that effect? This probability is called the power and we can compute it for different sample sizes or for different effect sizes.

We first determine what is a practically significant result. Suppose that company researchers care about finding any effect on blood pressure that is 3 mmHg or larger vs. the standard medication. Here, 3 mmHg is the minimum effect size and we want to know how likely we are to detect this size of an effect in the study.

We need to determine an appropriate sample size to ensure we can be reasonably confident that we'll detect any effects that are practically important in a study.