Power is the probability of correctly rejecting a false null hypothesis. Power is therefore defined as: 1 - β where β is the Type II error probability. If the power of an experiment is low, then there is a good chance that the experiment will be inconclusive. That is why it is so important to consider power in the design of experiments. There are methods for estimating the power of an experiment before the experiment is conducted. If the power is too low, then the experiment can be redesigned by changing one of the factors that determine power.

Sometimes more than one test statistic is used to conduct a hypothesis test. In this case the relative power of the test needs to be computed for the competing statistics; the test statistic that is most powerful must be selected.

If you want to know more about "power of a test," read the following (not required for Level I candidates):

Consider a hypothetical experiment designed to test whether rats brought up in an enriched environment can learn mazes faster than rats brought up in the typical laboratory environment (the control condition). Two groups of 12 rats are tested. Although the experimenter does not know it, the population mean number of trials it takes to learn the maze is 20 for rats from the enriched environment and 32 for rats from the control condition. The null hypothesis that the enriched environment makes no difference is therefore false.

The question is, What is the probability that the experimenter is going to be able to demonstrate that the null hypothesis is false by rejecting it at the 0.05 level? This is the same as asking, What is the power of the test? Before the power of the test can be determined, the standard deviation (s) must be known. If s = 10, then the power of the significance test is 0.80. This means that there is a 0.80 probability that the experimenter will be able to reject the null hypothesis. Since power = 0.80, b = 1 - 0.80 = 0.20.

It is important to keep in mind that power is not about whether or not the null hypothesis is true (it is assumed to be false). It is the probability the data gathered in an experiment will be sufficient to reject the null hypothesis. The experimenter does not know that the null hypothesis is false. The experimenter asks the question: If the null hypothesis is false with specified population means and standard deviation, what is the probability that the data from the experiment will be sufficient to reject the null hypothesis?

If the experimenter discovers that the probability of rejecting the null hypothesis is low (power is low), even if the null hypothesis is false to the degree expected (or hoped for), then it is likely that the experiment should be redesigned. Otherwise, considerable time and expense will go into a project that has little chance of being conclusive even if the theoretical ideas behind it are correct.