Statistical testing based on samples is always open to a certain degree of error. For instance, when testing if a new drug "A" is better than an old drug "B", there is the possibility that the wrong conclusion will be reached. Two types of error are possible. A good analogy is when an accused person is in court. If an innocent person is found to be not guilty, and the guilty person is found guilty, then there has been no error. The two types of mistake that can be made are when an innocent person is found guilty, and the other type of error is when a guilty person is found innocent. In statistical testing, the same two types of error can occur.
A free sample size calculator is being developed, as logged at blogger.com
Confidence Level and Alpha ('α') in Sample Size Calculations
In sample size estimation, the confidence level is the level of certainty that no difference will be found when there actually is no difference. In the case of new drug "A", it means the level of confidence that drug "A" will not appear to be better than drug "B", if drug "A" is indeed no better than drug "B". In the legal analogy, it is the chance that an innocent person will be found not guilty.
Alpha, usually written as the Greek letter α, is the level of uncertainty. If the level of uncertainty is α then the level of certainty or confidence is 1 - α. In the legal analogy, α is the equivalent of finding an innocent person to be guilty. Typical values for α depend on the industry and application of the sample sizes, but 5% or 0.05 is commonly used for α, and 95% or 0.95 is used for the confidence value.
Power and Beta ('β') Level in Sample Size Calculations
The statistical "power", in the legal analogy, is the level of certainty that a guilty person will be found to be guilty. It is analogous to "Confidence", in that it refers to the probability of making a correct decision about a population, based on a sample. In the analogy of drug "A" and drug "B", the power represents the level of certainty that if drug "A" is better than drug "B", this will be borne out by the statistical analysis.
The beta ('β') level represents the risk of not finding a difference where one exists. In the legal analogy, it is the risk of finding a guilty person to be innocent. Beta is often taken as twice the value of alpha, and is often set at 10% or 0.1
Significant Difference (δ or Δ) in Sample Size Calculations
The difference referred to here is not a statistical difference, but a practical or critical difference. In the case of drugs "A" and "B", the difference in the effect of the two drugs may be statistically significant, but numerically insignificant. For instance, if the sample sizes are very large, drug A may extend patient life by, on average, one hour. This would not be deemed to be medically significant, even if it were statistically significant.
Sample size calculation uses the ratio of the practical difference to the standard deviation. It is therefore not essential to know the practical difference, but the ratio of the practical difference to the standard deviation:
Δ = δ / σ
Summary of Confidence, Power, Alpha, Beta and Effective Difference in Sample Size Calculations
The definitions of the variables required to do a sample size calculation have been defined and described. Other articles by the author describe the sample selection method, and the equations needed to calculate sample sizes for mean, standard deviation, proportion and rates.
A free sample size calculator is being developed, as logged at blogger.com
Sample Size Method References
Julious, S.A., (2009), Sample Sizes for Clinical Trials. Boca Raton: CRC Press
Martin Bell - Martin holds a B.Sc. degree in chemical engineering, and an M.Sc. degree in electronics and computing. He has spent more than 25 years ...
Join the Conversation