Recently, I am working on a paper that based on an apprentice project I did before. I cannot remember how, but the concept of "power analysis" came into my mind. It seems that power analysis is very important in statistical analysis, but ignored by many, if not most, educational researchers. "[I]t is extremely surprising that very few researchers conduct and report power analyses for their studies (Brewer, 1972; Cohen, 1962, 1965, 1988, 1992; Keselman et al., 1998; Onwuegbuzie, 2002; Sherron, 1988) even though statistical power has been promoted actively since the 1960s (Cohen, 1962, 1965, 1969) and even though for many types of statistical analyses (e.g., r, z, F, ?2), tables have been provided by Cohen (1988, 1992) to determine the necessary sample size. Even when a priori power has been calculated, it is rarely reported (Wooley & Dawson, 1983)." (Onwuegbuzie & Leech, 2004, p. 207)

The "power analysis" concept was less or even not talked about in my previous research methods and statistics courses. Here are some reasons why power analyses were less used or reported (Onwuegbuzie & Leech, 2004).

  1. Researchers do not sufficiently understand the concept of statistical power.
  2. The concept and applications of power are not taught, or adequately covered, in many undergraduate- and graduate-level statistical courses. And, power is not recognized as important as other concepts.
  3. No sufficient information is provided on how to report statistical power.
  4. Research resource constraints do not allow research have enough sample size as required by the result of a priori power analysis.
  5. It is difficulty to estimate effect sizes and standard deviations before conducting a research because of the uncertainties involved.
  6. SPSS, SAS, and other software package do not have the function of conducting power analyses. Users need to use other software to do that. And, software for power analysis normally do not do other analysis.

Definition

Why it should be reported

A priori or post hoc?

When I first touched the term "power analysis", I thought, aha, I can use the power analysis result to prove how confident (or correct) I was with the result of my statistical analysis, especially if I got a significant result (p < .05). However, this thought is wrong. First, many researchers did not like the idea of performing power analysis after the data has been collected and analyzed, they call it "post hoc power analysis". They propose a priori power analysis, meaning that power analysis should be performed as a part of research plan. Second, even post hoc power analysis is favored by some researchers, most of them suggest reporting post hoc power analysis only when there is a non-significant result.

How to Perform Power Analysis

Resources