How to Read Effect Size in Research Papers
By Dr. Zubair Khalid, DVM, MS, PhD ·
Quick Answer
- Effect size measures the magnitude of a difference or association, giving you practical meaning beyond a p-value that only tells you if a result could be chance.
- Choose your metric based on your research design: Cohen's d for group mean differences, correlation coefficients for relationships, and odds ratios for binary outcomes.
- No single cutoff defines a "good" effect size, so interpret your value within your field's conventions and the specific context of your study.
What Effect Size Means in Practice
When you read a research paper, the statistical significance test answers one narrow question: could this result have happened by chance? A p-value below 0.05 tells you the finding is unlikely to be random, but it says nothing about whether the finding matters. A study with a very large sample can produce a statistically significant result for a trivial difference that has no practical value.
Effect size answers a different question: how large is the effect? It quantifies the magnitude of the difference between groups or the strength of a relationship between variables. This distinction matters for anyone reading scientific literature, whether you are a student evaluating evidence for a paper, a researcher designing a study, or a professional deciding whether a published finding should change your practice.
Consider a hypothetical example. A feed trial with 10,000 animals per group might show a statistically significant weight gain difference of 50 grams. The p-value confirms the difference is real, but the effect size tells you whether 50 grams is meaningful for your operation. In a different trial with 20 animals per group, a 2 kilogram difference might fail to reach statistical significance, yet the effect size could be large and practically important. The study was simply underpowered to detect it.
Statistical significance depends heavily on sample size. Effect size is designed to be less dependent on sample size, giving you a standardized measure you can compare across studies. This is why reporting effect sizes has become a standard expectation in many fields and why journals increasingly require them alongside p-values.
Core Principles of Effect Size
Standardized vs. Unstandardized Measures
Effect sizes come in two broad categories. Unstandardized measures retain the original units of your data. A mean difference of 2.5 kilograms in body weight is an unstandardized effect size. These are often the most interpretable for practical decisions because they speak directly in the units you work with.
Standardized measures remove the original units, expressing the effect in terms of variability. Cohen's d, for example, expresses the mean difference in units of standard deviation. This allows comparison across studies that used different measurement scales. The tradeoff is that standardized values can be harder to translate back into practical terms.
The Family of Effect Size Metrics
Different research designs call for different effect size metrics. Understanding which metric fits which design helps you read papers critically and choose appropriate measures for your own work.
Cohen's d is used when comparing two group means. It divides the difference between group means by the pooled standard deviation. A d of 0.2 is generally considered small, 0.5 medium, and 0.8 large, though these benchmarks are conventions, not universal laws.
Correlation coefficients, such as Pearson's r, measure the strength and direction of a linear relationship between two continuous variables. Values range from -1 to +1, with zero indicating no linear relationship. Squaring the correlation gives the coefficient of determination, which tells you the proportion of variance shared between the two variables.
Odds ratios are used when outcomes are binary, such as survival versus death or disease versus no disease. An odds ratio of 1 means no association. Values above 1 indicate increased odds of the outcome, and values below 1 indicate decreased odds. Odds ratios are common in epidemiological and clinical research.
Why Effect Size Matters for Study Design
Effect size is also for interpreting completed studies. It plays a central role in planning research. Power analysis, which determines the sample size needed to detect an effect, requires you to specify an expected effect size. A realistic effect size estimate, often drawn from prior literature or pilot data, helps you avoid two common errors: designing a study too small to detect a meaningful effect, or wasting resources on a study larger than necessary.
Funding agencies and institutional review processes increasingly expect researchers to justify their sample sizes with explicit effect size assumptions. The National Institutes of Health provides guidance on grant applications and review criteria, and reviewers commonly evaluate whether the proposed sample size is adequate to detect the effect of interest. A well-reasoned effect size justification strengthens a funding application.
At a Glance
| Metric | When to Use | Interpretation | Practical Note |
|---|---|---|---|
| Cohen's d | Comparing two group means | 0.2 small, 0.5 medium, 0.8 large | Standardized, comparable across studies |
| Pearson's r | Relationship between two continuous variables | -1 to +1, zero means no linear relationship | Square it to get shared variance |
| Odds ratio | Binary outcomes in cohort or case-control studies | 1 means no association | Common in epidemiology, not intuitive for non-statisticians |
How to Read Effect Size in a Research Paper
Step 1: Identify the Research Design
Before you interpret any effect size, determine what kind of study you are reading. A randomized controlled trial comparing two treatments will likely report Cohen's d or a similar mean difference measure. An observational study examining the relationship between two continuous variables will likely report a correlation coefficient. A study with a binary outcome, such as mortality or disease incidence, will likely report an odds ratio or relative risk.
The design determines which effect size metric is appropriate. If a paper reports an effect size that does not match its design, that is a red flag worth investigating.
Step 2: Locate the Effect Size in the Results
Many papers report effect sizes in the text of the results section, in tables, or in figures. Look for terms like "effect size," "Cohen's d," "eta squared," "odds ratio," or "correlation coefficient." Some papers report confidence intervals around the effect size, which give you a range of plausible values. A wide confidence interval indicates imprecise estimation, while a narrow interval indicates greater precision.
Step 3: Interpret the Magnitude in Context
Once you have the effect size, interpret it within the context of the field and the specific research question. A small effect size might still be practically important if the outcome is serious, such as mortality. A large effect size might be less important if the outcome is trivial or if the study has methodological flaws.
Consider the baseline risk or prevalence. An odds ratio of 1.5 might represent a meaningful increase in risk for a common outcome but a negligible increase for a rare outcome. Always interpret effect sizes in light of the absolute numbers, beyond the relative measure.
Step 4: Check for Confidence Intervals
A point estimate of an effect size is rarely the whole story. Confidence intervals tell you how precise the estimate is. A study with a large sample and a narrow confidence interval gives you more confidence in the effect size than a study with a small sample and a wide interval. If the confidence interval includes zero for a difference measure, or includes one for an odds ratio, the effect is not statistically significant at the conventional level.
Common Effect Size Metrics Explained
Cohen's d and Standardized Mean Differences
Cohen's d is one of the most widely used effect size measures. It is calculated by taking the difference between two group means and dividing by the pooled standard deviation. The result expresses the difference in standard deviation units.
A Cohen's d of 0.5 means the two group means differ by half a standard deviation. If you are comparing two feeding regimens and find a d of 0.5, the average animal in the treatment group is half a standard deviation above the average animal in the control group.
The conventional benchmarks of 0.2, 0.5, and 0.8 for small, medium, and large effects come from the work of Jacob Cohen. These are rough guidelines, not absolute thresholds. In some fields, a d of 0.3 might be considered large, while in others, a d of 1.0 might be considered modest.
Correlation Coefficients
Correlation coefficients measure the strength and direction of a linear relationship between two variables. Pearson's r is the most common, appropriate when both variables are continuous and approximately normally distributed. Spearman's rho is a nonparametric alternative used when the data do not meet the assumptions of Pearson's correlation.
The absolute value of the correlation coefficient indicates strength. A correlation of 0.8 is stronger than a correlation of 0.3. The sign indicates direction: positive correlations mean both variables increase together, while negative correlations mean one increases as the other decreases.
Correlation does not imply causation. Two variables can be strongly correlated without one causing the other. A third variable might drive both, or the relationship might be coincidental. When reading papers, pay attention to whether the authors acknowledge this limitation.
Odds Ratios and Relative Risk
Odds ratios are common in studies with binary outcomes. They compare the odds of an outcome in one group to the odds in another group. An odds ratio of 2 means the outcome is twice as likely in the first group, though the exact interpretation depends on the baseline risk.
Relative risk, also called the risk ratio, is a related measure that compares the probability of an outcome between two groups. Relative risk is often more intuitive than the odds ratio, but it can only be calculated directly in prospective studies where you can measure incidence.
Odds ratios are frequently reported in case-control studies, where relative risk cannot be calculated directly. They are also common in logistic regression models. When reading an odds ratio, remember that it overestimates the relative risk when the outcome is common, so the two measures can diverge substantially.
Practical Workflow for Applying Effect Size
Step 1: Define Your Research Question
Before you can choose an effect size metric, you need a clear research question. Are you comparing two groups? Are you examining a relationship between two continuous variables? Are you studying a binary outcome? The nature of your question determines the appropriate metric.
Step 2: Select the Appropriate Metric
Match the metric to your design. For a two-group comparison with a continuous outcome, use Cohen's d or a similar standardized mean difference. For a relationship between two continuous variables, use a correlation coefficient. For a binary outcome, use an odds ratio or relative risk.
Step 3: Calculate or Extract the Effect Size
If you are analyzing your own data, statistical software can calculate effect sizes directly. Many common statistical packages include effect size calculations as standard output or through add-on functions. If you are reading a paper, extract the effect size from the results section or calculate it from the reported means and standard deviations.
Step 4: Interpret with Confidence Intervals
Always interpret the effect size alongside its confidence interval. The point estimate gives you the best guess, but the confidence interval tells you the range of plausible values. A wide interval means the estimate is imprecise, and you should be cautious about drawing firm conclusions.
Step 5: Compare with Field Benchmarks
Compare your effect size with those reported in similar studies. This helps you determine whether your finding is consistent with the existing literature or whether it stands out as unusually large or small. Discrepancies might indicate methodological differences, different populations, or potential problems with one of the studies.
Records and Measurements
What to Record When Reading Papers
When you read a research paper and want to track effect sizes for later comparison, record the following information:
- The effect size metric used
- The point estimate
- The confidence interval
- The sample size
- The study design
- The population studied
- The outcome measured
Keeping this information in a structured format, such as a spreadsheet, allows you to compare effect sizes across studies and identify patterns.
What to Record When Conducting Research
If you are conducting your own research, document your effect size assumptions before data collection. Record the expected effect size, the basis for that expectation, and the sample size calculation. After analysis, record the observed effect size and its confidence interval. This documentation supports transparent reporting and helps future researchers build on your work.
Using Effect Size in Grant Applications
When preparing a grant application, reviewers will evaluate whether your proposed sample size is justified. The National Institutes of Health provides guidance on grant application and review processes, and a clear effect size justification demonstrates methodological rigor. State the expected effect size, cite the evidence supporting that expectation, and show how it translates into your proposed sample size.
Common Failure Patterns
Reporting Only Statistical Significance
The most common failure in research reporting is focusing exclusively on p-values while omitting effect sizes. A paper that reports "p < 0.05" without any measure of magnitude leaves the reader unable to judge practical importance. This pattern is increasingly criticized, and many journals now require effect sizes as part of standard reporting.
Misinterpreting Statistical Significance as Practical Importance
A statistically significant result is not necessarily practically important. With a large enough sample, even trivial differences become statistically significant. Reading papers critically means asking beyond whether a result is significant, but whether the effect size is large enough to matter.
Ignoring Confidence Intervals
Reporting an effect size without a confidence interval is incomplete reporting. The confidence interval conveys the precision of the estimate and allows readers to judge the range of plausible values. Without it, you cannot assess how much confidence to place in the point estimate.
Using the Wrong Metric for the Design
Applying a metric that does not match the research design leads to misinterpretation. Using a correlation coefficient for a binary outcome, or an odds ratio for a continuous outcome, produces meaningless numbers. Check that the reported metric matches the design before interpreting it.
Overinterpreting Small Effects
A small effect size might still be important, but it requires careful interpretation. Consider the outcome, the baseline risk, and the feasibility of intervention. A small effect on a common and serious outcome can have substantial public health importance, while a small effect on a trivial outcome might be negligible.
Limitations and Caveats
Effect Size Does Not Prove Causation
A large effect size in an observational study does not establish causation. Confounding variables, selection bias, and measurement error can all inflate or deflate effect sizes. Randomized controlled trials provide stronger evidence for causation, but even they have limitations.
Benchmarks Are Field-Specific
The conventional benchmarks for small, medium, and large effects are not universal. What counts as a large effect in one field might be considered small in another. Always interpret effect sizes within the context of your field and the specific research question.
Publication Bias Can Distort Effect Sizes
Studies with large effect sizes are more likely to be published than studies with small or null effects. This publication bias means that the effect sizes reported in the literature might overestimate the true effects. When reading papers, be aware that the published literature is not a random sample of all conducted studies.
Effect Size Does Not Capture All Relevant Information
Effect size summarizes the magnitude of an effect, but it does not capture all aspects of a study's findings. Absolute risk reductions, number needed to treat, and other measures provide complementary information. A complete understanding of a study requires looking at multiple measures.
Reporting Guidelines and Transparency
The Role of Reporting Guidelines
Reporting guidelines provide structured checklists for what to include in a research paper. They help ensure that authors report effect sizes, confidence intervals, and other essential information. The EQUATOR Network maintains a comprehensive collection of reporting guidelines for different study types, helping researchers select the appropriate guideline for their design.
Transparency in Research Reporting
Transparent reporting includes also effect sizes but also the methods used to calculate them, the assumptions made, and the limitations of the analysis. The Committee on Publication Ethics outlines core practices for ethical research and publication, including expectations for data integrity and transparent reporting. Following these practices strengthens the credibility of research and allows others to build on it.
Data Sharing and Reproducibility
Effect sizes are only useful if the underlying data are available for verification. The National Institutes of Health has established data management and sharing expectations for funded research, requiring researchers to plan for data sharing. Making data available allows others to verify effect size calculations and conduct secondary analyses.
Professional Escalation Criteria
When to Question an Effect Size
You should question an effect size when it seems implausibly large, when the confidence interval is extremely wide, or when the reported metric does not match the study design. These situations warrant closer scrutiny of the methods and possibly consultation with a statistician.
When to Seek Statistical Consultation
If you are conducting research and are uncertain about which effect size metric to use, how to calculate it, or how to interpret it, consult a statistician. Statistical expertise is valuable at the design stage, before data collection, and again during analysis. The cost of consultation is small compared with the cost of a flawed study.
When to Escalate Concerns About Published Research
If you identify serious problems with a published paper, such as missing effect sizes, implausible values, or apparent errors, consider contacting the journal. The Committee on Publication Ethics provides guidance on handling research misconduct and publication ethics concerns. Journals have processes for addressing concerns about published work.
Frequently Asked Questions
What is the difference between effect size and statistical significance?
Statistical significance tells you whether a result is likely due to chance, while effect size tells you how large the result is. A study can produce a statistically significant result with a trivial effect size, especially with a large sample. Effect size provides the practical context that significance testing alone cannot.
How do I choose which effect size metric to use?
Choose the metric that matches your research design. Use Cohen's d for comparing two group means, correlation coefficients for relationships between continuous variables, and odds ratios for binary outcomes. The design determines the appropriate metric, and using the wrong metric produces meaningless results.
What is a good effect size?
There is no universal answer. Conventional benchmarks exist, such as Cohen's suggestion that 0.2, 0.5, and 0.8 represent small, medium, and large effects for Cohen's d. However, these are rough guidelines, and what counts as a good effect size depends on your field, your outcome, and your practical context.
Why do some papers report effect sizes and others do not?
Reporting practices vary by field and journal. Some journals require effect sizes as part of standard reporting, while others do not. Reporting guidelines, such as those collected by the EQUATOR Network, increasingly recommend or require effect sizes. The trend is toward more complete reporting.
Can I calculate effect size from a paper that does not report it?
In many cases, yes. If the paper reports means and standard deviations for two groups, you can calculate Cohen's d. If it reports a correlation coefficient, that is already an effect size. If it reports the necessary data in tables, you can calculate the effect size yourself.
How does effect size relate to sample size?
Effect size and sample size are related through statistical power. Larger samples can detect smaller effects, while smaller samples require larger effects to reach statistical significance. Power analysis uses expected effect size to determine the sample size needed for a study.
What is the relationship between effect size and confidence intervals?
The confidence interval around an effect size tells you the precision of the estimate. A narrow confidence interval indicates a precise estimate, while a wide interval indicates imprecision. The confidence interval also allows you to assess statistical significance: if it includes zero for a difference measure, or one for an odds ratio, the effect is not statistically significant.
Why is effect size important for grant applications?
Grant reviewers evaluate whether the proposed sample size is adequate to detect the effect of interest. A clear effect size justification demonstrates methodological rigor and increases the likelihood of funding. The National Institutes of Health provides guidance on grant application and review processes, and reviewers commonly expect explicit effect size assumptions.
Using the Evidence
| Source | Best use in this topic | Important limitation |
|---|---|---|
| Research Methods Resources | official guidance | Check the linked page for current local requirements |
| EQUATOR Network | official guidance | Check the linked page for current local requirements |
| Core Practices | official guidance | Check the linked page for current local requirements |
Related Articles
- Effect Size in Statistics: Interpreting the Magnitude of Findings
- Effect Size in Statistics: Why It Matters and How to Interpret It
- Statistical Power: What It Is and Why It Matters in Research
- Binomial Effect Size Display: A Tool for Interpreting Effect Sizes
- Sample Size Symbol and Notation: What You Need to Know
References and Further Reading
- Research Methods Resources. National Library of Medicine.
- EQUATOR Network. EQUATOR Network.
- Core Practices. Committee on Publication Ethics.
- NIH Grants and Funding. National Institutes of Health.
- ORCID for Researchers. ORCID.
- Data Management and Sharing Policy. National Institutes of Health.
- NCBI Literature Resources. National Center for Biotechnology Information.
- PubMed. National Library of Medicine.
This article is educational and does not replace institutional policy, professional advice, or applicable safety and regulatory requirements.