Sample Size in Research Paper Methods
By Dr. Zubair Khalid, DVM, MS, PhD ·
Quick Answer
- Sample size adequacy is judged by reading the power analysis, effect size, and statistical methods section, not by the number alone.
- Look for a prospective justification that connects the sample size to the primary outcome and the smallest effect worth detecting.
- A missing or vague sample size justification is a red flag that the study may be underpowered to support its claims.
Why Sample Size Determines What a Study Can Claim
Sample size is the single most important design feature that determines whether a research paper can support its conclusions. A study with too few subjects cannot reliably detect a real effect, while a study with too many may waste resources and expose animals or participants to unnecessary procedures. When you read a research paper, the sample size tells you how much statistical confidence the authors can place in their results.
The core problem for readers is that sample size is not a standalone number. It only becomes meaningful when interpreted alongside the expected effect size, the variability of the measurements, and the chosen significance threshold. A study of 10 animals per group might be perfectly adequate for a large and consistent effect, while a study of 100 per group could be underpowered for a small and variable effect. This is why the methods section must explain how the number was derived.
Research reporting guidelines exist specifically to help authors disclose their sample size decisions transparently. The EQUATOR Network maintains a comprehensive library of reporting guidelines that tell authors exactly what to include in their methods sections. When a paper follows these guidelines, the sample size justification is easy to find and evaluate. When it does not, the reader must treat the results with caution.
At a Glance
| What to Look For | What It Tells You | Red Flag |
|---|---|---|
| Power analysis with named parameters | The authors calculated sample size to detect a specified effect | Power analysis missing or says "based on previous studies" without details |
| Effect size stated in meaningful units | You can judge whether the effect is practically important | Effect size reported only as a generic small, medium, or large label |
| Primary outcome clearly identified | The sample size is tied to the main question of the study | Multiple outcomes with no indication which one drove the sample size |
| Dropout or loss to follow-up accounted for | The final sample size will still have adequate power | No mention of expected attrition or how missing data were handled |
| Statistical software and version named | The analysis is reproducible | Software not named or version omitted |
| Ethics approval and animal welfare statement | The study met institutional requirements | No ethics statement or welfare information |
How to Find the Sample Size Justification in a Paper
Locate the Statistical Methods Subsection
Most research papers place the sample size justification in the methods section under a heading such as "Statistical Analysis," "Sample Size," or "Power Calculation." In animal studies, this information often appears near the description of experimental design and group allocation. The justification should explain how many animals or observations were included per group and why that number was chosen.
The National Library of Medicine hosts authoritative biomedical texts that describe the standard structure of research papers. These references confirm that the methods section must provide enough detail for another researcher to replicate the study. If the sample size justification is absent from the methods, check the figure legends, supplementary materials, or the results section where the authors may describe the actual number of animals used.
Identify the Primary Outcome
The sample size calculation must be anchored to a single primary outcome. This is the measurement that the study is designed to detect a difference in. For example, in a feed trial, the primary outcome might be average daily gain. In a vaccine study, it might be antibody titer. The authors should state which outcome was used for the power calculation.
When the paper lists multiple outcomes without identifying a primary one, the sample size becomes difficult to evaluate. Each additional outcome increases the chance of finding a false positive by chance alone. The Committee on Publication Ethics core practices emphasize that transparent reporting of study design is part of responsible research conduct. A paper that does not identify its primary outcome is not meeting this standard.
Check Whether the Justification Is Prospective or Retrospective
A prospective sample size justification means the authors calculated the required number before the study began. This is the correct approach because it prevents the researchers from deciding after seeing the results whether the sample was large enough. A retrospective justification, where the authors claim the sample size was adequate after the fact, is statistically weak and should be treated with skepticism.
Look for language that indicates the calculation was done in advance. Phrases such as "we calculated that X animals per group would provide 80 percent power" indicate a prospective approach. Phrases such as "the sample size was similar to previous studies" or "we used all available animals" do not constitute a valid prospective justification.
Core Principles of Sample Size Interpretation
Statistical Power
Statistical power is the probability that a study will detect a true effect when one exists. Power depends on four factors: the sample size, the effect size, the variability of the measurements, and the significance level. Increasing the sample size increases power, but the relationship is not linear. Doubling the sample size does not double the power.
The conventional target in biomedical research is 80 percent power, meaning the study has an 80 percent chance of detecting the specified effect. Some studies aim for 90 percent power, particularly when the outcome is clinically important or when the cost of missing a true effect is high. The National Institutes of Health grant review process expects applicants to justify their sample sizes with power calculations, and reviewers are trained to reject proposals with inadequate power.
Effect Size
The effect size is the magnitude of the difference the study is designed to detect. It can be expressed in the original units of measurement, such as a 2 kilogram difference in body weight, or as a standardized measure such as Cohen's d. The effect size is the most subjective part of the power calculation because the researchers must decide what difference is worth detecting.
A study that is powered to detect a very large effect may miss a smaller but still meaningful effect. Conversely, a study powered to detect a very small effect may require an impractically large sample. When you read a paper, ask whether the effect size used in the power calculation matches what would be considered practically important in the field.
Variability
Variability refers to how much the measurements differ between subjects within the same group. Higher variability requires a larger sample size to achieve the same power. In animal studies, variability can come from genetic differences, environmental conditions, or measurement error. The power calculation should state the expected variability, often as a standard deviation, and explain where that estimate came from.
When the actual variability observed in the study is much higher than the estimate used in the power calculation, the study may be underpowered even if the planned sample size was correct. This is why the results section should report the actual variability alongside the sample size.
Significance Level
The significance level, usually set at 0.05, is the probability of declaring a difference when none exists. This is the false positive rate. The power calculation must state the significance level used. Some studies use a more stringent level, such as 0.01, when multiple comparisons are being made or when the consequences of a false positive are severe.
Practical Workflow for Evaluating Sample Size
Step 1: Read the Abstract for the Sample Size Claim
The abstract often states the number of animals or participants used. This gives you a preliminary sense of the study scale, but it does not tell you whether the sample size was justified. Note the number and move to the methods section.
Step 2: Find the Power Analysis in the Methods
Search the methods section for the terms "power," "sample size," or "effect size." If the authors performed a power analysis, it will be here. Read the entire paragraph, beyond the final sample size number. The justification should include the expected effect size, the variability estimate, the significance level, and the target power.
Step 3: Evaluate Whether the Parameters Are Reasonable
Ask whether the effect size is realistic for the field. A power calculation that assumes a very large effect to justify a small sample size may be overly optimistic. Compare the variability estimate with what is known about the measurement system. If the authors used a standard deviation from a previous study, check whether that previous study used similar animals and methods.
Step 4: Check Whether the Final Sample Size Matches the Plan
The number of animals actually analyzed may differ from the number planned. Animals may die, samples may be lost, or equipment may fail. The paper should report both the planned and the actual sample sizes. If the actual sample size is substantially smaller than planned, the study may have less power than intended.
Step 5: Look for a CONSORT or ARRIVE Flow Diagram
The EQUATOR Network promotes the use of flow diagrams that show how many animals were in each group at each stage of the study. These diagrams make it easy to see whether animals were excluded and why. A flow diagram that shows large numbers of exclusions should raise concerns about bias.
Step 6: Consider Whether the Study Is Exploratory or Confirmatory
Exploratory studies are designed to generate hypotheses, and they may use smaller sample sizes because the goal is not to prove a definitive effect. Confirmatory studies are designed to test a specific hypothesis, and they require adequate power. The paper should state which type of study it is. If the authors describe the study as exploratory but draw confirmatory conclusions, the sample size becomes a critical limitation.
Common Failure Patterns in Sample Size Reporting
The Missing Justification
The most common failure is the complete absence of a sample size justification. The methods section describes the statistical tests but never explains how many animals were needed or why. This pattern is common in smaller studies and in papers from fields where sample size reporting is not yet standard practice. Without a justification, the reader cannot determine whether the study was adequately powered.
The Vague Justification
Some papers include a statement such as "sample size was determined based on previous studies" without providing the specific numbers from those studies. This is not a valid justification because it does not allow the reader to verify the calculation. A proper justification must include the actual parameters used in the calculation.
The Post Hoc Justification
A post hoc power analysis uses the observed effect size and variability to calculate the power of the completed study. This approach is circular because the observed effect size is already known. The National Library of Medicine resources on research methods explain that post hoc power analysis does not provide useful information about the adequacy of the study. If a paper reports only a post hoc power analysis, the sample size was likely not justified in advance.
The Overstated Effect Size
A power calculation that assumes an unrealistically large effect size can justify a small sample size. For example, a study that assumes a 50 percent difference between groups when the field expects a 10 percent difference will require far fewer animals than is appropriate. The reader should compare the assumed effect size with the effect sizes reported in similar studies.
The Ignored Attrition
Animal studies often lose subjects to death, illness, or technical failure. A power calculation that does not account for expected attrition will result in a final sample size that is too small. The methods section should state the expected dropout rate and explain how the planned sample size was inflated to compensate.
Records and Measurements That Support Sample Size Evaluation
What to Record When Reading a Paper
Keep a structured record of the key sample size information from each paper you read. This record will help you compare studies and identify patterns in the literature. For each paper, record the following:
- The planned sample size per group
- The actual sample size per group
- The primary outcome used for the power calculation
- The assumed effect size and variability
- The target power and significance level
- The software and version used for the calculation
- Whether the justification was prospective or retrospective
- Whether a flow diagram was included
How to Use These Records
These records allow you to build a picture of the typical sample sizes and effect sizes in your field. When you read a new paper, you can compare its sample size justification with the records from similar studies. If the new paper assumes an effect size that is much larger than what other studies have observed, you have reason to question the adequacy of the sample.
For researchers planning their own studies, these records provide the raw material for power calculations. The variability estimates and effect sizes from published studies can be used to justify the sample size in a new protocol. The National Institutes of Health expects grant applications to include this level of detail in the research strategy section.
Quality Controls and Professional Escalation Criteria
When to Accept a Sample Size Justification
Accept the sample size justification when the methods section includes a prospective power analysis with named parameters, the effect size is realistic compared with the literature, the variability estimate is reasonable, and the actual sample size matches the plan. The paper should also report the software used and provide enough detail for the calculation to be reproduced.
When to Question a Sample Size Justification
Question the sample size when the justification is missing, vague, or retrospective. Question it when the assumed effect size is much larger than what similar studies have found. Question it when the actual sample size is much smaller than planned and the paper does not explain why. Question it when the paper draws strong conclusions from a study that appears underpowered.
When to Escalate to a Professional
If you are evaluating a paper that will inform a regulatory submission, a clinical decision, or a significant financial investment, and the sample size justification is inadequate, escalate the concern to a statistician or a methodological expert. The Committee on Publication Ethics core practices emphasize that journals and authors share responsibility for the integrity of the published record. A statistician can determine whether the study results are robust despite the sample size limitations.
For animal owners and laboratory staff, an underpowered study does not necessarily mean the results are wrong. It means the study cannot distinguish a true effect from random variation. If a paper with an inadequate sample size is the basis for a treatment decision, seek additional evidence from larger studies or systematic reviews before changing practice.
Welfare and Safety Context for Animal Studies
The Ethical Obligation to Use Adequate Sample Sizes
The use of too few animals in a study is beyond a statistical problem. It is an ethical problem because the animals have been subjected to procedures without the possibility of producing a reliable answer. The National Institutes of Health requires that research involving animals be designed to minimize the number of animals used while still achieving the scientific objectives. An underpowered study wastes the lives of the animals involved.
The Obligation to Avoid Excessive Sample Sizes
The opposite problem, using more animals than necessary, also raises ethical concerns. The principle of reduction in animal research ethics requires that researchers use the minimum number of animals needed to achieve the scientific objective. A power analysis that is too conservative, assuming a very small effect size or very high variability, can lead to an unnecessarily large sample. The reader should consider whether the sample size is proportionate to the question being asked.
Reporting Animal Welfare Information
The methods section of an animal study should include information about housing, husbandry, and any procedures that affect animal welfare. The EQUATOR Network reporting guidelines for animal studies, such as the ARRIVE guidelines, require authors to report this information. When this information is missing, the reader cannot fully evaluate the study, and the sample size justification becomes harder to assess because welfare-related exclusions may have affected the final numbers.
Limitations of Sample Size Evaluation
You Cannot Determine Power From the Sample Size Alone
The most important limitation is that the sample size number by itself tells you nothing about the adequacy of the study. A study with 50 animals per group may be underpowered if the effect is small and the variability is high. A study with 8 animals per group may be adequately powered if the effect is large and consistent. You must read the power analysis to make a judgment.
Published Studies May Have Selective Reporting
The Committee on Publication Ethics core practices address the problem of selective reporting, where authors report only the outcomes that showed significant results. If the sample size justification is missing or unclear, the reader cannot rule out the possibility that the authors chose the sample size after seeing the results. This is a form of bias that cannot be detected from the paper alone.
You Cannot Verify the Assumptions
The power calculation depends on assumptions about effect size and variability that the reader cannot verify from the paper alone. The authors may have used estimates from unpublished pilot data or from studies with different methods. The reader must decide whether these assumptions are credible based on the broader literature.
The Sample Size Does Not Address Other Sources of Bias
An adequate sample size does not protect against bias from poor randomization, lack of blinding, or measurement error. A study can have a perfect power analysis and still produce misleading results if the animals were not randomly assigned to groups or if the outcome assessor knew which group each animal was in. The sample size is one component of study quality, not the whole picture.
Regulatory and Funding Context
NIH Grant Review Expectations
The National Institutes of Health grant application process requires investigators to justify their proposed sample sizes. Peer reviewers evaluate whether the sample size is adequate to answer the research question and whether the power analysis is based on reasonable assumptions. A grant application with an inadequate sample size justification is likely to receive a poor score regardless of the scientific merit of the hypothesis.
Data Management and Sharing Requirements
The National Institutes of Health requires that research data be managed and shared in ways that allow other researchers to verify the results. This includes the data that support the sample size calculation. When data are shared, other researchers can reanalyze the study and assess whether the sample size was adequate. The data management plan should describe how the data will be preserved and made available.
Researcher Identity and Reproducibility
The ORCID for Researchers resource explains how persistent researcher identifiers support the integrity of the scientific record. When researchers are consistently identified across their publications, it becomes easier to track their methods and assess the quality of their work over time. This context is relevant to sample size evaluation because it allows readers to see whether an author has a pattern of inadequate sample size reporting.
Practical Steps for Different Reader Roles
For Students Reading Papers
When you are reading a paper for a class or a literature review, use the sample size evaluation as a structured exercise. Find the power analysis, write down the key parameters, and assess whether the justification is adequate. This practice will build the skills you need to evaluate research independently.
For Researchers Planning Studies
Use the sample size evaluation of published papers to inform your own study design. Collect the effect sizes and variability estimates from similar studies and use them in your power calculations. Document your assumptions clearly so that reviewers can evaluate your justification.
For Laboratory Staff and Animal Owners
When a study is the basis for a treatment or management decision, check whether the sample size was adequate before changing your practices. If the study is underpowered, look for additional evidence from larger studies or systematic reviews. The National Library of Medicine provides access to systematic reviews that synthesize evidence across multiple studies.
For Job Seekers in Research Positions
During interviews for research positions, ask about the sample size practices of the laboratory or organization. A group that routinely performs prospective power analyses and follows reporting guidelines is more likely to produce reliable research. The EQUATOR Network can help you identify which reporting guidelines are relevant to the type of research the organization conducts.
How to Write a Sample Size Justification for Your Own Paper
State the Primary Outcome First
Begin the sample size justification by naming the primary outcome. This tells the reader what the study is designed to detect and anchors the entire calculation. Without a named primary outcome, the justification is incomplete.
Provide the Parameters Used
State the expected effect size, the variability estimate, the significance level, and the target power. Provide the source of these estimates, such as a previous study or a pilot experiment. The reader should be able to reproduce the calculation with the information you provide.
Name the Software and Version
Identify the statistical software and version used for the calculation. This allows the reader to verify the calculation and ensures that the analysis is reproducible. The National Institutes of Health expects this level of detail in grant applications and in publications arising from funded research.
Account for Attrition
State the expected dropout rate and explain how the planned sample size was adjusted to account for it. This shows the reader that the final sample size will be adequate even after expected losses.
Report the Actual Sample Size
In the results section, report the actual number of animals or observations in each group. If the actual number differs from the plan, explain why. This transparency is consistent with the reporting standards promoted by the EQUATOR Network.
Frequently Asked Questions
What is the minimum acceptable sample size for a research study?
There is no universal minimum sample size. The required number depends on the effect size, variability, significance level, and target power. A study with a large and consistent effect may need only a few animals per group, while a study with a small and variable effect may need dozens or hundreds. The adequacy of the sample size can only be judged by reading the power analysis in the methods section.
How do I know if a study is underpowered?
A study is underpowered when it has too few subjects to reliably detect the effect it is designed to measure. You can identify an underpowered study by checking whether the power analysis used a realistic effect size and whether the actual sample size matches the planned sample size. A paper that reports a nonsignificant result from a small sample may be underpowered, but you cannot conclude this from the p value alone.
What is the difference between a prospective and a retrospective power analysis?
A prospective power analysis is performed before the study begins and uses assumed effect sizes and variability to calculate the required sample size. A retrospective power analysis is performed after the study is complete and uses the observed effect size and variability. Retrospective power analysis is circular and does not provide useful information about the adequacy of the study.
Why do some papers not report a sample size justification?
Some papers do not report a sample size justification because the authors did not perform one, because the journal does not require it, or because the authors are following older reporting practices. The absence of a justification is a limitation of the paper and should be considered when evaluating the results. Reporting guidelines from the EQUATOR Network are designed to reduce this problem.
Can a study with a small sample size still be valid?
A study with a small sample size can be valid if the effect is large and consistent enough to be detected with the available number of subjects. The validity depends on whether the study was adequately powered for the effect it was designed to detect. A small study that reports a large and statistically significant effect may be valid, but the confidence intervals will be wide.
How does sample size affect the interpretation of a nonsignificant result?
A nonsignificant result from an underpowered study does not mean that no effect exists. It means that the study could not detect the effect with the available sample size. When you read a paper that reports a nonsignificant result, check whether the sample size was adequate to detect a meaningful effect. If the study was underpowered, the nonsignificant result is inconclusive.
What should I do if a paper I am relying on has an inadequate sample size?
If a paper with an inadequate sample size is the basis for a treatment or management decision, seek additional evidence from larger studies, systematic reviews, or meta-analyses. The National Library of Medicine provides access to these types of evidence. If the decision is significant, consult a statistician or methodological expert.
How can I verify the sample size calculation in a paper?
You can verify the sample size calculation by using the parameters reported in the methods section and running the calculation in statistical software. The paper should provide the effect size, variability, significance level, and target power. If any of these parameters are missing, you cannot fully verify the calculation.
Using the Evidence
| Source | Best use in this topic | Important limitation |
|---|---|---|
| Research Methods Resources | official guidance | Check the linked page for current local requirements |
| EQUATOR Network | official guidance | Check the linked page for current local requirements |
| Core Practices | official guidance | Check the linked page for current local requirements |
Related Articles
- Statistical Power And Sample Size
- Sample Size Symbol and Notation: What You Need to Know
- Statistical Power: What It Is and Why It Matters in Research
- The Role of the Research Ethics Committee: What Researchers Need to Know
- Sample Size Calculation for Diagnostic Accuracy Studies: A Practical Guide
References and Further Reading
- Research Methods Resources. National Library of Medicine.
- EQUATOR Network. EQUATOR Network.
- Core Practices. Committee on Publication Ethics.
- NIH Grants and Funding. National Institutes of Health.
- ORCID for Researchers. ORCID.
- Data Management and Sharing Policy. National Institutes of Health.
- NCBI Literature Resources. National Center for Biotechnology Information.
- PubMed. National Library of Medicine.
This article is educational and does not replace institutional policy, professional advice, or applicable safety and regulatory requirements.