How to Report Risk of Bias in Your Systematic Review

By Dr. Zubair Khalid, DVM, MS, PhD · 2026-08-27

Quick Answer

• Report risk of bias methods by naming the tool, the number of reviewers, disagreement resolution, and how judgments map to your synthesis.
• Present results in a dedicated figure and table that show per-study judgments across each bias domain.
• State limitations honestly, including whether unpublished data or non-English studies were excluded, since these choices affect confidence in your conclusions.

At a Glance

Why Risk of Bias Reporting Matters in Systematic Reviews

Systematic reviews are built on the premise that methods are transparent enough for another team to replicate. Risk of bias assessment is the step where you judge whether the studies you found can actually support the conclusions you draw. If you do not report this step clearly, readers cannot tell whether your confidence in the evidence is justified.

The EQUATOR Network maintains a searchable library of reporting guidelines that help authors decide what to include in their manuscripts. For systematic reviews, the relevant guideline is PRISMA, which explicitly requires authors to describe the risk of bias methods used and to present the results of those assessments. Journals increasingly check compliance with these guidelines during peer review, so a complete risk of bias section is often a precondition for publication.

Beyond meeting journal requirements, clear reporting serves a scientific purpose. Risk of bias judgments are interpretations, not raw data. Two reviewers can look at the same study and reach different conclusions about whether allocation was concealed or whether outcome measurement was blinded. When you report your judgments with supporting text, you allow readers to evaluate whether your interpretations are reasonable. When you do not, you force readers to either trust your conclusions or redo the assessment themselves.

The Committee on Publication Ethics core practices emphasize that authors should describe their methods accurately and completely. Inaccurate or incomplete method reporting is a form of publication misconduct because it misleads readers about the strength of the evidence. This is not an abstract concern. Clinical decisions, agricultural practices, and funding priorities are all influenced by systematic review conclusions, so the integrity of the reporting matters beyond the individual manuscript.

Selecting the Right Risk of Bias Tool

Your choice of tool must match the study designs included in your review. Using a tool designed for randomized trials on observational studies produces misleading judgments, and the reverse is equally problematic.

Tools for Randomized Trials

The Cochrane Risk of Bias tool for randomized trials, now in its revised version known as RoB 2, is the most widely accepted instrument for assessing randomized controlled trials. It covers five domains: bias arising from the randomization process, bias due to deviations from intended interventions, bias due to missing outcome data, bias in measurement of the outcome, and bias in selection of the reported result. Each domain receives a judgment of low risk, some concerns, or high risk.

When you report using RoB 2, you should state which version you used and whether you assessed individual results or entire studies. The tool is designed to assess a specific result within a trial, not the trial as a whole, so you need to clarify whether you assessed the primary outcome, multiple outcomes, or all reported results.

Tools for Non-Randomized Studies

For cohort studies, case-control studies, and other non-randomized designs, the Risk Of Bias In Non-randomized Studies of Interventions tool, known as ROBINS-I, is the standard choice. It covers seven domains including confounding, selection of participants, classification of interventions, deviations from intended interventions, missing data, measurement of outcomes, and selection of the reported result.

ROBINS-I requires you to specify the target trial, which is the hypothetical randomized trial that your non-randomized study is trying to emulate. This is a conceptual step that many authors skip, but it is essential for judging whether confounding was adequately addressed. Your methods section should state the target trial for each comparison or explain why a single target trial applied across all included studies.

Tools for Other Designs

If your review includes diagnostic accuracy studies, animal studies, or qualitative research, you need design-specific tools. The National Library of Medicine bookshelf hosts method guidance for many of these specialized designs, and the EQUATOR Network can help you identify the appropriate reporting guideline for your study type.

Your methods section should justify your tool selection in one or two sentences. For example, you might state that RoB 2 was used for randomized trials and ROBINS-I for non-randomized studies because the review included both designs and the tools are designed for those specific study types. This justification helps readers understand that your choice was deliberate instead of convenient.

Describing Your Risk of Bias Methods

The methods subsection for risk of bias should be detailed enough for another team to replicate your judgments. The EQUATOR Network reporting guidelines provide the framework for what to include, but the specific content depends on your review.

Number of Reviewers and Training

State how many reviewers independently assessed each study. The standard approach is two reviewers working independently, with disagreements resolved through discussion or by consulting a third reviewer. If you used more than two reviewers, say so and describe how the work was divided.

If your reviewers received training on the tool before starting, mention this. Training can be formal, such as a workshop or online module, or informal, such as pilot testing the tool on a sample of studies not included in the review. Pilot testing is particularly valuable because it reveals whether reviewers interpret the tool consistently before they assess the actual included studies.

Disagreement Resolution

Describe exactly how disagreements were handled. Common approaches include consensus discussion, arbitration by a third reviewer, or a combination of both. If you measured inter-rater agreement, report the statistic you used and the value you obtained. Cohen kappa is the most common choice, but you should report whatever measure is appropriate for your data.

Contacting Study Authors

State whether you attempted to contact study authors for missing information. Many risk of bias judgments depend on details that are not reported in the published manuscript, such as whether allocation was concealed or whether outcome assessors were blinded. Contacting authors can resolve these uncertainties, but it is not always successful.

If you contacted authors, report how many you contacted, how many responded, and how the information you received affected your judgments. If you did not contact authors, state this and acknowledge that some judgments may be based on incomplete information.

Assessment of Specific Outcomes

Clarify whether you assessed risk of bias for each outcome or for the study as a whole. This distinction matters because a study can be at low risk of bias for one outcome and high risk for another. For example, a trial might have objective outcome data that are unlikely to be biased by lack of blinding, while a subjective outcome in the same trial could be at higher risk.

If you assessed multiple outcomes per study, report how you handled the different judgments in your synthesis. Some reviews use the highest risk judgment across outcomes as the overall rating, while others report outcome-specific judgments separately. Your approach should be stated explicitly.

Presenting Risk of Bias Results

The results of your risk of bias assessment should be presented in two complementary formats: a figure showing judgments across domains and a table providing the supporting rationale for each judgment.

Creating the Risk of Bias Figure

The standard figure format is a traffic light plot with rows for each study and columns for each bias domain. Each cell is colored green for low risk, yellow for some concerns, and red for high risk. This format allows readers to see patterns across studies at a glance, such as whether most studies have low risk for randomization but high risk for blinding.

Your figure should include a legend explaining the color coding and the meaning of each judgment category. If you used different tools for different study designs, consider creating separate figures for each tool or clearly marking which studies were assessed with which tool.

Building the Risk of Bias Table

The table should provide the detailed rationale behind each judgment. Each row represents a study, and each column represents a bias domain. The cell content should include the judgment and a brief justification, such as "low risk, central randomization used" or "high risk, no blinding of outcome assessors and outcome was subjective."

This table is where you demonstrate that your judgments are based on evidence from the study reports instead of assumptions. A common failure is providing judgments without justification, which forces readers to take your word for the assessment. The table should be detailed enough that a reader can understand why each judgment was made without returning to the original study.

Supporting Data Availability

The NIH Data Management and Sharing Policy requires NIH-funded researchers to plan for data sharing, and this expectation extends to systematic review data in many cases. Your risk of bias assessments are data, and making them available supports transparency and reproducibility.

State in your manuscript where the full risk of bias assessments can be accessed. Options include a supplementary file, a repository such as Figshare or Zenodo, or the review protocol if it includes the assessment forms. If your data are available on request, provide the contact details and describe what is available.

Integrating Risk of Bias into Your Synthesis

Risk of bias assessment is not an end in itself. The purpose is to inform your synthesis and the confidence you place in your conclusions. Your results section should show how risk of bias influenced your analysis.

Sensitivity Analyses

A common approach is to conduct sensitivity analyses that exclude studies at high risk of bias or restrict the analysis to studies at low risk. If you used this approach, report which studies were excluded and how the results changed. If the conclusions remained the same after excluding high-risk studies, state this because it strengthens confidence in the findings.

If you did not conduct sensitivity analyses, explain why. In some reviews, the number of studies is too small to make sensitivity analysis meaningful, or all studies have similar risk of bias profiles. A brief explanation is sufficient, but the omission should be acknowledged instead of ignored.

Certainty of Evidence

Risk of bias is one of several domains considered when rating the certainty of evidence using frameworks such as GRADE. Other domains include inconsistency, indirectness, imprecision, and publication bias. Your risk of bias judgments feed into the certainty rating, so you should describe how this integration occurred.

If you used GRADE or a similar framework, state this and describe how risk of bias judgments contributed to the overall certainty rating. If you did not use a formal framework, describe how risk of bias influenced your interpretation of the findings.

Presenting Results by Risk of Bias Level

Consider presenting your main results stratified by risk of bias level. This can be done in a forest plot with subgroups for low risk, some concerns, and high risk, or in a table showing effect estimates for each subgroup. This presentation allows readers to see whether the effect estimate changes when only higher-quality studies are considered.

Common Failure Patterns in Risk of Bias Reporting

Understanding how risk of bias reporting commonly fails can help you avoid the same problems in your manuscript.

Reporting Only Overall Judgments

Some reviews report a single overall risk of bias rating for each study without providing domain-level judgments. This is problematic because the overall rating obscures important differences across domains. A study might be at low risk for randomization but high risk for missing outcome data, and this distinction matters for interpreting the results.

Using the Wrong Tool

Using a tool designed for randomized trials on observational studies produces judgments that are not meaningful. The domains do not map cleanly, and the signaling questions do not apply. This error is common in reviews that include mixed study designs and use a single tool for all studies.

Omitting the Rationale for Judgments

Providing judgments without supporting text is a frequent problem. A table that says "high risk" without explaining why is not useful to readers. The rationale should reference specific aspects of the study, such as the method of allocation or the proportion of missing outcome data.

Failing to Link Risk of Bias to Conclusions

Some reviews present risk of bias results in a figure and table but never mention them again in the discussion or conclusions. This disconnect leaves readers wondering whether the authors considered the risk of bias when interpreting the findings. Your discussion should explicitly state how risk of bias affected your confidence in the conclusions.

Ignoring Disagreements Between Reviewers

If two reviewers disagreed on judgments, the resolution process should be described. Some manuscripts report that disagreements were resolved by consensus without explaining what that involved. If a third reviewer was consulted, this should be stated.

Records and Documentation

Systematic review reporting is supported by thorough documentation throughout the review process. The NIH Grants and Funding resources emphasize the importance of rigorous methods in research, and this extends to the documentation practices that support your risk of bias assessment.

Protocol Documentation

Your review protocol should specify the risk of bias tool you plan to use, the number of reviewers, and the disagreement resolution process before you begin the review. This pre-specification prevents you from changing your methods after seeing the results, which would introduce bias into the review itself.

If you deviate from your protocol, document the deviation and explain why it was necessary. For example, you might have planned to use RoB 2 for all studies but discovered that some included studies were non-randomized and required ROBINS-I instead.

Assessment Forms

Develop a standardized form for recording risk of bias judgments. The form should include the signaling questions from the tool, space for the judgment, and a field for the rationale. Using a standardized form ensures that all studies are assessed consistently and that the documentation is complete.

Reviewer Training Records

Keep records of reviewer training and pilot testing. These records are useful if questions arise about the consistency of judgments during peer review. They also support the credibility of your assessment by demonstrating that reviewers were prepared for the task.

Data Management

The NIH Data Management and Sharing Policy describes expectations for managing and sharing research data. While this policy applies to NIH-funded research, the principles of good data management apply broadly. Your risk of bias assessments should be stored in a format that is accessible, organized, and preserved for future reference.

Professional Escalation Criteria

Some situations during risk of bias assessment warrant escalation to a supervisor, methodologist, or other authority. Recognizing these situations helps you avoid making inappropriate judgments or reporting incomplete assessments.

Unclear Tool Applicability

If you are uncertain whether a tool applies to a specific study design, escalate to a methodologist or systematic review expert before proceeding. Using the wrong tool produces invalid judgments that are difficult to correct after the fact.

Persistent Reviewer Disagreements

If two reviewers consistently disagree on judgments despite discussion, escalate to a third reviewer or a methodologist. Persistent disagreement may indicate that the tool is being interpreted differently or that the study reports are ambiguous in ways that require expert input.

Suspected Data Fabrication

If you suspect that a study report contains fabricated data, escalate to the journal editor or the Committee on Publication Ethics for guidance. Do not attempt to resolve suspected misconduct on your own, as this requires careful handling to protect all parties involved.

Requests for Unpublished Data

If you need unpublished data from study authors and the authors are unresponsive, escalate to your institution or funding body for guidance. The NIH Grants and Funding resources may provide context on expectations for data sharing in funded research.

Writing the Risk of Bias Section

The structure of your risk of bias section will depend on the journal requirements and the reporting guideline you are following. The EQUATOR Network can help you identify the appropriate guideline for your review type.

Methods Subsection

The methods subsection should describe the tool, the review process, and the synthesis integration. A typical structure includes a paragraph on tool selection, a paragraph on the review process, and a paragraph on how judgments were used in the synthesis.

Results Subsection

The results subsection should present the figure and table with brief narrative text highlighting key patterns. Avoid repeating in text what is already visible in the figure and table. Instead, draw attention to patterns that are not immediately obvious, such as consistent high risk of bias in a particular domain across studies.

Discussion Paragraph

The discussion should address how risk of bias affected the interpretation of findings. If the evidence base was limited by high risk of bias, state this and describe the implications for practice and future research.

Example Text for Your Methods Section

The following example text demonstrates how to describe risk of bias methods in a manuscript. Adapt the wording to match your specific review.

Two reviewers independently assessed the risk of bias of each included study using the Cochrane Risk of Bias tool for randomized trials (RoB 2) for randomized controlled trials and the Risk Of Bias In Non-randomized Studies of Interventions tool (ROBINS-I) for non-randomized studies. Disagreements were resolved through discussion, with a third reviewer consulted when consensus could not be reached. Reviewers were not blinded to study authors, journal, or results. Study authors were contacted for missing information when the published report did not provide sufficient detail for a judgment. Risk of bias was assessed for the primary outcome of each study. Judgments were incorporated into the synthesis through sensitivity analyses that excluded studies at high risk of bias.

Example Text for Your Results Section

The following example text demonstrates how to present risk of bias results.

Risk of bias assessments for the 12 included randomized trials are presented in Figure 2 and Table 3. All trials were at low risk of bias for the randomization process. Eight trials were at low risk of bias for deviations from intended interventions, while four trials had some concerns because of potential contamination between intervention groups. Missing outcome data were a concern in three trials, with high risk of bias judgments for two trials that had more than 20 percent missing outcome data. Blinding of outcome assessment was at low risk of bias for objective outcomes but at high risk of bias for subjective outcomes in five trials. All trials were at low risk of bias for selection of the reported result. Sensitivity analysis excluding the two trials at high risk of bias for missing outcome data did not change the direction or magnitude of the pooled effect estimate.

Frequently Asked Questions

Do I need to report risk of bias for every study in my review?

Yes, you should report risk of bias for every included study. Omitting assessments for some studies creates an incomplete picture of the evidence base and makes it difficult for readers to evaluate your conclusions. If a study could not be assessed because of missing information, report this and describe your attempts to obtain the information.

What is the difference between risk of bias and methodological quality?

Risk of bias refers to the likelihood that the design and conduct of a study introduce systematic error in the estimated effect. Methodological quality is a broader concept that includes aspects such as reporting quality and ethical conduct. Risk of bias tools focus specifically on bias, while quality scales often combine multiple dimensions into a single score.

Can I use the same risk of bias tool for all study designs?

No, you should use a tool that matches the study design. Tools designed for randomized trials include domains that do not apply to observational studies, such as bias arising from the randomization process. Using the wrong tool produces judgments that are not meaningful and may mislead readers about the strengths and weaknesses of the evidence.

How do I handle risk of bias assessments for studies with multiple outcomes?

Assess risk of bias for each outcome or specify which outcome you assessed. A study can be at different risk of bias levels for different outcomes, particularly when some outcomes are objective and others are subjective. State your approach in the methods section and apply it consistently across all studies.

Should I contact study authors for missing information?

Contacting study authors is recommended when the published report does not provide enough detail for a judgment. Report how many authors you contacted and how many responded. If you did not contact authors, state this and acknowledge that some judgments may be based on incomplete information.

How do risk of bias judgments affect my conclusions?

Risk of bias judgments should inform your confidence in the findings. If most studies are at high risk of bias, your conclusions should be more cautious than if most studies are at low risk. Describe how risk of bias influenced your interpretation in the discussion section.

What should I do if reviewers disagree on risk of bias judgments?

Disagreements should be resolved through discussion, with a third reviewer consulted if needed. Describe this process in your methods section. If you measured inter-rater agreement, report the statistic and value.

Where should I store my risk of bias data?

Store your risk of bias assessments in a format that is accessible and preserved. Options include supplementary files, repositories, or the review protocol. The NIH Data Management and Sharing Policy describes expectations for data sharing that may apply to your review.

Using the Evidence

Related Articles

• Meta-Analysis vs Systematic Review: What's the Difference?
• How to Write an Experiment Report: A Practical Template
• Meta-Analysis vs Systematic Review: Choosing the Right Synthesis Method
• Scoping Review vs Meta-Analysis: Clarifying Their Distinct Roles in Evidence Synthesis
• Research Figures: Making Methods and Results Easier to Evaluate

References and Further Reading

• Research Methods Resources. National Library of Medicine.
• EQUATOR Network. EQUATOR Network.
• Core Practices. Committee on Publication Ethics.
• NIH Grants and Funding. National Institutes of Health.
• ORCID for Researchers. ORCID.
• Data Management and Sharing Policy. National Institutes of Health.
• NCBI Literature Resources. National Center for Biotechnology Information.
• PubMed. National Library of Medicine.

This article is educational and does not replace institutional policy, professional advice, or applicable safety and regulatory requirements.