# How to Create a Risk of Bias Table for Your Systematic Review By the end of this guide, you will be able to choose the correct risk of bias tool for each study design in your review, build a risk of bias table in a spreadsheet, fill in judgments that a peer reviewer can trace back to the source paper, and summarize the table as a figure and a narrative paragraph. You will need a spreadsheet program such as Excel or Google Sheets, the published instruction manual for your chosen appraisal tool, the full-text PDFs of your included studies, and screening or extraction software such as Covidence if you are working with a team [1]. ## What a Risk of Bias Table Is and Why Your Review Needs One Risk of bias is the extent to which a study's design, conduct, and analysis distort the effect or accuracy it reports. A study can be well written, well funded, and published in a strong journal and still be biased. Bias lives in the methods, not the byline. A risk of bias table is the structured record of how biased each included study could be. It has one row per study and one column per domain, where a domain is a distinct way a study can go wrong, such as flawed randomization or a weak reference standard. For every domain you record a judgment, usually low risk, high risk, or unclear, plus a short justification with a page reference. The table does three jobs. First, it protects your synthesis, because you can weight or interpret high risk studies cautiously instead of pooling them as if they were equivalent to rigorous ones. Second, it feeds formal certainty grading. Reviewers using GRADE, the Grading of Recommendations Assessment, Development and Evaluation framework, downgrade certainty of evidence directly from their risk of bias judgments [2][3][4]. Third, it answers a question every reader, editor, and examiner will ask about any systematic review: can I trust the studies underneath the pooled estimate? Risk of bias sits at a specific point in the review workflow. A systematic literature search identifies records, title and abstract screening narrows them, full-text screening produces your included studies, and risk of bias assessment happens immediately after, before data synthesis. Reviews that skip or rush this step produce pooled estimates built on sand. One recent urology meta-analysis of artificial intelligence for urinary stone detection found that most included studies had a high risk of bias and lacked external validation, a finding that changed how confidently the pooled accuracy numbers could be interpreted [5]. ## Choosing the Right Tool for Each Study Design ![Decision path asking whether the study is randomized, a prediction model, a diagnostic accuracy study, or observational, ending at the matching tool.](/article-images/illustrations/risk-of-bias-table-systematic-review-tool-choice-triage.webp) The single most common failure in risk of bias assessment is using one generic checklist for every study type. Each study design has its own failure modes, and the validated tools target them specifically. Recent published reviews show how the pairing works in practice. | Tool | Designed for | Judgment scale | Used in recent reviews | |--|--|--|--| | Cochrane RoB 2 | Randomized controlled trials of interventions | Low, some concerns, high | Microwave needle radiofrequency acne scar trials [6], auricular therapy trials [7] | | ROBINS-I | Nonrandomized comparative studies | Low, moderate, serious, critical, no information | Nonrandomized comparative evidence in laser versus radiofrequency review [6] | | QUADAS-2 | Diagnostic accuracy studies | Risk of bias plus applicability concerns, each low, high, or unclear | Ultrasound for axillary nodes [8], point-of-care ultrasound for deep vein thrombosis [9], CMV in chorionic villi [10], AI electrocardiography [11], deep learning for cerebral microbleeds [12] | | QUADAS-AI | AI diagnostic studies with dataset and pipeline issues | QUADAS-2 framework extended for AI | CT urinary stone detection AI [5] | | PROBAST | Clinical prediction models | Low, high, or unclear across four domains, plus applicability | Lymphatic filariasis risk models [13], pulmonary tuberculosis prediction models [4] | | PROBAST-AI | Prediction models using AI | PROBAST extended for AI | Large vessel occlusion triage AI [14] | | Joanna Briggs Institute (JBI) tools | Mixed designs including observational and quasi-experimental studies | Varies by checklist, yes/no/partial items with a final judgment | Facial synkinesis treatments [2], GLP-1 receptor agonist facial remodeling [15] | Match the tool to the design, not to your comfort level. If your review mixes designs, you will run more than one tool. The acne scar review, for example, applied Cochrane RoB 2 to randomized comparisons and reserved ROBINS-I for nonrandomized comparative evidence considered in narrative synthesis [6]. ## The Standard Risk of Bias Table Format There are two complementary formats, and most published reviews use both. **Format 1, the summary matrix.** One row per study, one column per domain, each cell holding the judgment. This is the risk of bias table systematic review format that readers see in the results section. For diagnostic accuracy reviews using QUADAS-2, the columns are patient selection, index test, reference standard, and flow and timing, with applicability concerns rated separately for the first three domains [8][11]. **Format 2, the detailed assessment.** One table or appendix entry per study, listing the signaling question answers, the reasoning, and the supporting quotation with a page number. This is the evidence trail. Editors rarely ask for it in the main text, but examiners and meta-analysts rely on it, and you cannot fill format 1 defensibly without it. A simplified summary matrix for a hypothetical diagnostic accuracy review, showing the QUADAS-2 domains, looks like this: | Study | Patient selection | Index test | Reference standard | Flow and timing | Overall | |--|--|--|--|--|--| | Study A | Low | Low | Low | Low | Low | | Study B | High | Low | Unclear | Low | High | | Study C | Low | Unclear | High | High | High | The overall judgment is not an average. In QUADAS-2, any high risk domain typically drives the overall judgment to high risk, and you record the reasoning for the overall call explicitly. ## Step-by-Step Walkthrough ### Step 1: Fix your question and protocol before you assess anything Write your review question in PICO form, population, intervention or index test, comparator, and outcome, and register the protocol in PROSPERO, the International Prospective Register of Systematic Reviews, before you begin appraisal. Registration is now routine in published reviews. The breast ultrasound review [8], the preterm birth biomarker review [16], the deep vein thrombosis review [9], the CMV prenatal diagnosis review [10], and the cerebral microbleeds review [12] all registered their protocols in PROSPERO with registration numbers reported in the methods. Registration locks in your appraisal plan and lets readers spot post hoc deviations. Not every review follows this standard. A recent systematic review of lymphatic filariasis prediction models explicitly reported that it was not prospectively registered, which is a transparency limitation readers had to weigh against the findings [13]. Register early and quote your registration number in your methods section. ### Step 2: Run a systematic literature search that feeds the table Your risk of bias table can only be as good as the set of studies beneath it, and that set comes from a systematic literature search. This means a documented, reproducible search across multiple databases from inception, not a casual PubMed query. Reviews model this well. The antidepressant long-term treatment network meta-analysis searched ten sources including the Cochrane Register, MEDLINE, Embase, CINAHL, PsycINFO, LILACS, and regulatory agency websites from inception to the search date [17]. The sensory-based dental anxiety review searched MEDLINE, Scopus, Embase, PsycINFO, and CENTRAL to a stated cutoff date, supplemented by manual reference screening [3]. The deep learning microbleeds review ran an initial search and then updated the search during revision to catch newly eligible studies [12]. Record every database, the exact date the search was run, and the full search strings. When you later say "we included 14 studies," each of those 14 must appear in your risk of bias table. ### Step 3: Decide your tool per study design before full-text screening Write the tool selection into your protocol. If your eligibility criteria allow randomized trials, nonrandomized comparisons, and observational cohorts, plan to apply RoB 2, ROBINS-I, and the relevant JBI checklist respectively, as mixed-design reviews do [2][15][6]. If your review is a [diagnostic test accuracy](/blog/guides/understanding-diagnostic-test-accuracy-sensitivity-specificity-and-predictive-values) review, follow PRISMA-DTA reporting and use QUADAS-2, as the AI electrocardiography review did [11]. If your review covers prediction models, use PROBAST, which assesses four bias domains, participants, predictors, outcome, and analysis, plus applicability [13][4]. For AI prediction and AI diagnostic reviews, the newer extensions PROBAST-AI and QUADAS-AI address dataset leakage, training set contamination, and validation practices that older tools miss [14][5]. ### Step 4: Build the table skeleton in a spreadsheet Set up one master spreadsheet with the columns your tool requires. A QUADAS-2 template in CSV form: ```csv Study ID,Author,Year,Patient selection judgment,Patient selection rationale,Patient selection page,Index test judgment,Index test rationale,Index test page,Reference standard judgment,Reference standard rationale,Reference standard page,Flow and timing judgment,Flow and timing rationale,Flow and timing page,Applicability concerns,Overall judgment,Reviewer initials,Date assessed,Disagreement resolved by ``` An RoB 2 template for intervention trials: ```csv Study ID,Author,Year,D1 Randomization,D2 Deviations from intended interventions,D3 Missing outcome data,D4 Outcome measurement,D5 Selective reporting,Overall judgment,Rationale summary,Reviewer initials,Date assessed ``` Keep the judgment vocabulary strict. Use exactly three values for RoB 2 and QUADAS-2, Low, High, Unclear, and use the full five-level scale for ROBINS-I, Low, Moderate, Serious, Critical, No information. Anything else breaks downstream plotting software and confuses readers. If you plan to plot with robvis in R, this discipline matters, as the next step shows. ### Step 5: Pilot the tool on a small sample Before assessing all included studies, two reviewers independently assess the same five to ten studies and compare judgments. This calibration round exposes ambiguous signaling questions and forces your team to agree on interpretations, for example how to judge a study that reports a "modified" reference standard. Document your shared decisions in a short assessment guide. Every large review team works this way. The preterm birth biomarker review had two reviewers assess risk of bias with a third reviewer resolving disagreements [16], the CMV review performed assessment in duplicate and blinded [10], and the auricular therapy network meta-analysis had two reviewers independently screen, extract, and assess risk of bias [7]. ### Step 6: Assess every study, domain by domain, in duplicate Work through one study at a time. For each domain, answer the tool's signaling questions, then assign the judgment, then write the rationale before moving on. A defensible QUADAS-2 entry for the index test domain of a hypothetical ultrasound study might read: ``` Domain: Index test (risk of bias) Judgment: Low Rationale: The sonographer was blinded to the reference standard results and used a predefined three-point scanning protocol. Scan protocol was described in Methods, page 4. Blinding stated explicitly, Methods, page 5. ``` And a high risk entry for patient selection: ``` Domain: Patient selection (risk of bias) Judgment: High Rationale: Case-control design. Patients with known disease status enrolled from a oncology referral center. Unclear whether the sample was consecutive or random. Results section, page 7. Case-control design inflates discrimination relative to a clinical cohort. ``` Two habits separate professional assessments from student ones. First, always anchor the judgment to text on a specific page, so a co-reviewer or editor can verify it in under a minute. Second, judge the risk of bias to the review's own question, not to the study's stated aim. A cohort assembled for one purpose may still answer your question cleanly, or may not, and applicability concerns capture that separately in QUADAS-2 and PROBAST [8]. ### Step 7: Handle disagreements, overlap, and missing data Record every disagreement between reviewers and how you resolved it, whether by discussion, by consulting a third reviewer as in the preterm birth review [16], or by consensus meeting. Two further issues trip up real reviews. **Overlapping cohorts.** When several publications report the same patients, their biases are not independent, and pooling them double counts. The axillary ultrasound review examined potentially overlapping cohorts in sensitivity analyses [8], the deep vein thrombosis review included one dataset per study in the primary analysis to avoid duplicate weighting of paired cohorts [9], and the CMV review reduced overlapping cohorts to the largest, most completely verified report [10]. Note overlap in your risk of bias table so the synthesis team can act on it. **Unclear judgments.** Do not leave a domain "unclear" out of laziness. Attempt to contact the authors, check trial registry entries, and check supplementary material. If the information genuinely cannot be obtained, an unclear or no-information judgment is correct and should be stated plainly, not padded into an optimistic call. ### Step 8: Summarize the table as a figure and a narrative Most journals expect two outputs. The first is a traffic light plot, one row per study, one column per domain, colored cells for low, high, and unclear judgments. The second is a weighted bar plot showing the percentage of studies at each judgment level per domain. Both can be generated with the robvis package in R, which accepts your spreadsheet as input: ```r library(robvis) # Read your master CSV, keep column names exactly as robvis expects rob_data <- read.csv("rob2_master.csv") # Traffic light plot, one row per study rob_traffic_light(data = rob_data, tool = "ROB2") # Weighted summary bar plot by domain rob_summary(data = rob_data, tool = "ROB2") ``` Expected output for the traffic light call is a saved figure with one colored cell per study per domain, a legend mapping colors to low, some concerns, and high, and a final "Overall" column on the right. If you used QUADAS-2, pass the judgments through the same plotting approach or construct the figure in your spreadsheet with conditional formatting. Close the risk of bias section with a short narrative naming the weakest domains and the studies most affected. Published reviews do this directly. The urinary stone AI review reported that most included studies had a high risk of bias, lacked external validation, and frequently used shared datasets [5]. The tuberculosis prediction model review found limited external validation and uncertain clinical utility across the models it appraised with PROBAST [4]. The lymphatic filariasis review used PROBAST across 26 studies of statistical, Bayesian spatial, ecological niche, and machine learning models to characterize methodological quality and validation practices [13]. ### Step 9: Carry the judgments into synthesis and certainty grading The table is not decorative. Use it three ways. First, run sensitivity analyses restricted to low risk studies and compare pooled estimates. Second, when heterogeneity appears, risk of bias differences are a leading explanation to explore. Third, downgrade GRADE certainty for the relevant outcomes where bias is present. The facial synkinesis review assessed risk of bias with JBI tools and then evaluated certainty with GRADE [2]. The dental anxiety review applied formal GRADE certainty assessment across its eligible trials [3]. The tuberculosis review paired PROBAST with a GRADE adaptation for diagnostic test accuracy [4]. The preterm birth review rated certainty with GRADE for test accuracy and, because few studies provided usable data and two shared participants, presented study-level estimates in a paired forest plot rather than pooling [16]. That restraint is risk of bias thinking applied to synthesis. ## Checking That Your Table Is Right Before you submit, run these checks. 1. Every included study has a completed row, with no empty judgment cells. 2. The set of studies in the table exactly matches the set in your PRISMA flow diagram and your data extraction tables. The transumbilical breast augmentation review, for example, included 17 studies in both its extraction and its appraisal [18], and the counts must reconcile everywhere they appear. 3. Judgments use only the permitted vocabulary of the chosen tool. 4. Every high risk or unclear judgment carries a rationale with a page reference. 5. Two reviewers' judgments were compared, and the resolution is documented. 6. The overall column is consistent with the domain judgments under your tool's rules. 7. Any deviations from your registered protocol are declared in the methods. ## Common Mistakes and Limitations - **Confusing risk of bias with reporting quality.** A checklist that scores whether a paper reported a CONSORT item measures reporting, not truth. Risk of bias asks whether the underlying conduct threatens the result. The tools above target conduct. Avoid unvalidated scorecards that sum items into a total quality score. - **Using one checklist for all designs.** Applying QUADAS-2 to a prediction model study or RoB 2 to a cohort study produces judgments that miss the design's real vulnerabilities. Use PROBAST for prediction models [13][4], QUADAS-2 for diagnostic accuracy [8][11], PROBAST-AI and QUADAS-AI where AI pipelines add failure modes [14][5], and JBI checklists for heterogeneous observational designs [2][15]. - **Single reviewer assessment.** Self-review hides errors. Reviews published in strong journals consistently use duplicate assessment, sometimes blinded [10], with a third reviewer or structured discussion to resolve disagreements [16]. - **Pre-committing to a verdict.** If you have already decided the pooled effect, your risk of bias judgments will drift that way. Assess in duplicate, blind where feasible, and write rationales anchored to text before discussing with your co-reviewer. - **Ignoring preclinical and mechanistic studies.** Reviews of basic science evidence, such as the berberine immunotherapy review [19], still need structured bias appraisal for the in vitro and in vivo studies they include, rather than narrative cherry-picking. - **Overlooking cohort overlap and duplicate weighting.** Several reviews explicitly engineered their analyses around this, keeping one dataset per study in primary models and testing alternatives in sensitivity analyses [8][9]. - **Treating the table as the end point.** A table of red and yellow cells changes nothing unless it shapes the synthesis, the certainty rating, and the conclusions. The facial remodeling review assessed risk of bias with JBI tools and still concluded that all four observational studies reported favorable within-group findings with heterogeneous populations, which is exactly the honest framing a risk of bias table should force [15]. - **Expecting the table to prove accuracy.** Low risk of bias does not mean the result is true. It means the study was designed and conducted so that systematic error is unlikely. Applicability, precision, and consistency are separate dimensions. ## Frequently Asked Questions ### What is a risk of bias table in a systematic review? It is a structured table with one row per included study and one column per bias domain, recording a judgment of low, high, or unclear risk for each domain plus the reasoning. It documents how much systematic error could distort each study's results and feeds directly into certainty grading and sensitivity analyses. ### Which risk of bias tool should I use for randomized controlled trials? Use the Cochrane RoB 2 tool, which judges five domains, randomization, deviations from intended interventions, missing outcome data, outcome measurement, and selective reporting, on a scale of low risk, some concerns, and high risk. Recent reviews of randomized trials, including a network meta-analysis of auricular therapy for smoking cessation, applied this approach [7]. ### Can I use QUADAS-2 for prediction model studies? QUADAS-2 is built for diagnostic accuracy studies that compare an index test against a reference standard in a two by two table. For prediction model studies, use PROBAST, which assesses bias across participants, predictors, outcome, and analysis domains and adds separate applicability judgments [13][4]. For AI-based prediction models, PROBAST-AI extends the framework. ### How many reviewers need to assess risk of bias? Two reviewers working independently is the accepted standard, with a third reviewer or consensus process resolving disagreements. Published reviews follow this pattern, including duplicate blinded assessment [10] and two-reviewer assessment with a third reviewer adjudicating [16]. Single-reviewer assessment should be declared as a limitation if unavoidable. ### What do low, some concerns, and high risk mean in RoB 2? Low risk means the bias domain is unlikely to materially alter the results. Some concerns means the domain could shift the result but the evidence leaves doubt, and high risk means bias or missing information makes the result untrustworthy. The overall judgment reflects the most concerning domain, and any high risk domain makes the overall judgment high risk. ### How do I present the risk of bias table in my manuscript? Place a summary matrix, one row per study, one column per domain, in the results or a supplement, add a traffic light plot and a per-domain bar plot as figures, and write a short narrative naming the weakest domains. Tools such as the robvis R package generate the standard plot formats from your spreadsheet data [12]. ### Do I need to assess risk of bias for every included study? Yes, every study that contributes to your synthesis gets a completed assessment. If you include studies only in a narrative synthesis, for example nonrandomized evidence retained separately [3] or descriptive studies in a mixed review [15], they still receive an appraisal with the tool appropriate to their design, and the judgment informs how much weight their findings carry. ### How does risk of bias relate to GRADE? Risk of bias is one of the five GRADE domains, alongside inconsistency, indirectness, imprecision, and publication bias, used to downgrade certainty in a body of evidence. Reviews routinely downgrade certainty for outcomes supported mainly by high risk studies, as demonstrated in reviews pairing JBI or QUADAS-2 assessments with GRADE ratings [2][16][3]. ## Sources 1. [Peripherally acting μ-opioid receptor antagonists for recovery of bowel function after radical cystectomy: a systematic review and meta-analysis.](https://pubmed.ncbi.nlm.nih.gov/42866437/) 2. [Surgical Treatment and Botulinum Toxin Type A for the Management of Facial Synkinesis: A Systematic Review.](https://pubmed.ncbi.nlm.nih.gov/42995268/) 3. [Sensory-based non-pharmacological interventions for reducing dental anxiety and improving behavioural cooperation in paediatric patients: a systematic review with GRADE assessment.](https://pubmed.ncbi.nlm.nih.gov/42952639/) 4. [Limited External Validation and Uncertain Clinical Utility of Prediction Models for Bacteriologically Negative Pulmonary Tuberculosis: A Systematic Review.](https://pubmed.ncbi.nlm.nih.gov/42948785/) 5. [Too good to be true? AI performance in CT-based urinary stone detection and classification: EAU endourology systematic review and diagnostic accuracy meta-analysis.](https://pubmed.ncbi.nlm.nih.gov/42986462/) 6. [Microneedling Radiofrequency Versus Fractional Laser for the Treatment of Acne Scars: A Systematic Review with Descriptive Quantitative Synthesis.](https://pubmed.ncbi.nlm.nih.gov/42834161/) 7. [Efficacy of auricular therapy for smoking cessation: A systematic review and network meta-analysis.](https://pubmed.ncbi.nlm.nih.gov/42964821/) 8. [Diagnostic accuracy of ultrasound-based imaging modalities for assessing residual axillary lymph node disease after neoadjuvant chemotherapy in patients with breast cancer: a systematic review and meta-analysis.](https://pubmed.ncbi.nlm.nih.gov/42997962/) 9. [Diagnostic accuracy of point-of-care ultrasound for proximal lower-extremity deep vein thrombosis in emergency settings: A systematic review and meta-analysis.](https://pubmed.ncbi.nlm.nih.gov/42961238/) 10. [Detection of Cytomegalovirus in Chorionic Villi for the Prenatal Diagnosis of Congenital Infection: A Systematic Review of Diagnostic Test Accuracy.](https://pubmed.ncbi.nlm.nih.gov/42942467/) 11. [Diagnostic Accuracy of Artificial Intelligence-Based Electrocardiography for the Detection of Heart Diseases: A Systematic Review and Meta-Analysis.](https://pubmed.ncbi.nlm.nih.gov/42793799/) 12. [Accuracy of Deep Learning in Detecting Cerebral Microbleeds: Systematic Review and Meta-Analysis.](https://pubmed.ncbi.nlm.nih.gov/42767630/) 13. [Lymphatic filariasis risk prediction models: A systematic review of modelling approaches, predictors, validation practices, and methodological quality.](https://pubmed.ncbi.nlm.nih.gov/43008045/) 14. [AI Decision Support for Emergency Triage of Large Vessel Occlusion Using Noncontrast Computed Tomography: Systematic Review and Bayesian Diagnostic Test Accuracy Network Meta-Analysis.](https://pubmed.ncbi.nlm.nih.gov/42785737/) 15. [Beyond Weight Loss: Facial Remodeling Associated with GLP-1 Receptor Agonists-A Systematic Review and Therapeutic Framework.](https://pubmed.ncbi.nlm.nih.gov/42864543/) 16. [Prognostic accuracy of the maternal serum IGFBP4/SHBG ratio for spontaneous preterm birth in asymptomatic pregnancies: a systematic review.](https://pubmed.ncbi.nlm.nih.gov/42971281/) 17. [Comparative efficacy and acceptability of antidepressants in the long-term treatment of major depressive disorder: a systematic review and network meta-analysis.](https://pubmed.ncbi.nlm.nih.gov/43008763/) 18. [The Transumbilical Breast Augmentation: Safety and Satisfaction in a Systematic Review and Meta-analysis of 13,210 Breasts.](https://pubmed.ncbi.nlm.nih.gov/42993310/) 19. [Enhancing Cancer Immunotherapy with Berberine and Its Loaded Nanoparticles: A Systematic Review.](https://pubmed.ncbi.nlm.nih.gov/42852751/)