JBI Critical Appraisal Tools vs. CASP

By Dr. Zubair Khalid, DVM, MS, PhD · 2026-08-27

Quick Answer

• Choose JBI tools when your review includes diverse study designs and you need design-specific checklists with detailed appraisal guidance.
• Choose CASP when you need a rapid, accessible appraisal for qualitative research, randomized controlled trials, or systematic reviews with limited time.
• Both tools assess study quality, but neither replaces transparent reporting guidelines or ethical oversight from your institution.

Understanding Critical Appraisal Tools

Critical appraisal tools help reviewers evaluate the methodological quality of studies included in a literature review. These instruments provide structured questions that guide your assessment of whether a study was designed, conducted, and reported well enough to support its conclusions. For students, researchers, and laboratory staff preparing systematic reviews or evidence syntheses, selecting the right appraisal tool affects the credibility of your findings and the defensibility of your conclusions.

The Joanna Briggs Institute (JBI) and the Critical Appraisal Skills Programme (CASP) are two widely used families of appraisal checklists. Both originated to support evidence-based practice, but they differ in scope, structure, and intended use. Understanding these differences helps you match the tool to your review design and study types.

Your choice of appraisal tool should align with the types of studies you plan to include. A review of qualitative studies has different quality concerns than a review of randomized trials or diagnostic accuracy studies. Using a tool designed for a different study design can lead you to ask irrelevant questions or miss important sources of bias.

At a Glance

Core Principles of Critical Appraisal

Critical appraisal rests on the principle that not all published studies are equally trustworthy. Flaws in study design, conduct, or analysis can distort results and lead reviewers to incorrect conclusions. Appraisal tools operationalize this principle by asking standardized questions about key methodological features.

The EQUATOR Network maintains a comprehensive collection of reporting guidelines that help authors write transparent research reports. These guidelines, such as CONSORT for randomized trials and PRISMA for systematic reviews, complement appraisal tools by focusing on completeness of reporting instead of methodological conduct. When you appraise a study, you assess what the authors did, while reporting guidelines help you see whether they described their methods fully enough for you to judge.

Research Methods Resources from the National Library of Medicine provides access to authoritative biomedical books and research method references. These resources can help you understand the methodological concepts behind appraisal questions, such as blinding, allocation concealment, and handling of missing data.

JBI Critical Appraisal Tools

The Joanna Briggs Institute offers a suite of critical appraisal checklists designed for systematic reviews. Each checklist corresponds to a specific study design, allowing reviewers to select the tool that matches the studies they are evaluating.

Available JBI Checklists

JBI provides checklists for a broad range of study types. These include checklists for randomized controlled trials, quasi-experimental studies, cohort studies, case-control studies, analytical cross-sectional studies, case series, case reports, diagnostic test accuracy studies, economic evaluations, qualitative research, text and opinion papers, and prevalence studies. This breadth makes JBI tools particularly useful for reviews that include multiple study designs.

Each JBI checklist contains questions specific to the methodological features that matter for that design. For example, the randomized controlled trial checklist asks about randomization, allocation concealment, blinding, and completeness of follow-up. The qualitative checklist asks about congruity between philosophical perspective and methodology, researcher reflexivity, and representation of participant voices.

Using JBI Tools in Practice

When you use a JBI checklist, you answer each question with yes, no, unclear, or not applicable. The JBI approach encourages reviewers to record their answers and use them to inform decisions about including studies in a review. Studies with significant methodological flaws may be excluded or given less weight in the synthesis.

JBI tools are designed to work within the JBI systematic review methodology. If you are conducting a JBI-style review, the appraisal results feed directly into your review protocol and synthesis decisions. The tools also work independently for reviewers who want a structured approach to quality assessment without adopting the full JBI methodology.

CASP Checklists

The Critical Appraisal Skills Programme provides checklists that are widely used in health research education and practice. CASP checklists are designed to be accessible to reviewers with varying levels of methodological expertise.

Available CASP Checklists

CASP offers checklists for qualitative research, randomized controlled trials, systematic reviews, cohort studies, case-control studies, diagnostic test accuracy studies, economic evaluations, and clinical prediction rules. Each checklist includes a small number of screening questions followed by detailed questions about validity, results, and applicability.

The CASP format uses three response options: yes, no, and cannot tell. Each question includes prompts that help you think about what to look for in the study report. The final question in each checklist asks you to consider whether the results are applicable to your local context or population.

Using CASP Tools in Practice

CASP checklists are well suited for educational settings and for reviewers who need a quick but structured approach to appraisal. The prompts embedded in each question help novice reviewers understand what to look for, making CASP a common choice for student projects and introductory research methods courses.

CASP does not prescribe a specific scoring system. Reviewers typically use the checklists to guide their thinking and record their judgments, then make an overall decision about study quality. This flexibility is useful when you need to adapt your appraisal approach to the specific questions of your review.

Comparing JBI and CASP Across Key Domains

Study Design Coverage

JBI provides more checklists covering a wider range of study designs than CASP. If your review includes case series, case reports, text and opinion papers, or prevalence studies, JBI offers a specific checklist for each. CASP does not provide checklists for these designs, so you would need to adapt another tool or use a generic appraisal approach.

For common designs such as randomized controlled trials, cohort studies, and qualitative research, both JBI and CASP offer appropriate checklists. The choice between them depends on your preference for question format and the level of detail you need.

Question Format and Detail

JBI checklists use a yes, no, unclear, or not applicable format. This categorical approach allows you to record your judgment clearly and compare assessments across studies. JBI also provides accompanying guidance documents that explain the rationale behind each question.

CASP checklists use a yes, no, or cannot tell format with embedded prompts. The prompts are designed to help you identify what to look for in the study report. This format is more instructive for novice reviewers but may feel less precise for experienced reviewers who want to record nuanced judgments.

Integration with Review Methodology

JBI tools are integrated into a complete systematic review methodology. If you follow JBI guidance for your review, the appraisal tools fit naturally into your workflow. JBI also provides software and training to support reviewers using their approach.

CASP tools are standalone checklists that you can use within any review methodology. They do not require you to adopt a specific approach to systematic reviewing. This flexibility makes CASP useful for reviews conducted outside a specific methodological framework.

Time and Training Requirements

CASP checklists are generally quicker to complete than JBI checklists. The prompts help you move through the questions efficiently, and the three-response format reduces the time needed to record judgments. CASP is therefore a practical choice when you have limited time or are appraising a large number of studies.

JBI checklists require more time because they include more questions and you may need to consult the accompanying guidance to answer accurately. The additional time investment can improve the thoroughness of your appraisal, particularly for complex study designs.

Decision Table for Tool Selection

Practical Workflow for Choosing and Using an Appraisal Tool

Step 1: Define Your Review Question and Study Designs

Before selecting an appraisal tool, clarify what types of studies your review will include. List the study designs you expect to encounter based on your preliminary literature search. This list determines which checklists you need.

If your review question is broad and you expect to include multiple study designs, JBI offers the advantage of design-specific checklists for each type. If your review focuses on a single design such as qualitative studies or randomized trials, either tool works well.

Step 2: Match Tools to Your Study Designs

Review the available checklists from JBI and CASP and match them to your expected study designs. For each design, identify whether both tools offer a checklist or whether only one does. This comparison reveals whether you need to use JBI for certain designs and CASP for others.

Using different appraisal tools for different study designs within the same review is acceptable, but you should document your choice and rationale in your review protocol. Consistency in your approach helps readers understand your quality assessment.

Step 3: Pilot the Checklists on Sample Studies

Select two or three studies from your review and complete the appraisal using your chosen tool. This pilot test reveals whether the questions are clear, whether you can answer them from the information in the study reports, and how long the process takes.

If you find that the checklist questions do not fit your studies well, consider whether a different tool would work better. Piloting before you commit to a tool saves time and improves the quality of your appraisal.

Step 4: Document Your Appraisal Results

Record your appraisal answers for each study in a table or spreadsheet. Include the study identifier, your answer to each question, and any notes about concerns you identified. This documentation supports transparency and allows readers to verify your quality assessment.

Your appraisal results should inform your synthesis. Studies with serious methodological flaws may be excluded from the review or given less weight in your conclusions. Document how you used the appraisal results to make these decisions.

Step 5: Report Your Appraisal Approach

Describe your appraisal tool selection and process in your review methods section. State which tool you used for each study design and how you handled studies that did not fit the checklist. This transparency allows readers to assess the credibility of your quality assessment.

The EQUATOR Network provides reporting guidelines that can help you structure your methods section. Following an appropriate reporting guideline ensures that you describe your appraisal approach completely and transparently.

Records and Measurements for Appraisal

Maintain clear records of your appraisal process to support the credibility of your review. Your records should include the version of the checklist you used, the date of appraisal, and your responses to each question for every study.

A useful record format includes columns for study identifier, checklist used, response to each question, overall quality judgment, and notes. This format allows you to compare studies and identify patterns in methodological quality across your included studies.

Measure inter-rater reliability if multiple reviewers are appraising studies. Having two reviewers independently appraise a subset of studies and compare their results helps identify disagreements and ensures consistency. Disagreements should be resolved through discussion or consultation with a third reviewer.

Common Failure Patterns in Critical Appraisal

Using the Wrong Checklist for the Study Design

A common error is using a checklist designed for one study design to appraise a different design. For example, using a randomized controlled trial checklist to appraise a cohort study leads you to ask questions that do not apply and miss design-specific concerns. Always match the checklist to the actual study design.

Treating Appraisal as a Simple Score

Some reviewers sum the yes answers and treat the total as a quality score. This approach oversimplifies appraisal because not all questions carry equal weight and some flaws are more serious than others. Use your judgment to interpret the pattern of responses instead of relying on a numeric score.

Failing to Document Appraisal Decisions

Reviewers sometimes complete appraisal checklists but do not record how the results influenced their review decisions. Without documentation, readers cannot see how you handled studies with methodological concerns. Record your decisions about inclusion, exclusion, and weighting in your review.

Ignoring Reporting Quality

Appraisal tools assess methodological conduct, but you can only judge what authors report. Poor reporting can make a well-conducted study appear flawed, and incomplete reporting may hide methodological problems. Use reporting guidelines from the EQUATOR Network to assess whether studies described their methods fully enough for appraisal.

Limitations of Critical Appraisal Tools

Critical appraisal tools have inherent limitations that you should acknowledge in your review. The tools assess what is reported, not necessarily what was done. Authors may omit details of their methods even when they followed good practices, leading you to underestimate study quality.

Appraisal tools also cannot capture all aspects of study quality. Some methodological concerns are specific to a particular research question or context and may not be addressed by generic checklist questions. Use your professional judgment alongside the checklist to identify concerns that the tool does not cover.

The response options in appraisal checklists require you to make categorical judgments about complex methodological features. A study may partially meet a criterion, but the checklist forces you to choose yes, no, or unclear. Record nuances in your notes to preserve important detail.

Welfare and Safety Context for Appraisal

When your review includes studies involving animals or human participants, appraisal should consider ethical conduct alongside methodological quality. The Committee on Publication Ethics core practices address authorship, peer review, data handling, conflicts of interest, and misconduct. These practices provide a framework for assessing whether a study was conducted and reported ethically.

For animal studies, appraisal should consider whether the research followed appropriate welfare standards and whether the authors reported ethical approval and compliance with relevant regulations. For human studies, appraisal should consider informed consent, privacy protection, and ethical oversight.

The National Institutes of Health provides guidance on responsible conduct of research through its grants and funding resources. Researchers planning studies should consult this guidance to understand expectations for ethical conduct and reporting.

Professional Escalation Criteria

If you identify serious methodological or ethical concerns during appraisal, escalate the issue according to your institutional policies. Concerns that warrant escalation include evidence of data fabrication, plagiarism, undisclosed conflicts of interest, or ethical violations in the conduct of research.

The Committee on Publication Ethics core practices describe expectations for handling allegations of misconduct. If you suspect misconduct in a study you are appraising, consult your institution's research integrity office or follow the journal's policies for raising concerns.

For studies funded by the National Institutes of Health, the NIH Grants and Funding resources describe expectations for grantee conduct and reporting. Researchers who identify concerns about NIH-funded studies should consider reporting them through appropriate channels.

Researcher Identity and Record Keeping

Maintaining accurate researcher identity records supports the integrity of your review work. ORCID provides a persistent identifier that distinguishes you from other researchers and links your publications and review activities to your profile. Registering for an ORCID identifier and keeping your record current helps ensure your contributions are properly attributed.

When you conduct critical appraisal for a review, your appraisal records become part of your research documentation. Preserve these records according to your institution's data management policies. The NIH Data Management and Sharing Policy describes expectations for planning and sharing research data, which may apply to your review work if it is NIH-funded.

Data Management for Review Records

Plan how you will manage the data generated during your critical appraisal process. Your appraisal records, including completed checklists and decision logs, should be stored securely and organized for easy retrieval. Consider whether your data management plan addresses the retention and sharing of these records.

The NIH Data Management and Sharing Policy provides guidance on developing data management plans that address data types, formats, preservation, and access. Even if your review is not NIH-funded, following these principles supports good research practice and prepares you for future funding applications.

Frequently Asked Questions

What is the main difference between JBI and CASP tools?

JBI provides a broader range of design-specific checklists integrated into a complete systematic review methodology. CASP offers fewer checklists with a more accessible format that includes prompts to guide reviewers. Your choice depends on the study designs in your review and your preferred level of detail.

Can I use both JBI and CASP tools in the same review?

Yes, you can use different tools for different study designs within the same review. Document your choice and rationale for each tool in your review protocol. Consistency in your approach helps readers understand your quality assessment.

Which tool is better for a student learning critical appraisal?

CASP checklists are often easier for students because the embedded prompts explain what to look for in each question. The three-response format is straightforward and the checklists are shorter than JBI tools. As you gain experience, you may prefer the more detailed JBI approach.

Do JBI and CASP tools produce the same quality judgments?

The tools may produce different judgments for the same study because they ask different questions and use different response formats. JBI tools tend to be more detailed and may identify concerns that CASP does not address. Your overall quality judgment should consider the pattern of responses instead of a single score.

How do I choose the right checklist for my study design?

Identify the specific design of each study you are appraising, then select the checklist that matches that design. JBI provides checklists for more designs than CASP, so you may need JBI for less common designs such as case series or text and opinion papers.

Do I need to report which appraisal tool I used?

Yes, you should describe your appraisal tool selection and process in your review methods section. State which tool you used for each study design and how you handled studies that did not fit the checklist. This transparency allows readers to assess the credibility of your quality assessment.

Can I modify an appraisal checklist to fit my review?

You can adapt checklists to your review context, but you should document any modifications and explain your rationale. Substantial modifications may affect the validity of the tool, so consider whether an alternative tool would fit better without changes.

How do appraisal tools relate to reporting guidelines?

Appraisal tools assess methodological quality, while reporting guidelines such as those in the EQUATOR Network focus on completeness of reporting. Both are important for evaluating studies. Use reporting guidelines to assess whether authors described their methods fully enough for appraisal.

Related Articles

• Understanding Study Designs in Research: A Comparative Overview
• Bioinformatics vs. Computational Biology: What's the Difference (and Which Should You Choose)?
• Factorial Designs: When and How to Use Them
• Epidemiological Study Designs: An Overview for Researchers
• Epidemiology Study Designs: A Guide to Selection and Application

References and Further Reading

• Research Methods Resources. National Library of Medicine.
• EQUATOR Network. EQUATOR Network.
• Core Practices. Committee on Publication Ethics.
• NIH Grants and Funding. National Institutes of Health.
• ORCID for Researchers. ORCID.
• Data Management and Sharing Policy. National Institutes of Health.
• NCBI Literature Resources. National Center for Biotechnology Information.
• PubMed. National Library of Medicine.

This article is educational and does not replace institutional policy, professional advice, or applicable safety and regulatory requirements.