How to Use the CASP Checklist for Critical Appraisal in a Systematic Review

By Dr. Zubair Khalid, DVM, MS, PhD · 2026-08-27

Quick Answer

• Use the CASP checklists as a structured screening tool to assess the trustworthiness, relevance, and results of individual studies before deciding whether they merit inclusion in your systematic review.
• Work through each checklist item systematically, recording your answers and reasoning in a review matrix so that your appraisal decisions are transparent and reproducible.
• The CASP checklists are screening aids, not scoring instruments, and they do not replace the need for a prespecified protocol, dual reviewer assessment, or reporting guidelines like those indexed by the EQUATOR Network.

Understanding the Role of Critical Appraisal in a Systematic Review

A systematic review is built on a foundation of methodical searching, transparent selection, and rigorous evaluation of the evidence you find. Critical appraisal sits at the center of that process. It is the step where you judge whether the studies you have located are trustworthy enough to answer your research question and whether their findings should influence your conclusions.

The Critical Appraisal Skills Programme, commonly called CASP, provides a family of checklists designed for this purpose. Each checklist is tailored to a specific study design, such as randomized controlled trials, cohort studies, case control studies, diagnostic test accuracy studies, qualitative research, and systematic reviews themselves. The checklists guide you through a series of questions that help you assess methodological quality, the risk of bias, and the applicability of a study to your own context.

For researchers working in the life sciences, including those studying animal health, laboratory methods, or agricultural systems, the CASP checklists offer a practical entry point into critical appraisal. They are widely taught in university courses and used in research teams because they are short, accessible, and structured around the core questions that matter when judging evidence.

The purpose of this article is to give you a practical, step-by-step approach to applying the CASP checklists within a systematic review workflow. You will learn how to prepare for appraisal, how to work through each checklist section, how to record your judgments, and how to avoid common mistakes that undermine the credibility of a review.

At a Glance: CASP Checklist Application Overview

The table below summarizes the key decisions you will make when integrating CASP checklists into your systematic review process.

Preparing Your Appraisal Framework Before You Start

Critical appraisal cannot be improvised at the point of data extraction. You need a clear framework in place before you begin screening search results. This preparation protects your review from bias and ensures that your decisions are consistent across all studies.

Define Your Research Question and Eligible Study Designs

Your research question determines which study designs are eligible for inclusion. A question about the effectiveness of an intervention, such as a new vaccine protocol in poultry, points toward randomized controlled trials. A question about risk factors for a disease outbreak in dairy herds points toward cohort or case control studies. A question about how farmers perceive biosecurity measures points toward qualitative research.

Write down your inclusion criteria in your review protocol. Specify the study designs you will accept and match each design to the corresponding CASP checklist. This step prevents you from applying the wrong checklist to a study and producing misleading appraisal results.

Select the Correct CASP Checklist for Each Study Design

CASP publishes separate checklists for different study types. Using the correct checklist matters because each one asks questions that are specific to the strengths and weaknesses of that design. A randomized controlled trial checklist focuses on randomization, blinding, and follow up. A qualitative checklist focuses on the appropriateness of the research design, data collection methods, and the relationship between researcher and participants.

If you are reviewing a body of literature that mixes study designs, you will need multiple checklists. Keep a reference document that maps each eligible design to its checklist. This mapping becomes part of your review protocol and should be recorded before you begin appraisal.

Pilot the Checklist on a Small Sample of Studies

Before you appraise your full set of included studies, test the checklist on two or three representative papers. This pilot run helps you identify ambiguous questions, clarify your interpretation of each item, and calibrate your judgments with any co reviewers. It also reveals whether the checklist is practical for your specific body of literature or whether you need to supplement it with additional design specific questions.

The pilot phase is also the time to decide how you will handle studies that do not clearly fit a single checklist. Some papers report mixed methods or multiple studies within one article. In those cases, you may need to appraise each component separately or choose the checklist that matches the primary analysis that answers your review question.

Working Through the CASP Checklist Section by Section

Each CASP checklist is organized into three broad sections. The first section establishes whether the study is valid and worth continuing with. The second section examines the results in detail. The third section asks whether the results are applicable to your local context. Understanding this structure helps you move efficiently through the appraisal.

Section One: Screening Questions for Study Validity

The opening questions on every CASP checklist are screening questions. They are designed to give you a quick indication of whether the study is methodologically sound enough to warrant a full appraisal. If a study fails these initial questions, you may decide to exclude it from your review or to note serious limitations in your appraisal record.

For a randomized controlled trial, the screening questions ask whether the study addressed a clearly focused issue and whether the assignment of participants to interventions was randomized. For a cohort study, they ask whether the cohort was recruited in an acceptable way and whether the exposure was accurately measured to minimize bias. For a qualitative study, they ask whether there was a clear statement of research aims and whether a qualitative methodology was appropriate.

These screening questions are not meant to be answered with a simple yes or no in isolation. You should consider the context of the study, the population studied, and the research question. Record your reasoning alongside each answer so that your judgment is transparent.

Section Two: Detailed Assessment of Results

Once a study passes the screening questions, you move into the detailed assessment. This section asks about the precision, magnitude, and reliability of the results. The specific questions depend on the study design.

In a randomized controlled trial, you will be asked about blinding of participants, personnel, and outcome assessors. You will also be asked whether all participants who entered the trial were accounted for at its conclusion and whether the groups were treated equally apart from the intervention. These questions target the major sources of bias in experimental studies.

In a cohort study, the detailed questions ask about confounding factors, follow up completeness, and the precision of the results. You will consider whether the study identified and adjusted for important confounders and whether the length of follow up was sufficient for outcomes to occur.

In a diagnostic accuracy study, the questions focus on whether the reference standard was appropriate, whether the test results were interpreted without knowledge of the reference standard, and whether the same reference standard was applied to all participants.

Work through each question methodically. Do not skip items because a study appears strong overall. A single serious flaw in one area can undermine the credibility of the entire study, even if other aspects are well conducted.

Section Three: Applicability to Your Local Context

The final section of each CASP checklist asks whether the results can be applied to your local population or setting. This is where you judge the relevance of the study to your review question and to the context in which you will use the findings.

Consider whether the study participants are similar enough to the population you are interested in. Consider whether the intervention or exposure described in the study is feasible in your setting. Consider whether the outcomes measured are the ones that matter to your decision making.

This section is particularly important for life science research, where findings from one species, production system, or geographic region may not transfer to another. A study on antibiotic alternatives in broiler chickens raised in a temperate climate may have limited applicability to a free range system in a tropical region. Record your assessment of applicability clearly in your appraisal notes.

Building Your Appraisal Matrix and Recording Decisions

A systematic review requires transparency. Every decision you make during appraisal must be recorded in a way that another researcher could follow. An appraisal matrix is the practical tool for this purpose.

Design a Matrix That Captures Checklist Responses

Create a table with one row for each included study and one column for each checklist item. Enter your response for each item, using a consistent notation such as yes, no, unclear, or not applicable. Add a column for your overall judgment about the study quality and a column for your reasoning.

Your matrix should also include basic study identifiers, such as the first author, publication year, and study design. This makes it easy to cross reference your appraisal results with your data extraction tables and your synthesis.

Record Reasoning, beyond Answers

A matrix that only contains yes or no answers is of limited value. The reasoning behind each judgment is what allows other researchers to understand your decisions and to assess whether your appraisal was fair and consistent.

For each checklist item, write a brief note explaining your judgment. For example, if you answered no to a question about blinding in a randomized controlled trial, note whether blinding was impossible due to the nature of the intervention or whether the authors simply failed to report it. This distinction matters for your overall quality judgment.

Use the Matrix to Inform Your Synthesis

Your appraisal matrix is not an end in itself. It feeds directly into your synthesis. Studies that are judged to be at high risk of bias may be given less weight in your conclusions. You may also choose to conduct sensitivity analyses that exclude high risk studies to see whether your findings change.

The matrix also helps you identify patterns across the literature. If all studies in a particular area fail the same checklist item, that is an important finding for your review. It tells you where the evidence base is weak and where future research is needed.

Common Failure Patterns in CASP Appraisal

Researchers applying CASP checklists tend to make the same types of mistakes. Recognizing these patterns in your own work can help you avoid them.

Treating the Checklist as a Scoring System

The CASP checklists are not designed to produce a numerical score. There is no official threshold that separates high quality from low quality studies. Some research teams create their own scoring systems, but this practice is not part of the CASP approach and can introduce false precision.

Instead of scoring, use the checklist to develop a qualitative judgment about each study. Describe the strengths and limitations in your own words and classify the study as having low, moderate, or high risk of bias based on your overall assessment.

Applying the Wrong Checklist to a Study Design

Using a randomized controlled trial checklist to appraise a cohort study produces meaningless results. The questions do not match the design, and you will end up penalizing the study for features that are not relevant to its design.

Always confirm the study design before selecting a checklist. Read the methods section carefully to determine whether the study is truly a randomized controlled trial, a cohort study, a case control study, or another design. Some papers are poorly labeled, and the actual design may differ from what the authors claim.

Appraising Without a Second Reviewer

Critical appraisal involves judgment, and judgment is subject to personal bias. A single reviewer may be overly lenient or overly strict in their assessments. The risk of bias in the appraisal process itself is reduced when two reviewers independently appraise each study and then compare their results.

If you are working alone, consider asking a colleague to independently appraise a sample of your studies. Compare your judgments and discuss any disagreements. This process improves the reliability of your appraisal and strengthens the credibility of your review.

Failing to Distinguish Reporting Quality from Methodological Quality

A study can be well conducted but poorly reported. Conversely, a study can be well reported but methodologically flawed. The CASP checklists ask you to judge what the authors actually did, beyond what they wrote.

When a checklist item cannot be answered because the authors did not report the relevant information, record it as unclear instead of assuming the worst or the best. In your reasoning, note whether the missing information is likely to reflect poor reporting or poor methodology.

Practical Implementation Steps for Your Review Team

Integrating CASP checklists into your systematic review requires planning and coordination. The steps below outline a practical workflow for a research team.

Step One: Agree on the Appraisal Protocol

Before screening begins, hold a team meeting to agree on the appraisal protocol. Decide which checklists will be used for each study design, how disagreements will be resolved, and how appraisal results will be recorded. Write these decisions into your review protocol.

Step Two: Train All Reviewers on the Checklists

Every person who will perform appraisal needs training on the CASP checklists. Provide examples of studies that illustrate different quality levels and work through the checklists together. This training ensures that all reviewers interpret the questions consistently.

Step Three: Conduct Independent Appraisal in Duplicate

Each included study should be appraised independently by two reviewers. The reviewers should not discuss their judgments before recording them. This independence prevents one reviewer from influencing the other.

Step Four: Resolve Disagreements Through Discussion

After both reviewers have recorded their judgments, compare the results. Discuss any disagreements and try to reach consensus. If consensus cannot be reached, involve a third reviewer or the full team to make a final decision.

Step Five: Integrate Appraisal Results into the Synthesis

Use the completed appraisal matrix to inform your synthesis. Consider whether to exclude high risk studies, conduct sensitivity analyses, or downgrade the certainty of evidence based on methodological limitations.

Records and Measurements for a Transparent Appraisal

The records you keep during appraisal are part of the audit trail of your systematic review. They demonstrate that your review was conducted systematically and that your conclusions are based on a transparent assessment of the evidence.

Maintain a Full Audit Trail

Keep all versions of your appraisal matrix, including the independent judgments of each reviewer and the final consensus judgments. Record the date of each appraisal and the names of the reviewers. This audit trail is important if your review is submitted for publication or used to inform policy decisions.

Document Checklist Version and Date

CASP checklists are periodically updated. Record which version of each checklist you used and the date of that version. This information allows readers to understand exactly what questions were asked and ensures that your appraisal can be replicated.

Link Appraisal Records to Data Extraction

Your appraisal matrix should be linked to your data extraction tables so that you can trace the quality judgment for each study back to the specific data that informed your synthesis. This linkage is particularly important when you are deciding how much weight to give to individual studies.

Limitations of the CASP Approach

The CASP checklists are useful tools, but they have limitations that you should understand before relying on them exclusively.

Checklists Do Not Capture All Sources of Bias

The CASP checklists focus on the major sources of bias for each study design, but they do not capture every possible threat to validity. Some biases are specific to particular research questions or settings and may not be addressed by the standard checklist items.

Consider supplementing the CASP checklists with additional questions that are relevant to your specific review topic. For example, if you are reviewing studies on animal welfare outcomes, you may need to ask about the validity and reliability of the welfare measures used.

Appraisal Is Only One Component of Evidence Synthesis

Critical appraisal tells you about the quality of individual studies. It does not tell you about the overall certainty of the evidence across a body of research. For that, you need a framework such as GRADE, which considers additional factors including consistency, directness, and publication bias.

Use the CASP checklists as one input into your broader evidence synthesis process. Do not treat them as a complete system for judging the certainty of evidence.

The Checklists Require Interpretation

The CASP checklist questions are not self executing. They require interpretation and judgment. Two experienced reviewers may legitimately disagree about whether a study meets a particular criterion. This is why dual independent appraisal and consensus discussion are essential parts of the process.

Welfare and Safety Context in Life Science Reviews

When your systematic review involves animal studies, laboratory research, or agricultural interventions, you have additional responsibilities beyond methodological appraisal.

Consider Animal Welfare Standards in Your Appraisal

Studies involving animals should have been conducted in accordance with ethical standards for animal care and use. While the CASP checklists do not explicitly ask about animal welfare, you should consider whether the study reports ethical approval and whether the methods appear consistent with accepted welfare standards.

If a study raises serious welfare concerns, this should be noted in your appraisal and considered in your overall judgment about whether the study should inform your conclusions. The Committee on Publication Ethics core practices provide context on the ethical expectations for research conduct and reporting.

Consider Research Integrity and Publication Ethics

Your appraisal should also consider whether the study appears to have been conducted and reported honestly. Look for signs of selective reporting, undisclosed conflicts of interest, or other practices that undermine the integrity of the research. The Committee on Publication Ethics core practices outline expectations for authorship, data handling, and conflict disclosure that apply across the life sciences.

Align Your Review with Reporting Guidelines

When you write up your systematic review, you should follow a reporting guideline that is appropriate for your review type. The EQUATOR Network maintains a comprehensive database of reporting guidelines and can help you select the right one for your work. Following a reporting guideline ensures that your review is reported transparently and completely.

Professional Escalation Criteria in the Appraisal Process

There are situations where you should escalate concerns beyond your immediate review team. Knowing when to escalate protects the integrity of your review and the safety of any decisions based on it.

Escalate When You Suspect Research Misconduct

If your appraisal reveals evidence of fabricated data, duplicated publication, or other forms of research misconduct, you should escalate the concern to the appropriate authority. This may include the journal that published the study, your institutional research integrity office, or a funding agency. The Committee on Publication Ethics core practices provide guidance on handling allegations of misconduct.

Escalate When You Identify Serious Safety Concerns

If a study reports findings that raise serious safety concerns, such as an intervention that caused unexpected harm to animals or humans, you should escalate these concerns promptly. This may involve notifying your institutional animal care and use committee, a regulatory body, or the relevant funding agency.

Escalate When You Lack the Expertise to Judge a Study

Some studies may involve specialized methods or statistical techniques that are outside your expertise. If you cannot confidently appraise a study, you should seek input from a colleague with the relevant expertise instead of making a judgment you cannot defend. This is particularly important for complex statistical analyses or specialized laboratory methods.

Integrating CASP Appraisal with Your Broader Review Workflow

The CASP checklists fit into a larger systematic review workflow that includes protocol development, literature searching, screening, data extraction, and synthesis. Understanding where appraisal sits in this workflow helps you plan your time and resources.

Position Appraisal After Full Text Screening

Appraisal should occur after you have screened your search results and selected the full text articles that meet your inclusion criteria. Appraising studies that you will ultimately exclude for other reasons wastes time and resources.

Connect Appraisal to Your Data Extraction Process

Data extraction and appraisal are closely linked. You will often extract information about study methods and results at the same time as you appraise methodological quality. Keeping these processes coordinated ensures that you have the information you need to make informed quality judgments.

Use Appraisal Results to Guide Your Synthesis Decisions

The results of your appraisal should directly inform your synthesis. Studies at high risk of bias may be given less weight, excluded in sensitivity analyses, or used only to explore hypotheses instead of to draw firm conclusions. Document these decisions in your review protocol and report them transparently in your final review.

Frequently Asked Questions

How do I choose the right CASP checklist for a study?

Read the methods section of the study carefully to determine the actual study design. Match that design to the corresponding CASP checklist. If a study uses mixed methods, identify the primary design that answers your review question and use the matching checklist.

Can I modify the CASP checklist questions?

You can add supplementary questions that are relevant to your specific review topic, but you should not remove or alter the core CASP questions. If you modify the checklist, document your changes and explain your reasoning in your review protocol.

Should I exclude studies that fail the screening questions?

The decision to exclude a study based on screening questions should be made according to your prespecified protocol. Some review teams exclude studies that fail the initial screening questions, while others include them and note the limitations in their appraisal. Decide before you begin and apply your decision consistently.

How many reviewers should appraise each study?

The standard approach is to have two reviewers independently appraise each study and then resolve disagreements through discussion. This reduces the risk of individual bias and improves the reliability of your judgments.

Do I need to report CASP results in my published review?

Yes. Systematic review reporting guidelines require you to describe your appraisal methods and report the results of your quality assessment. This transparency allows readers to judge the credibility of your conclusions.

Can I use CASP checklists for studies outside the life sciences?

The CASP checklists were developed for health research but are widely used in related fields including veterinary science, agriculture, and environmental health. The core principles of critical appraisal apply across these disciplines, though you may need to adapt the questions to your specific context.

Using the Evidence

Related Articles

• Conducting a Study: A Practical Checklist for Life Science Researchers
• Epidemiological Study Designs: An Overview for Researchers
• Ethical Considerations in Research: A Practical Checklist for Study Design
• Protein Synthesis Step By Step
• Replication Study for Dissertation: A Practical Guide

References and Further Reading

• Research Methods Resources. National Library of Medicine.
• EQUATOR Network. EQUATOR Network.
• Core Practices. Committee on Publication Ethics.
• NIH Grants and Funding. National Institutes of Health.
• ORCID for Researchers. ORCID.
• Data Management and Sharing Policy. National Institutes of Health.
• NCBI Literature Resources. National Center for Biotechnology Information.
• PubMed. National Library of Medicine.

This article is educational and does not replace institutional policy, professional advice, or applicable safety and regulatory requirements.