Setting Inclusion and Exclusion Criteria for a Literature Review

By Dr. Zubair Khalid, DVM, MS, PhD · 2026-08-27

Quick Answer

• Set inclusion and exclusion criteria before searching so you can decide which studies answer your question and which do not.
• Write criteria around study design, population, intervention or exposure, comparators, outcomes, time period, language, and publication type.
• A major limitation is that overly narrow criteria can exclude relevant evidence, while overly broad criteria can make screening unmanageable.

Why Criteria Matter in a Literature Review

A literature review is only as useful as the decisions you make about which studies to include. Without clear inclusion and exclusion criteria, you risk collecting studies that do not answer your question, or missing studies that do. This problem appears across disciplines, from clinical medicine to animal science to laboratory research.

Inclusion criteria define the characteristics a study must have to be considered for your review. Exclusion criteria define the characteristics that disqualify a study. Together, they form a screening framework that you apply consistently to every record you find.

The practical consequence of unclear criteria is wasted effort. You may spend hours reading full texts that you later discard. You may also introduce bias by unconsciously favoring studies that support a particular view. A written, pre-specified set of criteria helps you avoid both problems.

For life-science reviews, the stakes are concrete. A review of antimicrobial use in dairy cattle needs criteria that distinguish between clinical trials, observational studies, and case reports. A review of laboratory animal welfare needs criteria that separate studies using rodents from those using rabbits or non-human primates. The criteria you set determine which evidence enters your synthesis.

At a Glance

Core Principles of Criteria Design

Start With a Focused Research Question

Your criteria should flow from a clearly stated research question. A vague question produces vague criteria. A focused question gives you the variables you need to define.

Consider the difference between these two questions:

• What is known about lameness in dairy cattle?
• What is the effect of foot trimming frequency on lameness incidence in lactating Holstein cows housed in free-stall barns?

The second question identifies the population (lactating Holstein cows in free-stall barns), the intervention (foot trimming frequency), the outcome (lameness incidence), and the setting. Each of these elements becomes a criterion.

The National Library of Medicine provides access to research-methods references that describe how to structure questions and search strategies for biomedical literature [<a href="#ref-1">1</a>]. These same principles apply to animal science and other life-science fields.

Use a Framework to Organize Criteria

A structured framework helps you identify all the elements your criteria should cover. The PICO framework is common in health-related reviews:

• Population or problem
• Intervention or exposure
• Comparator
• Outcome

For reviews that do not involve interventions, you can adapt the framework. A review of prevalence studies might use Population, Exposure, Outcome. A review of diagnostic tests might use Population, Index test, Reference standard, Outcome.

Write your criteria in plain language first. Then translate them into search terms and database filters. This two-step process reduces the chance that you miss an important element.

Distinguish Between Inclusion and Exclusion

Inclusion criteria are the minimum requirements for a study to enter your review. Exclusion criteria are the reasons you remove a study after it has been screened.

Some reviewers use only inclusion criteria and treat exclusion as the inverse. This approach works when the criteria are simple. In practice, you will often need both because some exclusions are not the logical opposite of an inclusion criterion.

For example, you might include studies of dairy cows but exclude studies that report only economic outcomes. The exclusion is not the inverse of the inclusion criterion. It is a separate decision about relevance.

Write both lists before you begin screening. Keep them in a document you can share with collaborators or include in a methods appendix.

Practical Workflow for Setting Criteria

Step 1: Define the Scope of Your Review

Decide what your review will and will not cover. Write a single paragraph that states the population, the exposure or intervention, the outcomes of interest, and the setting. This paragraph becomes the basis for your criteria.

For a review of biosecurity practices in poultry flocks, your scope paragraph might state that you are interested in commercial layer and broiler operations in temperate climates, that you will consider studies of vaccination, sanitation, and visitor control, and that your primary outcomes are mortality and pathogen detection.

Step 2: List Candidate Criteria

Brainstorm every characteristic that might matter for your question. Do not filter yourself at this stage. Write down everything that comes to mind, including study design, sample size, geographic location, species, breed, production system, and publication date.

Then review your list and mark each item as essential, desirable, or unnecessary. Essential criteria become your inclusion list. Desirable criteria may become secondary analyses. Unnecessary criteria are discarded.

Step 3: Test Your Criteria Against Known Studies

Before you finalize your criteria, test them against studies you already know are relevant. Take three to five studies that you believe should be in your review and check whether your criteria would include them.

If a study you consider essential fails your criteria, you have a problem. Either the study is not as relevant as you thought, or your criteria are too narrow. Adjust accordingly.

This testing step is quick and prevents the most common error in criteria design, which is setting thresholds that exclude the very evidence you need.

Step 4: Write the Criteria in a Reproducible Format

Write your criteria as a numbered list. Each criterion should be a single sentence that states a requirement. Avoid vague terms like "relevant" or "significant" unless you define them.

A well-written criterion might read: "Studies must report at least one of the following outcomes: mortality rate, morbidity rate, or pathogen prevalence."

A poorly written criterion might read: "Studies must be relevant to poultry health." This tells you nothing about what to include or exclude.

Step 5: Pilot the Criteria on a Sample of Records

Apply your criteria to a sample of 20 to 50 records from your initial search. Track how many records you include, exclude, and find ambiguous. If you are uncertain about more than a few records, your criteria need refinement.

Piloting also reveals practical problems. You may find that your criteria require information that is not available in abstracts, which means you will need to retrieve full texts for screening. You may also find that your criteria are too strict, leaving you with too few studies for a meaningful synthesis.

The EQUATOR Network provides reporting guidelines that can help you describe your review methods transparently, including how you applied your criteria [<a href="#ref-2">2</a>]. Transparent reporting allows others to assess whether your criteria were appropriate and reproducible.

Common Criteria Categories

Study Design

Study design is often the first criterion you set. It determines the level of evidence you will consider and the types of bias you must address.

For questions about intervention effectiveness, randomized controlled trials are usually the preferred design. For questions about risk factors or prevalence, cohort and cross-sectional studies may be more appropriate. For questions about rare outcomes, case-control studies may be the only feasible design.

You may choose to include multiple designs and analyze them separately. This approach is common in systematic reviews that assess both efficacy and safety, where trials provide efficacy data and observational studies provide safety data.

Be explicit about what you mean by each design. A "cohort study" can mean different things in different fields. Define the design features that matter for your question, such as prospective versus retrospective data collection or the presence of a comparison group.

Population and Setting

Population criteria define the subjects of the studies you will include. In animal science, this includes species, breed, age, sex, production stage, and health status.

Setting criteria define where the studies were conducted. This may include geographic region, climate, production system, or housing type.

Population and setting criteria are often where reviewers make their most consequential decisions. A review of swine respiratory disease that includes only indoor, intensive systems will reach different conclusions than one that includes outdoor and pasture-based systems. Neither is wrong, but you must be clear about which you intend.

Consider whether your population criteria should be based on biological characteristics or management characteristics. For example, you might include all studies of weaned piglets, or you might include only studies of piglets weaned at 21 to 28 days of age. The choice depends on your question.

Intervention, Exposure, or Comparator

If your review addresses an intervention, you must define what counts as the intervention. This includes the type of intervention, the dose or intensity, the duration, and the route of administration.

For example, a review of probiotics in calf diets must decide whether to include studies using single strains, multi-strain products, or both. It must also decide whether to include studies that report the viable cell count of the probiotic and whether to set a minimum dose.

Comparator criteria define what the intervention is compared against. Common comparators include placebo, no treatment, standard care, or a different dose of the same intervention.

If your review addresses an exposure instead of an intervention, define the exposure and how it is measured. A review of heat stress in feedlot cattle must decide whether heat stress is defined by temperature-humidity index, by observed panting behavior, or by physiological measures such as respiration rate.

Outcomes

Outcome criteria define what the studies must measure for you to include them. This is often the most contested area of criteria design because different studies measure different things.

For a review of lameness treatments in dairy cattle, you might require studies to report locomotion score, gait score, or a diagnosis of claw horn lesions. You might exclude studies that report only weight gain or milk production, even though these outcomes are related.

Decide whether you will include studies that report your outcomes of interest as primary outcomes, secondary outcomes, or incidental findings. This decision affects how many studies you include and how confident you can be in your synthesis.

Time Period

Time period criteria limit your review to studies published within a certain date range. This is a practical decision that balances comprehensiveness against currency.

For rapidly changing fields, a recent time window may be appropriate. For established fields, a longer window may be necessary to capture foundational studies.

You may also set a cutoff date for your search. This is the date after which you stop looking for new studies. The cutoff should be reported in your methods so readers know the currency of your evidence.

Language and Geography

Language criteria determine whether you include studies published in languages other than English. This is a practical decision with scientific consequences.

Excluding non-English studies can introduce language bias, where the results of your review differ from what you would find if you included all languages. However, translating studies is time-consuming and may not be feasible for every review.

Geography criteria determine whether you include studies from all regions or only specific regions. This matters when the question is region-specific, such as a review of tick-borne disease in cattle in the southeastern United States.

Publication Type

Publication type criteria determine whether you include peer-reviewed journal articles, conference abstracts, preprints, theses, government reports, or other document types.

Peer-reviewed articles are the standard for most reviews because they have undergone external review. However, they may miss recent findings that have not yet been published. Preprints and conference abstracts can provide early access to new evidence but have not been peer-reviewed.

The Committee on Publication Ethics describes core practices for peer review and publication that can help you understand the differences between publication types [<a href="#ref-3">3</a>]. Understanding these differences helps you make informed decisions about what to include.

Records and Measurements

Document Your Criteria Decisions

Keep a record of every decision you make about your criteria. This includes the initial list, the rationale for each criterion, and any changes you make during the review process.

A criteria decision log is a simple table with columns for the date, the criterion, the decision, and the rationale. This log serves two purposes. It helps you write your methods section accurately, and it provides a record if questions arise about your review process.

Track Screening Decisions

During screening, track how many records you include and exclude at each stage. The standard stages are title screening, abstract screening, and full-text screening.

For each stage, record the number of records screened, the number included, and the number excluded. Record the reasons for exclusion so you can report them in your final review.

This tracking serves a practical purpose. If you find that you are excluding a large proportion of records at full-text screening, your abstract screening criteria may be too broad. If you are excluding very few, your criteria may be too narrow.

Measure Inter-Rater Agreement

If more than one person screens records, measure how often the screeners agree. This is called inter-rater agreement.

A simple approach is to have two screeners independently apply the criteria to a sample of records and compare their decisions. Disagreements are discussed and resolved. The proportion of agreements is a useful indicator of whether your criteria are clear enough for consistent application.

Low agreement suggests your criteria are ambiguous and need revision. High agreement suggests your criteria are clear and can be applied consistently.

Common Failure Patterns

Criteria That Are Too Broad

Broad criteria include too many studies, many of which are only tangentially related to your question. This creates a screening burden and a synthesis problem.

The signs of overly broad criteria are a search that returns thousands of records, a screening process that includes most of them, and a final set of studies that are so heterogeneous that you cannot compare them meaningfully.

If you find yourself in this situation, revisit your criteria and add specificity. You may need to narrow your population, require a specific outcome, or limit the study designs you include.

Criteria That Are Too Narrow

Narrow criteria exclude relevant studies, leaving you with too few to answer your question. This is a less obvious failure because the problem appears only after you have done substantial screening work.

The signs of overly narrow criteria are a search that returns few records, a screening process that excludes most of them, and a final set of studies that is too small for a meaningful synthesis.

If you find yourself in this situation, broaden your criteria. You may need to relax your time window, include additional study designs, or expand your population definition.

Criteria That Are Not Reproducible

Criteria that depend on subjective judgment are not reproducible. If two screeners apply the same criteria and reach different decisions, the criteria are not clear enough.

The fix is to define your terms. If you include "relevant" studies, define what relevance means. If you include "significant" outcomes, define the threshold for significance.

Criteria That Change During Screening

Changing your criteria after you have started screening introduces bias. You may unconsciously adjust criteria to include studies you like or exclude studies you dislike.

If you must change a criterion, document the change and the date. Re-screen the records you have already processed to ensure consistent application.

Welfare and Safety Context

Animal Welfare Considerations

When you set criteria for a review involving animals, consider whether your criteria will capture studies that report welfare outcomes. Welfare outcomes include behavioral measures, physiological stress indicators, injury and disease prevalence, and mortality.

A review of housing systems for laying hens should include studies that report feather pecking, cannibalism, keel bone fractures, and mortality, also studies that report egg production. These welfare outcomes are essential for a balanced synthesis.

The National Institutes of Health provides guidance on data management and sharing that applies to research it funds, including expectations for sharing data that may be relevant to animal studies [<a href="#ref-4">4</a>]. While this guidance is specific to NIH-funded research, the principle of transparent data sharing supports the reproducibility of your review.

Research Integrity and Ethics

Your criteria should reflect the ethical standards of the research you review. This includes considering whether the studies you include were conducted ethically and reported transparently.

The Committee on Publication Ethics describes core practices for ethical research and publication, including expectations for data integrity and responsible conduct [<a href="#ref-3">3</a>]. When you set criteria, consider whether you will exclude studies with ethical concerns or include them with caveats.

Researcher Identity and Accountability

As you conduct your review, maintain a clear record of your own contributions and decisions. The ORCID system provides a persistent identifier for researchers that can be linked to your publications and review activities [<a href="#ref-5">5</a>]. While not required for a literature review, maintaining an accurate researcher record supports accountability in the research process.

Limitations and Escalation Criteria

Know the Limits of Your Criteria

Every set of criteria has limitations. Your criteria may exclude studies that are relevant but do not meet your design requirements. They may include studies that are relevant but of low quality. They may miss studies published in languages you cannot read.

Acknowledge these limitations in your review. Describe what your criteria excluded and why. This transparency allows readers to assess whether your criteria were appropriate for your question.

When to Escalate to a More Formal Review

If your literature review is intended to inform a significant decision, such as a clinical guideline, a policy recommendation, or a funding decision, you may need to escalate to a more formal review process.

Systematic reviews follow a structured protocol that includes pre-specified criteria, comprehensive searching, and formal quality assessment. The EQUATOR Network provides reporting guidelines for systematic reviews and other study types [<a href="#ref-2">2</a>]. These guidelines describe the elements you should report, including your search strategy and your inclusion and exclusion criteria.

If you are conducting a review to support a grant application, the National Institutes of Health provides guidance on the application and review process [<a href="#ref-6">6</a>]. Understanding the expectations for preliminary evidence can help you decide whether your review is sufficient or whether a more formal approach is needed.

When to Seek Professional Help

If you are uncertain about your criteria or your review methods, seek help from a librarian, a statistician, or a colleague with systematic review experience.

Librarians can help you design search strategies and identify databases. Statisticians can help you plan your synthesis and assess heterogeneity. Experienced reviewers can help you anticipate problems and interpret ambiguous evidence.

Seek help early. The cost of fixing a poorly designed review increases as you progress through screening and synthesis.

Frequently Asked Questions

What is the difference between inclusion and exclusion criteria?

Inclusion criteria define the characteristics a study must have to be considered for your review. Exclusion criteria define the characteristics that disqualify a study. You apply both during screening, and you should write both before you begin.

How many criteria should I have?

There is no fixed number. A typical review has 5 to 15 criteria covering study design, population, intervention or exposure, outcomes, time period, language, and publication type. The right number depends on your question and the diversity of the literature.

Should I include non-English studies in my review?

This is a practical decision. Including non-English studies reduces language bias but increases the time and cost of screening and translation. If you exclude non-English studies, report this as a limitation.

How do I decide on a time period for my review?

Choose a time period that captures the evidence relevant to your question. For rapidly changing fields, a recent window may be appropriate. For established fields, a longer window may be necessary. Report your time period and justify it in your methods.

What should I do if my criteria exclude too many studies?

Broaden your criteria. You may need to relax your time window, include additional study designs, or expand your population definition. Document any changes and re-screen records you have already processed.

What should I do if my criteria include too many studies?

Narrow your criteria. You may need to require a specific outcome, limit the study designs you include, or add population restrictions. Revisit your research question to ensure your criteria align with what you actually want to know.

Can I change my criteria after I start screening?

You can, but you must document the change and re-screen records you have already processed. Changing criteria without documentation introduces bias and reduces the reproducibility of your review.

How do I know if my criteria are clear enough?

Test your criteria with a second screener. If two people applying the same criteria reach different decisions, your criteria are ambiguous. Revise them until you achieve consistent application.

Using the Evidence

Related Articles

• Scientific Literature Review Format: A Reproducible Structure for Researchers
• Conducting a Study: A Practical Checklist for Life Science Researchers
• Bacterial Genome Annotation: A Practical Quality Checklist
• Data Management Plan Template: A Practical Guide for Researchers
• Scientific Literature Review: A Reproducible Search and Synthesis Workflow

References and Further Reading

This article is educational and does not replace institutional policy, professional advice, or applicable safety and regulatory requirements.