# How to Use the Newcastle-Ottawa Scale for Cohort and Case-Control Studies ## Quick Answer - Use the Newcastle-Ottawa Scale to judge the quality of non-randomized cohort and case-control studies by scoring selection, comparability, and outcome or exposure domains. - Apply the scale item by item with a star system, then interpret the total score against study design limitations instead of as a single universal cutoff. - The scale does not replace statistical analysis or reporting guidelines, and its items require your judgment about what counts as adequate for each study. ## What the Newcastle-Ottawa Scale Measures The Newcastle-Ottawa Scale (NOS) is a tool for appraising the quality of non-randomized studies included in systematic reviews and meta-analyses. It was developed to assess cohort studies and case-control studies separately because these designs answer different questions and carry different risks of bias. The scale divides quality assessment into three domains. For cohort studies, those domains are selection of the study groups, comparability of the groups, and ascertainment of the outcome of interest. For case-control studies, the domains are selection of the cases and controls, comparability of the groups, and ascertainment of the exposure. Each domain contains specific items that you score with stars. A study can receive a maximum of nine stars across all domains. The selection domain carries a maximum of four stars, the comparability domain carries a maximum of two stars, and the outcome or exposure domain carries a maximum of three stars. The NOS does not produce a continuous measure of bias. It gives you a structured way to record whether a study met predefined criteria for key methodological features. You still need to interpret what those features mean in the context of the specific research question. ## Why Researchers Use the NOS Systematic reviews depend on transparent and reproducible quality assessment. Without a structured tool, reviewers may apply inconsistent standards across studies, and readers cannot tell how the reviewers reached their judgments. The NOS provides a common language for describing study quality. When you report that a cohort study received seven stars, another researcher can see which items were satisfied and which were not. This transparency supports the broader goal of complete and accurate reporting in health research, which organizations like the EQUATOR Network promote through reporting guidelines and educational resources. The scale also helps you separate design quality from reporting quality. A study may be well designed but poorly reported, or well reported but poorly designed. The NOS items ask you to judge what the investigators actually did, beyond what they wrote. ## How the NOS Relates to Other Quality Tools The NOS is one of several tools available for appraising non-randomized studies. Other tools include the Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I) and various checklists from the Joanna Briggs Institute. Each tool takes a different approach to defining and measuring bias. The NOS is simpler and faster to apply than ROBINS-I, which requires a detailed assessment of confounding and selection bias across multiple domains. For this reason, the NOS is often used in reviews that include a large number of studies or that cover topics where the evidence base is heterogeneous. The simplicity of the NOS is also its main limitation. The scale does not provide explicit guidance on how to judge confounding, and it does not require you to specify which confounders you consider important. Two reviewers can apply the same NOS items to the same study and reach different scores if they disagree about what counts as adequate adjustment. When you choose a quality assessment tool, consider the purpose of your review and the nature of the studies you are appraising. The EQUATOR Network provides access to reporting guidelines that can help you decide what information a study should contain, which complements the quality assessment you perform with the NOS. ## At a Glance | NOS Domain | Cohort Studies | Case-Control Studies | Maximum Stars | | --- | --- | --- | --- | | Selection | Representativeness of exposed cohort, selection of non-exposed cohort, ascertainment of exposure, demonstration that outcome was not present at start | Adequate case definition, representativeness of cases, selection of controls, definition of controls | 4 | | Comparability | Comparability of cohorts on the basis of design or analysis | Comparability of cases and controls on the basis of design or analysis | 2 | | Outcome or Exposure | Assessment of outcome, adequacy of follow-up length, adequacy of follow-up completeness | Ascertainment of exposure, same method of ascertainment for cases and controls, non-response rate | 3 | ## Step-by-Step Application for Cohort Studies ### Step 1: Assess Selection of Study Groups The selection domain for cohort studies contains four items. Each item receives one star if the study meets the criterion. The first item asks whether the exposed cohort is truly representative of the average person or animal in the community or population of interest. A study that recruits participants from a defined population registry or a random sample of the target population earns the star. A study that recruits volunteers or uses a convenience sample does not. The second item asks how the non-exposed cohort was selected. The non-exposed group should come from the same community or population as the exposed group. Drawing the non-exposed group from a different setting, such as a different hospital or a different geographic region, introduces selection bias and loses the star. The third item asks how the exposure was ascertained. Secure records, structured interviews, or written self-reports earn the star. The key question is whether the exposure information is likely to be accurate and free from recall bias. The fourth item asks whether the study demonstrated that the outcome of interest was not present at the start of the study. For a cohort study, this means confirming that the participants did not already have the disease or condition being studied. If the outcome could have been present at baseline, the study loses the star. ### Step 2: Assess Comparability of Cohorts The comparability domain carries a maximum of two stars. This domain is where you apply your judgment about confounding. The first comparability star is awarded when the study design or analysis controls for the most important confounding factor. The second star is awarded when the study controls for additional confounding factors. You must decide which confounders are most important for the specific research question. For example, in a study of a dietary exposure and heart disease, age and sex might be the most important confounders, with smoking and physical activity as additional confounders. In a study of a vaccine and infection, age and underlying health status might be the priority. The study can earn comparability stars through matching in the design phase or through statistical adjustment in the analysis phase. Matching on the most important confounder earns one star, and matching on additional confounders earns the second star. The same logic applies to multivariable adjustment. ### Step 3: Assess Outcome Ascertainment and Follow-Up The outcome domain for cohort studies contains three items. The first item asks how the outcome was assessed. Independent blind assessment, linkage to records, or secure records earn the star. Self-report without verification does not. The second item asks whether the follow-up was long enough for the outcome to occur. The adequacy of follow-up duration depends on the natural history of the condition being studied. A follow-up period that is too short to capture the outcome loses the star. The third item asks whether follow-up was complete. The study earns the star if all subjects were accounted for, or if the loss to follow-up was unlikely to introduce bias. A study with a high dropout rate or with different dropout rates between the exposed and non-exposed groups loses the star. ## Step-by-Step Application for Case-Control Studies ### Step 1: Assess Selection of Cases and Controls The selection domain for case-control studies contains four items. The first item asks whether the case definition is adequate. The study earns the star if the cases are independently validated, such as through linkage to a disease registry or confirmation by a second clinician. The study loses the star if the cases are self-reported or based on a single unverified record. The second item asks whether the cases are representative of the population of interest. Consecutive cases or a random sample of eligible cases earn the star. A convenience sample or a sample with obvious selection patterns loses the star. The third item asks how the controls were selected. Community controls earn the star. Hospital controls may earn the star only if they are drawn from a population that is clearly free of the outcome and representative of the same base population as the cases. The fourth item asks how the controls were defined. The study earns the star if the controls have no history of the outcome of interest. The study loses the star if the control definition is not explicit or if controls could have the outcome without being detected. ### Step 2: Assess Comparability of Cases and Controls The comparability domain for case-control studies follows the same logic as for cohort studies. You award up to two stars based on whether the study controlled for the most important confounders and for additional confounders. The key difference is that comparability in a case-control study is achieved through matching cases and controls on confounders during the design phase, or through statistical adjustment during the analysis phase. Matching on age and sex is common, and you must decide whether the matching or adjustment addressed the confounders that matter for your research question. ### Step 3: Assess Exposure Ascertainment The exposure domain for case-control studies contains three items. The first item asks how the exposure was ascertained. Secure records or structured interviews where the interviewer is blinded to the case or control status earn the star. Interviews that are not blinded lose the star. The second item asks whether the same method of ascertainment was used for cases and controls. If cases were interviewed and controls were assessed from records, the study loses the star because the exposure information is not comparable. The third item asks about the non-response rate. The study earns the star if the non-response rate is the same for cases and controls. A differential non-response rate between cases and controls loses the star. ## Practical Workflow for a Systematic Review ### Step 1: Pilot the Scale on a Small Sample Before you apply the NOS to all studies in your review, pilot it on three to five studies that represent the range of designs and quality you expect to encounter. This pilot step helps you identify ambiguous items and agree on interpretation rules with your co-reviewers. During the pilot, record your decisions for each item and note where you disagreed. Discuss the disagreements and write down the reasoning that resolves them. This written guidance becomes your reference for the full review. ### Step 2: Apply the Scale Independently Two reviewers should apply the NOS independently to each study. Independent assessment reduces the risk that one reviewer's assumptions drive the quality scores. Each reviewer records stars for each item without seeing the other reviewer's scores. ### Step 3: Resolve Disagreements After both reviewers complete their assessments, compare the scores item by item. Disagreements on individual items are common and do not indicate a failure of the process. Resolve disagreements through discussion, and if discussion does not resolve the issue, involve a third reviewer. Record the final consensus score for each study. Keep the individual scores and the resolution notes in your review documentation so that the process is transparent. ### Step 4: Report the Scores Transparently Report the NOS scores for each study in a table within your review. Include the scores for each domain, beyond the total. This level of detail lets readers see which aspects of study quality were strong and which were weak. The EQUATOR Network emphasizes the importance of transparent reporting in research. When you report your quality assessment methods and results clearly, you support the reproducibility of your review and help readers interpret your findings. ## Records and Measurements ### What to Record for Each Study For each study you assess, record the following information in your review spreadsheet or data extraction form: - Study identifier and full citation - Study design (cohort or case-control) - NOS score for each item in the selection domain - NOS score for each item in the comparability domain - NOS score for each item in the outcome or exposure domain - Total NOS score - Reviewer initials and date of assessment - Notes on any disagreements and how they were resolved ### How to Structure Your Data Use one row per study and one column per NOS item. This structure allows you to sort and filter studies by their scores and to create summary tables for your review manuscript. Include a column for the total score and a column for your overall judgment about whether the study should be included in the primary analysis or used only in sensitivity analyses. The NOS score alone does not determine inclusion, but it informs your decisions about how much weight to give each study. ### Common Recording Errors A common error is recording only the total score without the domain-level scores. The total score can hide important differences between studies. Two studies with the same total score may have very different strengths and weaknesses, and those differences matter when you interpret the evidence. Another common error is failing to record the reasoning behind each star. If you do not write down why you awarded or withheld a star, you cannot defend your decision later, and you cannot resolve disagreements with your co-reviewers efficiently. ## Common Failure Patterns When Using the NOS ### Applying the Scale Without a Written Protocol Reviewers who apply the NOS without a written protocol often score the same study differently at different times. The scale items require judgment, and your judgment can shift as you read more studies and develop expectations about what studies in your field typically report. Write your interpretation rules before you start the full review. Specify which confounders you will consider most important for your research question, what you will accept as adequate outcome ascertainment, and how you will judge follow-up completeness. ### Treating the Total Score as a Quality Threshold Some reviews use a total NOS score cutoff, such as seven stars, to classify studies as high or low quality. This practice is common but problematic because the cutoff is arbitrary and has no universal justification. The NOS was not designed with a validated threshold for high or low quality. Use the total score as a descriptive measure and combine it with domain-level scores and your qualitative judgment about the specific biases that matter for your research question. ### Ignoring the Difference Between Design and Reporting A study can lose NOS stars because the investigators did not report a methodological detail, even if they actually performed the method correctly. The NOS assesses what you can determine from the study report, and poor reporting can lower a study's score. When you encounter a study with unclear reporting, check whether the study followed a reporting guideline. The EQUATOR Network provides access to reporting guidelines that specify what information authors should include, and these guidelines can help you determine whether missing information reflects poor design or poor reporting. ### Using the NOS for Study Designs It Was Not Built For The NOS was developed for cohort and case-control studies. It is not appropriate for cross-sectional studies, randomized trials, or qualitative research. Applying the NOS to these designs produces misleading scores because the items do not map onto the features that matter for those designs. If your review includes study designs beyond cohort and case-control studies, select a quality assessment tool that matches each design. The EQUATOR Network can help you identify appropriate reporting guidelines, and your review protocol should specify which quality tool you will use for each design. ## Limitations of the Newcastle-Ottawa Scale ### Limited Guidance on Confounding The comparability domain asks you to judge whether the study controlled for the most important confounders, but the scale does not tell you which confounders matter. This decision is left entirely to the reviewer. Different reviewers may identify different confounders as most important for the same research question. This variability reduces the reliability of the comparability scores and can lead to different quality assessments for the same study. ### No Explicit Handling of Bias Direction The NOS produces a score that reflects the presence or absence of certain methodological features. It does not tell you the direction or magnitude of bias that those features might introduce. A study that loses stars for inadequate follow-up may have biased results in either direction, depending on whether the participants lost to follow-up were more or less likely to experience the outcome. The NOS score does not capture this nuance. ### Arbitrary Scoring Weights Each NOS item carries one star, which means each item contributes equally to the total score. This equal weighting may not reflect the actual importance of each item for the specific research question. For example, inadequate control of a strong confounder may introduce more bias than a slightly incomplete follow-up, but the NOS gives both issues equal weight in the total score. ### Poor Inter-Rater Reliability in Some Contexts Studies that examine the reliability of the NOS have found variable agreement between reviewers, particularly for the comparability domain. The subjectivity of the comparability items is the main source of disagreement. You can improve reliability by writing detailed interpretation rules before the review, piloting the scale, and resolving disagreements through discussion. These practices do not eliminate subjectivity, but they make the process more consistent. ## Welfare and Safety Context for Animal Studies When you apply the NOS to animal studies, the same methodological principles apply, but you must adapt the items to the animal research context. For cohort studies in animals, the exposed cohort might be animals on a specific farm or in a specific management system, and the non-exposed cohort might be animals in a different system. The comparability domain requires you to consider confounders such as breed, age, housing density, and veterinary care. The outcome domain requires you to consider how the outcome was diagnosed and whether the follow-up period was long enough for the condition to develop. For case-control studies in animals, the cases might be animals with a specific disease, and the controls might be animals without the disease from the same herd or flock. The exposure of interest might be a management practice, a feed ingredient, or a housing condition. The NOS does not assess animal welfare directly. It assesses methodological quality, which is related to welfare because poorly designed studies can produce misleading conclusions that lead to harmful management decisions. When you read animal studies, consider both the methodological quality and the welfare implications of the findings. The National Library of Medicine provides access to biomedical books and research-method references that can help you understand the principles of study design and quality assessment in the context of health research, including animal health research. ## Professional Escalation Criteria ### When to Seek Additional Guidance If you are applying the NOS as part of a systematic review that will inform clinical or policy decisions, and you encounter persistent disagreements between reviewers that you cannot resolve, escalate the issue to a methodologist or a librarian with systematic review expertise. If your review includes studies with complex designs, such as nested case-control studies or case-cohort studies, seek guidance on how to apply the NOS to these designs. The standard NOS items may not map cleanly onto these designs, and a methodologist can help you adapt the tool appropriately. ### When to Consider a Different Quality Tool If your review includes non-randomized studies of interventions, consider whether the NOS is the most appropriate tool. The Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I) provides a more detailed assessment of bias for intervention studies, and some reviewers prefer it for this purpose. If your review includes a small number of studies and you have the resources to conduct a detailed bias assessment, a more comprehensive tool may be appropriate. The NOS is efficient, but efficiency comes at the cost of detail. ### When to Consult Publication Ethics Resources If you encounter a study with suspected fabrication, duplicate publication, or undisclosed conflicts of interest, do not attempt to resolve the issue through the NOS. The NOS assesses methodological quality, not research integrity. The Committee on Publication Ethics provides core practices for handling allegations of misconduct, including guidance on authorship, peer review, data handling, and conflicts of interest. Consult these resources and follow the procedures of the journal or institution where the research was conducted. ## How to Integrate NOS Scores Into Your Review ### Use Scores to Inform Sensitivity Analyses A common approach is to conduct the primary meta-analysis with all eligible studies and then repeat the analysis restricted to studies with higher NOS scores. This sensitivity analysis shows whether the conclusions change when lower-quality studies are excluded. Report the results of both analyses so that readers can see the impact of study quality on the findings. If the conclusions differ substantially between the full analysis and the restricted analysis, interpret the findings with caution. ### Do Not Use Scores as Weights Some reviewers consider weighting studies by their NOS scores in the meta-analysis. This practice is not standard and is not recommended by the developers of the NOS. The NOS score is an ordinal measure of quality, and treating it as a continuous weight implies a level of precision that the scale does not have. Use the scores to describe study quality and to inform sensitivity analyses, not to assign statistical weights. ### Report the Assessment Process Describe your NOS assessment process in the methods section of your review. Include the number of reviewers, the pilot process, the disagreement resolution procedure, and the interpretation rules you used for ambiguous items. This description supports the transparency that the EQUATOR Network promotes and allows readers to assess the credibility of your quality assessment. ## Frequently Asked Questions ### What is the maximum score on the Newcastle-Ottawa Scale? The maximum score is nine stars. The selection domain carries up to four stars, the comparability domain carries up to two stars, and the outcome or exposure domain carries up to three stars. ### Can I use the Newcastle-Ottawa Scale for randomized controlled trials? No. The NOS was developed for non-randomized cohort and case-control studies. Randomized trials have different sources of bias and should be assessed with tools designed for randomized designs, such as the Cochrane Risk of Bias tool. ### What does a low Newcastle-Ottawa Scale score mean? A low score means the study did not meet several of the methodological criteria in the scale. It does not automatically mean the study results are wrong, but it indicates a higher risk of bias that should be considered when interpreting the findings. ### How do I decide which confounders are most important for the comparability domain? Base your decision on the research question and the existing evidence in the field. The most important confounders are those that are strongly associated with both the exposure and the outcome. Write your decisions in your review protocol before you start the assessment. ### Can I modify the Newcastle-Ottawa Scale for my review? Modifications are common, but you must report them clearly. If you change the items, the scoring, or the interpretation rules, describe the changes in your methods section and explain why the modifications were necessary. ### How many reviewers should apply the Newcastle-Ottawa Scale? Two reviewers working independently is the standard approach. Independent assessment reduces bias in the scoring process, and disagreements are resolved through discussion or with a third reviewer. ### Does the Newcastle-Ottawa Scale assess reporting quality? The NOS assesses features that you can determine from the study report, so poor reporting can lower a study's score. The scale does not distinguish between a study that failed to perform a method and a study that performed the method but did not report it. ### Where can I find reporting guidelines for the studies I am assessing? The EQUATOR Network provides access to reporting guidelines for many study designs. These guidelines specify what information authors should include in their reports, which helps you determine whether missing information reflects poor design or poor reporting. ## Using the Evidence | Source | Best use in this topic | Important limitation | |---|---|---| | [Research Methods Resources](https://www.ncbi.nlm.nih.gov/books) | official guidance | Check the linked page for current local requirements | | [EQUATOR Network](https://www.equator-network.org/) | official guidance | Check the linked page for current local requirements | | [Core Practices](https://publicationethics.org/core-practices) | official guidance | Check the linked page for current local requirements | ## Related Articles - [Case-Control vs Cohort Studies: A Practical Comparison for Epidemiological Research](/blog/guides/case-control-vs-cohort-studies-a-practical-comparison-for-epidemiological-research) - [Observational Study Types: A Guide to Cohort, Case-Control, and Cross-Sectional Designs](/blog/guides/observational-study-types-a-guide-to-cohort-case-control-and-cross-sectional-designs) - [Cohort Studies: A Practical Guide to Design, Conduct, and Interpretation](/blog/guides/cohort-studies-a-practical-guide-to-design-conduct-and-interpretation) - [Case-Control Study Design: A Practical Guide for Planning and Conducting](/blog/guides/case-control-study-design-a-practical-guide-for-planning-and-conducting) - [Cross-Sectional vs. Case-Control Studies: Key Differences and How to Choose](/blog/guides/cross-sectional-vs-case-control-studies-key-differences-and-how-to-choose) ## References and Further Reading - [Research Methods Resources](https://www.ncbi.nlm.nih.gov/books). National Library of Medicine. - [EQUATOR Network](https://www.equator-network.org/). EQUATOR Network. - [Core Practices](https://publicationethics.org/core-practices). Committee on Publication Ethics. - [NIH Grants and Funding](https://grants.nih.gov/). National Institutes of Health. - [ORCID for Researchers](https://info.orcid.org/researchers). ORCID. - [Data Management and Sharing Policy](https://sharing.nih.gov/data-management-and-sharing-policy). National Institutes of Health. - [NCBI Literature Resources](https://www.ncbi.nlm.nih.gov/). National Center for Biotechnology Information. - [PubMed](https://pubmed.ncbi.nlm.nih.gov/). National Library of Medicine. > This article is educational and does not replace institutional policy, professional advice, or applicable safety and regulatory requirements.