# Multiple Testing Correction in Proteomics: Choosing Between FDR, Bonferroni, and q-Values Proteomics experiments routinely measure hundreds to thousands of proteins simultaneously, creating a statistical challenge that demands explicit correction for multiple testing. When a researcher tests 1,000 proteins for association with a biological condition, the probability of observing false positives by chance alone becomes substantial. This article explains the core problem, compares the most common correction approaches, and provides practical decision criteria for selecting the appropriate method in proteomics workflows. The central issue is straightforward. A conventional significance threshold of p < 0.05 means that 5% of tests will appear significant purely by random chance. With 1,000 proteins measured, approximately 50 false positives would be expected even when no true biological differences exist. Multiple testing correction methods adjust significance thresholds or p-values to control this inflation of false discoveries. Three approaches dominate proteomics practice: Bonferroni correction, false discovery rate (FDR) control, and q-value estimation. Each method addresses a different error concept, and each has distinct strengths and limitations depending on the experimental context. The choice between them affects which proteins are reported as significant, how many candidates proceed to validation, and ultimately the reproducibility of findings across independent cohorts. ## The Multiple Testing Problem in High-Dimensional Proteomics Modern proteomics platforms generate data on an unprecedented scale. Affinity-based assays can measure hundreds of proteins per sample, while mass spectrometry approaches can identify and quantify thousands of proteins in a single run. This dimensionality creates a statistical environment where uncorrected testing produces unacceptable error rates. Consider a typical discovery experiment comparing protein levels between two groups. For each protein, a statistical test produces a p-value representing the probability of observing the measured difference if no true difference exists. When 500 proteins are tested at the conventional 0.05 threshold, roughly 25 false positives are expected. When 5,000 proteins are tested, that number rises to 250. These false positives are not statistical abstractions. They consume validation resources, misdirect follow-up experiments, and erode confidence in published findings. The multiple testing problem is not unique to proteomics. Genomics, metabolomics, and other high-dimensional fields face the same challenge. However, proteomics presents specific complications. Protein measurements are often correlated with each other because proteins function within shared biological pathways. This correlation violates the assumption of independent tests that underlies some correction methods. Additionally, proteomics data frequently contain missing values, batch effects, and technical variability that interact with statistical correction in complex ways. The practical consequence is that researchers must decide also which statistical test to apply but also how to interpret results after correction. A protein with an uncorrected p-value of 0.001 may or may not survive correction depending on the method chosen and the total number of tests performed. Understanding the relationship between these factors is essential for designing experiments with adequate statistical power and for interpreting results correctly. ## Core Principles of Error Control Statistical correction methods operate by controlling different types of error. The family-wise error rate (FWER) and the false discovery rate (FDR) represent two distinct philosophies with different practical implications. ### Family-Wise Error Rate The family-wise error rate is the probability of making at least one false positive among all tests performed. Controlling the FWER at 0.05 means that the chance of any single false positive across the entire experiment is limited to 5%. This is a stringent criterion that becomes increasingly conservative as the number of tests grows. The Bonferroni correction is the most widely known FWER method. It divides the significance threshold by the number of tests performed. If 1,000 proteins are tested, the adjusted threshold becomes 0.05 divided by 1,000, or 0.00005. Only proteins with p-values below this adjusted threshold are declared significant. The strength of Bonferroni correction is its simplicity and its guarantee of FWER control. The weakness is its conservatism. With thousands of proteins, the adjusted threshold becomes so stringent that true but modest effects may be missed. This trade-off between false positives and false negatives is the fundamental tension in multiple testing correction. ### False Discovery Rate The false discovery rate takes a different approach. Instead of controlling the probability of any false positive, FDR controls the expected proportion of false positives among the tests declared significant. An FDR of 0.05 means that approximately 5% of the proteins reported as significant are expected to be false positives. The Benjamini-Hochberg procedure is the most commonly used FDR method. It ranks all p-values from smallest to largest and compares each to a threshold that increases with rank. This approach is less conservative than Bonferroni correction while still providing meaningful error control. The FDR framework is particularly well suited to discovery-oriented proteomics where the goal is to identify candidate proteins for further investigation instead of to make definitive claims about individual proteins. The q-value is a natural companion to the FDR approach. For each protein, the q-value represents the minimum FDR at which that protein would be considered significant. A protein with a q-value of 0.03 would be significant at an FDR threshold of 0.05 but not at a threshold of 0.01. Reporting q-values alongside or instead of p-values provides a direct measure of the expected false discovery proportion for each result. ## At a Glance | Method | Error Controlled | Stringency | Best Use Case | Limitation | |--------|-----------------|------------|---------------|------------| | Bonferroni | Family-wise error rate | Most stringent | Small protein panels, confirmatory claims | Misses modest true effects with many tests | | Benjamini-Hochberg FDR | False discovery rate | Moderate | Discovery proteomics, candidate generation | Allows some false positives by design | | Q-values | False discovery rate per protein | Flexible | Reporting across thresholds, meta-analysis | Requires understanding of FDR concept | ## Comparing Correction Methods The choice between Bonferroni, FDR, and q-values depends on the experimental goals, the number of proteins measured, and the tolerance for false positives versus false negatives. Each method has a distinct profile of strengths and limitations. ### Bonferroni Correction Bonferroni correction is appropriate when the cost of any false positive is high and when the number of tests is relatively small. It is the method of choice when researchers need to make definitive claims about individual proteins without the possibility of follow-up validation. The method is computationally simple and easy to explain. The adjusted threshold is calculated by dividing the desired significance level by the number of tests. This transparency is valuable in settings where statistical methods must be communicated to non-specialists. The primary limitation is loss of statistical power. With large numbers of proteins, the adjusted threshold becomes extremely stringent. A protein with a p-value of 0.0001 would not survive Bonferroni correction in an experiment testing 1,000 proteins, even though this p-value represents strong evidence of a true effect. Researchers using Bonferroni correction must accept that some true associations will be missed. ### Benjamini-Hochberg FDR The Benjamini-Hochberg procedure is the standard FDR method in proteomics. It provides a balance between error control and discovery power that suits the exploratory nature of most proteomics experiments. The procedure works by sorting p-values in ascending order and comparing each to a threshold based on its rank. The largest p-value that falls below its threshold is declared significant, along with all smaller p-values. This approach adapts to the distribution of p-values in the dataset, providing more power when many true effects exist and less power when few true effects exist. FDR control is particularly valuable in proteomics because the field is inherently discovery oriented. Researchers typically use proteomics to generate candidate lists for further investigation. An FDR of 0.05 or 0.10 provides reasonable confidence that most reported candidates are genuine while allowing sufficient sensitivity to detect modest effects. ### Q-Values Q-values are the FDR analog of p-values. For each protein, the q-value is the minimum FDR at which that protein would be declared significant. This formulation allows researchers to apply different FDR thresholds without recalculating the entire analysis. The q-value approach is especially useful when reporting results to diverse audiences. A researcher can report that a protein has a q-value of 0.02, allowing readers to apply their own significance criteria. This flexibility is valuable in multi-center studies or when results will be used for different purposes. Q-values also facilitate comparison across studies. Because q-values are standardized measures of evidence, they can be compared between experiments with different numbers of proteins or different platforms. This comparability is important for meta-analyses and for evaluating the consistency of findings across independent cohorts. ## Practical Workflow for Applying Corrections Implementing multiple testing correction in proteomics requires a systematic approach that begins before data collection and extends through interpretation and reporting. ### Step 1: Define the Testing Framework Before running any statistical tests, determine the total number of comparisons that will be performed. This number should include all proteins measured, including those that pass initial quality filters. The total number of tests directly affects the correction stringency and therefore the statistical power of the experiment. Consider whether the analysis involves a single comparison or multiple comparisons. A study comparing protein levels between two groups involves one test per protein. A study comparing protein levels across four groups involves multiple pairwise comparisons, which multiplies the number of tests and requires appropriate adjustment. ### Step 2: Select the Correction Method Choose the correction method based on the experimental goals and the consequences of false positives versus false negatives. For discovery experiments aimed at generating candidate lists, FDR control with q-values is typically appropriate. For confirmatory experiments where individual protein claims will be made without validation, Bonferroni correction may be necessary. Document the chosen method and the rationale in the analysis plan. This documentation is essential for reproducibility and for defending the analytical choices in peer review. ### Step 3: Apply the Correction Apply the chosen correction method to the full set of p-values. Most statistical software packages include functions for Bonferroni correction and Benjamini-Hochberg FDR. The R programming environment, widely used in bioinformatics, provides these functions in base packages and through the [Bioconductor project](https://bioconductor.org/), which offers specialized tools for genomic and proteomic analysis. Verify that the correction was applied correctly by checking the output against manual calculations for a subset of proteins. This verification step catches errors in data handling or software configuration that could invalidate the results. ### Step 4: Interpret Corrected Results Interpret the corrected results in the context of the experimental design and the biological questions being addressed. A protein that survives FDR correction at 0.05 is a strong candidate for further investigation, but it is not a confirmed biological finding. The q-value indicates the expected proportion of false positives among all proteins declared significant, not the probability that any individual protein is a false positive. Consider the biological plausibility of the significant proteins. Proteins that cluster in known biological pathways or that have established roles in the condition being studied are more credible candidates than isolated proteins with no obvious biological connection. ### Step 5: Report Methods and Results Transparently Report the correction method, the total number of tests, and the significance threshold in all publications and presentations. This information is essential for readers to evaluate the strength of the evidence and to compare results across studies. Include both uncorrected p-values and corrected q-values in supplementary tables. This transparency allows readers to apply their own thresholds and to assess the sensitivity of the findings to different correction approaches. ## Options and Tradeoffs in Correction Methods The selection of a correction method involves tradeoffs that should be made explicit instead of defaulting to a single approach for all analyses. ### Tradeoff Between False Positives and False Negatives The fundamental tradeoff in multiple testing correction is between false positives and false negatives. Stringent correction methods reduce false positives but increase false negatives. Lenient methods have the opposite effect. In discovery proteomics, the optimal balance depends on the downstream validation capacity. If validation is expensive or difficult, a more stringent correction is appropriate. If validation is straightforward, a more lenient correction may be acceptable because false positives can be filtered out in subsequent experiments. ### Tradeoff Between Power and Certainty Bonferroni correction provides high certainty about individual findings but sacrifices statistical power. FDR methods preserve more power but provide less certainty about individual proteins. The q-value framework allows researchers to express this tradeoff explicitly by reporting the FDR at which each protein would be significant. ### Tradeoff Between Simplicity and Flexibility Bonferroni correction is simple to implement and explain but offers no flexibility in interpretation. FDR methods are more complex but provide greater flexibility in applying different thresholds and in comparing results across studies. The choice between these approaches depends on the audience and the intended use of the results. ## Observations and Measurements in Practice Real proteomics studies demonstrate how correction methods affect results and interpretation. The published literature provides concrete examples of the consequences of different analytical choices. ### FDR in Biomarker Discovery A study of plasma proteomics in type 2 diabetes used targeted profiling of 368 proteins to identify predictors of chronic kidney disease. The researchers applied FDR correction and considered a q-value below 0.2 as statistically significant. This relatively lenient threshold reflected the discovery-oriented nature of the study and the intention to validate candidate proteins in independent cohorts. The analysis identified nine independent protein predictors that were subsequently tested in a large independent cohort from the UK Biobank Pharma Proteomics Project. This workflow illustrates the common pattern in proteomics where discovery-phase FDR thresholds are more lenient than confirmatory-phase thresholds. The resulting prediction model, combining eGFR with a five-protein panel, achieved good performance with an area under the receiver operating characteristic curve of at least 0.80 across populations ([A Five-Plasma Protein-Based Algorithm for Predicting Incident CKD in Type 2 Diabetes](https://pubmed.ncbi.nlm.nih.gov/41563825)). ### FDR in Mechanistic Studies A study of serum proteomics in multiple sclerosis used the Olink 48 Cytokine panel to measure immune-related proteins. The researchers applied FDR correction and considered q-values below 0.05 as significant. This more stringent threshold reflected the smaller number of proteins measured and the mechanistic focus of the study. The analysis identified specific cytokine alterations associated with different disease subtypes, including decreased GM-CSF in patients with low glial fibrillary acidic protein and decreased EGF in patients with high glial fibrillary acidic protein. The q-values for these findings ranged from 0.002 to below 0.0001, indicating strong evidence that survived correction ([New serum soluble factors predicting inflammatory and non-inflammatory disability worsening in multiple sclerosis](https://pubmed.ncbi.nlm.nih.gov/41425552)). ### FDR in Clinical Trial Analysis A post-hoc analysis of the REHAB-PH trial in pulmonary arterial hypertension applied FDR control to evaluate the molecular effects of famotidine treatment. The analysis found significant changes across 191 proteomic pathways at a q-value threshold of 0.05, while no metabolomic changes remained significant after correction. This contrast between proteomic and metabolomic findings illustrates how correction methods can affect conclusions across different molecular data types. The proteomic changes did not translate into clinical benefit, highlighting the importance of interpreting molecular findings in the context of clinical outcomes ([Harnessing metabolomics and proteomics in a clinical trial for pulmonary arterial hypertension](https://pubmed.ncbi.nlm.nih.gov/41849831)). ### When Correction Eliminates All Significant Results A study of cerebrospinal fluid proteomics in postoperative delirium measured 1,305 proteins in a small sample of 24 matched case-control pairs. The initial analysis identified 32 proteins associated with delirium at an uncorrected p-value threshold of 0.05. However, none of these proteins remained significant after Benjamini-Hochberg correction and q-value analysis. The researchers attributed this loss of significance to the limited sample size instead of to the absence of true biological effects. They used alternative analytical approaches, including hierarchical clustering and principal components analysis, to separate delirium cases from controls. The receiver operating characteristic curve yielded an area under the curve of 0.91, demonstrating that the protein data contained useful biological information even though individual proteins did not survive correction ([Aptamer-Based Proteomics Measuring Preoperative Cerebrospinal Fluid Protein Alterations Associated with Postoperative Delirium](https://pubmed.ncbi.nlm.nih.gov/37759795)). This example illustrates the important distinction between statistical significance and biological relevance, and it demonstrates the need for adequate sample sizes in proteomics discovery studies. ### FDR in Subtype Comparisons A study of serum proteomic profiles in depressive subtypes measured 171 proteins using a multi-analyte profiling platform. The researchers applied FDR correction and considered q-values below 0.05 as significant. The analysis identified 9 analytes differing between atypical and melancholic subtypes and 23 analytes differing between atypical depression and controls. Eight of the nine markers distinguishing the subtypes overlapped with markers from the comparison between atypical depression and controls, including mesothelin, leptin, IGFBP1, IGFBP2, FABPa, insulin, C3, and B2M. These markers were mainly involved in cellular communication, signal transduction, and immune response. No markers differed significantly between the melancholic subtype and controls ([Serum proteomic profiles of depressive subtypes](https://pubmed.ncbi.nlm.nih.gov/27404283)). This study demonstrates how FDR correction can reveal subtype-specific biological differences while also showing that some comparisons may yield no significant findings after correction. ## Records and Documentation Requirements Proper documentation of multiple testing correction is essential for reproducibility and for the credibility of proteomics findings. ### Analysis Plan Documentation Document the correction method, the total number of tests, and the significance threshold before conducting the analysis. This pre-specification prevents post-hoc selection of thresholds that favor particular results. The analysis plan should also specify how missing data will be handled, because missing values can affect the number of tests and the validity of the correction. ### Software and Version Documentation Record the software packages and versions used for statistical analysis. Different implementations of the same correction method can produce slightly different results, particularly for FDR methods that involve estimation procedures. The [Bioconductor project](https://bioconductor.org/) provides versioned packages for genomic and proteomic analysis, and documenting these versions is essential for reproducibility. ### Data and Code Archiving Archive the raw data, the analysis code, and the output files in a manner that allows independent verification of the results. Public repositories and workflow platforms such as those supported by the [Galaxy Training Network](https://training.galaxyproject.org/) and [nf-core documentation](https://nf-co.re/docs) provide infrastructure for reproducible analysis. The [Carpentries lessons](https://carpentries.org/lessons) offer training in the computational skills needed to implement reproducible workflows. The [EMBL-EBI training portal](https://www.ebi.ac.uk/training) provides learning pathways for bioinformatics data analysis that include guidance on statistical methods and reproducible practices. ### Reporting in Publications Report the correction method, the number of tests, and the significance threshold in the methods section of publications. Include both uncorrected and corrected statistics in supplementary tables. This transparency allows readers to evaluate the robustness of the findings and to apply alternative correction methods if they disagree with the authors' choices. ## Common Failure Patterns in Multiple Testing Correction Several recurring problems undermine the validity of multiple testing correction in proteomics. Recognizing these patterns is the first step toward avoiding them. ### Failure to Correct at All The most fundamental error is failing to apply any correction when testing hundreds or thousands of proteins. This error produces inflated false positive rates and findings that do not replicate in independent cohorts. The problem is particularly common in exploratory analyses where researchers report uncorrected p-values without acknowledging the multiple testing issue. ### Incorrect Number of Tests Applying correction based on the wrong number of tests produces incorrect thresholds. Underestimating the number of tests leads to insufficient correction and excess false positives. Overestimating the number of tests leads to excessive correction and loss of statistical power. The number of tests should include all comparisons performed, including pairwise comparisons in multi-group studies. ### Applying Correction to Filtered Data Some researchers apply quality filters to remove poorly measured proteins before correction, then apply correction only to the remaining proteins. This approach is problematic because the filtering decisions are often based on the same data used for significance testing. The correction should be applied to the full set of tests that could potentially be declared significant. ### Ignoring Correlation Between Tests Most correction methods assume independence or weak dependence between tests. Proteomics data violate this assumption because proteins within shared biological pathways are correlated. The Benjamini-Hochberg procedure is relatively robust to moderate correlation, but extreme correlation can affect its performance. Researchers should be aware of this limitation and consider whether their data contain highly correlated protein groups. ### Misinterpreting Q-Values Q-values are frequently misinterpreted as the probability that a specific protein is a false positive. The correct interpretation is that a q-value of 0.05 means that among all proteins declared significant at this threshold, approximately 5% are expected to be false positives. This distinction is subtle but important for interpreting individual findings. ### Using Correction as a Substitute for Validation Multiple testing correction reduces false positives but does not eliminate them. Even with stringent correction, some reported findings will be false positives. Correction is not a substitute for validation in independent cohorts or through orthogonal experimental methods. The study of cerebrospinal fluid proteomics in delirium illustrates this point, where proteins that did not survive correction were still biologically plausible candidates worthy of further investigation. ## Limitations of Correction Methods Multiple testing correction methods have inherent limitations that researchers must understand to interpret results appropriately. ### Statistical Power Limitations All correction methods reduce statistical power to some degree. The power loss is most severe with Bonferroni correction and least severe with FDR methods, but even FDR methods can miss true effects when the number of tests is large and the effect sizes are modest. Researchers should conduct power calculations before experiments to ensure that the sample size is adequate for the expected effect sizes and the chosen correction method. ### Assumption Limitations Correction methods make assumptions about the distribution of p-values and the independence of tests. When these assumptions are violated, the correction may be too lenient or too stringent. Proteomics data frequently violate independence assumptions because of biological correlation between proteins. Researchers should be aware of these limitations and consider whether alternative approaches, such as permutation-based methods, are more appropriate for their data. ### Interpretation Limitations Corrected statistics provide information about error rates but not about biological significance. A protein that survives correction may still be biologically irrelevant, and a protein that does not survive correction may still be biologically important. The study of postoperative delirium illustrates this point, where proteins that did not survive correction were still used to build predictive models with good performance. ### Replication Limitations Correction methods cannot compensate for inadequate study design or poor data quality. Findings from a single study, even with stringent correction, may not replicate in independent cohorts. Replication is the ultimate test of validity, and correction methods are not a substitute for it. ## Quality Controls and Verification Implementing quality controls around multiple testing correction improves the reliability of proteomics findings. ### Positive and Negative Controls Include proteins with known biological behavior as controls in the analysis. Positive controls should show significant differences between groups, and negative controls should not. The behavior of these controls under different correction methods provides a check on the validity of the analysis. ### Sensitivity Analysis Test the sensitivity of the findings to different correction methods and thresholds. If the same proteins are significant under Bonferroni correction and FDR correction, the findings are robust. If significance depends heavily on the correction method, the findings are less certain and should be interpreted with caution. ### Permutation Testing Permutation-based approaches provide an alternative to analytical correction methods. By randomly shuffling group labels and recalculating test statistics, permutation methods estimate the null distribution directly from the data. This approach accounts for correlation between tests and does not rely on distributional assumptions. Permutation methods are computationally intensive but provide more accurate error control for complex data structures. ### Independent Validation The most powerful quality control is independent validation. Proteins identified in a discovery cohort should be tested in an independent cohort using the same or orthogonal methods. The study of plasma proteomics in type 2 diabetes exemplifies this approach, where discovery findings were validated in the UK Biobank Pharma Proteomics Project. ## Safety and Regulatory Context Multiple testing correction has implications for the safety and regulatory context of proteomics research, particularly in clinical applications. ### Biomarker Claims Proteins identified through proteomics may be proposed as biomarkers for diagnosis, prognosis, or treatment response. Regulatory agencies require evidence that biomarker claims are not based on false positives. Appropriate multiple testing correction is a necessary component of the evidence base for biomarker claims. ### Clinical Trial Analysis Proteomics analyses in clinical trials must account for multiple testing to avoid false claims about treatment effects. The REHAB-PH trial analysis illustrates the importance of correction in clinical contexts, where proteomic changes were identified but did not correspond to clinical benefit. Researchers and regulators must interpret molecular findings in the context of clinical outcomes. ### Diagnostic Test Development Proteins that survive multiple testing correction may be candidates for diagnostic test development. The transition from discovery to diagnostic test requires extensive validation, including assessment of sensitivity, specificity, and reproducibility. Multiple testing correction is only the first step in this process. ## Professional Escalation Criteria Researchers should seek additional statistical expertise when certain conditions are present in their proteomics data. ### Complex Study Designs Studies with multiple groups, longitudinal sampling, or nested designs require statistical approaches beyond simple correction methods. A biostatistician or bioinformatics specialist should be consulted to ensure that the analysis appropriately accounts for the study design. ### High-Dimensional Data with Strong Correlation When proteomics data contain large blocks of highly correlated proteins, standard correction methods may not provide adequate error control. Specialized approaches, such as permutation-based FDR or methods that account for correlation structure, may be necessary. ### Conflicting Results Across Methods When different correction methods produce substantially different conclusions, the reasons for the discrepancies should be investigated. A statistical consultant can help determine whether the discrepancies reflect genuine uncertainty in the data or errors in the analysis. ### Regulatory or Clinical Applications Proteomics findings intended for regulatory submission or clinical use require rigorous statistical analysis and documentation. Consultation with a biostatistician is essential to ensure that the analysis meets regulatory standards. ## Decision Framework for Selecting a Correction Method Choosing between Bonferroni, FDR, and q-values requires a structured decision process that accounts for experimental goals, validation capacity, and the consequences of different error types. A practical framework helps researchers move beyond default choices and select the method that best matches their specific experimental context. ### Step 1: Classify the Experimental Purpose The first decision point is whether the experiment serves a discovery or confirmatory purpose. Discovery experiments aim to generate candidate protein lists for further investigation. Confirmatory experiments test specific hypotheses about individual proteins with pre-existing evidence. For discovery experiments, FDR-based methods are generally appropriate because they preserve statistical power while controlling the expected proportion of false positives. The study of plasma proteomics in type 2 diabetes exemplifies this approach, where researchers used FDR correction with a q-value threshold of 0.2 to identify candidate predictors for subsequent validation in an independent cohort ([A Five-Plasma Protein-Based Algorithm for Predicting Incident CKD in Type 2 Diabetes](https://pubmed.ncbi.nlm.nih.gov/41563825)). For confirmatory experiments, Bonferroni correction may be more appropriate because it controls the probability of any false positive across the entire experiment. This stringent approach is suitable when individual protein claims will be made without follow-up validation. ### Step 2: Assess Validation Capacity The downstream validation capacity should influence the correction stringency. When validation is expensive, time-consuming, or technically difficult, a more stringent correction reduces the burden of pursuing false leads. When validation is straightforward, a more lenient threshold may be acceptable because false positives can be filtered out in subsequent experiments. The multiple sclerosis study illustrates this principle. Researchers measured 48 cytokines using a targeted panel and applied FDR correction with a q-value threshold of 0.05. The smaller number of tests and the mechanistic focus of the study supported a more stringent threshold than the diabetes study, which measured 368 proteins and used a q-value threshold of 0.2 for discovery purposes ([New serum soluble factors predicting inflammatory and non-inflammatory disability worsening in multiple sclerosis](https://pubmed.ncbi.nlm.nih.gov/41425552)). ### Step 3: Evaluate the Number of Tests The total number of tests directly affects the tradeoff between false positives and false negatives. With fewer than 100 proteins, Bonferroni correction may be feasible without excessive power loss. With hundreds or thousands of proteins, FDR methods become necessary to preserve the ability to detect true effects. The depressive subtypes study measured 171 proteins and applied FDR correction with a q-value threshold of 0.05. This approach identified 9 analytes differing between atypical and melancholic subtypes and 23 analytes differing between atypical depression and controls. A Bonferroni correction would have required a threshold of approximately 0.0003, which would likely have eliminated most of these findings ([Serum proteomic profiles of depressive subtypes](https://pubmed.ncbi.nlm.nih.gov/27404283)). ### Step 4: Consider the Correlation Structure Proteins within shared biological pathways are often correlated, violating the independence assumption of some correction methods. The Benjamini-Hochberg procedure is relatively robust to moderate correlation, but extreme correlation can affect its performance. When the data contain large blocks of highly correlated proteins, consider whether permutation-based approaches might provide more accurate error control. These methods estimate the null distribution directly from the data by shuffling group labels, accounting for the correlation structure without relying on distributional assumptions. ### Step 5: Document the Decision Record the correction method, the rationale for its selection, and the significance threshold in the analysis plan before conducting the analysis. This pre-specification prevents post-hoc selection of thresholds that favor particular results and supports reproducibility. ## Decision Matrix for Common Proteomics Scenarios | Experimental Scenario | Recommended Method | Rationale | |----------------------|-------------------|-----------| | Discovery screen, hundreds of proteins, validation planned | FDR with q-value threshold 0.05 to 0.20 | Preserves power for candidate generation | | Targeted panel, fewer than 100 proteins, mechanistic focus | FDR with q-value threshold 0.05 | Balances error control with detection of modest effects | | Confirmatory claim about individual proteins, no validation planned | Bonferroni | Controls probability of any false positive | | Small sample size, exploratory analysis | FDR with sensitivity analysis | Avoids excessive power loss while acknowledging uncertainty | | Multi-omic comparison across data types | FDR applied consistently across all tests | Enables direct comparison of findings across platforms | ## Implementing the Decision Framework The decision framework translates into a practical workflow that can be applied before data collection and refined during analysis. ### Pre-Experiment Planning Before collecting data, document the experimental purpose, the expected number of proteins to be measured, and the downstream validation capacity. Use this information to select the correction method and significance threshold. Conduct power calculations to ensure the sample size is adequate for the expected effect sizes and the chosen correction method. ### Analysis Execution Apply the selected correction method to the full set of p-values. Verify the implementation by checking output against manual calculations for a subset of proteins. The [Bioconductor project](https://bioconductor.org/) provides versioned packages for multiple testing correction and related statistical analyses in the R programming environment. ### Sensitivity Assessment After applying the primary correction method, assess the sensitivity of the findings to alternative methods. If the same proteins are significant under Bonferroni and FDR correction, the findings are robust. If significance depends heavily on the correction method, interpret the results with caution and consider whether the uncertainty reflects genuine limitations in the data. ### Reporting Report the correction method, the total number of tests, the significance threshold, and the software used in all publications. Include both uncorrected p-values and corrected q-values in supplementary tables. This transparency allows readers to evaluate the robustness of the findings and to apply alternative correction methods if they disagree with the authors' choices. ## Troubleshooting the Decision Framework Several common problems can undermine the decision framework and produce inappropriate correction choices. ### Defaulting to a Single Method Using the same correction method for all experiments without considering the experimental context is a common error. The appropriate method depends on the number of tests, the validation capacity, and the consequences of false positives versus false negatives. The decision framework should be applied to each experiment individually. ### Ignoring the Validation Context Selecting a stringent correction method when validation is planned wastes statistical power and may eliminate true candidates. Conversely, selecting a lenient method when no validation is planned produces false claims that cannot be filtered out. The validation capacity should be a primary consideration in the decision. ### Applying Correction to Filtered Data Some researchers apply quality filters to remove poorly measured proteins before correction, then apply correction only to the remaining proteins. This approach is problematic because the filtering decisions are often based on the same data used for significance testing. The correction should be applied to the full set of tests that could potentially be declared significant. ### Misinterpreting the Decision Output The decision framework produces a correction method and threshold, but these choices do not guarantee that all significant findings are true. Even with stringent correction, some reported findings will be false positives. The framework should be used in conjunction with biological plausibility assessment and independent validation. ## Professional Escalation Criteria for the Decision Framework Consult a biostatistician or bioinformatics specialist when the decision framework produces uncertain results or when the experimental context is complex. ### Complex Study Designs Studies with multiple groups, longitudinal sampling, or nested designs require statistical approaches beyond simple correction methods. A specialist should be consulted to ensure that the analysis appropriately accounts for the study design and that the correction method is compatible with the overall analytical approach. ### Strong Correlation Structures When proteomics data contain large blocks of highly correlated proteins, standard correction methods may not provide adequate error control. A specialist can help determine whether permutation-based approaches or methods that account for correlation structure are more appropriate. ### Conflicting Results Across Methods When different correction methods produce substantially different conclusions, the reasons for the discrepancies should be investigated. A statistical consultant can help determine whether the discrepancies reflect genuine uncertainty in the data or errors in the analysis. ### Regulatory or Clinical Applications Proteomics findings intended for regulatory submission or clinical use require rigorous statistical analysis and documentation. Consultation with a biostatistician is essential to ensure that the analysis meets regulatory standards and that the correction method is appropriate for the intended use. ## Frequently Asked Questions ### What is the difference between a p-value and a q-value? A p-value represents the probability of observing a test statistic as extreme as the one measured, assuming no true effect exists. A q-value represents the minimum false discovery rate at which a test would be declared significant. For a single test, the q-value is the expected proportion of false positives among all tests declared significant at that threshold. Q-values are always equal to or larger than the corresponding p-values because they account for the multiple testing burden. ### When should I use Bonferroni correction instead of FDR? Use Bonferroni correction when the cost of any false positive is high and when you need to make definitive claims about individual proteins without follow-up validation. Use FDR when you are conducting discovery-oriented analysis and plan to validate candidate proteins in subsequent experiments. Bonferroni is appropriate for small protein panels, while FDR is better suited to experiments measuring hundreds or thousands of proteins. ### How do I choose the FDR threshold for my proteomics experiment? The choice of FDR threshold depends on the experimental goals and the downstream validation capacity. A threshold of 0.05 is common for confirmatory analyses, while thresholds of 0.10 or 0.20 may be appropriate for discovery analyses where candidates will be validated in independent cohorts. The study of plasma proteomics in type 2 diabetes used a q-value threshold of 0.2 for discovery, reflecting the intention to validate candidates in a large independent cohort. ### Why did my significant proteins disappear after multiple testing correction? Proteins that are significant at an uncorrected threshold may not survive correction because the correction accounts for the total number of tests performed. This outcome is expected when the number of tests is large and the effect sizes are modest. The study of cerebrospinal fluid proteomics in postoperative delirium identified 32 proteins at an uncorrected threshold, but none survived Benjamini-Hochberg correction due to the limited sample size. This result does not necessarily mean the proteins are biologically irrelevant, but it does indicate that the evidence is not strong enough to distinguish true effects from chance. ### Does multiple testing correction apply to targeted proteomics panels? Yes, correction applies whenever multiple statistical tests are performed, regardless of whether the proteins were measured using targeted or untargeted approaches. A targeted panel measuring 48 cytokines requires correction for 48 tests, just as an untargeted approach measuring 1,000 proteins requires correction for 1,000 tests. The study of serum proteomics in multiple sclerosis applied FDR correction to data from the Olink 48 Cytokine panel. ### Can I use different correction methods for different parts of my analysis? Using different correction methods for different parts of an analysis is generally not recommended because it complicates interpretation and can introduce bias. The correction method should be pre-specified in the analysis plan and applied consistently to all tests. If different correction methods are used for sensitivity analysis, the results should be compared and reported transparently. ### How does sample size affect multiple testing correction? Sample size affects statistical power, which determines the ability to detect true effects after correction. Larger sample sizes provide more power to detect modest effects and are more likely to produce findings that survive correction. The study of cerebrospinal fluid proteomics in delirium had a small sample size of 24 matched pairs, which contributed to the failure of findings to survive correction. Power calculations should be conducted before the experiment to ensure adequate sample sizes. ### What should I report in my publication regarding multiple testing correction? Report the correction method, the total number of tests, the significance threshold, and the software used for the analysis. Include both uncorrected p-values and corrected q-values in supplementary tables. This transparency allows readers to evaluate the robustness of the findings and to apply alternative correction methods if they disagree with the authors' choices. ## Related Bioinformatics Guides - [Genomic Data Analysis Tools: A Comparative Guide for Researchers](/knowledge/bioinformatics/genomic-data-analysis-tools-a-comparative-guide-for-researchers) - [Metagenomic Contamination Control: Best Practices for Clean Data](/knowledge/bioinformatics/metagenomic-contamination-control-best-practices-for-clean-data) - [Olink Proteomics: A Practical Guide to Panel Selection and Data Interpretation](/knowledge/bioinformatics/olink-proteomics-a-practical-guide-to-panel-selection-and-data-interpretation) - [Proteomics Mass Spectrometry: From Sample Preparation to Data Analysis](/knowledge/bioinformatics/proteomics-mass-spectrometry-from-sample-preparation-to-data-analysis) - [Spatial Proteomics vs. Single-Cell Proteomics: Choosing the Right Approach](/knowledge/bioinformatics/spatial-proteomics-vs-single-cell-proteomics-choosing-the-right-approach) ## References and Further Reading - [NCBI Data Resources](https://www.ncbi.nlm.nih.gov/). National Center for Biotechnology Information. - [EMBL-EBI Training](https://www.ebi.ac.uk/training). European Bioinformatics Institute. - [Bioconductor](https://bioconductor.org/). Bioconductor Project. - [Galaxy Training Network](https://training.galaxyproject.org/). Galaxy Project. - [nf-core Documentation](https://nf-co.re/docs). nf-core. - [The Carpentries Lessons](https://carpentries.org/lessons). The Carpentries. - [A Five-Plasma Protein-Based Algorithm for Predicting Incident CKD in Type 2 Diabetes.](https://pubmed.ncbi.nlm.nih.gov/41563825). Journal of the American Society of Nephrology : JASN, 2026. - [New serum soluble factors predicting inflammatory and non-inflammatory disability worsening in multiple sclerosis.](https://pubmed.ncbi.nlm.nih.gov/41425552). Frontiers in immunology, 2025. - [Aptamer-Based Proteomics Measuring Preoperative Cerebrospinal Fluid Protein Alterations Associated with Postoperative Delirium.](https://pubmed.ncbi.nlm.nih.gov/37759795). Biomolecules, 2023. - [Harnessing metabolomics and proteomics in a clinical trial for pulmonary arterial hypertension: insights from post-hoc analysis of the REHAB-PH trial.](https://pubmed.ncbi.nlm.nih.gov/41849831). EBioMedicine, 2026. - [Serum proteomic profiles of depressive subtypes.](https://pubmed.ncbi.nlm.nih.gov/27404283). Translational psychiatry, 2016. > This article is educational and does not replace validated analysis plans, institutional policy, clinical interpretation, or specialist review.