# Building a Custom PTM Search Workflow in MaxQuant: Configuring Modifications for Optimal Identification

MaxQuant is a computational platform for mass spectrometry-based shotgun proteomics, and its default configuration supports common post-translational modifications (PTMs) such as phosphorylation, acetylation, and oxidation. Default settings are not optimized for every biological question. Researchers studying less common modifications, large protein-based modifiers, or unusual amino acid combinations often find that standard parameter files produce poor identification rates or high false discovery rates. This article explains how to build a custom PTM search workflow in MaxQuant, covering modification configuration, localization scoring, false discovery rate (FDR) control, and practical validation steps. The guidance applies to biology students, researchers, laboratory professionals, and life-science practitioners who generate their own proteomics data or analyze public datasets.

## Understanding MaxQuant Search Architecture and Modification Handling

MaxQuant operates through a pipeline that includes peak detection, peptide identification via the Andromeda search engine, protein assembly, and quantification. The Andromeda search engine matches tandem mass spectra against a protein sequence database, and its scoring accounts for the mass shifts introduced by modifications. When you configure a variable modification, you instruct Andromeda to consider a specific mass delta at a specific amino acid residue, and the search space expands accordingly.

The key distinction in MaxQuant is between fixed modifications and variable modifications. Fixed modifications are applied to every occurrence of the specified residue, such as carbamidomethylation of cysteine after alkylation. Variable modifications are searched as optional mass shifts, meaning the engine must decide for each peptide whether the modification is present. Each variable modification increases the search space combinatorially, which affects both search time and statistical confidence.

For PTM-focused experiments, the standard workflow involves setting phosphorylation of serine, threonine, and tyrosine as variable modifications, with a neutral loss option for phosphopeptide fragmentation. This configuration works well for phosphoproteomics because phosphorylation produces characteristic neutral loss peaks in collision-induced dissociation spectra. Other modifications require different configurations. Acetylation of lysine and N-terminal residues, ubiquitination remnants (diglycine on lysine), and methylation of arginine or lysine each have distinct mass shifts and fragmentation behaviors.

The choice of modification set directly influences identification performance. A search with too many variable modifications dilutes the statistical power of the scoring algorithm because the engine must evaluate more candidate peptides. A search with too few modifications misses biologically relevant PTMs. The practical approach is to design separate search workflows for different biological questions instead of attempting one comprehensive search.

## At a Glance: Key Configuration Decisions for PTM Searches in MaxQuant

| Configuration Decision | Default Setting | Custom PTM Consideration | Practical Recommendation |
| --- | --- | --- | --- |
| Variable modifications | Phospho (STY), Oxidation (M), Acetyl (Protein N-term) | Add only modifications supported by your biological hypothesis or preliminary evidence | Start with 3 to 5 variable modifications, expand only after evaluating initial identification rates |
| Fixed modifications | Carbamidomethyl (C) | Keep fixed modifications constant unless your sample preparation uses different alkylation chemistry | Verify that the fixed modification matches your digestion and alkylation protocol |
| FDR control | 1% peptide and protein FDR | Tighten to 0.1% for PTM site-level claims or when validating novel modifications | Report both peptide-level and site-level FDR separately |
| Localization scoring | PTM score and localization probability calculated automatically | Enable site localization probability output and inspect scores for ambiguous sites | Use a localization probability threshold of 0.75 for confident site assignment |
| Enzyme specificity | Trypsin with up to 2 missed cleavages | Adjust for non-trypsin proteases or higher missed cleavage tolerance for modified peptides | Document enzyme choice in methods, consider semi-tryptic search for unusual modifications |

## Core Principles of Modification Configuration

### Mass Shift Accuracy and Elemental Composition

Every modification is defined by a monoisotopic mass shift. For common modifications, MaxQuant provides built-in definitions with accurate mass values. For custom modifications, you must calculate the mass shift from the elemental composition of the added group. A mass error of even 0.01 Da can reduce identification sensitivity because the precursor mass tolerance in MaxQuant is typically set to 4.5 ppm for Orbitrap data and 20 ppm for quadrupole time-of-flight data.

When defining a custom modification, verify the mass shift against public databases such as the [NCBI Data Resources](https://www.ncbi.nlm.nih.gov/) for sequence and structure information, and consult training materials from [EMBL-EBI Training](https://www.ebi.ac.uk/training) for guidance on modification mass calculations. The practical check is to apply the modification to a known standard peptide and confirm that the observed precursor mass matches the calculated value within instrument tolerance.

### Residue Specificity and Amino Acid Context

Modifications are residue-specific. Phosphorylation occurs on serine, threonine, and tyrosine. Acetylation occurs on lysine and protein N-termini. Methylation occurs on lysine and arginine. Ubiquitination leaves a diglycine remnant on lysine after trypsin digestion. When configuring a modification, you must specify the correct residue or residue set. Incorrect residue assignment leads to missed identifications because the search engine will not consider the modification at the actual site.

Some modifications show context-dependent behavior. Phosphorylation of serine and threonine is common, but tyrosine phosphorylation is less frequent and may require higher scoring thresholds for confident identification. The MaxQuant localization scoring algorithm evaluates the probability that a modification is assigned to a specific residue within a peptide, and this probability depends on the fragmentation pattern observed in the tandem mass spectrum.

### Fragmentation Behavior and Neutral Losses

The fragmentation behavior of a modified peptide affects its identification. Phosphorylated peptides often lose phosphoric acid (98 Da) or phosphate (80 Da) during collision-induced dissociation, producing neutral loss peaks that can dominate the spectrum. MaxQuant includes options for specifying neutral losses for variable modifications, and enabling these options improves identification of phosphopeptides.

Large protein-based modifiers, such as Small Ubiquitin-like Modifier (SUMO) and ubiquitin, fragment during tandem mass spectrometry and produce diagnostic ions that are not accounted for by standard search engines. Recent work has shown that a sequence-based modifier search strategy within MaxQuant can account for the fragmentation behavior of these large modifiers, improving identification of SUMOylated peptides from human cell lines by approximately 13% and SUMOylation sites in mouse embryonic cells by approximately 22% compared to standard searches. This approach relies on identifying diagnostic ions and PTM ions that are characteristic of the modifier fragmentation pattern.

## Practical Workflow for Custom PTM Configuration

### Step 1: Define the Biological Question and Modification Hypothesis

Before configuring MaxQuant, write down the specific PTM you intend to study and the residue or residues it targets. Consult the published literature for your organism and sample type to identify which modifications have been reported. If you are studying a novel modification, search public repositories for evidence of its existence and mass shift.

The [NCBI Data Resources](https://www.ncbi.nlm.nih.gov/) provide access to protein sequences and functional annotations that can help you predict which residues are likely to be modified. The [EMBL-EBI Training](https://www.ebi.ac.uk/training) portal offers structured learning pathways for bioinformatics analysis, including modules on proteomics data analysis and modification identification. These resources help you build a defensible modification hypothesis before you spend computational time on a search.

### Step 2: Prepare the Protein Sequence Database

MaxQuant requires a protein sequence database in FASTA format. The database should contain the complete proteome of your organism, including isoforms and common contaminants. For human samples, include both canonical and isoform sequences. For cross-species experiments, such as patient-derived xenografts, include both human and mouse sequences to enable species-specific peptide assignment.

Database quality directly affects PTM identification. A database that is too small misses peptides from unannotated isoforms. A database that is too large increases search time and may inflate false discovery rates. The practical approach is to download the reference proteome from a public repository and add common contaminants such as keratins and trypsin.

### Step 3: Configure Variable Modifications in the Parameter File

In the MaxQuant graphical interface, navigate to the Modifications tab. Select the variable modifications that match your hypothesis. For each modification, verify the following parameters:

- Modification name and mass shift
- Residue specificity
- Neutral loss settings if applicable
- Whether the modification is applied to peptide N-terminus, protein N-terminus, or internal residues

For custom modifications not present in the built-in list, use the "Add" function to define a new modification. Enter the monoisotopic mass shift calculated from the elemental composition. Specify the residue or residues. If the modification affects fragmentation, add the appropriate neutral loss masses.

### Step 4: Set Search Parameters for PTM Sensitivity

The search parameters in MaxQuant include precursor mass tolerance, fragment mass tolerance, enzyme specificity, and missed cleavage allowance. For PTM searches, consider the following adjustments:

- Precursor mass tolerance: 4.5 ppm for high-resolution Orbitrap data, 20 ppm for Q-TOF data
- Fragment mass tolerance: 20 ppm for HCD spectra, 0.5 Da for CID spectra
- Enzyme: Trypsin with up to 2 missed cleavages as a starting point
- Variable modifications: Limit to 3 to 5 to control search space

The Andromeda score threshold is automatically adjusted based on the FDR setting. For PTM site-level claims, use a more stringent FDR, such as 0.1%, to reduce the risk of incorrect site assignment.

### Step 5: Enable Localization Scoring and Site Probability Output

MaxQuant calculates a localization probability for each modified site. This probability reflects the confidence that the modification is assigned to the correct residue within the peptide. Enable the output of localization probabilities in the parameter file. After the search, inspect the modificationSpecificPeptides.txt file, which contains the localization probability for each modified peptide.

For phosphorylation, a localization probability above 0.75 is commonly used as a threshold for confident site assignment. For other modifications, the appropriate threshold depends on the fragmentation quality and the number of potential sites in the peptide. Peptides with multiple potential modification sites require higher localization probabilities for confident assignment.

### Step 6: Run the Search and Evaluate Identification Metrics

Execute the MaxQuant search and monitor the output. The key output files for PTM analysis include:

- msms.txt: contains all identified peptide-spectrum matches
- modificationSpecificPeptides.txt: contains modified peptide identifications with localization scores
- Phospho (STY)Sites.txt: contains phosphorylation site quantifications (when phosphorylation is configured)
- Evidence.txt: contains all peptide evidence with modification information

Evaluate the number of identified modified peptides and the distribution of localization probabilities. A successful PTM search typically identifies thousands of modified peptides from a standard proteomics experiment. If the number is unexpectedly low, review your modification configuration and search parameters.

## Options and Tradeoffs in PTM Search Design

### Single Search Versus Multiple Searches

One search with many variable modifications is computationally expensive and statistically less powerful than multiple targeted searches. The combinatorial expansion of modification types and sites is a known limitation of database searching for modified peptides. A practical strategy is to run an initial search with common modifications to establish a baseline of protein identifications, then run targeted searches for specific PTMs of interest.

For example, a phosphoproteomics experiment might use one search with phosphorylation as the only variable modification, plus oxidation of methionine as a common artifact. A separate search for acetylation would include acetyl on lysine and protein N-terminus. This approach keeps the search space manageable and improves the statistical confidence for each modification class.

### Open Search Versus Closed Search

Closed searches restrict the search space to a predefined set of modifications and amino acid substitutions. Open searches allow a wider mass tolerance to detect unexpected modifications. Recent benchmarking using patient-derived xenograft proteomics data showed that open search approaches can achieve greater than 65% sensitivity and greater than 70% precision for detecting amino acid substitutions in high-quality samples. The same study found that closed searches targeting predefined amino acid changes exhibited poor precision, with PTM mislocalization identified as a key bottleneck.

For PTM discovery, an open search can reveal unexpected modifications, but the results require careful validation. For targeted PTM analysis, a closed search with a well-defined modification set provides more reliable quantification.

### Database Search Versus De Novo Sequencing

Database searching remains the standard for identifying modified peptides from tandem mass spectra, but it is limited by the combinatorial expansion of modification types and sites. De novo sequencing offers an alternative that does not require a predefined modification set. Recent developments in de novo sequencing models have expanded the range of detectable PTMs. One model, trained on a large dataset of spectra from endogenous and synthetic peptides, covers 19 biologically relevant amino acid-PTM combinations including phosphorylation, acetylation, and ubiquitination, and achieves robust performance across these combinations while maintaining performance on unmodified peptides.

For MaxQuant users, de novo sequencing is not a built-in feature. De novo tools can complement MaxQuant results by identifying modified peptides that database search misses. The practical workflow is to run MaxQuant for quantitative analysis and use de novo sequencing for discovery of unexpected modifications.

## Observations and Measurements for PTM Search Validation

### Diagnostic Ion Monitoring

For large protein-based modifiers, diagnostic ions provide evidence of modification presence. The sequence-based modifier search strategy identifies distinct diagnostic features and characteristic mass shifts associated with modifier fragmentation. These features, referred to as diagnostic ions and PTM ions, improve spectral annotation and increase identification rates.

When validating a custom PTM search, inspect the tandem mass spectra of identified modified peptides for the presence of diagnostic ions. If the modification produces characteristic fragment ions, their presence in the spectrum supports the identification. Their absence does not rule out the modification, but it reduces confidence.

### Localization Probability Distributions

After a search, examine the distribution of localization probabilities across identified modified peptides. A healthy distribution shows a majority of peptides with high localization probability (above 0.9) and a tail of lower-confidence assignments. If most peptides have localization probabilities below 0.75, the fragmentation quality may be insufficient for confident site assignment, or the modification may be misconfigured.

The localization probability distribution also helps identify systematic errors. If a specific residue position consistently shows low localization probability, the modification may be assigned to the wrong residue in the configuration.

### Andromeda Score Distributions

The Andromeda score reflects the quality of the peptide-spectrum match. For modified peptides, the score distribution should overlap with that of unmodified peptides, though modified peptides often score slightly lower due to the additional search space. The sequence-based modifier search approach has been shown to improve spectral annotation of protein-based modifiers by up to 9% increase in the median Andromeda score.

Compare the Andromeda score distributions between modified and unmodified peptides in your search results. A large gap between the distributions suggests that the modification configuration is suboptimal or that the modification produces poor fragmentation.

## Records and Documentation for Reproducible PTM Searches

### Parameter File Documentation

MaxQuant generates a parameter file (mqpar.xml) for each search. Archive this file with your raw data and results. The parameter file contains all search settings, including modification definitions, enzyme specificity, tolerances, and FDR settings. Without this file, the search cannot be reproduced exactly.

For publication, include the parameter file as supplementary material or deposit it in a public repository. The [nf-core Documentation](https://nf-co.re/docs) emphasizes the importance of pipeline configuration and usage documentation for reproducibility, and the same principle applies to MaxQuant searches.

### Modification Definition Records

For custom modifications, maintain a record of the mass shift calculation, the elemental composition, and the residue specificity. This record should include the source of the modification information, such as a published paper or a public database. If the modification is novel, document the evidence supporting its existence.

The [Galaxy Training Network](https://training.galaxyproject.org/) provides accessible workflow training that emphasizes documentation and reproducibility. Apply the same standards to your MaxQuant parameter files and modification definitions.

### Search Version and Software Environment

Record the MaxQuant version and the operating system used for the search. Different versions of MaxQuant may produce slightly different results due to algorithm updates. The Andromeda search engine has been updated over time, and version-specific behavior affects identification rates.

Also record the version of the protein sequence database and the date of download. Databases are updated regularly, and the version affects which peptides can be identified.

## Common Failure Patterns in PTM Searches

### Failure Pattern 1: Too Many Variable Modifications

Adding many variable modifications to a single search dilutes statistical power. The search engine must evaluate more candidate peptides, and the FDR calculation accounts for the increased search space. The result is fewer identified modified peptides and lower confidence in the identifications that are made.

**Correction:** Limit variable modifications to 3 to 5 per search. Run separate searches for different modification classes.

### Failure Pattern 2: Incorrect Residue Specificity

Specifying the wrong residue for a modification causes the search engine to miss modified peptides. For example, configuring methylation on lysine only will miss arginine methylation. Some modifications occur on multiple residues, and the configuration must include all relevant residues.

**Correction:** Verify the residue specificity of each modification against published literature or public databases before running the search.

### Failure Pattern 3: Ignoring Neutral Losses

For modifications that produce characteristic neutral losses during fragmentation, failing to configure neutral loss options reduces identification sensitivity. Phosphorylation is the most common example, but other modifications also produce neutral losses.

**Correction:** Enable neutral loss options for modifications known to fragment in this way. Review the literature for your modification of interest.

### Failure Pattern 4: Inappropriate FDR Thresholds

Using a single FDR threshold for all levels of identification can lead to incorrect PTM site claims. Peptide-level FDR and site-level FDR are different metrics. A peptide may be confidently identified, but the modification site assignment within that peptide may be uncertain.

**Correction:** Report peptide-level and site-level FDR separately. Use a more stringent threshold for site-level claims.

### Failure Pattern 5: Poor Database Quality

An incomplete or contaminated protein sequence database reduces identification rates. Missing isoforms or species-specific sequences cause peptides to be unmatched. Contaminant sequences inflate the number of identifications but reduce the biological relevance.

**Correction:** Download the reference proteome from a public repository, add common contaminants, and verify the database contains the expected proteins for your sample.

### Failure Pattern 6: Ignoring Localization Probabilities

Reporting modified sites without inspecting localization probabilities leads to incorrect biological conclusions. A modification assigned to the wrong residue within a peptide produces misleading site-level results.

**Correction:** Filter modified peptides by localization probability and report the threshold used. Inspect spectra for ambiguous site assignments.

## Limitations of Custom PTM Searches in MaxQuant

### Search Space Explosion

The combinatorial expansion of modification types and sites is a fundamental limitation of database searching for modified peptides. Each additional variable modification multiplies the number of candidate peptides. For large modifications or modifications with many potential sites, the search space becomes computationally prohibitive.

### Fragmentation Behavior of Large Modifiers

Standard search engines are optimized for small, non-fragmenting modifications. Large protein-based modifiers such as SUMO and ubiquitin fragment during tandem mass spectrometry, producing complex spectra that standard scoring algorithms handle poorly. The sequence-based modifier search strategy addresses this limitation for specific modifiers, but it requires specialized configuration and validation.

### Isobaric Modifications

Some modifications have identical mass shifts, making them indistinguishable by mass alone. Isobaric modifications require additional evidence, such as fragmentation patterns or retention time behavior, for discrimination. MaxQuant does not automatically distinguish isobaric modifications.

### Quantification Challenges

PTM quantification is complicated by the fact that modified peptides are often present at lower abundance than unmodified peptides. Isobaric labeling strategies such as tandem mass tag (TMT) and isobaric tags for relative and absolute quantitation (iTRAQ) enable multiplexed quantification, but the processing of PTM quantification results requires specialized tools. Recent work has shown that alternative computational platforms can quantify more phosphorylated sites and deliver more robust quantification performance compared to MaxQuant for TMT data, though MaxQuant remains a valid choice for label-free quantification.

## Quality Controls and Validation Steps

### Standard Peptide Validation

Before running a large-scale PTM search, validate your modification configuration using standard peptides with known modification sites. Synthetic modified peptides are available from commercial suppliers for common modifications such as phosphorylation and acetylation. Run these standards through your MaxQuant workflow and confirm that the expected modification sites are identified with high localization probability.

### Replicate Consistency

Run technical replicates of the same sample and compare PTM identification rates. Consistent identification across replicates indicates a stable workflow. High variability suggests problems with sample preparation, chromatography, or search configuration.

### Cross-Validation with Alternative Tools

Compare your MaxQuant results with results from an alternative search engine or computational platform. Discrepancies in identified modified peptides highlight areas where the search configuration may be suboptimal. The intersection of different search strategies enhances precision, though at the expense of reduced sensitivity.

### Spectral Visualization

Inspect the tandem mass spectra of identified modified peptides to confirm the modification assignment. Look for fragment ions that support the modification site. For phosphorylated peptides, the presence of neutral loss peaks supports the identification. For large modifiers, the presence of diagnostic ions supports the assignment.

## Safety and Regulatory Context for PTM Data Analysis

### Data Management and Reproducibility

Proteomics data analysis must follow reproducible research practices. Archive raw data, parameter files, and analysis scripts. Deposit processed data in public repositories where appropriate. The [Carpentries Lessons](https://carpentries.org/lessons) provide foundational training in data management and reproducible analysis practices that apply to proteomics workflows.

### Reporting Standards

When reporting PTM identifications, include the search parameters, database version, FDR thresholds, and localization probability thresholds. This information allows other researchers to evaluate the confidence of your identifications and reproduce your analysis. The [nf-core Documentation](https://nf-co.re/docs) emphasizes the importance of pipeline configuration documentation, and the same principle applies to MaxQuant parameter reporting.

### Professional Escalation Criteria

Consult a bioinformatics specialist or proteomics core facility when:

- The number of identified modified peptides is unexpectedly low despite correct configuration
- Localization probabilities are consistently below 0.75 across many peptides
- The search time is excessive due to a large modification set
- You need to identify a novel modification without published mass shift information
- You are working with large protein-based modifiers and standard searches fail

## Building a Decision Framework for PTM Search Configuration Based on Evidence Quality

Selecting the right modification set for a MaxQuant search requires more than listing biologically plausible PTMs. The configuration decision should follow a structured evaluation of evidence quality, expected modification abundance, and the fragmentation characteristics of each candidate modification. This section provides a practical decision framework that separates modifications into tiers based on the strength of supporting evidence and the likelihood of successful identification.

### Tier Classification for Candidate Modifications

Before configuring any modification in MaxQuant, classify each candidate PTM into one of three evidence tiers. This classification prevents the common failure of adding too many variable modifications to a single search and provides a defensible rationale for modification selection.

**Tier 1 modifications** have direct experimental evidence in your sample type, published mass spectrometry identifications in a similar biological context, and known fragmentation behavior that MaxQuant can score effectively. Phosphorylation of serine, threonine, and tyrosine in a phosphoproteomics experiment is a Tier 1 modification when you have performed phosphopeptide enrichment. Acetylation of lysine in a study of histone modifications is Tier 1 when you have evidence from antibody-based enrichment or published datasets from the same cell type.

**Tier 2 modifications** have indirect evidence such as antibody detection without site-specific mass spectrometry confirmation, homology-based predictions from related organisms, or published identifications in a different tissue or condition. These modifications warrant inclusion only when the search space remains manageable. For example, methylation of lysine or arginine in a study where you have antibody evidence but no prior mass spectrometry data from your specific sample would be Tier 2.

**Tier 3 modifications** are speculative candidates with no direct evidence in your system, no published mass spectrometry identifications in a related context, or known poor fragmentation behavior that standard search engines handle inadequately. Large protein-based modifiers such as Small Ubiquitin-like Modifier (SUMO) and ubiquitin remnants fall into this tier for standard MaxQuant searches because their fragmentation behavior requires specialized search strategies. Recent work has demonstrated that a sequence-based modifier search strategy within MaxQuant can account for the fragmentation behavior of these large modifiers, improving identification of SUMOylated peptides from human cell lines by approximately 13% and SUMOylation sites in mouse embryonic cells by approximately 22% compared to standard searches. However, this approach requires specialized configuration and validation that goes beyond standard parameter selection.

The tier classification directly informs search design. Run Tier 1 modifications in your primary search. Add Tier 2 modifications only if the primary search produces acceptable identification rates and the computational time remains practical. Exclude Tier 3 modifications from routine searches and design dedicated experiments for them instead.

### Evidence Quality Assessment for Modification Selection

For each candidate modification, evaluate five evidence dimensions before adding it to the search configuration. Record your assessment in a modification decision log that accompanies your parameter file.

**Dimension 1: Direct mass spectrometry evidence.** Search public repositories and published literature for prior identifications of the modification in your organism or a closely related one. The [NCBI Data Resources](https://www.ncbi.nlm.nih.gov/) provide access to protein sequence databases and associated functional annotations that can help you assess whether a modification has been reported for your proteins of interest. The [EMBL-EBI Training](https://www.ebi.ac.uk/training) portal offers structured learning pathways for bioinformatics analysis that include guidance on searching public proteomics repositories and interpreting modification evidence.

**Dimension 2: Biochemical plausibility.** Evaluate whether the enzymatic machinery for the modification exists in your sample. For example, lysine acetylation requires acetyltransferases, and their expression in your cell type affects the likelihood of detecting acetylated peptides. This assessment draws on your biological knowledge and the published literature for your system.

**Dimension 3: Expected stoichiometry and abundance.** Modifications that occur at low stoichiometry produce fewer modified peptides, and the search may not detect them even with correct configuration. If published data suggest that a modification affects less than 1% of the target protein population, consider whether your sample preparation includes an enrichment step. Without enrichment, low-abundance modifications are unlikely to be identified regardless of search settings.

**Dimension 4: Fragmentation behavior compatibility.** Assess whether the modification produces fragment ions that Andromeda can score. Small modifications that remain intact during fragmentation are well suited to standard searches. Modifications that fragment themselves, such as large protein-based modifiers, produce complex spectra that standard scoring handles poorly. The sequence-based modifier search strategy addresses this limitation for specific modifiers by identifying distinct diagnostic features and characteristic mass shifts associated with modifier fragmentation, referred to as diagnostic ions and PTM ions.

**Dimension 5: Mass shift uniqueness.** Determine whether the modification mass shift is unique or overlaps with other modifications or amino acid substitutions. Isobaric modifications with identical mass shifts cannot be distinguished by mass alone and require additional evidence such as fragmentation patterns or retention time behavior. If your candidate modification shares a mass shift with a more common modification, the search may assign spectra to the wrong modification.

### Modification Decision Log Template

Maintain a structured record for each modification considered for your search. This log serves as documentation for reproducibility and provides a reference when troubleshooting poor identification rates.

| Modification | Residue | Mass Shift (Da) | Evidence Tier | Direct MS Evidence | Biochemical Plausibility | Expected Abundance | Fragmentation Compatibility | Mass Shift Uniqueness | Decision |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Phosphorylation | STY | +79.966 | 1 | Published in similar samples | Kinases expressed | High after enrichment | Neutral losses configured | Unique | Include |
| Acetylation | K | +42.011 | 1 | Published in similar samples | Acetyltransferases expressed | Moderate | Stable during fragmentation | Unique | Include |
| Methylation | K | +14.016 | 2 | Antibody evidence only | Methyltransferases expressed | Low | Stable during fragmentation | Overlaps with oxidation on some residues | Evaluate after primary search |
| SUMO remnant | K | +343.206 | 3 | Published in specialized studies | SUMO pathway active | Very low | Fragments during MS | Unique | Dedicated search required |

Record the source of evidence for each decision, including database identifiers, publication references, or preliminary experimental data. The [Galaxy Training Network](https://training.galaxyproject.org/) provides accessible workflow training that emphasizes documentation and reproducibility, and the same standards apply to modification selection records.

### Search Space Budgeting and Modification Prioritization

Each variable modification added to a MaxQuant search multiplies the candidate peptide space. The computational cost and statistical penalty grow with each addition, so a practical budget helps prioritize modifications.

**Step 1: Define your search space budget.** For a standard proteomics experiment analyzed on a typical workstation, limit the total number of variable modifications to 3 to 5. This budget accommodates the most common PTMs while keeping search times practical and statistical power acceptable.

**Step 2: Allocate budget by evidence tier.** Assign the first 2 to 3 slots to Tier 1 modifications with the strongest evidence and highest expected abundance. Reserve 1 to 2 slots for Tier 2 modifications that address specific biological questions. Do not allocate slots to Tier 3 modifications in a routine search.

**Step 3: Consider modification combinations.** Some modifications co-occur or share regulatory pathways. For example, phosphorylation and acetylation often regulate the same proteins. If your biological question concerns crosstalk between modifications, include both in the same search to enable detection of co-modified peptides. This consideration may override the tier classification when the biological question requires it.

**Step 4: Evaluate the budget after the first search.** Run the search with the initial modification set and evaluate identification rates. If the number of identified modified peptides is unexpectedly low, review the modification configuration before adding more modifications. Adding modifications to a failing search rarely solves the underlying problem.

### Comparison of Search Strategies for Different Evidence Scenarios

The evidence tier classification maps to specific search strategies. Match the strategy to your evidence quality and biological question.

**Scenario 1: High-confidence modification with established fragmentation behavior.** Use a closed search with the modification configured as a variable modification with appropriate neutral loss settings. This scenario applies to phosphorylation in enriched samples, acetylation in histone studies, and oxidation as a common artifact. The closed search restricts the search space to the predefined modification set and provides reliable quantification.

**Scenario 2: Multiple candidate modifications with moderate evidence.** Run separate closed searches for each modification class instead of combining them in one search. This approach keeps the search space manageable and improves statistical confidence for each modification. For example, run one search with phosphorylation as the only variable modification and a separate search with acetylation. Compare the identification rates and localization probability distributions between searches to assess which modification is better supported by the data.

**Scenario 3: Discovery of unexpected modifications.** Use an open search with a wider mass tolerance to detect modifications you did not anticipate. Recent benchmarking using patient-derived xenograft proteomics data showed that open search approaches can achieve greater than 65% sensitivity and greater than 70% precision for detecting amino acid substitutions in high-quality samples. The same study found that closed searches targeting predefined amino acid changes exhibited poor precision, with PTM mislocalization identified as a key bottleneck. Open search results require careful validation because the wider tolerance increases the risk of false identifications.

**Scenario 4: Large protein-based modifiers.** Standard MaxQuant searches do not account for the fragmentation behavior of large modifiers such as SUMO and ubiquitin. The sequence-based modifier search strategy within MaxQuant addresses this limitation by accounting for the fragmentation behavior of these modifiers during peptide identification. This approach identified distinct diagnostic features and characteristic mass shifts associated with modifier fragmentation and improved spectral annotation of protein-based modifiers by up to 9% increase in the median Andromeda score. If you study these modifiers, consult the published literature for the specific configuration parameters and validate the search with known positive controls.

### Decision Points for Search Reconfiguration

Monitor specific metrics after each search to determine whether the configuration requires adjustment. These decision points provide objective criteria for reconfiguration instead of relying on subjective impressions of search quality.

**Decision Point 1: Modified peptide identification rate.** Calculate the number of modified peptides identified per 1000 total peptide identifications. If this rate falls below 5% for a modification you expect to be abundant, review the configuration. For phosphorylation in enriched samples, the rate should be substantially higher, often exceeding 50% of all identifications.

**Decision Point 2: Localization probability distribution.** Examine the distribution of localization probabilities for each modification class. If more than 30% of modified peptides have localization probabilities below 0.75, the fragmentation quality may be insufficient for confident site assignment. Consider whether the modification produces diagnostic fragment ions that support site localization.

**Decision Point 3: Andromeda score comparison.** Compare the Andromeda score distributions between modified and unmodified peptides. A large gap between the distributions suggests that the modification configuration is suboptimal or that the modification produces poor fragmentation. The sequence-based modifier search approach has been shown to improve spectral annotation of protein-based modifiers by up to 9% increase in the median Andromeda score, demonstrating that configuration changes can directly affect scoring quality.

**Decision Point 4: False discovery rate stability.** Monitor whether the FDR remains stable across biological replicates. If the FDR fluctuates widely between replicates, the search may be operating at the edge of statistical power. Consider reducing the number of variable modifications or increasing the stringency of the FDR threshold.

### Practical Implementation Steps for the Decision Framework

**Step 1: Create a modification candidate list.** List all PTMs relevant to your biological question, including those with direct evidence, indirect evidence, and speculative support. For each modification, record the residue specificity, mass shift, and known fragmentation behavior.

**Step 2: Classify each modification into evidence tiers.** Apply the five evidence dimensions to each candidate. Assign Tier 1, Tier 2, or Tier 3 status based on the strength of evidence across all dimensions.

**Step 3: Allocate search space budget.** Select the modifications for your primary search based on tier classification and the 3 to 5 modification budget. Document the rationale for each selection in your modification decision log.

**Step 4: Configure and run the primary search.** Enter the selected modifications in the MaxQuant parameter file with correct residue specificity, mass shifts, and neutral loss settings. Run the search and record the identification metrics.

**Step 5: Evaluate against decision points.** Compare the search results against the four decision points described above. If the search meets the criteria, proceed with downstream analysis. If not, use the decision points to identify which configuration parameter requires adjustment.

**Step 6: Iterate with targeted searches.** For Tier 2 modifications that did not make the primary search budget, run separate targeted searches after evaluating the primary search results. For Tier 3 modifications, design dedicated experiments with appropriate enrichment and specialized search strategies.

### Records and Measurements for Configuration Decisions

Maintain the following records to support configuration decisions and enable troubleshooting:

**Modification decision log.** Record the evidence tier, evidence dimensions, and selection decision for each modification considered. Include the source of evidence for each assessment.

**Search parameter history.** Archive the parameter file for each search iteration, including the modification set, tolerances, enzyme specificity, and FDR settings. The [nf-core Documentation](https://nf-co.re/docs) emphasizes the importance of pipeline configuration and usage documentation for reproducibility, and the same principle applies to MaxQuant parameter tracking.

**Identification metrics summary.** Record the number of identified modified peptides, localization probability distributions, Andromeda score distributions, and FDR values for each search. This summary enables comparison across search iterations and biological replicates.

**Troubleshooting notes.** Document any configuration changes made in response to poor identification rates or other issues. Include the observed problem, the change made, and the effect on identification metrics.

### Common Configuration Decision Errors

**Error 1: Including Tier 3 modifications in a routine search.** Adding speculative modifications to a standard search dilutes statistical power and rarely produces confident identifications. Reserve Tier 3 modifications for dedicated experiments with appropriate enrichment and validation.

**Error 2: Ignoring fragmentation behavior in modification selection.** Modifications that fragment during tandem mass spectrometry produce complex spectra that standard scoring handles poorly. Selecting modifications without considering fragmentation behavior leads to poor identification rates even when the modification is present in the sample.

**Error 3: Failing to document modification selection rationale.** Without a documented decision log, you cannot reproduce your search configuration or explain your modification choices to reviewers. The modification decision log provides the evidence basis for your configuration.

**Error 4: Adding modifications to compensate for poor enrichment.** If your sample preparation did not enrich for the modification of interest, adding more variable modifications will not compensate. Low-abundance modifications require enrichment before mass spectrometry analysis, not additional search configurations.

**Error 5: Using the same modification set for different biological questions.** A modification set optimized for phosphoproteomics is not appropriate for studying acetylation or methylation. Design the modification set for each biological question based on the evidence tier classification.

### Professional Escalation Criteria for Configuration Decisions

Consult a bioinformatics specialist or proteomics core facility when:

- The evidence tier classification is unclear for a modification central to your biological question
- Multiple Tier 1 modifications exceed the search space budget and you need guidance on prioritization
- The decision points indicate poor search performance but the cause is not identifiable from the configuration
- You need to study a large protein-based modifier and standard searches fail
- You are working with a novel modification without published mass spectrometry evidence

The [Carpentries Lessons](https://carpentries.org/lessons) provide foundational training in data management and reproducible analysis practices that apply to proteomics workflows, and the [Bioconductor](https://bioconductor.org/) project offers packages for downstream analysis of proteomics results that can help evaluate search quality. These resources support the decision framework described here and help you build defensible modification configurations for your PTM searches.

## Frequently Asked Questions

### How do I calculate the mass shift for a custom modification in MaxQuant?

Calculate the monoisotopic mass shift from the elemental composition of the added group. Sum the monoisotopic masses of each atom in the added group, then subtract the mass of any atoms removed during the modification reaction. Verify the calculated mass against published values or public databases. Apply the modification to a known standard peptide and confirm the observed precursor mass matches the calculated value within instrument tolerance.

### What is the difference between fixed and variable modifications in MaxQuant?

Fixed modifications are applied to every occurrence of the specified residue. Variable modifications are searched as optional mass shifts, and the search engine decides for each peptide whether the modification is present. Fixed modifications do not increase the search space, while each variable modification increases the search space combinatorially. Use fixed modifications for complete chemical modifications such as cysteine alkylation, and variable modifications for biologically variable PTMs.

### How many variable modifications should I include in a single MaxQuant search?

Limit variable modifications to 3 to 5 per search. Each additional variable modification increases the search space and dilutes statistical power. Run separate searches for different modification classes. For example, use one search for phosphorylation and a separate search for acetylation.

### What localization probability threshold should I use for confident PTM site assignment?

A localization probability of 0.75 is commonly used as a threshold for confident site assignment in phosphorylation studies. For other modifications, the appropriate threshold depends on the fragmentation quality and the number of potential sites in the peptide. Peptides with multiple potential modification sites require higher localization probabilities. Report the threshold used in your methods.

### How do I configure MaxQuant for large protein-based modifiers such as SUMO?

Standard MaxQuant configurations do not account for the fragmentation behavior of large protein-based modifiers. Recent work has developed a sequence-based modifier search strategy within MaxQuant that accounts for the fragmentation behavior of these modifiers during peptide identification. This approach identifies diagnostic ions and PTM ions characteristic of the modifier fragmentation pattern. Consult the published literature for the specific configuration parameters for your modifier of interest.

### Why does my PTM search identify very few modified peptides?

Low identification rates can result from too many variable modifications, incorrect residue specificity, missing neutral loss options, poor database quality, or inappropriate FDR thresholds. Review each configuration parameter systematically. Validate your modification configuration using standard peptides with known modification sites.

### Can MaxQuant distinguish between isobaric modifications?

MaxQuant cannot distinguish between modifications with identical mass shifts by mass alone. Isobaric modifications require additional evidence such as fragmentation patterns or retention time behavior for discrimination. If you are studying isobaric modifications, consider using alternative search strategies or additional validation experiments.

### Should I use open search or closed search for PTM identification?

Closed searches restrict the search space to a predefined set of modifications and provide more reliable quantification for targeted PTM analysis. Open searches allow a wider mass tolerance to detect unexpected modifications but require careful validation. Recent benchmarking shows that open search approaches can achieve high sensitivity and precision for detecting amino acid substitutions, but closed searches targeting predefined changes exhibit poor precision due to PTM mislocalization. Use closed searches for targeted analysis and open searches for discovery.

## Related Bioinformatics Guides

- [Single-Cell Annotation: A Workflow for Cell Type Identification](/knowledge/bioinformatics/single-cell-annotation-a-workflow-for-cell-type-identification)
- [Metabolomics Data Analysis in R: A Practical Workflow](/knowledge/bioinformatics/metabolomics-data-analysis-in-r-a-practical-workflow)
- [Bottom-Up Proteomics: Principles, Workflow, and Applications](/knowledge/bioinformatics/bottom-up-proteomics-principles-workflow-and-applications)
- [Genomic Surveillance for Antimicrobial Resistance: A Bioinformatics Workflow](/knowledge/bioinformatics/genomic-surveillance-for-antimicrobial-resistance-a-bioinformatics-workflow)
- [Metabolomics Data Analysis Workflow: From Raw Data to Biological Insight](/knowledge/bioinformatics/metabolomics-data-analysis-workflow-from-raw-data-to-biological-insight)

## References and Further Reading

- [NCBI Data Resources](https://www.ncbi.nlm.nih.gov/). National Center for Biotechnology Information.
- [EMBL-EBI Training](https://www.ebi.ac.uk/training). European Bioinformatics Institute.
- [Bioconductor](https://bioconductor.org/). Bioconductor Project.
- [Galaxy Training Network](https://training.galaxyproject.org/). Galaxy Project.
- [nf-core Documentation](https://nf-co.re/docs). nf-core.
- [The Carpentries Lessons](https://carpentries.org/lessons). The Carpentries.
- [Improved Peptide Search for Identification of SUMO and Sequence-Based Modifiers, in MaxSBM.](https://doi.org/10.1016/j.mcpro.2026.101589). 2026.
- [Analysis of isobaric quantitative proteomic data using TMT-Integrator and FragPipe computational platform.](https://doi.org/10.1038/s41467-026-70118-7). 2026.
- [Modanovo: A Unified Model for Post-translational Modification-Aware De Novo Sequencing Using Experimental Spectra From In Vivo and Synthetic Peptides.](https://doi.org/10.1016/j.mcpro.2025.101501). 2026.
- [Assessing the Performance of Mass Spectrometry Search Strategies in Identifying Translational Errors Using PDX Proteomics Data.](https://doi.org/10.1016/j.mcpro.2025.101500). 2026.
- [The MaxQuant computational platform for mass spectrometry-based shotgun proteomics](https://doi.org/10.1038/nprot.2016.136). Nature Protocols, 2016.

> This article is educational and does not replace validated analysis plans, institutional policy, clinical interpretation, or specialist review.