Sample Metadata Standards for Sequencing Projects

If you are a researcher, bioinformatician, or data manager planning a sequencing project, you need a practical metadata schema that captures sample origin, methods, reagent versions, consent constraints, and persistent identifiers. This guide defines the minimum set of fields that make your data reusable and compliant with public repositories like the NCBI Sequence Read Archive. You should adopt this schema at the project design stage, not after sequencing is complete.

Well structured metadata is the difference between a dataset that others can interpret and a collection of uninterpretable files. The EMBL EBI Training resources emphasize that metadata completeness directly affects data reuse and reproducibility. In this guide, I walk through a standard schema, how to decide what to include, a practical implementation workflow, common pitfalls, and the limits of current best practices.

At a Glance

The following table summarizes the essential metadata fields for a sequencing project. Each field belongs to one of four categories: sample identity, experimental methods, consent and ethics, or identifiers.

Decision Criteria for Metadata Inclusion

Not every field is equally important for every project. Use these criteria to decide which metadata to capture.

Reuse potential. If your sample comes from a human subject with privacy restrictions, consent fields are mandatory. The Toward a quality managed operational architecture for wastewater surveillance study shows that even environmental surveillance requires clear consent and use limitation metadata to enable future public health analysis.

Technical reproducibility. Methods fields like library kit and sequencer model are needed for any pipeline that might be rerun. Include reagent lot numbers when possible, as RD OMICS an integrative multi omics data inventory found that lot to lot variability can introduce systematic bias.

Versions matter. Software and reference genome versions are often omitted, but they are critical. The highly sensitive amplicon sequencing workflow for Usutu virus demonstrates that version mismatches between mapping tools and references can change variant calls. Always record the exact version used for each step.

Consent constraints. For human data, you must capture the consent code and data use limitation. Many repositories reject submissions that do not have a clear consent category. Use the standardized codes from the informed consent or ethical approval.

Identifiers. Do not rely on sample names alone. Use BioSample and BioProject accessions from the NCBI Bookshelf to create persistent links. Without these, your data cannot be found by search tools or cited by others.

Practical Workflow for Metadata Collection

Follow this sequence to build a complete metadata record for each sample.

Step 1: Define your sample IDs before starting the lab work. Use a consistent naming scheme that encodes the project, sample type, and replicate. For example, PROJ_SERUM_01. Record this ID on every tube and in your lab notebook. Do not change the ID after sequencing.

Step 2: Create a template spreadsheet. Include all fields from the At a Glance table. Use columns for each field and rows for each sample. Share this template with your team and fill it in as you go. The Galaxy Training Network provides example spreadsheets for common sequencing protocols. Adapt one to your project.

Step 3: Record methods at the bench. When you process a sample, immediately note the library preparation kit number, the sequencer model, and the software version. Do not rely on memory. Write the lot number of each reagent kit onto the spreadsheet.

Step 4: Capture consent and ethics information. For human samples, record the consent code (PUB, CON, HMB, etc.) and the data use limitation (e.g., General Research Use, Disease Specific). Keep the original consent form or approval letter as a reference. For non human samples or environmental samples, record the permit or approval number if applicable.

Step 5: Submit to a public repository. When you are ready to upload FASTQ or BAM files, use your completed spreadsheet to fill out the repository submission forms. Most repositories, including the NCBI Sequence Read Archive, accept a tab separated metadata file. Double check that every required field is filled. After submission, record the BioSample and Run accessions back into your spreadsheet.

Step 6: Add versioning information. At the time of data analysis, record the version of every tool used. For example, if you ran FastQC, note the version number. If you aligned to a reference genome, note the genome build and the alignment software version. Include these versions in your final metadata file or in a separate analysis_versions.txt file.

Step 7: Validate and finalize. Run the metadata file through a validator. Several tools exist, such as the NIH metadata validator or the Bioconductor package metagenomeFeatures, which can check that fields are present and correctly formatted. Fix any errors. Then attach the metadata file to your project in the repository.

Common Mistakes and How to Avoid Them

Mistake 1: Inconsistent sample IDs. Researchers often rename samples during the project, leading to lost links between raw data and the original sample. Solution: decide on a single ID scheme at the start and never change it. If you must add information, append a suffix rather than altering the base ID.

Mistake 2: Missing consent constraints for human data. Many submitters assume that deposition in a public repository means the data is fully public. In fact, controlled access data requires explicit consent codes. Without them, the data may be quarantined or removed. The Global Landscape of Publicly Available Human Oral Microbiome Data paper highlights how inconsistent consent metadata hampered reuse of oral microbiome studies. Always include the consent code.

Mistake 3: Omitting reagent versions. Kit lot numbers and software versions are often left out. This is a serious problem because batch effects can be traced back to the reagent lot. As the Public genomic surveillance of African Klebsiella pneumoniae study shows, technical variation can obscure biological signals if metadata are incomplete. Record lot numbers in your spreadsheet.

Mistake 4: Using ambiguous collection dates. Common errors are missing the date entirely or using only the year. Repositories require a full date (YYYY MM DD) or at least a month. If exact date is unknown due to privacy, use a range or a season. The Mapping potential pathogen profiling in cetacean blow paper demonstrates that temporal metadata is crucial for tracking pathogen emergence. Always provide the best approximation.

Mistake 5: Not linking to the BioProject. Some submitters upload samples without associating them to a project. That makes the samples invisible in searches and impossible to cite as a cohesive dataset. Always create a BioProject first and assign all samples to it.

Limits and Uncertainty of Current Standards

Metadata standards for sequencing projects are still evolving. Different repositories accept different field names, and there is no universal schema that covers every type of omics data. The NCBI Bookshelf provides detailed submission guides, but those guides change every few years. What is required today may be optional tomorrow.

For human data, consent categories vary by jurisdiction and by institution. The GA4GH consent codes are widely used but not always understood by local ethics boards. If your consent form uses language that does not map cleanly to a standard code, you may need to contact the repository help desk before submission. Some repositories accept custom data use limitations, but they are not automatically searchable in global databases.

Environmental and pathogen surveillance projects face a different challenge: they often lack consent metadata but still need to describe the geographic location and collection method precisely. The standards for these fields are less mature than for human clinical samples. The wastewater surveillance literature Toward a quality managed operational architecture calls for a coordinated metadata framework that has not yet been fully implemented.

Another limit is version control for analysis software. Most repositories capture the sequencing platform and library kit version, but they do not require the bioinformatics tools version. That means two researchers downloading the same raw data may use different software versions and get different results. Until repositories enforce tool version metadata, each project must self document in a separate file.

Finally, there is no widely adopted machine readable format for metadata that includes all the fields described here. Spreadsheets are human readable but error prone. JSON or YAML formats are gaining traction, but repositories still prefer tab separated values. The community is moving toward standards like ISA Tab, but adoption is slow. For now, the best approach is to follow the repository guidelines and add your own fields for versions and consent details.

Frequently Asked Questions

Q: Do I need to include metadata for every single sample if they were processed identically?
Yes. Even if all samples share the same library kit and sequencing platform, you must record that information per sample. Repository submission forms require sample level metadata. You can use a group level description for shared fields, but each sample record must still contain the values.

Q: Can I use a non standard ID format for my samples?
You can use any identifier you like internally, but the external repository may convert it to a BioSample ID. To avoid confusion, use a simple alphanumeric ID without special characters. Avoid spaces, slashes, and underscores as the first character. These characters can cause parsing errors in some submission tools.

Q: What should I do if my consent form does not match any standard consent code?
Contact the repository help desk before submission. Provide them with the exact language from the consent form. Many repositories have a process for reviewing and approving custom data use limitations. Do not force your data into a code that does not fit, as that can violate the original consent terms.

Q: How do I record version information for open source bioinformatics tools that do not have official version numbers?
If a tool does not have a version number, use the git commit hash or the date of the download. Record that string in your metadata file. For example, kraken2 commit a1b2c3d or Kraken2 downloaded 2024 05 10. This gives future users enough information to reconstruct your environment.

References and Further Reading

• NCBI Bookshelf Authoritative technical references for sequence submission guidelines and metadata formats.
• EMBL EBI Training Official training modules with practical exercises for metadata annotation and repository submission.
• Galaxy Training Network Open tutorials for creating metadata spreadsheets and running validation tools.
• Bioconductor R packages for metadata validation and integration with common sequencing workflows.
• NCBI Sequence Read Archive Primary public repository for sequencing data and its metadata submission portal.
• Toward a quality managed operational architecture for wastewater surveillance Case study on metadata needs for environmental pathogen monitoring.
• RD OMICS an integrative multi omics data inventory in rare diseases Example of multi omics metadata harmonization with lot and version tracking.
• A highly sensitive amplicon sequencing workflow for Usutu virus Demonstration of version specific metadata impact on variant calling.
• Global Landscape of Publicly Available Human Oral Microbiome Data Analysis of metadata completeness gaps in human microbiome studies.
• Public genomic surveillance of African Klebsiella pneumoniae Study showing how incomplete metadata restricts reuse of genomic surveillance data.

Related Articles

• Single Cell RNA seq Workflow A Practical Analysis Roadmap
• Single Cell RNA seq Quality Control Cells Genes and Mitochondrial Reads
• Cell Type Annotation in Single Cell RNA seq A Layered Evidence Framework
• Batch Effects in RNA seq How to Recognize and Plan Around Them
• Pseudobulk Analysis for Single Cell RNA seq