The Essential Metadata Checklist for Bioinformatics Files

By Dr. Zubair Khalid, DVM, MS, PhD ·

The Essential Metadata Checklist for Bioinformatics Files

By the end of this guide you will have a 30-field metadata checklist you can apply to any sequencing, alignment, or variant file, a YAML and JSON template you can copy, shell commands that populate the checklist automatically, and a validation step that fails loudly when a required field is missing. You will also know exactly which fields belong inside a file, which belong in a sidecar, and which formats quietly discard metadata you assumed was stored.

You need a POSIX shell with coreutils (sha256sum, zcat, date), Python 3.9 or newer with the jsonschema package, seqkit for sequence file statistics, and jq for JSON inspection. Nothing else is strictly required. A git repository helps but is optional.

What Metadata Has to Do for a Bioinformatics File

Metadata is the set of recorded facts about a data file that are not the measurement itself. For a FASTQ file that means everything except the nucleotide calls and their quality scores. Metadata answers four separate questions, and a checklist that answers only one or two of them is incomplete.

The first question is discovery. Can another researcher find this file at all? A minimal metadata schema designed for bioinformatics deliberately reduces the entire field to a small set of descriptors so that search across heterogeneous datasets becomes possible [1]. If your file cannot be retrieved by organism, assay type, or project, it is effectively invisible regardless of its quality.

The second question is interpretation. Given the file, can a competent analyst reproduce the biological meaning without asking you a question? That requires the reference build, the library chemistry, the read layout, and the coordinate convention.

The third question is trust. Can someone verify that the file is the file you described? That requires a checksum and a record count. Metadata integrity is a direct determinant of whether data-driven findings hold up, and a single metadata error can propagate through publications and reanalyses before anyone notices [2].

The fourth question is execution. Can a machine, not a human, select this dataset and run the right tool against it? Work on FAIR metadata for bioinformatics tools identified minimal properties needed for identification, selection, validation, and execution of tools, then aligned those properties to dataset metadata so the two could be integrated for automated analytics [3]. A checklist written for human eyes only will fail this test.

The Four Layers of a Metadata Record

Every field in the checklist belongs to one of four layers. Keeping the layers separate stops you from producing a flat list that nobody can extend.

Administrative metadata covers identity, ownership, and policy. File names, identifiers, checksums, contact people, licenses, and access restrictions live here.

Descriptive metadata covers what the data are about. Organism, tissue, developmental stage, genotype, phenotype, and experimental condition.

Structural and technical metadata covers how the data were physically produced and encoded. Format, compression, read length, quality encoding, instrument, and reference build.

Provenance metadata covers how the data were transformed. Tool names, versions, container digests, parameters, and input checksums.

A field belongs in the layer where it changes least often. Put the organism in descriptive metadata once, not in the parameter list of every tool invocation.

The Bioinformatics Metadata Checklist

The checklist below has thirty fields grouped into five blocks. Treat the first twenty-one as required for any file that will be shared or archived. Fields 22 to 30 are required for any file that has passed through an analysis step.

Block 1: File Identity (fields 1 to 7)

  1. file_id is a stable identifier you assign, not the path. Use a pattern like PRJ0123_S001_R1. Paths change. Identifiers should not.
  1. file_name is the current base name including extension, for example S001_R1.fastq.gz. Record it even though it duplicates the filesystem, because it is the only thing that lets a checksum manifest survive a rename.
  1. file_format is the format as declared by the file, not as assumed from the extension. .fq.gz and .fastq.gz are the same format. A .txt file may be a FASTA. Record one of a fixed vocabulary: FASTQ, FASTA, SAM, BAM, CRAM, VCF, BCF, BED, GFF3, GTF, TSV, H5AD.
  1. format_version records the specification version where one exists. SAM and BAM are at 1.6. VCF is at 4.2 or 4.3. GFF3 is at 1.26. This field is the one most often skipped and the one that causes the most silent parsing failures.
  1. checksum_sha256 is a 64-character lowercase hex digest of the file exactly as stored. Compute it on the compressed file if the file is compressed, and compute it once, at the moment of writing, so that every downstream copy inherits the same digest.
  1. record_count is the number of primary records. For FASTQ that is reads. For FASTA it is sequences. For BAM it is alignments excluding unmapped secondary and supplementary records, which you should note separately.
  1. created_utc is an ISO 8601 timestamp with an explicit offset, for example 2026-11-10T14:22:05Z. A timestamp without a timezone is not a timestamp.

Block 2: Technical and Assay Metadata (fields 8 to 15)

  1. assay_type names the experiment class, for example WGS, WES, RNA-seq, 16S, ChIP-seq, ATAC-seq, amplicon, methylation. Use a controlled list. Free text here breaks faceted search.
  1. platform is the sequencing or detection platform, for example Illumina NovaSeq 6000, PacBio Sequel IIe, Oxford Nanopore PromethION, 10x Genomics Chromium.
  1. instrument_id is the specific machine, not the model. Instruments drift in quality over their service life, and a batch effect traced to one instrument is impossible to find later if you recorded only the model.
  1. run_id is the provider's run identifier, for example a flowcell serial or a SRA run accession. This is your only reliable link back to the raw output of the sequencing provider.
  1. library_kit and chemistry_version record the preparation kit and its version. For Illumina this is the kit name plus the read-length configuration, for example Nextera XT with 2x150.
  1. layout is single or paired. Paired files must cross-reference each other through mate_file_id. A paired dataset missing its mate is the single most common cause of a pipeline that runs to completion with half the data.
  1. read_length records the nominal read length in cycles, and index_reads records the index length. Record both even when they are the default for the kit.
  1. reference_build is required the moment the file is aligned or annotated. Record the assembly name and accession together, for example GRCh38.p14 with GCF_000001405.40. Never record a bare hg38, because hg38 and GRCh38 differ in their mitochondrial contig and in their decoy content.

Block 3: Sample Biology (fields 16 to 21)

  1. sample_accession is the persistent identifier for the biological sample. Use the repository accession (SAMN..., ERS...) where one exists. A local identifier is acceptable only if it maps to a repository accession in the same record.
  1. organism_taxid is the NCBI Taxonomy identifier as an integer. For human it is 9606. For Escherichia coli K-12 MG1655 it is 511145. A scientific name string alone is ambiguous across strains and assemblies.
  1. tissue_or_source names the biological source using an ontology term where possible, for example UBERON:0000178 for blood. Free text such as blood draw will not merge with whole blood in an aggregation.
  1. genotype and treatment record the genetic background and any perturbation. wild type, knockout, vehicle control, and treated are not sufficient without the gene or compound name.
  1. donor_id records the individual. Replicates from one donor are not biological replicates. Getting this wrong inflates significance in every downstream test.
  1. condition maps the sample to the experimental design, for example case or control, with the matching variable named in the design table.

Block 4: Rights, People, and Policy (fields 22 to 26)

  1. project_id is the umbrella identifier that ties this file to a grant, a study, or a repository submission.
  1. contact_name and contact_orcid identify the person who can answer questions about the file in five years. Use an ORCID rather than an email, because institutional addresses expire.
  1. license records the reuse terms, for example CC-BY-4.0, CC0-1.0, or restricted. A file with no stated license is not reusable even when it is public.
  1. data_use_restriction records consent limitations for human data. Genomic data are sensitive, and the restriction travels with every derivative file, not just the raw reads [2].
  1. retention_policy records how long the file is guaranteed to exist. A DOI pointing at storage that expires in two years is worse than no DOI.

Block 5: Provenance and Analysis (fields 27 to 30)

  1. tool_versions is a list of every tool that touched the file, with its exact version string. bwa is not a version. bwa 0.7.17-r1188 is.
  1. container_digest records the image digest, not the tag. Tags are mutable. The CREDO project exists precisely because reproducible container construction from recorded package metadata is the hard part of rerunning an analysis [4].
  1. command_line is the literal command with all parameters, redacted only for credentials. Store it as a list of strings so it can be re-executed without shell parsing.
  1. input_file_ids lists the file_id values consumed. Combined with checksums, this makes the record a node in a verifiable provenance graph rather than a leaf note.

Choosing a Bioinformatics Metadata Checklist Format

There is no single correct carrier. There are four practical options, and the right one depends on how many files you have and who consumes them.

Option 1: A TSV Sample Sheet

One row per sample and one column per field. This is the format most pipeline orchestration systems accept, and it is readable in any spreadsheet application. It breaks when a field needs to hold a list, and it has no type system, so a numeric field silently becomes text.

Option 2: A YAML Sidecar

One file per data file, with the same base name plus .meta.yaml. Example: S001_R1.fastq.gz and S001_R1.meta.yaml. YAML handles nesting and lists cleanly and diffs well in version control. It is indentation-sensitive, and a stray tab produces a parse error that confuses beginners.

Option 3: JSON Validated Against a Schema

One file per data file, or one record per line in JSON Lines. This is the only option that supports formal validation with types, enumerations, and required fields. It is verbose and awkward to hand-edit, so pair it with a generator script.

Option 4: Metadata Embedded in the File Header

Some formats carry structured metadata natively. SAM and BAM have @HD, @SQ, @RG, and @PG header lines. VCF has ##INFO and ##contig lines. GFF3 has ##sequence-region directives. Where the format provides a place for a fact, put the fact there as well as in the sidecar. A BAM read group that carries the sample name and library is far more robust than a sidecar that can be separated from the file.

Newer work extends this idea to formats that previously had no place for method documentation. The DeNoFo format exists because de novo gene annotation lacked a standard way to record its methodology, and it was designed to integrate with established formats such as FASTA and GFF while making datasets comparable across studies [5].

FormatBest forStrengthsFailure modes
TSV sample sheetMany samples, pipeline inputUniversal, spreadsheet-editableNo nested fields, no types, silent type coercion
YAML sidecarOne file per data fileNested lists, readable, git-friendlyIndentation errors, no validation without extra tooling
JSON plus schemaArchived, machine-consumedTyped, enumerable, formally validatedVerbose, needs generation and validation scripts
In-file headerAlignments, variants, annotationsTravels with the data, survives copiesFormat-limited, often lost on conversion

A workable default is to keep the authoritative record as JSON validated against a schema, generate the TSV sample sheet from it for pipeline input, and write the essential subset into the BAM header's read group at alignment time.

A Minimal Template

file_id: PRJ0123_S001_R1
file_name: S001_R1.fastq.gz
file_format: FASTQ
format_version: "Sanger FASTQ"
checksum_sha256: 0000000000000000000000000000000000000000000000000000000000000000
record_count: 25000000
created_utc: "2026-11-10T14:22:05Z"
assay_type: RNA-seq
platform: Illumina NovaSeq 6000
instrument_id: A00123
run_id: 2026-A00123_0456
library_kit: TruSeq Stranded mRNA
chemistry_version: "2x150"
layout: paired
mate_file_id: PRJ0123_S001_R2
read_length: 150
index_reads: 8
reference_build: GRCh38.p14 (GCF_000001405.40)
sample_accession: SAMN00000001
organism_taxid: 9606
tissue_or_source: UBERON:0000178
genotype: wild type
treatment: none
donor_id: D001
condition: control
project_id: PRJ0123
contact_name: Family Name, Given Name
contact_orcid: 0000-0000-0000-0000
license: CC-BY-4.0
data_use_restriction: none
retention_policy: repository, indefinite
tool_versions: []
container_digest: null
command_line: []
input_file_ids: []

Replace the placeholder checksum with a real digest. A template that ships with a valid-looking fake digest is a hazard, so keep the placeholder obviously invalid until it is populated.

A Bioinformatics Metadata Checklist Example

Consider one paired-end RNA-seq library, S001, sequenced as 2x150 on a NovaSeq 6000 and delivered as two gzipped FASTQ files. The first read of S001_R1.fastq.gz carries this header:

@A00123:1:H7L7YDSX2:1:1101:1000:1000 1:N:0:ACGTACGT+AGTCAGTC

Read that header field by field before writing anything down. The instrument identifier is A00123. The run number is 1. The flowcell is H7L7YDSX2. The lane is 1. The tile is 1101. The x and y coordinates within the tile are 1000 and 1000. The read number is 1, meaning this is read one of a pair. The filter flag is N. The control flag is 0. The index sequence is ACGTACGT+AGTCAGTC, an eight-base i7 followed by an eight-base i5.

That header supplies fields 10 and 11 and confirms field 13. It supplies nothing else. It does not tell you the organism, the tissue, the reference build, or the library kit. Those must come from the sample sheet and the sequencing provider's run manifest, which is why the sidecar cannot be skipped.

Cross-check the mate file. The first header of S001_R2.fastq.gz should carry the same instrument, flowcell, lane, tile, and coordinates with a read number of 2. If the coordinates differ, the files are not mates and no amount of metadata will fix the biology.

Bioinformatics File Formats and What They Carry Natively

Understanding what each format stores is what tells you which fields are genuinely missing.

FormatStructureNative metadataWhat it never records
FASTA2 lines per recordSequence identifier line, descriptionQuality, sample, organism, reference
FASTQ4 lines per recordInstrument, coordinates, read number, indexSample, organism, library kit, reference
SAM/BAM/CRAMHeader plus alignmentsRead groups, reference sequences, program recordsBiological interpretation, consent terms
VCF/BCFHeader plus variant recordsContigs, INFO definitions, sample columnsTissue, treatment, library chemistry
BEDTab-delimited intervalsTrack and browser linesAlmost everything else
GFF3/GTFTab-delimited featuresSequence regions, ontology referencesAssay and provenance detail

FASTQ vs FASTA: The Metadata Difference

The structural difference is easy. FASTA uses two lines per record, an identifier line beginning with > followed by the sequence. FASTQ uses four lines per record: an @ identifier line, the sequence, a + separator, and a quality string of the same length as the sequence.

The metadata difference is the one that matters here. FASTQ carries per-base quality scores, which are ASCII-encoded Phred values. The encoding is either Phred+33 (ASCII 33 to 126) or Phred+64, and the two are indistinguishable without knowing which was used. Illumina moved from Phred+64 to Phred+33 with the Casava 1.8 pipeline, so any FASTQ older than that transition is a candidate for the older encoding. Always record the encoding explicitly.

A FASTA file carries no quality scores at all, which makes it unsuitable for base-calling or adapter trimming but ideal for reference sequences, assemblies, and protein sets. A FASTA defline from a public repository looks like this:

>NC_000001.11 Homo sapiens chromosome 1, GRCh38.p14 Primary Assembly

That defline encodes the accession, the organism, the assembly, and the sequence class in free text. It is informative to a human and nearly useless to a parser, which is why the reference build field in the checklist must record the assembly accession separately.

Neither format records organism as a structured field, neither records the library kit, and neither records consent or license. Use a sidecar for all of them.

The quickest way to confirm which format you actually have is to ask the file, not the extension:

seqkit stats -a reads/S001_R1.fastq.gz reads/reference.fa
file                        format  type   num_seqs  sum_len  min_len  avg_len  max_len  Q1  Q2  Q3  N50  Q20(%)  Q30(%)  AvgQual  GC(%)
reads/S001_R1.fastq.gz      FASTQ   DNA           4      600      150      150      150 150 150 150  150    97.8    94.1    37.4   48.6
reads/reference.fa          FASTA   DNA           2       60       30       30       30  30  30  30   30     0.0     0.0     0.0   50.0

The format and type columns are the fields to trust. A file with a .fa extension that reports FASTQ in that column is a FASTQ file with a misleading name, and you should rename it before anyone builds a pipeline around it.

Building the Checklist: A Numbered Walkthrough

This walkthrough takes a freshly delivered FASTQ pair through to a validated, version-controlled metadata record.

Step 1: Create the layout

mkdir -p project/{reads,metadata,schemas}
cd project
project/
├── reads/
├── metadata/
└── schemas/

Keep metadata in its own directory at the same level as the data it describes. Do not nest metadata inside a per-sample data folder, because that makes a whole-project metadata query require a recursive walk.

Step 2: Identify the format from the file itself

zcat reads/S001_R1.fastq.gz | head -4
@A00123:1:H7L7YDSX2:1:1101:1000:1000 1:N:0:ACGTACGT+AGTCAGTC
ACGTACGTACGTACGTACGTACGTACGTACGTACGTACGT
+
IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII

Four lines with an @ first character and a + third line is FASTQ. The quality characters being I (ASCII 73) means a Phred score of 40 under Phred+33, which is a plausible value. Characters in the range @ to h with a median around h would suggest Phred+64 instead.

Step 3: Compute checksums and write a manifest

sha256sum reads/S001_R1.fastq.gz reads/S001_R2.fastq.gz > metadata/checksums.sha256
cat metadata/checksums.sha256
8f14e45fceea167a5a36dedd4bea2543...  reads/S001_R1.fastq.gz
c9f0f895fb98ab9159f51fd0297e236d...  reads/S001_R2.fastq.gz

Verify the manifest later, after any copy or transfer:

sha256sum -c metadata/checksums.sha256
reads/S001_R1.fastq.gz: OK
reads/S001_R2.fastq.gz: OK

Step 4: Extract counts and quality automatically

seqkit stats -a -T reads/S001_R1.fastq.gz reads/S001_R2.fastq.gz > metadata/seqkit_stats.tsv
column -t -s $'\t' metadata/seqkit_stats.tsv

The -T flag produces tab-separated output that pipes directly into a spreadsheet or a Python dataframe. The num_seqs column populates field 6, and Q30(%) gives you a quality figure to store alongside the record count.

Step 5: Populate the record

Copy the template and edit it. Populate record_count from the num_seqs column, not from the sequencing provider's quoted yield. The provider quotes the yield of the run, not the number of reads in your file, and the two frequently disagree by several percent after demultiplexing.

Step 6: Validate against a schema

python3 -m pip install --quiet jsonschema
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "title": "Bioinformatics file metadata record",
  "type": "object",
  "required": [
    "file_id", "file_name", "file_format", "checksum_sha256",
    "record_count", "organism_taxid", "assay_type", "created_utc"
  ],
  "properties": {
    "file_format": {"enum": ["FASTQ", "FASTA", "SAM", "BAM", "CRAM", "VCF", "BCF", "BED", "GFF3", "GTF", "TSV", "H5AD"]},
    "checksum_sha256": {"type": "string", "pattern": "^[a-f0-9]{64}$"},
    "record_count": {"type": "integer", "minimum": 0},
    "organism_taxid": {"type": "integer", "minimum": 1},
    "created_utc": {"type": "string", "format": "date-time"},
    "layout": {"enum": ["single", "paired"]}
  },
  "additionalProperties": true
}
import json
import sys
from jsonschema import validate, ValidationError

schema = json.load(open("schemas/file_metadata.schema.json"))
record = json.load(open(sys.argv[4]))

try:
    validate(instance=record, schema=schema)
except ValidationError as exc:
    print(f"INVALID: {exc.message}")
    sys.exit(1)

print(f"OK: {record['file_name']} ({record['file_format']}, {record['record_count']} records)")
python3 validate_metadata.py metadata/S001_R1.meta.json
OK: S001_R1.fastq.gz (FASTQ, 25000000 records)

Step 7: Check the paired relationship

jq -r '[.file_id, .mate_file_id, .layout] | @tsv' metadata/*.meta.json | sort
PRJ0123_S001_R1	PRJ0123_S001_R2	paired
PRJ0123_S001_R2	PRJ0123_S001_R1	paired

Every mate_file_id must resolve to a file_id that exists. A one-directional mate reference is a broken pair.

Step 8: Freeze and commit

git add metadata schemas
git commit -m "Add validated metadata for S001 read pair"

Commit the metadata at the same time as, or before, the data are moved. Metadata written after a six-month analysis is reconstructed from memory and contains errors.

How to Check the Result Is Right

Four checks catch nearly every mistake.

First, re-run sha256sum -c on a copy of the data in its final resting place. A checksum that validates on the source but not on the destination means the transfer truncated the file.

Second, confirm that record_count in the metadata equals num_seqs in the seqkit output. Do not eyeball it, diff it:

jq -r .record_count metadata/S001_R1.meta.json
awk -F'\t' 'NR==2 {print $4}' metadata/seqkit_stats.tsv

Third, confirm that organism_taxid resolves in NCBI Taxonomy to the organism you intend. A taxid is an integer and it will validate against the schema whether or not it names the right species.

Fourth, confirm that every input_file_ids entry exists in the metadata directory and that its checksum matches the manifest.

Automating Capture and Enforcement

Manual metadata entry degrades at scale. Three layers of automation keep a checklist honest.

Controlled entry. Rather than a blank text file, give data producers a form with fixed vocabularies and required fields. LabPipe was built for exactly this problem in clinical bioinformatics, providing reusable configurations for experiment metadata and local data collection while linking the entered metadata to vendor-generated files. It was piloted in a multi-site clinical breathomics study and provided a consistent, controlled approach to handling metadata across sites [6].

Workflow-level enforcement. Make the pipeline refuse to start if the checklist is incomplete. A schema check at the head of a workflow costs seconds and prevents a week of analysis on unlabeled data. Standardized output naming and standardized output file formats make downstream records predictable, which is the approach taken by public health bioinformatics workflows built on WDL and distributed through Dockstore with version-controlled source in public repositories [7].

Container pinning. Record the image digest, not the tag. CREDO addresses reproducibility by generating Docker images from recorded package and dependency information stored in a GitHub-compatible format, which allows an image to be rebuilt from scratch rather than pulled from a mutable registry entry [4].

Report generation. When a quality control step runs, have it emit a machine-readable summary alongside the human-readable one. The AssemblyQC pipeline, for example, produces an HTML report with method descriptions, tables, and visualizations covering contiguity, completeness, contamination, and k-mer-based quality metrics, and each of those underlying values is a candidate metadata field for the assembly record [8].

Discovery and indexing. Once records exist, they need to be findable. EBI Search provides a unified metadata search across EMBL-EBI resources, covering more than 6.5 billion data items through both a web interface and an OpenAPI-compliant programmatic interface [9]. Submitting well-structured records to such an index is what converts a private checklist into reusable data.

Machine consumption. The next consumer of your metadata may be an agent rather than a person. Work adapting the Model Context Protocol to bioinformatics web services explicitly associates endpoints with scientific concepts and detailed metadata so that large language models can explore databases programmatically, and aligns tool metadata with dataset metadata to support automated analytics [10]. Fields that are free text today will be parsed by machines tomorrow.

Common Mistakes and Limitations

Recording the checksum before compression, or after decompression. The digest must describe the bytes on disk. A checksum of the uncompressed stream will not validate against the stored .gz file and will send you chasing a corruption that does not exist.

Recording layout: paired without a mate reference. The field is a claim, not a fact. Enforce it with a cross-reference check in Step 7.

Guessing the quality encoding. Phred+33 and Phred+64 produce identical-looking FASTQ files. A file read with the wrong offset produces quality scores off by 31, which is enough to flip variant calls in low-coverage regions. Record the encoding field explicitly.

Mixing chromosome naming conventions. chr1 and 1 name the same chromosome and will not intersect in any tool that joins on contig name. Record the naming convention used by the reference build, and convert once, at the boundary, not scattered through the pipeline.

Using unknown or NA as a value. These strings pass a naive schema check and carry no information. Either make the field required so the record fails, or omit the key entirely so absence is unambiguous. Do not encode ignorance as data.

Treating metadata as write-once. Files get renamed, references get updated, and tools get re-run with different parameters. Version the metadata record alongside the data, and never edit a record in place after a result has been published from it.

Underestimating the consequence of a single wrong field. A metadata error discovered in patient data published across two high-impact journals prompted a broader call for availability, curation, and accuracy of metadata, because integrity of those records is what supports the reliability of data-driven findings [2]. The error was small and the propagation was not.

Over-engineering with ontologies. Ontology terms are valuable where they enable cross-study aggregation, particularly for tissue, cell type, and phenotype. Applying full ontological annotation to file_name or read_length adds work and no value. Match the vocabulary to the field. Social tagging studies found that free-form tagging covered document space well but agreed with controlled MeSH indexing at low rates, which is the practical trade-off in miniature [11].

Assuming a public repository accession is complete metadata. A repository record covers what the repository requires. It rarely captures the analysis parameters used downstream, and it never captures the container digest. The checklist complements repository submission, it does not replace it.

A note on scope. Every dataset has a specific biological and regulatory context that a generic checklist cannot resolve. Human genomic data carry consent restrictions that must be read from the original protocol, and clinical data carry de-identification requirements that vary by institution.

Frequently Asked Questions

What is a bioinformatics metadata checklist?

A bioinformatics metadata checklist is a fixed list of fields recorded for every data file, covering file identity, technical provenance, sample biology, rights, and analysis history, so that the file can be found, interpreted, verified, and rerun without asking the original author. It works as a template that is filled once per file and validated automatically.

What is the minimum metadata for a FASTQ file?

The minimum set is a stable file identifier, file name, format, SHA-256 checksum, record count, creation timestamp, assay type, platform, instrument, run identifier, layout, read length, sample accession, and organism taxonomy identifier. Add the mating file identifier for paired data. Everything else improves reuse but these fields make the file interpretable.

What is the difference between FASTQ and FASTA metadata?

FASTQ stores four lines per record and includes per-base quality scores plus an instrument and coordinate header. FASTA stores two lines per record and includes only an identifier and a free-text description, with no quality data. Neither format records organism, library kit, reference build, or consent terms, so both need a sidecar record.

Should metadata go inside the file or in a sidecar?

Use both where the format allows it. Put read groups and program records in a BAM header, contig and INFO definitions in a VCF header, and sequence regions in a GFF3 header. Keep the authoritative record in a validated sidecar, because headers are frequently stripped during format conversion and merging.

What format should a metadata checklist take?

JSON validated against a JSON Schema is the best authoritative format because it enforces types, enumerations, and required fields. Generate a TSV sample sheet from it for pipeline input, and write the essential subset into file headers at the point of alignment or variant calling.

How do I check metadata integrity?

Verify the SHA-256 checksum against the stored file with sha256sum -c, compare the recorded record count against a fresh count from seqkit stats, resolve the taxonomy identifier against NCBI Taxonomy, and confirm every input_file_ids and mate_file_id resolves to an existing record. Run all four checks after any transfer.

Do I need controlled vocabularies or ontologies?

Use controlled vocabularies for fields that must aggregate across studies, such as assay type, organism, tissue, and license. Keep free text for narrative fields. Ontology terms pay off for tissue, cell type, and phenotype, and add little for file names, read lengths, or instrument identifiers.

How much metadata is enough for FAIR compliance?

Enough that a machine can identify the dataset, select it for a query, validate it, and execute an analysis against it without human intervention. That threshold requires structured, typed fields rather than prose descriptions, and it requires linkage between the dataset record and the tool record so automated analytics can join them.

Sources

  1. A Minimal Metadata Schema and Its Tool to Improve the Searchableness of Research Data in Bioinformatics
  2. Metadata integrity in bioinformatics: Bridging the gap between data and knowledge
  3. FAIR in practice: minimum metadata schema for bioinformatics analytics by machines
  4. CREDO: a friendly Customizable, REproducible, DOcker file generator for bioinformatics applications.
  5. DeNoFo: a file format and toolkit for standardized, comparable de novo gene annotation.
  6. LabPipe: an extensible bioinformatics toolkit to manage experimental data and metadata
  7. Accelerating bioinformatics implementation in public health.
  8. AssemblyQC: a Nextflow pipeline for reproducible reporting of assembly quality.
  9. EBI Search: providing discovery tools for biological metadata in 2025
  10. MCPmed: a call for Model Context Protocol-enabled bioinformatics web services for LLM-driven discovery
  11. Social tagging in the life sciences: characterizing a new metadata resource for bioinformatics