# Genomics Data QC Checks Pipeline: Step-by-Step Guide By the end of this guide you will have a working genomics data qc checks pipeline that takes raw sequencing files from a sequencing provider or in-house instrument, verifies their integrity, profiles read quality, trims problematic sequence, checks alignments and variant calls, produces a single consolidated QC report, and leaves behind the metadata and storage discipline needed for reproducible work. Every step includes the actual command, the parameters that matter, what correct output looks like, and the exact error messages you are most likely to hit. To follow along you need a Linux or macOS workstation or H2 login, roughly 16 GB of RAM, and the following tools installed through Conda. Any recent FASTQ files, a matching reference genome, and a sample sheet with one row per sample are the only inputs. ```bash mamba create -n qc -c bioconda -c conda-forge \ fastqc fastp multiqc samtools bwa bcftools qualimap mamba activate qc ``` If your analysis is expression-based rather than whole-genome alignment, steps 5 and 6 swap out for platform-specific checks covered later in this guide. The pipeline structure stays the same. ## Why QC Checks Come Before Any Analysis Sequencing data inherits every upstream failure. Bad extraction, a degraded library, a miscalibrated flow cell, a sample swap, or a metadata typo all surface as downstream errors that are far more expensive to diagnose than the original problem. Data quality research across sectors consistently identifies accuracy, completeness, consistency, timeliness, and accessibility as the dimensions that determine whether a dataset can support decisions, and poor quality produces financial losses and loss of trust that compound over time [1]. Health and genomic datasets add governance obligations on top: quality management, access control, privacy, accountability, and traceability must be planned rather than patched on [2]. Two properties of genomics data make this acute. First, the files are large and the analyses are long, so a bad input wastes hours of compute before anyone notices. Second, genomics datasets are frequently reused by people who did not generate them, which means quality problems travel further than you expect. Documenting what you checked, and what you found, is part of the QC job, not an afterthought [3]. ## The Genomics Data QC Checks Pipeline at a Glance A defensible pipeline runs the following checks in order. Each stage produces a report that the next stage depends on. 1. File integrity and metadata validation (checksums, sample sheet) 2. Raw read profiling (FastQC) 3. Adapter and low-quality base trimming (fastp) 4. Post-trim profiling and cross-sample comparison (FastQC plus MultiQC) 5. Alignment-level QC (bwa, samtools flagstat, samtools coverage, QualiMap) 6. Variant-level QC (bcftools stats) 7. Automated dataset-level checks (null counts, duplicates, distribution shifts) 8. Sensitive data screening and quality annotation 9. Validation against curated test datasets 10. Storage, retention, and FAIR metadata ## Prerequisites: Set Up the Directory Layout Reproducibility starts with structure. Create a project directory that separates raw data, which you never modify, from processed outputs. ```bash mkdir -p project/{raw,trimmed,qc,ref,aligned,variants,logs} cd project cp /path/to/fastqs/*.fastq.gz raw/ ``` Keep the raw directory read-only once populated. ```bash chmod -R a-w raw/ ``` This layout encodes a principle from the Guidelines for Research Data Integrity, which recommend planning data collection, variable definitions, and processing before analysis rather than deciding mid-project [4]. ## Step 1: Verify File Integrity and the Sample Sheet ### Compute and record checksums Sequencing providers deliver MD5 sums for a reason. Verify before you run anything. ```bash cd raw md5sum *.fastq.gz > md5_raw.txt md5sum -c md5_raw.txt ``` Expected output for a clean delivery: ``` S1_R1.fastq.gz: OK S1_R2.fastq.gz: OK S2_R1.fastq.gz: OK S2_R2.fastq.gz: OK ``` If you received checksums from the provider, compare theirs against yours. Any `FAILED` line means a truncated transfer, which is the most common first-day failure in any pipeline. Re-download before debugging anything else. ### Validate the sample sheet A sample sheet with duplicated identifiers, missing read paths, or inconsistent naming will fail silently downstream. A short check catches the obvious cases. ```python import csv required = {"sample_id", "lane", "read1", "read2", "library"} with open("samplesheet.csv") as fh: rows = list(csv.DictReader(fh)) ids = [r["sample_id"] for r in rows] assert len(ids) == len(set(ids)), "duplicate sample_id found" missing = [r["sample_id"] for r in rows if not r["read1"] or not r["read2"]] print("rows:", len(rows)) print("rows missing read paths:", missing) ``` Expected output: ``` rows: 24 rows missing read paths: [] ``` Integrity failures also come from the data itself, not just the files. Large observational datasets have been shown to contain bot-generated or otherwise fabricated entries, and one survey study found that a documented five-step cleaning protocol removed 61.8 percent of first-wave responses because the cleaning happened only after suspicious patterns were noticed [5]. The lesson for genomics teams is to schedule integrity checks as a planned stage, not as a reaction to a weird result. ## Step 2: Profile Raw Reads with FastQC ![Decision path asking whether a FastQC warning is per-base quality, adapter content, sequence content or GC content, and what to do for each.](/article-images/illustrations/genomics-data-qc-checks-pipeline-fastqc-warn-triage.webp) FastQC produces per-file module reports covering per-base sequence quality, per-sequence quality, per-base sequence content, per-sequence GC content, per-base N content, sequence duplication levels, overrepresented sequences, and adapter content. ```bash mkdir -p qc/fastqc_raw fastqc -t 8 -o qc/fastqc_raw raw/*.fastq.gz ``` Expected output files per FASTQ: ``` S1_R1_fastqc.html S1_R1_fastqc.zip ``` Inside the zip, `summary.txt` gives you a one-line verdict per module: ``` Pass Per base sequence quality S1_R1.fastq Warn Per sequence GC content S1_R1.fastq Pass Sequence duplication levels S1_R1.fastq ``` The modules that deserve your attention first are per-base sequence quality, which warns when the lower quartile of Phred scores drops below 10 and fails when it drops below 5, and adapter content, which warns when adapter contamination grows toward the end of reads. Per-base sequence content warns are normal and benign for many library types. Random-primed libraries, for example, show biased bases in the first dozen positions, and this is expected behavior rather than a defect. Bisulfite and small-RNA libraries shift GC distributions for biological reasons. Read the warn in context of the library type before treating it as a problem. A common error at this step: ``` fastqc: could not open file raw/S1_R1.fastq.gz ``` Fix: the file path is wrong or the file is still transferring. Confirm with `ls -lh raw/` and check the file size against the provider manifest. ## Step 3: Trim Adapters and Low-Quality Bases with fastp fastp performs adapter detection by overlap analysis for paired-end data automatically, applies sliding-window quality trimming with a window of 4 bases and a mean quality of 15 by default, and reports every statistic it computed as machine-readable JSON. ```bash mkdir -p trimmed qc/fastp for r1 in raw/*_R1.fastq.gz do base=$(basename "$r1" _R1.fastq.gz) r2="raw/${base}_R2.fastq.gz" fastp \ -i "$r1" -I "$r2" \ -o "trimmed/${base}_R1.trim.fastq.gz" \ -O "trimmed/${base}_R2.trim.fastq.gz" \ -h "qc/fastp/${base}.html" \ -j "qc/fastp/${base}.json" \ -q 20 -l 25 -n 5 -w 8 \ 2> "qc/fastp/${base}.log" done ``` Parameters that matter: | Parameter | What it controls | Default behavior | When to change it | |--|--|--|--| | `-q` | Per-base minimum Phred quality for sliding-window trimming | fastp trims with window size 4 and mean quality 15 | Raise to 20 for variant calling on short reads | | `-l` | Minimum read length after trimming | 15 bases | Raise to 25 to avoid tiny fragments inflating mapping noise | | `-n` | Maximum N bases allowed per read | 5 | Lower for stringent variant work | | `-w` | Worker threads | 3 | Match to available cores | | `-c` | Base correction by overlap analysis | Off | Enable when read 1 and read 2 overlap heavily | The key parameters are `-q`, `-l`, and `-n` because they define what counts as a usable read for everything downstream. `2>` in the command captures fastp's verbose log separately from the HTML report so logs stay greppable. Expected key entries in the JSON report: ```json { "summary": { "before_filtering": { "total_reads": 52345612, "q20_rate": 0.931, "q30_rate": 0.887 }, "after_filtering": { "total_reads": 51230044, "q20_rate": 0.978, "q30_rate": 0.951 } }, "filtering_result": { "passed_filter_reads": 51230044, "low_quality_reads": 849203, "adapter_trimmed_reads": 2140763 } } ``` Interpretation. Roughly 2 percent of reads were removed for low quality, and about 4 percent were adapter-trimmed, which is typical for standard Illumina WGS material. If adapter-trimmed reads exceed 20 to 30 percent, something went wrong at library prep or the run overread into adapter, and you should talk to the sequencing core before proceeding. If Q30 after filtering stays below roughly 85 percent, the flow cell or the sample chemistry was marginal and the rest of the pipeline inherits that uncertainty. ## Step 4: Compare All Samples with MultiQC Single-sample QC hides batch effects. The point of a consolidated report is comparison across every sample in the run. ```bash multiqc qc/ -o qc/multiqc -n qc_report ``` Expected output: ``` Searching 'qc' Found 24 reports multiqc_report.html multiqc_data/ ``` Open `multiqc_report.html` and scan two views. First, the per-module plots should cluster tightly across samples from the same library prep. One sample with visibly lower Q30, higher duplication, or shorter trimmed lengths is a candidate for exclusion or a re-sequence decision made now, not after differential analysis. Second, the fastp adapter trimming plot reveals library-prep batch effects that would otherwise surface months later as unexplained covariates. This consolidated-report pattern generalizes beyond sequence data. Automated quality-check frameworks designed for large linked research datasets produce exactly this kind of navigable HTML report with interactive tables and graphs so that reviewers can explore issues with minimal effort [6]. The same design goal applies to genomics: make the report the artifact people actually open. ## Step 5: Alignment-Level QC ### Align with bwa and sort ```bash bwa index ref/grch38.fa for base in $(ls trimmed/ | sed 's/_R1.trim.fastq.gz//' | sort -u) do bwa mem -t 8 ref/grch38.fa \ "trimmed/${base}_R1.trim.fastq.gz" \ "trimmed/${base}_R2.trim.fastq.gz" \ 2> "logs/${base}.bwa.log" \ | samtools sort -@ 8 -o "aligned/${base}.sorted.bam" samtools index "aligned/${base}.sorted.bam" done ``` ### Mark duplicates Marking duplicates before flagstat gives you honest duplication statistics in the same pass. ```bash for bam in aligned/*.sorted.bam do base=$(basename "$bam" .sorted.bam) samtools collate -O -u "$bam" \ | samtools fixmate -m -u - "aligned/${base}.fixmate.bam" samtools sort -@ 8 -o "aligned/${base}.fixmate.sorted.bam" \ "aligned/${base}.fixmate.bam" samtools markdup "aligned/${base}.fixmate.sorted.bam" \ "aligned/${base}.markdup.bam" samtools index "aligned/${base}.markdup.bam" done ``` ### Check mapping statistics ```bash samtools flagstat aligned/S1.markdup.bam ``` Expected output shape: ``` 1042680 + 0 in total (QC-passed reads + QC-failed reads) 1021104 + 0 primary 981245 + 0 primary mapped (96.10% : N/A) 958432 + 0 primary paired (93.85% : N/A) 912086 + 0 properly paired (95.16% : N/A) ``` The three numbers that matter are primary mapped, properly paired, and duplicate rate. For a healthy human short-read WGS library, primary mapped above 95 percent, properly paired above 90 percent, and a duplicate rate in the single digits are reasonable expectations. Shorter-read or fragment-based libraries differ, so compare against expectations for your library type rather than a universal rule. ### Check depth and breadth of coverage ```bash samtools coverage aligned/S1.markdup.bam ``` Expected output columns include `numreads`, `covbases` (fraction of reference positions with at least one read), `meandepth`, and `meancov`. For human WGS you generally want around 30x mean depth and breadth of coverage above 95 percent. For exomes and panels the targets differ, so use the coverage contract your assay was designed around. QualiMap gives a deeper view of coverage distribution and per-contiguous-sequence statistics when you need it for a report or an audit. ```bash qualimap bamqc -bam aligned/S1.markdup.bam -outdir qc/qualimap/S1 ``` A frequent error at this step: ``` [E::hts_open_format] Failed to open file aligned/S1.markdup.bam ``` Fix: the `samtools index` step from the previous command failed or the BAM does not exist. Check `ls -lh aligned/` and rerun the index. If markdup fails with a message about missing MC or ms tags, your fixmate step did not run with `-m`, which adds the mate tags markdup depends on. For RNA-seq data the corresponding checks are mapping rate against the transcriptome, exonic versus intronic and intergenic fractions, insert size, 5-prime to 3-prime coverage bias, and gene body coverage uniformity. The same discipline applies: profile before trimming, trim, then re-profile and compare. ## Step 6: Variant-Level QC Alignment metrics can look perfect while variant calls are contaminated, mislabeled, or systematically biased. Variant statistics catch a distinct failure class. ```bash bcftools mpileup -f ref/grch38.fa -Ou aligned/S1.markdup.bam \ | bcftools call -mv -Oz -o variants/S1.vcf.gz bcftools index variants/S1.vcf.gz bcftools stats variants/S1.vcf.gz > qc/stats/S1.bcfstats.txt ``` Look for the TSTV record in the output, which reports transitions and transversions: ``` TSTV 0 1192334 2.01 117845 2.02 2.01 ``` For human WGS, the transition to transversion ratio should land near 2.0. Exome data runs higher. A value far below expectation suggests contamination, poor mapping, or a wrong reference build. The record also reports per-sample variant counts and quality distributions, which is where sample swaps become visible: a sample with dramatically fewer heterozygous variants than its peers is frequently a duplicate, a swap, or a contaminated library. Depth and quality filters applied at the variant level trade sensitivity for precision, and the correct cut-offs depend on the downstream application. Confirmatory or clinical contexts demand stricter filtering than exploratory screens, and any threshold you adopt should be documented with the dataset so the choice travels with the results [3]. For expression platforms, pre-analytic RNA integrity has documented limits as a gate. In a study of Affymetrix Gene 1.0 ST arrays, 7 of 34 arrays, or 20.6 percent, were flagged by array-level quality assessment even though the paired design meant the flagged samples carried usable information, and arrays with RNA integrity numbers below 7 passed quality assessment, including one sample with an RIN below 3. Transcript-length bias from degradation appeared in only 5.8 percent of transcript clusters [7]. The practical takeaway is that a hard RIN cut-off discards usable samples while failing to guarantee quality, so array-level metrics should complement rather than replace RIN. ## Step 7: Automate the Dataset-Level Checks Per-sample tool reports answer whether one sample looks good. Dataset-level checks answer whether the collection as a whole is complete, consistent, and comparable. Manual checking does not scale, and automated generic checks have been shown to report data issues across null counts, duplication, value distributions, ranges, agreement with reference sources, and comparison against previous versions with minimal configuration [6]. In the SAIL databank, quality tests were written against the dimensions of validity, consistency, accuracy, and completeness, using SQL scripts that evaluated not just each dataset in isolation but the consistency of shared fields between datasets [8]. A compact automated check for a genomics project looks like this: ```python import csv from collections import Counter with open("qc/all_sample_qc.tsv") as fh: rows = list(csv.DictReader(fh)) checks = [] # Completeness: every sample has every field gaps = [r["sample_id"] for r in rows if not r["mean_depth"] or not r["dup_rate"]] checks.append(("completeness", "missing depth or dup fields", gaps)) # Consistency: values inside expected ranges for the assay bad_depth = [r["sample_id"] for r in rows if float(r["mean_depth"]) < 20] checks.append(("consistency", "mean depth below 20x", bad_depth)) # Distribution shift: is one sample an outlier against its peers depths = [float(r["mean_depth"]) for r in rows] median = sorted(depths)[len(depths) // 2] outliers = [r["sample_id"] for r in rows if abs(float(r["mean_depth"]) - median) > 0.5 * median] checks.append(("consistency", "depth outlier vs cohort", outliers)) for name, desc, hits in checks: print(name, desc, hits if hits else "OK") ``` Expected output: ``` completeness missing depth or dup fields OK consistency mean depth below 20x ['S17'] consistency depth outlier vs cohort ['S17'] ``` The sample flagged twice becomes your first investigative target. Write these checks once, run them on every batch, and compare results against the previous batch so drift is visible early. Parallel work in other domains points the same direction: frameworks that profile data continuously in real time and use historical patterns to predict quality problems before they propagate are an active area of development, with reported improvements in quality indicators after deployment [9]. Review work on AI-assisted data governance similarly emphasizes automated profiling, anomaly detection, and lineage tracking as scalable replacements for purely manual audits. ## Step 8: Screen for Sensitive Data and Annotate Known Issues Genomics files carry re-identification risk, and metadata is where leaks hide. Free text fields, sample labels, and supplementary files frequently contain direct identifiers that slipped in during collection. A regex-driven flagging approach built for large health datasets scans identifiers such as user IDs, postcodes, phone numbers, and addresses across millions of records and renders findings in a human-readable HTML report for review, flagging more than 25 million records across over 50 columns in under 150 seconds [10]. A minimal version for your sample sheet: ```bash grep -nE '([0-9]{3}[-.][0-9]{4}|[A-Z][0-9]{2} ?[0-9][A-Z]{2})' samplesheet.csv ``` Expected output when a phone number pattern appears: ``` 7:study_notes,contact at 555-0183 for re-extraction ``` Any hit means the field gets redacted before the dataset moves anywhere. When a dataset has known quality problems but still has value for method development, training, or assay validation, annotate those problems explicitly rather than quietly publishing it. The PHA4GE consortium developed standardized quality control contextual data tags precisely so that public health sequence datasets with known issues can be shared, searched, and reused without being mistaken for higher-quality data [3]. Applying this pattern internally costs nothing and prevents a colleague from building on a file you already knew was compromised. ## Step 9: Validate the Pipeline with Curated Test Datasets A QC pipeline is itself software, and it needs its own testing. Curated test datasets, meaning sets of well-characterized sequences designed to evaluate accuracy, consistency, and performance of sequencing workflows, let a laboratory benchmark its pipeline and confirm comparability across platforms. Canadian public health work assembled exactly such datasets for SARS-CoV-2 analysis from Illumina and [Nanopore sequencing](/blog/guides/nanopore-sequencing) of characterized samples, including clean controls and a range of lineages, and used them to validate pipeline performance across settings [11]. Run your full pipeline, including QC gates, against a test dataset with known answers before you point it at new samples. Expected behavior: known variants are called, known QC failures are flagged, and results match the reference outputs. If a pipeline update changes results on the test set, that difference is a finding, and it gets resolved before production runs continue. ## Step 10: Storage, Retention, and Metadata Discipline QC produces large intermediate files and the raw data is irreplaceable. Bioinformatics [data management](/blog/guides/data-management-basics-principles-processes-and-best-practices) at this point means three concrete practices. First, checksum everything that moves. Recompute MD5 sums after any transfer and store the checksum file alongside the data, not inside a private notebook. Second, tier your genomics data storage. Raw FASTQ files and final QC reports are the irreplaceable tier and belong on backed-up, access-controlled storage. BAM files and trimmed FASTQs are regenerable from raw data and can live on cheaper scratch or secondary storage with a documented regeneration path. This policy keeps capacity spending proportional to data value and is consistent with stewardship frameworks that treat storage design as part of quality management for genomic datasets [2]. Third, capture metadata at the moment of generation. End-to-end traceability in regulated production environments links every result to the batch, the instruments, and the processing steps that produced it, and auditable computerized systems are treated as a requirement rather than a preference in that setting [12]. The equivalent habit for a research lab is simple: every dataset ships with the sample sheet, tool versions, reference build, QC reports, and the QC gate decisions that admitted it. Data quality governance frameworks emphasize that quality assurance is continuous across the data lifecycle and that transparency about collection and curation underpins trust and reuse [1]. Guidelines for research data integrity formalize this as planned data handling across the full research process [4], and the underlying point is older than any framework: the value of a data collection depends on whether people can believe it. ## Key QC Gates Summary ![Comparison of delivery, trimmed, alignment, coverage and variant QC gates with the healthy signal and the action to take when each fails.](/article-images/illustrations/genomics-data-qc-checks-pipeline-qc-gate-thresholds.webp) | Stage | Metric | Healthy signal for standard short-read WGS | Action when failed | |--|--|--|--| | Delivery | md5sum | All OK | Re-transfer, contact provider | | Raw reads | Per-base quality (FastQC) | Lower quartile above Phred 20 for most of read | Trim, consider re-sequencing | | Raw reads | Adapter content (FastQC) | Near zero | Trim, review library prep | | Trimmed | Q30 rate (fastp) | Above roughly 85 percent | Escalate to sequencing core | | Alignment | Primary mapped | Above 95 percent | Check reference build, contamination | | Alignment | Properly paired | Above 90 percent | Check insert size, fragmentation | | Alignment | Duplicate rate | Single-digit percent | Review library complexity | | Coverage | Mean depth and breadth | Around 30x, breadth above 95 percent for WGS | Downsample expectations or re-sequence | | Variants | Transition to transversion ratio | Near 2.0 for human WGS | Investigate contamination and filters | ## Common Mistakes and Limitations **Treating FastQC warnings as failures.** FastQC was designed for standard Illumina libraries and its modules warn on patterns that are expected for many modern library types. Per-base sequence content warnings from random priming and GC warnings from targeted capture are usually benign. The module verdicts are conversation starters, not gates. **Trimming before diagnosing.** Aggressive trimming masks problems. A sample with 40 percent adapter contamination has a library prep failure, and trimming it into shape hides the root cause while leaving degraded complexity behind. Look at the raw profile first, then decide whether trimming is treatment or anesthesia. **Ignoring cross-sample comparison.** A sample can pass every per-sample check and still be an outlier. Depth outliers, heterozygosity outliers, and batch-level drift only appear when samples are plotted together. MultiQC or the automated dataset checks from step 7 exist for this reason. **Skipping the alignment QC when raw QC looks fine.** Raw quality says nothing about index hopping, sample swaps, or reference build mismatches. Those failures appear first as anomalous mapping and variant statistics, which is why steps 5 and 6 are not optional. **Using RIN as a sole RNA quality gate.** The Gene 1.0 ST array study showed that arrays with RIN values below 7 can pass array-level QC while other samples with acceptable RIN get flagged, and that degradation effects were transcript-specific in a small fraction of clusters [7]. Combine RIN with platform-level metrics rather than substituting one for the other. **Treating quality as a one-time event.** QC gates run at delivery, after trimming, after alignment, after variant calling, and at every new batch. Data quality frameworks consistently frame assurance as continuous across the lifecycle rather than a single checkpoint [1], and multi-source integrations face distinct threats at the collection stage that are best mitigated before integration processing begins [13]. **Storage sprawl without metadata.** Copying every intermediate to every location until nobody knows which BAM is authoritative is a storage policy failure, not a convenience. Define the tiers in step 10 once and enforce them. Genomics data storage that lacks associated metadata effectively destroys the dataset's reuse value even when the bytes are perfectly preserved [2]. **Undocumented gate decisions.** When you exclude sample S17 for low depth, that decision and its evidence belong in the project record. Undocumented exclusions are indistinguishable from cherry-picking to anyone reviewing the work later, and quality failures erode confidence in every downstream claim. **One honest limitation.** QC metrics tell you whether data is fit for the analysis you planned. They do not replace biological judgment about a specific sample, an unusual assay, or a clinical decision, which requires domain expertise applied to the individual case. ## Genomics Data QC Checks Pipeline Tips That Save Reruns 1. Run MultiQC after every stage, not just at the end. Trends across stages locate problems faster than any single report. 2. Freeze tool versions in the environment file and commit it with the pipeline. `mamba env export > environment.yml` takes seconds and makes results reproducible. 3. Write QC gate decisions into a plain text file in the project directory as you make them. Retrospective reconstruction of exclusions is where integrity problems start. 4. Run the automated dataset checks from step 7 on every incoming batch with the previous batch as the comparison set. Drift becomes a scheduled observation instead of a crisis. 5. Keep a test dataset from step 9 permanently in the repository and run the pipeline against it in continuous integration after every change. 6. Budget disk before the run: for human WGS, raw and trimmed FASTQ plus BAM plus index files routinely total well over 100 GB per 30x genome. Plan the tiers in step 10 before data arrives, not after the scratch volume fills. ## Frequently Asked Questions ### What does a genomics data QC checks pipeline actually check? It checks file integrity with checksums, raw read quality with per-base and per-sequence metrics, adapter and low-quality base content, alignment rates and duplicate rates, coverage depth and breadth, variant-level statistics such as the transition to transversion ratio, and dataset-level properties such as completeness and cross-sample consistency. ### Can I skip QC if my sequencing provider says they already ran it? No. Provider QC covers their instrument and library process, not your reference build, your pipeline, your sample identity, or your downstream analysis requirements. Independent QC is how you catch transfer corruption and sample swaps that provider reports cannot see. ### Which FastQC modules actually require action? Per-base sequence quality, adapter content, overrepresented sequences, and sequence duplication levels are the modules most often actionable. Per-base sequence content and GC content warnings are frequently expected for particular library types, so interpret them in context. ### How much duplication is too much? There is no universal cut-off because it depends on library type and depth. Single-digit duplicate rates are healthy for standard human short-read WGS, while high-depth applications naturally accumulate duplicates. Compare each sample against its cohort and against previous batches from the same library prep. ### Should I trim adapters before or after alignment? Before. Aligners are affected by adapter sequence, and trimming first produces cleaner mapping statistics that make the alignment QC in step 5 interpretable. The exception is specialized tools that handle adapter-bearing reads internally, where you follow the tool's documentation. ### How do I verify my QC pipeline is working correctly? Run it against a curated test dataset with known answers, including known quality failures that should be flagged, after every pipeline change. This mirrors how public health laboratories use standardized test datasets to confirm accuracy and comparability across platforms. ### What QC metadata should I store alongside the data? Store the sample sheet, checksums, tool versions, reference build, all QC reports, the gate thresholds you applied, and the exclusion decisions with their evidence. This record is what makes the dataset reusable and auditable by people who were not in the room. ### How do I share datasets that have known quality issues? Annotate the issues explicitly using standardized quality contextual tags of the kind developed by PHA4GE, so the dataset remains searchable and useful for training and validation without being mistaken for higher-quality data. ## Sources 1. [Data Quality in the Age of AI: A Review of Governance, Ethics, and the FAIR Principles](https://doi.org/10.3390/data10120201) 2. [Data stewardship and curation practices in AI-based genomics and automated microscopy image analysis for high-throughput screening studies: promoting robust and ethical AI applications](https://pubmed.ncbi.nlm.nih.gov/39988670/) 3. [PHA4GE quality control contextual data tags: standardized annotations for sharing public health sequence datasets with known quality issues to facilitate testing and training](https://pubmed.ncbi.nlm.nih.gov/38860884/) 4. [Guidelines for Research Data Integrity (GRDI)](https://pubmed.ncbi.nlm.nih.gov/39820528/) 5. [Ensuring survey research data integrity in the era of internet bots](https://pubmed.ncbi.nlm.nih.gov/34629553/) 6. [Data Quality Automation: a Generic Approach for Large Linked Research Datasets](https://doi.org/10.23889/IJPDS.V3I4.1000) 7. [Quality assessment and data handling methods for Affymetrix Gene 1.0 ST arrays with variable RNA integrity](https://pubmed.ncbi.nlm.nih.gov/23324084/) 8. [Investigation and reporting of Data Quality within and between linked SAIL datasets](https://doi.org/10.23889/ijpds.v1i1.99) 9. [Data Quality Quantized Framework: Ensuring Large-Scale Data Integration in Gig Economy Platforms](https://doi.org/10.1145/3770177.3770217) 10. [Sensitive Data Flagging within Data Quality reports: R and Regex Integration for Effective Text Flagging in Large Datasets](https://doi.org/10.23889/ijpds.v9i5.2730) 11. [Optimizing data-driven excellence: Canada’s approach to using pathogen test datasets for quality control, pipeline development and training initiatives](https://pubmed.ncbi.nlm.nih.gov/41591806/) 12. [Smart Pharmaceutical Manufacturing: Ensuring End-to-End Traceability and Data Integrity in Medicine Production](https://doi.org/10.1016/J.BDR.2020.100172) 13. [Data Quality Threat Mitigation Strategies for Multi-Sourced Linked Data](https://doi.org/10.5220/0013462900003967)