Downloading Sequencing Data From the Sequence Read Archive

If you need to obtain raw high-throughput sequencing data for reanalysis or method development, the Sequence Read Archive (SRA) at NCBI is the primary public repository. This guide walks you through downloading those data efficiently and correctly. It is written for bioinformaticians, students, and wet-lab researchers who have basic command-line familiarity and need to transfer SRA files to their local or cloud environment. You can start by visiting the NCBI Sequence Read Archive to explore available datasets.

The process involves understanding accession codes, locating the correct files, choosing between several download tools, verifying data integrity, and planning for storage. Official training from EMBL-EBI Training offers complementary material for European data sources. This guide provides a practical, step-by-step approach grounded in authoritative resources.

At a Glance

Understanding SRA Accession Types

Every dataset in SRA is organized hierarchically. A Study accession (e.g., SRP123456) groups all samples and experiments. A Sample accession (SRS) identifies the biological source. An Experiment accession (SRX) describes a specific sequencing assay. The actual sequence data are stored as Run accessions (SRR) for a single sequencing run or Run Set (SRZ) for large studies. European Bioinformatics Institute (ENA) uses analogous prefixes: ERP, ERS, ERX, ERR.

Knowing this structure helps you request exactly the files you need. For example, to download all runs from a study, you use the study accession. The NCBI Bookshelf provides detailed explanations of these identifiers in the SRA documentation.

Finding and Interpreting Metadata

Before downloading, you must understand which runs correspond to your conditions of interest. The NCBI Run Selector is the most common starting point. Enter a study accession, and the table shows sample attributes, library layout, platform, and byte sizes. You can filter by column values and download a metadata file (SraRunTable.txt). For data hosted at ENA, the EMBL-EBI Training materials explain how to use the ENA Browser to retrieve similar metadata.

Make sure to check library design: single end versus paired end, strandedness, and read length. These details affect downstream processing. Tools like q2-fondue can also automate metadata acquisition and integration, as described in a reproducibility-focused publication. Always record the accession numbers of all selected runs for your analysis records.

Choosing a Download Method

Several download methods exist, each with its own tradeoffs.

SRA Toolkit command line is the most reliable method. Install the toolkit from NCBI, then use prefetch to download SRA files and fasterq-dump to convert them to FASTQ format. For example: prefetch SRR123456 and then fasterq-dump SRR123456. The toolkit automatically handles decompression and split files for paired-end reads. The Galaxy Training Network offers interactive tutorials on using SRA Toolkit within Galaxy workflows.

Aspera is a faster alternative for large datasets. If you have Aspera Connect installed, you can use ascp with SRA's dedicated endpoint. This method is often used in high-performance computing environments.

Graphical and programmatic options exist. The R/Bioconductor package SRAdb allows querying and downloading directly from R. For those using QIIME 2, the q2-fondue plugin provides guided acquisition, as noted in the q2-fondue publication. Another tool, COMMAND>_, described in this transcriptomic data integration paper, assists with downloading and organizing data from multiple repositories.

Cloud and mirror sites can be used. The AWS Open Data program hosts SRA data in S3, you can use aws s3 cp or wget on the corresponding URLs. ENA provides FTP downloads for many runs.

Recommendation: Use fasterq-dump for most scenarios. It is actively maintained and produces standard FASTQ files. For very large projects, consider Aspera or cloud direct download to avoid timeouts.

Ensuring Data Integrity

After downloading, verify that the file is not corrupted. NCBI provides MD5 checksums for every SRA file. After running prefetch, you can check the .sra file integrity using vdb-validate SRR123456. This command recomputes the checksum and compares it to the stored value in the SRA metadata. For FASTQ files produced by fasterq-dump, you should also compare the size and read count against the information in the Run Selector. The NCBI Bookshelf contains the official documentation for validation commands.

If you use ENA downloads, the FTP site provides .md5 files. Use md5sum on Linux or certutil on Windows to verify.

Planning Storage and Organization

Sequencing data are large. A single RNA-seq run (30 million paired-end reads, 100 bp) is roughly 6 GB compressed. A project with 100 runs needs 600 GB. Plan your storage accordingly. Compress FASTQ files with gzip or pigz to reduce space by about 60%.

Organize your files by study and run. For example:

data/SRP123456/
   metadata/
   raw/
      SRR123456_1.fastq.gz
      SRR123456_2.fastq.gz
   alignment/

Keep a manifest file listing each run accession, its original download date, and the tool used. This practice supports reproducibility. The Bioinformatics training from Bioconductor offers guidance on file organization within analysis projects.

Responsible Reuse and Citation

When you reuse public data, you must credit the original authors. Cite the study accession (SRP/ERP) and, if possible, the associated publication. Many studies have embargo periods, check the release date in the SRA record. Do not redistribute raw data, but you may share derived files (e.g., count matrices) with proper attribution. The NCBI Sequence Read Archive includes data use policies for each study.

Common Mistakes

• Downloading the wrong run: Always verify that the run matches your experimental condition. Use metadata filters.
• Using fastq-dump instead of fasterq-dump: The older fastq-dump is slower and may produce incorrect split files for paired-end data. Use the newer fasterq-dump.
• Ignoring checksums: Skipping validation can lead to using corrupted files and wasted downstream analysis.
• Not planning for disk space: Running out of storage mid-download causes incomplete files. Check available space and use streaming or chunking if needed.
• Assuming all data are in SRA: Some datasets are only in ENA or DDBJ. Use meta-search tools like NCBI's global query.
• Mixing paired-end read files: Ensure that _1 and _2 files correspond correctly. Use fasterq-dump --split-files to produce properly ordered pairs.

Limits and Uncertainty

SRA data are not always consistent. Some older submissions may lack metadata or have incorrect library layouts. The SRA Toolkit occasionally changes its API or command syntax, so always check the current version. Download speeds can be slow for large studies, consider using mirrors or Aspera. Not all data are available for direct download due to controlled access (e.g., human genomic data requiring dbGaP authorization). In such cases, apply for access through the appropriate portal. Finally, cloud-hosted copies may lag behind the main repository, so verify you have the latest version.

Frequently Asked Questions

1. How do I download only paired-end reads from a study? Use the Run Selector to filter by Library Layout equal to "PAIRED". Then download the list of SRR accessions and pass them to prefetch. When you run fasterq-dump, use the --split-files option to get separate _1 and _2 FASTQ files.

2. What is the difference between prefetch and fasterq-dump? prefetch downloads the SRA binary file (.sra) to your cache. It does not convert to FASTQ. fasterq-dump reads the SRA file and writes FASTQ output, optionally compressing it. You can run fasterq-dump directly on an accession without first running prefetch, but it will download the data on the fly if not cached.

3. Can I download SRA data from a European mirror? Yes. The European Nucleotide Archive (ENA) hosts copies of most SRA data. You can use their API or FTP to download FASTQ files directly, often faster than from NCBI. The EMBL-EBI Training resources explain how to use ENA's download services.

4. How do I verify that a download is complete without checksums? You can compare the number of reads in the FASTQ file to the "spots" count listed in the SRA Run Selector metadata. Use wc -l and divide by 4 (for single end) or by 8 (for paired end) to get the read count. This method is less reliable than checksum validation but provides a quick sanity check.

References and Further Reading

• NCBI Bookshelf , Official SRA documentation and command reference.
• EMBL-EBI Training , Tutorials on European data access and metadata retrieval.
• Galaxy Training Network , Hands-on workflows for downloading and processing SRA data.
• Bioconductor , R packages like SRAdb and ShortRead for programmatic access.
• NCBI Sequence Read Archive , Primary repository, start here to search for datasets.
• q2-fondue publication , Reproducible acquisition and management of nucleotide sequence metadata.
• COMMAND>_ publication , Transcriptomic data acquisition and integration tool.

Related Articles

• RNA Sequencing Analysis: From FASTQ Files to Biological Questions
• RNA-seq Quality Control: What to Check Before Differential Expression
• How to Plan a Bulk RNA-seq Differential Expression Study
• Single-Cell RNA-seq Workflow: A Practical Analysis Roadmap
• Single-Cell RNA-seq Quality Control: Cells, Genes, and Mitochondrial Reads