# bgzip vs gzip: Which Compression Tool Should You Use gzip makes a file smaller. bgzip makes a file smaller and also lets software jump directly to any genomic coordinate without decompressing everything that comes before it. The rule of thumb is short: if a downstream tool needs to seek, meaning tabix, bcftools, samtools, IGV, bedtools against an index, or an aligner reading a reference, use bgzip. If the file is a dead end that only needs to be archived, uploaded, or emailed (a FASTQ delivery, a tarball of results, a supplementary table), gzip is fine and usually produces a slightly smaller file. The two formats are closer than the names suggest. bgzip output is valid gzip output, so every tool that reads gzip can read bgzip. The reverse is not true, and that asymmetry is the source of most of the errors people hit in real pipelines. ## What gzip actually does gzip is a general-purpose lossless compressor built on DEFLATE, the same algorithm inside PNG images and ZIP archives. Lossless means the decompressed bytes are identical to the input, bit for bit, which matters because a single wrong base or a corrupted quality score can invalidate a variant call. DEFLATE combines two stages. First, LZ77 scans the input through a 32 KB sliding window and replaces repeated substrings with back-references (a distance and a length). Second, Huffman coding assigns short bit patterns to frequent symbols and long patterns to rare ones. A `.gz` file consists of a short header, one continuous DEFLATE stream, a CRC32 checksum, and the uncompressed size modulo 2³². The format is specified in RFC 1952, which is why any language with a zlib binding can read it. Compression levels 1 through 9 trade processor time for file size, with 6 as the default and 9 as the usual choice for archival. The important architectural detail is that a `.gz` file is one stream. To read the byte at offset 900,000,000, a decompressor has to start at offset 0 and inflate everything in front of it. gzip has no concept of records, chromosomes, coordinates, or blocks. It is a byte pipe that happens to be small. gzip tolerates concatenation. Two `.gz` files joined end to end form a valid multi-member gzip file, and `gunzip` will emit both payloads in order. That quirk is why bgzip files remain readable by gzip: BGZF is simply a disciplined sequence of gzip members. gzip dominates bioinformatics storage because it is universal and free. In annotation work it is the accepted default, and GPress reports an average 61% size reduction over gzip when compressing GFF files, which only makes sense if gzip is the baseline everyone compares against [1]. The same holds for whole-genome alignments, where MAFCO gains 34% to 57% over gzip depending on the dataset [2]. General-purpose compressors are not built for the statistics of sequencing data, a point made directly by the developers of SCALCE, who note that gzip does not exploit the limited alphabet or the high similarity among reads [3]. ## What bgzip actually does bgzip (block gzip) is part of HTSlib, the C library behind samtools, bcftools, and tabix. It writes BGZF, a format defined in the SAM/BAM specification. BGZF cuts the input into blocks of up to 65,280 bytes of uncompressed data. Each block is compressed independently and wrapped as a complete gzip member, with an extra header field labeled `BC` that records the compressed length of that block. The file is terminated by a special 28-byte empty block that marks end of file. Three consequences follow. First, a BGZF file is a valid gzip file. `gunzip`, `zcat`, Python's `gzip` module, and Java's `GZIPInputStream` all read it correctly. They see a series of gzip members and concatenate the output. This is the single most useful fact about bgzip and the reason you can rename a `.vcf.gz` to `.gz` and still inspect it with standard Unix tools. Second, each block decompresses on its own. A BGZF-aware reader can seek to any block boundary, and a companion `.gzi` index maps uncompressed offsets to compressed offsets. The virtual offset used by HTSlib packs two numbers into 64 bits: the upper 48 bits give the compressed block start, and the lower 16 bits give the offset inside the decompressed block. That scheme is what allows samtools to fetch a 200 bp region from a 60 GB BAM in milliseconds. Third, a coordinate index can be layered on top. tabix builds a `.tbi` index (or a `.csi` index for very large chromosomes or many contigs) over a bgzipped, position-sorted, tab-delimited file. The index stores the genomic intervals in bins, so a query for `chr1:1000000-1200000` resolves to a handful of virtual offsets and reads only those blocks. The cost is modest but real. Independent blocks compress slightly worse than one long stream, and each block carries a few bytes of overhead. bgzip is also slower per byte than gzip on the same hardware, though HTSlib's `-@` flag parallelizes block compression across threads and can close the gap on multi-core machines. File extensions in the wild are inconsistent: `.vcf.gz`, `.bed.gz`, `.gz`, and less commonly `.bgz` all appear. The extension tells you nothing reliable. What matters is whether the bytes are BGZF, and there is a one-command test for that. ## Side-by-side comparison | Property | gzip | bgzip (BGZF) | |--|--|--| | Underlying format | RFC 1952 single DEFLATE stream | Series of independent gzip members with `BC` extra field | | Common extensions | `.gz`, `.tgz` | `.gz`, `.bgz` | | Readable by standard gzip tools | Yes | Yes | | Random access to uncompressed offsets | No | Yes, via `.gzi` index | | Coordinate queries | No | Yes, via tabix or HTSlib | | Index files produced | None | `.gzi`, `.tbi`, `.csi` | | Compatible with tabix | No | Yes | | Compatible with `bcftools -r` and `samtools view chr:start-end` | No | Yes | | Compression ratio | Slightly better at equal settings | Slightly worse, due to block boundaries | | Compression speed | Faster per byte | Slower per byte, scales with `-@` threads | | Multithreading | Only through `pigz`, a separate tool | Built in with `-@` | | Streaming through pipes | Yes | Yes, with block-sized buffering | | Typical block or window | 32 KB LZ77 window | 65,280-byte independent blocks | | Best use | Archival, FASTQ, tarballs, terminal files | VCF, BED, GFF, SAM, reference FASTA, GAF, anything queried | ## How to tell which format you have `file sample.vcf.gz` is not enough. It prints "gzip compressed data" for both formats. Use one of these instead. `htsfile sample.vcf.gz` is the direct answer from HTSlib and reports "gzip compressed" for plain gzip and "BGZF-compressed" for bgzip. If `htsfile` is not installed, `bcftools view -h sample.vcf.gz` will succeed on BGZF and fail on plain gzip for indexed operations. A quick byte check also works: BGZF files carry the `BC` subfield in the gzip extra header, which starts at byte 12, and the ASCII bytes `B` and `C` appear near the beginning of every block. The practical test most people run first is `tabix -l sample.vcf.gz`. On a plain gzip file it fails immediately with a message about the file not being BGZF. That error is diagnostic, not a code bug. ## Commands you will actually type Compress to bgzip and keep the original: ``` bgzip -k input.vcf ``` Compress to standard output (the usual form in a pipeline): ``` bgzip -c input.vcf > input.vcf.gz ``` Compress with eight threads at maximum level: ``` bgzip -@ 8 -l 9 input.vcf ``` Index a sorted VCF or BED for region queries: ``` tabix -p vcf input.vcf.gz tabix -p bed input.bed.gz ``` Build a CSI index instead of a TBI when a chromosome exceeds 512 Mb: ``` tabix -C -p vcf input.vcf.gz ``` Query a region without decompressing the whole file: ``` tabix input.vcf.gz chr1:1000000-1200000 bcftools view -r chr1:1000000-1200000 input.vcf.gz ``` Decompress, or decompress with plain gzip because the format is compatible: ``` bgzip -d input.vcf.gz gunzip -c input.vcf.gz > input.vcf ``` Convert an existing gzip file to bgzip. The data is unchanged, only the block structure is rewritten: ``` gzip -dc old.vcf.gz | bgzip -c > new.vcf.gz ``` Index a bgzipped FASTA for random access: ``` bgzip -r reference.fa samtools faidx reference.fa.gz ``` That last pair is worth internalizing. `bgzip -r` writes a `.gzi` index alongside the compressed file, and `samtools faidx` writes the `.fai` plus, for compressed input, the `.gzi` it needs. Both files must travel with the `.fa.gz`. ## Where each tool fits in real workflows ### Variant files: bgzip is mandatory VCF is line-oriented and sorted by chromosome then position, which makes it a natural target for interval indexing. The standard architecture is bgzip for compression plus tabix or a CSI index for queries. Any pipeline that ships a `.vcf.gz` without a companion `.tbi` is shipping a file that cannot be queried, only scanned. The scale of this problem is why new tools keep appearing around the bgzip and tabix baseline. STABIX was built specifically to improve on standard bgzip compression and Tabix querying, and across ten PanUKBB GWAS files its compressed data and indexes averaged 1.2 times smaller than the bgzip plus `.tbi` files, with per-gene decompression about 7 times faster than Tabix [4]. That result only makes sense against a backdrop where bgzip and tabix are the accepted default for distributed GWAS summary statistics. VCF compression also interacts with privacy-preserving access patterns. A portable JavaScript module has been described that fetches partial content from large compressed VCF files using byte-range HTTP requests, decompressing irregularly positioned compressed chunks on the fly and running a binary search to locate chromosome-position ranges [5]. Byte-range fetching only works because the file is chunked and addressable, which is precisely what BGZF provides. ### Interval and annotation files: bgzip plus tabix BED, GFF, GTF, and similar tab-delimited annotation formats follow the same logic. Sort by coordinate, bgzip, index with the matching tabix preset, query by region. The alternative is to gzip and accept that every query reads the entire file, which is tolerable for a 2 MB file and painful for a 2 GB one. Annotation files are often the largest text artifacts a lab produces after raw reads. GPress was developed because existing utilities like gffutils and gffread were not focused on storage, and in some cases increased file size, while gzip remained the de facto compressor for GFF [1]. bgzip does not beat gzip on ratio, but it makes the file queryable, which is usually the more valuable property. ### Alignment files: BAM is BGZF by definition A BAM file is a BGZF-compressed binary alignment. The BGZF layer is not optional and not a user choice. A SAM file, which is the plain-text version of the same records, can be bgzipped before conversion or simply streamed. CRAM, the reference-based successor, uses a different columnar compression scheme, but it preserves the same indexed-region-query behavior that makes alignment files usable at scale. Data structures and compression algorithms for high-throughput sequencing data have been studied since the earliest short-read era, and the consistent finding is that reference-aware encoding plus entropy coding beats general-purpose compression on both size and speed [6]. ### Reference FASTA: bgzip enables faidx Reference genomes are distributed as `.fa.gz` files that are bgzipped and paired with `.fai` and `.gzi` indexes. That combination lets `samtools faidx reference.fa.gz chr1:1000-2000` extract a region without inflating the 3 GB file. Plain gzip makes the file smaller but removes the ability to do this, which is why aligner indexes and reference bundles are almost always bgzipped. FASTA itself carries sequence and headers only. FASTQ carries the same sequence plus a quality string and a `+` separator line, which roughly triples or quadruples the per-base storage cost. That distinction drives most compression decisions downstream. The quality score is the least compressible part of a FASTQ file, since the base sequence has a four-letter alphabet and heavy repetition while quality strings encode per-base error probabilities with high entropy [7]. The same asymmetry explains why dedicated quality-score compressors exist at all. ### FASTQ: gzip is the right default FASTQ has no coordinates. You cannot ask for "reads overlapping chr7:55,000,000" because reads are not sorted by position and the file has no index. Random access is meaningless. That removes the main argument for bgzip. gzip is the interoperable choice. Every sequencing center, every SRA submission tool, every aligner, and every language reads it. GeneSqueeze, a reference-free FASTQ compressor, was benchmarked against gzip and SPRING, and both GeneSqueeze and gzip achieved 100% lossless compression across read identifiers, sequences, quality scores, and the `+` lines, with GeneSqueeze slower than gzip in its current Python implementation [8]. That is the honest trade: gzip is not the smallest option, but it is the fastest practical general-purpose one and it loses nothing. Dedicated FASTQ compressors do beat gzip on size. RBFQC reports a compression ratio of 80% to 140% better than GZIP on fixed-length datasets and 80% to 125% on variable-length datasets, by splitting the FASTQ into identifier, sequence, and quality streams and mapping the sequence stream against a reference [9]. DSRC 2 was built because specialized FASTQ compressors were often impractical due to slow processing or missing FASTQ variants, and it targets ratios comparable to the best existing tools while being several times faster [10]. None of these change the fact that gzip is what your collaborator can open without installing anything. ### Pangenome GAF: bgzip extends to new formats The Graph Alignment Format is a tab-delimited text format for alignments and path annotations in a pangenome graph. Recent work extended vg and HTSlib to sort, index, and query GAF files by compressing paths with BGZIP and indexing them with HTSlib and tabix through new format definitions [11]. Compared with the binary GAM format, GAF files were twice as fast to sort and half as large on disk [11]. The lesson generalizes: any new text format that needs region queries gets bgzip plus an index, not a new compressor. ### BigWig and BigBed: a different design, same principle BigWig and BigBed are compressed binary indexed files that store data at multiple resolutions, so the genome browser transmits only the data needed for the current view rather than the whole file [12]. They are not BGZF, and bgzip is not the right tool for producing them. The architectural idea is identical, though: compress in addressable units, add an index, and fetch by region. When a format already has a native indexed binary container, use that container rather than wrapping text with bgzip. ## What compression actually buys you The headline numbers from dedicated genomic compressors show how much headroom exists beyond general-purpose tools. A collection of 1,092 human diploid genomes was compressed about 9,500 times, needing roughly 700 MB against about 6.7 TB of uncompressed FASTA, which is about four times better than other existing compressors at the time [13]. For nanopore reads, NanoSpring uses 0.35 to 0.65 bits per base on recently basecalled high-quality datasets, which is 3 to 6 times lower than general-purpose compressors like gzip [14]. A nongreedy unbalanced Huffman tree compressor for FASTA and multi-FASTA files outperformed gzip, bzip2, and an earlier DNA-specific Huffman compressor on ratio while running seven times faster than the earlier Huffman approach [15]. Each of these is a specialized tool that trades universality for size. bgzip does not compete in that race. It is a format, not a ratio champion. Its value is that it costs a few percent in size relative to gzip and returns indexed random access, which is a different kind of currency. ## Worked scenarios **1. tabix refuses your VCF.** You downloaded `cohort.vcf.gz` from a collaborator, ran `tabix -p vcf cohort.vcf.gz`, and got a message that the file is not BGZF. The cause is that the file was made with `gzip` rather than `bgzip`. Fix it with `gzip -dc cohort.vcf.gz | bgzip -c > cohort.bgz.vcf.gz`, then index again. No data is lost, only the block structure changes. **2. You are archiving 2 TB of FASTQ.** Use gzip or pigz. Nothing downstream needs to seek inside a FASTQ file, and the smaller file size and faster compression reduce storage and transfer cost. Sort, rename, and validate the files first, because compression bakes in whatever mistakes are already there. If storage is the dominant constraint, a specialized FASTQ compressor will do better, at the cost of requiring that tool to decompress later [9][10]. **3. You are serving a BED track to a genome browser.** bgzip plus tabix. The browser issues hundreds of small range requests as the user scrolls, and a plain gzip file forces a full download for each one. BigWig or BigBed is better still if the data is numeric or interval-based and you can convert, because those formats store multiple resolutions and avoid sending detail the current zoom level cannot show [12]. **4. You need a reference FASTA for an aligner.** bgzip the FASTA, run `samtools faidx`, confirm that both `.fai` and `.gzi` exist, and keep all three files together. Aligners and variant callers will retrieve contigs by name. A plain gzipped reference works only if the tool decompresses the entire file to disk first, which doubles the storage requirement during indexing. **5. You are sharing a VCF with a clinical collaborator.** bgzip plus tabix plus the `.tbi`. Include a short note listing the reference build, because an unindexed or wrongly indexed VCF is a support ticket waiting to happen. If the file exceeds the 512 Mb chromosome limit that TBI indexes handle, use `tabix -C` to produce a CSI instead. **6. You are compressing GFF annotation.** bgzip plus `tabix -p gff`. The result is queryable by region and stays readable by every gzip tool. If storage is the only concern and nobody will query it, gzip is acceptable, and GPress can reduce GFF size by about 61% relative to gzip when query support is also wanted [1]. ## Common Mistakes and Limitations **Indexing a gzip file.** tabix, `bcftools index`, and `samtools index` all require BGZF. A plain gzip file will fail with a clear error if you are lucky and a confusing one if the tool tries to stream first. Always run `htsfile` before blaming the indexer. **Assuming `.gz` means bgzip.** The extension is shared. Only the byte-level format distinguishes them, and every `.vcf.gz` from a public repository should be checked once before you build a pipeline around it. **Forgetting to ship the index.** A `.vcf.gz` without its `.tbi` is a flat file. Region queries will fail or fall back to a full scan depending on the tool. Copy the index alongside the data, and regenerate it if the data changes, because a stale index silently returns wrong regions rather than raising an error. **Editing a bgzipped file in place.** Running `sed -i` on a `.gz` file, or appending lines with `>>`, corrupts the block structure. Decompress, edit, re-sort, re-compress, and re-index. The same applies to any tool that writes text without going through bgzip. **Losing the original with `bgzip`.** Unlike `gzip -k`, the default behavior of `bgzip` is to replace the input with the compressed output. Use `-k` when you want to keep both, and check before running it on a file that took three days to produce. **Expecting bgzip to compress better than gzip.** It usually compresses slightly worse at the same level, because independent blocks cannot exploit redundancy across block boundaries. The BC extra field and the EOF block add a little more overhead. On small files the difference is proportionally larger, and bgzip is a poor choice for a 50 KB configuration file. **Ignoring thread counts.** bgzip without `-@` uses one core and can be several times slower than gzip on a modern machine. For a 500 GB VCF conversion, set `-@` to the number of available cores or the job will run overnight for no reason. **Compressing unsorted data.** tabix requires position-sorted input. bgzipping an unsorted BED produces a valid compressed file that indexes incorrectly or fails to index at all. Sort first with `sort -k1,1 -k2,2n`, then compress. **Using gzip on a file that will be queried later.** This is the expensive mistake. Converting a 400 GB gzipped VCF to bgzip later means a full decompress and recompress cycle, and the intermediate uncompressed file may not fit on disk. Decide the format at creation time. **Truncated BGZF files.** A bgzip file missing its 28-byte EOF block will often still decompress, because the blocks themselves are intact, but integrity checks and some tools will flag it. If a transfer was interrupted, re-download rather than assuming the file is fine because `zcat` produced output. **Confusing compression with validation.** A small file is not a correct file. Compression preserves whatever errors, adapter contamination, duplicated reads, or wrong reference build the input contained. Run your quality checks before you compress, not after. **Limitations of the format itself.** bgzip does not make a text format queryable by content, only by coordinate. If you need to find all variants in a p-value range across a summary-statistics file, a coordinate index does not help, which is exactly the gap STABIX was designed to fill by adding summary-statistic-based queries on top of the bgzip and tabix model [4]. Individual research questions still need the right tool, and clinical interpretation of any specific variant requires a qualified professional. ## Frequently Asked Questions ### Can tabix index a regular gzip file? No. tabix requires BGZF, the block format written by bgzip. Convert the file first with `gzip -dc input.vcf.gz | bgzip -c > output.vcf.gz`, then index the result. ### Is bgzip just gzip with a different name? No, but the output is a valid gzip file. bgzip writes a series of independently compressed gzip members with an extra header field recording each block size, which is what enables seeking. Standard gzip tools read the result correctly because concatenated gzip members are legal. ### Which compresses smaller, bgzip or gzip? gzip usually wins by a small margin at the same compression level. bgzip compresses each block independently, so it cannot exploit redundancy across block boundaries, and it carries a few bytes of overhead per block. The difference is a few percent on typical genomic files. ### Do I need bgzip for FASTQ files? No. FASTQ files have no coordinates and nothing queries into them by position, so random access buys you nothing. gzip is faster, universally supported, and typically smaller. Reserve bgzip for formats that get indexed, such as VCF, BED, GFF, SAM, and reference FASTA. ### Can I use zcat on a bgzipped file? Yes. `zcat`, `gunzip -c`, and every standard gzip library will decompress a bgzipped file correctly because BGZF is built from valid gzip members. You will get the exact original bytes, just without the ability to seek. ### Is bgzip faster than gzip? No, it is slower per byte on a single core because of per-block overhead. Adding `-@` lets bgzip compress blocks in parallel across threads, which closes the gap on multi-core machines and can exceed gzip throughput on large files. ## Sources 1. [GPress: a framework for querying general feature format (GFF) files and expression files in a compressed form.](https://pubmed.ncbi.nlm.nih.gov/32609343/) 2. [MAFCO: a compression tool for MAF files.](https://pubmed.ncbi.nlm.nih.gov/25816229/) 3. [SCALCE: boosting sequence compression algorithms using locally consistent encoding.](https://pubmed.ncbi.nlm.nih.gov/23047557/) 4. [STABIX: summary-statistic-based GWAS indexing and compression.](https://pubmed.ncbi.nlm.nih.gov/40315153/) 5. [FAIR privacy-preserving operation of large genomic variant calling format (VCF) data without download or installation.](https://pubmed.ncbi.nlm.nih.gov/38827109/) 6. [Data structures and compression algorithms for high-throughput sequencing technologies.](https://pubmed.ncbi.nlm.nih.gov/20946637/) 7. [LCQS: an efficient lossless compression tool of quality scores with random access functionality.](https://pubmed.ncbi.nlm.nih.gov/32183707/) 8. [Lossless and reference-free compression of FASTQ/A files using GeneSqueeze.](https://pubmed.ncbi.nlm.nih.gov/39747361/) 9. [A new efficient referential genome compression technique for FastQ files.](https://pubmed.ncbi.nlm.nih.gov/37950100/) 10. [DSRC 2--Industry-oriented compression of FASTQ files.](https://pubmed.ncbi.nlm.nih.gov/24747219/) 11. [Efficient indexing and querying of annotations in a pangenome graph.](https://pubmed.ncbi.nlm.nih.gov/39464141/) 12. [BigWig and BigBed: enabling browsing of large distributed datasets.](https://pubmed.ncbi.nlm.nih.gov/20639541/) 13. [GDC 2: Compression of large collections of genomes.](https://pubmed.ncbi.nlm.nih.gov/26108279/) 14. [Reference-free lossless compression of nanopore sequencing reads using an approximate assembly approach.](https://pubmed.ncbi.nlm.nih.gov/36747011/) 15. [Nongreedy Unbalanced Huffman Tree Compressor for Single and Multifasta Files.](https://pubmed.ncbi.nlm.nih.gov/31553226/)