/bio-compressed-files

name: bio-compressed-files
description: Read and write compressed sequence files (gzip, bzip2, BGZF) using Biopython. Use when working with .gz or .bz2 sequence files. Use BGZF for indexable compressed files.
tool_type: python
primary_tool: Bio.bgzf

Version Compatibility

Reference examples tested with: BioPython 1.83+, samtools 1.19+

Before using code patterns, verify installed versions match. If versions differ:

• Python: `pip show <package>` then `help(module.function)` to check signatures

If code throws ImportError, AttributeError, or TypeError, introspect the installed package and adapt the example to match the actual API rather than retrying.

Compressed Files

Handle gzip, bzip2, and BGZF compressed sequence files with Biopython.

**"Read a compressed sequence file"** → Open a compressed file handle in text mode, then parse with the standard SeqIO interface.

• gzip: `gzip.open(path, 'rt')` (Python stdlib)
• bzip2: `bz2.open(path, 'rt')` (Python stdlib)
• BGZF: `bgzf.open(path, 'rt')` (BioPython) or direct `SeqIO.parse(path, fmt)`

**"Make a compressed file indexable"** → Convert to BGZF format. Only BGZF supports `SeqIO.index()` on compressed data.

Required Imports

import gzip
import bz2
from Bio import SeqIO
from Bio import bgzf

Reading Compressed Files

**Goal:** Parse sequence records from compressed files without decompressing to disk.

**Approach:** Open a decompression handle in text mode (`'rt'`), then pass the handle to `SeqIO.parse()`. The parser works identically to uncompressed input.

Gzip (.gz) (BioPython 1.83+)

with gzip.open('sequences.fasta.gz', 'rt') as handle:
    for record in SeqIO.parse(handle, 'fasta'):
        print(record.id, len(record.seq))

**Important:** Use `'rt'` (read text) mode, not `'rb'` (read binary).

Bzip2 (.bz2) (BioPython 1.83+)

with bz2.open('sequences.fasta.bz2', 'rt') as handle:
    for record in SeqIO.parse(handle, 'fasta'):
        print(record.id, len(record.seq))

BGZF (Block Gzip) (BioPython 1.83+)

BGZF files can be read like regular gzip, but also support indexing:

for record in SeqIO.parse('sequences.fasta.bgz', 'fasta'):
    print(record.id)

with bgzf.open('sequences.fasta.bgz', 'rt') as handle:
    for record in SeqIO.parse(handle, 'fasta'):
        print(record.id)

Writing Compressed Files

**Goal:** Save sequence records directly to compressed files without an intermediate uncompressed step.

**Approach:** Open a compression handle in text mode (`'wt'`), then pass it to `SeqIO.write()`.

Gzip (.gz)

with gzip.open('output.fasta.gz', 'wt') as handle:
    SeqIO.write(records, handle, 'fasta')

Bzip2 (.bz2)

with bz2.open('output.fasta.bz2', 'wt') as handle:
    SeqIO.write(records, handle, 'fasta')

BGZF (.bgz)

with bgzf.open('output.fasta.bgz', 'wt') as handle:
    SeqIO.write(records, handle, 'fasta')

BGZF: Indexable Compression

**Goal:** Enable random access to records in compressed sequence files.

**Approach:** Write sequences in BGZF (Block GZip Format) — the only compressed format supporting `SeqIO.index()` and `SeqIO.index_db()`. BGZF is a gzip variant used by BAM and tabix-indexed files.

Create Indexable Compressed File

from Bio import SeqIO, bgzf

records = SeqIO.parse('input.fasta', 'fasta')
with bgzf.open('output.fasta.bgz', 'wt') as handle:
    SeqIO.write(records, handle, 'fasta')

Index a BGZF File

records = SeqIO.index('sequences.fasta.bgz', 'fasta')
seq = records['target_id'].seq
records.close()

records = SeqIO.index_db('index.sqlite', 'sequences.fasta.bgz', 'fasta')

Convert Gzip to BGZF

**"Convert gzip to indexable format"** → Parse from gzip handle, write through BGZF handle.

from Bio import SeqIO, bgzf
import gzip

with gzip.open('input.fasta.gz', 'rt') as in_handle:
    with bgzf.open('output.fasta.bgz', 'wt') as out_handle:
        SeqIO.write(SeqIO.parse(in_handle, 'fasta'), out_handle, 'fasta')

Code Patterns

Read Gzipped FASTQ

with gzip.open('reads.fastq.gz', 'rt') as handle:
    records = list(SeqIO.parse(handle, 'fastq'))
print(f'Loaded {len(records)} reads')

Count Records in Gzipped File

with gzip.open('sequences.fasta.gz', 'rt') as handle:
    count = sum(1 for _ in SeqIO.parse(handle, 'fasta'))
print(f'{count} sequences')

Fast Count with Low-Level Parser

from Bio.SeqIO.FastaIO import SimpleFastaParser
import gzip

with gzip.open('sequences.fasta.gz', 'rt') as handle:
    count = sum(1 for _ in SimpleFastaParser(handle))

Convert Compressed to Uncompressed

with gzip.open('input.fasta.gz', 'rt') as in_handle:
    records = SeqIO.parse(in_handle, 'fasta')
    SeqIO.write(records, 'output.fasta', 'fasta')

Convert Uncompressed to Compressed

records = SeqIO.parse('input.fasta', 'fasta')
with gzip.open('output.fasta.gz', 'wt') as out_handle:
    SeqIO.write(records, out_handle, 'fasta')

Auto-Detect Compression

from pathlib import Path
from Bio import SeqIO, bgzf
import gzip
import bz2

def open_sequence_file(filepath, format):
    filepath = Path(filepath)
    suffix = filepath.suffix.lower()
    if suffix == '.gz':
        # Could be gzip or bgzf - bgzf handles both
        handle = bgzf.open(filepath, 'rt')
    elif suffix == '.bgz':
        handle = bgzf.open(filepath, 'rt')
    elif suffix == '.bz2':
        handle = bz2.open(filepath, 'rt')
    else:
        handle = open(filepath, 'r')
    return SeqIO.parse(handle, format)

Process Large Gzipped File (Memory Efficient)

with gzip.open('large.fastq.gz', 'rt') as handle:
    for record in SeqIO.parse(handle, 'fastq'):
        if len(record.seq) >= 100:
            process(record)

Compress Existing File (Raw Copy)

import shutil

with open('sequences.fasta', 'rb') as f_in:
    with gzip.open('seq