Skip to content

SYSTEM Cited by 1 source

bcftools / htslib

bcftools is the standard command-line toolkit for reading, filtering, and manipulating variant call data (VCF/BCF); it is built on htslib, the C library underpinning the SAM/BAM/CRAM/VCF file formats. The architecturally significant capability for this wiki is htslib's support for tabix-indexed random access over HTTP byte ranges: given a tabix index and a genomic region, htslib issues Range: bytes=X-Y requests to read only the bytes covering that region instead of downloading the whole file.

Why it appears here

In sBeacon, the performQuery Lambda passes an S3 URI directly to bcftools as a subprocess argument. htslib then issues HTTP byte-range requests (~1 KB per query) against the S3 REST API of whichever organization owns that bucket, reading only the tabix-indexed region of interest. (Source: sources/2026-09-18-aws-how-csiro-built-scalable-cost-optimized-genomic-variant-querying-on-aws)

This is what makes sBeacon's reference-not-copy design work:

  • The large raw VCF payload is never copied or ingested — only indexed for random access.
  • The raw genomic sequence bytes never pass through Lambda memory as returnable data; only the aggregate result (exists / count / variant record) is returned.
  • Because the read hits the owning org's bucket via byte-range, sBeacon's decentralization is achieved at the storage layer, keeping data movement and egress minimal (read only what you need).

The bcftools subprocess runs and exits within the performQuery Lambda's 10 s lifetime, leaving no persistent state.

Seen in

Last updated · 766 distilled / 2,225 read