Quick start

Installation

From PyPI

pip install vepyr

This installs the Python package and a vepyr executable — see Command line.

From bioconda

conda install -c conda-forge -c bioconda vepyr

The package is built for linux-64, osx-64 and osx-arm64 and ships the same vepyr executable. A new release reaches bioconda after its recipe update is merged, so it can trail PyPI by a few days — check the badge above for the current version.

From source (for development)

vepyr requires a Rust toolchain and Python 3.10+.

1. Install uv and Rust:

curl -LsSf https://astral.sh/uv/install.sh | sh
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

1. Clone and build:

git clone git@github.com:biodatageeks/vepyr.git
cd vepyr
RUSTFLAGS="-C target-cpu=native" uv sync --reinstall-package vepyr

1. Verify:

uv run python -c "import vepyr; print('build_cache' in vepyr.__all__)"
# True
uv run vepyr --version

Getting a cache

Annotation needs an Ensembl VEP cache in vepyr's optimized Parquet format. You have two options: download a prebuilt one (minutes, recommended) or build your own from an Ensembl VEP offline cache (hours of CPU).

Option A — download a prebuilt cache (recommended)

Release-116 GRCh38 caches for all three transcript sets are published on Hugging Face. Install the client once:

pip install -U "huggingface_hub[cli]"

Then pull the cache type you want:

mergedensemblrefseq

hf download biodatageeks/vepyr_116_GRCh38_merged \
  --repo-type dataset \
  --local-dir ~/vepyr_cache/116_GRCh38_merged

hf download biodatageeks/vepyr_116_GRCh38_ensembl \
  --repo-type dataset \
  --local-dir ~/vepyr_cache/116_GRCh38_ensembl

hf download biodatageeks/vepyr_116_GRCh38_refseq \
  --repo-type dataset \
  --local-dir ~/vepyr_cache/116_GRCh38_refseq

The download is resumable and the client verifies integrity, so there is no separate checksum step. Budget 31–36 G of disk per cache.

Confirm the cache reports the release you expect before annotating — this opens only the named contig's shards, so it is fast:

import os
import vepyr

cache = os.path.expanduser("~/vepyr_cache/116_GRCh38_merged")
print(vepyr.cache_contig_identity(cache, "chr22", expected_cache_version="116"))

For a second mirror, per-contig partial downloads, and the four prebuilt plugin caches, see Download Ensembl VEP and plugin caches.

Option B — build your own cache

Use this for release 115, for a cache type or assembly that is not mirrored, or when you want to convert a VEP cache you already hold.

Download and convert automatically

import vepyr

results = vepyr.build_cache(
    release=115,
    cache_dir="/data/vepyr_cache",
    cache_type="ensembl",
)
for path, rows in results:
    print(f"{path}: {rows:,} rows")

This downloads the Ensembl VEP 115 cache for homo_sapiens / GRCh38 and converts it to a partitioned Parquet cache.

Convert a local cache

If you already have the Ensembl VEP cache unpacked locally:

results = vepyr.build_cache(
    release=115,
    cache_dir="/data/vepyr_cache",
    cache_type="ensembl",
    local_cache="/data/ensembl_vep/homo_sapiens/115_GRCh38",
)

To rebuild a single raw entity without converting the full cache, pass entity (and optionally chroms):

results = vepyr.build_cache(
    release=116,
    cache_dir="/data/vepyr_cache",
    cache_type="merged",
    entity="motif",
    local_cache="/data/ensembl_vep/homo_sapiens_merged/116_GRCh38",
    overwrite=True,
)

A targeted build uses the same strict release/source validation as a full one and writes into the same <release>_<assembly>_<cache_type> directory, leaving the other entities untouched.

Options

Parameter	Default	Description
partitions	8	DataFusion partitions for parallel conversion
species	homo_sapiens	Species name
assembly	GRCh38	Genome assembly
cache_type	required	Ensembl VEP cache type: ensembl, merged, or refseq

Annotating variants

Annotation mode: Ensembl VEP --everything

vepyr always annotates the way Ensembl VEP does with --everything. It cannot be turned off, and there are no per-feature flags such as hgvs=, af= or check_existing=: every run produces the full --everything result. HGVS notation needs the reference sequence, so reference_fasta= (--fasta on the command line) is required.

In Ensembl VEP, --everything is a shortcut: it switches on a fixed set of other flags (@OPTION_SETS in Config.pm; the set is identical in releases 115 and 116). These are the flags it enables, with the CSQ fields each one adds:

Ensembl VEP flag	CSQ fields
--sift b	SIFT (prediction and score)
--polyphen b	PolyPhen (prediction and score)
--hgvs	HGVSc, HGVSp, HGVS_OFFSET
--symbol	SYMBOL, SYMBOL_SOURCE, HGNC_ID
--numbers	EXON, INTRON
--biotype	BIOTYPE
--canonical	CANONICAL
--mane	MANE, MANE_SELECT, MANE_PLUS_CLINICAL
--tsl	TSL
--appris	APPRIS
--ccds	CCDS
--protein	ENSP
--uniprot	SWISSPROT, TREMBL, UNIPARC, UNIPROT_ISOFORM
--domains	DOMAINS
--mirna	miRNA
--variant_class	VARIANT_CLASS
--gene_phenotype	GENE_PHENO
--regulatory	regulatory and motif consequences; MOTIF_NAME, MOTIF_POS, HIGH_INF_POS, MOTIF_SCORE_CHANGE, TRANSCRIPTION_FACTORS
--af	AF
--af_1kg	AF, AFR_AF, AMR_AF, EAS_AF, EUR_AF, SAS_AF
--af_gnomade	gnomADe_AF and the per-population gnomADe_*_AF
--af_gnomadg	gnomADg_AF and the per-population gnomADg_*_AF
--max_af	MAX_AF, MAX_AF_POPS
--pubmed	PUBMED

Some of those flags imply others in turn, through the same option sets:

• --hgvs implies --hgvsc and --hgvsp. Adding --hgvs to an --everything command line, as the Ensembl VEP reference runs in Testing vs Ensembl VEP do, changes nothing.
• --af, --af_1kg, --af_gnomade, --af_gnomadg, --max_af and --pubmed each imply --check_existing. It fills the base Existing_variation field and adds CLIN_SIG, SOMATIC and PHENO.

A field that two flags list (AF, or BIOTYPE, which --regulatory also lists) appears once. The field lists follow Ensembl VEP's FLAG_FIELDS in Constants.pm.

--everything does not set the HGVS formatting options. 3' shifting of HGVS indels is on by default in Ensembl VEP as well, and --shift_hgvs 0, --no_escape, --remove_hgvsp_version and --hgvsp_use_prediction change the notation. vepyr keeps these four as annotate() parameters (shift_hgvs=False, no_escape=, remove_hgvsp_version=, hgvsp_use_prediction=).

--everything also leaves some things out. Transcript selection (--pick and its variants, --gencode_basic, --exclude_predicted, …), frequency filtering (--check_frequency, --filter_common), --custom annotations and plugins are separate options, both in Ensembl VEP and in vepyr. Ensembl VEP also drops the frequency and PubMed flags when --everything is combined with --database; vepyr always reads a local cache, so that never applies.

The resulting CSQ layout has 80 fields for an ensembl cache, and more for refseq and merged caches, which add a transcript-source block. See CSQ output fields for the exact order.

Normalize the input VCF first

vepyr supports normalized VCFs only

vepyr annotates records as given and does not split multiallelic sites itself. Parity with Ensembl VEP is validated on inputs with one ALT allele per record, so split multiallelic records before annotating:

bcftools norm -m -both -Oz -o input.norm.vcf.gz input.vcf.gz
tabix -p vcf input.norm.vcf.gz

The index lets you use workers > 1. Pass input.norm.vcf.gz as vcf in the examples below. In Nextflow, the vcf_annotate_vepyr subworkflow runs this step for you; see Normalizing first.

Writing annotated VCF output

Write results directly to a VCF file with a CSQ INFO field:

import vepyr

out_path = vepyr.annotate(
    vcf="input.vcf.gz",
    cache_dir="/data/vepyr_cache/parquet/115_GRCh38_ensembl",
    reference_fasta="GRCh38.fa",
    output_vcf="annotated.vcf.gz",  # .vcf.gz for bgzf, .vcf for plain
)
print(f"Wrote annotated VCF to {out_path}")

For the supported scope the records are byte-identical to Ensembl VEP's own --everything --hgvs output — not merely equivalent — so the file is a drop-in replacement wherever a VEP VCF is expected. Only the provenance header lines (wall-clock time, cache paths, tool versions) differ, since they can never match. The e2e suite verifies this by hashing the record bodies of both files; see Checking byte-level agreement.

The input's own INFO keys and sample columns are written back unchanged, also when a key shares a name with a VEP field: an input AF= from gnomAD or bcftools +fill-tags stays as it was, and VEP's frequency is inside CSQ. An input that already carries CSQ has it replaced, as Ensembl VEP does: the old key and its header line are dropped and the new CSQ is appended last.

Annotating to a Polars LazyFrame

Omit output_vcf and annotate() returns a Polars LazyFrame instead:

import vepyr

lf = vepyr.annotate(
    vcf="input.vcf.gz",
    cache_dir="/data/vepyr_cache/parquet/115_GRCh38_ensembl",
    reference_fasta="GRCh38.fa",
)

df = lf.collect()
print(df.select("chrom", "start", "ref", "alt", "most_severe_consequence").head())

Annotation always runs Ensembl VEP --everything, so reference_fasta is required on both paths and there are no per-feature flags to switch on; see Annotation mode for what it enables.

workers controls how many within-contig annotation pipelines run concurrently, on both the LazyFrame and the output_vcf path. It requires a tabix-indexed (bgzip + .tbi or .csi) input VCF. Output is identical to workers=1, row order included.

df = vepyr.annotate(
    "input.vcf.gz",
    "/data/vepyr_cache/parquet/115_GRCh38_ensembl",
    reference_fasta="GRCh38.fa",
    workers=4,
).collect()

Filtering the LazyFrame on chrom, start or end is pushed into the engine before annotation; see Polars DataFrames.

From the command line

The quickest path from a VCF to an annotated VCF needs no Python at all:

vepyr annotate \
    -i input.vcf.gz \
    -o annotated.vcf.gz \
    --dir_cache ~/vepyr_cache/116_GRCh38_merged \
    --fasta GRCh38.fa \
    --fork 8

The flags follow Ensembl VEP's own spelling. The command covers the VCF-in / VCF-out path only; the LazyFrame path above is Python-side. See Command line for the full flag set, the plugin flags and the exit codes.