API reference

vepyr.build_cache(release, cache_dir, *, cache_type, entity=None, chroms=None, species='homo_sapiens', assembly=_DEFAULT_ASSEMBLY, partitions=8, cache_format='parquet', local_cache=None, download_retries=10, show_progress=True, on_progress=None, overwrite=False)

Download an Ensembl VEP cache and convert it to an optimized cache.

Parameters:

Name	Type	Description	Default
release	int	Ensembl release number (e.g. 115).	required
cache_dir	str	Root directory for cache data and Parquet output.	required
cache_type	str	Required Ensembl VEP cache type: "ensembl", "merged", or "refseq".	required
entity	str or None	Build a single raw entity instead of the whole cache, leaving the rest of the output directory untouched. One of "variation", "transcript", "exon", "translation", "regulatory", or "motif"; the raw translation entity produces the translation_core and translation_sift Parquet datasets. None (default) builds every entity.	None
chroms	list[str] or None	Restrict the build to specific contigs (e.g. ["chrX"]). None (default) builds every contig.	None
species	str	Species name (default: "homo_sapiens").	'homo_sapiens'
assembly	str	Genome assembly (default: "GRCh38").	_DEFAULT_ASSEMBLY
partitions	int	Number of DataFusion partitions for parallelism (default: 8).	8
cache_format	str	Cache format to build. Only "parquet" is supported (default).	'parquet'
local_cache	str or None	Path to an already-unpacked Ensembl VEP cache directory (the one containing info.txt). When provided, downloading and extraction are skipped entirely.	None
download_retries	int	Maximum number of resume-retries for the cache download (default: 10). Each retry resumes from the last byte received.	10
show_progress	bool	Show the download progress bar and the tqdm bars during conversion (default: True). .. note:: The partitioned-Parquet build path does not currently emit per-batch progress events, so no bars appear during the cache build regardless of show_progress / on_progress.	True
on_progress	callable or None	Custom progress callback with signature (entity, format, batch_rows, total_rows, total_expected). Overrides the default tqdm bars when provided. See the note on show_progress — the Parquet build path does not invoke it.	None
overwrite	bool	Rebuild existing cache outputs instead of skipping them.	False

Returns:

Type	Description
list[tuple[str, int]]	List of (parquet_file_path, row_count) for each written file.

vepyr.build_plugin_cache(plugin, version, *, source_path, cache_dir, plugin_cache_root, chroms=None, plugins_repo=None, overwrite=False, verify_source=True)

Build a per-chromosome plugin cache.

plugin/version select plugins/<plugin>/<plugin>.source.toml from the public vepyr-plugins repo at that git tag (or plugins_repo for offline). Tiering is inherited from the variation cache at cache_dir. Returns per-chrom (chrom, rows, warm, cold) tuples.

source_path points each [[source]] at a real file, since the paths a manifest ships are placeholders. A single-source manifest takes a plain path; a manifest that declares several sources (each with a part) takes a {part: path} mapping, and every part must be mapped::

build_plugin_cache(
    "cadd", "v1.0",
    source_path={"snv": ".../whole_genome_SNVs.tsv.gz",
                 "indel": ".../gnomad.genomes.r4.0.indel.tsv.gz"},
    ...
)

The sources are registered as plugin_<name>_src_<part> and combined by the manifest's own ingest_sql -- there is no need to concatenate them first.

verify_source guards against building from the wrong bytes. Before the first chromosome is ingested, each resolved source file is MD5-hashed once (streaming, bounded memory) and compared with the manifest's md5. True / "strict" (default) raises on a mismatch, naming the part, both digests and the upstream url; "warn" logs it and keeps building -- for a build input that is a derived artifact of the upstream file (AlphaMissense's BGZF re-compression; its plugin README documents the preprocessing); False / "skip" never hashes -- use it for a chromosome slice cut with tabix, whose digest can never match the whole file. A manifest that declares no md5 is never hashed. The digest actually verified, with the file's size and mtime, is recorded under sources in the emitted manifest.json; an incremental chroms=[...] build against an unchanged file trusts that record instead of re-hashing.

The cache manifest records the source as <version>@<commit SHA>. This keeps mutable refs auditable and prevents an incremental build from mixing chromosomes produced from different manifest revisions. Builds for the same plugin cache are serialized across processes so the revision check and shard update observe one cache revision. This coordination applies to calls through this API on filesystems that honor advisory locks; external writers are outside the lock.

vepyr.download_cache(release, cache_dir, *, cache_type, assembly='GRCh38', chroms=None, revision=None, show_progress=True)

Download a published vepyr cache from Hugging Face.

Parameters:

Name	Type	Description	Default
release	int	Ensembl release, e.g. 116.	required
cache_dir	str	Root directory; the cache is written to <cache_dir>/<release>_<assembly>_<cache_type>, the layout :func:build_cache produces. ~ is expanded.	required
cache_type	('ensembl', 'merged', 'refseq')	Cache type.	"ensembl"
assembly	str	Genome assembly (default: "GRCh38").	'GRCh38'
chroms	list of str or None	Contigs to download, e.g. ["22"] or ["chr22", "chrX"]; a name without chr also matches the prefixed one. None downloads every contig. The per-entity manifests and the root sidecars are always included.	None
revision	str or None	Dataset revision. None uses the revision pinned by this vepyr release; any branch, tag or commit (e.g. "main") is passed on.	None
show_progress	bool	Show download progress bars (default: True).	True

Returns:

Type	Description
str	The cache directory, ready for annotate(cache_dir=...).
Re-running resumes and skips files already present, and ``chroms`` can
add contigs to an earlier download. A directory holds one dataset
revision, recorded in ``.cache/vepyr/download.json``: downloading another
revision into it is refused, and so is downloading into a directory filled
some other way (e.g. by :func:`build_cache`) unless ``hf download`` wrote
it at the same revision.

vepyr.download_plugin_cache(plugin, release, plugin_cache_root, *, assembly='GRCh38', chroms=None, revision=None, show_progress=True)

Download a published vepyr plugin cache from Hugging Face.

Parameters:

Name	Type	Description	Default
plugin	str	Plugin name, e.g. "clinvar". dbNSFP is supported by :func:build_plugin_cache but not published.	required
release	int	Ensembl release, e.g. 116.	required
plugin_cache_root	str	Root of the plugin cache tree; the cache is written to <plugin_cache_root>/plugin/<plugin>. ~ is expanded.	required
assembly	str	Genome assembly (default: "GRCh38").	'GRCh38'
chroms	list of str or None	Contigs to download, as for :func:download_cache. The manifest is always included.	None
revision	str or None	Dataset revision; None uses the pinned one.	None
show_progress	bool	Show download progress bars (default: True).	True

Returns:

Type	Description
str	The plugin cache directory. Pass plugin_cache_root and plugins=[plugin] to :func:annotate.
Re-running resumes; the directory keeps one dataset revision, as for
func:`download_cache`.

vepyr.annotate(vcf, cache_dir, *, reference_fasta, shift_hgvs=None, no_escape=False, remove_hgvsp_version=False, hgvsp_use_prediction=False, expected_cache_version=None, distance=None, gencode_basic=False, gencode_primary=False, all_refseq=False, exclude_predicted=False, pick=False, pick_allele=False, per_gene=False, pick_allele_gene=False, flag_pick=False, flag_pick_allele=False, flag_pick_allele_gene=False, pick_order=None, buffer_size=5000, failed=0, workers=1, skip_csq=True, fields=None, plugin_cache_root=None, plugins=None, allow_non_variant=False, info_fields=None, format_fields=None, output_vcf=None, preserve_record_layout=None, show_progress=True, compression=None, on_batch_written=None)

Annotate variants from a VCF file with VEP consequences.

Reads the VCF, runs annotate_vep() against the partitioned parquet cache produced by :func:build_cache, and returns a polars LazyFrame.

The engine auto-discovers context tables (transcript, exon, translation, regulatory, motif) from cache_dir subdirectories.

Parameters:

Name	Type	Description	Default
vcf	str	Path to the input VCF file.	required
cache_dir	str	Path to the parquet cache directory produced by :func:build_cache, e.g. "/data/vep/wgs/parquet/115_GRCh38_ensembl". Cache directories are local filesystem paths.	required
reference_fasta	str	Path to the reference FASTA. Required: annotation always runs Ensembl VEP --everything, and HGVS needs it. --everything implies --sift b --polyphen b --ccds --hgvs --symbol --numbers --domains --regulatory --canonical --protein --biotype --af --af_1kg --af_gnomade --af_gnomadg --max_af --pubmed --uniprot --mane --tsl --appris --variant_class --gene_phenotype --mirna, and through those --hgvsc, --hgvsp and --check_existing.	required
shift_hgvs	bool or None	3' shift HGVS notation. None = VEP's default (on).	None
no_escape	bool	Don't URI-escape HGVS strings.	False
remove_hgvsp_version	bool	Remove version from HGVSp transcript ID.	False
hgvsp_use_prediction	bool	Use predicted rather than observed protein sequence.	False
distance	int or tuple[int, int] or None	Upstream/downstream distance for transcript overlap. Single int = both directions; tuple = (upstream, downstream).	None
gencode_basic	bool	Restrict to transcripts in the GENCODE basic set. Mutually exclusive with gencode_primary.	False
gencode_primary	bool	Restrict to transcripts in the GENCODE primary set (GRCh38 only). Mutually exclusive with gencode_basic.	False
all_refseq	bool	Keep all RefSeq transcripts including CCDS/EST-style rows.	False
exclude_predicted	bool	Exclude predicted RefSeq transcripts (XM_ / XR_ prefixes).	False
pick	bool	Emit one selected consequence per variant, matching VEP --pick.	False
pick_allele	bool	Emit one selected consequence per allele, matching VEP --pick_allele.	False
per_gene	bool	Emit one selected consequence per gene while retaining non-transcript rows, matching VEP --per_gene.	False
pick_allele_gene	bool	Emit one selected consequence per allele and gene, matching VEP --pick_allele_gene.	False
flag_pick	bool	Retain all consequences and add PICK=1 to one selected entry per variant, matching VEP --flag_pick.	False
flag_pick_allele	bool	Retain all consequences and add PICK=1 to one selected entry per allele, matching VEP --flag_pick_allele.	False
flag_pick_allele_gene	bool	Add a standalone PICK=1 CSQ field for the selected transcript per allele and gene, matching VEP --flag_pick_allele_gene.	False
pick_order	str or None	Comma-separated VEP pick ranking order, e.g. "biotype,rank,mane_select,tsl,canonical,appris,ccds,length".	None
buffer_size	int	Number of input variants per VEP-style annotation buffer. Defaults to Ensembl VEP's --buffer_size default of 5000.	5000
failed	int	Maximum allowed failed flag value from cache (default: 0).	0
expected_cache_version	str or None	Optional assertion against the cache version embedded in each requested chromosome's Parquet metadata. It cannot supply missing cache identity.	None
workers	int	Number of within-contig annotation pipelines (default: 1) on both output paths. Values greater than 1 require a tabix-indexed (bgzip + .tbi or .csi) input VCF. Results are identical to workers=1 row for row and in the same order. On the LazyFrame path the engine queues at most workers + lookahead runs of output ahead of the consumer, capped at VEP_STREAM_BUFFER_MB MiB of Arrow data (default max(1024, 512 * workers), so 4096 at 8 workers; a positive value replaces the default). The cap is a ceiling, not an allocation: the queue only fills while the first run in order lags. VEP_STREAM_LOOKAHEAD_RUNS (default workers) sets the lookahead.	1
skip_csq	bool	Exclude the raw CSQ column from the output (default: True). When True, only the parsed annotation columns are returned.	True
fields	'core'	Ordered base CSQ fields to emit, mirroring VEP --fields. "core" selects VEP's eleven VCF-side default output fields. A list or tuple preserves its supplied order. Plugin fields are always appended after the selected base block. On the DataFrame path, unselected annotation columns are omitted. None (default) keeps the full layout.	"core"
plugin_cache_root	str or None	Root of a plugin cache tree built by :func:build_plugin_cache, e.g. "/data/plugin_cache" holding plugin/<name>/ directories. Every plugin found under the root is applied — see plugins to narrow that — and its CSQ fields are added to the header and to each transcript. A plugin's version is fixed when its cache is built, not chosen here. None (default) applies no plugins and is byte-identical to a plugin-free run. Plugin values are appended to the CSQ field. With fields set, a LazyFrame also exposes each plugin field as a named list column, including when skip_csq=True.	None
plugins	list or tuple of str, or None	Restrict annotation to these plugin names, a subset of the directories under <plugin_cache_root>/plugin/. None (default) applies every plugin found there in alphabetical order. A supplied sequence is also the emitted CSQ block order; an empty sequence applies none. Requires plugin_cache_root. Duplicate or unknown names are errors.	None
info_fields	list of str	Input INFO fields the LazyFrame carries: None (default) for all, a list to select, [] for none -- the meaning polars_bio.scan_vcf gives the same argument. A field whose id is also an annotation column (AF) is carried as INFO_<id>; the annotation column keeps the bare name. A query only reads the fields it names. Ignored with output_vcf, which always keeps every field.	None
format_fields	list of str	Input FORMAT fields, same convention. Single-sample inputs get one column per field; multi-sample inputs a nested genotypes struct.	None
output_vcf	str or None	Path to write annotated VCF output. When set, annotation results are written directly to a VCF file and the output path is returned. When None (default), returns a polars LazyFrame. Compression is auto-detected from the file extension: .vcf for plain text, .vcf.gz or .vcf.bgz for block-gzipped (bgzf). Override with the compression parameter.	None
allow_non_variant	bool	Keep records that carry no alternate allele -- ALT=. (default: False). Ensembl VEP drops such a record entirely unless --allow_non_variant is given, and vepyr does the same: by default it is absent from the output, exactly as VEP leaves it out. With this on, the record is emitted with its original ALT and no consequence annotation -- no CSQ key on the VCF path, a null CSQ on the LazyFrame path. Note VEP tests only the first ALT, so ALT=.,C is non-variant while ALT=C,. is an ordinary record.	False
preserve_record_layout	bool	Write each record's INFO fields in the order the input wrote them, and its own FORMAT keys (default: on). On the LazyFrame it adds two string columns, _vcf_info_keys and _vcf_format_keys, which polars_bio.sink_vcf uses to write each record exactly as output_vcf would; they have to stay in the frame for that. Pass False to leave them out. The default is skipped for an input that cannot carry the layout (BCF, or a file declaring a field with either name); an explicit True raises for such an input. Both are per record and neither survives the typed columns, so turning this off reorders INFO to schema order and drops any FORMAT key whose value is missing in every sample. Ensembl VEP keeps both by copying the input line and only appending to INFO, so byte agreement with it needs this on.	None
show_progress	bool	Show a progress bar while annotating (default: True). On the VCF output path it is a determinate bar over the pre-counted input. On the LazyFrame path it counts the variants the engine yields on each collect(), with a total only when a limit (.head(n)) reaches the source unfiltered.	True
compression	str or None	VCF output compression. "bgzf" (block-gzip, tabix-compatible), "gzip", "plain", or None (auto-detect from extension). Only used when output_vcf is set.	None
on_batch_written	callable or None	Callback invoked after each batch is written to VCF, with signature (batch_rows: int, total_rows: int, total_input: int). total_rows is the cumulative number of VCF records written so far. total_input is the total number of input variants when known. Useful for driving tqdm progress bars in notebooks. Only used when output_vcf is set.	None

Notes

Filtering the returned LazyFrame on chrom, start or end restricts the input before annotation (region pushdown): unselected contigs are skipped and indexed inputs are read by seek. Results are identical to filtering after collect(). A RuntimeWarning is raised when the input has no .tbi/.csi index. See the "Region filters" section of the Polars DataFrames docs page for the recognised predicate shapes.

Returns:

Type

Description

LazyFrame or str

When output_vcf is None: annotated variants as a polars LazyFrame with typed annotation columns plus original VCF fields. Every column holds its --everything value. A select() only decides how much work that takes: the engine skips the HGVS lookup, the co-located lookup and the everything-only extras (SIFT, PolyPhen, motifs, ...) when no selected column needs them, which leaves the selected values unchanged. fields cannot be combined with a narrowing select(). When output_vcf is set: the output VCF file path.

Examples:

>>> import vepyr
>>> lf = vepyr.annotate(
...     "input.vcf",
...     "/data/vep/parquet/115_GRCh38_ensembl",
...     reference_fasta="/ref/GRCh38.fa",
... )
>>> lf.collect()

>>> # Write annotated VCF directly
>>> path = vepyr.annotate(
...     "input.vcf",
...     "/data/vep/parquet/115_GRCh38_ensembl",
...     reference_fasta="/ref/GRCh38.fa",
...     output_vcf="annotated.vcf",
... )