Data sources

Every dataset seqtree ships, downloads, or reads in a benchmark. Provenance is stated as experimental (measured from a biological sample), derived (computed from experimental data), or synthetic (generated by a model). Never conflate the three.

Bundled with the wheel

python/seqtree/data/control_human_trb_aa.txt

what	250,000 human TRB junction amino-acid sequences, the null distribution P₀
provenance	experimental, then derived by uniform subsampling
origin	HuggingFace antigenomics/vdjdb-cdr3-frequencies, human TRB pooled clonotype table
format	one junction per line, productive only (20 standard amino acids)
regenerate	python bench/gen_control.py --species human --chain trb --size 250000 --seed 0

The upstream table is sorted by clonotype count, descending. The sample is a seeded reservoir sample over unique clonotypes, then shuffled, so any prefix bundled[:k] is itself a uniform sub-sample. Taking the first k rows of the upstream table instead yields the abundance head, which is 25.8× more self-similar and carries 3.1× the ball mass — it violates ass:indep in appendix/evalue.tex and corrupts every E-value. See CHANGELOG.md [0.3.0] Breaking.

Out-of-frame clonotypes (VDJtools _) and in-frame stops (*) are dropped. _ marks a collapsed run of untranslatable positions, so the residue count is already lost and no amino-acid repair is possible; and out-of-frame junctions escape thymic selection, making them a sample of P_gen, which lem:hierarchy proves is not P₀.

Substitution matrices — src/blosum62.inc, pam250.inc, pam100.inc, structural.inc

provenance	derived (published log-odds / interaction-strength matrices)
origin	BLOSUM62 (NCBI); EMBOSS EPAM250, EPAM100; Miyazawa–Jernigan interaction strengths, transcribed into MJ_CONTACT in bench/gen_matrices.py from the MJ rows of tcren-ms/src/tcren/data/MJ_Keskin_potentials.csv (Miyazawa & Jernigan, J Mol Biol 1996, as tabulated by Keskin et al., Protein Sci 1998)
regenerate	python bench/gen_matrices.py

Stored as non-negative penalties via the Gram transform pen[a][b] = s[a][a] + s[b][b] − 2·s[a][b], clamped at zero, so penalty(a, a) == 0 and the score defines a ball.

Known corruption in the MJ source. MJ_Keskin_potentials.csv has no A,N row: the lower-triangle slot where N-A belongs is mangled to V,1 (mirrored 1,V) for both the MJ and Keskin potentials. MJ_CONTACT carries the correct A-N = 0.15 from that row, so the constant will not diff cleanly against the CSV at that pair. The Keskin rows are unused here. Fix not yet upstreamed to tcren-ms.

Downloaded on demand

load_control(name) → ~/.cache/seqtree/control_*.sqtree

what	larger control repertoires, e.g. mouse_trb_aa (694,241 productive clonotypes)
provenance	experimental, then derived by uniform subsampling
origin	HuggingFace antigenomics/vdjdb-cdr3-frequencies
fetch	seqtree.load_control("mouse_trb_aa") (uses huggingface_hub, an optional dependency)

The cached reference index is content-addressed by the bundled asset or download source, requested size, alphabet and sampling seed. Superseded index files are pruned after rebuilding.

Benchmark inputs — not shipped, passed by path

These live in sibling repositories. seqtree does not depend on either; the benchmarks that read them take --prototypes / --vdjdb arguments and parse the files with the standard library.

VDJdb slim dump

what	TCR↔epitope pairs with v.end / j.start templated-flank annotations
provenance	experimental (curated from published studies)
origin	../mirpy/airr_benchmark/vdjdb/vdjdb-2025-12-29/vdjdb.slim.txt.gz
upstream	antigenomics/vdjdb-db — release 2025-12-29
covers	human TRA/TRB, mouse TRA/TRB, macaque TRB. Dotted column names (v.segm, antigen.epitope)
used by	bench/gap_rules_by_locus.py (criterion A)

Per-locus prototype sets

what	10,000 clonotypes per (species, locus): v_call, j_call, junction_aa
provenance	experimental where a real control repertoire exists, otherwise synthetic (OLGA)
origin	../mirpy/mir/resources/prototypes/{species}_{locus}.tsv
covers	human TRA TRB TRG TRD IGH IGK IGL; mouse TRA TRB
regenerate	python ../mirpy/mir/resources/prototypes/generate_prototypes.py (committed; never rebuilt at test time)
used by	bench/gap_rules_by_locus.py (criteria B and C)

Junctions, not IMGT CDR3: they include both anchors (C…F or C…W). Median lengths run from 12 (IGK) to 20 (IGH), with IGH reaching 50 — which is why a fixed gap-position set calibrated on length-14 TRB does not transfer.

IMGT gene library

what	germline V/D/J nucleotide sequences, 2,710 (species, allele) entries
provenance	experimental (IMGT reference directory)
origin	../mirpy/mir/resources/gene_library/imgt_gene_library.txt
caveat	V genes translate in frame 0; J genes do not (TRBJ2-7 is frame 1, TRBJ1-1 frame 2). The companion region_annotations.txt carries jcdr3_aa, which excludes the conserved F/W — it is CDR3 convention, not junction, and it disagrees with the translated library on some alleles (TRBV1*01, IGHV1-2*01).

Not currently read by any committed benchmark. Recorded because a germline-templated-flank prior was built against it and falsified: it recovers the structurally correct block position 0.4% of the time on TRA against 42.4% for a central prior.

UniProt reference proteomes (isalgo/pmhc_data)

what	whole-proteome FASTA: mouse (UP000000589, 54,857 records / 23,131,234 residues), human (UP000005640, 147,506 records / 69,578,135 residues), plus ecoli_K12_UP000000625 and fluA_PR8_UP000009255 as size steps
provenance	experimental (UniProt reference proteomes, mirrored)
origin	HuggingFace dataset isalgo/pmhc_data, path proteome/<name>.fasta.gz
regenerate	hf_hub_download("isalgo/pmhc_data", "proteome/human.fasta.gz", repo_type="dataset")
used by	bench/bench_text_index.py (large tier only, RUN_BENCHMARK=1)

Carries residues the amino-acid codec does not name — 36 ``U`` (selenocysteine) in human and 33 in mouse — plus 8,417 human and 4,263 mouse X. TextIndex keeps the unnamed ones as holes and reports the count, rather than refusing the build or silently treating them as wildcards; X/B/Z/* are real symbols that match only themselves.