The portable repertoire signature

One AIRR repertoire in, one fixed, named, positional feature vector out — computable by anyone who pip install mirpy-lib, on their own samples, and directly comparable with yours. That is the whole design goal: a matrix you can hand a collaborator that drops into PCA, logistic regression, random forest, boosting or an MLP with no scaler of their own.

Quickstart — one command

No Python needed. This is the command to send a collaborator:

pip install mirpy-lib

mir signature --preset classify cohort/*.tsv.gz -o sig.parquet

Files sharing a sample id (the name up to the first dot) are joined into one multi-locus sample, so a donor sequenced on TRA and TRB is one signature with both loci filled rather than two half-empty rows. AIRR Rearrangement, native vdjtools, Parquet and the usual third-party exports are auto-detected.

mir signature --preset classify --describe        # the columns, reading no input
mir signature --preset classify --threads 0 cohort/*.tsv.gz -o sig.parquet   # every core
mir presets                                       # the named feature sets, ranked
mir presets classify                              # one in full: what, how, when

Three presets are marked recommended: compact (the smallest vector that still describes a repertoire, usable at n = 50), classify (general-purpose, the usual random-forest / boosting input), and transfer (for a model that must work on another lab’s samples). Unlike vdjtools signature, which serves the vsig half only, every preset resolves here in full — mirpy is where the two halves meet.

Warning

CDR3 vs junction. The reader prefers AIRR junction_aa (conserved anchors included) and falls back to IMGT cdr3_aa (anchors excluded). A file carrying only cdr3_aa is two residues short everywhere, which shifts the length, k-mer and Pgen features. Check your headers before you trust a matrix.

Two more things worth knowing: --standardize reference (the default) is what makes your vector comparable with anyone else’s, and you should not PCA-project the result — plain scaling beat projection at every rank tested.

The Python API

from mir.signature import signature, signature_cohort

v = signature({"TRB": df})                 # {column: value}, standardised, layout order
F = signature_cohort(samples)              # one row per sample, positional
F.write_parquet("cohort.parquet")

Two halves, one contract

The signature is the concatenation of two vectors that answer different questions about the same sample, joined on sample_id and namespaced so they never collide:

	vsig — statistics (vdjtools.signature)	rsig — geometry (mir.signature)
basis	the clone-size vector and the germline vocabulary	the prototype-sum measure \(\Phi(S) = \sum_\sigma w_\sigma z_\sigma\)
blocks	mask qc depth div clon len iso shm pair pgen aa pchem	depth div band contrast phiv phij phic
each column is	a defined statistic of the clone-size vector	a linear functional, a norm, or a mixture coefficient of \(\Phi\)

depth and div appear on both sides deliberately. They are different objects with the same name family — count-native Hill numbers at a frozen coverage level on one side, embedding-native effective sample size and Rao dispersion on the other — and which one carries a given phenotype is itself a result.

Why every column is transformed before you see it

A learner cannot be handed a log-scaled read count, an isotype fraction and a principal component in one matrix and be expected to weight them sensibly. Each feature therefore carries a variance-stabilising transform chosen from its support, not from taste, and each of those choices is denominator-aware — the alternative silently lies about shallow samples:

transform	where, and why that one
log10 / log1p	counts and norms, whose spread scales with their magnitude
logit	a proportion, Haldane–Anscombe corrected, so 0/3 and 0/500 are different numbers
arcsine	Anscombe’s variance-stabiliser for a binomial share; defined at exactly zero
clr	a composition, over the whole composition before any coordinate is selected, shipping k−1 parts because all k are linearly dependent and would put a guaranteed zero eigenvalue in any PCA

On top of that every column is rescaled against a frozen reference (median and \(1.4826\cdot\mathrm{MAD}\), clipped), which is what makes two people’s matrices comparable rather than each being internally consistent and mutually meaningless.

Geometry is not transformed, and that is not an oversight

The two halves of the signature end up with almost opposite transform tables, because they hold almost opposite kinds of object:

	vsig — statistics	rsig — geometry
dominant transform	arcsine ×20, clr ×7, logit ×7, log10 ×6	none ×116
support	\([0,1]\) or \([0,\infty)\), bounded, discrete	\(\mathbb{R}\), signed, continuous
mean–variance coupling	yes — binomial or Poisson; variance is a function of the mean	no — \(\operatorname{Var} \approx \sigma^2 / n_{\text{eff}}\), independent of the value

A coordinate of \(\Phi = \sum_\sigma w_\sigma z_\sigma\) is a linear functional of a weighted mean of fixed embedding vectors. It is signed and roughly symmetric, there is no boundary to compress against, and its variance does not depend on its own value — exactly the condition under which a variance-stabilising transform buys nothing. Applying one would not merely be useless: log, logit and arcsine all require a non-negative or \([0,1]\) domain, and these coordinates go negative. What they need instead is location–scale rescaling against the frozen reference, which is what they get.

The exceptions prove the rule. rsig transforms exactly where the quantity stops being a coordinate: the block norms (\(\lVert\Phi\rVert\), Rao dispersion) are non-negative, right-skewed magnitudes on \([0,\infty)\), so they take log1p; band is a genuine closed composition, so it takes clr.

One block breaks the pattern deliberately

contrast — \(\Psi = \mathrm{mass}\cdot(\Phi - \mathrm{naive})\) — is flagged magnitude=True: it is divided by one frozen scalar RMS for the whole block, and is never centred. Per-column z-scoring would force every coordinate to unit variance, which makes a sample sitting near zero — an immune desert, a repertoire that has barely moved from naive — look identical to a typical one. How far a repertoire is from naive is what that block exists to carry, so rescaling it away would delete precisely the feature. This is the opposite policy from every other column in the matrix, and it is a property of the block, not of the sample.

How many components should you keep

Not “enough for 90% of the variance”. In a repertoire matrix the leading variance is sequencing depth, batch and V-gene usage, so a variance-ranked criterion ranks nuisance first.

Measured on the emitted signature matrix — 14,553 samples × 1,369 columns across 182 studies, robust median/MAD scaling with clipping (benchmark_signature_dimension.py):

criterion	components	what it actually measures
90% cumulative variance	394	how much of this corpus you reproduce
Horn parallel analysis	241	how many exceed a column-permuted null
participation ratio (effective rank)	144	how spread the eigenvalue mass is
per-component \(|r| \ge 0.95\) across a study-disjoint refit	1	which individual axes are identified at all
per-component \(|r| \ge 0.90\)	1

The gap between 394 and 1 is the finding, not a contradiction. Split-half correlation per component was 0.949 for PC1, 0.614 for PC2 and 0.15–0.32 from PC3 onward, while eigenvalues 2–12 sit at 73, 56, 51, 48, 44, 42, 38, 34, 32, 31, 30 — near-degenerate. Components of nearly equal eigenvalue swap order between two refits, so a per-component correlation punishes a labelling artifact rather than a stability failure. That is why the same script also reports the rotation-invariant subspace overlap \(\lVert V_a V_b^\top\rVert_F^2 / k\): the subspace can be stable where the axes are not.

So what number do you actually use? Ask the only criterion that knows what the components are for. Mean AUC over the four largest tasks (2,199–8,016 samples, 21–70 studies), study-disjoint folds, rotation refit inside every fold:

components	8	16	64	256	512	all 1,369
mean AUC	0.575	0.581	0.589	0.591	0.584	0.555

The curve is flat from 16 to 256 and then falls off a cliff: the full 1,369-column matrix scores worse than 16 components. That is the curse of dimensionality, located. Per task the effect is large — l3_covid reads 0.727 at \(k = 64\) against 0.582 on all columns; l1_infection 0.659 against 0.577.

The recommendation: 16–64 components. 16 buys 98% of the achievable AUC at a quarter of the width; 64 is the plateau; beyond 256 you are paying for noise. Keep the full matrix only when the learner is regularised for it (L1, gradient boosting) or when you are hunting a rare, sparse signal that a rotation would average away.

Practical rules that follow:

• Never interpret an individual PC beyond the first as though it were a named feature. It is a coordinate of the corpus that fitted it.

• Select rank by out-of-study reproducibility, not by explained variance — and prefer the subspace-overlap criterion to the per-component one, which fails on degeneracy alone.

• Refit the rotation inside every cross-validation fold. Fitting it once on the whole task and cross-validating only the classifier lets the components see the test studies’ covariance.

• Report a permutation null for anything chosen by looking at the labels. A maximum over 64 components reached AUC 0.84 by chance on a 26-vs-7 contrast (p = 0.20).

• For a rare, discriminative signal, do not project at all — keep sparse columns and an L1 model. The SVD optimises for variance, and a motif carried by a handful of donors has none.

This is also why no corpus-fitted rotation ships in the artifact: the phiv / phij / phic bases come from the prototype cloud, which involves zero samples and so has no corpus to be unstable with respect to.

Holes are never zeros

A locus that was not sequenced, a compartment with too few clonotypes to be a compartment, a statistic the sample is too shallow to estimate — each yields nan and a mask: column. A model that reads “absent” as “zero” reads an unsequenced chain as a biological finding. Most learners take nan natively; those that do not should impute and keep the mask.

The signature filters for you — do not pre-filter

signature() sanitises before it embeds, and sanitise=True is the default. Leave it there. The reason is specific to the geometry: a stop codon does not raise in the distance code. * is in seqtree’s alphabet, so an unfiltered frame used to return a finite, meaningless distance and contaminate Φ silently — which is strictly worse than crashing.

Since 3.12.0 embed() refuses it instead:

junction	default	allow_nonstandard=True
null	raises	raises
outside [ACDEFGHIKLMNPQRSTVWY*_]	raises — a corrupt table	raises
_ (out-of-frame marker)	raises — crashes gapblock	raises
* (stop codon)	raises	embedded

allow_nonstandard covers stop codons and nothing else. A guard against a crash, and a guard against a damaged file, cannot be switched off; only the guard against a silently-wrong number has an opt-out, and taking it has to be written down. Measured over 6,047,716 rows of real clinical AIRR: zero corrupt characters and zero _, so the strict default costs nothing on well-formed data.

Neither predicate filters on length — a two-residue junction and a sixty-residue one both pass. The question asked is only whether the string is a plain amino-acid string.

What pre-filtering actually changes. Exactly one column per locus, vsig:qc:<locus>:nonstd_aa_frac, because sanitise reports the weight fraction it dropped and a pre-filtered frame has nothing left to drop. Measured on 1,168 blood samples from a clinical AIRR cohort at tier="standard": of 688 columns, 7 move and 681 are bit-identical — including all 528 rsig geometry columns, which do not move because rsig is handed sanitised frames either way. Under compact, transfer or classify — every recommended preset — zero columns move, because they drop the qc block. If you want the column honest on a pre-filtered corpus, pass signature(..., prefiltered=True) and it reports a hole rather than a floor.

Tiers

core ⊂ standard ⊂ full, as exact index subsets of one frozen layout — so a narrower tier is a slice of a wider one and never a differently-computed number.

from mir.signature import columns, describe

len(columns("core")), len(columns("standard")), len(columns("full"))
describe("standard")     # column, sig, block, locus, feature, tier, transform, flags

What is fitted on data, and what is not

Two artifacts ship, and the split is the design.

	the geometry artifact	the scale artifact
holds	slot rotations, prototype-cloud location and scale, the naive reference	per-column location and scale, the measured cstar and pgen_q05
fitted on	nothing — bundled resources only	a reference corpus draw
re-fit risk	none; the rebuild is bit-identical	low; a median and a MAD are identified at any n

Fitting a scale and fitting a basis are different statistical problems, and only one of them is safe at the sample sizes anyone actually has. A rotation over \(p = 256\) coordinates per slot is not column-identified at a few thousand samples — measured split-half column agreement of a fitted junction basis is 0.23 — whereas a per-column median and MAD converge as \(1/\sqrt{n}\). So the rotation is taken from the prototype cloud instead: bundled receptors embedded against bundled receptors, zero samples, nothing to re-fit and nothing of any corpus in it.

Batch is the thing to check first

A frozen reference removes the scaling difference between two cohorts. It does not remove a batch effect, and nothing in this vector should be read as if it did — sequencing protocol, depth and sample handling all move real columns. Use the depth: columns as covariates, and check a batch label before believing a between-cohort contrast.

How strong is that warning? On a clonal-density read-out, the flagged fraction fell from 19.4% to 0.9% when the background was drawn from within the same study instead of across studies. Nearly all of it was batch.

Regenerating these numbers

Every measured table on this page comes from a script in the companion 2026-mirpy-analysis repo, so it can be re-measured rather than believed:

script	what it re-measures
benchmark_signature_dimension.py	variance / Horn / effective rank / split-half rank criteria, both scaling arms
benchmark_signature_scale_convergence.py	how many samples a frozen median and MAD need
benchmark_signature_rotation.py	prototype-cloud rotation vs a corpus-fitted one
benchmark_density_ankspond.py	the within-study vs cross-study background comparison

Measurements on this page were last taken 2026-08-13.

Feature presets — pick by intent, not by column

The signature is over 1,400 columns. Almost nobody wants all of them, and which subset is right depends on the question — a model that must run on another lab’s samples wants different columns from one scoring samples inside a single study. vdjtools.signature.presets names those choices, documents each, and ranks it:

recommended

Use this unless you have a reason not to.

specific

Correct for a stated purpose and wrong outside it.

avoid

A control, a baseline, or a measured dead end. Named so that choosing it is deliberate.

preset	rank	columns	what it is
compact	recommended	152	The smallest vector that still describes a repertoire. Start here.
transfer	recommended	550	For models that must work on another lab’s samples. Drops the columns whose level moves most between studies.
classify	recommended	615	The general-purpose set. Best measured task performance when train and test come from comparable cohorts.
statistics	specific	101	Classical repertoire statistics only. Needs no embedding, so vdjtools alone suffices.
bcell	specific	286	B-cell receptor work: the immunoglobulin loci with somatic hypermutation and isotype.
geometry	specific	514	Embedding coordinates only — no count statistics at all.
full	specific	1403	Every contract column. For feature selection, not for fitting.
nuisance	avoid	73	Sequencing protocol only. A control, not a feature set.

Every preset resolves to a column list from the frozen layout alone — block names, loci, tier. No corpus, no fitted artifact and no private data is involved, so two people selecting the same preset get the same columns in the same order.

mir presets                      # the table above
mir presets transfer             # one preset in full: features, how, use cases, caveats
mir signature *.tsv --preset transfer --describe    # the exact columns it selects

from vdjtools.signature import presets

presets.get("transfer").rank        # 'recommended'
cols = presets.columns("compact")   # a concrete, ordered column list
presets.table()                     # the whole registry as a DataFrame

Where the rankings come from

A benchmark over a public multi-study AIRR corpus — several hundred study groups, tens of thousands of samples — scored with study-disjoint folds: fit on some studies, predict on studies the fit never saw. Under that split a column that merely encodes sequencing protocol scores at chance, which is the point. Three findings shaped the presets:

• A nuisance floor of depth + presence masks + call quality is a surprisingly strong predictor on many contrasts. Any feature set worth using has to beat its own floor, which is why nuisance ships as a named control rather than being hidden.

• Projection did not help. Plain robust or asinh scaling beat PCA at every rank tested, so no preset projects by default and full is documented as a feature-selection tool rather than a model input.

• The two halves have opposite nuisance profiles. The embedding geometry carries several times less study-to-study variance than the count statistics and the most donor-to-donor variance, and is nearly unaffected by whether a sample is blood or tissue — but wins fewer supervised tasks outright. Hence transfer and geometry for robustness, classify and statistics for raw accuracy.

Anyone with a comparable SRA/AIRR corpus can reproduce this; none of it depends on a private dataset.

API

mir.signature — the geometry half of the portable repertoire signature.

The column contract lives in vdjtools.signature, which mirpy already depends on, so there is one implementation of the layout, the transforms and the frozen reference rescaling rather than two that can drift apart. This package supplies the rsig blocks: features of the prototype-sum measure Φ(S) = Σ_σ w_σ z_σ, each a linear functional, a norm, or a mixture coefficient of it.

mir.signature.columns(tier='standard', sig=None, blocks_=None)

The column names of a signature, in emitted order.

Parameters:

• tier (str) – "core", "standard" or "full". Tiers are nested and each is an exact index subset of "full".

• sig (str | None) – "vsig", "rsig", or None for both concatenated.

• blocks – Restrict to these block names (both signatures unless sig is given).

• blocks_ (tuple[str, ...] | None)

Returns:

Column names as <sig>:<block>:<locus>:<feature>.

Raises:

ValueError – If tier is unknown, or blocks_ names a block that is not registered.

Return type:

list[str]

mir.signature.describe(tier='standard', sig=None)

The column dictionary — one row per column, so a collaborator can read the contract.

Returns:

A pl.DataFrame with column, sig, block, locus, feature, tier, transform, attributable, exempt, magnitude.

Parameters:

• tier (str)

• sig (str | None)

mir.signature.load_reference(path=None)

Load (and cache) the frozen reference.

Raises:

FileNotFoundError – If the artifact is missing — with the command that rebuilds it, since it is derived entirely from bundled resources and needs no data to regenerate.

Parameters:

path (str | Path | None)

Return type:

SignatureReference

mir.signature.rsig(sample, *, tier='standard', species='human', weight='log2p1', reference=None, chunk=50000, min_clonotypes=5)

The rsig (geometry) half of one sample’s signature, as {column_name: value}.

Parameters:

• sample – {locus: clonotype frame}, or one frame with a locus column. Frames should already be sanitised (vdjtools.signature.blocks.sanitise); malformed junctions are silently accepted by the embedder and would quietly contaminate every coordinate here.

• tier (str) – "core", "standard" or "full".

• species (str) – Prototype panel species.

• weight (str) – Clone-size weight g.

• reference – A SignatureReference, or None for the bundled one.

• chunk (int) – Rows embedded at a time; bounds memory, not time.

• min_clonotypes (int) – Floor for a compartment to count as present. Not a floor on the sample — a whole repertoire is never dropped for being small.

Returns:

Every column the layout lists for tier and "rsig", in order, nan where the sample could not support one.

Raises:

ValueError – If tier is unknown.

Return type:

dict[str, float]

mir.signature.self_test(path=None)

Check the installed reference end to end, and return what it found.

Verifies the prototype hashes, then embeds a fixed synthetic repertoire and confirms the projection is finite, the right width, and — the property that matters — that centring actually buys discrimination. Cheap enough to run on import in a notebook.

Parameters:

path (str | Path | None)

Return type:

dict

mir.signature.signature(sample, *, tier='standard', species='human', weight='log2p1', reference=None, scale=None, standardize='reference', clip=8.0, sanitise=True, prefiltered=False, **vsig_kw)

Both halves of one sample’s signature, concatenated — the hand-off object.

With standardize="reference" (the default) every column is rescaled against the frozen reference, so the result is dimensionless and on a common scale and a downstream model needs no scaler of its own. That is the entire point: a collaborator’s matrix and ours are directly comparable, rather than each being internally consistent and mutually meaningless.

Parameters:

• sample – {locus: clonotype frame} or one frame with a locus column.

• tier (str) – "core", "standard" or "full".

• species (str) – Prototype panel species.

• weight (str) – Clone-size weight g, shared by both halves so they describe one measure.

• reference – Frozen geometry reference, or None for the bundled one.

• scale – A ScaleReference, or None for the bundled one.

• standardize (str) – "reference" to rescale against it, "none" for raw values. Asking for "reference" when none is installed raises rather than silently handing back raw numbers that look standardised.

• clip (float) – Bound in robust standard deviations when standardising.

• sanitise (bool) – Drop unusable clonotypes first. Leave this on. Turning it off no longer contaminates the geometry silently — TCREmp.embed now refuses a junction outside the 20 standard amino acids — but it does mean the call raises rather than returning a vector, which is the intended outcome and not a reason to reach for allow_nonstandard.

• prefiltered (bool) – Say True when non-functional rearrangements were already removed upstream. vsig:qc:*:nonstd_aa_frac then reports nan rather than a confident floor value, because on a pre-filtered input there is nothing left to measure. This is the ONLY part of the vector that pre-filtering affects: every other column is computed on the rows that survive sanitise, and pre-filtering does not change which rows those are. If you do not want the column at all, drop it with a preset (classify, transfer and compact all do) — that is exact, whereas filtering the repertoire is not.

• **vsig_kw – Forwarded to vdjtools.signature.vsig (cstar, pgen_q05, strict…). cstar and pgen_q05 default to the measured values in the scale reference.

Returns:

{column: value} for vsig then rsig, in layout order.

Raises:

ValueError – If standardize is unknown, or is "reference" with none available.

Return type:

dict[str, float]

mir.signature.signature_cohort(samples, *, tier='standard', n_jobs=1, columns=None, **kw)

Assemble a cohort: one row per sample, sample_id plus every signature column.

Parameters:

• samples – {sample_id: {locus: frame}} or an iterable of (sample_id, frames).

• tier (str) – Column tier.

• n_jobs (int) – Worker processes. 1 runs in-process; 0 uses every core. A cohort is embarrassingly parallel over samples – each is independent and the frozen artifacts are read-only – and per-sample cost runs from milliseconds on shallow blood to minutes on a deep tissue biopsy, so this is the difference between minutes and hours.

• columns (list[str] | None) – Explicit column subset (e.g. from a preset). Defaults to the whole tier.

Returns:

One row per sample, sample_id first, then the requested columns in layout order.

Return type:

DataFrame

mir.signature.blocks

The geometry half of the signature — features of the prototype-sum measure.

Every column here is a linear functional, a norm, or a mixture coefficient of one object:

Φ(S) = Σ_σ w_σ z_σ          z_σ = TCREmp row for clonotype σ,  w_σ = g(a_σ)/Σg

That object is fit-free: z_σ is a vector of distances to a fixed, bundled prototype panel, so no basis is estimated from anybody’s cohort and two people who never share data land in the same coordinate system. It is also linear in the clone-weight measure, which is what makes the compartment shares well-posed rather than heuristic: for a genuine partition Φ(S) = Σ_c π_c Φ(c) holds exactly, so π can be measured instead of assumed.

Three structural facts the layout depends on, all verified against the library:

• TCREmp.embed lays columns out interleaved per prototype as [V, J, junction], so the three slots are the exact strides Φ[0::3], Φ[1::3], Φ[2::3]. Attribution across them is exact, not a surrogate model.

• Prototype k’s three columns never move as the panel grows, so a K-prototype embedding is a bit-exact prefix of a larger one. Widening the panel is therefore a superset, never a new coordinate system.

• Distances to an unrecognised V or J allele silently take the germline max-distance fallback. Nothing here can detect that; vsig:qc:*:*_fallback_frac reports it, and a vector with a high fallback fraction is not comparable to one without.

Φ must be centred against the frozen reference before it means anything. Every prototype distance is large and positive, so every repertoire’s Φ sits in almost the same place: measured on eight unrelated donors, the cosine between different people’s raw Φ spans 0.9989–0.9999, and the shared offset is about 55× the between-donor signal in norm. Subtract the reference mean and the same eight span −0.81 to +0.66 — roughly a thousandfold more discriminative. So centring is not a tidying step: without it the leading component is the constant everyone shares and the identity block is nearly blank.

The centre is the naive reference, not the prototype-cloud mean — both are fit-free, but they average different things, and only one is where repertoires live. See mir.signature.reference.LocusReference.standardize() for the measurement. The same argument is why contrast() subtracts that reference rather than reporting Φ directly — it lives there, with the naive it needs, not here.

mir.signature.blocks.CHUNK = 50000

Rows embedded per chunk. The embedding itself is cheap — measured 0.01 s for 2,000 clonotypes at K=512 — so this exists to bound memory, not time: the full matrix for a 500,000-clonotype sample would be 3 GB at K=512, while the accumulators are 1536 floats.

mir.signature.blocks.BANDS: dict[str, callable] = {'middle': <function <lambda>>, 'singleton': <function <lambda>>, 'top': <function <lambda>>}

Abundance compartments, as row predicates over the clone-size vector. A partition — unlike mir.repertoire.band_frames, whose top is deliberately a subset of expanded. The mixture identity is only exact for a partition, and an NNLS over overlapping parts is not a composition at all: its weights need not sum to one and an individual share can exceed it. These are depth-fragile, and deliberately not corrected. An abundance compartment’s share genuinely moves with sequencing depth: the singleton fraction grows as rarer clones are sampled, and a 1% quantile selects 20 clonotypes in a 2,000-clonotype sample against 1,000 in a 100,000-clonotype one. Measured on one repertoire across a 67x depth range, band:top spans about 6.9 in log-ratio coordinates. Bounding the quantile to a clonotype count (as mir.repertoire.band_frames’ top_clip does) was tried and merely relocated the discontinuity, so it was reverted rather than shipped as a fix.

The signature’s answer to a depth-fragile column is not to correct it but to carry the covariate: vsig:depth:* and rsig:depth:* are in every tier precisely so a downstream model can adjust. vsig:clon:*:clr_f1 is documented the same way for the same reason.

mir.signature.blocks.ISOTYPE_BANDS: dict[str, tuple[str, ...]] = {'IgA': ('IGHA1', 'IGHA2'), 'IgG': ('IGHG1', 'IGHG2', 'IGHG3', 'IGHG4'), 'IgM': ('IGHM', 'IGHD')}

IGH isotype compartments, by constant-gene call. IGHGP is a pseudogene and IGHC is ambiguous, so neither is called; roughly two fifths of IGH reads carry no call at all and form their own part rather than being folded into IgM.

mir.signature.blocks.weights(counts, weight='log2p1')

Normalised clone weights w = g(a)/Σg.

Raises:

ValueError – If weight is unknown, or no clonotype carries any weight.

Parameters:

• counts (ndarray)

• weight (str)

Return type:

ndarray

mir.signature.blocks.prototype_sum(df, model, w, *, chunk=50000)

Φ = Σ w_σ z_σ and its Rao dispersion, in one chunked pass.

Both quantities are running sums over the rows, so the full (n, 3K) matrix is never held: the accumulators are Σ w z and Σ w‖z‖², and Rao’s Q telescopes out of the pair as 2(Σw‖z‖² − ‖Φ‖²) (see mir.repertoire.rao_dispersion()).

Parameters:

• df (DataFrame) – One locus of one sample, already sanitised. Row order must match w.

• model – A TCREmp for that locus.

• w (ndarray) – Normalised clone weights, from weights().

• chunk (int) – Rows embedded at a time.

Returns:

(phi, mean_sq_norm) — phi is (3K,), mean_sq_norm is Σ w‖z‖².

mir.signature.blocks.slots(phi)

Split Φ into its V / J / junction strides.

Exact by construction — these are literal column strides of the embedding, not an attribution model — which is what makes “how much of this distance is V?” answerable without SHAP, sampling, or a surrogate.

Parameters:

phi (ndarray)

Return type:

dict[str, ndarray]

mir.signature.blocks.depth_block(counts, w, mass)

Effective size and retained mass — the geometry’s own reading of depth.

n_eff = 1/Σw² is a Hill number of the weights the geometry actually uses, which is not the same quantity as the richness of the count vector: it says how many clonotypes are effectively contributing to Φ, and so predicts how noisy this sample’s Φ is. mass is 1 − M₀, the share of the repertoire that was ever drawn.

Parameters:

• counts (ndarray)

• w (ndarray)

• mass (float)

Return type:

dict[str, float]

mir.signature.blocks.band_shares(df, w, *, bands=None, min_clonotypes=5)

Compartment shares of Φ, in closed form rather than by NNLS.

Because Φ is linear in the clone-weight measure and the compartments partition the clonotypes, the share of Φ owned by compartment c is just its share of the weight:

Φ(S) = Σ_c π_c Φ(c)        with     π_c = Σ_{σ ∈ c} w_σ

exactly, with no fitting. Solving a non-negative least squares for the same quantity — the obvious alternative — is both slower and worse posed: over overlapping compartments the weights need not sum to one and a share can exceed it, which then breaks any log-ratio coordinate downstream.

A compartment below min_clonotypes is recorded absent (its share is dropped from the composition) rather than set to zero. Zero is a measurement; absent is not.

Takes no embedder and no Φ: by the identity above the shares are a property of the weights alone, so asking for either would be asking the caller to compute something this cannot use.

Returns:

{band: share} over the bands that cleared the floor, plus _residual for whatever no compartment owned. Shares are raw, un-transformed; the caller closes them into log-ratio coordinates.

Parameters:

• df (DataFrame)

• w (ndarray)

• bands (dict | None)

• min_clonotypes (int)

Return type:

dict[str, float]

mir.signature.blocks.isotype_shares(df, w, *, min_clonotypes=5)

Isotype shares of Φ(IGH), by the same mixture identity as band_shares().

A share of the geometry, which is a different quantity from the read fraction the statistics half reports — the two answer different questions and the signature carries both rather than picking the flattering one.

Parameters:

• df (DataFrame)

• w (ndarray)

• min_clonotypes (int)

Return type:

dict[str, float]

mir.signature.reference

The frozen reference: load it, check it is the one you think, and project through it.

A signature is only portable if everyone projects through the same basis. This module owns that basis — the artifact built by mir/resources/signature/build_rsig.py — and the one operation that turns a raw prototype-sum into signature coordinates:

Φ  →  (Φ − naive)/sd_phi  →  split into V/J/junction slots  →  rotate by R_*

The centring is not cosmetic. Every prototype distance is large and positive, so all repertoires sit in nearly the same place: across unrelated donors the raw between-donor cosine spans about 0.001, while the shared offset is ~55× the between-donor signal. Rotate without subtracting a centre and the leading component is the constant everyone shares, so the identity block comes out nearly blank. Centred, the same donors span 1.48. The centre is naive rather than the prototype cloud’s own mu_phi, which is a measured distinction — see LocusReference.standardize().

Comparability is checked, not assumed. The artifact records the prototype hash it was built against, and SignatureReference.verify() refuses a mismatch rather than silently producing numbers in a different coordinate system that look perfectly reasonable. That is the same contract RepertoireSpace and CodecBundle enforce.

mir.signature.reference.DEFAULT_PATH = PosixPath('/home/runner/work/mirpy/mirpy/src/mir/resources/signature/rsig_v2.npz')

Where the bundled artifact lives. Alongside the germline distance matrices and the prototype panels, since it is the same kind of object: a versioned, baked resource.

mir.signature.reference.SLOTS: dict[str, tuple[str, int]] = {'C': ('phic', 2), 'J': ('phij', 1), 'V': ('phiv', 0)}

Slot name in the artifact -> the layout block it feeds, and its stride in Φ.

class mir.signature.reference.LocusReference(mu_phi, sd_phi, naive, naive_sem, rotations, prototype_hash, gaps=None)

Bases: object

The frozen basis for one locus.

Parameters:

• mu_phi (ndarray)

• sd_phi (ndarray)

• naive (ndarray)

• naive_sem (ndarray)

• rotations (dict[str, ndarray])

• prototype_hash (str)

• gaps (dict[str, ndarray] | None)

mu_phi: ndarray

sd_phi: ndarray

naive: ndarray

naive_sem: ndarray

rotations: dict[str, ndarray]

prototype_hash: str

gaps: dict[str, ndarray] | None = None

slot -> (k,) relative eigenvalue gap to the next component. See exchangeable().

property n_prototypes: int

exchangeable(slot, *, tol=0.02)

Indices of components whose neighbour sits within tol relative eigenvalue.

Such a pair spans a well-determined plane, but which of the two is the earlier coordinate is determined by nothing: rebuild the artifact against a different panel size or a different LAPACK and they exchange. Measured on the junction slot, components matched by |cos| between a 5,000-prototype rotation and the whole-panel one drop to 0.01 exactly at the near-degenerate pairs while their neighbours stay above 0.95.

The shipped artifact is frozen, so nothing exchanges in practice. It matters for what a coordinate is allowed to mean: a linear model spanning the plane is unaffected, a per-coordinate feature-importance read-out on one of the pair is not interpretable.

Returns an empty array if the artifact predates the stored gaps.

Parameters:

• slot (str)

• tol (float)

Return type:

ndarray

standardize(phi)

(Φ − naive)/sd_phi — centre on an unselected repertoire, then scale.

The centre is naive, not mu_phi, and the difference is large. Both are fit-free, but they are means over different things: naive is a sample-level Φ (an unselected repertoire, weighted over its clonotypes), while mu_phi averages the prototype panel, which is a differently-constituted set. Real repertoires live near the former. Measured on two cohorts, between-donor cosine spread after centring:

centre	cohort A	cohort B
none (raw)	0.0105	0.0010
mu_phi (prototype cloud)	0.4008	0.0785
naive (unselected)	1.2660	1.6148
own sample mean (oracle)	1.9031	1.9088

naive recovers 67–85% of what an oracle centred on the cohort’s own mean achieves; mu_phi recovers 4–21%. The residual offset explains it: ‖mu_phi − sample mean‖ is 5–7× the between-donor spread, ‖naive − sample mean‖ well under it.

mu_phi is still shipped, because it is the centre the rotation was fitted against and is needed to reproduce the artifact — it is simply not the right origin for a sample.

Parameters:

phi (ndarray)

Return type:

ndarray

project(phi, slot, k=None)

Standardise Φ, take one slot’s stride, and rotate into the frozen basis.

Parameters:

• phi (ndarray) – One sample’s raw (3K,) prototype-sum.

• slot (str) – "V", "J" or "C".

• k (int | None) – Keep the leading k components; None keeps every stored one. The narrower tiers are prefixes, so this is a slice rather than a different projection.

Raises:

ValueError – If slot is unknown, phi has the wrong width, or k exceeds what the artifact stores.

Return type:

ndarray

contrast(phi, mass)

Ψ = mass·(Φ − naive) in raw prototype coordinates.

Deliberately not standardised by sd_phi: this block carries its meaning in its magnitude, and a per-coordinate rescale would make a sample that deviates barely at all look like one that deviates a lot.

Parameters:

• phi (ndarray)

• mass (float)

Return type:

ndarray

class mir.signature.reference.SignatureReference(loci, meta, path)

Bases: object

The frozen bases for every locus the artifact covers.

Parameters:

• loci (dict[str, LocusReference])

• meta (dict)

• path (Path)

loci: dict[str, LocusReference]

meta: dict

path: Path

property version: str

verify(*, species='human')

Check each locus’s basis against the prototype panel currently installed.

Returns:

{locus: True} when every hash matches.

Raises:

ValueError – On the first mismatch. Loud, because the failure it prevents is silent — a mismatched panel still yields a full, plausible vector, in coordinates nobody else shares.

Parameters:

species (str)

Return type:

dict

mir.signature.reference.load_reference(path=None)

Load (and cache) the frozen reference.

Raises:

FileNotFoundError – If the artifact is missing — with the command that rebuilds it, since it is derived entirely from bundled resources and needs no data to regenerate.

Parameters:

path (str | Path | None)

Return type:

SignatureReference

mir.signature.reference.self_test(path=None)

Check the installed reference end to end, and return what it found.

Verifies the prototype hashes, then embeds a fixed synthetic repertoire and confirms the projection is finite, the right width, and — the property that matters — that centring actually buys discrimination. Cheap enough to run on import in a notebook.

Parameters:

path (str | Path | None)

Return type:

dict

mir.signature.scale

The corpus-fitted half of the reference: per-column location and scale.

The geometry is fit-free — a rotation and a naive reference derived from bundled resources, with no cohort involved. This module owns the one part that must come from data, because there is no other way to know it: where each column typically sits and how far it typically moves.

That is deliberately the cheap part to estimate. A rotation over p=256 coordinates is not identified at a thousand samples, which is why none is fitted here; a per-column median and MAD are identified at any n, converge as 1/sqrt(n), and are what the whole re-fit story rests on. Fitting a scale is a different statistical problem from fitting a basis, and only one of them is safe at the sample sizes anyone actually has.

Three rules the estimator follows, each because the alternative silently corrupts something:

• Robust, not moment-based. Median and 1.4826·MAD. A handful of pathological repertoires in a reference corpus would otherwise set the scale for everyone.

• Observed entries only, before any imputation. Filling holes first and measuring afterwards deflates the scale in proportion to how sparse a column is, so the least-observed locus ends up with the largest apparent values and dominates every distance and principal component.

• Refuse a column the corpus barely saw. A location fitted on nine samples is not a reference. Below min_n_obs the column ships as “unscaled” and passes through, rather than carrying a confident-looking number derived from almost nothing.

It also measures the two constants the statistics half cannot pick for itself: the per-locus coverage level cstar at which Hill numbers are compared, and the Pgen quantile below which a clonotype counts as atypical. Both are quantiles of what the corpus actually attains, not values chosen by taste — a textbook C* = 0.95 puts every real repertoire into extrapolation.

mir.signature.scale.DEFAULT_PATH = PosixPath('/home/runner/work/mirpy/mirpy/src/mir/resources/signature/rsig_scale_v2.npz')

Bundled alongside the geometry artifact.

mir.signature.scale.MIN_N_OBS = 1000

A column observed fewer times than this ships unscaled. A reference is a claim about a population; a hundred samples cannot support one for 716 columns.

mir.signature.scale.MIN_N_GROUPS = 20

…and a claim about a population is not supportable from a handful of studies either, however many samples they hold. Only checked when group= is given. 20 is where the measured acceptance rate is still 0.16 (benchmarks/SIGNATURE_SCALE_N.md), i.e. the floor below which a reference is worse than none; the 947-group blood corpus clears it on every column with room to spare (minimum 566 groups behind any scaled column), so this costs nothing on a broad corpus and only bites on a narrow one — which is exactly the case that used to pass silently.

mir.signature.scale.MIN_PER_GROUP = 25

the higher floor admitted 40 of the blood reference’s 947 studies — the largest 40, which are also the most protocol-homogeneous — and missed 102 batch-loaded columns. See _batch_ratio().

Type:

Samples a group needs before its centre counts towards batch_ratio. 25, not 100

mir.signature.scale.CSTAR_QUANTILE = 0.1

real repertoires reach Good-Turing coverage 0.24-0.58, so anything near the textbook 0.95 forces every sample into extrapolation, where diversity inflates roughly tenfold.

Type:

Coverage level, as a quantile of what the reference corpus attains. Low on purpose

mir.signature.scale.COVERAGE_CEILING = 0.99

Attained coverage at or above this is treated as “no singleton tail”, not as deep sequencing. Real repertoires reach 0.24-0.58; a value here means the input was truncated or pre-collapsed.

class mir.signature.scale.ScaleReference(columns, loc, scale, n_obs, cstar, pgen_q05, meta, batch_ratio=None)

Bases: object

Per-column location and scale, plus the constants the blocks need.

Parameters:

• columns (list[str])

• loc (ndarray)

• scale (ndarray)

• n_obs (ndarray)

• cstar (dict[str, float])

• pgen_q05 (dict[str, float])

• meta (dict)

• batch_ratio (ndarray | None)

columns: list[str]

loc: ndarray

scale: ndarray

n_obs: ndarray

cstar: dict[str, float]

pgen_q05: dict[str, float]

meta: dict

batch_ratio: ndarray | None = None

Between-corpus spread of the column’s median over within-corpus spread, measured on the reference draw. Above 1 the column separates cohorts better than it separates donors within one. Diagnostic only — nothing here divides by it — but it is the number that says which columns a cross-cohort model should treat as nuisance. nan where unmeasured.

property scaled: ndarray

Mask of columns with a usable scale.

apply(values, *, clip=8.0)

Rescale one sample’s columns against the reference.

Unknown columns and columns without an established scale pass through untouched, so a caller always gets back exactly the keys it handed in. A hole stays a hole: nan is not something to centre.

Parameters:

• values (dict[str, float]) – {column: value}, e.g. from mir.signature.signature().

• clip (float) – Bound in robust standard deviations. Wide enough that a genuine outlier stays one, narrow enough that a single pathological sample cannot set a model’s scale.

Return type:

dict[str, float]

property batch_dominated: ndarray

Mask of columns that separate reference corpora better than donors within one.

Not a defect of the scaling — a property of the feature. Standardising cannot remove a batch effect, and a cross-cohort model should treat these as nuisance or residualise them. All False when no grouping was supplied at fit time.

report()

What this reference can and cannot standardise.

Return type:

dict

mir.signature.scale.fit_scale(frame, *, min_n_obs=1000, min_n_groups=20, cstar=None, pgen_q05=None, group=None, weight_by_group=True, meta=None)

Fit location and scale from an assembled cohort matrix.

The estimator is the median and 1.4826·MAD, and that is a measured choice rather than a stylistic one: the columns are heavy-tailed after their block transform (excess kurtosis 0.6-207, and 0.4-3% of samples beyond five robust deviations against the 6e-7 a normal would give), so a standard deviation is set by a handful of samples and moves when they do. Against a fit on held-out studies, the moment estimator never reaches the robust one.

One vote per study, not one per sample (weight_by_group, on by default when group is given). Samples inside a study share a protocol, a batch and a donor pool, so an unweighted fit lets whichever studies happen to be large set the reference. Measured on 23,234 blood samples over 947 SRA study groups, six independent 70/30 study splits, scoring the fit against a fit on the held-out studies — the fraction of columns whose location lands within 0.10 scales and whose scale lands within ±10%:

weighting	pass rate	sd over splits	scale swing
per sample	0.630	0.064	0.063
per study	0.792	0.089	0.044

“Scale swing” is the sd across splits of the median pool/target scale ratio: the reference is only defined up to a global multiplicative factor, and which studies you fit on moves it. One vote per study shrinks that by a third and tightens the per-column scatter four-fold (IQR of the log ratio 0.092±0.020 to 0.079±0.005).

The unit that has to converge is therefore the study, and the sample count follows from it. On the same corpus, drawing whole studies: 80 studies (~2,000 samples) is where every column clears min_n_obs; 160 studies (~3,800) reaches 0.66, 320 (~7,000) 0.74 and 640 (~14,500) 0.85. An earlier revision of this docstring quoted “median/MAD converges at N=1000” from a benchmark that drew samples IID and scored them against the same corpus’s own fit; that measures the estimator’s own noise, not what a new corpus needs, and is withdrawn.

Parameters:

• frame – A pl.DataFrame from mir.signature.signature_cohort() — one row per sample, sample_id plus signature columns.

• min_n_obs (int) – Columns observed fewer times than this get no scale.

• min_n_groups (int) – Columns observed in fewer group values than this get no scale, and a corpus with fewer groups than this in total is refused outright. Ignored without group. Pass 0 to fit a narrow corpus deliberately.

• pgen_q05 (dict | None) – Measured constants to carry alongside (see measure_constants()).

• group – Optional per-sample corpus/batch label (a column name in frame, or a sequence). Supplying it records batch_ratio per column — diagnostic only, nothing divides by it — and, unless weight_by_group=False, weights the fit itself.

• weight_by_group (bool) – Give each group one vote, spread over its samples. No effect without group. Pass False to reproduce a pre-3.11 artifact.

• meta (dict | None) – Provenance recorded into the artifact.

• cstar (dict | None)

• pgen_q05

Returns:

A ScaleReference.

Raises:

ValueError – If the frame has no signature columns.

Return type:

ScaleReference

mir.signature.scale.measure_constants(samples, *, loci=None, group=None, cstar_quantile=0.1, n_pgen=2000, threads=0)

Measure cstar and pgen_q05 per locus from a reference draw.

cstar is a low quantile of attained Good–Turing coverage, so most samples interpolate rather than extrapolate; pgen_q05 is the 5th percentile of log10 Pgen pooled over the draw, which is what “atypically improbable” is measured against.

Both are quantiles over the corpus, so both inherit fit_scale()’s problem: unweighted, a 500-sample study casts 500 votes and a 5-sample study 5, and pgen_q05 is worse — it pools n_pgen junctions per sample, so the same study puts a million junctions into a pool the small one contributes ten thousand to. Pass group to give each study one vote. Measured on the 23,234-sample / 947-group blood reference, the shift in cstar is −0.019 (TRA), −0.025 (TRB), −0.023 (IGH), −0.046 (IGK), −0.040 (IGL), −0.014 (TRG), +0.014 (TRD) — 11–17% relative on the T-cell and IGK/IGL loci, and downward, because the large studies are the deep ones. Lower is the safer direction: fewer samples pushed into extrapolation.

Parameters:

• samples – Iterable of (sample_id, {locus: frame}).

• loci – Restrict to these loci; None measures whatever appears.

• group – Optional {sample_id: label} mapping. Given, each label gets one vote, spread over its samples (and over their junctions, for pgen_q05). Samples with no entry fall into a shared None group rather than being dropped.

• cstar_quantile (float) – Quantile of attained coverage to freeze.

• n_pgen (int) – Junctions sampled per repertoire for the Pgen pool, via the same pgen_junctions() draw the per-sample block uses. This can be far smaller than the per-sample n_max: the pool is one percentile over the whole corpus, so 400 samples at 2,000 each is 800,000 junctions to place a single q05 that ~20,000 already pins. Lower it to make a reference fit cheap – Pgen is ~all of the cost, and IGH alone is ~80% of it at 1.5 ms/junction against 0.0-0.2 elsewhere.

• threads (int) – Worker threads for the Pgen batch; 0 = auto. Pgen is essentially the whole cost of this function — coverage is a one-line reduction, and the models load once — so leaving this unplumbed pins a reference fit to whatever the library defaults to regardless of the machine it was given. vsig’s pgen_block has always taken it.

Returns:

(cstar, pgen_q05), each {locus: value}.

Return type:

tuple[dict, dict]

mir.signature.scale.save_scale(ref, path=PosixPath('/home/runner/work/mirpy/mirpy/src/mir/resources/signature/rsig_scale_v2.npz'))

Write the scale artifact (and its json sidecar).

Parameters:

• ref (ScaleReference)

• path (str | Path)

Return type:

Path

mir.signature.scale.load_scale(path=None)

Load the scale artifact, or None if none is installed.

None rather than an exception only for the default path: a signature without a scale reference is still a perfectly usable raw feature vector, and the caller is told which it got via standardize=.

An explicitly supplied path that does not exist RAISES. Returning None there conflates “you did not ask for a reference” with “the reference you named is missing” – so a typo in --scale would silently produce an unstandardised matrix that looks exactly like a standardised one, and the caller has already said they want a specific artifact.

Raises:

FileNotFoundError – If path was given and does not exist.

Parameters:

path (str | Path | None)

Return type:

ScaleReference | None