Repertoire signatures

A signature is a fixed-order, name-addressed feature vector for one repertoire. mirpy emits the geometry half, rsig; the statistics half, vsig, comes from vdjtools and shares the same contract, the same corpus machinery and the same column grammar.

What this half measures, in words. The statistics half asks how diverse a repertoire is. This half asks where its receptors are. Every clonotype is placed at a point in a fixed coordinate system by measuring how similar it is to each of a panel of 256 reference receptors — so two clonotypes that use related V and J genes and have similar junctions land near each other, regardless of whether they share a single identical sequence. The repertoire is then summarised by the average of those points, weighted by clone size. Two donors can have identical diversity, clonality and gene usage and still sit in different regions of this space, which is why both halves exist.

Formally, everything in this half rests on one object:

\[\Phi(S) = \sum_\sigma w_\sigma\, z_\sigma\]

z_\sigma is a clonotype’s vector of distances to a fixed, bundled prototype panel — K = 256 receptors per locus, embedded by germline V/J distance plus junction gapblock alignment — and w_\sigma is its normalised clone weight, so \(\Phi\) is a weighted average of positions. Here Phi denotes a prototype-distance mean; mir.repertoire uses the same symbol for an RFF-based repertoire representation. TCREmp.embed interleaves the three components per prototype as [V, J, junction], so Φ[0::3], Φ[1::3] and Φ[2::3] are the exact V / J / junction slots. Literal column strides, not an attribution model, which is what makes “how much of this distance is V?” answerable without SHAP, sampling or a surrogate.

For raw sample RFF vectors and their optional PCA, see Clonotype and sample embeddings. Existing rsig is a prototype-distance geometry representation, not an RFF subset or extension. The two use different fitted artifacts.

Quickstart

# nine corpora are published; name one and the artifact is fetched on first use
mir signature --corpus blood cohort/*.tsv.gz -o rsig.tsv

# pre-warm the cache instead of fetching lazily -- the install-time step
mir corpus --fetch all

# or build your own -- still uses no samples from anybody's cohort
mir corpus --corpus naive --smoke -o naive_rsig.npz

mir signature --corpus naive_rsig.npz cohort/*.tsv.gz -o rsig.tsv
mir signature --corpus naive_rsig.npz --components 32 --describe

from mir.signature import rsig, rsig_cohort, Corpus

corpus = Corpus.load("naive_rsig.npz")
row = rsig({"TRB": trb, "IGH": igh}, corpus)
frame = rsig_cohort({"S1": s1, "S2": s2}, corpus, n_jobs=0)

Joining the two halves

There is no joined entry point, deliberately. Each half has its own artifact, so the wrapper’s only real job — applying one scale reference over both — no longer exists. Two calls and a polars join is the entire procedure:

from mir.signature import rsig_cohort
from vdjtools.signature import vsig_cohort
from vdjtools.signature.corpus import Corpus

v = vsig_cohort(samples, Corpus.load("vsig_naive.npz"))
r = rsig_cohort(samples, Corpus.load("rsig_naive.npz"))
full = v.join(r, on="sample_id", how="inner")

Use the same corpus name and seed for both halves. vdjtools corpus and mir corpus draw the same synthetic repertoires from the same seeds, which is what makes the join meaningful rather than merely type-correct.

What changed in 4.0, and why

Important

The old rotation was fitted on the wrong unit. build_rsig.py fitted R_V, R_J and R_C on (10_000, 768) — one row per clonotype, from the bundled prototype panel — while every one of the 399 PC columns it produced was a repertoire statistic, obtained by projecting a clone-weighted mean through those axes. PCA over 10,000 receptors evaluated at a mean is not PCA of repertoires: the two maximise variance in different units and give different axes. A sample’s coordinate averages ~400 effective clones. Under an independent-clone approximation, averaging reduces the coordinate variance by a factor of order \(1/400\); the corresponding standard-deviation factor is \(1/\sqrt{400}\). Clone dependence changes this effective count.

The artifact’s centre and scale were fitted on zero rows and arrived from a separate corpus of real samples. Two independent fits, stitched — which is how one shipped reference came to pair a centre of exactly 0.0 with a scale plainly fitted from data, putting a corpus-typical sample 81 robust deviations out, with 885 of 885 samples of an independent cohort outside the bound.

Now the rotation, the bounds, the centre and the scale all come out of one pass over one matrix of repertoires, per locus.

There is no ``contrast`` group any more, and nothing was lost. It was Ψ = mass·(Φ − naive), with naive a separately drawn 20,000-sequence reference, because the rotation was fit-free and needed an explicit subtraction point. The corpus centre now is that point: rotating through the naive corpus subtracts the median Φ of unselected repertoires, which is what the contrast measured. 231 columns, one frozen vector, and one whole failure mode — a naive drawn against a different release of the recombination models, which moved every contrast column by 0.1–1.6% per locus — replaced by choosing a corpus. mass remains a feature in its own right, so the rotation still sees it.

Raw feature groups

Every group at a locus is concatenated into one feature vector and rotated together.

group	width	what
phiv	256	V-germline slot of Φ: distance from each clonotype’s V gene to each prototype’s, clone-weighted
phij	256	J-germline slot
phic	256	Junction slot — the gapblock alignment distance
depth	2	n_eff = 1/Σw² and mass = 1 − M₀. A Hill number of the weights the geometry actually uses, so it predicts how noisy this sample’s Φ is
band	2	Clone-size compartment shares of Φ, in clr coordinates
band_igh	3	IGH only: isotype shares of Φ(IGH)

p_L is 772 per locus, 775 at IGH.

Note

rsig:phiv is not V-gene usage, and should not be described as such. It is the clone-weighted mean of the clonotypes’ V-germline-similarity profiles: it encodes usage only softly — a sample dominated by one V sits near that V’s row in germline space — and it is not a histogram over V genes. Explicit V and J usage are vsig features.

The compartment shares are exact, not fitted

Φ is linear in the clone-weight measure and the compartments partition the clonotypes, so

\[\Phi(S) = \sum_c \pi_c\, \Phi(c), \qquad \pi_c = \sum_{\sigma \in c} w_\sigma\]

holds exactly and the shares are read off the weights in closed form. Solving a non-negative least squares for the same quantity is both slower and worse posed: over overlapping compartments the weights need not sum to one and one share can exceed it, which breaks every log-ratio coordinate downstream.

A compartment below min_clonotypes is recorded absent — dropped from the composition — rather than set to zero. Zero is a measurement; absent is not, and a clr cannot tell them apart afterwards. When every compartment falls below the floor only the closing residual is left, which is no composition at all, so the coordinates are holes rather than an invented ratio.

Warning

The compartment shares are depth-fragile and deliberately uncorrected. A compartment’s share genuinely moves with sequencing depth: the singleton fraction grows as rarer clones are sampled, and a 1% quantile selects 20 clonotypes in a 2,000-clonotype sample against 1,000 in a 100,000-clonotype one. Measured on one repertoire across a 67x depth range, band:top spans about 6.9 in log-ratio coordinates. Bounding the quantile to a clonotype count was tried and merely relocated the discontinuity.

The answer to a depth-fragile column is to carry the covariate, not to correct it — which is why depth is in the rotation and cov:*:cstar is a channel on the other half.

Channels

Never rotated, never clamped. See Channels.

• rsig:div:<locus>:rao — log1p of Rao quadratic entropy in embedding coordinates, self-pair corrected. It sees that two clonotypes are one substitution apart, which no Hill number can, and it telescopes out of the same chunked pass that computes Φ: Q = 2(Σw‖z‖² − ‖Φ‖²) before multiplication by n_eff / (n_eff − 1) for n_eff > 1 and the final log1p transform. Carried in its own units because the head-to-head against the statistics half’s Hill numbers is the point.

• rsig:qc:-:winsor_frac — what fraction of this row the corpus’s bounds clamped.

Supports, and why Φ is one-sided

Every Φ coordinate is a weighted mean of distances, so it is non-negative and its tail runs upward only: the slots declare support="nonneg" and only their top tail is trimmed. n_eff is a count, likewise nonneg. The compartment shares are clr coordinates of a composition and are therefore two-sided, real. The full support table is in vdjtools’ signature reference — one table for both halves, since one module applies it.

Building a corpus

mir corpus --corpus synthetic-blood  -o synthetic-blood_rsig.npz
mir corpus --corpus synthetic-tissue -o synthetic-tissue_rsig.npz
mir corpus --corpus naive  -o naive_rsig.npz
mir corpus --corpus memory --size n_eff --components 0.95 -o memory_rsig.npz
mir corpus --smoke -o /tmp/smoke.npz
mir corpus --corpus naive -j 8 -o naive_rsig.npz     # 8 worker threads, within allocation

Nine corpora are published: four synthetic — naive, memory, synthetic-blood, synthetic-tissue — and five real, fitted on repertoires (blood, tissue, deep-tcr, and an uncapped variant of the first two). The commands above build a synthetic one, which needs no cohort and no download; every published artifact, real or synthetic, is instead fetched on first use from a release and cached, so --corpus blood needs no build at all. Match the corpus to how your samples were produced — the selection table in the vdjtools half is the same for both halves.

Among the synthetic four, synthetic-blood and synthetic-tissue are the ones to reach for: each repertoire is a naive/memory mixture drawn from three quantile ladders measured per locus on the compartment it is named after – richness, reads per expanded clone, and the singleton fraction that stands in for the naive compartment – through that compartment’s measured rank correlations. naive and memory are the pure regimes, and neither varies what a real cohort varies: memory at a fixed size has the same read count in every sample. The construction, the acceptance table and why reads per expanded clone is the drawable quantity are in the vdjtools half, since one module draws for both.

Both halves must be built from the same corpus name and seed: they resolve the name through one corpus_plan and write the manifest through one corpus_meta, so the cohort, the three ladders, the rank correlations and the germline fingerprint cannot disagree between vsig_<name> and rsig_<name> – which is what makes joining them on sample_id meaningful.

Learned-model pool generation uses one native batch per locus, loading each model once. Feature extraction then runs across worker threads, one contiguous block of samples each; -j/--jobs defaults to every core the process may use (-j 1 stays in-process). Both halves share one builder, so a sample is the same repertoire in either – its generator is seeded from (seed, locus, index), which is also why the artifact is identical at every --jobs.

Note

--jobs selects native pool-generation threads and subsequent sample-map threads, with only one parallel layer active at a time. Feature workers use serial inner kernels. Explicit OLGA legacy sources retain reference sampling and fixed chunk seeds; only these legacy generation paths need spawned processes and an importable entry point with a if __name__ == "__main__": guard.

The build is a deterministic function of (corpus, loci, samples, size, seed, source) and uses the isalgo vdjtools models by default. Select --source olga-legacy (or source="olga-legacy" in Python) for historical reproduction. olga aliases the frozen OLGA family; learned and arda alias fresh isalgo. Their old weights require a pinned previous release. Existing frozen corpus manifests keep their recorded model source and fingerprints. The build is required to be byte-identical across processes, worker counts and thread counts:

mir corpus --corpus naive --loci TRG,TRD --samples 24 --size 80 -o /tmp/a.npz
OMP_NUM_THREADS=1 POLARS_MAX_THREADS=1 \
  mir corpus --corpus naive --loci TRG,TRD --samples 24 --size 80 -o /tmp/b.npz
cmp /tmp/a.npz /tmp/b.npz          # must be identical

Measured on a small TRG/TRD corpus, 5 components reach 0.91–0.93 of the variance — far more compressible than the statistics half, where the same count reaches 0.34–0.48, because the 256 Φ coordinates are highly correlated distances to one panel while the vsig groups are genuinely heterogeneous.

Traps

• The SHM columns must never reach the embedder. v_identity and v_mutations silently switch TCREmp.embed to SHM-aware V distances, which is a different coordinate system under the same column names — the numbers move and nothing says so. They are dropped before embedding.

• ``chunk`` bounds memory. It must be a positive integer. The default is 4,096 rows: 24 MiB per float64 embedding block at 256 prototypes. Running sums avoid retaining the whole repertoire embedding. Changing block size preserves the calculation up to floating-point rounding.

• ``–jobs`` selects sample threads. The default is 0 (allocated cores); 1 selects serial execution. mir corpus first uses generation processes, then sample-map threads with the same budget. Positive counts must fit that allocation and are capped by sample count. Workers share frozen resources and read one sample at a time; output follows input order. Deferred readers may be closures. Callbacks must not mutate shared resources. A preflight guard estimates numerical workspace and retained output memory; reader-owned input frames need additional memory. If the guard rejects the allocation, request fewer workers with --jobs.

• A pool that cannot start raises. Startup and calculation errors propagate; there is no serial fallback.

API

Repertoire signatures, geometry half (rsig).

The column contract lives in vdjtools.signature.layout – mirpy depends on vdjtools and not the reverse – and is re-exported here so a caller of mir.signature need not know that. The rsig raw groups and channels are declared in mir.signature.signature, which registers them into that same registry at import.

Two modules:

• features – the prototype-sum measure Phi and its functionals, a pure function of one sample plus the bundled prototype panel.

• signature – rsig / rsig_cohort / synthesize, using vdjtools.signature.corpus for the fit, the winsorization and the artifact. The same machinery fits both halves, so they cannot drift into different notions of “standardised”.

class mir.signature.Channel(sig, name, features, loci=None)

A named family carried through untouched – never winsorized, never rotated.

Parameters:

• sig (str) – Owning signature.

• name (str) – Channel name.

• features (dict[str, str]) – {name: support}. The support is recorded for documentation and for a range assertion; no trimming is applied either way.

• loci (tuple[str, ...] | None) – As RawGroup.

class mir.signature.Corpus(sig, name, vocab, fits, meta=<factory>)

A fitted corpus: one LocusFit per locus, plus the manifest.

A load verifies rather than trusts: the raw column order is re-derived from the stored vocabulary through the current layout and compared, so a layout change that would silently re-index the rotation raises instead. The bundled-model version is checked the same way, because the synthetic corpora are drawn from those models and retraining them moves the result by 0.1-1.6% per locus.

Parameters:

• sig (str)

• name (str)

• vocab (dict[str, dict[str, list[str]]])

• fits (dict[str, LocusFit])

• meta (dict)

columns(n_components=None, named=())

The signature columns this corpus emits: rotated, then channels, then named blocks.

Parameters:

• n_components (int | float | None)

• named (bool | Sequence[str])

Return type:

list[str]

property k: dict[str, int]

{locus: components}. A locus absent here emits no columns – how a corpus that could not fit a locus declares it, rather than shipping a rotation of zeros.

classmethod load(path, *, verify=True)

Read an artifact, and check it is the one the caller thinks it is.

Parameters:

• path (str | Path) – The .npz path; the .json sidecar must sit beside it.

• verify (bool) – Re-derive the raw column order from the stored vocabulary through the current layout and compare, and check the recorded bundled-model version against the installed one. Turn it off only to inspect a deliberately stale artifact.

Raises:

ValueError – On a column-order or model-version mismatch. Both would otherwise produce numbers in a different coordinate system that look perfectly reasonable.

Return type:

Corpus

resolve_k(n_components=None)

{locus: k} after apply-time truncation.

Parameters:

n_components (int | float | None) – None keeps what was fitted. An int truncates every locus to that count. A float in (0, 1) truncates each locus to the fewest components reaching that cumulative variance.

Raises:

ValueError – If an int exceeds what a locus was fitted with, or a float target is not reached by the stored spectrum. Neither ever pads: a count is capped at each locus’s stored k (the same cap the fit applied, so a 5-feature cross-locus block yields 5 whether you asked for 5 or 128), and a variance fraction the stored spectrum cannot reach raises rather than silently returning the whole rotation under the requested name.

Return type:

dict[str, int]

save(path)

Write <path>.npz (arrays) and <path>.json (manifest and vocabulary).

Parameters:

path (str | Path)

Return type:

Path

verify()

Raise unless the stored column order and model version match this installation.

Return type:

None

class mir.signature.RawGroup(sig, name, features=<factory>, loci=None, dynamic=None, named=False)

One named family of raw features – an input to the rotation, never an output.

Parameters:

• sig (str) – Owning signature, "vsig" or "rsig".

• name (str) – Group name, unique within a sig.

• features (dict[str, tuple[str, str]]) – {name: (transform, support)}, most easily built with feats(). Empty for a group whose column names are not knowable without the germline; such a group declares dynamic instead.

• loci (tuple[str, ...] | None) – Loci the group is emitted for. None means all of LOCI; an empty tuple means the group is not per-locus and uses NO_LOCUS.

• dynamic (tuple[str, str] | None) – For a group whose width depends on the germline vocabulary (vus, jus, spec): one (transform, support) pair shared by every column, with the names resolved at fit time and then recorded in the corpus artifact. Keeping the vocabulary out of here is what lets import vdjtools stay free of arda.

• named (bool) – This group’s features are interpretable domain scalars – a diversity index, a read count, a singleton fraction – that a caller may legitimately want back in their own units rather than only as a rotation input. Declared here rather than inferred, because “is this number reportable” is a property of the feature and not of its width: pchem is 30 static columns and is not reportable, shm is one and is. Groups marked here are what named=True selects; see named_groups().

columns(locus, names=None)

Raw column names for one locus.

Parameters:

• locus (str) – A member of emitted_loci.

• names (list[str] | None) – Required for a dynamic group – the germline-resolved column names, in the order the artifact recorded them.

Return type:

list[str]

property emitted_loci: tuple[str, ...]

Loci this group emits for; (NO_LOCUS,) when it is not per-locus.

spec(feature)

(transform, support) for one of this group’s features.

Parameters:

feature (str)

Return type:

tuple[str, str]

mir.signature.band_shares(df, w, *, bands=None, min_clonotypes=5)

Compartment shares of Phi, in closed form rather than by NNLS.

Phi is linear in the clone-weight measure and the compartments partition the clonotypes, so compartment c’s share of Phi is exactly its share of the weight:

Phi(S) = sum_c pi_c Phi(c)        with   pi_c = sum_{sigma in c} w_sigma

with no fitting. A compartment below min_clonotypes is recorded absent – dropped from the composition – rather than set to zero. Zero is a measurement; absent is not.

Returns:

{band: share} over the bands that cleared the floor, plus _residual. Raw shares; the caller closes them into log-ratio coordinates.

Parameters:

• df (DataFrame)

• w (ndarray)

• bands (dict | None)

• min_clonotypes (int)

Return type:

dict[str, float]

mir.signature.channel_columns(sig, *, feature_definition=None)

Every pass-through channel column for one sig, in emitted order.

Parameters:

sig (str)

Return type:

list[str]

mir.signature.channels(sig=None)

Registered pass-through channels, in declaration order.

Parameters:

sig (str | None)

Return type:

list[Channel]

mir.signature.isotype_shares(df, w, *, min_clonotypes=5, masks=None)

Isotype shares of Phi(IGH), by the same mixture identity as band_shares().

A share of the geometry, which is a different quantity from the read fraction the statistics half reports. The signature carries both rather than picking the flattering one.

The masks come from isotype_masks() rather than from a second copy of the same allele-stripping comprehension — two copies is how the two came to disagree about a null. Pass already computed masks to share that parse with the dispersion pass.

Parameters:

• df (DataFrame)

• w (ndarray)

• min_clonotypes (int)

• masks (dict | None)

Return type:

dict[str, float]

mir.signature.parse(column)

Split a column name into (sig, block, locus, feature).

Also the definition of “is this a signature column”, used as an allow-list wherever a frame may carry a caller’s own joined columns – so an age or n_reads column can never be silently winsorized or rotated.

Raises:

ValueError – If the name is not four colon-separated parts.

Parameters:

column (str)

Return type:

tuple[str, str, str, str]

mir.signature.pc_columns(sig, k)

Rotated column names, given the component count per locus.

Parameters:

• sig (str) – "vsig" or "rsig".

• k (dict[str, int]) – {locus: n_components}, from the corpus artifact. A locus absent from k emits nothing, which is how a corpus that could not fit a locus declares it.

Return type:

list[str]

mir.signature.prototype_sum(df, model, w, *, chunk=4096)

Phi = sum w_sigma z_sigma and its Rao dispersion, in one chunked pass.

Both are running sums over the rows, so the full (n, 3K) matrix is never held: the accumulators are sum w z and sum w ||z||^2, and Rao’s Q telescopes out of the pair as 2(sum w||z||^2 - ||Phi||^2).

Returns:

(phi, mean_sq_norm) – phi is (3K,), mean_sq_norm is sum w ||z||^2.

Parameters:

• df (DataFrame)

• w (ndarray)

• chunk (int)

mir.signature.rao_of(phi, mean_sq, n_eff)

Rao quadratic entropy in embedding coordinates, self-pair corrected.

Sees that two clonotypes are one substitution apart, which no Hill number can. The n_eff/(n_eff-1) correction removes the self-pair bias, which is O(1/n_eff) and therefore tracks depth – exactly the confounder this column would otherwise smuggle in.

Parameters:

• phi (ndarray)

• mean_sq (float)

• n_eff (float)

Return type:

float

mir.signature.raw_and_channels(frames, vocab, *, species='human', weight='log2p1', min_clonotypes=5, chunk=4096)

Every raw rsig feature and every rsig channel for one sample.

Parameters:

• frames (dict[str, DataFrame]) – {locus: frame}, already sanitised.

• vocab (dict[str, dict]) – {locus: {}} from the corpus artifact. rsig has no germline-width group, so the per-locus dicts are empty; the keys are what say which loci the corpus models.

• species (str) – Prototype panel species.

• weight (str) – Clone-size weight, a key of vdjtools.signature.features.WEIGHTS.

• min_clonotypes (int) – Floor for a compartment to count as present. Below it the compartment is dropped from the composition, not zeroed. Never drops a whole sample.

• chunk (int) – Rows embedded per pass; bounds memory, not time.

Returns:

(raw, channels) – both {column: value}, holes as nan.

Return type:

tuple[dict[str, float], dict[str, float]]

mir.signature.raw_columns(sig, locus, vocab=None, *, feature_definition=None)

Every raw feature name for one (sig, locus), in rotation-input order.

This is the column order of one row of the corpus matrix M_L, so it is also the row order of the rotation. It must be reproduced exactly at apply time, which is why the corpus artifact stores it verbatim rather than recomputing it.

Parameters:

• sig (str) – "vsig" or "rsig".

• locus (str) – A locus, or NO_LOCUS for the cross-locus group.

• vocab (dict[str, list[str]] | None) – {group_name: [column names]} for every dynamic group emitted at this locus. Required unless no dynamic group applies.

Return type:

list[str]

mir.signature.raw_groups(sig=None)

Registered raw groups, in declaration order.

Parameters:

sig (str | None)

Return type:

list[RawGroup]

mir.signature.rsig(sample, corpus, *, mode=None, winsor_p=None, n_components=None, species='human', weight='log2p1', sanitise=True, on_duplicate='error', min_clonotypes=5, chunk=4096, named=(), columns=None)

The geometry half of the signature for one sample.

Parameters:

• sample – {locus: frame}, one frame with a locus column, or a zero-argument callable.

• corpus (Corpus) – A fitted corpus for sig="rsig".

• mode (str | None) – Winsorization mode override – "features", "pcs" or "none".

• winsor_p (float | None) – Which stored percentile to clamp at.

• n_components (int | float | None) – Truncate to this many components, or this variance fraction.

• species (str) – Prototype panel species.

• weight (str) – Clone-size weight.

• sanitise (bool) – Drop non-productive clonotypes first.

• on_duplicate (str) – "error" or "sum" for a repeated amino-acid clonotype key.

• min_clonotypes (int) – Compartment presence floor, and the floor below which the whole div and disp family of a locus is a hole rather than a number. A one-clonotype locus has a Rao of exactly 0.0 – arithmetically right, and not a diversity measurement.

• chunk (int) – Rows embedded per pass.

• named (bool | tuple[str, ...]) – Also return the reportable raw blocks (band, band_igh, depth) – True for all of them, or an explicit sequence. () emits the rotated columns and channels only. Values carry their declared transform; channel_table() reports which.

• columns (list[str] | None) – Restrict the output to these columns, in layout order.

Return type:

dict[str, float]

mir.signature.rsig_cohort(samples, corpus, *, n_jobs=0, columns=None, **kw)

One row per sample, sample_id first.

Parameters:

• samples – {sample_id: sample} or an iterable of pairs. A sample may be a zero-argument callable, which defers the read into the worker and holds peak memory at O(n_jobs) samples rather than the whole cohort.

• corpus (Corpus) – A fitted corpus.

• n_jobs (int) – Worker threads; 0 (default) uses allocated cores, 1 is serial.

• columns (list[str] | None) – Restrict the output columns.

• **kw – Forwarded to rsig().

Return type:

DataFrame

mir.signature.signature_columns(sig, k, *, feature_definition=None)

The emitted signature for one half: rotated columns then channels, in that order.

Parameters:

• sig (str)

• k (dict[str, int])

Return type:

list[str]

mir.signature.slots(phi)

Split Phi into its V / J / junction strides, exactly.

Parameters:

phi (ndarray)

Return type:

dict[str, ndarray]

mir.signature.support_of(column, vocab=None, *, feature_definition=None)

The declared support of a raw feature or channel column.

Parameters:

• column (str) – A raw or channel column name.

• vocab (dict[str, dict[str, list[str]]] | None) – Unused for lookup – a dynamic group’s support is declared on the group, not per column – and accepted so callers can pass their artifact’s vocab uniformly.

Raises:

ValueError – If no registered group or channel declares the column.

Return type:

str

mir.signature.synthesize(corpus_name, *, loci=('TRA', 'TRB', 'TRG', 'TRD', 'IGH', 'IGK', 'IGL'), n_samples=10000, size=None, seed=20260927, n_components=128, mode='features', winsor_p=0.01, source='isalgo', species='human', progress=None, n_jobs=0, depth_spread=None)

Build and fit an rsig corpus on the same synthetic repertoires vsig uses.

The receptors, the seeds and the drawn depths are all vdjtools.signature.corpus.synthesize()’s, so the two halves of a corpus name describe the same repertoires – which is what makes joining vsig_<name> and rsig_<name> on sample_id meaningful rather than merely type-correct.

Parameters:

• corpus_name (str) – One of vdjtools.signature.corpus.SYNTHETIC – "naive" / "memory" for the pure regimes, "synthetic-blood" / "synthetic-tissue" for the naive/memory mixture drawn across that cohort’s measured per-locus richness, read-depth and singleton-fraction bands.

• n_jobs (int) – Worker threads (not kernel threads). 0 means every available core; 1, the default, runs in-process. The artifact is bit-identical at every value.

• size (int | str | None) – None is the corpus’s own – 10,000 for a pure regime, the geometric centre of the cohort’s measured richness band for a synthetic-* one.

• depth_spread (float | str | None) – Multiplicative depth range each repertoire’s size is drawn log-uniformly across, around size; None is the corpus’s own. Must match the vsig half’s value for the two to describe the same repertoires.

• loci (tuple[str, ...])

• n_samples (int)

• seed (int)

• n_components (int | float)

• mode (str)

• winsor_p (float)

• source (str)

• species (str)

Returns:

(corpus, mats) – mats being {locus: (matrix, columns)}, the corpus matrix itself.

mir.signature.weights(counts, weight='log2p1')

Normalised clone weights w = g(a)/sum(g).

Raises:

ValueError – If weight is unknown, or no clonotype carries any weight.

Parameters:

• counts (ndarray)

• weight (str)

Return type:

ndarray

Raw rsig features for one sample: the prototype-sum measure and its functionals.

Everything rests on one object:

Phi(S) = sum_sigma  w_sigma * z_sigma

z_sigma is a clonotype’s vector of distances to a fixed, bundled prototype panel (K receptors per locus, embedded by germline V/J distance plus junction gapblock alignment), and w_sigma is its normalised clone weight. TCREmp.embed interleaves the three components per prototype as [V, J, junction], so Phi[0::3], Phi[1::3] and Phi[2::3] are the exact V / J / junction slots – literal column strides, not an attribution model, which is what makes “how much of this distance is V?” answerable without SHAP or a surrogate.

No rotation and no standardisation here. Both need a corpus, and both live in vdjtools.signature.corpus, which this half shares with vsig. That is the correction this rewrite exists for: the previous rotation was fitted on 10,000 individual clonotypes from the prototype panel while every column it produced was a repertoire statistic.

There is no ``contrast`` group any more, and nothing was lost. It used to be Psi = mass * (Phi - naive), with naive a separately drawn 20,000-sequence reference, because the rotation was fit-free and needed an explicit subtraction point. The corpus centre now is that point: rotating through the naive corpus subtracts the median Phi of unselected repertoires, which is what the contrast measured. 231 columns, one frozen vector and one whole failure mode – a naive drawn against a different release of the recombination models – replaced by choosing a corpus. mass remains a feature in its own right, so the rotation still sees it.

mir.signature.features.CHUNK = 4096

Phi and the Rao accumulator are both running sums, so the full (n, 3K) matrix is never held. At K=256, one float64 block uses 24 MiB. A fixed row budget, independent of depth or worker count.

Type:

Rows embedded per pass. Bounds memory, not time

mir.signature.features.K = 256

Prototypes per locus. The panel is a bundled resource and this is its size; changing it changes the raw column set, which the corpus artifact’s load-time check will refuse rather than reindex.

mir.signature.features.BANDS: dict[str, callable] = {'middle': <function <lambda>>, 'singleton': <function <lambda>>, 'top': <function <lambda>>}

Abundance compartments, as row predicates over the clone-size vector. A partition – the mixture identity below is only exact for a partition, and an NNLS over overlapping parts is not a composition at all: its weights need not sum to one and one share can exceed it.

These are depth-fragile, deliberately uncorrected. A compartment’s share genuinely moves with depth: the singleton fraction grows as rarer clones are sampled, and a 1% quantile selects 20 clonotypes in a 2,000-clonotype sample against 1,000 in a 100,000-clonotype one. Measured on one repertoire across a 67x depth range, band:top spans about 6.9 in log-ratio coordinates. Bounding the quantile to a clonotype count was tried and merely relocated the discontinuity. The answer to a depth-fragile column is to carry the covariate, not to correct it, which is why depth is in the rotation and cov:*:cstar is a channel.

mir.signature.features.ISOTYPE_BANDS: dict[str, tuple[str, ...]] = {'IgA': ('IGHA1', 'IGHA2'), 'IgG': ('IGHG1', 'IGHG2', 'IGHG3', 'IGHG4'), 'IgM': ('IGHM', 'IGHD')}

IGH isotype compartments, by constant-gene call. IGHGP is a pseudogene and IGHC is ambiguous, so neither is called; roughly two fifths of IGH reads carry no call at all and form their own part rather than being folded into IgM.

mir.signature.features.weights(counts, weight='log2p1')

Normalised clone weights w = g(a)/sum(g).

Raises:

ValueError – If weight is unknown, or no clonotype carries any weight.

Parameters:

• counts (ndarray)

• weight (str)

Return type:

ndarray

mir.signature.features.prototype_sum(df, model, w, *, chunk=4096)

Phi = sum w_sigma z_sigma and its Rao dispersion, in one chunked pass.

Both are running sums over the rows, so the full (n, 3K) matrix is never held: the accumulators are sum w z and sum w ||z||^2, and Rao’s Q telescopes out of the pair as 2(sum w||z||^2 - ||Phi||^2).

Returns:

(phi, mean_sq_norm) – phi is (3K,), mean_sq_norm is sum w ||z||^2.

Parameters:

• df (DataFrame)

• w (ndarray)

• chunk (int)

class mir.signature.features.Dispersion(phi, mean_sq, stride_sq, second, n_eff, n_rows, uni_phi, uni_mean_sq, bands, rows=None)

Everything the embedding-diversity family needs, from one chunked pass over a repertoire.

Every field is a running sum, so the (n, 3K) clonotype matrix is never held. The one field that is not a vector – second, the (3K, 3K) weighted second moment – is what the spectrum comes from, and accumulating it rather than the n x n Gram is what keeps the pass linear in sequencing depth: the Gram is quadratic, which on a deep repertoire is the whole cost of the signature rather than a few milliseconds of it.

Variables:

• phi (numpy.ndarray) – sum w z, shape (3K,).

• mean_sq (float) – sum w ||z||^2.

• stride_sq (numpy.ndarray) – sum w ||z[s::3]||^2 for s in V, J, junction order, shape (3,).

• second (numpy.ndarray) – sum w z z^T, shape (3K, 3K).

• n_eff (float) – 1 / sum w^2, the effective clonotype count under the weighting.

• n_rows (int) – Clonotypes that entered the pass.

• uni_phi (numpy.ndarray) – phi again under uniform weights – the reference for evenness.

• uni_mean_sq (float) – mean_sq under uniform weights.

• bands (dict) – {band: (mass, phi_b, mean_sq_b, sum_w2_b, n_b)} for each clone-size or isotype compartment, with sums taken over that compartment only and not renormalised; the caller divides by mass to get the within-compartment measure.

• rows (tuple[numpy.ndarray, numpy.ndarray] | None) – The embedded clonotypes and their weights, (z, w), kept only when there are no more of them than there are coordinates. That is the one case where the n x n weighted Gram is cheaper to decompose than the p x p covariance, and both have the same non-zero eigenvalues. None above that, where keeping them would be the quadratic-in-depth term this whole design exists to avoid.

Parameters:

• phi (ndarray)

• mean_sq (float)

• stride_sq (ndarray)

• second (ndarray)

• n_eff (float)

• n_rows (int)

• uni_phi (ndarray)

• uni_mean_sq (float)

• bands (dict)

• rows (tuple[ndarray, ndarray] | None)

mir.signature.features.dispersion_pass(df, model, w, *, band_masks=None, chunk=4096)

One chunked pass yielding every accumulator diversity_channels() reads.

A superset of prototype_sum(), and it re-embeds nothing: the same z that produces phi produces the strides, the uniform-weight sums, the compartment sums and the second moment. Adding a metric that reads these costs no extra embedding.

Parameters:

• df (DataFrame) – One locus’s sanitised clonotype frame, SHM columns already dropped.

• model – The embedder for this locus.

• w (ndarray) – Normalised clone weights, sum w == 1.

• band_masks (dict[str, ndarray] | None) – {name: boolean mask over df's rows}; each becomes a bands entry.

• chunk (int) – Rows embedded per pass; bounds memory, not time.

Return type:

Dispersion

mir.signature.features.diversity_channels(acc, locus)

The embedding-diversity channel family for one locus, from one dispersion_pass().

Rao quadratic entropy is the metric-space analogue of Simpson diversity, and until now it was the only diversity read-out the geometry half gave up – everything else it knew was inside a rotated component, which has no name a domain reader can use. These are the rest of the analogues, all from accumulators the Phi pass already forms:

rao	Rao’s Q over the whole embedding. Simpson, with distance.
q_v/q_j/q_c	The same restricted to the V, J and junction strides. The strides are literal column offsets, so “how much of this is V-driven” needs no attribution model.
q_frac_*	Each stride’s share of the total dispersion, in clr coordinates – a composition of where the diversity sits. No counting index has this.
evenness	Q(w) / Q(uniform): the clone-size weighting’s effect with the composition held fixed. Bounded, and far less depth-fragile than richness.
eff_dim	exp(H(lambda)) of the weighted covariance spectrum – richness as directions of receptor space occupied.
eff_dim_pr	(sum lambda)^2 / sum lambda^2, the order-2 version, led by the dominant directions.
q_top,	Rao inside a clone-size compartment, weights renormalised within it –
q_singleton	the diversity of the expanded compartment rather than its share, which band already carries.
q_ratio_top	q_top / q_singleton, the two compared directly.

Returns:

{column: value}, every value a rsig:div:<locus>:* channel. Holes are nan.

Parameters:

• acc (Dispersion)

• locus (str)

Return type:

dict[str, float]

mir.signature.features.displacement_channels(acc, locus)

disp channels: distances and cosines between compartment centroids of one locus.

A displacement is a quantity no counting index has: where the expanded compartment sits relative to the singleton one, rather than how large either is.

NOTE: only ever within one locus. Each locus has its own panel of K prototype receptors, so Phi(TRA) and Phi(TRB) are vectors in different spaces and a distance between them is arithmetic without a meaning. Cross-locus comparison belongs in vsig:pair, which is a ratio of scalars.

Parameters:

• acc (Dispersion)

• locus (str)

Return type:

dict[str, float]

mir.signature.features.slots(phi)

Split Phi into its V / J / junction strides, exactly.

Parameters:

phi (ndarray)

Return type:

dict[str, ndarray]

mir.signature.features.rao_of(phi, mean_sq, n_eff)

Rao quadratic entropy in embedding coordinates, self-pair corrected.

Sees that two clonotypes are one substitution apart, which no Hill number can. The n_eff/(n_eff-1) correction removes the self-pair bias, which is O(1/n_eff) and therefore tracks depth – exactly the confounder this column would otherwise smuggle in.

Parameters:

• phi (ndarray)

• mean_sq (float)

• n_eff (float)

Return type:

float

mir.signature.features.band_shares(df, w, *, bands=None, min_clonotypes=5)

Compartment shares of Phi, in closed form rather than by NNLS.

Phi is linear in the clone-weight measure and the compartments partition the clonotypes, so compartment c’s share of Phi is exactly its share of the weight:

Phi(S) = sum_c pi_c Phi(c)        with   pi_c = sum_{sigma in c} w_sigma

with no fitting. A compartment below min_clonotypes is recorded absent – dropped from the composition – rather than set to zero. Zero is a measurement; absent is not.

Returns:

{band: share} over the bands that cleared the floor, plus _residual. Raw shares; the caller closes them into log-ratio coordinates.

Parameters:

• df (DataFrame)

• w (ndarray)

• bands (dict | None)

• min_clonotypes (int)

Return type:

dict[str, float]

mir.signature.features.isotype_masks(df)

{isotype: boolean mask} over df’s rows, for the IGH displacement channels.

Allele-stripped for the same reason isotype_shares() strips: the classes are gene names matched by equality, so IGHG1*01 would match nothing and every read would come back uncalled — a composition rather than an error.

One polars pass, three is_in expressions. It used to to_list() the whole column and run a Python in test per clonotype per band: 8.63 ms at 10,000 clonotypes and 132.2 ms at 200,000, to answer three questions about eight gene names. A null c_call is False in every band, as None in names was.

Parameters:

df (DataFrame)

Return type:

dict[str, ndarray]

mir.signature.features.isotype_shares(df, w, *, min_clonotypes=5, masks=None)

Isotype shares of Phi(IGH), by the same mixture identity as band_shares().

A share of the geometry, which is a different quantity from the read fraction the statistics half reports. The signature carries both rather than picking the flattering one.

The masks come from isotype_masks() rather than from a second copy of the same allele-stripping comprehension — two copies is how the two came to disagree about a null. Pass already computed masks to share that parse with the dispersion pass.

Parameters:

• df (DataFrame)

• w (ndarray)

• min_clonotypes (int)

• masks (dict | None)

Return type:

dict[str, float]

rsig and rsig_cohort: the geometry half, through a corpus rotation.

The three stages are the same as the statistics half’s, and the machinery is literally the same – vdjtools.signature.corpus fits and applies both, so the two halves cannot drift into different notions of “standardised”:

features.raw_and_channels   ->  corpus.apply  ->  rsig:pc:<locus>:PCnn  +  channels

This module also declares the rsig half of the column contract, by registering its raw groups and channels into vdjtools.signature.layout at import. The contract lives in vdjtools because mirpy depends on vdjtools and not the reverse; nothing in vdjtools imports mir.

mir.signature.signature.CORPUS_REPO = 'antigenomics/mirpy'

The repo whose release assets carry the rsig artifacts. The vsig half lives in vdjtools’ releases; each half is published by the library that owns it, so a corpus name resolves to two downloads from two repos rather than one repo having to know about the other’s layout.

mir.signature.signature.bundled_names()

Every corpus name this version knows – installed, cached, or downloadable.

Return type:

list[str]

mir.signature.signature.bundled_path(name)

Resolve a corpus name or a filesystem path to an rsig artifact, else None.

Downloads on first use if the name is in the shipped index and not yet cached. One resolver, shared with the vsig half, so the two cannot disagree about precedence – a local path always wins over a download.

mir.signature.signature.raw_and_channels(frames, vocab, *, species='human', weight='log2p1', min_clonotypes=5, chunk=4096)

Every raw rsig feature and every rsig channel for one sample.

Parameters:

• frames (dict[str, DataFrame]) – {locus: frame}, already sanitised.

• vocab (dict[str, dict]) – {locus: {}} from the corpus artifact. rsig has no germline-width group, so the per-locus dicts are empty; the keys are what say which loci the corpus models.

• species (str) – Prototype panel species.

• weight (str) – Clone-size weight, a key of vdjtools.signature.features.WEIGHTS.

• min_clonotypes (int) – Floor for a compartment to count as present. Below it the compartment is dropped from the composition, not zeroed. Never drops a whole sample.

• chunk (int) – Rows embedded per pass; bounds memory, not time.

Returns:

(raw, channels) – both {column: value}, holes as nan.

Return type:

tuple[dict[str, float], dict[str, float]]

mir.signature.signature.rsig(sample, corpus, *, mode=None, winsor_p=None, n_components=None, species='human', weight='log2p1', sanitise=True, on_duplicate='error', min_clonotypes=5, chunk=4096, named=(), columns=None)

The geometry half of the signature for one sample.

Parameters:

• sample – {locus: frame}, one frame with a locus column, or a zero-argument callable.

• corpus (Corpus) – A fitted corpus for sig="rsig".

• mode (str | None) – Winsorization mode override – "features", "pcs" or "none".

• winsor_p (float | None) – Which stored percentile to clamp at.

• n_components (int | float | None) – Truncate to this many components, or this variance fraction.

• species (str) – Prototype panel species.

• weight (str) – Clone-size weight.

• sanitise (bool) – Drop non-productive clonotypes first.

• on_duplicate (str) – "error" or "sum" for a repeated amino-acid clonotype key.

• min_clonotypes (int) – Compartment presence floor, and the floor below which the whole div and disp family of a locus is a hole rather than a number. A one-clonotype locus has a Rao of exactly 0.0 – arithmetically right, and not a diversity measurement.

• chunk (int) – Rows embedded per pass.

• named (bool | tuple[str, ...]) – Also return the reportable raw blocks (band, band_igh, depth) – True for all of them, or an explicit sequence. () emits the rotated columns and channels only. Values carry their declared transform; channel_table() reports which.

• columns (list[str] | None) – Restrict the output to these columns, in layout order.

Return type:

dict[str, float]

mir.signature.signature.rsig_cohort(samples, corpus, *, n_jobs=0, columns=None, **kw)

One row per sample, sample_id first.

Parameters:

• samples – {sample_id: sample} or an iterable of pairs. A sample may be a zero-argument callable, which defers the read into the worker and holds peak memory at O(n_jobs) samples rather than the whole cohort.

• corpus (Corpus) – A fitted corpus.

• n_jobs (int) – Worker threads; 0 (default) uses allocated cores, 1 is serial.

• columns (list[str] | None) – Restrict the output columns.

• **kw – Forwarded to rsig().

Return type:

DataFrame

mir.signature.signature.synthesize(corpus_name, *, loci=('TRA', 'TRB', 'TRG', 'TRD', 'IGH', 'IGK', 'IGL'), n_samples=10000, size=None, seed=20260927, n_components=128, mode='features', winsor_p=0.01, source='isalgo', species='human', progress=None, n_jobs=0, depth_spread=None)

Build and fit an rsig corpus on the same synthetic repertoires vsig uses.

The receptors, the seeds and the drawn depths are all vdjtools.signature.corpus.synthesize()’s, so the two halves of a corpus name describe the same repertoires – which is what makes joining vsig_<name> and rsig_<name> on sample_id meaningful rather than merely type-correct.

Parameters:

• corpus_name (str) – One of vdjtools.signature.corpus.SYNTHETIC – "naive" / "memory" for the pure regimes, "synthetic-blood" / "synthetic-tissue" for the naive/memory mixture drawn across that cohort’s measured per-locus richness, read-depth and singleton-fraction bands.

• n_jobs (int) – Worker threads (not kernel threads). 0 means every available core; 1, the default, runs in-process. The artifact is bit-identical at every value.

• size (int | str | None) – None is the corpus’s own – 10,000 for a pure regime, the geometric centre of the cohort’s measured richness band for a synthetic-* one.

• depth_spread (float | str | None) – Multiplicative depth range each repertoire’s size is drawn log-uniformly across, around size; None is the corpus’s own. Must match the vsig half’s value for the two to describe the same repertoires.

• loci (tuple[str, ...])

• n_samples (int)

• seed (int)

• n_components (int | float)

• mode (str)

• winsor_p (float)

• source (str)

• species (str)

Returns:

(corpus, mats) – mats being {locus: (matrix, columns)}, the corpus matrix itself.