Reliability: scoring one modelled structure#

A generator’s confidence is not a binding prediction. On the balanced VDJdb panel, models in ipTM’s top decile are still 26.2 % [18.7, 35.5] non-binders — and that is exactly the band where the coordinates carry the most information the confidence does not read. This module is the read-out for one structure at a time.

S: three blocks, one divide#

\[S \;=\; \frac{Q}{\sigma_Q} \;+\; \frac{T}{\sigma_T} \;+\; \frac{\Pi - \mu_\Pi}{\sigma_\Pi}\]

Each block is a fit-free directional score \(z(x)^\top C^{-1} s\) standardized against the Native2026 crystals: \(Q\) the interface geometry (tcren.cohort.Q_FEATURES_GEOM), \(T\) the footprint shape free of its size (T_FEATURES_TOPO), and \(\Pi\) the interface energy read against the partition function (neg_energy from tcren.potts).

The outer transform is a divide, not a z-score. A block score’s native mean is 0 by construction, so re-centring does nothing; its variance is \(s^\top C^{-1} s\), which is not 1. Measured native spreads are 1.43 (\(Q\)), 1.61 (\(T\)) and 14.13 (\(\Pi\)), so without the division the energy would carry ten times the weight of the geometry. The claim the formula makes is equal weight in native-sd units, and the division is what makes it true.

Why this and not a cohort-refit posterior#

The latent-class posterior this module replaced refitted on every call and raised when a cohort had fewer rows than features — so it was undefined for a single structure, and its value depended on what else was scored alongside it. Neither property survives contact with a user holding one model. It was discarded in 2.26.0. S fits nothing at call time.

The generator diagnostic#

af_band() looks a confidence up in a frozen band table: how often a model this confident is a non-binder, with a Wilson interval, and s_roc_in_band — what S still separates inside that band. Bands are deciles of the benchmark’s own confidence distribution, never scanned for an effect. Values outside the range clamp to the end bands.

From the command line#

$ tcren features -s models/ -i placement,interface,topology,energetics -o feats.tsv
$ tcren recognize --features feats.tsv -o scores.tsv

  S: Q + T in native-sd units, 618 of 618 finite

$ tcren assess --features feats.tsv -o assessed.tsv --band 'tcrvdb|ipTM'

Two commands, because they answer different questions and print different lines. recognize --features is what emits S and reports how many rows it is finite on. assess reads the frozen model, ranks on binder_score, and — when the table carries a confidence column — adds the band diagnostic: for the tcrvdb|ipTM table the top decile is one where 15.3 % [8.2 %, 26.5 %] of benchmark models are non-binders and S still separates them at 0.773 ROC-AUC. Without a confidence column assess says so and skips that block rather than imputing one.

Add the energy term by joining tcren potts score’s neg_energy; without it recognize emits the two-block form and says so rather than imputing.

API#

tcren.reliability.s_score(table, reference=None, energy=None)[source]#

S, the recommended single-structure binder score. Higher = more native-like.

Parameters:

  • table – a tcren features table (dict / pandas / polars) carrying the four geometry and five topology descriptors.

  • reference – overrides reliability_reference().

  • energy – \(\Pi\) per row, from tcren.potts.bound_unbound()’s neg_energy. When None the energy term is dropped and the score is Q/sd + T/sd — still defined, and reported as such rather than silently imputed.

Returns:

One value per row. NaN only where a whole block is unavailable.

Return type:

ndarray

tcren.reliability.t_score(table, reference=None)[source]#

Topology block T — the same construction as tcren.cohort.q_score(), other terms.

T reads the SHAPE of the footprint rather than its size, which is why it survives where the geometry block does not: on the balanced VDJdb panel it loses 0.06 ROC-AUC when the epitope has no solved complex to template on, against Q’s 0.24.

Return type:

ndarray

tcren.reliability.af_band(iptm, reference='binder_bm|ipTM')[source]#

Look each confidence up in the frozen band table: how often is a model this confident wrong?

The bands are deciles of the benchmark’s own confidence distribution, never scanned for an effect. Each entry carries p_nonbinder with a Wilson interval and s_roc_in_band — what S still separates INSIDE that band, which is the actionable half: on the balanced VDJdb panel the top ipTM decile is 26% non-binders and is also where S reads highest.

Values outside the reference range clamp to the end bands rather than extrapolating.

Parameters:

reference (str)

Return type:

list[dict]

tcren.reliability.reliability_reference()[source]#

The Q and T descriptors plus the Potts energies over the Native2026 crystals.

Same role as tcren.cohort.native_reference(), extended to the topology block and the partition-function terms so a single user structure can be standardized against the crystal manifold for all three blocks at once.

Return type:

dict

tcren.reliability.available_bands()[source]#

The frozen confidence-band tables, <benchmark>|<confidence>.

Return type:

list[str]

tcren.reliability.moments()[source]#

Every frozen constant this module reads, from data/reliability_moments.json.

Four keys, and none of them is a fit against a binding label. blocks carries the native

mean and spread of each block, keyed by Q, T and the \(\Pi\) column named in pi_frozen; those spreads are the divisors in s_score() and are measured on the Native2026 crystals. af_bands carries the confidence-band tables af_band() reads, which are quantile bins of the generator’s confidence with the observed non-binder fraction and its Wilson interval in each. phi carries the per-interface energy spreads.

The out-of-fold-fitted sections that used to sit here — the Platt links and the confidence correction — were removed in 2.28.0; see the manuscript repository’s LEGACY.md.

Returns:

The parsed JSON. Cached, so callers may treat it as read-only.

Return type:

dict

tcren.reliability.inversion_flag(table, reference=None, energy=None)[source]#

Is this model’s recognition energy running backwards? The forced-pose detector.

A co-folding model that has been pushed into a confident but wrong pose does not produce a random interface. To seat the chains it selects residue pairs it believes are favourable, so the recognition energy comes out good — better, often, than a genuine complex of the same epitope. The energy therefore inverts under forcing rather than degrading, and the inversion is a signal with its sign flipped, not an absence of signal.

Measured on a 24-cohort forced-pose panel (1,707 structures), the TCR:peptide energy reads macro ROC-AUC 0.4952 against the forced poses and is below 0.5 in 15 of the 24 cohorts, where the generator’s own ipTM reads 0.6093. That is what this flag reads off one structure: an energy far better than the crystal manifold’s while the geometry and topology blocks are not is the forced-pose signature, because a generator can fake favourable contacts far more easily than it can fake a well-formed footprint.

Parameters:
Returns:

the energy block’s native-sd score minus the mean of the geometry and topology blocks’. Large positive means the energy is vouching for a structure the shape does not, which is the pattern to distrust. It is a diagnostic to rank and inspect by, and the package ships no probability to threshold on.

Return type:

One value per row

tcren.reliability.screening_yield(score, budget=0.1, prevalence=None)[source]#

What a caller gets for testing the top budget fraction of a scored set.

ROC-AUC answers “does the score order the set”. It does not answer the question an experimenter asks, which is “I can test ten of these hundred models – how many binders do I get?”. This is that number.

Parameters:

  • score – the score for every candidate in the set, higher = more likely to bind.

  • budget (float) – fraction of the set that can be tested, in (0, 1].

  • prevalence (float | None) – the hit rate expected if the set were tested at random. When given, the returned expected_hits is what testing that slice blindly would yield – the number the score has to beat to have been worth computing.

Returns:

n_tested, the threshold score at the cut, and rank_cut, the percentile it corresponds to; plus expected_hits when prevalence is given.

Return type:

dict

Note

Enrichment – hits over the random baseline – is deliberately not returned. It needs the labels this function does not have, and returning it as NaN would read like a measurement. The benchmark computes it where the labels are, in bench/metrics.screening_yield.

tcren.reliability.T_FEATURES_TOPO = ('D2_pep24', 'fp_b0_frac_r7', 'H_cell', 'L_canon', 'ab_imb')#

the SHAPE of the contact set, free of its size.

Type:

The topology block

tcren.reliability.T_SIGNS = (1.0, -1.0, 1.0, 1.0, 1.0)#

Its orientation. Every term rises towards a native interface except the footprint’s connected-component fraction at 7 A, which falls.

tcren.reliability.PI_FROZEN = 'neg_energy'#

The partition-function-referenced energy S spends. See the module docstring.

Nothing here is fitted against a binding label#

Version 2.28.0 removed the last read-outs that were: the frozen Platt links behind p_binder and the four-coefficient confidence correction behind tcren diagnose, both of which were fitted out of fold on the benchmarks. Every quantity this module now returns is a directional score or an empirical band table, so a value computed today depends on the structure it was computed from and on the 374 Native2026 crystals, and on nothing else. The removed read-outs and the numbers they produced are recorded in the manuscript repository’s LEGACY.md.