TCRvdb: replay the frozen score and inspect recorded figure tables

Optional supplied-data replay: the retained result bundle and companion dataset driver are not distributed with TCRen. Use the self-contained confidence-motif notebook for a first example.

Set TCREN_FINAL to the retained final result bundle containing predictions.parquet, scores.tsv, and comparisons/. This notebook keeps all 2,034 unlabelled-context TCRvdb rows, including 252 ambiguous calls, and checks the frozen score replay. Classification summaries remain the recorded library outputs for 1,782 supplied binary calls. It never refits weights, re-extracts contacts, or recalculates performance. No model identifiers are displayed.

Use the release Python kernel. Inputs and outputs stay local; do not publish private model identities, labels, provenance or notebook outputs. Raw-source reproduction is a separate explicit full-panel run, described below.

# Report the active kernel and package versions.
import sys, json, time
from pathlib import Path
from importlib.metadata import version
from importlib.resources import files
import numpy as np
import polars as pl
from tcren.contacts.motif_optimize import apply_confidence_score
started = time.monotonic()
print({name: version(name) for name in ["tcren", "numpy", "polars"]})
print("Python", sys.version.split()[0])
resource_path = files("tcren.data").joinpath("motif_confidence_reference.json")
resource = json.loads(resource_path.read_text())

# Resolve the supplied retained bundle; no machine-specific path is embedded.
import os
final = Path(os.environ["TCREN_FINAL"]).expanduser().resolve()
for name in ["predictions.parquet", "scores.tsv", "comparisons/cohort_metrics.tsv", "comparisons/summary.tsv"]:
    if not (final / name).is_file():
        raise FileNotFoundError(name)

# Retain all unlabelled-context rows; outcome labels never enter scoring.
columns = ["id", "dataset", "stratum", "epitope", "iptm", "betti", "enrichment", "plddt"]
features = (pl.scan_parquet(final / "predictions.parquet")
            .filter(pl.col("dataset") == "tcrvdb").select(columns).collect().sort("id"))
assert features["id"].n_unique() == features.height
scored = apply_confidence_score(features, resource)
print({"context_models": features.height, "cohorts": features["epitope"].n_unique()})

# Verify the frozen library replay against the recorded final scores by exact ID.
recorded = (pl.read_csv(final / "scores.tsv", separator="\t", infer_schema_length=10000)
            .filter(pl.col("dataset") == "tcrvdb").select("id", pl.col("score").alias("recorded_score")))
check = scored.select("id", "score").join(recorded, on="id", how="full", coalesce=True, validate="1:1")
assert check.height == features.height and not check.null_count().to_numpy().any()
np.testing.assert_allclose(check["score"], check["recorded_score"], rtol=0, atol=1e-15)
print("Frozen score replay agrees within 1e-15 for every retained model")

# Display recorded AP/AUROC and exact outcome counts without re-evaluating labels.
selected = "round7-complementary/cohort-plddt-rank"
metrics = pl.read_csv(final / "comparisons/cohort_metrics.tsv", separator="\t")
(metrics.filter((pl.col("experiment") == selected) & (pl.col("dataset") == "tcrvdb")
                & (pl.col("population") == "all_labelled"))
 .select("epitope", "arm", "n", "binders", "nonbinders", "AP", "AUROC")
 .sort("epitope", "arm"))

Figure tables and raw-input reproduction

In the companion benchmark checkout, export the retained comparisons with:

python bench/scripts/prepare_motif_release_tables.py --final "$TCREN_FINAL" -o "$NEW_TABLE_DIRECTORY"

The reshaper preserves all 27 cohorts, both population masks, the selected score, ipTM, Betti, mock enrichment, the previous ipTM/Betti score and both prior triplets. It copies recorded AP/AUROC; it neither fits nor evaluates. VDJdb template strata remain separate.

For fresh raw-coordinate reproduction, use an immutable released wheel and record its SHA256. For each complete epitope/pMHC cohort separately, supply confidence TSV columns complex.id, iptm, plddt (generator scale 0–100) and run:

tcren screen -s models.tar.gz --confidence confidence.tsv -t 0 -o scores.tsv

Do not mix epitopes into one screen call. Preserve the 30 contact-bearing-model floor and zero-contact reasons. Provide the original recovered pLDDT values; the existing motif-confidence-inputs command documents hash-verified recovery from generator coordinates. Missing metadata otherwise invokes frozen imputation and does not reproduce those recovered values. Use the library evaluation commands with original supplied calls. Source manifests and original commands belong to the private benchmark SOURCES record. Never substitute a partial cohort, discard ambiguous context rows, reuse cached feature blocks, or treat this frozen-feature replay as proof of a fresh raw-coordinate reproduction.

Interpretation

The selected result is adaptive in-sample TCRvdb development; VDJdb is reported separately as transfer. The selected baseline retains the recorded template-absent tradeoff. All comparison rows remain available. A future untouched-panel test is a separate scientific step.

# Record replay diagnostics only; no random sampling or scientific rerun occurred.
print({"context_models": features.height, "seconds": round(time.monotonic() - started, 3),
       "replay_tolerance": 1e-15, "fitting": False, "raw_coordinate_reproduction": False})