Skip to content

TrustBench Documentation

Welcome to the documentation for TrustBench, a modern, framework-independent machine learning evaluation library for model reliability, robustness, uncertainty, and fairness.

Traditional model evaluation focuses almost exclusively on test accuracy. TrustBench extends evaluation to answer critical operational questions:

  • Calibration: Does the model's confidence accurately match its empirical probability of being correct?
  • Robustness: How severely does performance degrade when inputs suffer from physical, digital, or environmental corruptions?
  • Uncertainty & Error Detection: Can uncertainty metrics (predictive entropy, free energy) catch incorrect predictions before they cause failures?
  • Selective Prediction: What risk-coverage trade-off is achievable if low-confidence predictions are deferred?
  • Multi-Task & Multi-Framework: Does evaluation work seamlessly across Classification, Regression, Information Retrieval, and Semantic Segmentation across PyTorch, Scikit-learn, ONNX Runtime, and experimental adapters for Keras and JAX?

Documentation Contents