Skip to content

Reporting & Visualization

TrustBench generates both machine-readable JSON reports and standalone, interactive HTML reports viewable in any modern web browser without running a server.


1. Standalone HTML Reports

Generated via trustbench evaluate or generate_html_report(): - Zero-Dependency Viewing: Fully self-contained HTML with embedded Plotly charts. Does not require TrustBench, Node.js, or an active internet connection. - Interactive Visualizations: - Confusion Matrix: Heatmap with normalized percentages and class labels. - Reliability Diagram: Expected vs observed accuracy across confidence bins with calibration gap shading. - Confidence Histogram: Frequency of predicted confidence scores. - Risk-Coverage Curve: Error rate vs fraction of retained predictions. - AUROC Curve: Error detection trade-off. - Robustness Degradation: Multi-line plot tracking accuracy degradation across severity levels.


2. Machine-Readable JSON Reports

Reports adhere to a versioned schema (REPORT_SCHEMA_VERSION = 1.0):

{
  "schema_version": "1.0",
  "generated_at": "2026-11-09T00:00:00Z",
  "metadata": {
    "experiment_name": "resnet18_cifar10",
    "model": {"name": "ResNet18", "checkpoint_sha256": "..."},
    "dataset": {"name": "cifar10", "num_samples": 10000}
  },
  "metrics": {
    "classification": {"accuracy": 0.9539, "macro_f1": 0.9539},
    "calibration": {"ece": 0.0262, "brier_score": 0.0712},
    "uncertainty": {"mean_entropy": 0.1824}
  }
}

3. Generating Reports Programmatically

from trustbench.reporting.html_report import generate_html_report
from trustbench.reporting.schema import generate_json_report

# Generate JSON
json_str = generate_json_report(result, experiment_name="run_01")

# Generate HTML
html_str = generate_html_report(result, title="ResNet18 Reliability Report")
with open("report.html", "w", encoding="utf-8") as f:
    f.write(html_str)