Insights Object

The insights object contains derived or aggregated metrics computed across one or more test runs. Insights help identify trends, patterns, and changes in test behavior over time, enabling cross-run analysis, flakiness detection, and baseline comparisons.

info

The insights object is optional and is typically added by consumers during post-processing rather than by test reporters directly.

Run-Level vs Test-Level Insights

Insights can be computed at two levels:

Run-Level Insights

The insights object at the root level contains metrics aggregated across multiple test runs. These provide an overall view of test health and trends.

Test-Level Insights

The insights object within individual test objects contains metrics for that specific test case across multiple executions. This enables per-test trend analysis and flakiness detection.

---

Run-Level Insights Properties

Property	Type	Required	Description
passRate	metricDelta	Optional	Overall pass rate across all tests, with baseline comparison.
failRate	metricDelta	Optional	Overall failure rate across all tests.
flakyRate	metricDelta	Optional	Rate of flaky tests across all tests.
averageRunDuration	metricDelta	Optional	Average total duration of test runs.
p95RunDuration	metricDelta	Optional	95th percentile of total test run duration.
averageTestDuration	metricDelta	Optional	Average duration per individual test.
runsAnalyzed	Integer	Optional	Number of historical runs included in the insights calculation.
extra	Object	Optional	Extension point for custom metrics or metadata.

---

Test-Level Insights Properties

Property	Type	Required	Description
passRate	metricDelta	Optional	Pass rate for this specific test across runs.
failRate	metricDelta	Optional	Failure rate for this specific test.
flakyRate	metricDelta	Optional	Flaky rate for this specific test.
averageTestDuration	metricDelta	Optional	Average duration of all attempts of this test across analyzed runs.
p95TestDuration	metricDelta	Optional	95th percentile execution duration for this test.
executedInRuns	Integer	Optional	Number of runs in which this test was executed.
extra	Object	Optional	Extension point for custom metrics or metadata.

---

Metric Delta

The metricDelta object represents a metric's value with optional baseline comparison, enabling delta analysis between runs.

Property	Type	Required	Description
current	Number	Optional	The current value of the metric.
baseline	Number	Optional	The value of the metric from the baseline run for comparison.
change	Number	Optional	The computed difference between current and baseline values.

---

Example

{
  "insights": {
    "runsAnalyzed": 10,
    "passRate": { "current": 0.95, "baseline": 0.92, "change": 0.03 },
    "failRate": { "current": 0.05, "baseline": 0.08, "change": -0.03 },
    "flakyRate": { "current": 0.02, "baseline": 0.03, "change": -0.01 },
    "averageRunDuration": { "current": 45000, "baseline": 42000, "change": 3000 }
  },
  "results": {
    "tests": [
      {
        "name": "should authenticate user",
        "status": "passed",
        "duration": 300,
        "insights": {
          "passRate": { "current": 0.90, "baseline": 0.85, "change": 0.05 },
          "flakyRate": { "current": 0.10, "baseline": 0.15, "change": -0.05 },
          "averageTestDuration": { "current": 280, "baseline": 250, "change": 30 },
          "executedInRuns": 10
        }
      }
    ]
  }
}

See the Metrics Reference for detailed definitions of each metric and the Baseline Object for information on baseline comparisons.