ILLUSTRATIVE · NASA / SVS · NOT ADITYA-L1 DATA
Model cardsrandom_forest
Model card · candidate detector
Random forest
An ensemble of decision trees, averaged.
What is this?
A model card for Random forest, one of eight detectors evaluated on the frozen Aditya-L1 flare benchmark.
Why should I trust it?
Every figure below is read at build time from the committed benchmark artifact; none is transcribed. The judgements are traceable to the written evaluation protocol and the dataset limitations record.
Where can I verify it?
- The validation evidenceComputed from 192,541 held-out predictions
- Dataset limitationsL-1 … L-10
- Evidence traceabilityclaim → artifact → commit
- The raw payloadfindings/models.json
Section 1
Purpose and intended use
Why it is in the benchmark. A high-capacity, low-tuning learner. If nonlinearity and feature interaction were the missing ingredient, this is where it should have shown up.
Intended use. Benchmark comparison and impurity-based feature attribution.
Out of scope. Impurity importance is biased toward high-cardinality continuous features and says nothing about direction. It is not a substitute for a physical account.
Section 2
Inputs, outputs, training
Inputs. Fourteen features per minute, all derived from the SoLEXS 1-minute count rate in T1: the log rate, rolling means over 5/15/30/60 minutes, a 15-minute rolling maximum, rolling standard deviations, a background-excess term, two rise rates, and three data-quality terms (GTI fraction, seconds present, partial-minute flag).
Outputs. A probability per minute, from the fraction of trees voting positive.
Training protocol. Chronological split with a held-out test period beginning at the frozen test start date; no shuffling, so no future minute can inform a past one. The protocol — split, seed, metric set and bootstrap scheme — was fixed in EVALUATION_PROTOCOL.md before any model was fitted.
Operating point. Decision threshold 0.9126, selected on the training period and then frozen.
Section 3
Evaluation
Both tasks, both reported.
M/X NOWCAST · 192,541 held-out minutes · base rate 1.24%
| Metric | Value | What it measures |
|---|---|---|
| ROC-AUC | 0.966 | Ranking quality across all thresholds |
| PR-AUC | 0.699 | The informative curve at a low base rate |
| Precision | 0.766 | Of the minutes flagged, how many were flares |
| Recall | 0.590 | Of the flare minutes, how many were flagged |
| F1 | 0.667 | Harmonic mean of the two above |
| MCC | 0.669 | Correlation, robust to imbalance |
| Brier | 0.019 | Probability accuracy; undefined for hard classifiers |
| Balanced accuracy | 0.794 | Mean of the two class recalls |
- Events detected
- 80 / 82
- Event recall 0.976 · 95% CI 0.930–1.000
- False alarm runs
- 61
- Of 149 alarm runs raised. A run, not a minute — the unit an operator actually responds to.
- ROC-AUC interval
- 0.956–0.976
- 95%, day-block bootstrap.
M/X 30-MIN PREDICTION · 192,541 held-out minutes · base rate 1.29%
| Metric | Value | What it measures |
|---|---|---|
| ROC-AUC | 0.784 | Ranking quality across all thresholds |
| PR-AUC | 0.115 | The informative curve at a low base rate |
| Precision | 0.152 | Of the minutes flagged, how many were flares |
| Recall | 0.332 | Of the flare minutes, how many were flagged |
| F1 | 0.208 | Harmonic mean of the two above |
| MCC | 0.210 | Correlation, robust to imbalance |
| Brier | 0.051 | Probability accuracy; undefined for hard classifiers |
| Balanced accuracy | 0.654 | Mean of the two class recalls |
- Events detected
- 34 / 82
- Event recall 0.415 · 95% CI 0.220–0.547
- False alarm runs
- 243
- Of 276 alarm runs raised. A run, not a minute — the unit an operator actually responds to.
- ROC-AUC interval
- 0.694–0.850
- 95%, day-block bootstrap.
Confusion matrix · M/X NOWCAST
True negative
189,717
False positive
431
False negative
980
True positive
1,413
Section 4
Feature attribution
Which of the fourteen features the fit relied on.
Attribution is not explanation. These features are strongly collinear — four of them are rolling means of the same series — so credit assigned to one is credit withheld from another, and none of it constitutes a physical account of flare onset.
Section 5
Strengths and failure modes
Strengths
- Captures interactions and thresholds without them being specified.
- Insensitive to feature scaling and to monotone transforms.
Failure modes
- Capacity far exceeds the effective sample size, so it can fit event-specific noise (L-1).
- Probabilities from vote fractions are not calibrated by construction.
Section 6
Limitations and ethical considerations
Ethical considerations. Its opacity makes an unexplained alarm hard to contest. On a surface where every claim must be checkable, that is a cost, not a neutral property.
Governing limitation clauses
Clause text is not restated here. It is published in full on the dataset card, from the frozen DATASET_LIMITATIONS_FOR_ML.md report, so there is exactly one copy of every caveat.