ILLUSTRATIVE · NASA / SVS · NOT ADITYA-L1 DATA
Model cardsthreshold_rate
Model card · candidate detector
Threshold on the count rate
Raises an alarm whenever the SoLEXS 1-minute count rate exceeds a single fixed value.
What is this?
A model card for Threshold on the count rate, one of eight detectors evaluated on the frozen Aditya-L1 flare benchmark.
Why should I trust it?
Every figure below is read at build time from the committed benchmark artifact; none is transcribed. The judgements are traceable to the written evaluation protocol and the dataset limitations record.
Where can I verify it?
- The validation evidenceComputed from 192,541 held-out predictions
- Dataset limitationsL-1 … L-10
- Evidence traceabilityclaim → artifact → commit
- The raw payloadfindings/models.json
Section 1
Purpose and intended use
Why it is in the benchmark. The operational recommendation. It exists to answer the question every ML benchmark should have to answer first: what does the simplest thing that could possibly work do?
Intended use. Flagging minutes of elevated soft X-ray flux in the frozen archive for human review, and serving as the baseline any proposed detector must beat before it is worth deploying.
Out of scope. Not a forecaster — it describes the present minute, not a future one. Not a severity estimate: without an instrument response there is no calibrated flux, so a rate cannot be converted to a GOES class.
Section 2
Inputs, outputs, training
Inputs. One number per minute: the SoLEXS count rate from T1 `rate_total`.
Outputs. A binary alarm per minute. The score is a physical rate, not a probability, which is why no reliability diagram is published for it — there is nothing to be calibrated against.
Training protocol. No fitting in the machine-learning sense. The threshold is selected on the training period only and then frozen; the test period never influences it.
Operating point. Decision threshold 6.2335, selected on the training period and then frozen.
Section 3
Evaluation
Both tasks, both reported.
M/X NOWCAST · 192,541 held-out minutes · base rate 1.24%
| Metric | Value | What it measures |
|---|---|---|
| ROC-AUC | 0.954 | Ranking quality across all thresholds |
| PR-AUC | 0.486 | The informative curve at a low base rate |
| Precision | 0.548 | Of the minutes flagged, how many were flares |
| Recall | 0.589 | Of the flare minutes, how many were flagged |
| F1 | 0.568 | Harmonic mean of the two above |
| MCC | 0.562 | Correlation, robust to imbalance |
| Brier | — | Probability accuracy; undefined for hard classifiers |
| Balanced accuracy | 0.791 | Mean of the two class recalls |
- Events detected
- 76 / 82
- Event recall 0.927 · 95% CI 0.875–0.976
- False alarm runs
- 15
- Of 88 alarm runs raised. A run, not a minute — the unit an operator actually responds to.
- ROC-AUC interval
- 0.940–0.966
- 95%, day-block bootstrap.
M/X 30-MIN PREDICTION · 192,541 held-out minutes · base rate 1.29%
| Metric | Value | What it measures |
|---|---|---|
| ROC-AUC | 0.792 | Ranking quality across all thresholds |
| PR-AUC | 0.074 | The informative curve at a low base rate |
| Precision | 0.119 | Of the minutes flagged, how many were flares |
| Recall | 0.310 | Of the flare minutes, how many were flagged |
| F1 | 0.172 | Harmonic mean of the two above |
| MCC | 0.175 | Correlation, robust to imbalance |
| Brier | — | Probability accuracy; undefined for hard classifiers |
| Balanced accuracy | 0.640 | Mean of the two class recalls |
- Events detected
- 36 / 82
- Event recall 0.439 · 95% CI 0.250–0.571
- False alarm runs
- 208
- Of 240 alarm runs raised. A run, not a minute — the unit an operator actually responds to.
- ROC-AUC interval
- 0.708–0.856
- 95%, day-block bootstrap.
Confusion matrix · M/X NOWCAST
True negative
188,985
False positive
1,163
False negative
984
True positive
1,409
Section 4
Strengths and failure modes
Strengths
- Auditable end to end — the decision rule is one comparison, inspectable by anyone.
- No training artifacts, no serialised weights, no dependency on a modelling library.
- Degrades transparently: when it is wrong, the reason is visible in the light curve.
Failure modes
- Blind to any flare whose peak rate stays below the threshold, regardless of shape.
- Fires on instrument artifacts that raise the rate without a solar cause.
- Alarm runs cluster: an active day produces many alarms, so the burden is uneven in time.
Section 5
Limitations and ethical considerations
Ethical considerations. The main risk is misplaced authority. This is a detector over one archive from one detector on one instrument, in one phase of the solar cycle. It is not a space-weather warning service and must never be presented as one.
Governing limitation clauses
Clause text is not restated here. It is published in full on the dataset card, from the frozen DATASET_LIMITATIONS_FOR_ML.md report, so there is exactly one copy of every caveat.