The investigation, not just the answer

Aditya-L1 publishes soft X-ray measurements from two payloads at one-minute cadence. Can a model trained on that archive detect solar flares better than the simplest rule an operator would write by hand?

The question was fixed before the data was touched, along with the metrics that would answer it. That ordering is what makes the eventual negative result publishable rather than a story assembled after the fact.

What the project is →

Expected · Machine learning would beat the threshold, and spectral features would be why.

The reasoning was ordinary and, at the time, sound. A single count-rate threshold uses one number per minute and discards everything else — the shape of the rise, the recent variance, the spectral distribution. A learner with fourteen features and access to the spectrum should have had a great deal to work with.

Neither half of the hypothesis survived contact with the evaluation.

The models that were tried →

Two years of archive products were parsed onto a canonical one-minute grid: 1,985 Parquet files across seven tables, frozen at a single digest and never modified since. Missing minutes were left missing.

The limitations were written down before modelling, not after. Two of them decided the shape of everything that followed: the effective sample size is events rather than minutes, and no instrument response exists, so physical severity targets are unconstructible from this archive.

The ten limitation clauses →

Expected · A threshold would be a floor to clear.

A single threshold on the SoLEXS count rate detected 76 of 82 M/X events (event recall 0.927) while raising 15 false alarm runs.

The baseline was not a floor. It was, and remained, the best operational detector in the study. Everything after this point is an attempt to displace it.

Threshold detector card →

Expected · Gradient boosting would win.

Logistic regression, a random forest and LightGBM were fitted on the same fourteen features under a protocol frozen in advance — seed 20260718, chronological split, no shuffling. LightGBM ranked minutes best of anything evaluated: PR-AUC 0.707 against the threshold's 0.486.

On the ranking metric, the hypothesis looked confirmed. This is the point at which a less careful project would have published.

LightGBM model card →

Expected · Better ranking would mean better detection.

It did not. LightGBM detected 81 of 82 events against the threshold's 76 — no more flares — and the confidence intervals on ROC-AUC overlapped the baseline's, so no model was statistically distinguishable from it.

Ranking and detection came apart. At a base rate near one percent, being better at ordering minutes does not translate into catching more events, because the events were already at the top of both orderings.

ROC and PR curves →

Alarms were then counted as runs — contiguous blocks of predicted-positive minutes — because that is what an operator responds to. LightGBM raised 79 false runs against the threshold's 15: about 5 times as many, for no additional events detected.

This settled the question. The comparison is decided on the metric that matters operationally, and on that metric the learned models lose decisively.

The recall–burden trade-off →

The remaining defence of a learned model is that its probabilities are useful even if its alarms are not. LightGBM's reliability diagram and Brier score (0.0155) are published so that argument can be evaluated rather than asserted.

The threshold detector emits a physical rate, not a probability, so it has no reliability diagram at all. Where a comparison is impossible, no comparison is drawn.

The reliability diagram →

A one-line detector on the SoLEXS count rate remains the operational recommendation. Machine learning ranks individual minutes better, detects no more flares, and raises roughly 5× the false alarms. Added spectral resolution changed ROC-AUC by 0.0033 — a confirmed null.

The negative result is published at full weight, and the model that lost has the most detailed card on the site. A platform whose central claim is that its evidence is checkable earns that claim by publishing the finding a positive-result incentive would bury.

The findings →