Uncertainties¶
The updated uncertainty checkpoint predicts a raw variance contribution for each patch using its MLP head. However, because we are working in two different spaces, i.e. normalized space for the loss and real physical space, we need to find a transformation from raw flux variance to normalized flux variance.
Derivation note: The derivation below was developed with assistance from ChatGPT 6.1 Sol. The authors retain responsibility for its correctness, implementation, and interpretation.
Transformation of Raw Flux Variance to Normalized Logarithmic Variance¶
We begin with the normalization applied to the raw flux \(X\):
where \(a\) and \(b\) are the mean and standard deviation used for normalization (the values stored in sxr_norm.npy),
respectively, and \(\epsilon=10^{-8}\) W/m² is the normalization offset that we apply to make sure we don’t get weird nans when working in log space.
We assume that the normalized flux follows a Gaussian predictive distribution,
where \(\hat{y}\) is the predicted normalized flux and \(v\) is the normalized predictive variance.
Our objective is to determine \(v\) given the predicted raw flux \(\hat{x}\) and raw predictive variance \(S^2\).
Inverse Normalization¶
Rearranging the normalization gives
This is the transformation from normalized prediction space into raw prediction space.
Variance Transformation¶
Starting from the definition of variance,
Substituting the inverse normalization,
For a Gaussian random variable \(Y\sim\mathcal{N}(\hat{y},v)\), the moment-generating function gives
See this Wikipedia link.
Using \(10^z=e^{z\ln(10)}\), the first moment becomes
Similarly, the second moment is
Substituting both moments into the variance definition,
Solving for normalized variance¶
The inverse-transformed point prediction satisfies
Here, \(\hat{x}\) is the raw prediction.
For convenience, define
The raw variance equation simplifies to
Dividing by \(M^2\),
Introducing the substitution \(u=e^t\) gives
Solving the quadratic equation and selecting the physically valid root \(u\geq 1\),
Since \(u=e^t\), taking the natural logarithm yields
Finally, using \(t=(b\ln(10))^2v\), we obtain
Numerically stable implementation¶
Define the dimensionless ratio
The logarithmic term can be rewritten as
Rationalizing the fraction,
Therefore,
This form is mathematically equivalent to the original expression, but avoids
subtracting nearly equal quantities and allows the use of a numerically stable
log1p implementation.
Prediction Intervals¶
The API’s nominal 68% and 95% full-disk intervals use \(z=1\) and \(z=1.96\), respectively. With \(t=(b\ln(10))^2v\), their endpoints in W/m² are
These are the values returned in Prediction.prediction_intervals.
The uncertainty loss and the model’s full-disk variance output use the
normalized Gaussian space defined above. The API reuses
forecasting.inference.uncertainty_from_prediction to form
\(\hat{y}\pm z\sqrt{v}\) in that space, then inverse-transform each endpoint
to W/m². The standard deviation \(\sqrt{v}\) is in normalized log space;
it must not be added to or subtracted from the raw flux.
The bounds are asymmetric around total_flux; a displayed
total_flux ± sqrt(S²) label reports the median and raw standard deviation,
not these interval endpoints.
Reconstructing the full-disk bounds from its raw flux and summed patch
variance gives the same result as inverse-transforming the normalized
Gaussian endpoints, provided the same \(\epsilon\) is used. The raw variance
is exact, but total_flux ± sqrt(S²) is a different interval.