Knowing the weights: four Bayesian lenses on uncertainty

I. The boat and its weights

In Essay #4, we gave the model a voice.

Learning to Say "I Don't Know": The First Floor of Awareness Essay #4 in the Humble Model Series

It learned to express uncertainty and defer when an input looked unfamiliar. But the first experiment revealed a serious limitation: the model could recognize natural distributional shift, yet remain silent under adversarial attack.

In Essay #5, we added a depth sensor.

The Geometry of Fragility: Feeling the Decision Boundary A CIFAR-10 experiment comparing confidence, gradient sensitivity, and directional boundary search as signals of…

The model began to measure its proximity to the decision boundary. Confidence alone was not enough. A model could be highly confident and still be standing dangerously close to a boundary that a small perturbation could cross.

In Essay #6, we changed the boat itself.

Architectural awareness: building the sensor into the boat Essay #6 in the Humble Model Series

Instead of attaching uncertainty after training, we built uncertainty signals into the architecture. Multi-head disagreement and evidential evidence became internal signals of instability. Under the transferred baseline-PGD protocol, these architecture-native signals produced substantially lower adversarial risk than ordinary confidence or boundary distance.

But all of these approaches still left one question unanswered.

They observed the model's outputs.

They measured its geometry.

They changed its architecture.

But what about the model's parameters themselves?

The weights are where the model stores its learned representation of the world. They are the memory of its training experience. They determine how it transforms an input into a prediction.

Can the model know whether its own weights are reliable?

This is the question explored in Essay #7.

II. The Bayesian view

In ordinary deep learning, training produces one set of weights. These weights are point estimates. After training, the model behaves as if one particular parameter configuration is the answer.

But the data may not determine one perfect configuration.

Several different parameter settings may explain the training data almost equally well. Some regions of parameter space may be strongly supported by the data. Other regions may be poorly constrained.

The Bayesian view attempts to represent this uncertainty with a distribution over weights rather than a single point. For a new input, the predictive distribution then averages over all plausible weight settings, weighted by how well each one explains the training data.

That average is intractable for any realistic neural network. We therefore use approximations.

This essay studies four of them:

Monte Carlo Dropout: using stochastic dropout at inference time.

Deep Ensembles: training several models and measuring their disagreement.

SWAG: fitting a Gaussian approximation to the trajectory of training weights.

Variational Inference, learning distributions over selected model parameters.

They all ask a similar question:

If the model had learned differently, would it still make the same prediction?

If the answer is no, the prediction may deserve caution.

III. The experimental setting

All four methods were evaluated using the same CIFAR-10 setting and the same CNN backbone family used in Essay #6.

The CIFAR-10 test set was divided into two parts:

• 2,000 images for calibration
• 8,000 images for final evaluation

The calibration split was used to select an uncertainty threshold for each method. The evaluation split remained untouched until the final measurement.

The target was approximately 25% clean deferral. Each method received its own threshold, because the numerical scale of its uncertainty signal was different.

Each method used a fixed sampling budget: 50 stochastic forward passes for Monte Carlo Dropout, 5 independently trained members for Deep Ensembles, and 30 posterior samples each for SWAG and Variational Inference. All four were scored with the same uncertainty measure — the maximum per-class variance of the predictive distribution — so that the comparison reflects differences in the underlying approximation rather than differences in how uncertainty was read off.

The risk definition remained consistent with the earlier essays:

Risk = 100 − accuracy on non-deferred predictions.

Deferred predictions were not counted as incorrect predictions. Coverage and deferral were reported separately.

The adversarial examples were generated against the baseline CNN using projected gradient descent, then transferred to the uncertainty methods. This is therefore a transferred baseline-PGD experiment, not an adaptive attack designed against each Bayesian method individually.

The baseline CNN achieved 78.61% clean accuracy and 21.09% accuracy under the transferred attack.

The question was not whether the Bayesian models could resist every possible attack.

The question was narrower:

Do their uncertainty signals help them avoid making dangerous predictions under the same transferred attack?

IV. Four ways of looking at the weights

1. Monte Carlo Dropout

Dropout is normally used during training as a regularization technique. During inference, the model usually becomes deterministic.

Monte Carlo Dropout keeps dropout active during inference and performs multiple stochastic forward passes. Each pass uses a slightly different effective network.

The model does not produce one prediction. It produces a collection of predictions.

If those predictions agree, the input appears stable. If they disagree, the input may be associated with parameter uncertainty.

In this experiment, 50 stochastic forward passes were used for each input.

Code snippet (PyTorch)

def mc_dropout_predict(model, image, num_passes=50):
    model.train()  # enable dropout
    predictions = []
    for _ in range(num_passes):
        with torch.no_grad():
            pred = torch.softmax(model(image), dim=1)
            predictions.append(pred)
    model.eval()
    predictions = torch.stack(predictions)
    mean_pred = predictions.mean(dim=0)
    variance = predictions.var(dim=0)
    return mean_pred, variance

def mc_dropout_predict(model, image, num_passes=50):
    model.train()  # enable dropout
    predictions = []
    for _ in range(num_passes):
        with torch.no_grad():
            pred = torch.softmax(model(image), dim=1)
            predictions.append(pred)
    model.eval()
    predictions = torch.stack(predictions)
    mean_pred = predictions.mean(dim=0)
    variance = predictions.var(dim=0)
    return mean_pred, variance

2. Deep Ensembles

Deep Ensembles train several models independently, with different random initializations and training trajectories. Each model produces a prediction. The ensemble combines them and measures their disagreement.

Deep Ensembles are not literal samples from the exact Bayesian posterior. They are a practical approximation to uncertainty arising from multiple plausible learned solutions.

Their strength is also their cost: instead of one model, several must be trained and evaluated.

The question is whether that additional computation creates a more useful uncertainty signal.

Code snippet (PyTorch)

def ensemble_predict(models, image):
    predictions = []
    for model in models:
        model.eval()
        with torch.no_grad():
            pred = torch.softmax(model(image), dim=1)
            predictions.append(pred)
    predictions = torch.stack(predictions)
    mean_pred = predictions.mean(dim=0)
    variance = predictions.var(dim=0)
    return mean_pred, variance

def ensemble_predict(models, image):
    predictions = []
    for model in models:
        model.eval()
        with torch.no_grad():
            pred = torch.softmax(model(image), dim=1)
            predictions.append(pred)
    predictions = torch.stack(predictions)
    mean_pred = predictions.mean(dim=0)
    variance = predictions.var(dim=0)
    return mean_pred, variance

3. SWAG

SWAG, or Stochastic Weight Averaging Gaussian, models uncertainty through the trajectory of weights during training.

Instead of treating only the final weights as meaningful, it collects weights from different points in the training process and fits a Gaussian approximation to them. At inference time, new weight samples are drawn from this approximation, and the resulting predictions are combined into a mean prediction and an uncertainty estimate.

Conceptually, SWAG asks:

What does the path taken through weight space tell us about the solutions the model might have learned?

Code snippet (PyTorch)

# Collect weights during training
swag_weights = []
for epoch in range(num_epochs):
    train_one_epoch(model)
    swag_weights.append(model.state_dict().copy())

# Fit Gaussian
mean_weights = average(swag_weights)
covariance = compute_covariance(swag_weights)

# Sample and predict
def swag_predict(model, image, num_samples=30):
    predictions = []
    for _ in range(num_samples):
        sampled_weights = sample_from_gaussian(mean_weights, covariance)
        model.load_state_dict(sampled_weights)
        with torch.no_grad():
            pred = torch.softmax(model(image), dim=1)
            predictions.append(pred)
    predictions = torch.stack(predictions)
    return predictions.mean(dim=0), predictions.var(dim=0)

# Collect weights during training
swag_weights = []
for epoch in range(num_epochs):
    train_one_epoch(model)
    swag_weights.append(model.state_dict().copy())

# Fit Gaussian
mean_weights = average(swag_weights)
covariance = compute_covariance(swag_weights)

# Sample and predict
def swag_predict(model, image, num_samples=30):
    predictions = []
    for _ in range(num_samples):
        sampled_weights = sample_from_gaussian(mean_weights, covariance)
        model.load_state_dict(sampled_weights)
        with torch.no_grad():
            pred = torch.softmax(model(image), dim=1)
            predictions.append(pred)
    predictions = torch.stack(predictions)
    return predictions.mean(dim=0), predictions.var(dim=0)

4. Variational Inference

Variational Inference explicitly assigns distributions to model parameters. Instead of learning one value for a weight, the model learns a mean and a variance.

During training, weights are sampled from these distributions. The loss includes both the prediction error and a term that keeps the learned distribution close to a prior.

In this notebook, the variational treatment was applied to the fully connected layers while the convolutional layers remained deterministic.

This distinction matters. The model is not a fully Bayesian CNN. It is a partially Bayesian architecture with variational fully connected layers.

Code snippet (PyTorch)

class BayesianLinear(nn.Module):
    def __init__(self, in_features, out_features):
        super().__init__()
        self.weight_mean = nn.Parameter(torch.randn(out_features, in_features))
        self.weight_logvar = nn.Parameter(torch.randn(out_features, in_features))
        self.bias_mean = nn.Parameter(torch.randn(out_features))
        self.bias_logvar = nn.Parameter(torch.randn(out_features))

    def forward(self, x):
        weight_std = torch.exp(0.5 * self.weight_logvar)
        bias_std = torch.exp(0.5 * self.bias_logvar)
        weight = self.weight_mean + weight_std * torch.randn_like(weight_std)
        bias = self.bias_mean + bias_std * torch.randn_like(bias_std)
        return F.linear(x, weight, bias)

class BayesianLinear(nn.Module):
    def __init__(self, in_features, out_features):
        super().__init__()
        self.weight_mean = nn.Parameter(torch.randn(out_features, in_features))
        self.weight_logvar = nn.Parameter(torch.randn(out_features, in_features))
        self.bias_mean = nn.Parameter(torch.randn(out_features))
        self.bias_logvar = nn.Parameter(torch.randn(out_features))

    def forward(self, x):
        weight_std = torch.exp(0.5 * self.weight_logvar)
        bias_std = torch.exp(0.5 * self.bias_logvar)
        weight = self.weight_mean + weight_std * torch.randn_like(weight_std)
        bias = self.bias_mean + bias_std * torch.randn_like(bias_std)
        return F.linear(x, weight, bias)

V. The results

The results were not uniform.

The strongest result came from Deep Ensembles, with an adversarial risk of 15.42%. Monte Carlo Dropout followed closely at 17.15%.

Both were modestly lower than the architecture-native signals measured in Essay #6:

• Multi-head disagreement: 19.86%
• Evidential evidence: 21.31%

This comparison deserves a caveat. The Essay #6 values are reference results carried forward, not re-evaluated in this run; the differences are a few percentage points on a single 8,000-image evaluation without confidence intervals. The ranking is suggestive, not established.

What can be said more firmly is that both Bayesian methods land in the same general region as the strongest architecture-native signals, and far below the post-hoc baselines from Essay #6: confidence at 78.70% and boundary distance at 77.06%.

This suggests that uncertainty arising from model diversity can be useful. When several independently trained models disagree, that disagreement may reveal regions where the learned solution is unstable.

But the result changed with SWAG.

SWAG produced an adversarial risk of 25.95%, higher than Deep Ensembles, MC Dropout, and both architecture-native signals from Essay #6. Its more structured approximation to the training trajectory did not translate into a stronger signal than simply training several models and letting them disagree.

It is worth keeping the scale in view, however. SWAG still reduced adversarial risk to roughly a third of what plain confidence achieved in Essay #6. The honest reading is not that SWAG failed, but that it was outperformed. A principled approximation can be genuinely useful and still lose to a simpler one.

Placing these results alongside the reference values from Essay #6 gives the full picture:

Essay #6 values are reference results, carried forward from the previous experiment, not re-evaluated in this run.

The variational result

The most dramatic result came from Variational Inference.

Its clean risk was 90.62%, and its adversarial risk was 89.69%. Its accuracy on retained predictions, 9.38% clean, 10.31% adversarial, is indistinguishable from random guessing on ten classes.

This is not an underfit model. It is a model that never trained at all.

The training loss began near 2.37 and ended near 2.33, essentially flat across twenty epochs. The cross-entropy of a uniform guess over ten classes is about 2.303. The model spent its entire training run sitting on that number.

The cause is visible in the initialization of the variational layers. The log-variance parameters were initialized near zero, which sets the sampled weight standard deviation near 1.0, while the weight means were initialized at a scale of roughly 0.1. The noise injected on every forward pass was therefore about ten times larger than the signal it was perturbing. No stable function could be learned through it. Standard Bayes-by-Backprop implementations initialize this standard deviation several orders of magnitude smaller.

This result should not be read as evidence that Variational Inference is ineffective. It is a broken configuration, not a verdict on the method.

But the failure is more instructive than a simple training bug, because of how the uncertainty behaved.

The variational model deferred on 30.40% of clean inputs and only 24.80% of adversarial ones. It became less cautious when the input was attacked. Its maximum predictive variance was 0.000237, three orders of magnitude below the other three methods. The model was not merely wrong. It was quietly and consistently wrong, with an uncertainty signal that carried almost no information about the danger it was facing.

This is the cleanest illustration in the series so far of a distinction the earlier essays kept circling. The model possessed a genuine distribution over its weights. It sampled from that distribution at inference. Every mathematical ingredient of Bayesian uncertainty was present.

And the resulting signal was decoupled from the thing it was supposed to detect.

A distribution over weights is not awareness. Awareness requires that the variation correspond to the failure.

VI. What did the model gain?

The Bayesian methods added a new kind of sensor.

Earlier essays focused on uncertainty in the input, the output, the decision boundary, or the architecture. This experiment introduced uncertainty connected to the learned parameters themselves.

The most successful methods gained the ability to ask:

Would another plausible version of this model make the same prediction?

Deep Ensembles provided the strongest answer in this experiment. MC Dropout provided a similar answer at a lower computational cost.

It is also worth noting what these methods did not cost. Deep Ensembles achieved 10.31% clean risk against the baseline's 21.39%. The gate was not buying adversarial safety by sacrificing clean performance. At roughly 21.5% clean deferral, the retained predictions were substantially more reliable than the undeferred baseline's.

But the gain was not universal.

SWAG showed that a more structured approximation to the training trajectory does not automatically produce a more useful signal.

Variational Inference showed something more important: uncertainty is not valuable if the underlying predictive model has not learned the task.

The goal is not to create variation for its own sake.

The goal is to create variation that corresponds to genuine uncertainty about the prediction.

VII. What we learned

The question was: can the model know its own weights?

The answer is partial.

Some approximations to weight-level uncertainty produced a stronger relationship between uncertainty and adversarial error than the signals examined in Essay #6. Deep Ensembles produced the lowest adversarial risk, followed by Monte Carlo Dropout.

This suggests that parameter-level variation can help the model recognize situations in which its prediction is unstable.

But the experiment did not show that all Bayesian methods are equally useful.

SWAG performed worse than the strongest architecture-native signals, though still far better than the post-hoc baselines.

Variational Inference failed to learn a useful classifier in its current form.

The deeper lesson is this:

Uncertainty is not automatically awareness.

A distribution over weights is not enough. The distribution must be connected to a model that has learned meaningful structure. Its variation must also correlate with the situations in which the model is likely to fail.

The Bayesian view therefore adds a new lens, but not a universal solution.

Deep Ensembles and MC Dropout suggest that model diversity can improve awareness.

SWAG reminds us that a principled approximation can still fail to produce the strongest signal.

Variational Inference reminds us that mathematical uncertainty cannot compensate for a model that has not learned the task.

The boat can carry many kinds of sensors.

But a sensor becomes meaningful only when it responds to the danger the boat is actually facing.

VIII. What we still do not know

This experiment leaves several questions open.

• We do not yet know whether the advantage of Deep Ensembles and MC Dropout will remain on larger architectures and more complex datasets.
• We do not know whether the relative ranking will change under adaptive attacks designed specifically against each uncertainty method. Every number here comes from a transferred attack.
• We do not know whether the Bayesian signals can be usefully combined with the architecture-native signals from Essay #6, or how either would compare to attention entropy, the direction deferred back in Essay #6 and planned for Essay #8. Given how often the simpler method has won in this series, that comparison is worth taking seriously rather than assuming sophistication will carry it. We do not know whether a correctly initialized variational model would behave like the other Bayesian methods, or whether it would reveal a different failure mode of its own. The configuration tested here never learned the task, so it tells us nothing about the approach itself.
• We also need to distinguish more carefully between uncertainty measures. The present comparison uses maximum class-probability variance as a common experimental score. Other measures, such as predictive entropy or mutual information, may reveal different behavior.

The current experiment is therefore not the final word on Bayesian uncertainty.

It is another measurement.

And the measurement changes the question.

The question is no longer simply whether a model can know its weights.

The question is:

Which representation of uncertainty becomes useful when the model is asked to recognize its own possible failure?

IX. The next floor

In Essay #4, the model learned to say: I am not sure.

In Essay #5, it learned to sense the ground beneath its decision boundary.

In Essay #6, the sensors became part of the boat.

In Essay #7, we looked below the visible structure and asked whether the weights themselves could carry uncertainty.

The answer was neither a simple success nor a simple failure.

Some methods made the model more sensitive to its own instability. Some added complexity without producing a stronger signal. One failed to learn a useful classifier at all.

That is not a disappointment. It is the kind of answer a humble model should give us.

Not every principled idea becomes a working sensor. Not every distribution represents useful knowledge. And not every uncertainty estimate deserves to become a decision.

There is one more thing worth noticing about where this ended.

The series moved steadily inward. Essay #4 read the model's outputs. Essay #5 measured the geometry around its decisions. Essay #6 rebuilt its architecture. Essay #7 went further still, into the weights themselves, and the method that performed best was Deep Ensembles, which was already sitting in Essay #4.

Nothing was wasted in getting here. Without the intermediate essays there would be no protocol strict enough to trust this comparison, and no way to know that the simplest answer was also the strongest. But it is worth saying plainly: the deepest lens did not win. The oldest one did.

Perhaps that is the most fitting result for a series about humility. We went looking for awareness in increasingly sophisticated places, and found that training several models and listening to their disagreement remains hard to beat.

The model becomes more aware not when it possesses more mathematical machinery, but when its uncertainty begins to correspond to the places where it can genuinely be wrong.

The sensor is no longer only attached to the boat. It has begun to look into the weights that hold the boat together.

But looking inward is not the same as understanding.

That remains the next question.

Come with me.

Resources

• Essay #4:

Learning to Say "I Don't Know": The First Floor of Awareness Essay #4 in the Humble Model Series

• Essay #5:

The Geometry of Fragility: Feeling the Decision Boundary A CIFAR-10 experiment comparing confidence, gradient sensitivity, and directional boundary search as signals of…

• Essay #6:

Architectural awareness: building the sensor into the boat Essay #6 in the Humble Model Series

• Notebook:

Adversarial-Notebooks/#7/Note07_Bayesian_Uncertainty.ipynb at master ·… Research notebooks and essays on adversarial attacks, defenses, uncertainty, OOD detection, and model awareness. …

References

Blundell, C., Cornebise, J., Kavukcuoglu, K., & Wierstra, D. (2015). Weight uncertainty in neural networks. ICML.

Gal, Y., & Ghahramani, Z. (2016). Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. ICML.

Lakshminarayanan, B., Pritzel, A., & Blundell, C. (2017). Simple and scalable predictive uncertainty estimation using deep ensembles. NeurIPS.

Maddox, W. J., Izmailov, P., Garipov, T., Vetrov, D. P., & Wilson, A. G. (2019). A simple baseline for Bayesian uncertainty in deep learning. NeurIPS.

Sensoy, M., Kaplan, L., & Kandemir, M. (2018). Evidential deep learning to quantify classification uncertainty. NeurIPS.