Got it. Thanks so much for your help!! Still a lot to learn here.

Coming from a world of building software where things are very binary (it works or it doesn't), it's also really tough to judge how good is "good enough". There is a point of diminishing returns, and not sure at what point to say that it's good enough vs continuing to learn and improve it.

Really appreciate your help here tho.

So someone else suggested to reduce the learning rate. I tried that and at least to me it looks a lot more stable between runs. All the code is my original code (none of the suggestions you made) but I reduced the learning rate to 0.00001 instead of 0.0001.

Not quite sure what that means exactly tho. Or if more adjustments are needed.

As for the confusion matrix. I think the issue is the difference between smoothed values in TensorBoard vs the actual values. But I just ran it again with the previous values to verify. It does look like it matches up if you look at the actual value instead of the smoothed value.

Sorry for the delayed reply. I really appreciate your help so far.

Here is the raw link to the confusion matrix: https://eventfrontier.com/pictrs/image/1a2bc13e-378b-4920-b7f6-e5b337cd8c6f.webm

I changed it to keras.layers.Conv2D(16, 10, strides=(5, 5), activation='relu'). Dense units still at 64.

And in case the confusion matrix still doesn't work, here is a still image from the last run.

EDIT: The wrong image was uploaded originally.

Ok I changed the Conv2D layer to be 10x10. I also changed the dense units to 64. Here is just a single run of that with a Confusion Matrix.

I don't really see a bias towards non-blurred images.

So does the fact that they aren't converging near the same point indicate there is a problem with my architecture and model design?

Got it. I'll try with some more values and see what that leads to.

So does that mean my learning rate might be too high and it's overshooting the optimal solution sometimes based on those random weights?

I think what you’re referring to with iterating through algorithms and such is called hyper parameter tuning. I think there is a tool called Keras Tuner you can use for this.

However. I’m incredibly skeptical that will work in this situation because of how variable the results are between runs. I run it with the same input, same code, everything, and get wildly different results. So I think in order for that to be effective it needs to be fairly consistent between runs.

I could be totally off base here tho. (I haven’t worked with this stuff a ton yet).