What is the MOST likely cause of this issue?
The class distribution in the dataset is imbalanced.
Dataset shuffling is disabled.
The batch size is too big.
The learning rate is very high.
Explanations:
An imbalanced class distribution can lead to poor performance or biased predictions, but it typically causes issues with overall accuracy and model generalization, rather than oscillations in training accuracy.
While disabling dataset shuffling can lead to poor convergence and might make the training process less effective, it does not typically cause oscillations in accuracy. Instead, it may result in a more stable but potentially slower learning process.
A batch size that is too big may reduce the noise in gradient estimates, leading to more stable updates rather than oscillations. Larger batches usually smooth out the learning process and can result in less variance in accuracy.
A very high learning rate can cause the model’s weights to update too aggressively, leading to overshooting the optimal solution, which results in oscillations in training accuracy as the model repeatedly overshoots in either direction.
I rank that the answer is:
The learning rate is very high.