grad_norm_gauge
- class cockpit.instruments.grad_norm_gauge(self, fig, gridspec)[source]
Showing the gradient norm versus iteration.
If the training gets stuck, due to a small
UpdateSize
it can be the result of both a badly chosen learning rate, or from a flat plateau in the loss landscape. This instrument shows the gradient norm at each iteration, overlayed with an exponentially weighted average, and can thus distinguish these two cases.Preview
Requires
The gradient norm instrument requires data from the
GradNorm
quantity class.- Parameters
self (CockpitPlotter) – The cockpit plotter requesting this instrument.
fig (matplotlib.figure.Figure) – Figure of the Cockpit.
gridspec (matplotlib.gridspec.GridSpec) – GridSpec where the instrument should be placed