grad_norm_gauge

class cockpit.instruments.grad_norm_gauge(self, fig, gridspec)

Showing the gradient norm versus iteration.

If the training gets stuck, due to a small UpdateSize it can be the result of both a badly chosen learning rate, or from a flat plateau in the loss landscape. This instrument shows the gradient norm at each iteration, overlayed with an exponentially weighted average, and can thus distinguish these two cases.

Preview

Requires

The gradient norm instrument requires data from the GradNorm quantity class.

Parameters

• self (CockpitPlotter) – The cockpit plotter requesting this instrument.

• fig (matplotlib.figure.Figure) – Figure of the Cockpit.

• gridspec (matplotlib.gridspec.GridSpec) – GridSpec where the instrument should be placed