Alpha
- class cockpit.quantities.Alpha(track_schedule, verbose=False)[source]
Alpha Quantity class for the normalized step length α.
The normalized step length α uses a noise-aware quadratic loss landscape fit to estimate whether current local steps over- or undershoot the local minimum.
The fit uses zero- and first-order information, including uncertainties, between two consecutive iterations which are referred to as
'start'
and'end'
point, respectively. This information needs to be projected onto the update step.Note
This quantity requires the optimizer be specified in the
'optimizer'
info
entry of acockpit(...)
context.Note
For SGD with default parameters the projections onto the search direction can be performed during a backward pass without storing large tensors between start and end point.
Initialization sets the tracking schedule & creates the output dict.
- Parameters
track_schedule (callable) – Function that maps the
global_step
to a boolean, which determines if the quantity should be computed.verbose (bool, optional) – Turns on verbose mode. Defaults to
False
.