Skip to content

llmcompressor.metrics.utils.frequency_manager

Frequency management utilities for metrics logging.

Provides classes and functions for managing logging frequencies and determining when metrics should be recorded during training and compression workflows. Supports both epoch-based and step-based logging with configurable modes and intervals.

Classes:

  • FrequencyManager –

    Class for managing the frequency of logging and model updates

Functions:

  • log_ready –

    Check if we are ready to log again based on the given parameters

FrequencyManager 

FrequencyManager(
    log_frequency: LogStepType = None,
    mode: LoggingModeType = DEFAULT_LOGGING_MODE,
    frequency_type: FrequencyType = DEFAULT_FREQUENCY_TYPE,
)

Class for managing the frequency of logging and model updates

Parameters:

  • log_frequency

    (LogStepType, default: None ) –

    The frequency to log at

  • mode

    (LoggingModeType, default: DEFAULT_LOGGING_MODE ) –

    The logging mode to use, either "on_change" or "exact", "on_change" will log when the model has been updated since the last log, "exact" will log at the given frequency regardless of model updates

  • frequency_type

    (FrequencyType, default: DEFAULT_FREQUENCY_TYPE ) –

    The frequency type to use, either "epoch", "step", or "batch" controls what the frequency manager is tracking, e.g. if the frequency type is "epoch", then the frequency manager will track the number of epochs that have passed since the last log, if the frequency type is "step", then the frequency manager will track the number of optimizer steps

Methods:

  • log_ready –

    Check if the frequency manager is ready to log

  • log_written –

    Sets the last log step to the given step

  • model_updated –

    Sets the last model update to the given step

Attributes:

Source code in llmcompressor/metrics/utils/frequency_manager.py 
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
def __init__(
    self,
    log_frequency: LogStepType = None,
    mode: LoggingModeType = DEFAULT_LOGGING_MODE,
    frequency_type: FrequencyType = DEFAULT_FREQUENCY_TYPE,
):
    # sets self._logging_mode and self._check_model_update
    self._logging_mode = self._set_logging_mode(mode=mode)

    # sets self._frequency_type and self._valid_python_types
    self.frequency_type = self._set_frequency_type(frequency_type=frequency_type)

    self._validate_log_frequency(log_frequency=log_frequency)
    self._log_frequency = log_frequency

    self.last_log_step: LogStepType = None
    self.last_model_update_step: LogStepType = None

is_epoch_frequency_manager property 

is_epoch_frequency_manager: bool

Returns:

  • bool –

    True if the frequency manager is tracking epochs, False otherwise

is_optim_frequency_manager property 

is_optim_frequency_manager: bool

Returns:

  • bool –

    True if the frequency manager is tracking optimizer steps, False otherwise

log_frequency property writable 

log_frequency: LogStepType

Returns:

  • LogStepType –

    The log frequency

log_ready 

log_ready(
    current_log_step: LogStepType,
    check_model_update: bool = False,
)

Check if the frequency manager is ready to log Conditions for readiness: - log frequency is not None - current log step is None - current log step greater than or equal to the last log step plus the log frequency - if check_model_update is True, or self._check_model_update is True, then the last model update step must be greater than or equal to the last log step, and the current log step must be greater than or equal to the last model update step plus the log frequency

Parameters:

  • current_log_step

    (LogStepType) –

    The current log step

  • check_model_update

    (bool, default: False ) –

    If True, will check if the model has been updated since the last log step and if _log_frequency steps have passed since the last model update; Defaults to False.

Returns:

  • –

    True if the frequency manager is ready to log, False otherwise

Source code in llmcompressor/metrics/utils/frequency_manager.py 
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
def log_ready(
    self,
    current_log_step: LogStepType,
    check_model_update: bool = False,
):
    """
    Check if the frequency manager is ready to log
    Conditions for readiness:
        - log frequency is not None
        - current log step is None
        - current log step greater than or equal to the last log step
            plus the log frequency
        - if check_model_update is True, or self._check_model_update is True,
            then the last model update step must be greater than or equal
            to the last log step, and the current log step must be greater
            than or equal to the last model update step plus the log frequency

    :param current_log_step: The current log step
    :param check_model_update: If True, will check if the model has been updated
        since the last log step and if _log_frequency steps have passed since the
        last model update; Defaults to False.
    :return: True if the frequency manager is ready to log,
        False otherwise
    """
    # check_model_update is used to override self._check_model_update
    # e.g. if check_model_update is True, then the model update check
    # will be performed even if self._check_model_update is False

    check_model_update = check_model_update or self._check_model_update

    return log_ready(
        current_log_step=current_log_step,
        last_log_step=self.last_log_step,
        log_frequency=self.log_frequency,
        last_model_update_step=self.last_model_update_step,
        check_model_update=check_model_update,
    )

log_written 

log_written(step: LogStepType = None) -> None

Sets the last log step to the given step

:post-cond: The last log step is set to the given step

Parameters:

  • step

    (LogStepType, default: None ) –

    The step to set the last log step to

Source code in llmcompressor/metrics/utils/frequency_manager.py 
115
116
117
118
119
120
121
122
123
def log_written(self, step: LogStepType = None) -> None:
    """
    Sets the last log step to the given step

    :param step: The step to set the last log step to
    :post-cond: The last log step is set to the given step
    """
    self._validate_log_step(log_step=step)
    self.last_log_step = step

model_updated 

model_updated(step: LogStepType = None) -> None

Sets the last model update to the given step

:post-cond: The last model update step is set to the given step

Parameters:

  • step

    (LogStepType, default: None ) –

    The step to set the last model update to

Source code in llmcompressor/metrics/utils/frequency_manager.py 
105
106
107
108
109
110
111
112
113
def model_updated(self, step: LogStepType = None) -> None:
    """
    Sets the last model update to the given step

    :param step: The step to set the last model update to
    :post-cond: The last model update step is set to the given step
    """
    self._validate_log_step(log_step=step)
    self.last_model_update_step = step

log_ready 

log_ready(
    current_log_step: Optional[LogStepType],
    last_log_step: Optional[LogStepType],
    log_frequency: Optional[LogStepType],
    last_model_update_step: Optional[LogStepType] = None,
    check_model_update: bool = False,
)

Check if we are ready to log again based on the given parameters (Stateless version of FrequencyManager().log_ready)

Conditions for readiness: - log frequency is not None - current log step is None - current log step greater than or equal to the last log step plus the log frequency - if check_model_update is True, then the last model update step must be greater than or equal to the last log step, and the current log step must be greater than or equal to the last model update step plus the log frequency

Parameters:

  • current_log_step

    (Optional[LogStepType]) –

    The current log step

  • last_log_step

    (Optional[LogStepType]) –

    The last step at which logging occurred

  • log_frequency

    (Optional[LogStepType]) –

    The frequency to log at

  • last_model_update_step

    (Optional[LogStepType], default: None ) –

    The last step at which the model was updated

  • check_model_update

    (bool, default: False ) –

    If True, will check if the model has been updated since the last log step and if log_frequency steps have passed since the last model update; Defaults to False.

Returns:

  • –

    True if logging cadence has been reached again False otherwise

Source code in llmcompressor/metrics/utils/frequency_manager.py 
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
def log_ready(
    current_log_step: Optional[LogStepType],
    last_log_step: Optional[LogStepType],
    log_frequency: Optional[LogStepType],
    last_model_update_step: Optional[LogStepType] = None,
    check_model_update: bool = False,
):
    """
    Check if we are ready to log again based on the given parameters
    (Stateless version of FrequencyManager().log_ready)

    Conditions for readiness:
        - log frequency is not None
        - current log step is None
        - current log step greater than or equal to the last log step
            plus the log frequency
        - if check_model_update is True, then the last model update step
            must be greater than or equal to the last log step, and the
            current log step must be greater than or equal to the
            last model update step plus the log frequency

    :param current_log_step: The current log step
    :param last_log_step: The last step at which logging occurred
    :param log_frequency: The frequency to log at
    :param last_model_update_step: The last step at which the model was updated
    :param check_model_update: If True, will check if the model has been updated
        since the last log step and if log_frequency steps have passed since the
        last model update; Defaults to False.
    :return: True if logging cadence has been reached again False otherwise
    """
    # format is used to avoid floating point errors
    # e.g. 0.1 + 0.2 != 0.3
    # format(0.1 + 0.2, ".4f") == format(0.3, ".4f")

    cadence_reached: bool = log_frequency is not None and (
        current_log_step is None
        or last_log_step is None
        or current_log_step >= float(format(last_log_step + log_frequency, ".4f"))
    )

    if not cadence_reached or not check_model_update:
        # early return if cadence not reached or,
        # model update check not requested
        return cadence_reached

    model_updated_since_last_log: bool = (
        last_model_update_step is None
        or last_log_step is None
        or current_log_step is None
        or (
            last_model_update_step >= last_log_step
            and current_log_step
            >= float(format(log_frequency + last_model_update_step, ".4f"))
        )
    )

    return cadence_reached and model_updated_since_last_log