Skip to content

llmcompressor.core.events

LLM Compressor Core Events Package

This package provides the core components and lifecycle management for events used in the LLM Compressor framework. It includes definitions for various event types and lifecycles that are critical for managing the state and execution flow of the model compression and training processes.

Modules:

  • event –

    Module for defining and managing events in the LLM Compressor.

Classes:

  • Event –

    A class for defining an event that can be triggered during sparsification.

  • EventType –

    An Enum for defining the different types of events that can be triggered

Event dataclass 

Event(
    type_: Optional[EventType] = None,
    steps_per_epoch: Optional[int] = None,
    batches_per_step: Optional[int] = None,
    invocations_per_step: int = 1,
    global_step: int = 0,
    global_batch: int = 0,
)

A class for defining an event that can be triggered during sparsification.

Parameters:

  • type_

    (Optional[EventType], default: None ) –

    The type of event.

  • steps_per_epoch

    (Optional[int], default: None ) –

    The number of steps per epoch.

  • batches_per_step

    (Optional[int], default: None ) –

    The number of batches per step where step is an optimizer step invocation. For most pathways, these are the same. See the invocations_per_step parameter for more details when they are not.

  • invocations_per_step

    (int, default: 1 ) –

    The number of invocations of the step wrapper before optimizer.step was called. Generally can be left as 1 (default). For older amp pathways, this is the number of times the scaler wrapper was invoked before the wrapped optimizer step function was called to handle accumulation in fp16.

  • global_step

    (int, default: 0 ) –

    The current global step.

  • global_batch

    (int, default: 0 ) –

    The current global batch.

Methods:

  • new_instance –

    Creates a new instance of the event with the provided keyword arguments.

  • should_update –

    Determines if the event should trigger an update.

Attributes:

  • current_index (float) –

    Calculates the current index of the event.

  • epoch (int) –

    Calculates the current epoch.

  • epoch_based (bool) –

    Determines if the event is based on epochs.

  • epoch_batch (int) –

    Calculates the current batch within the current epoch.

  • epoch_full (float) –

    Calculates the current epoch with the fraction of the current step.

  • epoch_step (int) –

    Calculates the current step within the current epoch.

current_index property writable 

current_index: float

Calculates the current index of the event.

Returns:

  • float –

    The current index of the event, which is either the global step or the epoch with the fraction of the current step.

Raises:

  • ValueError –

    if the event is not epoch based or if the steps per epoch are too many.

epoch property 

epoch: int

Calculates the current epoch.

Returns:

  • int –

    The current epoch.

Raises:

  • ValueError –

    if the event is not epoch based.

epoch_based property 

epoch_based: bool

Determines if the event is based on epochs.

Returns:

  • bool –

    True if the event is based on epochs, False otherwise.

epoch_batch property 

epoch_batch: int

Calculates the current batch within the current epoch.

Returns:

  • int –

    The current batch within the current epoch.

Raises:

  • ValueError –

    if the event is not epoch based.

epoch_full property 

epoch_full: float

Calculates the current epoch with the fraction of the current step.

Returns:

  • float –

    The current epoch with the fraction of the current step.

Raises:

  • ValueError –

    if the event is not epoch based.

epoch_step property 

epoch_step: int

Calculates the current step within the current epoch.

Returns:

  • int –

    The current step within the current epoch.

Raises:

  • ValueError –

    if the event is not epoch based.

new_instance 

new_instance(**kwargs) -> Event

Creates a new instance of the event with the provided keyword arguments.

Parameters:

  • kwargs

    –

    Keyword arguments to set in the new instance.

Returns:

  • Event –

    A new instance of the event with the provided kwargs.

Source code in llmcompressor/core/events/event.py 
239
240
241
242
243
244
245
246
247
248
249
250
251
def new_instance(self, **kwargs) -> "Event":
    """
    Creates a new instance of the event with the provided keyword arguments.

    :param kwargs: Keyword arguments to set in the new instance.
    :return: A new instance of the event with the provided kwargs.
    :rtype: Event
    """
    logger.debug("Creating new instance of event with kwargs: {}", kwargs)
    instance = deepcopy(self)
    for key, value in kwargs.items():
        setattr(instance, key, value)
    return instance

should_update 

should_update(
    start: Optional[float],
    end: Optional[float],
    update: Optional[float],
) -> bool

Determines if the event should trigger an update.

Parameters:

  • start

    (Optional[float]) –

    The start index to check against, set to None to ignore start.

  • end

    (Optional[float]) –

    The end index to check against, set to None to ignore end.

  • update

    (Optional[float]) –

    The update interval, set to None or 0.0 to always update, otherwise must be greater than 0.0, defaults to None.

Returns:

  • bool –

    True if the event should trigger an update, False otherwise.

Source code in llmcompressor/core/events/event.py 
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
def should_update(
    self, start: Optional[float], end: Optional[float], update: Optional[float]
) -> bool:
    """
    Determines if the event should trigger an update.

    :param start: The start index to check against, set to None to ignore start.
    :type start: Optional[float]
    :param end: The end index to check against, set to None to ignore end.
    :type end: Optional[float]
    :param update: The update interval, set to None or 0.0 to always update,
        otherwise must be greater than 0.0, defaults to None.
    :type update: Optional[float]
    :return: True if the event should trigger an update, False otherwise.
    :rtype: bool
    """
    current = self.current_index
    logger.debug(
        "Checking if event should update: "
        "current_index={}, start={}, end={}, update={}",
        current,
        start,
        end,
        update,
    )
    if start is not None and current < start:
        return False
    if end is not None and current > end:
        return False
    return update is None or update <= 0.0 or current % update < 1e-10

EventType

Bases: Enum

An Enum for defining the different types of events that can be triggered during model compression lifecycles. The purpose of each EventType is to trigger the corresponding modifier callback during training or post training pipelines.

Parameters:

  • INITIALIZE

    –

    Event type for initialization.

  • FINALIZE

    –

    Event type for finalization.

  • BATCH_START

    –

    Event type for the start of a batch.

  • LOSS_CALCULATED

    –

    Event type for when loss is calculated.

  • BATCH_END

    –

    Event type for the end of a batch.

  • CALIBRATION_EPOCH_START

    –

    Event type for the start of a calibration epoch.

  • SEQUENTIAL_EPOCH_END

    –

    Event type for the end of a layer calibration epoch, specifically used by src/llmcompressor/pipelines/sequential/pipeline.py

  • CALIBRATION_EPOCH_END

    –

    Event type for the end of a calibration epoch.

  • OPTIM_PRE_STEP

    –

    Event type for pre-optimization step.

  • OPTIM_POST_STEP

    –

    Event type for post-optimization step.