Skip to content

llmcompressor.pipelines

Compression pipelines for orchestrating different compression strategies.

Provides various compression pipelines including basic, sequential, independent, layer-sequential, and data-free approaches. Each pipeline coordinates different compression techniques and workflows for optimal model optimization based on specific requirements and constraints.

Modules:

Classes:

BasicPipeline

Bases: CalibrationPipeline

CalibrationPipeline

Bases: ABC, RegistryMixin

Methods:

  • from_modifiers –

    Infer which calibration pipeline to use based on the available modifiers and

from_modifiers classmethod 

from_modifiers(
    modifiers: List[Modifier], user: Optional[str] = None
) -> CalibrationPipeline

Infer which calibration pipeline to use based on the available modifiers and any user specifications

Parameters:

  • modifiers

    (List[Modifier]) –

    modifiers to apply to model

  • user

    (Optional[str], default: None ) –

    pipeline name passed by user

Returns:

  • CalibrationPipeline –

    CalibrationPipeline instance to be called with data (if not datafree)

Source code in llmcompressor/pipelines/registry.py 
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
@classmethod
def from_modifiers(
    cls, modifiers: List[Modifier], user: Optional[str] = None
) -> "CalibrationPipeline":
    """
    Infer which calibration pipeline to use based on the available modifiers and
    any user specifications

    :param modifiers: modifiers to apply to model
    :param user: pipeline name passed by user
    :return: CalibrationPipeline instance to be called with data (if not datafree)
    """
    user = standardize_lookup_name(user) if user else None
    inferred = standardize_lookup_name(cls._infer_pipeline(modifiers))
    independent = standardize_lookup_name("independent")

    if user == independent:
        inferred = independent

    if user is not None and user != inferred:
        logger.warning(
            f"Calibration pipeline is set to `{user}`, but it is recommended to "
            f"use `{inferred}`"
        )

    pipeline = user or inferred
    return cls.load_from_registry(pipeline)

DataFreePipeline

Bases: CalibrationPipeline

IndependentPipeline

Bases: CalibrationPipeline

LayerSequentialPipeline

Bases: CalibrationPipeline

SequentialPipeline

Bases: CalibrationPipeline