Skip to content

llmcompressor.transformers.utils

Utilities for applying sparsification algorithms to Hugging Face transformers flows

Modules:

  • helpers –

    Helper variables and functions for integrating LLM Compressor with

  • preprocessing_functions –

    Dataset preprocessing functions for text generation tasks.

Functions:

is_model_ct_quantized_from_path 

is_model_ct_quantized_from_path(path: str) -> bool

Determine if model from path is quantized based on the config

Parameters:

  • path

    (str) –

    path to the model or HF stub

Returns:

  • bool –

    True if config contains quantization_config from the given path

Source code in llmcompressor/transformers/utils/helpers.py 
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
def is_model_ct_quantized_from_path(path: str) -> bool:
    """
    Determine if model from path is quantized based
    on the config

    :param path: path to the model or HF stub
    :return: True if config contains quantization_config from the given path

    """
    config = AutoConfig.from_pretrained(path)
    if config is not None:
        if (
            hasattr(config, "quantization_config")
            and config.quantization_config["quant_method"] == "compressed-tensors"
        ):
            return True
    return False