Skip to content

llmcompressor

LLM Compressor is a library for compressing large language models utilizing the latest techniques and research in the field for both training aware and post-training techniques.

The library is designed to be flexible and easy to use on top of PyTorch and HuggingFace Transformers, allowing for quick experimentation.

Modules:

  • args –

    Arguments package for LLM Compressor.

  • core –

    Provides the core compression framework for LLM Compressor.

  • datasets –

    Provides dataset utilities for model calibration and processing.

  • entrypoints –

    Provides entry points for model compression workflows.

  • logger –

    Provides a flexible logging configuration for LLM Compressor.

  • metrics –

    Metrics logging and monitoring framework for compression workflows.

  • modeling –

    Model preparation and fusion utilities for compression workflows.

  • modifiers –

    Compression modifiers for applying various optimization techniques.

  • observers –

    Framework for monitoring and analyzing model behavior during compression.

  • pipelines –

    Compression pipelines for orchestrating different compression strategies.

  • pytorch –

    PyTorch-specific utilities and tools for model compression workflows.

  • recipe –

    Recipe system for defining and managing compression workflows.

  • sentinel –

    Sentinel value implementation for LLM compression workflows.

  • transformers –

    Tools for integrating LLM Compressor with transformers training flows.

  • typing –

    Defines type aliases for the llm-compressor library.

  • utils –

    General utility functions used throughout LLM Compressor.

Functions:

configure_logger 

configure_logger(
    config: Optional[LoggerConfig] = None,
) -> None

Configure the logger for LLM Compressor.

This function sets up the console and file logging as per the specified or default parameters.

Note: Environment variables take precedence over function parameters.

Parameters:

  • config

    (Optional[LoggerConfig], default: None ) –

    The configuration for the logger to use.

Source code in llmcompressor/logger.py 
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
def configure_logger(config: Optional[LoggerConfig] = None) -> None:
    """
    Configure the logger for LLM Compressor.

    This function sets up the console and file logging
    as per the specified or default parameters.

    **Note**: Environment variables take precedence over function parameters.

    :param config: The configuration for the logger to use.
    :type config: LoggerConfig
    """
    logger_config = config or LoggerConfig()

    # env vars get priority
    if (disabled := os.getenv("LLM_COMPRESSOR_LOG_DISABLED")) is not None:
        logger_config.disabled = disabled.lower() == "true"
    if (clear_loggers := os.getenv("LLM_COMPRESSOR_CLEAR_LOGGERS")) is not None:
        logger_config.clear_loggers = clear_loggers.lower() == "true"
    if (console_log_level := os.getenv("LLM_COMPRESSOR_LOG_LEVEL")) is not None:
        logger_config.console_log_level = console_log_level.upper()
    if (log_file := os.getenv("LLM_COMPRESSOR_LOG_FILE")) is not None:
        logger_config.log_file = log_file
    if (log_file_level := os.getenv("LLM_COMPRESSOR_LOG_FILE_LEVEL")) is not None:
        logger_config.log_file_level = log_file_level.upper()

    if logger_config.disabled:
        logger.disable("llmcompressor")
        return

    logger.enable("llmcompressor")

    if logger_config.clear_loggers:
        logger.remove()

    if logger_config.console_log_level:
        # log as a human readable string with the time, function, level, and message
        logger.add(
            sys.stdout,
            level=logger_config.console_log_level.upper(),
            format="{time} | {function} | {level} - {message}",
            filter=support_log_once,
        )

    if logger_config.log_file or logger_config.log_file_level:
        log_file = logger_config.log_file or "llmcompressor.log"
        log_file_level = logger_config.log_file_level or "INFO"
        # log as json to the file for easier parsing
        logger.add(
            log_file,
            level=log_file_level.upper(),
            serialize=True,
            filter=support_log_once,
        )

    if logger_config.metrics_disabled or "METRIC" in logger._core.levels.keys():
        return

    # initialize metric logger on loguru
    logger.level("METRIC", no=38, color="<yellow>", icon="📈")