llmcompressor.core.state
Module for managing LLM Compressor state.
Provides classes for holding and updating the state information related to data, hardware, and model compression.
Classes:
-
Data–A dataclass to hold different data sets for training, validation,
-
Hardware–A dataclass to hold information about the hardware being used.
-
ModifiedState–A dataclass to represent a modified model, optimizer, and loss.
-
State–State class holds information about the current compression state.
Data dataclass
Data(
train: Optional[Any] = None,
val: Optional[Any] = None,
test: Optional[Any] = None,
calib: Optional[Any] = None,
)
A dataclass to hold different data sets for training, validation, testing, and/or calibration. Each data set is a ModifiableData instance.
Parameters:
-
(trainOptional[Any], default:None) –The training data set
-
(valOptional[Any], default:None) –The validation data set
-
(testOptional[Any], default:None) –The testing data set
-
(calibOptional[Any], default:None) –The calibration data set
Hardware dataclass
Hardware(
device: Optional[str] = None,
devices: Optional[List[str]] = None,
rank: Optional[int] = None,
world_size: Optional[int] = None,
local_rank: Optional[int] = None,
local_world_size: Optional[int] = None,
distributed: Optional[bool] = None,
distributed_strategy: Optional[str] = None,
)
A dataclass to hold information about the hardware being used.
Parameters:
-
(deviceOptional[str], default:None) –The current device being used for training
-
(devicesOptional[List[str]], default:None) –List of all devices to be used for training
-
(rankOptional[int], default:None) –The rank of the current device
-
(world_sizeOptional[int], default:None) –The total number of devices being used
-
(local_rankOptional[int], default:None) –The local rank of the current device
-
(local_world_sizeOptional[int], default:None) –The total number of devices being used on the local machine
-
(distributedOptional[bool], default:None) –Whether or not distributed training is being used
-
(distributed_strategyOptional[str], default:None) –The distributed strategy being used
ModifiedState dataclass
A dataclass to represent a modified model, optimizer, and loss.
Parameters:
-
(modelOptional[Any]) –The modified model
-
(optimizerOptional[Any]) –The modified optimizer
-
(lossOptional[Any]) –The modified loss
-
(modifier_dataOptional[List[Dict[str, Any]]]) –The modifier data used to modify the model, optimizer, and loss
Initialize the ModifiedState with the given parameters.
Parameters:
-
(modelAny) –The modified model
-
(optimizerAny) –The modified optimizer
-
(lossAny) –The modified loss
-
(modifier_dataList[Dict[str, Any]]) –The modifier data used to modify the model, optimizer, and loss
Source code in llmcompressor/core/state.py
State dataclass
State(
model: Any = None,
teacher_model: Any = None,
optimizer: Any = None,
optim_wrapped: bool = None,
loss: Any = None,
batch_data: Any = None,
data: Data = Data(),
hardware: Hardware = Hardware(),
loggers: Optional[LoggerManager] = None,
model_log_cadence: Optional[float] = None,
_last_log_step: Union[float, int, None] = None,
)
State class holds information about the current compression state.
Parameters:
-
(modelAny, default:None) –The model being used for compression
-
(teacher_modelAny, default:None) –The teacher model being used for compression
-
(optimizerAny, default:None) –The optimizer being used for training
-
(optim_wrappedbool, default:None) –Whether or not the optimizer has been wrapped
-
(lossAny, default:None) –The loss function being used for training
-
(batch_dataAny, default:None) –The current batch of data being used for compression
-
(dataData, default:Data()) –The data sets being used for training, validation, testing, and/or calibration, wrapped in a Data instance
-
(hardwareHardware, default:Hardware()) –Hardware instance holding info about the target hardware being used
-
(loggersOptional[LoggerManager], default:None) –LoggerManager instance holding all the loggers to log
-
(model_log_cadenceOptional[float], default:None) –The cadence to log model information w.r.t epochs. If 1, logs every epoch. If 2, logs every other epoch, etc. Default is 1.
Methods:
-
update–Update the state with the given parameters.
Attributes:
-
compression_ready(bool) –Check if the model and optimizer are set for compression.
compression_ready property
Check if the model and optimizer are set for compression.
Returns:
-
bool–True if model and optimizer are set, False otherwise
update
update(
model: Any = None,
teacher_model: Any = None,
optimizer: Any = None,
attach_optim_callbacks: bool = True,
train_data: Any = None,
val_data: Any = None,
test_data: Any = None,
calib_data: Any = None,
copy_data: bool = True,
start: float = None,
steps_per_epoch: int = None,
batches_per_step: int = None,
loggers: Union[
None, LoggerManager, List[BaseLogger]
] = None,
model_log_cadence: Optional[float] = None,
**kwargs,
) -> Dict
Update the state with the given parameters.
Parameters:
-
(modelAny, default:None) –The model to update the state with
-
(teacher_modelAny, default:None) –The teacher model to update the state with
-
(optimizerAny, default:None) –The optimizer to update the state with
-
(attach_optim_callbacksbool, default:True) –Whether or not to attach optimizer callbacks
-
(train_dataAny, default:None) –The training data to update the state with
-
(val_dataAny, default:None) –The validation data to update the state with
-
(test_dataAny, default:None) –The testing data to update the state with
-
(calib_dataAny, default:None) –The calibration data to update the state with
-
(copy_databool, default:True) –Whether or not to copy the data
-
(startfloat, default:None) –The start index to update the state with
-
(steps_per_epochint, default:None) –The steps per epoch to update the state with
-
(batches_per_stepint, default:None) –The batches per step to update the state with
-
(loggersUnion[None, LoggerManager, List[BaseLogger]], default:None) –The metrics manager to setup logging important info and milestones to, also accepts a list of BaseLogger(s)
-
(model_log_cadenceOptional[float], default:None) –The cadence to log model information w.r.t epochs. If 1, logs every epoch. If 2, logs every other epoch, etc. Default is 1.
-
–kwargsAdditional keyword arguments to update the state with
Returns:
-
Dict–The updated state as a dictionary
Source code in llmcompressor/core/state.py
127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 | |