llmcompressor.core.state
Module for managing LLM Compressor state.
Provides classes for holding and updating the state information related to data, hardware, and model compression.
Classes:
-
Data
–A dataclass to hold different data sets for training, validation,
-
Hardware
–A dataclass to hold information about the hardware being used.
-
ModifiedState
–A dataclass to represent a modified model, optimizer, and loss.
-
State
–State class holds information about the current compression state.
Data dataclass
Data(
train: Optional[Any] = None,
val: Optional[Any] = None,
test: Optional[Any] = None,
calib: Optional[Any] = None,
)
A dataclass to hold different data sets for training, validation, testing, and/or calibration. Each data set is a ModifiableData instance.
Parameters:
-
train
Optional[Any]
, default:None
) –The training data set
-
val
Optional[Any]
, default:None
) –The validation data set
-
test
Optional[Any]
, default:None
) –The testing data set
-
calib
Optional[Any]
, default:None
) –The calibration data set
Hardware dataclass
Hardware(
device: Optional[str] = None,
devices: Optional[List[str]] = None,
rank: Optional[int] = None,
world_size: Optional[int] = None,
local_rank: Optional[int] = None,
local_world_size: Optional[int] = None,
distributed: Optional[bool] = None,
distributed_strategy: Optional[str] = None,
)
A dataclass to hold information about the hardware being used.
Parameters:
-
device
Optional[str]
, default:None
) –The current device being used for training
-
devices
Optional[List[str]]
, default:None
) –List of all devices to be used for training
-
rank
Optional[int]
, default:None
) –The rank of the current device
-
world_size
Optional[int]
, default:None
) –The total number of devices being used
-
local_rank
Optional[int]
, default:None
) –The local rank of the current device
-
local_world_size
Optional[int]
, default:None
) –The total number of devices being used on the local machine
-
distributed
Optional[bool]
, default:None
) –Whether or not distributed training is being used
-
distributed_strategy
Optional[str]
, default:None
) –The distributed strategy being used
ModifiedState dataclass
A dataclass to represent a modified model, optimizer, and loss.
Parameters:
-
model
Optional[Any]
) –The modified model
-
optimizer
Optional[Any]
) –The modified optimizer
-
loss
Optional[Any]
) –The modified loss
-
modifier_data
Optional[List[Dict[str, Any]]]
) –The modifier data used to modify the model, optimizer, and loss
Initialize the ModifiedState with the given parameters.
Parameters:
-
model
Any
) –The modified model
-
optimizer
Any
) –The modified optimizer
-
loss
Any
) –The modified loss
-
modifier_data
List[Dict[str, Any]]
) –The modifier data used to modify the model, optimizer, and loss
Source code in llmcompressor/core/state.py
State dataclass
State(
model: Any = None,
teacher_model: Any = None,
optimizer: Any = None,
optim_wrapped: bool = None,
loss: Any = None,
batch_data: Any = None,
data: Data = Data(),
hardware: Hardware = Hardware(),
loggers: Optional[LoggerManager] = None,
model_log_cadence: Optional[float] = None,
_last_log_step: Union[float, int, None] = None,
)
State class holds information about the current compression state.
Parameters:
-
model
Any
, default:None
) –The model being used for compression
-
teacher_model
Any
, default:None
) –The teacher model being used for compression
-
optimizer
Any
, default:None
) –The optimizer being used for training
-
optim_wrapped
bool
, default:None
) –Whether or not the optimizer has been wrapped
-
loss
Any
, default:None
) –The loss function being used for training
-
batch_data
Any
, default:None
) –The current batch of data being used for compression
-
data
Data
, default:Data()
) –The data sets being used for training, validation, testing, and/or calibration, wrapped in a Data instance
-
hardware
Hardware
, default:Hardware()
) –Hardware instance holding info about the target hardware being used
-
loggers
Optional[LoggerManager]
, default:None
) –LoggerManager instance holding all the loggers to log
-
model_log_cadence
Optional[float]
, default:None
) –The cadence to log model information w.r.t epochs. If 1, logs every epoch. If 2, logs every other epoch, etc. Default is 1.
Methods:
-
update
–Update the state with the given parameters.
Attributes:
-
compression_ready
(bool
) –Check if the model and optimizer are set for compression.
compression_ready property
Check if the model and optimizer are set for compression.
Returns:
-
bool
–True if model and optimizer are set, False otherwise
update
update(
model: Any = None,
teacher_model: Any = None,
optimizer: Any = None,
attach_optim_callbacks: bool = True,
train_data: Any = None,
val_data: Any = None,
test_data: Any = None,
calib_data: Any = None,
copy_data: bool = True,
start: float = None,
steps_per_epoch: int = None,
batches_per_step: int = None,
loggers: Union[
None, LoggerManager, List[BaseLogger]
] = None,
model_log_cadence: Optional[float] = None,
**kwargs
) -> Dict
Update the state with the given parameters.
Parameters:
-
model
Any
, default:None
) –The model to update the state with
-
teacher_model
Any
, default:None
) –The teacher model to update the state with
-
optimizer
Any
, default:None
) –The optimizer to update the state with
-
attach_optim_callbacks
bool
, default:True
) –Whether or not to attach optimizer callbacks
-
train_data
Any
, default:None
) –The training data to update the state with
-
val_data
Any
, default:None
) –The validation data to update the state with
-
test_data
Any
, default:None
) –The testing data to update the state with
-
calib_data
Any
, default:None
) –The calibration data to update the state with
-
copy_data
bool
, default:True
) –Whether or not to copy the data
-
start
float
, default:None
) –The start index to update the state with
-
steps_per_epoch
int
, default:None
) –The steps per epoch to update the state with
-
batches_per_step
int
, default:None
) –The batches per step to update the state with
-
loggers
Union[None, LoggerManager, List[BaseLogger]]
, default:None
) –The metrics manager to setup logging important info and milestones to, also accepts a list of BaseLogger(s)
-
model_log_cadence
Optional[float]
, default:None
) –The cadence to log model information w.r.t epochs. If 1, logs every epoch. If 2, logs every other epoch, etc. Default is 1.
-
kwargs
Additional keyword arguments to update the state with
Returns:
-
Dict
–The updated state as a dictionary
Source code in llmcompressor/core/state.py
127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 |
|