bi=RddlmZGddeZGddeZGddeZy) )Optimizer1Statec2eZdZ dfd ZxZS)SGDc \|dk(r tdt | d||||fd|||| | | y)a Base SGD optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr (`float`): The learning rate. momentum (`float`, defaults to 0): The momentum value speeds up the optimizer by taking bigger steps. dampening (`float`, defaults to 0): The dampening value reduces the momentum of the optimizer. weight_decay (`float`, defaults to 0.0): The weight decay value for the optimizer. nesterov (`bool`, defaults to `False`): Whether to use Nesterov momentum. optim_bits (`int`, defaults to 32): The number of bits of the optimizer state. args (`object`, defaults to `None`): An object with additional arguments. min_8bit_size (`int`, defaults to 4096): The minimum number of elements of the parameter tensors for 8-bit optimization. percentile_clipping (`int`, defaults to 100): Adapts clipping threshold automatically by tracking the last 100 gradient norms and clipping the gradient at a certain percentile to improve stability. block_wise (`bool`, defaults to `True`): Whether to independently quantize each block of tensors to reduce outlier effects and improve stability. r&SGD without momentum is not supported!momentumNNotImplementedErrorsuper__init__) selfparamslrr dampening weight_decaynesterov optim_bitsargs min_8bit_sizepercentile_clipping block_wise __class__s Q/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/bitsandbytes/optim/sgd.pyr z SGD.__init__ sLR q=%&NO O     y !        ) rrrF NdT__name__ __module__ __qualname__r __classcell__rs@rrrs*  7 7 rrc0eZdZ dfd ZxZS)SGD8bitc \|dk(r tdt | d||||fd|d||| | y)a+ 8-bit SGD optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr (`float`): The learning rate. momentum (`float`, defaults to 0): The momentum value speeds up the optimizer by taking bigger steps. dampening (`float`, defaults to 0): The dampening value reduces the momentum of the optimizer. weight_decay (`float`, defaults to 0.0): The weight decay value for the optimizer. nesterov (`bool`, defaults to `False`): Whether to use Nesterov momentum. args (`object`, defaults to `None`): An object with additional arguments. min_8bit_size (`int`, defaults to 4096): The minimum number of elements of the parameter tensors for 8-bit optimization. percentile_clipping (`int`, defaults to 100): Adapts clipping threshold automatically by tracking the last 100 gradient norms and clipping the gradient at a certain percentile to improve stability. block_wise (`bool`, defaults to `True`): Whether to independently quantize each block of tensors to reduce outlier effects and improve stability. rrrr Nr rrrrrrrrrrrrs rr zSGD8bit.__init__DsLL q=%&NO O     y !       rrrrFNrrTrr$s@rr&r&C'  4 4 rr&c0eZdZ dfd ZxZS)SGD32bitc \|dk(r tdt | d||||fd|d||| | y)a, 32-bit SGD optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr (`float`): The learning rate. momentum (`float`, defaults to 0): The momentum value speeds up the optimizer by taking bigger steps. dampening (`float`, defaults to 0): The dampening value reduces the momentum of the optimizer. weight_decay (`float`, defaults to 0.0): The weight decay value for the optimizer. nesterov (`bool`, defaults to `False`): Whether to use Nesterov momentum. args (`object`, defaults to `None`): An object with additional arguments. min_8bit_size (`int`, defaults to 4096): The minimum number of elements of the parameter tensors for 8-bit optimization. percentile_clipping (`int`, defaults to 100): Adapts clipping threshold automatically by tracking the last 100 gradient norms and clipping the gradient at a certain percentile to improve stability. block_wise (`bool`, defaults to `True`): Whether to independently quantize each block of tensors to reduce outlier effects and improve stability. rrrr rNr r)s rr zSGD32bit.__init__|sLL q=%&NO O     y !        rr*rr$s@rr-r-{r+rr-N)bitsandbytes.optim.optimizerrrr&r-rrr1s0 98 /8 v5 o5 p5 5 r