biU\ddlmZmZddlmZmZddlmZddlm Z eGdde Z y) ) dataclassfield)AnyOptional)TrainingArguments) SFTConfigcjeZdZUdZej dgzZedddiZee d<eddd iZ ee d <eddd iZ ee d <ed ddiZ e e d<edddiZeee d<edddiZeeeefe d<edddiZee d<edddiZee d<fdZxZS) GKDConfiga8 Configuration class for [`GKDTrainer`]. This class includes only the parameters that are specific to GKD training. For a full list of training arguments, please refer to the [`~transformers.TrainingArguments`] and [`SFTConfig`] documentation. Args: temperature (`float`, *optional*, defaults to `0.9`): Temperature for sampling. The higher the temperature, the more random the completions. lmbda (`float`, *optional*, defaults to `0.5`): Lambda parameter that controls the student data fraction (i.e., the proportion of on-policy student-generated outputs). beta (`float`, *optional*, defaults to `0.5`): Interpolation coefficient between `0.0` and `1.0` of the Generalized Jensen-Shannon Divergence loss. When beta is `0.0`, the loss is the KL divergence. When beta is `1.0`, the loss is the Inverse KL Divergence. max_new_tokens (`int`, *optional*, defaults to `128`): Maximum number of tokens to generate per completion. teacher_model_name_or_path (`str` or `None`, *optional*, defaults to `None`): Model name or path of the teacher model. If `None`, the teacher model will be the same as the model being trained. teacher_model_init_kwargs (`dict[str, Any]]` or `None`, *optional*, defaults to `None`): Keyword arguments to pass to `AutoModelForCausalLM.from_pretrained` when instantiating the teacher model from a string. disable_dropout (`bool`, *optional*, defaults to `True`): Whether to disable dropout in the model. seq_kd (`bool`, *optional*, defaults to `False`): Seq_kd parameter that controls whether to perform Sequence-Level KD (can be viewed as supervised FT on teacher-generated output). teacher_model_init_kwargsg?helpzVTemperature for sampling. The higher the temperature, the more random the completions.)defaultmetadata temperatureg?zwLambda parameter that controls the student data fraction (i.e., the proportion of on-policy student-generated outputs).lmbdazInterpolation coefficient between `0.0` and `1.0` of the Generalized Jensen-Shannon Divergence loss. When beta is `0.0`, the loss is the KL divergence. When beta is `1.0`, the loss is the Inverse KL Divergence.betaz4Maximum number of tokens to generate per completion.max_new_tokensNzrModel name or path of the teacher model. If `None`, the teacher model will be the same as the model being trained.teacher_model_name_or_pathzwKeyword arguments to pass to `AutoModelForCausalLM.from_pretrained` when instantiating the teacher model from a string.Tz'Whether to disable dropouts in `model`.disable_dropoutFzSeq_kd parameter that controls whether to perform Sequence-Level KD (can be viewed as supervised FT on teacher-generated output).seq_kdct||jdks|jdkDr td|jdks|jdkDr tdy)Ngg?z&lmbda must be in the range [0.0, 1.0].z%beta must be in the range [0.0, 1.0].)super __post_init__r ValueErrorr)self __class__s Q/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/trl/trainer/gkd_config.pyrzGKDConfig.__post_init__jsW  :: tzzC/EF F 99s?dii#oDE E.)__name__ __module__ __qualname____doc__r_VALID_DICT_FIELDSrrfloat__annotations__rrrintrrstrr dictrrboolrr __classcell__)rs@rr r sI<+==A\@]]rsK * E5  D% PQNC16 # 1 ;@ + ;xS#X7"CDOT / FDFFrr N) dataclassesrrtypingrr transformersr sft_configr r rrr1s4) *! XF XF XFr