biPddlmZmZddlmZmZddlmZeGddeZy)) dataclassfield)AnyOptional)TrainingArgumentsceZdZUdZej dgzZedddiZee d<eddd iZ ee d <ed dd iZ e e e d <edddiZe ee d<edddiZe ee d<ed ddiZe ee d<edddiZee d<edddiZe e d<edddiZee d<ed ddiZe ee d <ed!d"d!d#gd$Zee d%<ed&dd'iZe e d(<ed dd)iZe e e d*<ed dd+iZe eeefe d<ed dd,iZe ee d-<fd.ZxZS)/ ORPOConfigu8 Configuration class for the [`ORPOTrainer`]. This class includes only the parameters that are specific to ORPO training. For a full list of training arguments, please refer to the [`~transformers.TrainingArguments`] documentation. Note that default values in this class may differ from those in [`~transformers.TrainingArguments`]. Using [`~transformers.HfArgumentParser`] we can turn this class into [argparse](https://docs.python.org/3/library/argparse#module-argparse) arguments that can be specified on the command line. Parameters: max_length (`int` or `None`, *optional*, defaults to `1024`): Maximum length of the sequences (prompt + completion) in the batch. This argument is required if you want to use the default data collator. max_prompt_length (`int` or `None`, *optional*, defaults to `512`): Maximum length of the prompt. This argument is required if you want to use the default data collator. max_completion_length (`int` or `None`, *optional*, defaults to `None`): Maximum length of the completion. This argument is required if you want to use the default data collator and your model is an encoder-decoder. beta (`float`, *optional*, defaults to `0.1`): Parameter controlling the relative ratio loss weight in the ORPO loss. In the [paper](https://huggingface.co/papers/2403.07691), it is denoted by λ. In the [code](https://github.com/xfactlab/orpo), it is denoted by `alpha`. disable_dropout (`bool`, *optional*, defaults to `True`): Whether to disable dropout in the model. label_pad_token_id (`int`, *optional*, defaults to `-100`): Label pad token id. This argument is required if you want to use the default data collator. padding_value (`int` or `None`, *optional*, defaults to `None`): Padding value to use. If `None`, the padding value of the tokenizer is used. truncation_mode (`str`, *optional*, defaults to `"keep_end"`): Truncation mode to use when the prompt is too long. Possible values are `"keep_end"` or `"keep_start"`. This argument is required if you want to use the default data collator. generate_during_eval (`bool`, *optional*, defaults to `False`): If `True`, generates and logs completions from the model to W&B or Comet during evaluation. is_encoder_decoder (`bool` or `None`, *optional*, defaults to `None`): When using the `model_init` argument (callable) to instantiate the model instead of the `model` argument, you need to specify if the model returned by the callable is an encoder-decoder model. model_init_kwargs (`dict[str, Any]` or `None`, *optional*, defaults to `None`): Keyword arguments to pass to `AutoModelForCausalLM.from_pretrained` when instantiating the model from a string. dataset_num_proc (`int` or `None`, *optional*, defaults to `None`): Number of processes to use for processing the dataset. model_init_kwargsgư>helpz$The initial learning rate for AdamW.)defaultmetadata learning_rate zLog every X updates steps. Should be an integer or a float in range `[0,1)`. If smaller than 1, will be interpreted as ratio of total training steps. logging_stepsNzWhether to use bf16 (mixed) precision instead of 32-bit. Requires Ampere or higher NVIDIA architecture or Intel XPU or using CPU (use_cpu) or Ascend NPU. If not set, it defaults to `True` if `fp16` is not set.bf16izCMaximum length of the sequences (prompt + completion) in the batch. max_lengthizMaximum length of the prompt. This argument is required if you want to use the default data collator and your model is an encoder-decoder.max_prompt_lengthzMaximum length of the completion. This argument is required if you want to use the default data collator and your model is an encoder-decoder.max_completion_lengthg?uiParameter controlling the relative ratio loss weight in the ORPO loss. In the paper, it is denoted by λ.betaTz(Whether to disable dropout in the model.disable_dropoutiz[Label pad token id. This argument is required if you want to use the default data collator.label_pad_token_idzLPadding value to use. If `None`, the padding value of the tokenizer is used. padding_valuekeep_endz3Truncation mode to use when the prompt is too long. keep_start)r choicestruncation_modeFzRIf `True`, generates and logs completions from the model to W&B during evaluation.generate_during_evalzWhen using the `model_init` argument (callable) to instantiate the model instead of the `model` argument, you need to specify if the model returned by the callable is an encoder-decoder model.is_encoder_decoderzoKeyword arguments to pass to `AutoModelForCausalLM.from_pretrained` when instantiating the model from a string.z6Number of processes to use for processing the dataset.dataset_num_proccv|j |j n |j|_t| y)N)rfp16super __post_init__)self __class__s R/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/trl/trainer/orpo_config.pyr#zORPOConfig.__post_init__s*'+yy'8Odii  ) __name__ __module__ __qualname____doc__r_VALID_DICT_FIELDSrrfloat__annotations__rrrboolrintrrrrrrrstrrrr dictrrr# __classcell__)r%s@r&r r sR+Z+==AT@UU!@AM5! D M5! ! D(4.!&_`!J (- = (x},1 = ,8C=  D%"DEOT$ q  $)hi$M8C=!I"L1 OS"'no"$*/ o *38  3xS#X/',RS'hsm   r'r N) dataclassesrrtypingrr transformersrr r'r&r8s/) * M "M  M r'