bi'jddlZddlZddlmZmZddlmZmZddlm Z ddl m Z eGddZ y) N) dataclassfield)AnyOptional)is_bitsandbytes_available) flatten_dictc eZdZUdZeej jejdde d ddiZ e e d<ed dd iZe e d <eddd iZee d <eddddgdZee e d<edddiZee d<eeddiZee efe d<eeddiZee efe d<eeddiZee efe d<edddiZe e d<ed dd!iZe e d"<ed#dd$iZee d%<eddd&iZee d'<ed(dd)iZee d*<ed+d,gd-dZe e d.<ed/dd0iZe e d1<ed dd2iZ!e e d3<ed4dd5iZ"ee d6<ed7dd8iZ#e$e d9<ed:dd;iZ%e$e d<<eddd=iZ&ee d><ed?dd@iZ'e e dA<edBddCiZ(e$e dD<edEddFiZ)e$e dG<edHddIiZ*e$e dJ<edKddLiZ+e$e dM<edNddOiZ,e$e dP<edddQiZ-ee dR<ed7ddSiZ.e$e dT<edddUiZ/ee e dV<ed/ddWiZ0e e dX<edYddZiZ1ee d[<ed\dd]iZ2e3eefe d^<ed?dd_iZ4e e d`<daZ5dbZ6y)cAlignPropConfiga Configuration class for the [`AlignPropTrainer`]. Using [`~transformers.HfArgumentParser`] we can turn this class into [argparse](https://docs.python.org/3/library/argparse#module-argparse) arguments that can be specified on the command line. Parameters: exp_name (`str`, *optional*, defaults to `os.path.basename(sys.argv[0])[: -len(".py")]`): Name of this experiment (defaults to the file name without the extension). run_name (`str`, *optional*, defaults to `""`): Name of this run. seed (`int`, *optional*, defaults to `0`): Random seed for reproducibility. log_with (`str` or `None`, *optional*, defaults to `None`): Log with either `"wandb"` or `"tensorboard"`. Check [tracking](https://huggingface.co/docs/accelerate/usage_guides/tracking) for more details. log_image_freq (`int`, *optional*, defaults to `1`): Frequency for logging images. tracker_kwargs (`dict[str, Any]`, *optional*, defaults to `{}`): Keyword arguments for the tracker (e.g., `wandb_project`). accelerator_kwargs (`dict[str, Any]`, *optional*, defaults to `{}`): Keyword arguments for the accelerator. project_kwargs (`dict[str, Any]`, *optional*, defaults to `{}`): Keyword arguments for the accelerator project config (e.g., `logging_dir`). tracker_project_name (`str`, *optional*, defaults to `"trl"`): Name of project to use for tracking. logdir (`str`, *optional*, defaults to `"logs"`): Top-level logging directory for checkpoint saving. num_epochs (`int`, *optional*, defaults to `100`): Number of epochs to train. save_freq (`int`, *optional*, defaults to `1`): Number of epochs between saving model checkpoints. num_checkpoint_limit (`int`, *optional*, defaults to `5`): Number of checkpoints to keep before overwriting old ones. mixed_precision (`str`, *optional*, defaults to `"fp16"`): Mixed precision training. allow_tf32 (`bool`, *optional*, defaults to `True`): Allow `tf32` on Ampere GPUs. resume_from (`str`, *optional*, defaults to `""`): Path to resume training from a checkpoint. sample_num_steps (`int`, *optional*, defaults to `50`): Number of sampler inference steps. sample_eta (`float`, *optional*, defaults to `1.0`): Eta parameter for the DDIM sampler. sample_guidance_scale (`float`, *optional*, defaults to `5.0`): Classifier-free guidance weight. train_batch_size (`int`, *optional*, defaults to `1`): Batch size for training. train_use_8bit_adam (`bool`, *optional*, defaults to `False`): Whether to use the 8bit Adam optimizer from `bitsandbytes`. train_learning_rate (`float`, *optional*, defaults to `1e-3`): Learning rate. train_adam_beta1 (`float`, *optional*, defaults to `0.9`): Beta1 for Adam optimizer. train_adam_beta2 (`float`, *optional*, defaults to `0.999`): Beta2 for Adam optimizer. train_adam_weight_decay (`float`, *optional*, defaults to `1e-4`): Weight decay for Adam optimizer. train_adam_epsilon (`float`, *optional*, defaults to `1e-8`): Epsilon value for Adam optimizer. train_gradient_accumulation_steps (`int`, *optional*, defaults to `1`): Number of gradient accumulation steps. train_max_grad_norm (`float`, *optional*, defaults to `1.0`): Maximum gradient norm for gradient clipping. negative_prompts (`str` or `None`, *optional*, defaults to `None`): Comma-separated list of prompts to use as negative examples. truncated_backprop_rand (`bool`, *optional*, defaults to `True`): If `True`, randomized truncation to different diffusion timesteps is used. truncated_backprop_timestep (`int`, *optional*, defaults to `49`): Absolute timestep to which the gradients are backpropagated. Used only if `truncated_backprop_rand=False`. truncated_rand_backprop_minmax (`tuple[int, int]`, *optional*, defaults to `(0, 50)`): Range of diffusion timesteps for randomized truncated backpropagation. push_to_hub (`bool`, *optional*, defaults to `False`): Whether to push the final model to the Hub. rNz.pyhelpzJName of this experiment (defaults to the file name without the extension).)defaultmetadataexp_namezName of this run.run_namez Random seed for reproducibility.seedz)Log with either 'wandb' or 'tensorboard'.wandb tensorboard)r choiceslog_withzFrequency for logging images.log_image_freqz:Keyword arguments for the tracker (e.g., `wandb_project`).)default_factoryrtracker_kwargsz&Keyword arguments for the accelerator.accelerator_kwargszKKeyword arguments for the accelerator project config (e.g., `logging_dir`).project_kwargstrlz$Name of project to use for tracking.tracker_project_namelogsz2Top-level logging directory for checkpoint saving.logdirdzNumber of epochs to train. num_epochsz2Number of epochs between saving model checkpoints. save_freqz:Number of checkpoints to keep before overwriting old ones.num_checkpoint_limitfp16zEMixed precision training. Possible values are 'fp16', 'bf16', 'none'.)r&bf16nonemixed_precisionTzAllow `tf32` on Ampere GPUs. allow_tf32z*Path to resume training from a checkpoint. resume_from2z"Number of sampler inference steps.sample_num_stepsg?z#Eta parameter for the DDIM sampler. sample_etag@z Classifier-free guidance weight.sample_guidance_scalezBatch size for training.train_batch_sizeFz;Whether to use the 8bit Adam optimizer from `bitsandbytes`.train_use_8bit_adamgMbP?zLearning rate.train_learning_rateg?zBeta1 for Adam optimizer.train_adam_beta1g+?zBeta2 for Adam optimizer.train_adam_beta2g-C6?z Weight decay for Adam optimizer.train_adam_weight_decayg:0yE>z!Epsilon value for Adam optimizer.train_adam_epsilonz&Number of gradient accumulation steps.!train_gradient_accumulation_stepsz,Maximum gradient norm for gradient clipping.train_max_grad_normzrGrKrHrFr r sKZ  !-mU <fgHc"8K/LMHcMa63U*VWD#W#ESZ\iRjkHhsm V=\4]^NC^%*VW&NDcN*/0X'Y*S#X&+gh&NDcN!&evGm>n o#o&:n1opFCpC6;W2XYJY18l/mnIsn %V%ab!#![/ OSTV=[4\]J]R6;g2hiKi!"@d7efcfcV=b4cdJd#(Hj?k#l5l!!v?Y6Z[c[ %)f g!"'tvGW>X!YY#C6C^:_`e`#EVE`rbs5 ( 2 ff frH