biŻddlZddlZddlZddlZddlmZddlmZddlmZddl m Z ddl m Z m Z mZmZmZddlZddlmZddlmZddlmZmZdd lmZdd lmZmZmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&dd l'm(Z(dd l)m*Z*dd l+m,Z,ddl-m.Z.ddl/m0Z0m1Z1m2Z2m3Z3m4Z4ddl5m6Z6m7Z7ddl8m9Z9ddl:m;Z;mZ>m?Z?e.rddl@Z@ddl@mAZAmBZBmCZCmDZDe&rddlEZEedeFeZGdeGdeGfdZHeGdde(ZIGdde$ZJy)N) defaultdict)Mapping) dataclass)Path)AnyCallableOptionalTypeVarUnion) PartialState)DatasetIterableDataset)version) AutoModelForCausalLM AutoTokenizerBaseImageProcessor DataCollatorFeatureExtractionMixinPreTrainedModelPreTrainedTokenizerBaseProcessorMixinTrainerTrainingArgumentsis_wandb_available)DataCollatorMixin)TrainerCallback)EvalPrediction)is_peft_available)is_conversationalis_conversational_from_valuemaybe_convert_to_chatml pack_datasettruncate_dataset)clone_chat_templateget_act_offloading_ctx_manager) SFTConfig)ConstantLengthDatasetgenerate_model_cardget_comet_experiment_urlpadpeft_module_casting_to_bf16) PeftConfig PeftModelget_peft_modelprepare_model_for_kbit_trainingTListOrMappingexamplereturnc ^t|tr1|Dcgc]%}t|ttfr t|n|'c}St|trG|j Dcic]+\}}|$|t|ttfr t|n|-c}}St dcc}wcc}}w)a Recursively removes entries with `None` values from a nested structure (list or dictionary). Args: example (`list` or `Mapping`): Input nested structure (list or dictionary) from which to remove `None`. Example: ```python >>> [{ ... "a": {"aa": None, ... "ab": 1}, ... "b": "my_string", ... }] >>> remove_none_values(example) [{'a': {'ab': 1}, 'b': 'my_string'}] ``` z%Input must be a list or a dictionary.) isinstancelistdictremove_none_valuesritems TypeError)r3valuekeys R/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/trl/trainer/sft_trainer.pyr9r9Ms&'4 elm\aZd|-L"5)RWWmm GW %&mmo U  jt .M#E*SX X  ?@@n s *B$%0B)ceZdZUdZeed<dZeed<dZeed<dZ eed<dZ e eed <d Z e ed <d eeeeeee effd ee effdZy)DataCollatorForLanguageModelinga Data collator used for language modeling data. Inputs are dynamically padded to the maximum length of a batch. This collator expects each example in the input list to be a dictionary containing at least the `"input_ids"` key. If the input contains a `"completion_mask"`, it is used to set the labels to `-100` for tokens that are not in the completion. If `"assistant_masks"` are present, they are used to set the labels to `-100` for tokens that are not in the assistant part of the sequence. The collator returns a dictionary containing the following keys: - `"input_ids"`: Tensor of input IDs, padded to the maximum length of the batch. - `"attention_mask"`: Tensor of attention mask, padded to the maximum length of the batch. - `"position_ids"`: Tensor of position IDs, padded to the maximum length of the batch. - `"labels"`: Tensor of labels, padded to the maximum length of the batch. If `completion_only_loss` is set to `True`, tokens that are not in the completion are set to -100. If `assistant_masks` are present, tokens that are not in the assistant part of the sequence are set to -100. Args: pad_token_id (`int`): Token ID to use for padding. completion_only_loss (`bool`, *optional*, defaults to `True`): When the input contains a completion mask (`completion_mask`), the labels are set to -100 for the tokens that are no in the completion. padding_free (`bool`, *optional*, defaults to `False`): If set to `True`, the sequences will be flattened into a single sequence, and the position IDs will be generated accordingly. The attention mask will be set to 1 for all tokens. pad_to_multiple_of (`int` or `None`, *optional*, defaults to `None`): If set, the sequences will be padded to a multiple of this value. return_tensors (`str`, *optional*, defaults to `"pt"`): Type of Tensor to return. Only `"pt"` is currently supported. Examples: ```python >>> from trl import DataCollatorForLanguageModeling >>> collator = DataCollatorForLanguageModeling(pad_token_id=0) >>> examples = [{"input_ids": [1, 2, 3]}, {"input_ids": [4, 5]}] >>> collator(examples) {'input_ids': tensor([[ 1, 2, 3], [ 4, 5, 0]]), 'attention_mask': tensor([[ 1, 1, 1], [ 1, 1, 0]]), 'position_ids': tensor([[0, 1, 2], [0, 1, 0]]), 'labels': tensor([[ 1, 2, 3], [ 4, 5, -100]])} >>> # With completion mask >>> examples = [ ... {"input_ids": [1, 2, 3], "completion_mask": [0, 1, 1]}, ... {"input_ids": [4, 5], "completion_mask": [0, 1]}, ... ] >>> collator(examples) {'input_ids': tensor([[ 1, 2, 3], [ 4, 5, 0]]), 'attention_mask': tensor([[ 1, 1, 1], [ 1, 1, 0]]), 'position_ids': tensor([[0, 1, 2], [0, 1, 0]]), 'labels': tensor([[-100, 2, 3], [-100, 5, -100]])} >>> # With padding_free >>> collator = DataCollatorForLanguageModeling(pad_token_id=0, padding_free=True) >>> collator(examples) {'input_ids': tensor([[ 1, 2, 3, 4, 5]]), 'attention_mask': tensor([[1, 1, 1, 1, 1]]), 'position_ids': tensor([[0, 1, 2, 0, 1]]), 'labels': tensor([[1, 2, 3, 4, 5]])} ``` pad_token_idTcompletion_only_lossF padding_freereturn_position_idsNpad_to_multiple_ofptreturn_tensorsexamplesr4c|Dcgc]}tj|d}}|jxrd|dvxr |j}|s"|Dcgc]}tj|}}|jrXd|dvr&|Dcgc]}tj|d}}n+Dcgc] }tj t |"}}d|dvr&|Dcgc]}tj|d}}n%|Dcgc]}tj|d}}|jr,d|dvr%|Dcgc]}tj|d} }d|dvr%|Dcgc]}tj|d} }i} |jr0tjdjd| d<|s)tjdjd| d<|jr)tjdjd| d<tj|djd| d<|jr8d|dvr1tj djd} d | d| dk(<d|dvr1tj djd} d | d| dk(<| St|jd |j | d<tdd |j | d<|jrtdd |j | d<t|d d |j | d<|jr+d|dvr$t dd |j } d | d| dk(<d|dvr$t dd |j } d | d| dk(<| Scc}wcc}wcc}wcc}wcc}wcc}wcc}wcc}w) N input_ids position_idsrlabelscompletion_maskassistant_masksdimattention_maskright) padding_value padding_siderE) torchtensorrDrC ones_likearangelenrBcat unsqueezer,rArE) selfrHr3rJhas_packed_position_idsrQrKidsrLrMrNoutputs r> torch_callz*DataCollatorForLanguageModeling.torch_callsGOPGU\\'+"67P P#'":":"r~QYZ[Q\?\"raearar'JSTYeooi8TNT  # #!,U]^' W^-D E^ ^BKL3 SX 6L L x{ "EMN'ell78#45NFNHPQWell7;#78QFQ  $ $):hqk)IW_`Gu||G4E,FG`O`  +W_`Gu||G4E,FG`O`   "'))I1"="G"G"JF; *+099^+K+U+UVW+X'('').(1+-M"'))O"C"M"Ma"P9=x A!56 HQK/"'))O"C"M"Ma"P9=x A!568 5#&"//$#'#:#: #F;  (+agZ^ZqZq(F# $''), \`\s\s*~& #dUYUlUl F8 ((->(1+-M"%#17_c_v_v#:>x A!56 HQK/"%#17_c_v_v#:>x A!56 EQU _LNQ``s/N0N5N:*%N?OO :O&O)__name__ __module__ __qualname____doc__int__annotations__rBboolrCrDrEr rGstrr7r rr8rar>r@r@lsCJ!%$%L$ $$(, ,NCD4d3id38n.L(M#NDSWX[]`X`SaDrkr@c xeZdZdZddgZ d)deeeje fde ee e fde e de eeefd e eeeeeffd e eeeeefd e ed e eegefd e eedee ej6j8e ej6j:j<fde eeej6j8eee ffde eejBejBgejBfde dde eegefffd Z"dede de fdZ#de de de de fdZ$de de de fdZ%de de de fdZ&deeefd eeeeefde de'de eegefdedeeeffdZ(dZ)d*fd Z*fd!Z+d+d"eee,fd#e e,ddffd$ Z-fd%Z. d,d&e ede ed'eeeedffd(Z/xZ0S)- SFTTrainera$ Trainer for Supervised Fine-Tuning (SFT) method. This class is a wrapper around the [`transformers.Trainer`] class and inherits all of its attributes and methods. Example: ```python from datasets import load_dataset from trl import SFTTrainer dataset = load_dataset("roneneldan/TinyStories", split="train[:1%]") trainer = SFTTrainer(model="Qwen/Qwen2-0.5B-Instruct", train_dataset=dataset) trainer.train() ``` Args: model (`Union[str, PreTrainedModel]`): Model to be trained. Can be either: - A string, being the *model id* of a pretrained model hosted inside a model repo on huggingface.co, or a path to a *directory* containing model weights saved using [`~transformers.PreTrainedModel.save_pretrained`], e.g., `'./my_model_directory/'`. The model is loaded using [`~transformers.AutoModelForCausalLM.from_pretrained`] with the keyword arguments in `args.model_init_kwargs`. - A [`~transformers.PreTrainedModel`] object. Only causal language models are supported. args ([`SFTConfig`], *optional*, defaults to `None`): Configuration for this trainer. If `None`, a default configuration is used. data_collator (`DataCollator`, *optional*): Function to use to form a batch from a list of elements of the processed `train_dataset` or `eval_dataset`. Will default to a custom [`DataCollatorForLanguageModeling`]. train_dataset ([`~datasets.Dataset`] or [`~datasets.IterableDataset`]): Dataset to use for training. SFT supports both [language modeling](#language-modeling) type and [prompt-completion](#prompt-completion) type. The format of the samples can be either: - [Standard](dataset_formats#standard): Each sample contains plain text. - [Conversational](dataset_formats#conversational): Each sample contains structured messages (e.g., role and content). The trainer also supports processed datasets (tokenized) as long as they contain an `input_ids` field. eval_dataset ([`~datasets.Dataset`], [`~datasets.IterableDataset`] or `dict[str, Union[Dataset, IterableDataset]]`): Dataset to use for evaluation. It must meet the same requirements as `train_dataset`. processing_class ([`~transformers.PreTrainedTokenizerBase`], *optional*, defaults to `None`): Processing class used to process the data. If `None`, the processing class is loaded from the model's name with [`~transformers.AutoTokenizer.from_pretrained`]. callbacks (list of [`~transformers.TrainerCallback`], *optional*, defaults to `None`): List of callbacks to customize the training loop. Will add those to the list of default callbacks detailed in [here](https://huggingface.co/docs/transformers/main_classes/callback). If you want to remove one of the default callbacks used, use the [`~transformers.Trainer.remove_callback`] method. optimizers (`tuple[torch.optim.Optimizer, torch.optim.lr_scheduler.LambdaLR]`, *optional*, defaults to `(None, None)`): A tuple containing the optimizer and the scheduler to use. Will default to an instance of [`AdamW`] on your model and a scheduler given by [`get_linear_schedule_with_warmup`] controlled by `args`. optimizer_cls_and_kwargs (`Tuple[Type[torch.optim.Optimizer], Dict[str, Any]]`, *optional*, defaults to `None`): A tuple containing the optimizer class and keyword arguments to use. Overrides `optim` and `optim_args` in `args`. Incompatible with the `optimizers` argument. Unlike `optimizers`, this argument avoids the need to place model parameters on the correct devices before initializing the Trainer. preprocess_logits_for_metrics (`Callable[[torch.Tensor, torch.Tensor], torch.Tensor]`, *optional*, defaults to `None`): A function that preprocess the logits right before caching them at each evaluation step. Must take two tensors, the logits and the labels, and return the logits once processed as desired. The modifications made by this function will be reflected in the predictions received by `compute_metrics`. Note that the labels (second parameter) will be `None` if the dataset does not have them. peft_config ([`~peft.PeftConfig`], *optional*, defaults to `None`): PEFT configuration used to wrap the model. If `None`, the model is not wrapped. formatting_func (`Optional[Callable]`): Formatting function applied to the dataset before tokenization. Applying the formatting function explicitly converts the dataset into a [language modeling](#language-modeling) type. trlsftNmodelargs data_collator train_dataset eval_datasetprocessing_classcompute_loss_funccompute_metrics callbacks optimizersoptimizer_cls_and_kwargspreprocess_logits_for_metrics peft_configr.formatting_funcc, t|tr|n|jj}|#|j dd}t |d}n[t|t rKt|t s;|j}|j|d<|jdt d%i|}|tj|}|jL|j}|j|}|&td|d|jj d||_|j$%t|tst'j(d t|tr|j+||}|j,t.j0j3|j,rQ|j,j5d r6t7|j,d 5}|j9|_dddnt=|||j,\}}| |j?|| |}|j@xs|jBxr|jDd k(|_ |j@r| td|jBr$|jDdk(rt'j(d|jjFdk7rt'j(d|jHdk(r!|jBst'j(d|jJtMtO|}d|v|_%n|jJ|_%||jPxs|jPxs |j}|j|}|&td|d|jj dtS||jJ|j@|jjFdk(|jT}|jBr=|jDd k(r.|jjFdk7rt'j(d|jVrtY|ds td|jZduxs|jZj]dd }|r|jJr |r td|j_||||jB|d}||j` |jBn |j`}t|tbr8|jeDcic]\}}||j_||||||}}}n|j_|||||d }tgthtgthd!|_5d|_6tn|||||||||| | | | " |jrjtrtw|jx#|_=nt}j~|_=t|jxd$r&|jxj|jyy#1swYxYwcc}}w)&N/z-SFT hub_tokenpush_to_hub_tokenzThe specified `eos_token` ('zC') is not found in the vocabulary of the given `processing_class` (zX). Ensure that the `eos_token` exists in the vocabulary before using it as an EOS token.zYou passed model_init_kwargs to the `SFTConfig`, but your model is already instantiated. The `model_init_kwargs` will be ignored.)z.jinjaz.j2zutf-8)encodingffdzHPassing a custom data collator is not supported when using padding-free.wrappedzYou are passing `padding_free=True` with the 'wrapped' packing strategy, which is not recommended. Please refer to the documentation to understand why this is not recommended.flash_attention_2aPadding-free training is enabled, but the attention implementation is not set to 'flash_attention_2'. Padding-free training flattens batches into a single sequence, and 'flash_attention_2' is the only known attention mechanism that reliably supports this. Using other implementations may lead to unexpected behavior. To ensure compatibility, set `attn_implementation='flash_attention_2'` in the model configuration, or verify that your attention mechanism can handle flattened sequences.r'zYou are using a per_device_train_batch_size of 1 with padding-free training. Using a batch size of 1 anihilate the benefits of padding-free training. Please consider increasing the batch size to at least 2.promptzThe specified `pad_token` ('z[). Ensure that the `pad_token` exists in the vocabulary before using it as a padding token.)rArBrCrDrEaYou are using packing, but the attention implementation is not set to 'flash_attention_2'. Packing flattens batches into a single sequence, and 'flash_attention_2' is the only known attention mechanism that reliably supports this. Using other implementations may lead to cross-contamination between batches. To avoid this, either disable packing by setting `packing=False`, or set `attn_implementation='flash_attention_2'` in the model configuration.rzYou set `assistant_only_loss=True`, but the dataset is not conversational. This option is only supported for conversational datasets.skip_prepare_datasetFaEA formatting function was provided while `completion_only_loss=True`, which is incompatible. Using a formatter converts the dataset to a language modeling type, conflicting with completion-only loss. To resolve this, apply your formatting function before passing the dataset, or disable `completion_only_loss` in `SFTConfig`.traineval)rr) rprqrrrsrtrurvrwrxryrzr{)rpadd_model_tagsrj)Cr6riconfig _name_or_pathsplitr(rto_dictrpoprfrom_pretrained eos_tokenconvert_tokens_to_ids ValueError __class__rb eos_token_idmodel_init_kwargswarningswarn_create_model_from_pathchat_template_pathospathisfileendswithopenread chat_templater%_prepare_peft_modelrCpackingpacking_strategy_attn_implementationper_device_train_batch_sizerBnextiter pad_tokenr@rEassistant_only_lossr dataset_kwargsget_prepare_dataset eval_packingr8r:rr7_metrics_total_train_tokenssuper__init__rqactivation_offloadingr&rp maybe_activation_offload_context contextlib nullcontexthasattrr _tag_names)r]rprqrrrsrtrurvrwrxryrzr{r|r}model_id model_name dict_argsrrchat_template_file first_examplerrApreprocess_datasetrr=datasetrs r>rzSFTTrainer.__init__Ns('uc25 8R8R <!,R0J |401D / 0D)9T I%)^^Ik " MM- .)y)D  #,<> %I+AA)LL# 29+>++;+E+E+N+N*OPII -9  )  ! ! -j6L MM;  eS !00=E  " " .ww~~d5564;R;R;[;[\m;n$11GDOHZ5G5L5L5N$2OO+>eEUW[WnWn*o''  ",,UKFE !--b$,,2a4CXCX\aCa   ( !kll|| 5 5 B p||004GG J//14T\\ %  $ $ , m!45M(0M(AD %(,(A(AD %  b*:*D*DbHXHbHbI+AA)LL# 29+>++;+E+E+N+N*OPLL <)%)%>%>!..$)LL$E$EI\$\#'#:#: M LL%%. 115HH MMX   # #,=mA>N,O9  "00D8v@S@S@W@WXnpu@v,>,@$(CT2274J4J4LD 1 4::/ 0 JJ % %doo 6 1UOOB$sX:!XX  model_pathr4c"|jxsi}|jd}t|tjs|dk(s|n5t|t rt t|}||d<ntd|dtj|fi|}|S)z0Creates a model from a path or model identifier. torch_dtypeautozInvalid `torch_dtype` passed to `SFTConfig`. Expected either 'auto' or a string representing a `torch.dtype` (e.g., 'float32'), but got .) rrr6rVdtyperigetattrrrr)r]rrqrrrps r>rz"SFTTrainer._create_model_from_paths 228b'++M: k5;; /;&3HKL_   S )!%5K/: m ,>>I]!M %44ZUCTU rkcNts tdt|tst dt |dt|t r|St|ddxs t|dd}d}t|ddrU|jD]B\}}|jjdk(s |jjj dv}n|r,|s*|j||}tj|d }n|j r|j#||}t%j&t(j*t%j&d k\rt|ddr|rt-||d }n t-||}|j.rt|ddr |s t1||S) z#Prepares a model for PEFT training.z9To use PeftModel, you need to install the `peft` library.z#Expected PeftConfig object but got z[. If you want to use the PeftModel, you need to pass a PeftConfig object to the SFTTrainer.is_loaded_in_4bitFis_loaded_in_8bit Params4bit>cpumeta)gradient_checkpointingz0.12)autocast_adapter_dtype)r ImportErrorr6r.rtyper/rnamed_parametersrrbdatadevice _prepare_model_for_kbit_training dataclassesreplacer_enable_gradient_checkpointingrparsepeft __version__r0bf16r-)r]rpr|rqis_qlorais_sharded_qlora_params r>rzSFTTrainer._prepare_peft_model*s "YZ Z+z25d;6G5HIAA  eY 'L5"5u=kPcejAk  5-u 5!224 5??++|;',zz'8'8'='='P$  ,99%FE&&tEJD  ( (77tDE MM$** +w}}V/D D2E: "5+eTE"5+6E 99(;UCL\ ' . rkcT|j|jxsid}t|fi|S)z-Prepares a quantized model for kbit training.)use_gradient_checkpointinggradient_checkpointing_kwargs)rrr1)r]rprqprepare_model_kwargss r>rz+SFTTrainer._prepare_model_for_kbit_training[s7+/*E*E-1-O-O-USU  /uM8LMMrkc|jxsi}d|vxs|d}|r@t|dr|j|Sd}|jj ||S)z-Enables gradient checkpointing for the model. use_reentrantenable_input_require_gradsc&|jdy)NT)requires_grad_)moduleinputr`s r>make_inputs_require_gradzKSFTTrainer._enable_gradient_checkpointing..make_inputs_require_gradps))$/rk)rrrget_input_embeddingsregister_forward_hook)r]rprqrrrs r>rz)SFTTrainer._enable_gradient_checkpointingdst(,(J(J(Pb% #@ @ rDabqDr  u:;002 0**,BBC[\ rkrr dataset_namec t|tr|St|tr|jt}t t t|j}d|v}i} t|tr|j| d<tj5|rtjdt6|s4t|tr d|d| d<fd} |j| fdd i| }|s%t t|} t%| rZt|tr d |d| d<t t|j}|jt&fdd|vrdndi| }t t|} t)| sDt|tr d|d| d<d} |j| fd|j*id|vrdndd| }t|tr d|d| d<d}|j|fd||j,|j.di| }|rd|j0 t3dt|tr d|d| d<|j5d}t7||j0|j8| }n<|j00t|tr d|d| d<t;||j0| }|j<r,|j5ddhj?|j@}ddd|S#t $r>} tjd | d t"|j| fdd i| }Yd} ~ Nd} ~ wwxYw#1swY|SxYw)NrJnum_proczYou passed a dataset that is already processed (contains an `input_ids` field) together with a formatting function. Therefore `formatting_func` will be ignored. Either remove the `formatting_func` or pass a dataset that is not already processed.z Applying formatting function to z datasetdesccd|iS)Ntextrj)r3r}s r>_funcz*SFTTrainer._prepare_dataset.._funcs"OG$<==rkbatchedFzDFailed to apply the formatting function due to the following error: a3. This may be because the function is designed for batched input. Please update it to process one example at a time (i.e., accept and return a single example). For now, we will attempt to apply the function in batched mode, but note that batched formatting is deprecated and will be removed in version 0.21.Tz Converting z dataset to ChatMLremove_columns conversationszAdding EOS to cd|vr!|dj|s |d|z|d<|Sd|vr|dj|s |d|z|d<|S)Nr completion)r)r3rs r>add_eosz,SFTTrainer._prepare_dataset..add_eossj!W,WV_5M5Mi5X.5fo .IGFO '*W4W\=R=[=[\e=f4;L4II4UGL1&rkrmessages) fn_kwargsrz Tokenizing cFd|vr t|ru|j|dfd|jdi|jdi}|j|d|dzfd|jdi|jdi}n2||dj}||d|dzj}|dt ||k(st j ddgt |zdgt |t |z zz}||d }|St|ri|j|d fd ||jdd |jdi}d |vrd|d vr tddDcic] }||vs|||}}|Sd|||ji}|Scc}w)Nrtoolschat_template_kwargsr)rzMismatch between tokenized prompt and the start of tokenized prompt+completion. This may be due to unexpected tokenizer behavior, whitespace issues, or special token handling. Verify that the tokenizer is processing text consistently.rr')rJrMrT) return_dictreturn_assistant_tokens_maskrrNu?You're using `assistant_only_loss=True`, but at least one example has no assistant tokens. This usually means the tokenizer's chat template doesn't generate assistant masks — it may be missing the `{% generation %}` keyword. Please check the template and ensure it's correctly configured to support assistant masking.)rJrNrJ)r apply_chat_templaterrJrZrr RuntimeError) r3rudataset_text_fieldr prompt_idsprompt_completion_idsrM processedks r>tokenizez-SFTTrainer._prepare_dataset..tokenizes7*,W5)M)9)M)M ' 1*&-kk'&:*#*++.Db"I*J 5Y4D4X4X ' 1GL4I I5&-kk'&:5#*++.Db"I51 *:wx?P)Q)[)[J4D%,X%69N%N5'i2 55Fs:G:U$MM!m,-#J*?1#MbIcfijtfuIuBv*v2G\k$l ,%$'-W5(L(8(L(L ' 3),0=P&-kk'&: ) #*++.Db"I )I 1I=!9UfKgBg&2%/'"!"Cc(uQfgktftIaL(uI(u%$*56FGTfLg6h6r6r(sI$$)vs . F8Fr)rurrz6When packing is enabled, `max_length` can't be `None`.zPacking z Truncating rK)!r6r)r with_transformr9r7rrkeysdataset_num_procr main_process_firstrr UserWarningmap ExceptionDeprecationWarningr!r"r rrr max_lengthrselect_columnsr#rr$use_liger_kernel intersection column_names)r]rrurqrr}rr is_processed map_kwargsrerrr s ` r>rzSFTTrainer._prepare_datasetws g4 5N gw ',,-?@GDg/4467 "l2  gw '%)%:%:Jz " ^ . . 0Q s*| Y *<gw/+KL>Ya)bJv&> M)gkk%MM*MG $T'] 3 / >!'73/:<.HZ-[ 6*#'W #6#;#;#=L)gkk/:I\:Y_c%G!%T'] 3 (7!'73/=l^8-T 6*'*gkk#.0@0J0J"K5?<5OzUY% Ggw/+6|nH)MJv&3%j&'++-=.2.E.E/3/G/G!??*$%]^^gw/+3L>)JJv&!00=&wAVAVXbc,gw/+6|nH)MJv&*7DOOZP$$!00+~1N1[1[\c\p\p1qrcQ sfA! MMM^_`^ab++ + *gkk%LLLG M'Q sfs8!AM#L8HM M  3MMM  MMc0|j gd|_yy)N)rJrLrKrMrN)_signature_columns)r]s r> _set_signature_columns_if_neededz+SFTTrainer._set_signature_columns_if_needed's  " " *'D # +rkc|jjrdnd}t| ||d|\}}|dk(rd|vrI|jj |dj j j}n}d|vrntj|djd|dj} |jj | j j}n td |xj|z c_ |jg|j|d <d |vr#|jj s |j"d d dd d fj%} |d d dd fj%} | j'd} | dk7} | | k(| z}| j }|j }|jj |}|jj |}|j }|dkDr!|j |z jnd}|j|dj)||r||fS|S)zQ Compute training loss and additionally compute token accuracies rrT)return_outputsnum_items_in_batchrQrKr')rz6Expected 'attention_mask' or 'position_ids' in inputs. num_tokensrL.NrrOrRrgmean_token_accuracy)rptrainingr compute_loss acceleratorgather_for_metricssumitemrVrWsizerrrrrqrlogits contiguousargmaxappend)r]rpinputsrr modelossoutputsnum_tokens_in_batchlocal_num_tokens shift_logits shift_labels predictionsmaskcorrect_predictions total_tokenscorrect_tokens total_sumaccuracyrs r>r$zSFTTrainer.compute_loss5sR**--w6'. 6$CU/ w 7? 6)&*&6&6&I&I&QaJbJfJfJh&i&m&m&o&t&t&v#6)#(<<~0F0K0KA0NW]^lWmWtWt#u &*&6&6&I&IJZ&[&_&_&a&f&f&h# !YZZ  $ $(; ; $-1-E-E,F dL) v dii&@&@">>#ssA+6AACL!(+CG4??AL'--"-5K 4'D$/,#>$"F 88:L0446N"--@@PN++>>|LL%((*IDMPQM**,y8>>@WZH MM$  5 6 = =h G"0g:d:rkcf|j5t||i|cdddS#1swYyxYwN)rr training_step)r]rqkwargsrs r>r?zSFTTrainer.training_stepfs3  2 2 :7($9&9 : : :s'0logs start_timec |jjrdnd}|j|jDcic]\}}|t |t |z }}}|dk(r&|jDcic] \}}d|| }}}i||}t ||||j|jycc}}wcc}}w)Nrreval_) rpr#rr:r'rZrlogclear)r]rArBr/r=valmetricsrs r>rEzSFTTrainer.logjs**--w6<@MM$:A--/Jhc3se}c)JGJ"$"'"  D*% d!!#Y Ks "C9Cc|jj*t|jjj}n(|jjj dd}|j |t|!||y)Nrr)r) rq hub_model_idr output_dirnamercreate_model_cardr_save_checkpoint)r]rptrialrrs r>rNzSFTTrainer._save_checkpointxsl 99 ! ! )dii22388J//55c:2>J *5  .rkrtagsc h|jsyt|jjdr^tj j |jjjs!|jjj}nd}| t}nt|tr|h}n t|}t|jjdr|jd|j|jt|||j|t!|t#r.t$j&t$j&j)ndt+d}|j-tj j/|j0j2dy)a Creates a draft of a model card using the information available to the `Trainer`. Args: model_name (`str` or `None`, *optional*, defaults to `None`): Name of the model. dataset_name (`str` or `None`, *optional*, defaults to `None`): Name of the dataset used for training. tags (`str`, `list[str]` or `None`, *optional*, defaults to `None`): Tags to be associated with the model card. Nrunsloth_versionunslothSFT) base_modelrrJrrP wandb_url comet_url trainer_namez README.md)is_world_process_zerorrprrrisdirrsetr6riaddupdaterr*rJr7rwandbrunget_urlr+savejoinrqrK)r]rrrPrU model_cards r>rMzSFTTrainer.create_model_cards-"))+  4::$$o 6rww}}TZZM^M^MlMl?m**88JJ <5D c "6Dt9D 4::$$&7 8 HHY  DOO$(!!**%d-?-AeiiF[eii'')ae.0    TYY%9%9;GHrk) NNNNNNNN)NNNNNN)FNr>)NNN)1rbrcrdrerr rinnModulerr r(rrr rr8rrrrrrr7rtuplerVoptim Optimizer lr_schedulerLambdaLRrrTensorrrrrrrhrrr$r?floatrErNrM __classcell__)rs@r>rmrms{HTJ ?C04CGEI 04FJ59jvaehl.2;?#C7S"))_45C7uY(99:;C7 - C7  g&> ?@ C7 uWd3<.@%@AB C7# )+=?UWee f C7$H-C7"(N+;T+A"BCC7D12C7(5;;#8#898EKKD\D\DeDe;ffgC7#+5ekk6K6K1LdSVX[S[n1\+]"^C7(0%,, 9UW\WcWc9c0d'eC7 l+!C7""(D63;"78#C7J#Y?.//s/R[/`o/bNoNYN[jNO9Yh&nw/0n 79KMces stn n  n "(D63;"78 nn w' (n` .;b: $S%Z( $huo $QU $/%)&*,0 1ISM1Ism1ICcD() 1Irkrm)Krrrr collectionsrcollections.abcrrpathlibrtypingrrr r r rVtorch.nnrd accelerater datasetsr r packagingr transformersrrrrrrrrrrrtransformers.data.data_collatorrtransformers.trainer_callbackrtransformers.trainer_utilsrtransformers.utilsr data_utilsr r!r"r#r$modelsr%r& sft_configr(utilsr)r*r+r,r-rr.r/r0r1r^r7r2r9r@rmrjrkr>rs ##!:: #-    >950I![[)49AA>A> Q&7Q Qhp Ip Irk