bi/PddlmZddlZddlZddlZddlZddlmZddlZddl Z ddlm Z m Z ddl m Z mZddlmZddlmZddlmZd d lmZd d lmZmZmZmZmZmZmZd d l m!Z!d Z"dZ# ddZ$ d ddZ% ddZ& d ddZ'dddZ( d ddZ)y)) annotationsN)Optional) file_existshf_hub_download)EntryNotFoundErrorLocalEntryNotFoundError) load_file)http_user_agent)PEFT_TYPE_TO_PREFIX_MAPPING)INCLUDE_LINEAR_LAYERS_SHORTHAND)EMBEDDING_LAYER_NAMESSAFETENSORS_WEIGHTS_NAME WEIGHTS_NAMEAuxiliaryTrainingWrappercheck_file_exists_on_hf_hub infer_devicematch_target_against_key)PeftTypect|dxrHt|jtjj tjj fS)z.Check if the layer has an embedding base layer base_layer)hasattr isinstancertorchnnLinear Embedding)layers S/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/peft/utils/save_and_load.pyhas_valid_embedding_base_layerr ,s; 5, ' oJu7G7G%((//[`[c[c[m[mIn,oocj|jD] \}}|s||k(s|t|ddk(s|cSy)z7Get the name of the embedding module for a given layer.rN) named_modulesgetattr)modelris_embedding_in_target_modulesnamemodules rget_embedding_layer_namer)1sE++- f.6U?vQXY^`lnrQsGsK r!c 2 !|r t|d|}|j| ||j} jtj tj fvr j}|dk(r|Dcic] }d|vs|||}}nh|dk(r|Dcic]}d|vsd|vs |||}}nF|dk(r;i}|D]3}d|vs||||<|jdddz}||vs,||||<5nt|jD cic]\}} d|vr||vsd|vs|| }}} jtj k(r^ j} | P| jD cic]\}} |jd |d | } }} | _ |j| ||} jrd |d !!fd } |jD cic]\}} | || }}} nK jtjk(r j}|dk(r|Dcic] }d|vs|||}}n|dk(r|Dcic]}d|vsd|vs |||}}n|dk(ri} jXdkrtJjZ}nP jXdkrtJj\}n0 jXdkrtJj^}ntJj`} jbr|D]}d |vs||je jd\}}|jg|d!z|jI|"i|jg|d#ztKjh|d$ddddddfjkin|Dcic] }d |vs|||}}|d%|z|d%|z<n_ jtmtvr,t8 j}|Dcic] }||vs|||}}ntSd& j|jCD]\}}to|tps|j#d'r|jsd'}|jD cic]/\}} |j#|d s|js|d | 1}}} |jg|ju||jD cic] \}} |d || c} }d(}tE d)rto jvtxrT jvtzk7rAtE|d*r|j}n|}t fd+|jCD}n$ jvrt fd,tD} jtjk(xst d-ddu}|d.k(r|r|st?j@d/d0}n|d.k(rtt|d1dd2d}t d3d}d(}|mtjjtjj|d4}|xs t|d4}|t?j@d5|d6d(}n|}|rN|rL|rJ||jjj|jk7rt?j@d7d0}nd(}|rtE|d8r||j|jfD]X}|r t|st|||}|s!|jg|jD cic] \}} ||vs || c} }Zn|rt?j@d9|jD cic]\}} |jd |d | }}} |Scc}wcc}wcc} }wcc} }wcc} }wcc}wcc}wcc}wcc}wcc}wcc}wcc}wcc}wcc} }wcc} }wcc} }wcc} }w):uQ Get the state dict of the Peft model. Args: model ([`PeftModel`]): The Peft model. When using torch.nn.DistributedDataParallel, DeepSpeed or FSDP, the model should be the underlying model/unwrapped model (i.e. model.module). state_dict (`dict`, *optional*, defaults to `None`): The state dict of the model. If not provided, the state dict of the passed model will be used. adapter_name (`str`, *optional*, defaults to `"default"`): The name of the adapter whose state dict should be returned. unwrap_compiled (`bool`, *optional*, defaults to `False`): Whether to unwrap the model if torch.compile was used. save_embedding_layers (`Union[bool, str]`, , *optional*, defaults to `auto`): If `True`, save the embedding layers in addition to adapter weights. If `auto`, checks the common embedding layers `peft.utils.other.EMBEDDING_LAYER_NAMES` in config's `target_modules` when available. Based on it sets the boolean flag. This only works for 🤗 transformers models. _orig_modNnonelora_allbias lora_onlyr.lora_magnitude_vector..weightc4|jr|dd}|S)Niendswith)knew_dora_suffixs rrenamed_dora_weightsz7get_peft_model_state_dict..renamed_dora_weightsvs::o.#2Ar!boft_ boft_only adaption_prefix_task_colsprefix_task_rowsprompt_embeddingsWindowszWindows has issues saving integers into safetensors. Hence, we convert shira_indices to float32 before saving on Windows OS. The shira_indices will always be converted to integers when loading. shira_indices.shira_indices.zbase_model.vera_A.zModel was initialised to not save vera_A and vera_B but config now specifies to save projection! Set `config.save_projection` to `False`.zbase_model.vera_B.internal_xlora_classifieril vblora_logits _topk_indices)dtype _topk_weightsdimzbase_model.vblora_vector_bank.zUnknown PEFT type passed: _fsdp_wrapped_module.Ftarget_modulesget_base_modelc3K|]5\}tfdtDrtj7yw)c3RK|]}tjd|d yw)z(.*\.)?$N)rematch).0er8s r z6get_peft_model_state_dict...s$Srxx71#Q3Ss$'N)anyrrrN)rU_r8configs @rrWz,get_peft_model_state_dict..s9(AqS=RSS))>)>B(s;?c3:K|]}|jvyw)N)rN)rUr8rZs rrWz,get_peft_model_state_dict..s'bqV-B-B(B'bstrainable_token_indicesautozXSetting `save_embedding_layers` to `True` as embedding layers found in `target_modules`.TrZ vocab_sizebase_model_name_or_pathz config.jsonz Could not find a config file in z4 - will assume that the vocabulary was not modified.zdSetting `save_embedding_layers` to `True` as the embedding layer has been resized during finetuning.get_input_embeddingsuYCould not identify embedding layer(s) because the model is not a 🤗 transformers model.)Or$ peft_config state_dict peft_typerLORAADALORAr/splitNotImplementedErroritems rank_patternreplace!resize_state_dict_by_rank_patternuse_doraBOFTADAPTION_PROMPT startswithis_prompt_learningMULTITASK_PROMPT_TUNINGprompt_encoderr?r@ embeddingweightinference_modeget_prompt_embedding_to_saveSHIRAr platformsystemwarningswarnr#rrCtorfloat32VERAsave_projection ValueErrorXLORAVBLORA num_vectorsuint8int16int32int64save_only_topk_weightstopkupdatesoftmax contiguouslistrr removeprefixadapter_state_dictrNstrr rOrXrTRAINABLE_TOKENSospathexistsjoinrrZ __class__from_pretrainedr^r`get_output_embeddingsr r))"r%rb adapter_nameunwrap_compiledsave_embedding_layersr/r8 to_return bias_namevrir:rA shira_prefixr'r( vera_prefix indices_dtypelogitsindicesprefixmodule_state_dictembedding_is_targeted_modelusing_trainable_tokensr^model_idhas_base_configlocal_config_existsrrembedding_module_namerZr9s" @@rget_peft_model_state_dictr9sj ({E2   | ,F%%' HMM8+;+;<<{{ 6>3=NaAJqM)NIN U]3=]aAQW[\Q\JqM)]I] [ I Ea<#-a=IaL ! 0 3f 3=NaAJqM)NIN U]3=]aAQW[\Q\JqM)]I] [ I Ea<#-a=IaL ! 0 3f 2*D @>HH\_kHk=lI*\9 :=GH\_kHk=lI*\9 :   X^^ +/9^!=X\]=]Q 1 %^ ^   X__ ,    $!KKM   % '!KKM   % '!KKM!KKM  ( ( s"a'&0m&8&8&EOFG$$a/&97::M:;Z%[\$$a/&95==UW;XYZ\]_b`b_bYb;c;n;n;p%qr  s 4>VaTUAUJqM)VIVEO ,| ;F 2\AB   T(^ +,V-=-=>/9I!Vq[Q 1 %I I5f6F6F5GHII++- f f6 767(()@A ;E:J:J:L!26!QPQP\P\`d_eef]gPh$qz*A-! !   .4.G.G Vg.h.n.n.pqdaD61#!q :"v'( f++S 1v7L7LPo7o07u>N/OU))+UZF$'("002(% !  " "$''bLa'b$b ! H555uIbdh9iqu9u&+@I_ pq $ & (WUHd;\4P 6# k<k<l$l,l  l l  lc|s|gfSg}|j}|jD]\}}||vr ||jddk(r(||jdz|jk(rH||j|jk7se|j ||j||jf|D] \}}}||= ||fS)Nr=r )rbrhshapenumelappend)r%peft_model_state_dictignore_mismatched_sizes mismatchedrbkeytensorrYs r_find_mismatched_keysrCs #$b((J!!#J,224 J V j  sO ! !" % *C1F1F1H11LPVP\P\P^1^  c? FLL 0   sFLL*S/2G2GH I J ' Q !# &' !* ,,r!c i}|jD]n\}}||vr`|j|d}d|vr;dj|jddd}|j||d|}n|d|}|||<j|||<p|S)zbUtility function to remap the state_dict keys to fit the PEFT model by inserting the adapter name.r r1N)rhrfrrj)rbrparameter_prefixrrvalsuffixsuffix_to_replaces r$_insert_adapter_name_into_state_dictr_s$$& -S s "YY/03Ff}$'HHV\\#->qr-B$C!kk"3 ~QGXFY5Z[Q|n-), !# &), !# & - ! r!c |j|}|}|jD]l\}}t|ts|j |} |j dr|j d}| D]} |d| } |d| | } || || <|| = n|js|jtjk(r|}n|jtjk(r|}n|jtvri}t|j} |jtjk(r^|jrQ|j|j \}}t#|j%}|D]} d| vs || j't(j*}| j-dd}|| j-dd}t)j.|d|j1dd z gd }t)j2|}t)j4g|j d d|j7t9d j'|j:j=d||}|||<|| =|| j-dd=t?||| }|jtj@k(r#|jB}||jE||n|jtjFk(rtIjJdk(rtMjNd|jD]^\}}tQ|ds|d||vs|jS|d|}|j't(jT|jV|<`n|jtjXk(rc|jZrd|vr t]d|jZsd|vrtMjNdn|jZstMjNdn|jtj^k(r4d| fd}|jaD cic]\} }|| |}} }n@|jtjbk(r#ted|Drt]dtfti|||\}}|rH|jk|dd}|jmD] }tQ|ds|jo|"n|jk|d}|jr/|jp|jrjkd|d id|jtjtk(r |jp|jk|d|rcd!jw|Dcgc]\}}}d"|d#|d$|d%c}}}}d&|jxjzd'|d}tMjN||Scc}} wcc}}}w)(a Set the state dict of the Peft model. Args: model ([`PeftModel`]): The Peft model. peft_model_state_dict (`dict`): The state dict of the Peft model. adapter_name (`str`, *optional*, defaults to `"default"`): The name of the adapter whose state dict should be set. ignore_mismatched_sizes (`bool`, *optional*, defaults to `False`): Whether to ignore mismatched in the state dict. low_cpu_mem_usage (`bool`, `optional`, defaults to `False`): This argument must be `True` if the `model` was loaded with adapter weights on the meta device, e.g. after calling `inject_adapter_in_model` with `low_cpu_mem_usage=True`. Otherwise, leave it as `False`. rMr1rHr2rJr r=T)keepdimrKNz-inf)rrrBzWindows has issues saving integers into safetensors. Hence, we had converted shira_indices to float32 before saving on Windows OS. The shira_indices will always be converted to integers when loading.rCrDzbase_model.vera_AzXSpecified to load vera_A and vera_B from state dictionary however they were not present!zSpecified to not load vera_A and vera_B from state dictionary however they are present in state dictionary! Consider using them to ensure checkpoint loading is correct on all platforms using `peft_config.save_projection = True`zSpecified to not load vera_A and vera_B from state dictionary. This means we will be relying on PRNG initialisation to restore these projections using `config.projection_prng_key`, which may not be accurate on all system configurations.r3c4|jr|dz}|S)Nr4r6)r8old_dora_suffixs rr:z7set_peft_model_state_dict..renamed_dora_weightss::o.I Ar!c3$K|]}d|v yw)z.oft_r.N)rUrs rrWz,set_peft_model_state_dict..sE9#EszTrying to load old OFT checkpoint, which is no longer supported. Please install PEFT <= v0.15.2 to load it or train a new OFT adapter.)rF)strictassign%_move_adapter_to_device_of_base_layer)rrtrA z- z: found shape z in the checkpoint and z in the model instantiatedzSome weights of zy were not initialized from the model checkpoint and are being ignored because you passed `ignore_mismatched_sizes=True`: )>rar#rradapter_state_dict_load_maprorrprcrrnrr rrvblora_vector_bankrrkeysr|rlongrjcatsumlogzerosfill_floatdevicescatterrreriresize_modules_by_rank_patternrwrxryrzr{rpopintrCr~rrrdrhOFTrXrgrload_state_dictmodulesrrrrsrqrr__name__)!r%rrrlow_cpu_mem_usagerZrbr'r(key_mapr8 lookup_key store_keyrrrYstate_dict_keysr original_key topk_weights topk_logitsmatrixrishira_indices_valuesr:mismatched_keys load_resultrshape1shape2mismatched_warningmsgrs! @rset_peft_model_state_dictrrs0  | ,F&J ++-+ f f6 788FG67(()@A + $vQqc] #fAgaj\2 (=j(I 9%z* ++&  F$4$48P8P$P *   X^^ + *   8 8 "6v7G7GH   x .63P3P"55lCIINK":??#45O$ P#a'"1 ((4A#$99_b#AL#-aii.Y#ZL#(99lA @P@PQS]a@P@bR=T=TUZU^U^=_,,\: `   .%%*=EZ*Z n++0CG\0\ < ++ E    .!7|nEO  MbLgLgLi$jDAq%9!%U.*?++,A%X\+] mmo KFvFG<<\J K++,A%+P     \*44DD ,-@A B4 E 8;;; \*::;PY^:_!YY,;  'CSEx/FvhNhi  u7789XXjWkkl n   c U%k@ s W,W2 T) weights_onlyc0tj|d|i|S)zCall torch.load and handle weights_only. Defaults to weights_only=True to anticipate upcoming switch on the PyTorch side. r)rload)rargskwargss r torch_loadr's ::t A, A& AAr!c jdd#tjj|dn|}| t }dfd }dvr t d<tjj tjj|tr(tjj|t}d}n7tjj tjj|tr'tjj|t}d}ntjjr.|d}jdd t||fddi}d}njd d} | jd d} |d}t||jd djd d| } | }| rt|tfi}n t|tfi}|rNt%t&j(dr&|t'j*dk(rt-|d} n.t-||} n t/|t'j*|} |s| } | Si} | j1D]\} }| j3drd}n| j3drd}n t#d| j5|} |j1D](\}}t7j8||| \}}|dkDs&|} n|| }|| |<| S#t$r|d}t||fddi}d}YBwxYw#t $r$t#d|d|dtdtd|d wxYw)aP A helper method to load the PEFT weights from the HuggingFace Hub or locally Args: model_id (`str`): The local path to the adapter weights or the name of the adapter to load from the HuggingFace Hub. device (`str`): The device to load the weights onto. key_mapping (dict, *optional*, defaults to None) Extra mapping of PEFT `state_dict` keys applied before loading the `state_dict`. When this mapping is applied, the PEFT-specific `"base_model.model"` prefix is removed beforehand and the adapter name (e.g. `"default"`) is not inserted yet. Only pass this argument if you know what you're doing. hf_hub_download_kwargs (`dict`): Additional arguments to pass to the `hf_hub_download` method when loading from the HuggingFace Hub. subfolderNTc|rtnt}jdd#tjj d|S|S)Nr)rrgetrrr)use_safetensors weights_namehf_hub_download_kwargss rget_hub_filenamez+load_peft_weights..get_hub_filenameKsK3B/ &))+t<H GGLL/ /@|, ww~~bggll4)ABC77<<&>?  T<8 977<<l3  " " 1 1'= ""#5t< $&xoPToXnoH"O'**7D9 =*../?FE'= &1!+// DA,00dC ' #6 &&()H  *8\\E[\ 5>>5 )ve9L/L-huE -hvF %hU\\&=QR $4!4 %$-%'!(..0 =HC~~12, .& w ""6*C(3(9(9(; $%'WWWk3%G"q=!C   "(.O9< %o 6' =* %$Q' $,EBL&xoPToXnoH#O  $>&  -hZtH:F22>tD\C]]lmulvvwy s#L+M#L?>L?-M/)NdefaultFr])F)r%ztorch.nn.Modulerdict[str, torch.Tensor]rboolreturnzRtuple[dict[str, torch.Tensor], list[tuple[str, tuple[int, ...], tuple[int, ...]]]])rbrrrrrrr)rFF)rrrr)NN)rrrz Optional[str]r zOptional[dict[str, str]]rdict)* __future__rrrxrSrztypingrrrrrhuggingface_hub.errorsrrsafetensors.torchr r transformers.utilsr peft.mappingr rr otherrrrrrrr peft_typesrr r)rrrrrrrr!rr$s#   8N9.46!p bhGVmr- -3J-ei-W-8!'!7:!NQ!!,$)# q" q  qj$(BZ^{%{%({%>V{% {%r!