bi 0ddlZddlmZddlZddlmZdej dej fdZdZdejdej fdZ ddejd ee ejd eej dej fd Z d edefd Zy)N)Optionalxreturnc|dd|jddzf}|d|jddzdf}tj| |fdS)a Rotate half the hidden dims of the input. This function was duplicated verbatim from: https://github.com/huggingface/transformers/blob/1de8ce9ee1191ba761a593ac15d9ccbf5851bfc5/src/transformers/models/llama/modeling_llama.py#L126 This was done to eliminate the Llama transformers implementation as a dependency of this file. Note that some other functions were also adapted from the transformers implementation but were modified. .Ndim)shapetorchcat)rx1x2s \/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/peft/tuners/adaption_prompt/utils.pyllama_rotate_halfrs\ 3"!''"+"" " #B 3 q " " #B 99rc2YB ''ct|jdk(r|ddddddf}|jd|jdd|jd}tj|j|jddddd|}tj|j|jddddd|}n(||j d}||j d}||zt ||zz}|S)a Apply rotary position embedding to query states in the Llama model. This function was adapted from: https://github.com/huggingface/transformers/blob/1de8ce9ee1191ba761a593ac15d9ccbf5851bfc5/src/transformers/models/llama/modeling_llama.py#L133 It was modified to remove unnecessary processing of key states. The method is compatible with transformers <= 4.34.2 and also with the latest version (>=4.35). Nrr)lenr repeatr gather unsqueezer)qcossin position_idsgather_indicesq_embeds rllama_apply_rotary_pos_embr!$s 399~%aq$&67'..q#))A,399Q<Pll3::n&:&:1&=q!QGN[ll3::n&:&:1&=q!QGN[,))!,,))!,3w,Q/#56G Nrmodelc N|jd}|jd}|jd}|j\}}}t|dr |j}n|jj }|j |j||||jjdd} |jj|jjz} |j|j|||| z|jjdd} |} |Dt|tr| |dj dz } n| |j#|j$z } d |vr>|d \} }| j'd} |j'd}| | zt)| |zzSdt+j,|j.j0j2vr$|j/| | \} }t5| | ||Sd}|z|&t7j8|||z| j: }nA|j=||j$}t7j8|||z| j: }|j'd}d|i}d t+j,|j.j0j2vr||z|d <|j.| fi|\} }t?| j d k(r"| j'd} |j'd}| | zt)| |zzS)a Compute query states for Llama models specifically. They need to be recomputed as the forward() method of the original LlamaModel in the transformers library does not return them. See the related discussion in the PR: https://github.com/huggingface/peft/pull/268 hidden_statesrpast_key_value num_headsrrrposition_embeddings)seq_len)devicer)r) getsizehasattrr&confignum_attention_headsq_projviewhead_dim transposek_proj in_features out_featuresv_proj isinstancetupler get_seq_length layer_idxrrinspect signature rotary_embforward parametersr!r aranger*get_usable_lengthr)r"kwargsr$rr%bszq_len_r& query_statesfactor value_statesr)rrpast_seen_tokensnew_cache_positionsrotary_emb_kwargss rllama_compute_query_statesrM=s JJ/M::n-LZZ 01N!&&(MCuk"OO LL44 << .33C 5>>ZddefhijL \\ % %)B)B BF<< .33CfATW\WeWefppqrtuvLG! ne , ~a(..r2 2G ~44U__E EG&/0SmmAmmAs"'8'F'LMMW..u/?/?/G/GHSSS##L'#BS),S,OO  !"',,ueemLL_L_"` -??uW "',,/?AQTYAYbnbubu"v *44Q7 '6G%%e&6&6&>&>?JJJ',/?'?)$u B0ABHC 399~mmAmmA 3 #4\#BS#H IIrr$encoder_hidden_statescn|At|ds#td|jjd|j |}n0|j |j |jd\}}}g|jddd|j}|j|jdd}|S) z Compute query states for GPT2 models. They need to be recomputed as the forward() method of the GPT@ in the transformers library does not return them. See the related discussion in the PR: Nq_attnzIf `z` is used as cross attention, the weights `q_attn` must be defined. Please make sure to instantiate it with `GPT2Attention(..., is_cross_attention=True)`.rr rr) r- ValueError __class____name__rPc_attnsplit split_sizer r2r1r3)r"r$rNrGrFshape_qs rgpt2_compute_query_statesrXs(uh'u//01ij ||M2 "\\-8>>u?O?OUV>W a< ""3B'<rhs  ( (%,, (2CJbiiCJellCJR59 99E%"3"345$ELL1 \\ 2999r