biqhddlZddlmZmZddlmZmZddlmZddl m Z m Z eGddeZ y)N) dataclassfield)LiteralOptional)PromptLearningConfig)PeftTypeTaskTypecteZdZUdZedddiZeeee d<edddiZ eeee d<eddd iZ eeee d <ed dd iZ ee d e d<edddiZeee d<edddiZeee d<edddiZeee d<edddiZeee d<dZdZy) CPTConfigaa CPT Configuration class extending PeftConfig for Context-aware Prompt Tuning (CPT). This class introduces additional parameters required for CPT, such as: - Token type masks - Prompt tuning initialization - Loss weighting - Projection settings For more details, see the paper: https://huggingface.co/papers/2410.17222 Nhelpz)Tensor of token IDs used for CPT prompts.)defaultmetadata cpt_token_idsz"Tensor mask applied to CPT tokens.cpt_maskz+Mask indicating the type of each CPT token.cpt_tokens_type_masknonez)Type of weighted loss: 'none' or 'decay'.)rdecayopt_weighted_loss_typeg?z/Factor for exponential decay in loss weighting.opt_loss_decay_factorg?z#Epsilon value for input projection.opt_projection_epsilonz$Epsilon value for format projection.opt_projection_format_epsilonz`The tokenizer to use for prompt tuning initialization. Only used if prompt_tuning_init is `TEXT`tokenizer_name_or_pathTcbd|_d|_d|_d|_d|_t j |_|jtjk7ratj|jjdtjjdt tj|_|j"dg|_t%|j"|_|j(|j"Dcgc]}dc}|_|j*|j"Dcgc]}dc}|_t%|j"t%|j(cxk(r2t%|j*cxk(r|j&k(st-dt-dycc}wcc}w)zh Post-initialization hook to set additional attributes after the config is initialized. TNz only supports task_type = zP, setting it automatically. This will raise an error starting from PEFT v0.18.0.rzKcpt_token_ids, cpt_mask and cpt_tokens_type_mask must have the same length.)is_prompt_learning num_layers token_dimnum_attention_headsnum_transformer_submodulesrCPT peft_type task_typer CAUSAL_LMwarningswarn __class____name__value FutureWarningrlennum_virtual_tokensrr ValueError)self_s Q/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/peft/tuners/cpt/config.py __post_init__zCPTConfig.__post_init__Hso #'#' *+'! >>X// / MM>>**++FxGYGYG_G_F`aaa  &//DN    %"#D "%d&8&8"9 == (,(:(:;1Q;DM  $ $ ,484F4F(Gq(GD % "" #s4=='9 vSAZAZ=[ v_c_v_v vjk k wjk k w <)Hs ? F'* F,)r' __module__ __qualname____doc__rrrlistint__annotations__rrrrrfloatrrrstrrr0r/r r s? */(ST*M8DI&%*$&JnAo$pHhtCy!p05(UV1(49- BG&*U!VBHW_%=>.3v'XY.8E? /4v'LM/HUO6;v'MN6!8E? -2 v -HSM"lr:r ) r$ dataclassesrrtypingrr peft.configr peft.utilsrr r r9r:r/r?s8($,) Rl$Rl Rlr: