bilddlmZddlmZmZddlmZmZmZddl m Z ddl m Z eGdde Z y) ) annotations) dataclassfield)LiteralOptionalUnion) PeftConfig)PeftTypecVeZdZUdZedddiZded<eddd iZd ed <eddd iZd ed <edddiZ ded<edddiZ ded<edddiZ ded<edddiZ ded<edddiZ ded<edddiZded <fd!ZxZS)" PolyConfiga This is the configuration class to store the configuration of a [`PolyModel`]. - [Polytropon (Poly)](https://huggingface.co/papers/2202.13914) - [Multi-Head Routing (MHR)](https://huggingface.co/papers/2211.03831) Args: r (`int`): Attention dimension of each Lora in Poly. target_modules (`Union[List[str],str]`): The names of the modules to apply Poly to. exclude_modules (`Optional[Union[List[str], str]]`): The names of the modules to not apply the adapter. When passing a string, a regex match will be performed. When passing a list of strings, either an exact match will be performed or it is checked if the name of the module ends with any of the passed strings. modules_to_save (`List[str]`): List of modules apart from Poly layers to be set as trainable and saved in the final checkpoint. init_weights (bool): Whether to perform initialization of Poly weights. poly_type (`Literal["poly"]`): The variant of the Poly module to use. Currently, only "poly" is supported. n_tasks (`int`): The number of tasks in a multitasking scenario. n_skills (`int`): The number of skills (LoRA) in each Poly layer. n_splits (`int`): The number of splits within each LoRA of a Poly layer. A value greater than 1 indicates the use of Multi-Head Routing (MHR). helpzLora attention dimension)defaultmetadataintrNzList of module names or regex expression of the module names to replace with Poly.For example, ['q', 'v'] or '.*decoder.*(SelfAttention|EncDecAttention).*(q|v)$' zOptional[Union[list[str], str]]target_moduleszRList of module names or regex expression of the module names to exclude from Poly.exclude_modulesaList of modules apart from Poly layers to be set as trainable and saved in the final checkpoint. For example, in Sequence Classification or Token Classification tasks, the final layer `classifier/score` are randomly initialized and as such need to be trainable and saved.zOptional[list[str]]modules_to_saveTzWhether to initialize the weights of the Poly layers with their default initialization. Don't change this setting, except if you know exactly what you're doing.bool init_weightspolyzDType of Poly modules to be used. Currently only "poly" is supported.zLiteral['poly'] poly_typez)Number of tasks in multitasking scenario.n_tasksz+Number of skills (LoRA) in each Poly layer.n_skillsz2Number of splits within each LoRA of a Poly layer.n_splitscXt|tj|_t |j trt|j n |j |_t |jtrt|j|_ y|j|_ y)N) super __post_init__r POLY peft_type isinstancerlistsetr)self __class__s R/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/peft/tuners/poly/config.pyr!zPolyConfig.__post_init___s !(243F3F(MC## $SWSfSf *4D4H4H$)OC$$ % UYUiUi )__name__ __module__ __qualname____doc__rr__annotations__rrrrrrrrr! __classcell__)r(s@r)r r s3.10J'K LAsL6; _ 7N38=no8O4,1 v ,O( N L$"'`a"IEFGSGHHcNOHc   r*r N) __future__r dataclassesrrtypingrrr peft.configr peft.utilsr r r*r)r7s7#(++" N N  N r*