bitddlmZddlZddlmZmZddlmZmZm Z ddl m Z ddl m Z eGdde Zy) ) annotationsN) dataclassfield)LiteralOptionalUnion) PeftConfig)PeftTypec8eZdZUdZeddddZded<ed d d d Zd ed<ed ddiZd ed<edddiZ ded<ed ddiZ ded<ed ddiZ ded<edddiZ ded<ed dd iZ d!ed"<fd#ZxZS)$ BoneConfiga This is the configuration class to store the configuration of a [`BoneModel`]. Args: r (`int`): The rank of Bone across different layers. It is best to set 'r' to an even number; otherwise, the default initialization method will not work. target_modules (`Optional[Union[List[str], str]]`): The names of the modules to apply the adapter to. If this is specified, only the modules with the specified names will be replaced. When passing a string, a regex match will be performed. When passing a list of strings, either an exact match will be performed or it is checked if the name of the module ends with any of the passed strings. If this is specified as 'all-linear', then all linear modules are chosen, excluding the output layer. If this is not specified, modules will be chosen according to the model architecture. If the architecture is not known, an error will be raised -- in this case, you should specify the target modules manually. exclude_modules (`Optional[Union[List[str], str]]`): The names of the modules to not apply the adapter. When passing a string, a regex match will be performed. When passing a list of strings, either an exact match will be performed or it is checked if the name of the module ends with any of the passed strings. init_weights (bool | Literal["bat"]): Different initializations correspond to different Bone variants. By default, setting True uses the Bone structure, while "bat" selects the Bat structure. layers_to_transform (`Union[List[int], int]`): The layer indices to transform. If a list of ints is passed, it will apply the adapter to the layer indices that are specified in this list. If a single integer is passed, it will apply the transformations on the layer at this index. layers_pattern (`str`): The layer pattern name, used only if `layers_to_transform` is different from `None`. modules_to_save (`List[str]`): List of modules apart from adapter layers to be set as trainable and saved in the final checkpoint. @z)The rank of Bone across different layers.zdIt is best to set 'r' to an even number; otherwise, the default initialization method will not work.)helpnote)defaultmetadataintrNzRList of module names or regex expression of the module names to replace with Bone.zPFor example, ['q', 'v'] or '.*decoder.*(SelfAttention|EncDecAttention).*(q|v)$' )rexamplezOptional[Union[list[str], str]]target_modulesrzRList of module names or regex expression of the module names to exclude from Bone.exclude_modulesTzWhether to initialize the weights of the Bone layers with their default initialization. Don't change this setting, except if you know exactly what you're doing.zbool | Literal['bat'] init_weightszThe layer indexes to transform, is this argument is specified, PEFT will transform only the layers indexes that are specified inside this list. If a single integer is passed, PEFT will transform only the layer at this index.zOptional[Union[list[int], int]]layers_to_transformzThe layer pattern name, used only if `layers_to_transform` is different to None and if the layer pattern is not in the common layers pattern.z Optional[str]layers_patternnonez7Bias type for Bone. Can be 'none', 'all' or 'Bone_only'strbiasaList of modules apart from Bone layers to be set as trainable and saved in the final checkpoint. For example, in Sequence Classification or Token Classification tasks, the final layer `classifier/score` are randomly initialized and as such need to be trainable and saved.zOptional[list[str]]modules_to_savec:t|tj|_t |j trt|j n |j |_t |jtrt|jn |j|_ t |j tr|j tdt |j tr|j tdtjdy)NzD`layers_to_transform` cannot be used when `target_modules` is a str.z?`layers_pattern` cannot be used when `target_modules` is a str.zBone will be removed in v0.19.0 of PEFT, use `MissConfig` instead. If you already have a Bone checkpoint, you can use `/scripts/convert-bone-to-miss.py` to convert it into )super __post_init__r BONE peft_type isinstancerlistsetrrr ValueErrorrwarningswarn)self __class__s R/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/peft/tuners/bone/config.pyr zBoneConfig.__post_init__ms !(243F3F(MC## $SWSfSf *4D4H4H$)OC$$ %UYUiUi  d))3 /D4L4L4Xcd d d))3 /D4G4G4S^_ _  x )__name__ __module__ __qualname____doc__rr__annotations__rrrrrrrr __classcell__)r*s@r+r r s"@?z As7<hi 7N38=no8O4+0 N +L'r9s:#(++" g g  g r,