Ë §Ùbi0 ãól—ddlmZddlmZmZddlmZmZmZddl m Z ddl m Z eGd„de ««Z y) é)Ú annotations)Ú dataclassÚfield)ÚLiteralÚOptionalÚUnion)Ú LycorisConfig)ÚPeftTypecó‡—eZdZUdZedddi¬«Zded<edddi¬«Zded <ed dd i¬«Zd ed <ed ddi¬«Z d ed<edddi¬«Z ded<edddi¬«Z ded<edddi¬«Z ded<edddi¬«Z ded<edddi¬«Zded<edddi¬«Zded <ed!dd"i¬«Zd#ed$<eddd%i¬«Zd&ed'<eddd(i¬«Zded)<eddd*i¬«Zd+ed,<ˆfd-„ZˆxZS).Ú LoKrConfiga¨ Configuration class of [`LoKrModel`]. Args: r (`int`): LoKr rank. alpha (`int`): The alpha parameter for LoKr scaling. rank_dropout (`float`): The dropout probability for rank dimension during training. module_dropout (`float`): The dropout probability for disabling LoKr modules during training. use_effective_conv2d (`bool`): Use parameter effective decomposition for Conv2d with ksize > 1 ("Proposition 3" from FedPara paper). decompose_both (`bool`): Perform rank decomposition of left kronecker product matrix. decompose_factor (`int`): Kronecker product decomposition factor. rank_dropout_scale ('bool) Whether to scale the rank dropout while training, defaults to `False`. target_modules (`Optional[Union[List[str], str]]`): The names of the modules to apply the adapter to. If this is specified, only the modules with the specified names will be replaced. When passing a string, a regex match will be performed. When passing a list of strings, either an exact match will be performed or it is checked if the name of the module ends with any of the passed strings. If this is specified as 'all-linear', then all linear/Conv1D modules are chosen, excluding the output layer. If this is not specified, modules will be chosen according to the model architecture. If the architecture is not known, an error will be raised -- in this case, you should specify the target modules manually. exclude_modules (`Optional[Union[List[str], str]]`): The names of the modules to not apply the adapter. When passing a string, a regex match will be performed. When passing a list of strings, either an exact match will be performed or it is checked if the name of the module ends with any of the passed strings. init_weights (`bool`): Whether to perform initialization of adapter weights. This defaults to `True`. Use "lycoris" to initialize weights in the style of the LYCORIS repository. Passing `False` is discouraged. layers_to_transform (`Union[List[int], int]`): The layer indices to transform. If a list of ints is passed, it will apply the adapter to the layer indices that are specified in this list. If a single integer is passed, it will apply the transformations on the layer at this index. layers_pattern (`Optional[Union[List[str], str]]`): The layer pattern name, used only if `layers_to_transform` is different from `None`. This should target the `nn.ModuleList` of the model, which is often called `'layers'` or `'h'`. rank_pattern (`dict`): The mapping from layer names or regexp expression to ranks which are different from the default rank specified by `r`. For example, `{'^model.decoder.layers.0.encoder_attn.k_proj': 16}`. alpha_pattern (`dict`): The mapping from layer names or regexp expression to alphas which are different from the default alpha specified by `alpha`. For example, `{'^model.decoder.layers.0.encoder_attn.k_proj': 16}`. modules_to_save (`Optional[List[str]]`): List of modules apart from adapter layers to be set as trainable and saved in the final checkpoint. éÚhelpz LoKr rank)ÚdefaultÚmetadataÚintÚrz LoKr alphaÚalphagz:The dropout probability for rank dimension during trainingÚfloatÚ rank_dropoutzBThe dropout probability for disabling LoKr modules during trainingÚmodule_dropoutFzhUse parameter effective decomposition for Conv2d 3x3 with ksize > 1 ("Proposition 3" from FedPara paper)ÚboolÚuse_effective_conv2dzr<s:ðõ#ç(ß+Ñ+å3Ýð ôsósó ñsr/