biiddlZddlZddlZddlmZmZmZmZmZddl Z ddl Z ddl Z ddlmZddlmZmZddlmZddl mZddlmZdd lmZdd lmZmZdd lmZmZdd l m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&e%rdd l'm(Z(dZ)e&jTe+Z,ddgddgddgdddgddgddgddgddgddgddZ-iZ.e-D]Z/e.jae-e/dZ1e jdjfGdde"Z4Gddee#Z5y)N)AnyDictListOptionalUnion) FrozenDict) create_reposnapshot_download)validate_hf_hub_args)Image)tqdm) ConfigMixin)FLAX_WEIGHTS_NAMEFlaxModelMixin)SCHEDULER_CONFIG_NAMEFlaxSchedulerMixin) CONFIG_NAME BaseOutputPushToHubMixinhttp_user_agentis_transformers_availablelogging)FlaxPreTrainedModelzdiffusion_flax_model.binsave_pretrainedfrom_pretrained)rrFlaxDiffusionPipeline)PreTrainedTokenizerPreTrainedTokenizerFastrFeatureExtractionMixinProcessorMixinImageProcessingMixin) diffusers transformersc  t|d|z}|S#t$rt||}Y|St$rtd|d|d|wxYw)NFlaxz Neither Flaxz nor z exist in )getattrAttributeError ValueError)module class_name class_objs b/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/diffusers/pipelines/pipeline_flax_utils.pyimport_flax_or_no_modelr.JspYFFZ$78   0FJ/   Y< |5 JvhWXXYsAAcheZdZUdZeeejjejfe d<y)FlaxImagePipelineOutputz Output class for image pipelines. Args: images (`List[PIL.Image.Image]` or `np.ndarray`) List of denoised PIL images of length `batch_size` or NumPy array of shape `(batch_size, height, width, num_channels)`. imagesN) __name__ __module__ __qualname____doc__rrPILr npndarray__annotations__r-r0r0Ws* $syy'3 44r;r0ceZdZdZdZdZ ddeeejfdee e fde fdZ eedeeeejffd Zed Zed e eeffd Zed ZdZdZy)ra Base class for Flax-based pipelines. [`FlaxDiffusionPipeline`] stores all components (models, schedulers, and processors) for diffusion pipelines and provides methods for loading, downloading and saving models. It also includes methods to: - enable/disable the progress bar for the denoising iteration Class attributes: - **config_name** ([`str`]) -- The configuration filename that stores the class and module names of all the diffusion pipeline's components. zmodel_index.jsonc ddlm}|jD]\}}||di}n|jj dd}|jj dd}|jj d}||vxr t ||} |t vs| r|}|jj} ||| fi}|jdi|t|||y)Nr pipelines)NN.r:) r#r?itemsr3splithasattrLOADABLE_CLASSES __class__r2register_to_configsetattr) selfkwargsr?namer* register_dictlibrary pipeline_dirpathis_pipeline_moduler+s r-register_modulesz&FlaxDiffusionPipeline.register_modulesvs'"LLN (LD&~!%| 4 !++11#6q9 &0066s;B? ((..s3%1T%9%^giQ]>^" "226H*G$--66 !%'< =  $D # # 4m 4 D$ '7 (r;save_directoryparams push_to_hubc |j|t|j}|jd|jd|jdd|r|jdd}|jdd}|jdd}|jd d} |jd |j t j jd } t| d || j} |jD]B} t|| } | | j} d}tjD]Y\}}tj |}|jD](\}}t||d}|t#| |s#|d}n|Ynt| |}dt%t'j(|j*jv}|r,|t j j-|| || n&|t j j-|| |s-|j/|  Ey)a Save all saveable variables of the pipeline to a directory. A pipeline variable can be saved and loaded if its class implements both a save and loading method. The pipeline is easily reloaded using the [`~FlaxDiffusionPipeline.from_pretrained`] class method. Arguments: save_directory (`str` or `os.PathLike`): Directory to which to save. Will be created if it doesn't exist. push_to_hub (`bool`, *optional*, defaults to `False`): Whether or not to push your model to the Hugging Face model hub after saving it. You can specify the repository you want to push to with `repo_id` (will default to the name of `save_directory` in your namespace). kwargs (`Dict[str, Any]`, *optional*): Additional keyword arguments passed along to the [`~utils.PushToHubMixin.push_to_hub`] method. _class_name_diffusers_version_moduleNcommit_messageprivate create_prFtokenrepo_idT)exist_okrZr\rrS)rS)r\rYr[) save_configdictconfigpoprCosrOsepr r]keysr'rFrErB importlib import_module issubclasssetinspect signature parametersjoin_upload_folder)rIrRrSrTrJmodel_index_dictrYrZr[r\r]pipeline_component_name sub_model model_clssave_method_name library_namelibrary_classesrM base_classsave_load_methodsclass_candidate save_methodexpects_paramss r-rz%FlaxDiffusionPipeline.save_pretraineds1. ( ,]+12Y- #ZZ(8$?NjjD1G ;6IJJw-EjjN,@,@,Mb,QRG!'D'QVW__G'7'<'<'>&  #&=>I !++I# 1A1G1G1I - o#11,?5D5J5J5L1J 1&-gz4&HO&2z)_7]+K%W->->{-K-V-V-[-[-])^^NGGLL1HIRXYpRqBGGLL9PQR##"#1' $A& r;pretrained_model_name_or_pathc 8|jdd}|jdd}|jdd}|jdd}|jdd}|jdd}|jd d} |jd d} |jd d} tjj|s|j |||||| } | j D cgc]} | j d r| }} |D cgc]"} tjj| d$}} |ttt|jgz }|sddgng}|gdz }|tk7r |j}n4| jd|j}|j dr|nd|z}d|i}t|}t!||||||||| }n|}|j |} |tk7r|}n^t#j$|j&j)dd}| dj dr| dnd| dz}t+||}|j-|\}}|D cic]} | |vs| |j| c} 8|D cic]} | |vs| |j| }} |j.| fi|\}}}|D cic]} | |vs| |j| }} i||}8fd}|j1D cic]\} }|| |s| |}} }t3|dkDr&t4j7d|d|jdi} ddlm}!|j1D]\}"\}#}|d||"<t=|!|#}$d}%d}&|"8vr|$st#j$|#}'t+|'|}(t>|#})|)j D*cic]}*|*t+|'|*d}+}*d},|+j1D]\}}-|- tA|(|-s|-},tA8|"jB|,setE8|"dtG8|"d|,8|""t4j7d |"d!|d"|d#d}&nt4j7d$8|"d%8|"}%n|$rCt+|!|#}.tI|.|}(tJ})tLjO|)j |(}+nSt#j$|#}'tI|'|}(t>|#})|)j D*cic]}*|*t+|'|*d}+}*|%8|&r5d}/+j1D]\}}-|- tA(|-s)|d&}/t+(|/}0tjjtjj||"r!tjj||"}1n|}%tA|(tPr|01|| | | '\}%}2|2| |"<nqtSr>tA|(tTr.|r|01|(}%|%jV}2|%`,n |01d)\}%}2|2| |"<n)tA|(tZr|01\}%}3|3| |"<n|01}%|%||"<t]|t]|j z }4t_8j }5t3|4dkDr+|4t]|5kr|4D]}68j|6d||6<nbt3|4dkDrTt]t_|j t_8j z|z }5tEd*|d+|d,|5d-|d.i|d | i}7|7| fScc} wcc} wcc} wcc} wcc} wcc}} wcc}*wcc}*w)/aW Instantiate a Flax-based diffusion pipeline from pretrained pipeline weights. The pipeline is set in evaluation mode (`model.eval()) by default and dropout modules are deactivated. If you get the error message below, you need to finetune the weights for your downstream task: ``` Some weights of FlaxUNet2DConditionModel were not initialized from the model checkpoint at stable-diffusion-v1-5/stable-diffusion-v1-5 and are newly initialized because the shapes did not match: ``` Parameters: pretrained_model_name_or_path (`str` or `os.PathLike`, *optional*): Can be either: - A string, the *repo id* (for example `stable-diffusion-v1-5/stable-diffusion-v1-5`) of a pretrained pipeline hosted on the Hub. - A path to a *directory* (for example `./my_model_directory`) containing the model weights saved using [`~FlaxDiffusionPipeline.save_pretrained`]. dtype (`jnp.dtype`, *optional*): Override the default `jnp.dtype` and load the model under this dtype. force_download (`bool`, *optional*, defaults to `False`): Whether or not to force the (re-)download of the model weights and configuration files, overriding the cached versions if they exist. proxies (`Dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, for example, `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request. output_loading_info(`bool`, *optional*, defaults to `False`): Whether or not to also return a dictionary containing missing keys, unexpected keys and error messages. local_files_only (`bool`, *optional*, defaults to `False`): Whether to only load local model weights and configuration files or not. If set to `True`, the model won't be downloaded from the Hub. token (`str` or *bool*, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, the token generated from `diffusers-cli login` (stored in `~/.huggingface`) is used. revision (`str`, *optional*, defaults to `"main"`): The specific model version to use. It can be a branch name, a tag name, a commit id, or any identifier allowed by Git. mirror (`str`, *optional*): Mirror source to resolve accessibility issues if you're downloading a model in China. We do not guarantee the timeliness or safety of the source, and you should refer to the mirror site for more information. kwargs (remaining dictionary of keyword arguments, *optional*): Can be used to overwrite load and saveable variables (the pipeline components) of the specific pipeline class. The overwritten components are passed directly to the pipelines `__init__` method. To use private or [gated models](https://huggingface.co/docs/hub/models-gated#gated-models), log-in with `hf auth login`. Examples: ```py >>> from diffusers import FlaxDiffusionPipeline >>> # Download pipeline from huggingface.co and cache. >>> # Requires to be logged in to Hugging Face hub, >>> # see more in [the documentation](https://huggingface.co/docs/hub/security-tokens) >>> pipeline, params = FlaxDiffusionPipeline.from_pretrained( ... "stable-diffusion-v1-5/stable-diffusion-v1-5", ... variant="bf16", ... dtype=jnp.bfloat16, ... ) >>> # Download pipeline, but use a different scheduler >>> from diffusers import FlaxDPMSolverMultistepScheduler >>> model_id = "stable-diffusion-v1-5/stable-diffusion-v1-5" >>> dpmpp, dpmpp_state = FlaxDPMSolverMultistepScheduler.from_pretrained( ... model_id, ... subfolder="scheduler", ... ) >>> dpm_pipe, dpm_params = FlaxStableDiffusionPipeline.from_pretrained( ... model_id, variant="bf16", dtype=jnp.bfloat16, scheduler=dpmpp ... ) >>> dpm_params["scheduler"] = dpmpp_state ``` cache_dirNproxieslocal_files_onlyFr\revisionfrom_ptuse_memory_efficient_attentionsplit_head_dimdtype)r~rrr\r_*z*.binz *.safetensors)z*.onnxz *.onnx_dataz*.xmlz*.pbrVr&pipeline_class)r~rrr\rallow_patternsignore_patterns user_agentr@rc&|dy|vr|yy)NrFTr:)rKvaluepassed_class_objs r- load_modulez:FlaxDiffusionPipeline.from_pretrained..load_modules*Qx'',Tz is of type: z, but should be zYou have passed `None` for z! to disable its functionality in z3. Note that this might lead to problems when using z and is not recommended.z&You have passed a non-standard module z2. We cannot verify whether it has the correct type)rrrr)r)_do_initz Pipeline z expected z , but only z were passed.r:)0rcrdrOisdir load_configrf startswithrnrrr config_namerr2getrr rgrhr3rCr'_get_signature_keysextract_init_dictrBlenloggerwarningr#r?rDrErirFr)typer.ALL_IMPORTABLE_CLASSESrafromkeysrrrrS_paramsrrjlist)9clsr|rJr~rrr\rrrrr config_dictk folder_namesrrrequested_pipeline_classr cached_folderrdiffusers_moduler+expected_modulesoptional_kwargspassed_pipe_kwargs init_dict unused_kwargsr init_kwargsrvrSr?rKrurPloaded_sub_modelsub_model_should_be_definedrMr,importable_classescclass_candidatesexpected_class_objrypipeline_moduleload_method_name load_methodloadable_folder loaded_paramsscheduler_statemissing_modulespassed_modulesr*modelrs9 @r-rz%FlaxDiffusionPipeline.from_pretrainedsblJJ{D1 **Y-!::&8%@ 7D)::j$/**Y.)/4TV[)\&$4e< 7D)ww}}:;//-#!1! *K(3'7'7'9S!cARASLS8CO8T5$o#2F4L'2?'C$ 0K _ 1D./#k6F6F6H2II.3356  ! #3~;N(N) I&6&:&:64&H F# I  !A % k&6&6&8!9DAQAVAVAX>> from diffusers import ( ... FlaxStableDiffusionPipeline, ... FlaxStableDiffusionImg2ImgPipeline, ... ) >>> text2img = FlaxStableDiffusionPipeline.from_pretrained( ... "stable-diffusion-v1-5/stable-diffusion-v1-5", variant="bf16", dtype=jnp.bfloat16 ... ) >>> img2img = FlaxStableDiffusionImg2ImgPipeline(**text2img.components) ``` Returns: A dictionary containing all the modules needed to initialize the pipeline. rz% has been incorrectly initialized or z& is incorrectly implemented. Expected z to be defined, but z are defined.)rrbrfrr'rjr)rF)rIrrr componentss r-rz FlaxDiffusionPipeline.components!s0150H0H0N--)-)9)9); $%1<rs1" 33 ,:6-JX0 '    H % -.?@02CD"35F!G !23DE$57H#I 13DE#46G"H,.?@!24E F   =G!!"27";<=  5j 5 5}+K}+r;